From stefan.karlsson at oracle.com Sat Jul 1 07:01:04 2017 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Sat, 1 Jul 2017 09:01:04 +0200 Subject: RFR: 8178495: Bug in the align_size_up_ macro In-Reply-To: References: <90b9016c-42b3-31e7-8c9c-28dafcad879a@oracle.com> <7aea18aa-7aab-8f80-72f5-b8d4304a829d@oracle.com> Message-ID: On 2017-07-01 01:21, Kim Barrett wrote: >> On Jun 30, 2017, at 11:15 AM, Stefan Karlsson wrote: >> >> Fixing the logging with SCOPED_TRACE was only tested locally, but failed on OSX. Here's a fix for that problem: >> http://cr.openjdk.java.net/~stefank/8178495/webrev.02 >> http://cr.openjdk.java.net/~stefank/8178495/webrev.02.delta >> >> Passes JPRT now. >> >> StefanK > Looks good. Thanks for reviewing. > > I think I see where things might have gone awry, but I?m curious what the OSX failure was. > I?m a bit surprised that it complained but other compilers didn?t. This was the error message: hotspot/test/native/utilities/test_align.cpp:40:18: note: expanded from macro 'log' #define log(...) SCOPED_TRACE(err_msg(__VA_ARGS__)) ^ test/fmw/gtest/include/gtest/gtest.h:2172:49: note: expanded from macro 'SCOPED_TRACE' __FILE__, __LINE__, ::testing::Message() << (message)) ^ hotspot/src/share/vm/utilities/formatBuffer.hpp:65:3: note: declared private here FormatBuffer(const FormatBuffer &); // prevent copies ^ The jprt run was aborted after this failure, so it could be that this would have failed on Windows and Solaris as well. StefanK > From thomas.stuefe at gmail.com Sun Jul 2 07:23:43 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sun, 2 Jul 2017 09:23:43 +0200 Subject: RFR(s): 8179327: gtestLauncher should run tests on a separate thread (optionally) Message-ID: Dear all, please review the following change: Bug: JDK-8179327 Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/ 8179327-gtestLauncher-should-have-an-option-to-avoid- primordial-thread-for-tests/webrev.00/webrev/ This fix is motivated by the inability of the AIX openjdk vm to run on the primordial thread (see https://bugs.openjdk.java.net/browse/JDK-8171505), which we unfortunately cannot do much about. This prevented us until now to run gtests successfully, because the gtestlauncher (in contrast to the standard java launcher) invokes the VM on the primordial thread. To fix this, I gave the gtest launcher the ability to run the tests in an own thread. Instead of sprinkling the source with "#ifdef _AIX", I added a command line option "-new-thread", which allows to start the tests in a new thread, and implemented this feature for all platforms (well, posix and windows). On all platforms save AIX this is by default off, so nothing changes. On AIX, tests will be default be executed in a new thread. Thanks for reviewing, Kind Regards, Thomas From hohensee at amazon.com Mon Jul 3 01:35:49 2017 From: hohensee at amazon.com (Hohensee, Paul) Date: Mon, 3 Jul 2017 01:35:49 +0000 Subject: RFR(XL): 8182299: Enable disabled clang warnings, build on OSX 10 + Xcode 8 In-Reply-To: References: <27FD0413-52BC-42E6-A5B0-3C92A49A2D6F@amazon.com> <1CBD62A9-9B1B-4B05-AAF9-BE2D52DE8C79@amazon.com> <6B296ABE-66C0-4C32-AC4E-8674BE103514@oracle.com> <3ACE75CF-CEA6-4081-952A-BBA138763582@amazon.com> <435C96F2-417E-4A47-8649-28E34E2908AE@amazon.com> <1d06f5fe-c3f8-f599-58bc-09b2099c819c@oracle.com> Message-ID: Jesper discovered that I?d forgotton to build Hotspot with DEBUG_LEVEL=fastdebug, so he made the necessary additional changes. Incremental webrev here http://cr.openjdk.java.net/~jwilhelm/8182299/webrev.01/hotspot-webrev/ Updated complete webrev including Jesper?s changes here http://cr.openjdk.java.net/~phh/8182299/webrev_hotspot.01/ Thanks, Paul On 6/30/17, 9:30 AM, "hotspot-dev on behalf of Hohensee, Paul" wrote: Thanks for picking up the closed work, Jesper, and thanks for the review, Erik. It?s good to be back working on server Java. ( Yes, I added the G1RootClosures virtual destructor because of the warning. In heapRegionType.cpp, I just followed along with the existing funky indentation and made the default clause also have a return because all the other clauses do. Clang would indeed be ok with a default clause with just a break. Also, I?ve used the same idiom in other files, so I?d change those too if I change these. Which would you prefer? I?m not invested in any particular format. Thanks, Paul On 6/30/17, 3:00 AM, "jesper.wilhelmsson at oracle.com" wrote: I ran the change through JPRT so at least it builds and runs on all platforms we support there. I'm currently fixing our closed code to build on Mac with this change. /Jesper > On 30 Jun 2017, at 10:35, Erik Helin wrote: > > Hi Paul, > > thanks for contributing! Please see my comments regarding the GC changes below. > > On 06/29/2017 03:31 PM, Hohensee, Paul wrote: >> I now have access to cr.openjdk.java.net, so the latest webrevs are at >> >> http://cr.openjdk.java.net/~phh/8182299/webrev_jdk.00/ >> http://cr.openjdk.java.net/~phh/8182299/webrev_hotspot.00/ > > gc/g1/g1RootClosures.cpp: > - // Closures to process raw oops in the root set. > + virtual ~G1RootClosures() {} > + > +// Closures to process raw oops in the root set. > > I assume this is added because there is some warning about having only pure virtual methods but not having a virtual destructor. None of the classes inheriting from G1RootClosures needs a destructor, nor does G1RootClosures itself (it is just an "interface"). So there is no problem with correctness here :) > > However, I like to have lots of warnings enabled, so for me it is fine to an empty virtual destructor here just to please the compiler. > > gc/g1/heapRegion.cpp: looks correct > > gc/g1/heapRegionType.cpp: > The indentation seems a bit funky to me here :) Also, have you compiled this on some other platforms? I think the last return is outside of the switch just to, as the comment says, "keep some compilers happy". > > Would clang be ok with having an empty default clause with just a break? And then fall through to the return outside of the switch? > > gc/g1/ptrQueue.cpp: looks correct > > gc/parallel/psPromotionManager.cpp: looks correct > > Thanks, > Erik > >> On 6/28/17, 3:50 PM, "hotspot-dev on behalf of Hohensee, Paul" wrote: >> >> Thanks for the review, Jesper. >> >> New webrev sent, has only a change to nativeInst_x86.cpp. >> >> In nativeInst_x86.cpp, I formatted the expression so I could easily understand it, but you?re right, not everyone does it that way (maybe only me!), so I?ve changed it to >> >> if (((ubyte_at(0) & NativeTstRegMem::instruction_rex_prefix_mask) == NativeTstRegMem::instruction_rex_prefix && >> ubyte_at(1) == NativeTstRegMem::instruction_code_memXregl && >> (ubyte_at(2) & NativeTstRegMem::modrm_mask) == NativeTstRegMem::modrm_reg) || >> (ubyte_at(0) == NativeTstRegMem::instruction_code_memXregl && >> (ubyte_at(1) & NativeTstRegMem::modrm_mask) == NativeTstRegMem::modrm_reg)) { >> >> In graphKit.cpp, the old code was >> >> #ifdef ASSERT >> case Deoptimization::Action_none: >> case Deoptimization::Action_make_not_compilable: >> break; >> default: >> fatal("unknown action %d: %s", action, Deoptimization::trap_action_name(action)); >> break; >> #endif >> >> In the non-ASSERT case, the compiler complained about the lack of Action_none, Action_make_not_compilable and default. If the warning had been turned off, the result would have been ?break;? for all three. In the ASSERT case, Action_none and Action_make_not_compilable result in ?break;?, and in the default case ?fatal(); break;? >> >> The new code is >> >> case Deoptimization::Action_none: >> case Deoptimization::Action_make_not_compilable: >> break; >> default: >> #ifdef ASSERT >> fatal("unknown action %d: %s", action, Deoptimization::trap_action_name(action)); >> #endif >> break; >> >> The compiler doesn?t complain about Action_none, Action_make_not_compilable or default anymore. In the non-ASSERT case, the result is ?break;? for all three, same as for the old code. In the ASSERT case, Action_none and Action_make_not_compilable result in ?break;?, and in the default case ?fatal(); break;?, again same as for the old code. >> >> Thanks, >> >> Paul >> >> On 6/28/17, 10:36 AM, "jesper.wilhelmsson at oracle.com" wrote: >> >> Hi Paul, >> >> Thanks for doing this change! In general everything looks really good, there are a lot of really nice cleanups here. I just have two minor questions/nits: >> >> * In hotspot/cpu/x86/vm/nativeInst_x86.hpp it seems the expression already have parenthesis around the & operations and the change here is "only" cleaning up the layout of the code which is not a bad thing in it self, but you move the logical operators to the beginning of each line which is a quite different style than the rest of the code in the same function where the operators are at the end of the line. >> >> * In hotspot/share/vm/opto/graphKit.cpp you moved the #ifdef ASSERT so that Action_none and Action_make_not_compilable are available also when ASSERT is not defined. I don't see this mentioned in your description of the change. Was this change intentional? >> >> Thanks, >> /Jesper >> >> >> > On 27 Jun 2017, at 21:34, Hohensee, Paul wrote: >> > >> > An attempt at better formatting. >> > >> > https://bugs.openjdk.java.net/browse/JDK-8182299 >> > http://cr.openjdk.java.net/~jwilhelm/8182299/webrev_jdk.00/ >> > http://cr.openjdk.java.net/~jwilhelm/8182299/webrev_hotspot.00/ >> > >> > Jesper has been kind enough to host the webrevs while I get my cr.openjdk.net account set up, and to be the sponsor. >> > >> > This rfe a combination of enabling disabled clang warnings and getting jdk10 to build on OSX 10 and Xcode 8. At least one enabled warning (delete-non-virtual-dtor) detected what seems to me a real potential bug, with the rest enforcing good code hygiene. >> > >> > These changes are only in OpenJDK, so I?m looking for a volunteer to make the closed changes. >> > >> > Thanks, >> > >> > Paul >> > >> > >> > Here are the jdk file-specific details: >> > >> > java_md_macosx.c splashscreen_sys.m >> > >> > Removed objc_registerThreadWithCollector() since it's obsolete and of questionable value in any case. >> > >> > NSApplicationAWT.m >> > >> > Use the correct NSEventMask rather than NSUInteger. >> > >> > jdhuff.c jdphuff.c >> > >> > Shifting a negative signed value is undefined. >> > >> > Here are the hotspot notes: >> > >> > Here are the lists of files affected by enabling a given warning: >> > >> > switch: all of these are lack of a default clause >> > >> > c1_LIRAssembler_x86.cpp c1_LIRGenerator_x86.cpp c1_LinearScan_x86.hpp >> > jniFastGetField_x86_64.cpp assembler.cpp c1_Canonicalizer.cpp >> > c1_GraphBuilder.cpp c1_Instruction.cpp c1_LIR.cpp c1_LIRGenerator.cpp >> > c1_LinearScan.cpp c1_ValueStack.hpp c1_ValueType.cpp >> > bcEscapeAnalyzer.cpp ciArray.cpp ciEnv.cpp ciInstance.cpp ciMethod.cpp >> > ciMethodBlocks.cpp ciMethodData.cpp ciTypeFlow.cpp >> > compiledMethod.cpp dependencies.cpp nmethod.cpp compileTask.hpp >> > heapRegionType.cpp abstractInterpreter.cpp bytecodes.cpp >> > invocationCounter.cpp linkResolver.cpp rewriter.cpp jvmciCompilerToVM.cpp >> > jvmciEnv.cpp universe.cpp cpCache.cpp generateOopMap.cpp >> > method.cpp methodData.cpp compile.cpp connode.cpp gcm.cpp graphKit.cpp >> > ifnode.cpp library_call.cpp memnode.cpp parse1.cpp >> > parse2.cpp phaseX.cpp superword.cpp type.cpp vectornode.cpp >> > jvmtiClassFileReconstituter.cpp jvmtiEnter.xsl jvmtiEventController.cpp >> > jvmtiImpl.cpp jvmtiRedefineClasses.cpp methodComparator.cpp methodHandles.cpp >> > advancedThresholdPolicy.cpp reflection.cpp relocator.cpp sharedRuntime.cpp >> > simpleThresholdPolicy.cpp writeableFlags.cpp globalDefinitions.hpp >> > >> > delete-non-virtual-dtor: these may be real latent bugs due to possible failure to execute destructor(s) >> > >> > decoder_aix.hpp decoder_machO.hpp classLoader.hpp g1RootClosures.hpp >> > jvmtiImpl.hpp perfData.hpp decoder.hpp decoder_elf.hpp >> > >> > dynamic-class-memaccess: obscure use of memcpy >> > >> > method.cpp >> > >> > empty-body: ?;? isn?t good enough for clang, it prefers {} >> > >> > objectMonitor.cpp mallocSiteTable.cpp >> > >> > format: matches printf format strings against arguments. debug output will be affected by >> > incorrect code changes to these. >> > >> > macroAssembler_x86.cpp os_bsd.cpp os_bsd_x86.cpp ciMethodData.cpp javaClasses.cpp >> > debugInfo.cpp logFileOutput.cpp constantPool.cpp jvmtiEnter.xsl jvmtiRedefineClasses.cpp >> > safepoint.cpp thread.cpp >> > >> > logical-op-parentheses: can be tricky to get correct. There are a few very long-winded predicates. >> > >> > nativeInst_x86.hpp archDesc.cpp output_c.cpp output_h.cpp c1_GraphBuilder.cpp >> > c1_LIRGenerator.cpp c1_LinearScan.cpp bcEscapeAnalyzer.cpp ciMethod.cpp >> > stackMapTableFormat.hpp compressedStream.cpp dependencies.cpp heapRegion.cpp >> > ptrQueue.cpp psPromotionManager.cpp jvmciCompilerToVM.cpp cfgnode.cpp >> > chaitin.cpp compile.cpp compile.hpp escape.cpp graphKit.cpp lcm.cpp >> > loopTransform.cpp loopnode.cpp loopopts.cpp macro.cpp memnode.cpp >> > output.cpp parse1.cpp parseHelper.cpp reg_split.cpp superword.cpp >> > superword.hpp jniCheck.cpp jvmtiEventController.cpp arguments.cpp >> > javaCalls.cpp sharedRuntime.cpp >> > >> > parentheses >> > >> > adlparse.cpp >> > >> > parentheses-equality >> > >> > output_c.cpp javaAssertions.cpp gcm.cpp >> > >> > File-specific details: >> > >> > GensrcAdlc.gmk CompileJvm.gmk >> > Left tautological-compare in place to allow null 'this' pointer checks in methods >> > intended to be called from a debugger. >> > >> > CompileGTest.gmk >> > Just an enhanced comment. >> > >> > MacosxDebuggerLocal.m >> > PT_ATTACH has been replaced by PT_ATTACHEXC >> > >> > ciMethodData.cp >> > " 0x%" FORMAT64_MODIFIER "x" reduces to "0x%llx", whereas >> > " " INTPTRNZ_FORMAT reduces to "0x%lx", which latter is what clang want. >> > >> > generateOopMap.cpp >> > Refactored duplicate code in print_current_state(). >> > >> > binaryTreeDictionary.cpp/hpp, hashtable.cpp/hpp >> > These provoked ?instantiation of variable required here, >> > but no definition is available?. >> > >> > globalDefinitions_gcc.hpp >> > Define FORMAT64_MODIFIER properly for Apple, needed by os.cpp. >> > >> > globalDefinitions.hpp >> > Add INTPTRNZ_FORMAT, needed by ciMethodData.cpp. >> >> >> >> >> From christoph.langer at sap.com Mon Jul 3 07:36:20 2017 From: christoph.langer at sap.com (Langer, Christoph) Date: Mon, 3 Jul 2017 07:36:20 +0000 Subject: RFR(M): 8183227: read/write APIs in class os shall return ssize_t In-Reply-To: References: Message-ID: Ping: Any comments on this change...? From: Langer, Christoph Sent: Donnerstag, 29. Juni 2017 12:50 To: hotspot-runtime-dev at openjdk.java.net Subject: RFR(M): 8183227: read/write APIs in class os shall return ssize_t Hi folks, please review the following change: Bug: https://bugs.openjdk.java.net/browse/JDK-8183227 Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8183227.0/ When looking at the read() and write() APIs and its siblings on the platforms, you find the return value mostly being of signed type ssize_t vs. the current unsigned return type size_t of the methods in class os. With my proposed patch I try to address that and change the signatures of these methods to ssize_t. At the places where the methods were called so far, I could find 2 places where usage of the wrong type could be critical: src/share/vm/compiler/compileLog.cpp (where actually ::read is used) and src/share/vm/compiler/directivesParser.cpp where a wrong array access could happen As for the class os, I'm wondering if the methods "read_at" and "restartable_read" are still required? My source code scan tells me that there are no users of them. Shall I maybe remove them completely from the code? Thanks & Best regards Christoph From mikael.gerdin at oracle.com Mon Jul 3 07:53:09 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Mon, 3 Jul 2017 09:53:09 +0200 Subject: RFR (S) 8183229: Implement WindowsSemaphore::trywait In-Reply-To: References: <4FC8F3FB-B5ED-4618-BF89-9D46A1C56560@oracle.com> Message-ID: <0e556cf3-6c4b-890c-1215-1b31ab26f444@oracle.com> Just realized I didn't send this to the list! /m On 2017-06-30 10:56, Mikael Gerdin wrote: > Hi Kim > > On 2017-06-30 02:01, Kim Barrett wrote: >>> On Jun 29, 2017, at 8:50 AM, Mikael Gerdin >>> wrote: >>> >>> Hi all, >>> >>> To help with some upcoming patches I'd like to add the trywait >>> operation to the platform-independent Semaphore class. It's currently >>> lacking a Windows-implementation. >>> >>> Testing: JPRT testing on Windows platforms >>> Webrev: http://cr.openjdk.java.net/~mgerdin/8183229/webrev.0/ >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8183229 >>> >>> Thanks >>> /Mikael >> >> Code looks good, but needs a test for the new function in >> hotspot/test/native/runtime/test_semaphore.cpp. > > Oh! Of course! > > How about: > http://cr.openjdk.java.net/~mgerdin/8183229/webrev.0_to_1/ > http://cr.openjdk.java.net/~mgerdin/8183229/webrev.1/ > > I've run the unit test on all JRPT platforms. > > /Mikael > >> From erik.helin at oracle.com Mon Jul 3 07:58:57 2017 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 3 Jul 2017 09:58:57 +0200 Subject: RFR(XL): 8182299: Enable disabled clang warnings, build on OSX 10 + Xcode 8 In-Reply-To: References: <27FD0413-52BC-42E6-A5B0-3C92A49A2D6F@amazon.com> <1CBD62A9-9B1B-4B05-AAF9-BE2D52DE8C79@amazon.com> <6B296ABE-66C0-4C32-AC4E-8674BE103514@oracle.com> <3ACE75CF-CEA6-4081-952A-BBA138763582@amazon.com> <435C96F2-417E-4A47-8649-28E34E2908AE@amazon.com> <1d06f5fe-c3f8-f599-58bc-09b2099c819c@oracle.com> Message-ID: <035f2cd3-de68-efe7-d32e-bb6f06c58f93@oracle.com> On 06/30/2017 06:30 PM, Hohensee, Paul wrote: > Thanks for picking up the closed work, Jesper, and thanks for the review, Erik. It?s good to be back working on server Java. ( > > Yes, I added the G1RootClosures virtual destructor because of the warning. > > In heapRegionType.cpp, I just followed along with the existing funky indentation and made the default clause also have a return because all the other clauses do. Clang would indeed be ok with a default clause with just a break. Also, I?ve used the same idiom in other files, so I?d change those too if I change these. Which would you prefer? I?m not invested in any particular format. I don't know about the other files, but given the indentation in heapRegionType.cpp, I would prefer: switch (_tag) { // ... bunch of case statements case ArchiveTag: return "ARC"; default: break; } ShouldNotReachHere(); return NULL; // just to keep some compilers happy } My reason for this is mainly that all the other case statements are on one line (`case` followed by `return`), so I would prefer to the `default` case to also be on one line (`default` followed by `break`). However, for in HeapRegionType::is_valid, I think your changes follows the existing style in that method: 37 case ArchiveTag: 38 return true; 39 default: 40 return false; 41 } I feel like this is quickly approaching bikeshed territory, but the above is at least my preference :) Thanks, Erik > Thanks, > > Paul > > On 6/30/17, 3:00 AM, "jesper.wilhelmsson at oracle.com" wrote: > > I ran the change through JPRT so at least it builds and runs on all platforms we support there. > I'm currently fixing our closed code to build on Mac with this change. > /Jesper > > > On 30 Jun 2017, at 10:35, Erik Helin wrote: > > > > Hi Paul, > > > > thanks for contributing! Please see my comments regarding the GC changes below. > > > > On 06/29/2017 03:31 PM, Hohensee, Paul wrote: > >> I now have access to cr.openjdk.java.net, so the latest webrevs are at > >> > >> http://cr.openjdk.java.net/~phh/8182299/webrev_jdk.00/ > >> http://cr.openjdk.java.net/~phh/8182299/webrev_hotspot.00/ > > > > gc/g1/g1RootClosures.cpp: > > - // Closures to process raw oops in the root set. > > + virtual ~G1RootClosures() {} > > + > > +// Closures to process raw oops in the root set. > > > > I assume this is added because there is some warning about having only pure virtual methods but not having a virtual destructor. None of the classes inheriting from G1RootClosures needs a destructor, nor does G1RootClosures itself (it is just an "interface"). So there is no problem with correctness here :) > > > > However, I like to have lots of warnings enabled, so for me it is fine to an empty virtual destructor here just to please the compiler. > > > > gc/g1/heapRegion.cpp: looks correct > > > > gc/g1/heapRegionType.cpp: > > The indentation seems a bit funky to me here :) Also, have you compiled this on some other platforms? I think the last return is outside of the switch just to, as the comment says, "keep some compilers happy". > > > > Would clang be ok with having an empty default clause with just a break? And then fall through to the return outside of the switch? > > > > gc/g1/ptrQueue.cpp: looks correct > > > > gc/parallel/psPromotionManager.cpp: looks correct > > > > Thanks, > > Erik > > > >> On 6/28/17, 3:50 PM, "hotspot-dev on behalf of Hohensee, Paul" wrote: > >> > >> Thanks for the review, Jesper. > >> > >> New webrev sent, has only a change to nativeInst_x86.cpp. > >> > >> In nativeInst_x86.cpp, I formatted the expression so I could easily understand it, but you?re right, not everyone does it that way (maybe only me!), so I?ve changed it to > >> > >> if (((ubyte_at(0) & NativeTstRegMem::instruction_rex_prefix_mask) == NativeTstRegMem::instruction_rex_prefix && > >> ubyte_at(1) == NativeTstRegMem::instruction_code_memXregl && > >> (ubyte_at(2) & NativeTstRegMem::modrm_mask) == NativeTstRegMem::modrm_reg) || > >> (ubyte_at(0) == NativeTstRegMem::instruction_code_memXregl && > >> (ubyte_at(1) & NativeTstRegMem::modrm_mask) == NativeTstRegMem::modrm_reg)) { > >> > >> In graphKit.cpp, the old code was > >> > >> #ifdef ASSERT > >> case Deoptimization::Action_none: > >> case Deoptimization::Action_make_not_compilable: > >> break; > >> default: > >> fatal("unknown action %d: %s", action, Deoptimization::trap_action_name(action)); > >> break; > >> #endif > >> > >> In the non-ASSERT case, the compiler complained about the lack of Action_none, Action_make_not_compilable and default. If the warning had been turned off, the result would have been ?break;? for all three. In the ASSERT case, Action_none and Action_make_not_compilable result in ?break;?, and in the default case ?fatal(); break;? > >> > >> The new code is > >> > >> case Deoptimization::Action_none: > >> case Deoptimization::Action_make_not_compilable: > >> break; > >> default: > >> #ifdef ASSERT > >> fatal("unknown action %d: %s", action, Deoptimization::trap_action_name(action)); > >> #endif > >> break; > >> > >> The compiler doesn?t complain about Action_none, Action_make_not_compilable or default anymore. In the non-ASSERT case, the result is ?break;? for all three, same as for the old code. In the ASSERT case, Action_none and Action_make_not_compilable result in ?break;?, and in the default case ?fatal(); break;?, again same as for the old code. > >> > >> Thanks, > >> > >> Paul > >> > >> On 6/28/17, 10:36 AM, "jesper.wilhelmsson at oracle.com" wrote: > >> > >> Hi Paul, > >> > >> Thanks for doing this change! In general everything looks really good, there are a lot of really nice cleanups here. I just have two minor questions/nits: > >> > >> * In hotspot/cpu/x86/vm/nativeInst_x86.hpp it seems the expression already have parenthesis around the & operations and the change here is "only" cleaning up the layout of the code which is not a bad thing in it self, but you move the logical operators to the beginning of each line which is a quite different style than the rest of the code in the same function where the operators are at the end of the line. > >> > >> * In hotspot/share/vm/opto/graphKit.cpp you moved the #ifdef ASSERT so that Action_none and Action_make_not_compilable are available also when ASSERT is not defined. I don't see this mentioned in your description of the change. Was this change intentional? > >> > >> Thanks, > >> /Jesper > >> > >> > >> > On 27 Jun 2017, at 21:34, Hohensee, Paul wrote: > >> > > >> > An attempt at better formatting. > >> > > >> > https://bugs.openjdk.java.net/browse/JDK-8182299 > >> > http://cr.openjdk.java.net/~jwilhelm/8182299/webrev_jdk.00/ > >> > http://cr.openjdk.java.net/~jwilhelm/8182299/webrev_hotspot.00/ > >> > > >> > Jesper has been kind enough to host the webrevs while I get my cr.openjdk.net account set up, and to be the sponsor. > >> > > >> > This rfe a combination of enabling disabled clang warnings and getting jdk10 to build on OSX 10 and Xcode 8. At least one enabled warning (delete-non-virtual-dtor) detected what seems to me a real potential bug, with the rest enforcing good code hygiene. > >> > > >> > These changes are only in OpenJDK, so I?m looking for a volunteer to make the closed changes. > >> > > >> > Thanks, > >> > > >> > Paul > >> > > >> > > >> > Here are the jdk file-specific details: > >> > > >> > java_md_macosx.c splashscreen_sys.m > >> > > >> > Removed objc_registerThreadWithCollector() since it's obsolete and of questionable value in any case. > >> > > >> > NSApplicationAWT.m > >> > > >> > Use the correct NSEventMask rather than NSUInteger. > >> > > >> > jdhuff.c jdphuff.c > >> > > >> > Shifting a negative signed value is undefined. > >> > > >> > Here are the hotspot notes: > >> > > >> > Here are the lists of files affected by enabling a given warning: > >> > > >> > switch: all of these are lack of a default clause > >> > > >> > c1_LIRAssembler_x86.cpp c1_LIRGenerator_x86.cpp c1_LinearScan_x86.hpp > >> > jniFastGetField_x86_64.cpp assembler.cpp c1_Canonicalizer.cpp > >> > c1_GraphBuilder.cpp c1_Instruction.cpp c1_LIR.cpp c1_LIRGenerator.cpp > >> > c1_LinearScan.cpp c1_ValueStack.hpp c1_ValueType.cpp > >> > bcEscapeAnalyzer.cpp ciArray.cpp ciEnv.cpp ciInstance.cpp ciMethod.cpp > >> > ciMethodBlocks.cpp ciMethodData.cpp ciTypeFlow.cpp > >> > compiledMethod.cpp dependencies.cpp nmethod.cpp compileTask.hpp > >> > heapRegionType.cpp abstractInterpreter.cpp bytecodes.cpp > >> > invocationCounter.cpp linkResolver.cpp rewriter.cpp jvmciCompilerToVM.cpp > >> > jvmciEnv.cpp universe.cpp cpCache.cpp generateOopMap.cpp > >> > method.cpp methodData.cpp compile.cpp connode.cpp gcm.cpp graphKit.cpp > >> > ifnode.cpp library_call.cpp memnode.cpp parse1.cpp > >> > parse2.cpp phaseX.cpp superword.cpp type.cpp vectornode.cpp > >> > jvmtiClassFileReconstituter.cpp jvmtiEnter.xsl jvmtiEventController.cpp > >> > jvmtiImpl.cpp jvmtiRedefineClasses.cpp methodComparator.cpp methodHandles.cpp > >> > advancedThresholdPolicy.cpp reflection.cpp relocator.cpp sharedRuntime.cpp > >> > simpleThresholdPolicy.cpp writeableFlags.cpp globalDefinitions.hpp > >> > > >> > delete-non-virtual-dtor: these may be real latent bugs due to possible failure to execute destructor(s) > >> > > >> > decoder_aix.hpp decoder_machO.hpp classLoader.hpp g1RootClosures.hpp > >> > jvmtiImpl.hpp perfData.hpp decoder.hpp decoder_elf.hpp > >> > > >> > dynamic-class-memaccess: obscure use of memcpy > >> > > >> > method.cpp > >> > > >> > empty-body: ?;? isn?t good enough for clang, it prefers {} > >> > > >> > objectMonitor.cpp mallocSiteTable.cpp > >> > > >> > format: matches printf format strings against arguments. debug output will be affected by > >> > incorrect code changes to these. > >> > > >> > macroAssembler_x86.cpp os_bsd.cpp os_bsd_x86.cpp ciMethodData.cpp javaClasses.cpp > >> > debugInfo.cpp logFileOutput.cpp constantPool.cpp jvmtiEnter.xsl jvmtiRedefineClasses.cpp > >> > safepoint.cpp thread.cpp > >> > > >> > logical-op-parentheses: can be tricky to get correct. There are a few very long-winded predicates. > >> > > >> > nativeInst_x86.hpp archDesc.cpp output_c.cpp output_h.cpp c1_GraphBuilder.cpp > >> > c1_LIRGenerator.cpp c1_LinearScan.cpp bcEscapeAnalyzer.cpp ciMethod.cpp > >> > stackMapTableFormat.hpp compressedStream.cpp dependencies.cpp heapRegion.cpp > >> > ptrQueue.cpp psPromotionManager.cpp jvmciCompilerToVM.cpp cfgnode.cpp > >> > chaitin.cpp compile.cpp compile.hpp escape.cpp graphKit.cpp lcm.cpp > >> > loopTransform.cpp loopnode.cpp loopopts.cpp macro.cpp memnode.cpp > >> > output.cpp parse1.cpp parseHelper.cpp reg_split.cpp superword.cpp > >> > superword.hpp jniCheck.cpp jvmtiEventController.cpp arguments.cpp > >> > javaCalls.cpp sharedRuntime.cpp > >> > > >> > parentheses > >> > > >> > adlparse.cpp > >> > > >> > parentheses-equality > >> > > >> > output_c.cpp javaAssertions.cpp gcm.cpp > >> > > >> > File-specific details: > >> > > >> > GensrcAdlc.gmk CompileJvm.gmk > >> > Left tautological-compare in place to allow null 'this' pointer checks in methods > >> > intended to be called from a debugger. > >> > > >> > CompileGTest.gmk > >> > Just an enhanced comment. > >> > > >> > MacosxDebuggerLocal.m > >> > PT_ATTACH has been replaced by PT_ATTACHEXC > >> > > >> > ciMethodData.cp > >> > " 0x%" FORMAT64_MODIFIER "x" reduces to "0x%llx", whereas > >> > " " INTPTRNZ_FORMAT reduces to "0x%lx", which latter is what clang want. > >> > > >> > generateOopMap.cpp > >> > Refactored duplicate code in print_current_state(). > >> > > >> > binaryTreeDictionary.cpp/hpp, hashtable.cpp/hpp > >> > These provoked ?instantiation of variable required here, > >> > but no definition is available?. > >> > > >> > globalDefinitions_gcc.hpp > >> > Define FORMAT64_MODIFIER properly for Apple, needed by os.cpp. > >> > > >> > globalDefinitions.hpp > >> > Add INTPTRNZ_FORMAT, needed by ciMethodData.cpp. > >> > >> > >> > >> > >> > > > From stefan.karlsson at oracle.com Mon Jul 3 08:05:48 2017 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 3 Jul 2017 10:05:48 +0200 Subject: RFR (S) 8183229: Implement WindowsSemaphore::trywait In-Reply-To: <0e556cf3-6c4b-890c-1215-1b31ab26f444@oracle.com> References: <4FC8F3FB-B5ED-4618-BF89-9D46A1C56560@oracle.com> <0e556cf3-6c4b-890c-1215-1b31ab26f444@oracle.com> Message-ID: <79c4790d-2994-e617-1387-7987bb0b877e@oracle.com> Looks good. StefanK On 2017-07-03 09:53, Mikael Gerdin wrote: > Just realized I didn't send this to the list! > /m > > On 2017-06-30 10:56, Mikael Gerdin wrote: >> Hi Kim >> >> On 2017-06-30 02:01, Kim Barrett wrote: >>>> On Jun 29, 2017, at 8:50 AM, Mikael Gerdin >>>> wrote: >>>> >>>> Hi all, >>>> >>>> To help with some upcoming patches I'd like to add the trywait >>>> operation to the platform-independent Semaphore class. It's >>>> currently lacking a Windows-implementation. >>>> >>>> Testing: JPRT testing on Windows platforms >>>> Webrev: http://cr.openjdk.java.net/~mgerdin/8183229/webrev.0/ >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8183229 >>>> >>>> Thanks >>>> /Mikael >>> >>> Code looks good, but needs a test for the new function in >>> hotspot/test/native/runtime/test_semaphore.cpp. >> >> Oh! Of course! >> >> How about: >> http://cr.openjdk.java.net/~mgerdin/8183229/webrev.0_to_1/ >> http://cr.openjdk.java.net/~mgerdin/8183229/webrev.1/ >> >> I've run the unit test on all JRPT platforms. >> >> /Mikael >> >>> From erik.helin at oracle.com Mon Jul 3 14:19:33 2017 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 3 Jul 2017 16:19:33 +0200 Subject: RFR(XL): 8181917: Refactor UL LogStreams to avoid using resource area In-Reply-To: References: <337330d2-512c-bc77-5a55-c05d33e376e5@oracle.com> Message-ID: On 06/30/2017 08:32 PM, Thomas St?fe wrote: > New > Version: http://cr.openjdk.java.net/~stuefe/webrevs/8181917-refactor-ul-logstream/all.webrev.03/webrev/ > > Delta to > last: http://cr.openjdk.java.net/~stuefe/webrevs/8181917-refactor-ul-logstream/delta-02-to-03/webrev/ Thanks, I will take a look later. On 06/30/2017 08:32 PM, Thomas St?fe wrote: > - should you add declarations of delete as well to prevent someone from > calling delete on LogStream pointer? > > > Tried this, but I think I ran into linker errors. I do not think it is > needed, though. Hmm, that seems strange, a linker error would/could indicate that some code _is_ calling delete on a LogStream*... On 06/30/2017 08:32 PM, Thomas St?fe wrote: > - is 128 bytes as default too much for _smallbuf? Should we start with > 64? > > I changed it to 64. Ok. On 06/30/2017 08:32 PM, Thomas St?fe wrote: > logStream.cpp > - one extra newline at line 74 > - the pointer returned from os::malloc is not checked. What should we > do if a NULL pointer is returned? Print whatever is buffered and then > do vm_out_of_memory? > - is growing by doubling too aggressive? should the LineBuffer instead > grow by chunking (e.g. always add 64 more bytes)? > > > Okay. I changed the allocation of the line buffer such that > - we do not allocate beyond a reasonable limit (1M hardwired), to > prevent runaway leaks. > - we cope with OOM > In both cases, LogStream continues to work but may truncate the output- > > I also added a test case for the former scenario (the 1M cap). Ok, sounds good, I will have a look. On 06/30/2017 08:32 PM, Thomas St?fe wrote: > - instead of growing the LineBuffer by reallocating and then copying > over the old bytes, should we use a chunked linked list instead (in > order to avoid copying the same data multiple times)? The only > "requirements" on the LineBuffer is fast append and fast iteration > (doesn't have to support fast index lookup). > > > Not really sure how this would work. The whole point of LogStream is to > assemble one log line, in the form of a single continuous > zero-terminated string, and pass that to the UL backend as one single > log call, yes? How would you do this with disjunct chunks? The key point here is "as one single log call". If we allow ourselves to pass a more elaborate data structure than a plain string (such as a linked list of strings), then we can enable this kind of "chunked expansion". The write call will eventually result in flockfile(_stream); // do stuff fflush(_stream); funlockfile(_stream); deep down in logFileStreamOutput.cpp. Whatever we do at `// do stuff` will become atomic wrt to other threads. We could for example iterate over a linked list of strings and print them (with space as separator). For an example, see LogFileStreamOutput::write(LogMessageBuffer::Iterator msg_iterator). The worst case for this patch is if the log line becomes larger than 512 bytes. In this case, there will be plenty of copying as LineBuffer gets expanded and then LogTagSet::vwrite will also allocate a buffer on the heap, copy everything over (again), print that buffer, free it. It seems like we can do better here :) Whether this should be done as part of this patch, or as a follow-up patch, that is another matter. > - LogStream::write is no longer in inline.hpp, is this a potential > performance problem? I think not, ,the old LogStream::write > definition was most likely in .inline.hpp because of template usage > > > I do not think so either. I do not like implementations in headers > (clutters the interface), unless it is worth it performance wise. Here, > LogStream::write is usually called from one of the > outputStream::print()... functions via virtual function call; not sure > how that could even be inlined. > > > > Great work thus far Thomas, the patch is becoming really solid! If > you want to discuss over IM, then you can (as always) find me in > #openjdk on irc.oftc.net . > > > Thanks :) And thanks for reviewing! Thanks, Erik > ...Thomas > > > > Thanks, > Erik > > Kind Regards, Thomas > > From kim.barrett at oracle.com Mon Jul 3 15:14:11 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 3 Jul 2017 11:14:11 -0400 Subject: RFR (S) 8183229: Implement WindowsSemaphore::trywait In-Reply-To: <0e556cf3-6c4b-890c-1215-1b31ab26f444@oracle.com> References: <4FC8F3FB-B5ED-4618-BF89-9D46A1C56560@oracle.com> <0e556cf3-6c4b-890c-1215-1b31ab26f444@oracle.com> Message-ID: <66793B73-B472-47B8-85E5-D26EBE8A67B8@oracle.com> > On Jul 3, 2017, at 3:53 AM, Mikael Gerdin wrote: > > Just realized I didn't send this to the list! > /m Looks good. > > On 2017-06-30 10:56, Mikael Gerdin wrote: >> Hi Kim >> On 2017-06-30 02:01, Kim Barrett wrote: >>>> On Jun 29, 2017, at 8:50 AM, Mikael Gerdin wrote: >>>> >>>> Hi all, >>>> >>>> To help with some upcoming patches I'd like to add the trywait operation to the platform-independent Semaphore class. It's currently lacking a Windows-implementation. >>>> >>>> Testing: JPRT testing on Windows platforms >>>> Webrev: http://cr.openjdk.java.net/~mgerdin/8183229/webrev.0/ >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8183229 >>>> >>>> Thanks >>>> /Mikael >>> >>> Code looks good, but needs a test for the new function in hotspot/test/native/runtime/test_semaphore.cpp. >> Oh! Of course! >> How about: >> http://cr.openjdk.java.net/~mgerdin/8183229/webrev.0_to_1/ >> http://cr.openjdk.java.net/~mgerdin/8183229/webrev.1/ >> I've run the unit test on all JRPT platforms. >> /Mikael From mikael.gerdin at oracle.com Mon Jul 3 15:21:55 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Mon, 3 Jul 2017 17:21:55 +0200 Subject: RFR (S) 8183229: Implement WindowsSemaphore::trywait In-Reply-To: <66793B73-B472-47B8-85E5-D26EBE8A67B8@oracle.com> References: <4FC8F3FB-B5ED-4618-BF89-9D46A1C56560@oracle.com> <0e556cf3-6c4b-890c-1215-1b31ab26f444@oracle.com> <66793B73-B472-47B8-85E5-D26EBE8A67B8@oracle.com> Message-ID: <74b6c274-4aef-69ff-1620-11dc40d0f5bc@oracle.com> Hi Kim, On 2017-07-03 17:14, Kim Barrett wrote: >> On Jul 3, 2017, at 3:53 AM, Mikael Gerdin wrote: >> >> Just realized I didn't send this to the list! >> /m > > Looks good. Thanks for the review! /Mikael > >> >> On 2017-06-30 10:56, Mikael Gerdin wrote: >>> Hi Kim >>> On 2017-06-30 02:01, Kim Barrett wrote: >>>>> On Jun 29, 2017, at 8:50 AM, Mikael Gerdin wrote: >>>>> >>>>> Hi all, >>>>> >>>>> To help with some upcoming patches I'd like to add the trywait operation to the platform-independent Semaphore class. It's currently lacking a Windows-implementation. >>>>> >>>>> Testing: JPRT testing on Windows platforms >>>>> Webrev: http://cr.openjdk.java.net/~mgerdin/8183229/webrev.0/ >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8183229 >>>>> >>>>> Thanks >>>>> /Mikael >>>> >>>> Code looks good, but needs a test for the new function in hotspot/test/native/runtime/test_semaphore.cpp. >>> Oh! Of course! >>> How about: >>> http://cr.openjdk.java.net/~mgerdin/8183229/webrev.0_to_1/ >>> http://cr.openjdk.java.net/~mgerdin/8183229/webrev.1/ >>> I've run the unit test on all JRPT platforms. >>> /Mikael > > From claes.redestad at oracle.com Mon Jul 3 15:24:59 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 3 Jul 2017 17:24:59 +0200 Subject: RFR (S) 8183229: Implement WindowsSemaphore::trywait In-Reply-To: <0e556cf3-6c4b-890c-1215-1b31ab26f444@oracle.com> References: <4FC8F3FB-B5ED-4618-BF89-9D46A1C56560@oracle.com> <0e556cf3-6c4b-890c-1215-1b31ab26f444@oracle.com> Message-ID: <7e1f5a4f-cb56-5fb5-8ec5-6f53d91b5f58@oracle.com> On 07/03/2017 09:53 AM, Mikael Gerdin wrote: >> http://cr.openjdk.java.net/~mgerdin/8183229/webrev.1/ Looks good! /Claes From mikael.gerdin at oracle.com Mon Jul 3 15:36:51 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Mon, 3 Jul 2017 17:36:51 +0200 Subject: RFR (S) 8183229: Implement WindowsSemaphore::trywait In-Reply-To: <7e1f5a4f-cb56-5fb5-8ec5-6f53d91b5f58@oracle.com> References: <4FC8F3FB-B5ED-4618-BF89-9D46A1C56560@oracle.com> <0e556cf3-6c4b-890c-1215-1b31ab26f444@oracle.com> <7e1f5a4f-cb56-5fb5-8ec5-6f53d91b5f58@oracle.com> Message-ID: Hi Claes, On 2017-07-03 17:24, Claes Redestad wrote: > > > On 07/03/2017 09:53 AM, Mikael Gerdin wrote: >>> http://cr.openjdk.java.net/~mgerdin/8183229/webrev.1/ > > Looks good! Thanks for the review! /m > > /Claes > From hohensee at amazon.com Mon Jul 3 16:47:44 2017 From: hohensee at amazon.com (Hohensee, Paul) Date: Mon, 3 Jul 2017 16:47:44 +0000 Subject: RFR(XL): 8182299: Enable disabled clang warnings, build on OSX 10 + Xcode 8 In-Reply-To: <035f2cd3-de68-efe7-d32e-bb6f06c58f93@oracle.com> References: <27FD0413-52BC-42E6-A5B0-3C92A49A2D6F@amazon.com> <1CBD62A9-9B1B-4B05-AAF9-BE2D52DE8C79@amazon.com> <6B296ABE-66C0-4C32-AC4E-8674BE103514@oracle.com> <3ACE75CF-CEA6-4081-952A-BBA138763582@amazon.com> <435C96F2-417E-4A47-8649-28E34E2908AE@amazon.com> <1d06f5fe-c3f8-f599-58bc-09b2099c819c@oracle.com> <035f2cd3-de68-efe7-d32e-bb6f06c58f93@oracle.com> Message-ID: <901E4351-42D1-44D4-85F6-FBD4C715797A@amazon.com> Fixed heapRegionType.cpp per your comment. http://cr.openjdk.java.net/~phh/8182299/webrev_hotspot.02/ Jesper says he?s ready to push the closed changes. Any other comments on the open ones? Thanks, Paul On 7/3/17, 12:58 AM, "Erik Helin" wrote: On 06/30/2017 06:30 PM, Hohensee, Paul wrote: > Thanks for picking up the closed work, Jesper, and thanks for the review, Erik. It?s good to be back working on server Java. ( > > Yes, I added the G1RootClosures virtual destructor because of the warning. > > In heapRegionType.cpp, I just followed along with the existing funky indentation and made the default clause also have a return because all the other clauses do. Clang would indeed be ok with a default clause with just a break. Also, I?ve used the same idiom in other files, so I?d change those too if I change these. Which would you prefer? I?m not invested in any particular format. I don't know about the other files, but given the indentation in heapRegionType.cpp, I would prefer: switch (_tag) { // ... bunch of case statements case ArchiveTag: return "ARC"; default: break; } ShouldNotReachHere(); return NULL; // just to keep some compilers happy } My reason for this is mainly that all the other case statements are on one line (`case` followed by `return`), so I would prefer to the `default` case to also be on one line (`default` followed by `break`). However, for in HeapRegionType::is_valid, I think your changes follows the existing style in that method: 37 case ArchiveTag: 38 return true; 39 default: 40 return false; 41 } I feel like this is quickly approaching bikeshed territory, but the above is at least my preference :) Thanks, Erik > Thanks, > > Paul > > On 6/30/17, 3:00 AM, "jesper.wilhelmsson at oracle.com" wrote: > > I ran the change through JPRT so at least it builds and runs on all platforms we support there. > I'm currently fixing our closed code to build on Mac with this change. > /Jesper > > > On 30 Jun 2017, at 10:35, Erik Helin wrote: > > > > Hi Paul, > > > > thanks for contributing! Please see my comments regarding the GC changes below. > > > > On 06/29/2017 03:31 PM, Hohensee, Paul wrote: > >> I now have access to cr.openjdk.java.net, so the latest webrevs are at > >> > >> http://cr.openjdk.java.net/~phh/8182299/webrev_jdk.00/ > >> http://cr.openjdk.java.net/~phh/8182299/webrev_hotspot.00/ > > > > gc/g1/g1RootClosures.cpp: > > - // Closures to process raw oops in the root set. > > + virtual ~G1RootClosures() {} > > + > > +// Closures to process raw oops in the root set. > > > > I assume this is added because there is some warning about having only pure virtual methods but not having a virtual destructor. None of the classes inheriting from G1RootClosures needs a destructor, nor does G1RootClosures itself (it is just an "interface"). So there is no problem with correctness here :) > > > > However, I like to have lots of warnings enabled, so for me it is fine to an empty virtual destructor here just to please the compiler. > > > > gc/g1/heapRegion.cpp: looks correct > > > > gc/g1/heapRegionType.cpp: > > The indentation seems a bit funky to me here :) Also, have you compiled this on some other platforms? I think the last return is outside of the switch just to, as the comment says, "keep some compilers happy". > > > > Would clang be ok with having an empty default clause with just a break? And then fall through to the return outside of the switch? > > > > gc/g1/ptrQueue.cpp: looks correct > > > > gc/parallel/psPromotionManager.cpp: looks correct > > > > Thanks, > > Erik > > > >> On 6/28/17, 3:50 PM, "hotspot-dev on behalf of Hohensee, Paul" wrote: > >> > >> Thanks for the review, Jesper. > >> > >> New webrev sent, has only a change to nativeInst_x86.cpp. > >> > >> In nativeInst_x86.cpp, I formatted the expression so I could easily understand it, but you?re right, not everyone does it that way (maybe only me!), so I?ve changed it to > >> > >> if (((ubyte_at(0) & NativeTstRegMem::instruction_rex_prefix_mask) == NativeTstRegMem::instruction_rex_prefix && > >> ubyte_at(1) == NativeTstRegMem::instruction_code_memXregl && > >> (ubyte_at(2) & NativeTstRegMem::modrm_mask) == NativeTstRegMem::modrm_reg) || > >> (ubyte_at(0) == NativeTstRegMem::instruction_code_memXregl && > >> (ubyte_at(1) & NativeTstRegMem::modrm_mask) == NativeTstRegMem::modrm_reg)) { > >> > >> In graphKit.cpp, the old code was > >> > >> #ifdef ASSERT > >> case Deoptimization::Action_none: > >> case Deoptimization::Action_make_not_compilable: > >> break; > >> default: > >> fatal("unknown action %d: %s", action, Deoptimization::trap_action_name(action)); > >> break; > >> #endif > >> > >> In the non-ASSERT case, the compiler complained about the lack of Action_none, Action_make_not_compilable and default. If the warning had been turned off, the result would have been ?break;? for all three. In the ASSERT case, Action_none and Action_make_not_compilable result in ?break;?, and in the default case ?fatal(); break;? > >> > >> The new code is > >> > >> case Deoptimization::Action_none: > >> case Deoptimization::Action_make_not_compilable: > >> break; > >> default: > >> #ifdef ASSERT > >> fatal("unknown action %d: %s", action, Deoptimization::trap_action_name(action)); > >> #endif > >> break; > >> > >> The compiler doesn?t complain about Action_none, Action_make_not_compilable or default anymore. In the non-ASSERT case, the result is ?break;? for all three, same as for the old code. In the ASSERT case, Action_none and Action_make_not_compilable result in ?break;?, and in the default case ?fatal(); break;?, again same as for the old code. > >> > >> Thanks, > >> > >> Paul > >> > >> On 6/28/17, 10:36 AM, "jesper.wilhelmsson at oracle.com" wrote: > >> > >> Hi Paul, > >> > >> Thanks for doing this change! In general everything looks really good, there are a lot of really nice cleanups here. I just have two minor questions/nits: > >> > >> * In hotspot/cpu/x86/vm/nativeInst_x86.hpp it seems the expression already have parenthesis around the & operations and the change here is "only" cleaning up the layout of the code which is not a bad thing in it self, but you move the logical operators to the beginning of each line which is a quite different style than the rest of the code in the same function where the operators are at the end of the line. > >> > >> * In hotspot/share/vm/opto/graphKit.cpp you moved the #ifdef ASSERT so that Action_none and Action_make_not_compilable are available also when ASSERT is not defined. I don't see this mentioned in your description of the change. Was this change intentional? > >> > >> Thanks, > >> /Jesper > >> > >> > >> > On 27 Jun 2017, at 21:34, Hohensee, Paul wrote: > >> > > >> > An attempt at better formatting. > >> > > >> > https://bugs.openjdk.java.net/browse/JDK-8182299 > >> > http://cr.openjdk.java.net/~jwilhelm/8182299/webrev_jdk.00/ > >> > http://cr.openjdk.java.net/~jwilhelm/8182299/webrev_hotspot.00/ > >> > > >> > Jesper has been kind enough to host the webrevs while I get my cr.openjdk.net account set up, and to be the sponsor. > >> > > >> > This rfe a combination of enabling disabled clang warnings and getting jdk10 to build on OSX 10 and Xcode 8. At least one enabled warning (delete-non-virtual-dtor) detected what seems to me a real potential bug, with the rest enforcing good code hygiene. > >> > > >> > These changes are only in OpenJDK, so I?m looking for a volunteer to make the closed changes. > >> > > >> > Thanks, > >> > > >> > Paul > >> > > >> > > >> > Here are the jdk file-specific details: > >> > > >> > java_md_macosx.c splashscreen_sys.m > >> > > >> > Removed objc_registerThreadWithCollector() since it's obsolete and of questionable value in any case. > >> > > >> > NSApplicationAWT.m > >> > > >> > Use the correct NSEventMask rather than NSUInteger. > >> > > >> > jdhuff.c jdphuff.c > >> > > >> > Shifting a negative signed value is undefined. > >> > > >> > Here are the hotspot notes: > >> > > >> > Here are the lists of files affected by enabling a given warning: > >> > > >> > switch: all of these are lack of a default clause > >> > > >> > c1_LIRAssembler_x86.cpp c1_LIRGenerator_x86.cpp c1_LinearScan_x86.hpp > >> > jniFastGetField_x86_64.cpp assembler.cpp c1_Canonicalizer.cpp > >> > c1_GraphBuilder.cpp c1_Instruction.cpp c1_LIR.cpp c1_LIRGenerator.cpp > >> > c1_LinearScan.cpp c1_ValueStack.hpp c1_ValueType.cpp > >> > bcEscapeAnalyzer.cpp ciArray.cpp ciEnv.cpp ciInstance.cpp ciMethod.cpp > >> > ciMethodBlocks.cpp ciMethodData.cpp ciTypeFlow.cpp > >> > compiledMethod.cpp dependencies.cpp nmethod.cpp compileTask.hpp > >> > heapRegionType.cpp abstractInterpreter.cpp bytecodes.cpp > >> > invocationCounter.cpp linkResolver.cpp rewriter.cpp jvmciCompilerToVM.cpp > >> > jvmciEnv.cpp universe.cpp cpCache.cpp generateOopMap.cpp > >> > method.cpp methodData.cpp compile.cpp connode.cpp gcm.cpp graphKit.cpp > >> > ifnode.cpp library_call.cpp memnode.cpp parse1.cpp > >> > parse2.cpp phaseX.cpp superword.cpp type.cpp vectornode.cpp > >> > jvmtiClassFileReconstituter.cpp jvmtiEnter.xsl jvmtiEventController.cpp > >> > jvmtiImpl.cpp jvmtiRedefineClasses.cpp methodComparator.cpp methodHandles.cpp > >> > advancedThresholdPolicy.cpp reflection.cpp relocator.cpp sharedRuntime.cpp > >> > simpleThresholdPolicy.cpp writeableFlags.cpp globalDefinitions.hpp > >> > > >> > delete-non-virtual-dtor: these may be real latent bugs due to possible failure to execute destructor(s) > >> > > >> > decoder_aix.hpp decoder_machO.hpp classLoader.hpp g1RootClosures.hpp > >> > jvmtiImpl.hpp perfData.hpp decoder.hpp decoder_elf.hpp > >> > > >> > dynamic-class-memaccess: obscure use of memcpy > >> > > >> > method.cpp > >> > > >> > empty-body: ?;? isn?t good enough for clang, it prefers {} > >> > > >> > objectMonitor.cpp mallocSiteTable.cpp > >> > > >> > format: matches printf format strings against arguments. debug output will be affected by > >> > incorrect code changes to these. > >> > > >> > macroAssembler_x86.cpp os_bsd.cpp os_bsd_x86.cpp ciMethodData.cpp javaClasses.cpp > >> > debugInfo.cpp logFileOutput.cpp constantPool.cpp jvmtiEnter.xsl jvmtiRedefineClasses.cpp > >> > safepoint.cpp thread.cpp > >> > > >> > logical-op-parentheses: can be tricky to get correct. There are a few very long-winded predicates. > >> > > >> > nativeInst_x86.hpp archDesc.cpp output_c.cpp output_h.cpp c1_GraphBuilder.cpp > >> > c1_LIRGenerator.cpp c1_LinearScan.cpp bcEscapeAnalyzer.cpp ciMethod.cpp > >> > stackMapTableFormat.hpp compressedStream.cpp dependencies.cpp heapRegion.cpp > >> > ptrQueue.cpp psPromotionManager.cpp jvmciCompilerToVM.cpp cfgnode.cpp > >> > chaitin.cpp compile.cpp compile.hpp escape.cpp graphKit.cpp lcm.cpp > >> > loopTransform.cpp loopnode.cpp loopopts.cpp macro.cpp memnode.cpp > >> > output.cpp parse1.cpp parseHelper.cpp reg_split.cpp superword.cpp > >> > superword.hpp jniCheck.cpp jvmtiEventController.cpp arguments.cpp > >> > javaCalls.cpp sharedRuntime.cpp > >> > > >> > parentheses > >> > > >> > adlparse.cpp > >> > > >> > parentheses-equality > >> > > >> > output_c.cpp javaAssertions.cpp gcm.cpp > >> > > >> > File-specific details: > >> > > >> > GensrcAdlc.gmk CompileJvm.gmk > >> > Left tautological-compare in place to allow null 'this' pointer checks in methods > >> > intended to be called from a debugger. > >> > > >> > CompileGTest.gmk > >> > Just an enhanced comment. > >> > > >> > MacosxDebuggerLocal.m > >> > PT_ATTACH has been replaced by PT_ATTACHEXC > >> > > >> > ciMethodData.cp > >> > " 0x%" FORMAT64_MODIFIER "x" reduces to "0x%llx", whereas > >> > " " INTPTRNZ_FORMAT reduces to "0x%lx", which latter is what clang want. > >> > > >> > generateOopMap.cpp > >> > Refactored duplicate code in print_current_state(). > >> > > >> > binaryTreeDictionary.cpp/hpp, hashtable.cpp/hpp > >> > These provoked ?instantiation of variable required here, > >> > but no definition is available?. > >> > > >> > globalDefinitions_gcc.hpp > >> > Define FORMAT64_MODIFIER properly for Apple, needed by os.cpp. > >> > > >> > globalDefinitions.hpp > >> > Add INTPTRNZ_FORMAT, needed by ciMethodData.cpp. > >> > >> > >> > >> > >> > > > From robbin.ehn at oracle.com Tue Jul 4 08:22:49 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 4 Jul 2017 10:22:49 +0200 Subject: RFR(s): 8179327: gtestLauncher should run tests on a separate thread (optionally) In-Reply-To: References: Message-ID: <438277b7-921c-f4e3-ee12-201ad45e3eef@oracle.com> Looks reasonable, thanks! /Robbin If you don't mind, please don't wrap lines, links do not work well with line breaks in them :) On 07/02/2017 09:23 AM, Thomas St?fe wrote: > Dear all, > > please review the following change: > > Bug: JDK-8179327 > Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/ > 8179327-gtestLauncher-should-have-an-option-to-avoid- > primordial-thread-for-tests/webrev.00/webrev/ > > This fix is motivated by the inability of the AIX openjdk vm to run on the > primordial thread (see https://bugs.openjdk.java.net/browse/JDK-8171505), > which we unfortunately cannot do much about. This prevented us until now to > run gtests successfully, because the gtestlauncher (in contrast to the > standard java launcher) invokes the VM on the primordial thread. > > To fix this, I gave the gtest launcher the ability to run the tests in an > own thread. Instead of sprinkling the source with "#ifdef _AIX", I added a > command line option "-new-thread", which allows to start the tests in a new > thread, and implemented this feature for all platforms (well, posix and > windows). On all platforms save AIX this is by default off, so nothing > changes. On AIX, tests will be default be executed in a new thread. > > Thanks for reviewing, > > Kind Regards, Thomas > From robbin.ehn at oracle.com Tue Jul 4 08:40:59 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 4 Jul 2017 10:40:59 +0200 Subject: RFR: 8178500: Replace usages of round_to and round_down with align_up and align_down In-Reply-To: References: Message-ID: <68ebaf00-bf89-f9c7-5de9-d869f334778e@oracle.com> Hi Stefan, Looks ok, thanks for fixing. /Robbin On 06/30/2017 06:12 PM, Stefan Karlsson wrote: > Hi all, > > Please review this patch to remove round_to and round_down and replace their usages with align_up and align_down. > > http://cr.openjdk.java.net/~stefank/8178500/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8178500 > > The round_to and round_down functions asserted that alignment was a power-of-2. I've added asserts to the corresponding align functions. > > This patch builds upon all the patches referred to in: > http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-June/027329.html > > Thanks, > StefanK From robbin.ehn at oracle.com Tue Jul 4 08:44:12 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 4 Jul 2017 10:44:12 +0200 Subject: RFR: 8178499: Remove _ptr_ and _size_ infixes from align functions In-Reply-To: References: Message-ID: Looks good. /Robbin On 06/30/2017 11:40 AM, Stefan Karlsson wrote: > Hi all, > > Please review this patch to remove the _ptr_ and _size_ infixes from align functions. > > http://cr.openjdk.java.net/~stefank/8178499/webrev.00 > http://bugs.openjdk.java.net/browse/JDK-8178499 > > Rename functions (and corresponding macros) from: > align_ptr_up > align_size_up > align_ptr_down > align_size_down > is_ptr_aligned > is_size_aligned > > to: > align_up > align_down > is_aligned > > The patch builds upon the changes in: > http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-June/027328.html > > Thanks, > StefanK From robbin.ehn at oracle.com Tue Jul 4 08:44:40 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 4 Jul 2017 10:44:40 +0200 Subject: RFR: 8178489: Make align functions more type safe and consistent In-Reply-To: <23768f79-d356-a7fc-833c-4932f18fc0d1@oracle.com> References: <23768f79-d356-a7fc-833c-4932f18fc0d1@oracle.com> Message-ID: <6509977d-16d0-4693-78d7-29372a275de4@oracle.com> Looks good. Thanks, Robbin On 06/30/2017 11:16 AM, Stefan Karlsson wrote: > Hi all, > > Please review this patch to make the align functions more type safe and consistent. > > http://cr.openjdk.java.net/~stefank/8178489/webrev.00 > https://bugs.openjdk.java.net/browse/JDK-8178489 > > Note that this patch needs to be applied on top of the following patches that are out for review: > http://cr.openjdk.java.net/~stefank/8178491/webrev.02 > http://cr.openjdk.java.net/~stefank/8178495/webrev.01 > > Currently, the align functions forces the user to often explicitly cast either the input parameters, or the return type, or both. > > Two examples of the current API: > inline intptr_t align_size_up(intptr_t size, intptr_t alignment); > inline void* align_ptr_up(const void* ptr, size_t alignment); > > I propose that we change the API to use templates to return the aligned value as the same type as the type of the unaligned input. > > The proposed API would look like this: > > template > inline T align_size_up(T size, A alignment); > > template > inline T* align_ptr_up(T* ptr, A alignment); > > and a follow-up RFE (JDK-8178499) would get rid of _size_ and _ptr_ from the names. > > Usages of these align functions would then look like: > > size_t aligned_size = align_up(alloc_size, os::vm_page_size()) > HeapWord* aligned_top = align_up(top, region_size) > > Please, take an extra close look at the reinterpret_cast I added in atomic.hpp. This was needed because the align_ptr_down now returns T* and not void*, and the compiler complained when we tried to do a static cast from a volatile jbyte* to a volatile jint*. > > Tested with the align unit test and JPRT. > > Thanks, > StefanK From volker.simonis at gmail.com Tue Jul 4 09:05:18 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 4 Jul 2017 11:05:18 +0200 Subject: [10] RFR(XS): 8183534: [TEST] Make detection of compilation mode more robust Message-ID: Hi, can you please review the following trivial change which makes the detection of compilation mode for JTreg more robust: http://cr.openjdk.java.net/~simonis/webrevs/2017/8183534/ https://bugs.openjdk.java.net/browse/JDK-8183534 The compilation mode in JTreg is parsed from the "java.vm.info" system property. The current implementation expects that "java.vm.info" contains exactly one of the strings "mixed mode", "compiled mode" or "interpreted mode" otherwise the detection will fail and JTreg will quit with an error. The detection can be done more robust by searching for the substrings "mixed mode", "compiled mode" or "interpreted mode" instead, thus allowing OpenJDK builders to store additional information in the "java.vm.info" system property. Thank you and best regards, Volker From thomas.stuefe at gmail.com Tue Jul 4 09:12:12 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 4 Jul 2017 11:12:12 +0200 Subject: RFR(s): 8179327: gtestLauncher should run tests on a separate thread (optionally) In-Reply-To: <438277b7-921c-f4e3-ee12-201ad45e3eef@oracle.com> References: <438277b7-921c-f4e3-ee12-201ad45e3eef@oracle.com> Message-ID: On Tue, Jul 4, 2017 at 10:22 AM, Robbin Ehn wrote: > Looks reasonable, thanks! > > Thank you! > /Robbin > > If you don't mind, please don't wrap lines, links do not work well with > line breaks in them :) > > Oops, sorry! > On 07/02/2017 09:23 AM, Thomas St?fe wrote: > >> Dear all, >> >> please review the following change: >> >> Bug: JDK-8179327 >> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/ >> 8179327-gtestLauncher-should-have-an-option-to-avoid- >> primordial-thread-for-tests/webrev.00/webrev/ >> >> This fix is motivated by the inability of the AIX openjdk vm to run on the >> primordial thread (see https://bugs.openjdk.java.net/browse/JDK-8171505), >> which we unfortunately cannot do much about. This prevented us until now >> to >> run gtests successfully, because the gtestlauncher (in contrast to the >> standard java launcher) invokes the VM on the primordial thread. >> >> To fix this, I gave the gtest launcher the ability to run the tests in an >> own thread. Instead of sprinkling the source with "#ifdef _AIX", I added a >> command line option "-new-thread", which allows to start the tests in a >> new >> thread, and implemented this feature for all platforms (well, posix and >> windows). On all platforms save AIX this is by default off, so nothing >> changes. On AIX, tests will be default be executed in a new thread. >> >> Thanks for reviewing, >> >> Kind Regards, Thomas >> >> From glaubitz at physik.fu-berlin.de Tue Jul 4 09:33:04 2017 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Tue, 4 Jul 2017 11:33:04 +0200 Subject: [PATCH] linux-sparc build fixes In-Reply-To: References: <20170609102041.GA2477@physik.fu-berlin.de> Message-ID: <20170704093304.GD28259@physik.fu-berlin.de> On Wed, Jun 14, 2017 at 01:50:06PM +0200, Erik Helin wrote: > I think the first three patches (hotspot-add-missing-log-header.diff, > hotspot-fix-checkbytebuffer.diff, rename-sparc-linux-atomic-header.diff) all > look good, thanks for fixing broken code. Consider them Reviewed by me. > Every patch needs a corresponding issue in the bug tracker, so I went ahead > and created: > - https://bugs.openjdk.java.net/browse/JDK-8182163 > - https://bugs.openjdk.java.net/browse/JDK-8182164 > - https://bugs.openjdk.java.net/browse/JDK-8182165 Any chance a second maintainer can review those? I have run the test suite as mentioned before: > https://people.debian.org/~glaubitz/openjdk-9_9~b170-2_sparc64.new.build.gz Thanks, Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From thomas.stuefe at gmail.com Tue Jul 4 10:05:07 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 4 Jul 2017 12:05:07 +0200 Subject: RFR(XL): 8181917: Refactor UL LogStreams to avoid using resource area In-Reply-To: References: <337330d2-512c-bc77-5a55-c05d33e376e5@oracle.com> Message-ID: Hi Erik, On Mon, Jul 3, 2017 at 4:19 PM, Erik Helin wrote: > On 06/30/2017 08:32 PM, Thomas St?fe wrote: > >> New >> Version: http://cr.openjdk.java.net/~stuefe/webrevs/8181917-refactor- >> ul-logstream/all.webrev.03/webrev/ >> >> Delta to >> last: http://cr.openjdk.java.net/~stuefe/webrevs/8181917-refactor- >> ul-logstream/delta-02-to-03/webrev/ >> > > Thanks, I will take a look later. > > On 06/30/2017 08:32 PM, Thomas St?fe wrote: > >> - should you add declarations of delete as well to prevent someone >> from >> calling delete on LogStream pointer? >> >> >> Tried this, but I think I ran into linker errors. I do not think it is >> needed, though. >> > > Hmm, that seems strange, a linker error would/could indicate that some > code _is_ calling delete on a LogStream*... > > Fair enough. I retried and got a ton of unhelpful linkage errors on Windows, basically almost every file has a reference to LogStream::operator delete(). Retried on Linux, and gcc refuses to compile, always with a variant of the following: In file included from /priv/d031900/openjdk/jdk10-hs/source/hotspot/test/native/logging/logTestUtils.inline.hpp:26:0, from /priv/d031900/openjdk/jdk10-hs/source/hotspot/test/native/logging/test_logTagSetDescriptions.cpp:25: /priv/d031900/openjdk/jdk10-hs/source/hotspot/src/share/vm/logging/logStream.hpp: In destructor ?virtual LogStreamTemplate<(LogLevel::type)5u, (LogTag::type)50u, (LogTag::type)0u, (LogTag::type)0u, (LogTag::type)0 u, (LogTag::type)0u, (LogTag::type)0u>::~LogStreamTemplate()?: /priv/d031900/openjdk/jdk10-hs/source/hotspot/src/share/vm/logging/logStream.hpp:61:6: error: ?static void LogStream::operator delete(void*)? is private void operator delete(void* p); ^ In file included from /priv/d031900/openjdk/jdk10-hs/source/hotspot/test/native/logging/logTestUtils.inline.hpp:26:0, from /priv/d031900/openjdk/jdk10-hs/source/hotspot/test/native/logging/test_logTagSetDescriptions.cpp:25: /priv/d031900/openjdk/jdk10-hs/source/hotspot/src/share/vm/logging/logStream.hpp:99:7: error: within this context class LogStreamTemplate : public LogStream { ^ First I wanted to know if this is a remnant of LogStream being a ResourceObj, so I removed the inheritance between outputStream and ResourceObj, but the same errors happen (beside new ones, of course). I believe this is caused by ~outputStream() being virtual - so a virtual ~LogStreamTemplate() gets synthesized, which wants to reference LogStream::operator delete()? I have the vague notion that this has something to do with the fact that operator delete is "static but polymorphic" in the sense that for polymorphic delete calls it needs to be looked up via the virtual destructor. Something like this: " https://stackoverflow.com/questions/2273187/why-is-the-delete-operator-required-to-be-static". However, there are lots of things I do not understand here. I still do not think this is a problem, since we disallow new() sucessfully. On 06/30/2017 08:32 PM, Thomas St?fe wrote: > >> - is 128 bytes as default too much for _smallbuf? Should we start with >> 64? >> >> I changed it to 64. >> > > Ok. > > > On 06/30/2017 08:32 PM, Thomas St?fe wrote: > >> logStream.cpp >> - one extra newline at line 74 >> - the pointer returned from os::malloc is not checked. What should we >> do if a NULL pointer is returned? Print whatever is buffered and >> then >> do vm_out_of_memory? >> - is growing by doubling too aggressive? should the LineBuffer instead >> grow by chunking (e.g. always add 64 more bytes)? >> >> >> Okay. I changed the allocation of the line buffer such that >> - we do not allocate beyond a reasonable limit (1M hardwired), to >> prevent runaway leaks. >> - we cope with OOM >> In both cases, LogStream continues to work but may truncate the output- >> >> I also added a test case for the former scenario (the 1M cap). >> > > Ok, sounds good, I will have a look. > > On 06/30/2017 08:32 PM, Thomas St?fe wrote: > >> - instead of growing the LineBuffer by reallocating and then copying >> over the old bytes, should we use a chunked linked list instead (in >> order to avoid copying the same data multiple times)? The only >> "requirements" on the LineBuffer is fast append and fast iteration >> (doesn't have to support fast index lookup). >> >> >> Not really sure how this would work. The whole point of LogStream is to >> assemble one log line, in the form of a single continuous >> zero-terminated string, and pass that to the UL backend as one single >> log call, yes? How would you do this with disjunct chunks? >> > > The key point here is "as one single log call". If we allow ourselves to > pass a more elaborate data structure than a plain string (such as a linked > list of strings), then we can enable this kind of "chunked expansion". The > write call will eventually result in > > flockfile(_stream); > // do stuff > fflush(_stream); > funlockfile(_stream); > > deep down in logFileStreamOutput.cpp. Whatever we do at > `// do stuff` will become atomic wrt to other threads. We could for > example iterate over a linked list of strings and print them (with space as > separator). For an example, see LogFileStreamOutput::write(LogMessageBuffer::Iterator > msg_iterator). > > I think it can be done, but I am not sure it is worth the trouble. Right now, LogOutput is implemented by only one concrete child, writing to a FILE*. I would not swap a single IO syscall (fprintf) against a number of them to save some memory reallocations. So, we still call fprintf() only once, and we would have to assemble all the chunked buffers beforehand. Granted, that still could be a bit better than reallocating the LogStream line buffer every 64byte, but that also depends on the line buffer allocation strategy (back to doubling the memory for the line buffer?) Thinking further ahead: Assuming that we want to implement more LogOutput childs in the future, I would be careful adding an API which exposes lockinf behaviour of the underlying implementation. This means I would not expose the sequence of functions ("lock(), write(),...,write(), unlock()") in the general LogOutput() API (just as an example, lets assume you want to have a LogOutput writing to a lock-free ring buffer in shared memory or some such). But I do not think you meant that, right? You meant something like a single call, "log(struct my_list_of_chunks_t*)" ? I think that could be done, but if we find that all possible LogOutput implementations we can think up will re-assemble the chunks anyway before doing something with it, it may not be worth the trouble. The worst case for this patch is if the log line becomes larger than 512 > bytes. In this case, there will be plenty of copying as LineBuffer gets > expanded and then LogTagSet::vwrite will also allocate a buffer on the > heap, copy everything over (again), print that buffer, free it. It seems > like we can do better here :) > > Yes. I think the backend could be a bit optimized to minimize copying. > Whether this should be done as part of this patch, or as a follow-up > patch, that is another matter. > > - LogStream::write is no longer in inline.hpp, is this a potential >> performance problem? I think not, ,the old LogStream::write >> definition was most likely in .inline.hpp because of template usage >> >> >> I do not think so either. I do not like implementations in headers >> (clutters the interface), unless it is worth it performance wise. Here, >> LogStream::write is usually called from one of the >> outputStream::print()... functions via virtual function call; not sure >> how that could even be inlined. >> >> >> >> Great work thus far Thomas, the patch is becoming really solid! If >> you want to discuss over IM, then you can (as always) find me in >> #openjdk on irc.oftc.net . >> >> >> Thanks :) And thanks for reviewing! >> > > Thanks, > Erik > > > Thanks! ..Thomas > ...Thomas >> >> >> >> Thanks, >> Erik >> >> Kind Regards, Thomas >> >> >> From thomas.schatzl at oracle.com Tue Jul 4 11:03:39 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 04 Jul 2017 13:03:39 +0200 Subject: RFR: 8178500: Replace usages of round_to and round_down with align_up and align_down In-Reply-To: References: Message-ID: <1499166219.2721.2.camel@oracle.com> Hi Stefan, On Fri, 2017-06-30 at 18:12 +0200, Stefan Karlsson wrote: > Hi all, > > Please review this patch to remove round_to and round_down and > replace? > their usages with align_up and align_down. > > http://cr.openjdk.java.net/~stefank/8178500/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8178500 > > The round_to and round_down functions asserted that alignment was a? > power-of-2. I've added asserts to the corresponding align functions. > > This patch builds upon all the patches referred to in: > ? http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-June/027329 > .html ? I think the following hunk should be reverted: --- old/src/cpu/ppc/vm/templateInterpreterGenerator_ppc.cpp 2017 -06-30 18:03:19.892762780 +0200 +++ new/src/cpu/ppc/vm/templateInterpreterGenerator_ppc.cpp 2017 -06-30 18:03:19.704756404 +0200 @@ -1032,8 +1032,8 @@ ???// Cut back area between esp and max_stack. ???__ addi(parent_frame_resize, parent_frame_resize, frame::abi_minframe_size - Interpreter::stackElementSize); ? -??__ round_to(top_frame_size, frame::alignment_in_bytes); -??__ round_to(parent_frame_resize, frame::alignment_in_bytes); +??__ align_up(top_frame_size, frame::alignment_in_bytes); +??__ align_up(parent_frame_resize, frame::alignment_in_bytes); ???// parent_frame_resize = (locals-parameters) - (ESP-SP-ABI48) Rounded to frame alignment size. ???// Enlarge by locals-parameters (not in case of native_call), shrink by ESP-SP-ABI48. It does not have anything to do with the C++ round_to() methods, but some code generation function that by chance has the same name. Also I believe it will cause errors in compilation of the ppc target. :) I do not need a re-review if you just reverted that change. Thanks, ? Thomas From stefan.karlsson at oracle.com Tue Jul 4 11:21:00 2017 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 4 Jul 2017 13:21:00 +0200 Subject: RFR: 8178500: Replace usages of round_to and round_down with align_up and align_down In-Reply-To: <1499166219.2721.2.camel@oracle.com> References: <1499166219.2721.2.camel@oracle.com> Message-ID: <46069047-baa3-ebcc-9fbf-fcbaf3e9de60@oracle.com> On 2017-07-04 13:03, Thomas Schatzl wrote: > Hi Stefan, > > On Fri, 2017-06-30 at 18:12 +0200, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to remove round_to and round_down and >> replace >> their usages with align_up and align_down. >> >> http://cr.openjdk.java.net/~stefank/8178500/webrev.00/ >> https://bugs.openjdk.java.net/browse/JDK-8178500 >> >> The round_to and round_down functions asserted that alignment was a >> power-of-2. I've added asserts to the corresponding align functions. >> >> This patch builds upon all the patches referred to in: >> http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-June/027329 >> .html > > I think the following hunk should be reverted: > > --- old/src/cpu/ppc/vm/templateInterpreterGenerator_ppc.cpp 2017 > -06-30 18:03:19.892762780 +0200 > +++ new/src/cpu/ppc/vm/templateInterpreterGenerator_ppc.cpp 2017 > -06-30 18:03:19.704756404 +0200 > @@ -1032,8 +1032,8 @@ > // Cut back area between esp and max_stack. > __ addi(parent_frame_resize, parent_frame_resize, > frame::abi_minframe_size - Interpreter::stackElementSize); > > - __ round_to(top_frame_size, frame::alignment_in_bytes); > - __ round_to(parent_frame_resize, frame::alignment_in_bytes); > + __ align_up(top_frame_size, frame::alignment_in_bytes); > + __ align_up(parent_frame_resize, frame::alignment_in_bytes); > // parent_frame_resize = (locals-parameters) - (ESP-SP-ABI48) > Rounded to frame alignment size. > // Enlarge by locals-parameters (not in case of native_call), shrink > by ESP-SP-ABI48. > > It does not have anything to do with the C++ round_to() methods, but > some code generation function that by chance has the same name. Also I > believe it will cause errors in compilation of the ppc target. :) You are right. Thanks for catching this. > > I do not need a re-review if you just reverted that change. Thanks! StefanK > > Thanks, > Thomas > From stefan.karlsson at oracle.com Tue Jul 4 11:43:50 2017 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 4 Jul 2017 13:43:50 +0200 Subject: RFR: 8178500: Replace usages of round_to and round_down with align_up and align_down In-Reply-To: <68ebaf00-bf89-f9c7-5de9-d869f334778e@oracle.com> References: <68ebaf00-bf89-f9c7-5de9-d869f334778e@oracle.com> Message-ID: Thanks, Robbin. StefanK On 2017-07-04 10:40, Robbin Ehn wrote: > Hi Stefan, > > Looks ok, thanks for fixing. > > /Robbin > > On 06/30/2017 06:12 PM, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to remove round_to and round_down and replace >> their usages with align_up and align_down. >> >> http://cr.openjdk.java.net/~stefank/8178500/webrev.00/ >> https://bugs.openjdk.java.net/browse/JDK-8178500 >> >> The round_to and round_down functions asserted that alignment was a >> power-of-2. I've added asserts to the corresponding align functions. >> >> This patch builds upon all the patches referred to in: >> >> http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-June/027329.html >> >> Thanks, >> StefanK From stefan.karlsson at oracle.com Tue Jul 4 11:44:04 2017 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 4 Jul 2017 13:44:04 +0200 Subject: RFR: 8178499: Remove _ptr_ and _size_ infixes from align functions In-Reply-To: References: Message-ID: <7a147d22-4d99-60a9-beaf-a555dbe9f127@oracle.com> Thanks, Robbin. StefanK On 2017-07-04 10:44, Robbin Ehn wrote: > Looks good. > > /Robbin > > On 06/30/2017 11:40 AM, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to remove the _ptr_ and _size_ infixes from >> align functions. >> >> http://cr.openjdk.java.net/~stefank/8178499/webrev.00 >> http://bugs.openjdk.java.net/browse/JDK-8178499 >> >> Rename functions (and corresponding macros) from: >> align_ptr_up >> align_size_up >> align_ptr_down >> align_size_down >> is_ptr_aligned >> is_size_aligned >> >> to: >> align_up >> align_down >> is_aligned >> >> The patch builds upon the changes in: >> >> http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-June/027328.html >> >> Thanks, >> StefanK From stefan.karlsson at oracle.com Tue Jul 4 11:44:23 2017 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 4 Jul 2017 13:44:23 +0200 Subject: RFR: 8178489: Make align functions more type safe and consistent In-Reply-To: <6509977d-16d0-4693-78d7-29372a275de4@oracle.com> References: <23768f79-d356-a7fc-833c-4932f18fc0d1@oracle.com> <6509977d-16d0-4693-78d7-29372a275de4@oracle.com> Message-ID: <015f7858-9654-ee82-de28-5842298f9001@oracle.com> Thanks, Robbin. StefanK On 2017-07-04 10:44, Robbin Ehn wrote: > Looks good. > > Thanks, Robbin > > On 06/30/2017 11:16 AM, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to make the align functions more type safe >> and consistent. >> >> http://cr.openjdk.java.net/~stefank/8178489/webrev.00 >> https://bugs.openjdk.java.net/browse/JDK-8178489 >> >> Note that this patch needs to be applied on top of the following >> patches that are out for review: >> http://cr.openjdk.java.net/~stefank/8178491/webrev.02 >> http://cr.openjdk.java.net/~stefank/8178495/webrev.01 >> >> Currently, the align functions forces the user to often explicitly >> cast either the input parameters, or the return type, or both. >> >> Two examples of the current API: >> inline intptr_t align_size_up(intptr_t size, intptr_t alignment); >> inline void* align_ptr_up(const void* ptr, size_t alignment); >> >> I propose that we change the API to use templates to return the >> aligned value as the same type as the type of the unaligned input. >> >> The proposed API would look like this: >> >> template >> inline T align_size_up(T size, A alignment); >> >> template >> inline T* align_ptr_up(T* ptr, A alignment); >> >> and a follow-up RFE (JDK-8178499) would get rid of _size_ and _ptr_ >> from the names. >> >> Usages of these align functions would then look like: >> >> size_t aligned_size = align_up(alloc_size, os::vm_page_size()) >> HeapWord* aligned_top = align_up(top, region_size) >> >> Please, take an extra close look at the reinterpret_cast I added in >> atomic.hpp. This was needed because the align_ptr_down now returns T* >> and not void*, and the compiler complained when we tried to do a >> static cast from a volatile jbyte* to a volatile jint*. >> >> Tested with the align unit test and JPRT. >> >> Thanks, >> StefanK From mikael.gerdin at oracle.com Tue Jul 4 11:55:54 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Tue, 4 Jul 2017 13:55:54 +0200 Subject: RFR: 8178489: Make align functions more type safe and consistent In-Reply-To: <23768f79-d356-a7fc-833c-4932f18fc0d1@oracle.com> References: <23768f79-d356-a7fc-833c-4932f18fc0d1@oracle.com> Message-ID: <450b2d5b-1878-c2a7-6299-e3723c57e996@oracle.com> Hi Stefan, On 2017-06-30 11:16, Stefan Karlsson wrote: > Hi all, > > Please review this patch to make the align functions more type safe and > consistent. > > http://cr.openjdk.java.net/~stefank/8178489/webrev.00 I looked through this and I think it looks good but I think you may need to add some extra brackets around the use of the parameters in is_size_aligned_ change #define is_size_aligned_(size, alignment) ((size) == (align_size_up_(size, alignment))) to #define is_size_aligned_(size, alignment) ((size) == (align_size_up_((size), (alignment)))) Thanks /Mikael > https://bugs.openjdk.java.net/browse/JDK-8178489 > > Note that this patch needs to be applied on top of the following patches > that are out for review: > http://cr.openjdk.java.net/~stefank/8178491/webrev.02 > http://cr.openjdk.java.net/~stefank/8178495/webrev.01 > > Currently, the align functions forces the user to often explicitly cast > either the input parameters, or the return type, or both. > > Two examples of the current API: > inline intptr_t align_size_up(intptr_t size, intptr_t alignment); > inline void* align_ptr_up(const void* ptr, size_t alignment); > > I propose that we change the API to use templates to return the aligned > value as the same type as the type of the unaligned input. > > The proposed API would look like this: > > template > inline T align_size_up(T size, A alignment); > > template > inline T* align_ptr_up(T* ptr, A alignment); > > and a follow-up RFE (JDK-8178499) would get rid of _size_ and _ptr_ from > the names. > > Usages of these align functions would then look like: > > size_t aligned_size = align_up(alloc_size, os::vm_page_size()) > HeapWord* aligned_top = align_up(top, region_size) > > Please, take an extra close look at the reinterpret_cast I added in > atomic.hpp. This was needed because the align_ptr_down now returns T* > and not void*, and the compiler complained when we tried to do a static > cast from a volatile jbyte* to a volatile jint*. > > Tested with the align unit test and JPRT. > > Thanks, > StefanK From thomas.schatzl at oracle.com Tue Jul 4 11:56:20 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 04 Jul 2017 13:56:20 +0200 Subject: RFR: 8178499: Remove _ptr_ and _size_ infixes from align functions In-Reply-To: References: Message-ID: <1499169380.2721.8.camel@oracle.com> Hi Stefan, On Fri, 2017-06-30 at 11:40 +0200, Stefan Karlsson wrote: > Hi all, > > Please review this patch to remove the _ptr_ and _size_ infixes from? > align functions. > > http://cr.openjdk.java.net/~stefank/8178499/webrev.00 > http://bugs.openjdk.java.net/browse/JDK-8178499 > > Rename functions (and corresponding macros) from: > ? align_ptr_up > ? align_size_up > ? align_ptr_down > ? align_size_down > ? is_ptr_aligned > ? is_size_aligned > > to: > ? align_up > ? align_down > ? is_aligned > > The patch builds upon the changes in: > ? http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-June/027328 > .html > ? looks good. I am not sure if I am happy about the renamings of the non-global align* methods, as they at least sometimes seemed to have been part of the item you wanted to align. E.g. ReservedSpace::page_align_size_* to ReservedSpace::page_align_*. There is at least one name clash because of that is resolved using the "::" specifiers (in space.cpp:~670). Or maybe this can be resolved by different naming in ReservedSpace. Unless somebody else also objects, consider this as not a blocking issue. Thanks, ? Thomas From thomas.schatzl at oracle.com Tue Jul 4 13:11:14 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 04 Jul 2017 15:11:14 +0200 Subject: RFR: 8178499: Remove _ptr_ and _size_ infixes from align functions In-Reply-To: <1499169380.2721.8.camel@oracle.com> References: <1499169380.2721.8.camel@oracle.com> Message-ID: <1499173874.3200.4.camel@oracle.com> Hi, On Tue, 2017-07-04 at 13:56 +0200, Thomas Schatzl wrote: > Hi Stefan, > > On Fri, 2017-06-30 at 11:40 +0200, Stefan Karlsson wrote: > > > > Hi all, > > > > Please review this patch to remove the _ptr_ and _size_ infixes > > from? > > align functions. > > > > http://cr.openjdk.java.net/~stefank/8178499/webrev.00 > > http://bugs.openjdk.java.net/browse/JDK-8178499 > > > > Rename functions (and corresponding macros) from: > > ? align_ptr_up > > ? align_size_up > > ? align_ptr_down > > ? align_size_down > > ? is_ptr_aligned > > ? is_size_aligned > > > > to: > > ? align_up > > ? align_down > > ? is_aligned > > > > The patch builds upon the changes in: > > ? http://mail.openjdk.java.net/pipermail/hotspot-dev/2017- > > June/027328.html > > > ? looks good. I am not sure if I am happy about the renamings of the > non-global align* methods, as they at least sometimes seemed to have > been part of the item you wanted to align. E.g. > ReservedSpace::page_align_size_* to ReservedSpace::page_align_*. > > There is at least one name clash because of that is resolved using > the "::" specifiers (in space.cpp:~670). Or maybe this can be > resolved by different naming in ReservedSpace. > ? Stefan made me aware that the last statement is untrue, the name clash has nothing to do with the renaming of ReservedSpace methods. The new names for the changed ReservedSpace just feel worse than before, but maybe this is not an issue for others. Feel free to ignore this comment. Sorry for the trouble with that, ? Thomas From volker.simonis at gmail.com Tue Jul 4 13:19:57 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 4 Jul 2017 15:19:57 +0200 Subject: RFR(M) JDK-8181809 PPC64: Leverage mtfprd/mffprd on POWER8 In-Reply-To: <232e0b79-9022-bf4b-4c40-88880c68d22e@linux.vnet.ibm.com> References: <8b53b855-f540-bb89-dcbb-bf0f5bb35b17@linux.vnet.ibm.com> <2a4fcb315f4d44199e8cc66935886f41@sap.com> <651ebdd4-3854-ac42-8e9c-54df77cbb5fc@linux.vnet.ibm.com> <56073cca-0c7a-0436-4e95-6d74a1bbe404@linux.vnet.ibm.com> <20c43bf4-a66b-2cc8-e62f-d58eb66df278@linux.vnet.ibm.com> <0f82ddbcf57348e8ac6e6cd9e51674f3@sap.com> <95d5cb36271e4ebf8398223702b61ac8@sap.com> <232e0b79-9022-bf4b-4c40-88880c68d22e@linux.vnet.ibm.com> Message-ID: Hi Matthew, Martin, thanks a lot for doing/reviewing this change! It looks good. Thumbs up from my side. Regards, Volker On Thu, Jun 29, 2017 at 3:37 PM, Matthew Brandyberry wrote: > Thanks Martin. That looks good. Is there anything I need to do to request > a 2nd review? > > > On 6/26/17 7:47 AM, Doerr, Martin wrote: >> >> Hi Matt, >> >> after some testing and reviewing the C1 part again, I found 2 bugs: >> >> c1_LIRAssembler: is_stack() can't be used for this purpose as the value >> may be available in a register even though it was forced to stack. I just >> changed src_in_memory = !VM_Version::has_mtfprd() to make it consistent with >> LIRGenerator and removed the assertions which have become redundant. >> >> c1_LIRGenerator: value.set_destroys_register() is still needed for >> conversion from FP to GP registers because they kill the src value by >> fctiwz/fctidz. >> >> I just fixed these issues here in a copy of your webrev v2: >> http://cr.openjdk.java.net/~mdoerr/8181809_ppc64_mtfprd/v2/ >> >> Please take a look and use this one for 2nd review. >> >> Best regards, >> Martin >> >> >> -----Original Message----- >> From: Doerr, Martin >> Sent: Montag, 26. Juni 2017 10:44 >> To: 'Matthew Brandyberry' ; >> ppc-aix-port-dev at openjdk.java.net; hotspot-dev at openjdk.java.net >> Subject: RE: RFR(M) JDK-8181809 PPC64: Leverage mtfprd/mffprd on POWER8 >> >> Hi Matt, >> >> you can run the pre-push check stand-alone: hg jcheck >> See: >> http://openjdk.java.net/projects/code-tools/jcheck/ >> >> I just had to add the commit message: >> 8181809: PPC64: Leverage mtfprd/mffprd on POWER8 >> Reviewed-by: mdoerr >> Contributed-by: Matthew Brandyberry >> >> (Note that the ':' after the bug id is important.) >> >> and replace the Tabs the 2 C1 files to get it passing. >> (I think that "Illegal tag name" warnings can be ignored.) >> >> So only the copyright dates are missing which are not checked by jcheck. >> But I don't need a new webrev if that's all which needs to be changed. >> >> Best regards, >> Martin >> >> >> -----Original Message----- >> From: Matthew Brandyberry [mailto:mbrandy at linux.vnet.ibm.com] >> Sent: Freitag, 23. Juni 2017 18:39 >> To: Doerr, Martin ; >> ppc-aix-port-dev at openjdk.java.net; hotspot-dev at openjdk.java.net >> Subject: Re: RFR(M) JDK-8181809 PPC64: Leverage mtfprd/mffprd on POWER8 >> >> Thanks Martin. Are there tools to help detect formatting errors like the >> tab characters? >> >> I'll keep an eye on this to see if I need to do anything else. >> >> -Matt >> >> On 6/23/17 4:30 AM, Doerr, Martin wrote: >>> >>> Excellent. Thanks for the update. The C1 part looks good, too. >>> >>> Also, thanks for checking "I could not find evidence of any config that >>> includes vpmsumb but not >>> mtfprd." >>> >>> There are only a few formally required things: >>> - The new C1 code contains Tab characters. It's not possible to push it >>> without fixing this. >>> - Copyright messages should be updated. >>> - Minor resolution to get vm_version_ppc applied to recent jdk10/hs. >>> >>> If no other changes get requested, I can handle these issues this time >>> before pushing. >>> But we need another review, first. >>> >>> Thanks and best regards, >>> Martin >>> >>> >>> -----Original Message----- >>> From: Matthew Brandyberry [mailto:mbrandy at linux.vnet.ibm.com] >>> Sent: Freitag, 23. Juni 2017 04:54 >>> To: Doerr, Martin ; >>> ppc-aix-port-dev at openjdk.java.net; hotspot-dev at openjdk.java.net >>> Subject: Re: RFR(M) JDK-8181809 PPC64: Leverage mtfprd/mffprd on POWER8 >>> >>> Updated webrev: http://cr.openjdk.java.net/~gromero/8181809/v2/ >>> >>> See below for responses inline. >>> >>> On 6/20/17 8:38 AM, Matthew Brandyberry wrote: >>>> >>>> Hi Martin, >>>> >>>> Thanks for the review. I'll take a look at these areas and report >>>> back -- especially the integration into C1. >>>> >>>> On 6/20/17 8:33 AM, Doerr, Martin wrote: >>>>> >>>>> Hi Matt, >>>>> >>>>> thanks for providing this webrev. I had already thought about using >>>>> these instructions for this purpose and your change matches pretty >>>>> much what I'd do. >>>>> >>>>> Here a couple of comments: >>>>> ppc.ad: >>>>> This was a lot of work. Thanks for doing it. >>>>> effect(DEF dst, USE src); is redundant if a match rule match(Set dst >>>>> (MoveL2D src)); exists. >>> >>> Fixed. >>>>> >>>>> vm_version: >>>>> This part is in conflict with Michihiro's change which is already >>>>> pushed in jdk10, but it's trivial to resolve. I'm ok with using >>>>> has_vpmsumb() for has_mtfprd(). In the past, we sometimes had trouble >>>>> with assuming that a certain Power processor supports all new >>>>> instructions if it supports certain ones. We also use the hotspot >>>>> code on as400 where certain instruction subsets were disabled while >>>>> other Power 8 instructions were usable. Maybe you can double-check if >>>>> there may exist configurations in which has_vpmsumb() doesn't match >>>>> has_mtfprd(). >>> >>> I could not find evidence of any config that includes vpmsumb but not >>> mtfprd. >>>>> >>>>> C1: >>>>> It should also be possible to use the instructions in C1 compiler. >>>>> Maybe you would like to take a look at it as well and see if it can >>>>> be done with feasible effort. >>>>> Here are some hints: >>>>> The basic decisions are made in LIRGenerator::do_Convert. You could >>>>> skip the force_to_spill or must_start_in_memory steps. >>>>> The final assembly code gets emitted in LIR_Assembler::emit_opConvert >>>>> where you could replace the store instructions. >>>>> For testing, you can use -XX:TieredStopAtLevel=1, for example. >>> >>> Done. Please take a look. >>>>> >>>>> Thanks and best regards, >>>>> Martin >>>>> >>>>> >>>>> -----Original Message----- >>>>> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On >>>>> Behalf Of Matthew Brandyberry >>>>> Sent: Montag, 19. Juni 2017 18:28 >>>>> To: ppc-aix-port-dev at openjdk.java.net; hotspot-dev at openjdk.java.net >>>>> Subject: RFR(M) JDK-8181809 PPC64: Leverage mtfprd/mffprd on POWER8 >>>>> >>>>> This is a PPC-specific hotspot optimization that leverages the >>>>> mtfprd/mffprd instructions for for movement between general purpose and >>>>> floating point registers (rather than through memory). It yields a >>>>> ~35% >>>>> improvement measured via a microbenchmark. Please review: Bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8181809 Webrev: >>>>> http://cr.openjdk.java.net/~gromero/8181809/v1/ Thanks, Matt >>>>> >>>>> > From stefan.karlsson at oracle.com Tue Jul 4 13:33:46 2017 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 4 Jul 2017 15:33:46 +0200 Subject: RFR: 8178489: Make align functions more type safe and consistent In-Reply-To: <450b2d5b-1878-c2a7-6299-e3723c57e996@oracle.com> References: <23768f79-d356-a7fc-833c-4932f18fc0d1@oracle.com> <450b2d5b-1878-c2a7-6299-e3723c57e996@oracle.com> Message-ID: <9557d5ff-b40a-d9e7-008e-af252b0444c5@oracle.com> Hi Mikael, Thanks for the review. Inlined: On 2017-07-04 13:55, Mikael Gerdin wrote: > Hi Stefan, > > On 2017-06-30 11:16, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to make the align functions more type safe >> and consistent. >> >> http://cr.openjdk.java.net/~stefank/8178489/webrev.00 > > I looked through this and I think it looks good but I think you may need > to add some extra brackets around the use of the parameters in > is_size_aligned_ > > change > #define is_size_aligned_(size, alignment) ((size) == > (align_size_up_(size, alignment))) > > to > #define is_size_aligned_(size, alignment) ((size) == > (align_size_up_((size), (alignment)))) This is a pre-existing issue with these macros, I just moved it in this file. I talked to Mikael offline and we agreed on creating a new Bug for this, and push after the current set of patches have been pushed. This way I won't have to rebase the follow-up patches. StefanK > > Thanks > /Mikael > > >> https://bugs.openjdk.java.net/browse/JDK-8178489 >> >> Note that this patch needs to be applied on top of the following >> patches that are out for review: >> http://cr.openjdk.java.net/~stefank/8178491/webrev.02 >> http://cr.openjdk.java.net/~stefank/8178495/webrev.01 >> >> Currently, the align functions forces the user to often explicitly >> cast either the input parameters, or the return type, or both. >> >> Two examples of the current API: >> inline intptr_t align_size_up(intptr_t size, intptr_t alignment); >> inline void* align_ptr_up(const void* ptr, size_t alignment); >> >> I propose that we change the API to use templates to return the >> aligned value as the same type as the type of the unaligned input. >> >> The proposed API would look like this: >> >> template >> inline T align_size_up(T size, A alignment); >> >> template >> inline T* align_ptr_up(T* ptr, A alignment); >> >> and a follow-up RFE (JDK-8178499) would get rid of _size_ and _ptr_ >> from the names. >> >> Usages of these align functions would then look like: >> >> size_t aligned_size = align_up(alloc_size, os::vm_page_size()) >> HeapWord* aligned_top = align_up(top, region_size) >> >> Please, take an extra close look at the reinterpret_cast I added in >> atomic.hpp. This was needed because the align_ptr_down now returns T* >> and not void*, and the compiler complained when we tried to do a >> static cast from a volatile jbyte* to a volatile jint*. >> >> Tested with the align unit test and JPRT. >> >> Thanks, >> StefanK From stefan.karlsson at oracle.com Tue Jul 4 14:24:18 2017 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 4 Jul 2017 16:24:18 +0200 Subject: RFR: 8183544: Missing parentheses in is_size_aligned_ macro Message-ID: <7b97c707-1baa-3282-4759-cc6226d05921@oracle.com> Hi all, Please review this trivial patch to add parentheses to the is_aligned_ macro (used to be named is_size_aligned_). http://cr.openjdk.java.net/~stefank/8183544/webrev.00/ MikaelG found this issue while reviewing another patch. Thanks, StefanK From mikael.gerdin at oracle.com Tue Jul 4 14:28:40 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Tue, 4 Jul 2017 16:28:40 +0200 Subject: RFR: 8183544: Missing parentheses in is_size_aligned_ macro In-Reply-To: <7b97c707-1baa-3282-4759-cc6226d05921@oracle.com> References: <7b97c707-1baa-3282-4759-cc6226d05921@oracle.com> Message-ID: <703684db-e4d4-e0a5-055f-4717da5b573f@oracle.com> Hi Stefan, On 2017-07-04 16:24, Stefan Karlsson wrote: > Hi all, > > Please review this trivial patch to add parentheses to the is_aligned_ > macro (used to be named is_size_aligned_). > > http://cr.openjdk.java.net/~stefank/8183544/webrev.00/ Looks great! :) /m > > MikaelG found this issue while reviewing another patch. > > Thanks, > StefanK From stefan.karlsson at oracle.com Tue Jul 4 14:34:26 2017 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 4 Jul 2017 16:34:26 +0200 Subject: 8178501: Replace usages of align macros with calls to the align inline functions Message-ID: <76431861-a2fe-2309-de4a-85abf302c3f2@oracle.com> Hi all, Please review this patch to replace usages of align macros with calls to the align inline functions. http://cr.openjdk.java.net/~stefank/8178501/webrev.00/ https://bugs.openjdk.java.net/browse/JDK-8178501 Thanks, StefanK From mikael.gerdin at oracle.com Tue Jul 4 14:38:07 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Tue, 4 Jul 2017 16:38:07 +0200 Subject: 8178501: Replace usages of align macros with calls to the align inline functions In-Reply-To: <76431861-a2fe-2309-de4a-85abf302c3f2@oracle.com> References: <76431861-a2fe-2309-de4a-85abf302c3f2@oracle.com> Message-ID: <1ee4f83a-0a4e-c15d-f880-1d9ac7c66a78@oracle.com> Hi Stefan, On 2017-07-04 16:34, Stefan Karlsson wrote: > Hi all, > > Please review this patch to replace usages of align macros with calls to > the align inline functions. > > http://cr.openjdk.java.net/~stefank/8178501/webrev.00/ Looks fine. /Mikael > https://bugs.openjdk.java.net/browse/JDK-8178501 > > Thanks, > StefanK From stefan.karlsson at oracle.com Tue Jul 4 14:38:30 2017 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 4 Jul 2017 16:38:30 +0200 Subject: 8178501: Replace usages of align macros with calls to the align inline functions In-Reply-To: <1ee4f83a-0a4e-c15d-f880-1d9ac7c66a78@oracle.com> References: <76431861-a2fe-2309-de4a-85abf302c3f2@oracle.com> <1ee4f83a-0a4e-c15d-f880-1d9ac7c66a78@oracle.com> Message-ID: <96a617a2-77f5-b69e-bcee-70bf7d0386d7@oracle.com> Thanks, Mikael. StefanK On 2017-07-04 16:38, Mikael Gerdin wrote: > Hi Stefan, > > On 2017-07-04 16:34, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to replace usages of align macros with calls >> to the align inline functions. >> >> http://cr.openjdk.java.net/~stefank/8178501/webrev.00/ > > Looks fine. > /Mikael > >> https://bugs.openjdk.java.net/browse/JDK-8178501 >> >> Thanks, >> StefanK From stefan.karlsson at oracle.com Tue Jul 4 14:38:48 2017 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 4 Jul 2017 16:38:48 +0200 Subject: RFR: 8183544: Missing parentheses in is_size_aligned_ macro In-Reply-To: <703684db-e4d4-e0a5-055f-4717da5b573f@oracle.com> References: <7b97c707-1baa-3282-4759-cc6226d05921@oracle.com> <703684db-e4d4-e0a5-055f-4717da5b573f@oracle.com> Message-ID: <84aaf10d-6d93-02d9-e58b-3d2ab3f3d8f5@oracle.com> On 2017-07-04 16:28, Mikael Gerdin wrote: > Hi Stefan, > > On 2017-07-04 16:24, Stefan Karlsson wrote: >> Hi all, >> >> Please review this trivial patch to add parentheses to the is_aligned_ >> macro (used to be named is_size_aligned_). >> >> http://cr.openjdk.java.net/~stefank/8183544/webrev.00/ > > Looks great! :) Thanks! ;) StefanK > /m > >> >> MikaelG found this issue while reviewing another patch. >> >> Thanks, >> StefanK From thomas.schatzl at oracle.com Tue Jul 4 14:43:46 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 04 Jul 2017 16:43:46 +0200 Subject: RFR: 8183544: Missing parentheses in is_size_aligned_ macro In-Reply-To: <7b97c707-1baa-3282-4759-cc6226d05921@oracle.com> References: <7b97c707-1baa-3282-4759-cc6226d05921@oracle.com> Message-ID: <1499179426.2757.2.camel@oracle.com> Hi Stefan, On Tue, 2017-07-04 at 16:24 +0200, Stefan Karlsson wrote: > Hi all, > > Please review this trivial patch to add parentheses to the > is_aligned_? > macro (used to be named is_size_aligned_). > > http://cr.openjdk.java.net/~stefank/8183544/webrev.00/ > > MikaelG found this issue while reviewing another patch. ? looks good. Thomas From thomas.schatzl at oracle.com Tue Jul 4 14:48:50 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 04 Jul 2017 16:48:50 +0200 Subject: 8178501: Replace usages of align macros with calls to the align inline functions In-Reply-To: <76431861-a2fe-2309-de4a-85abf302c3f2@oracle.com> References: <76431861-a2fe-2309-de4a-85abf302c3f2@oracle.com> Message-ID: <1499179730.2757.3.camel@oracle.com> Hi, On Tue, 2017-07-04 at 16:34 +0200, Stefan Karlsson wrote: > Hi all, > > Please review this patch to replace usages of align macros with calls > to? > the align inline functions. > > http://cr.openjdk.java.net/~stefank/8178501/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8178501 ? ship it. :) Thomas From stefan.karlsson at oracle.com Tue Jul 4 14:51:12 2017 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 4 Jul 2017 16:51:12 +0200 Subject: 8178501: Replace usages of align macros with calls to the align inline functions In-Reply-To: <1499179730.2757.3.camel@oracle.com> References: <76431861-a2fe-2309-de4a-85abf302c3f2@oracle.com> <1499179730.2757.3.camel@oracle.com> Message-ID: <2ebf6636-2675-3606-9763-e901486f5093@oracle.com> Thanks, Thomas! StefanK On 2017-07-04 16:48, Thomas Schatzl wrote: > Hi, > > On Tue, 2017-07-04 at 16:34 +0200, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to replace usages of align macros with calls >> to >> the align inline functions. >> >> http://cr.openjdk.java.net/~stefank/8178501/webrev.00/ >> https://bugs.openjdk.java.net/browse/JDK-8178501 > > ship it. :) > > Thomas > From stefan.karlsson at oracle.com Tue Jul 4 14:51:23 2017 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 4 Jul 2017 16:51:23 +0200 Subject: RFR: 8183544: Missing parentheses in is_size_aligned_ macro In-Reply-To: <1499179426.2757.2.camel@oracle.com> References: <7b97c707-1baa-3282-4759-cc6226d05921@oracle.com> <1499179426.2757.2.camel@oracle.com> Message-ID: <65180919-6a81-dd20-c395-7dc4a827f839@oracle.com> Thanks, Thomas. StefanK On 2017-07-04 16:43, Thomas Schatzl wrote: > Hi Stefan, > > On Tue, 2017-07-04 at 16:24 +0200, Stefan Karlsson wrote: >> Hi all, >> >> Please review this trivial patch to add parentheses to the >> is_aligned_ >> macro (used to be named is_size_aligned_). >> >> http://cr.openjdk.java.net/~stefank/8183544/webrev.00/ >> >> MikaelG found this issue while reviewing another patch. > > looks good. > > Thomas > From stefan.karlsson at oracle.com Tue Jul 4 22:25:03 2017 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 5 Jul 2017 00:25:03 +0200 Subject: RFR: 8183552: Move align functions to align.hpp Message-ID: <935675bc-b011-6606-73f8-6a8454a68d11@oracle.com> Hi all, Please review this patch to move the align functions to align.hpp. http://cr.openjdk.java.net/~stefank/8183552/webrev.00 https://bugs.openjdk.java.net/browse/JDK-8183552 The patch mainly moves the functions and adding #include "utilities/align.hpp" lines, but it also moves ScaleForWordSize to globals.hpp to prevent a cyclic dependency between align.hpp and globalDefinitions.hpp. Thanks, StefanK From kim.barrett at oracle.com Wed Jul 5 01:02:55 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 4 Jul 2017 21:02:55 -0400 Subject: RFR: 8183544: Missing parentheses in is_size_aligned_ macro In-Reply-To: <7b97c707-1baa-3282-4759-cc6226d05921@oracle.com> References: <7b97c707-1baa-3282-4759-cc6226d05921@oracle.com> Message-ID: <6259AC65-4308-4DBE-AE7A-BC6A3A974530@oracle.com> > On Jul 4, 2017, at 10:24 AM, Stefan Karlsson wrote: > > Hi all, > > Please review this trivial patch to add parentheses to the is_aligned_ macro (used to be named is_size_aligned_). > > http://cr.openjdk.java.net/~stefank/8183544/webrev.00/ > > MikaelG found this issue while reviewing another patch. > > Thanks, > StefanK I'm confused by the existing definition. Why isn't it #define is_aligned_(size, alignment) (((size) & align_mask(alignment)) == 0) That seems like it would be simpler, faster, and avoid multiple evaluation of the size expression. From thomas.stuefe at gmail.com Wed Jul 5 06:10:45 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 5 Jul 2017 08:10:45 +0200 Subject: RFR: 8183552: Move align functions to align.hpp In-Reply-To: <935675bc-b011-6606-73f8-6a8454a68d11@oracle.com> References: <935675bc-b011-6606-73f8-6a8454a68d11@oracle.com> Message-ID: Hi Stefan, this looks good. This change (and all the preceding ones) are nice cleanups! Kind Regards, Thomas On Wed, Jul 5, 2017 at 12:25 AM, Stefan Karlsson wrote: > Hi all, > > Please review this patch to move the align functions to align.hpp. > > http://cr.openjdk.java.net/~stefank/8183552/webrev.00 > https://bugs.openjdk.java.net/browse/JDK-8183552 > > The patch mainly moves the functions and adding #include > "utilities/align.hpp" lines, but it also moves ScaleForWordSize to > globals.hpp to prevent a cyclic dependency between align.hpp and > globalDefinitions.hpp. > > Thanks, > StefanK > From stefan.karlsson at oracle.com Wed Jul 5 06:35:22 2017 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 5 Jul 2017 08:35:22 +0200 Subject: RFR: 8183552: Move align functions to align.hpp In-Reply-To: References: <935675bc-b011-6606-73f8-6a8454a68d11@oracle.com> Message-ID: <25616a65-3f29-c408-1c70-8a4cab3d581f@oracle.com> Hi Thomas, On 2017-07-05 08:10, Thomas St?fe wrote: > Hi Stefan, > > this looks good. This change (and all the preceding ones) are nice > cleanups! Thanks for the review! StefanK > > Kind Regards, Thomas > > > > On Wed, Jul 5, 2017 at 12:25 AM, Stefan Karlsson > > wrote: > > Hi all, > > Please review this patch to move the align functions to align.hpp. > > http://cr.openjdk.java.net/~stefank/8183552/webrev.00 > > https://bugs.openjdk.java.net/browse/JDK-8183552 > > > The patch mainly moves the functions and adding #include > "utilities/align.hpp" lines, but it also moves ScaleForWordSize to > globals.hpp to prevent a cyclic dependency between align.hpp and > globalDefinitions.hpp. > > Thanks, > StefanK > > From thomas.stuefe at gmail.com Wed Jul 5 09:21:49 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 5 Jul 2017 11:21:49 +0200 Subject: RFR(M): 8183227: read/write APIs in class os shall return ssize_t In-Reply-To: References: Message-ID: Hi Christoph, good change so far. src/jdk.hotspot.agent/linux/native/libsaproc/ps_core.c: ok src/jdk.hotspot.agent/macosx/native/libsaproc/ps_core.c: ok src/os/aix/vm/os_aix.cpp: ok src/os/aix/vm/os_aix.inline.hpp: ok src/os/bsd/vm/os_bsd.cpp: ok src/os/bsd/vm/os_bsd.inline.hpp: ok src/os/linux/vm/os_linux.cpp: ok src/os/linux/vm/os_linux.inline.hpp: ok src/os/solaris/vm/os_solaris.cpp: ok src/os/windows/vm/os_windows.cpp: ok src/os/windows/vm/os_windows.inline.hpp: ok src/share/vm/classfile/classLoader.cpp: ok src/share/vm/classfile/sharedPathsMiscInfo.hpp: ok src/share/vm/compiler/compileLog.cpp: - size_t nr; // number read into buf from partial log + ssize_t nr; // number read into buf from partial log // Copy data up to the end of the last element: julong to_read = log->_file_end; while (to_read > 0) { if (to_read < (julong)buflen) - nr = (size_t)to_read; + nr = (ssize_t)to_read; else nr = buflen; nr = read(partial_fd, buf, (int)nr); if (nr <= 0) break; to_read -= (julong)nr; - file->write(buf, nr); + file->write(buf, (size_t)nr); } nr is used both as input and output variable. Could you disentangle this by providing two variables, that would make the cast unnecessary. Even cleaner would be to make the len/positon arguments for read/pread a size_t. src/share/vm/compiler/directivesParser.cpp: ok src/share/vm/memory/filemap.cpp: ok src/share/vm/runtime/os.hpp: ok -------------- General remarks (up to you if you fix them in this issue or not): - Please leave whitespace only changes out, makes it difficult to read the change - The length and, for pread(3), position arguments are size_t. To avoid casts, it would make sense to make the corresponding arguments for os::read/os::read_at etc size_t too. - os::read_at() and os::restartable_read() are not used anywhere and can be deleted. - A lot of those IO functions (e.g. os::write()) live in xx.inline.hpp header files. That is unncessary, they should live in xxx.cpp files. Kind Regards, Thomas On Mon, Jul 3, 2017 at 9:36 AM, Langer, Christoph wrote: > Ping: Any comments on this change...? > > From: Langer, Christoph > Sent: Donnerstag, 29. Juni 2017 12:50 > To: hotspot-runtime-dev at openjdk.java.net > Subject: RFR(M): 8183227: read/write APIs in class os shall return ssize_t > > Hi folks, > > please review the following change: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8183227 > Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8183227.0/ > > When looking at the read() and write() APIs and its siblings on the > platforms, you find the return value mostly being of signed type ssize_t > vs. the current unsigned return type size_t of the methods in class os. > With my proposed patch I try to address that and change the signatures of > these methods to ssize_t. > > At the places where the methods were called so far, I could find 2 places > where usage of the wrong type could be critical: > src/share/vm/compiler/compileLog.cpp (where actually ::read is used) and > src/share/vm/compiler/directivesParser.cpp where a wrong array access > could happen > > As for the class os, I'm wondering if the methods "read_at" and > "restartable_read" are still required? My source code scan tells me that > there are no users of them. Shall I maybe remove them completely from the > code? > > Thanks & Best regards > Christoph > > > From per.liden at oracle.com Wed Jul 5 09:32:00 2017 From: per.liden at oracle.com (Per Liden) Date: Wed, 5 Jul 2017 11:32:00 +0200 Subject: RFR: 8183552: Move align functions to align.hpp In-Reply-To: <935675bc-b011-6606-73f8-6a8454a68d11@oracle.com> References: <935675bc-b011-6606-73f8-6a8454a68d11@oracle.com> Message-ID: Looks good! /Per On 2017-07-05 00:25, Stefan Karlsson wrote: > Hi all, > > Please review this patch to move the align functions to align.hpp. > > http://cr.openjdk.java.net/~stefank/8183552/webrev.00 > https://bugs.openjdk.java.net/browse/JDK-8183552 > > The patch mainly moves the functions and adding #include > "utilities/align.hpp" lines, but it also moves ScaleForWordSize to > globals.hpp to prevent a cyclic dependency between align.hpp and > globalDefinitions.hpp. > > Thanks, > StefanK From christoph.langer at sap.com Wed Jul 5 09:33:03 2017 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 5 Jul 2017 09:33:03 +0000 Subject: [10] RFR(XS): 8183534: [TEST] Make detection of compilation mode more robust In-Reply-To: References: Message-ID: <9ebff9d98af743a29346070a3c8859fa@sap.com> Hi Volker, overall, the change looks good to me. When you create the Exception to print the stack trace (line 129 in the new file), why not add the value of 'java.vm.info' to the message? Also, I don't like the spaces around the '.' before PrintStackTrace() - but maybe that's just my personal taste. ;-) Best regards Christoph > -----Original Message----- > From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On > Behalf Of Volker Simonis > Sent: Dienstag, 4. Juli 2017 11:05 > To: HotSpot Open Source Developers ; > Java Core Libs > Subject: [10] RFR(XS): 8183534: [TEST] Make detection of compilation mode > more robust > > Hi, > > can you please review the following trivial change which makes the > detection of compilation mode for JTreg more robust: > > http://cr.openjdk.java.net/~simonis/webrevs/2017/8183534/ > https://bugs.openjdk.java.net/browse/JDK-8183534 > > The compilation mode in JTreg is parsed from the "java.vm.info" system > property. The current implementation expects that "java.vm.info" > contains exactly one of the strings "mixed mode", "compiled mode" or > "interpreted mode" otherwise the detection will fail and JTreg will > quit with an error. > > The detection can be done more robust by searching for the substrings > "mixed mode", "compiled mode" or "interpreted mode" instead, thus > allowing OpenJDK builders to store additional information in the > "java.vm.info" system property. > > Thank you and best regards, > Volker From stefan.karlsson at oracle.com Wed Jul 5 09:34:48 2017 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 5 Jul 2017 11:34:48 +0200 Subject: RFR: 8183552: Move align functions to align.hpp In-Reply-To: References: <935675bc-b011-6606-73f8-6a8454a68d11@oracle.com> Message-ID: <817aabbe-9b3e-9866-9b35-118bf205cd83@oracle.com> Thanks, Per! StefanK On 2017-07-05 11:32, Per Liden wrote: > Looks good! > > /Per > > On 2017-07-05 00:25, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to move the align functions to align.hpp. >> >> http://cr.openjdk.java.net/~stefank/8183552/webrev.00 >> https://bugs.openjdk.java.net/browse/JDK-8183552 >> >> The patch mainly moves the functions and adding #include >> "utilities/align.hpp" lines, but it also moves ScaleForWordSize to >> globals.hpp to prevent a cyclic dependency between align.hpp and >> globalDefinitions.hpp. >> >> Thanks, >> StefanK From stefan.karlsson at oracle.com Wed Jul 5 10:46:27 2017 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 5 Jul 2017 12:46:27 +0200 Subject: RFR: 8183544: Missing parentheses in is_size_aligned_ macro In-Reply-To: <6259AC65-4308-4DBE-AE7A-BC6A3A974530@oracle.com> References: <7b97c707-1baa-3282-4759-cc6226d05921@oracle.com> <6259AC65-4308-4DBE-AE7A-BC6A3A974530@oracle.com> Message-ID: <103de22f-a561-d1c3-411c-67c28e98830a@oracle.com> On 2017-07-05 03:02, Kim Barrett wrote: >> On Jul 4, 2017, at 10:24 AM, Stefan Karlsson wrote: >> >> Hi all, >> >> Please review this trivial patch to add parentheses to the is_aligned_ macro (used to be named is_size_aligned_). >> >> http://cr.openjdk.java.net/~stefank/8183544/webrev.00/ >> >> MikaelG found this issue while reviewing another patch. >> >> Thanks, >> StefanK > > I'm confused by the existing definition. Why isn't it > > #define is_aligned_(size, alignment) (((size) & align_mask(alignment)) == 0) > > That seems like it would be simpler, faster, and avoid multiple > evaluation of the size expression. I don't know. Maybe because it's a tad bit more readable in this "high-level" version, for some? I created and will fix: https://bugs.openjdk.java.net/browse/JDK-8183570 StefanK > From stefan.karlsson at oracle.com Wed Jul 5 10:53:41 2017 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 5 Jul 2017 12:53:41 +0200 Subject: RFR: 8183570: Rework is_aligned_ to avoid multiple evaluation of the size expression Message-ID: <7d13f891-a267-3661-4ffc-d724d351c519@oracle.com> Hi all, Please review this patch to rework is_aligned_ to avoid multiple evaluation of the size expression. http://cr.openjdk.java.net/~stefank/8183570/webrev.00 https://bugs.openjdk.java.net/browse/JDK-8183570 Thanks, StefanK From thomas.schatzl at oracle.com Wed Jul 5 12:36:48 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 05 Jul 2017 14:36:48 +0200 Subject: RFR: 8183570: Rework is_aligned_ to avoid multiple evaluation of the size expression In-Reply-To: <7d13f891-a267-3661-4ffc-d724d351c519@oracle.com> References: <7d13f891-a267-3661-4ffc-d724d351c519@oracle.com> Message-ID: <1499258208.15955.2.camel@oracle.com> Hi, On Wed, 2017-07-05 at 12:53 +0200, Stefan Karlsson wrote: > Hi all, > > Please review this patch to rework is_aligned_ to avoid multiple? > evaluation of the size expression. > > http://cr.openjdk.java.net/~stefank/8183570/webrev.00 > https://bugs.openjdk.java.net/browse/JDK-8183570 ? looks good. Thomas From stefan.karlsson at oracle.com Wed Jul 5 12:49:27 2017 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 5 Jul 2017 14:49:27 +0200 Subject: RFR: 8183570: Rework is_aligned_ to avoid multiple evaluation of the size expression In-Reply-To: <1499258208.15955.2.camel@oracle.com> References: <7d13f891-a267-3661-4ffc-d724d351c519@oracle.com> <1499258208.15955.2.camel@oracle.com> Message-ID: <26d77361-9e14-f43f-67f4-d9dba7e25e3e@oracle.com> Thanks, Thomas. StefanK On 2017-07-05 14:36, Thomas Schatzl wrote: > Hi, > > On Wed, 2017-07-05 at 12:53 +0200, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to rework is_aligned_ to avoid multiple >> evaluation of the size expression. >> >> http://cr.openjdk.java.net/~stefank/8183570/webrev.00 >> https://bugs.openjdk.java.net/browse/JDK-8183570 > > looks good. > > Thomas > From matthias.baesken at sap.com Wed Jul 5 13:36:21 2017 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 5 Jul 2017 13:36:21 +0000 Subject: jdk10 : UseSharedSpaces flag and INCLUDE_CDS macro Message-ID: <64495e2dba5d4d1788d101eaa6f89a3e@sap.com> Hello, when looking into CDS related build options, I noticed that most code-parts that are executed only when UseSharedSpaces is set, are guarded by the compile-time macro INCLUDE_CDS to support switching off compilation of this coding when CDS is disabled at compile time : See hotspot/make/lib/JvmFeatures.gmk : ifneq ($(call check-jvm-feature, cds), true) JVM_CFLAGS_FEATURES += -DINCLUDE_CDS=0 However some places miss the compile-time guarding ( #if INCLUDE_CDS ....) for example in share/vm/prims/jvmtiRedefineClasses.cpp share/vm/memory/universe.cpp (also some other places) Should I prepare a change and add the compile-time guard at the places where missing as well ? Best regards, Matthias From kim.barrett at oracle.com Wed Jul 5 14:32:37 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 5 Jul 2017 10:32:37 -0400 Subject: RFR: 8183570: Rework is_aligned_ to avoid multiple evaluation of the size expression In-Reply-To: <7d13f891-a267-3661-4ffc-d724d351c519@oracle.com> References: <7d13f891-a267-3661-4ffc-d724d351c519@oracle.com> Message-ID: > On Jul 5, 2017, at 6:53 AM, Stefan Karlsson wrote: > > Hi all, > > Please review this patch to rework is_aligned_ to avoid multiple evaluation of the size expression. > > http://cr.openjdk.java.net/~stefank/8183570/webrev.00 > https://bugs.openjdk.java.net/browse/JDK-8183570 > > Thanks, > StefanK Looks good. From stefan.karlsson at oracle.com Wed Jul 5 14:36:56 2017 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 5 Jul 2017 16:36:56 +0200 Subject: RFR: 8183570: Rework is_aligned_ to avoid multiple evaluation of the size expression In-Reply-To: References: <7d13f891-a267-3661-4ffc-d724d351c519@oracle.com> Message-ID: <23617ff4-dd41-2585-3b87-3053f2c09ed1@oracle.com> Thanks, Kim. StefanK On 2017-07-05 16:32, Kim Barrett wrote: >> On Jul 5, 2017, at 6:53 AM, Stefan Karlsson wrote: >> >> Hi all, >> >> Please review this patch to rework is_aligned_ to avoid multiple evaluation of the size expression. >> >> http://cr.openjdk.java.net/~stefank/8183570/webrev.00 >> https://bugs.openjdk.java.net/browse/JDK-8183570 >> >> Thanks, >> StefanK > > Looks good. > From volker.simonis at gmail.com Wed Jul 5 14:46:02 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 5 Jul 2017 16:46:02 +0200 Subject: [10] RFR(XS): 8183534: [TEST] Make detection of compilation mode more robust In-Reply-To: <9ebff9d98af743a29346070a3c8859fa@sap.com> References: <9ebff9d98af743a29346070a3c8859fa@sap.com> Message-ID: Hi Christoph, thanks for your review. On Wed, Jul 5, 2017 at 11:33 AM, Langer, Christoph wrote: > Hi Volker, > > overall, the change looks good to me. > > When you create the Exception to print the stack trace (line 129 in the new file), why not add the value of 'java.vm.info' to the message? Good point. I'll do so. > Also, I don't like the spaces around the '.' before PrintStackTrace() - but maybe that's just my personal taste. ;-) You're right. I somehow thought that the spaces are the default "coding style" for OpenJDK but I've checked and you're right that by default we don't use spaces when calling a method right on a 'new' expression. I'll update the change accordingly. Regards, Volker > > Best regards > Christoph > >> -----Original Message----- >> From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On >> Behalf Of Volker Simonis >> Sent: Dienstag, 4. Juli 2017 11:05 >> To: HotSpot Open Source Developers ; >> Java Core Libs >> Subject: [10] RFR(XS): 8183534: [TEST] Make detection of compilation mode >> more robust >> >> Hi, >> >> can you please review the following trivial change which makes the >> detection of compilation mode for JTreg more robust: >> >> http://cr.openjdk.java.net/~simonis/webrevs/2017/8183534/ >> https://bugs.openjdk.java.net/browse/JDK-8183534 >> >> The compilation mode in JTreg is parsed from the "java.vm.info" system >> property. The current implementation expects that "java.vm.info" >> contains exactly one of the strings "mixed mode", "compiled mode" or >> "interpreted mode" otherwise the detection will fail and JTreg will >> quit with an error. >> >> The detection can be done more robust by searching for the substrings >> "mixed mode", "compiled mode" or "interpreted mode" instead, thus >> allowing OpenJDK builders to store additional information in the >> "java.vm.info" system property. >> >> Thank you and best regards, >> Volker From thomas.stuefe at gmail.com Wed Jul 5 17:26:07 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 5 Jul 2017 19:26:07 +0200 Subject: [PATCH] linux-sparc build fixes In-Reply-To: <20170704093304.GD28259@physik.fu-berlin.de> References: <20170609102041.GA2477@physik.fu-berlin.de> <20170704093304.GD28259@physik.fu-berlin.de> Message-ID: Hi Adrian, Changes look fine to me. I cannot build and test though, do not have a sparc linux machine. Kind Regards, Thomas On Tue, Jul 4, 2017 at 11:33 AM, John Paul Adrian Glaubitz < glaubitz at physik.fu-berlin.de> wrote: > On Wed, Jun 14, 2017 at 01:50:06PM +0200, Erik Helin wrote: > > I think the first three patches (hotspot-add-missing-log-header.diff, > > hotspot-fix-checkbytebuffer.diff, rename-sparc-linux-atomic-header.diff) > all > > look good, thanks for fixing broken code. Consider them Reviewed by me. > > Every patch needs a corresponding issue in the bug tracker, so I went > ahead > > and created: > > - https://bugs.openjdk.java.net/browse/JDK-8182163 > > - https://bugs.openjdk.java.net/browse/JDK-8182164 > > - https://bugs.openjdk.java.net/browse/JDK-8182165 > > Any chance a second maintainer can review those? > > I have run the test suite as mentioned before: > > > https://people.debian.org/~glaubitz/openjdk-9_9~b170-2_ > sparc64.new.build.gz > > Thanks, > Adrian > > -- > .''`. John Paul Adrian Glaubitz > : :' : Debian Developer - glaubitz at debian.org > `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de > `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 > From glaubitz at physik.fu-berlin.de Wed Jul 5 17:27:12 2017 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Wed, 5 Jul 2017 19:27:12 +0200 Subject: [PATCH] linux-sparc build fixes In-Reply-To: References: <20170609102041.GA2477@physik.fu-berlin.de> <20170704093304.GD28259@physik.fu-berlin.de> Message-ID: On 07/05/2017 07:26 PM, Thomas St?fe wrote: > Changes look fine to me. I cannot build and test though, do not have a sparc linux machine. We have a fast SPARC T5 running Debian unstable available and I could create an account for you if you're interested. Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From thomas.stuefe at gmail.com Wed Jul 5 17:37:42 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 5 Jul 2017 19:37:42 +0200 Subject: [PATCH] linux-sparc build fixes In-Reply-To: References: <20170609102041.GA2477@physik.fu-berlin.de> <20170704093304.GD28259@physik.fu-berlin.de> Message-ID: On Wed, Jul 5, 2017 at 7:27 PM, John Paul Adrian Glaubitz < glaubitz at physik.fu-berlin.de> wrote: > On 07/05/2017 07:26 PM, Thomas St?fe wrote: > > Changes look fine to me. I cannot build and test though, do not have a > sparc linux machine. > > We have a fast SPARC T5 running Debian unstable available and I could > create an account for you if you're interested. > > Adrian > > Nah, I believe you :) Changes are trivial enough. ..Thomas > -- > .''`. John Paul Adrian Glaubitz > : :' : Debian Developer - glaubitz at debian.org > `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de > `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 > From glaubitz at physik.fu-berlin.de Wed Jul 5 17:39:10 2017 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Wed, 5 Jul 2017 19:39:10 +0200 Subject: [PATCH] linux-sparc build fixes In-Reply-To: References: <20170609102041.GA2477@physik.fu-berlin.de> <20170704093304.GD28259@physik.fu-berlin.de> Message-ID: <78dbbda5-f3ea-009e-3989-95a2292a0a46@physik.fu-berlin.de> On 07/05/2017 07:37 PM, Thomas St?fe wrote: > Nah, I believe you :) Changes are trivial enough. OK. So we're good to merge then? -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From thomas.stuefe at gmail.com Wed Jul 5 19:23:12 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 05 Jul 2017 19:23:12 +0000 Subject: [PATCH] linux-sparc build fixes In-Reply-To: <78dbbda5-f3ea-009e-3989-95a2292a0a46@physik.fu-berlin.de> References: <20170609102041.GA2477@physik.fu-berlin.de> <20170704093304.GD28259@physik.fu-berlin.de> <78dbbda5-f3ea-009e-3989-95a2292a0a46@physik.fu-berlin.de> Message-ID: On Wed 5. Jul 2017 at 19:39, John Paul Adrian Glaubitz < glaubitz at physik.fu-berlin.de> wrote: > On 07/05/2017 07:37 PM, Thomas St?fe wrote: > > Nah, I believe you :) Changes are trivial enough. > > OK. So we're good to merge then? > > -- > .''`. John Paul Adrian Glaubitz > : :' : Debian Developer - glaubitz at debian.org > `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de > `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 > Yes, you now have two reviewers. But for these changes (hotspot) you need a sponsor, which needs to be an Oracle employee, which I am not. Maybe Eric could sponsor the change? Best regards, Thomas From frederic.parain at oracle.com Wed Jul 5 19:34:15 2017 From: frederic.parain at oracle.com (Frederic Parain) Date: Wed, 5 Jul 2017 15:34:15 -0400 Subject: RFR: JDK-8172791 fixing issues with Reserved Stack Area In-Reply-To: <0b99ee01-4e40-dc92-b116-1abcd6ff257a@oracle.com> References: <3cd1c352-33fa-96fe-d2f6-a78a25c0543e@oracle.com> <0b99ee01-4e40-dc92-b116-1abcd6ff257a@oracle.com> Message-ID: Dan, Thank your for reviewing this code. My answers are in-lined below. > On Jun 30, 2017, at 19:56, Daniel D. Daugherty wrote: > > On 6/30/17 9:03 AM, Andrew Haley wrote: >> Just to explain why this patch is needed: without it, JEP 270 >> (ReservedStackArea) does not work at all if a method with a >> ReservedStack annotation is inlined. Which, in practice, is all the >> time, because aggressive inlining is what C2 does. >> >> Can somebody please have a look at this? >> >> >> On 18/04/17 15:47, Frederic Parain wrote: >>> Greetings, >>> >>> Please review this fix which addresses several issues with the >>> ReservedStackArea implementation: >>> 1 - the method look_for_reserved_stack_annotated_method() fails to >>> detect in-lined >>> annotated methods, making the annotation ineffective for >>> in-lined methods >>> 2 - sometime an assertion failure related to reserved area status >>> occurs after incorrect >>> restoring of guards pages during a return from runtime >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8172791 >>> >>> webrev: >>> http://cr.openjdk.java.net/~fparain/8172791/webrev.00/index.html > > src/cpu/x86/vm/interp_masm_x86.cpp > Is the deletion of: > > L1083: push(rthread); > > related to the assertion failure part of the fix? It looks like > it is just fixing a call protocol error (pushing rthread when it > is not needed), but I'm not sure. It?s fixing a call protocol error, this is not related to the assertion failure described in point 2 above. AFAIK, this error did not trigger issues, but it is cleaner to remove it. > > src/share/vm/code/compiledMethod.cpp > No comments. > > src/share/vm/code/compiledMethod.hpp > No comments. > > src/share/vm/opto/compile.cpp > No comments. > > src/share/vm/runtime/sharedRuntime.cpp > L3157: for (ScopeDesc *sd = nm->scope_desc_near(fr.pc()); sd; sd = sd->sender()) { > nit: implied boolean with 'sd;' > please change to: 'sd != NULL;? Fixed > > test/runtime/ReservedStack/ReservedStackTest.java > Why stop running the test on -Xint? > Running in interpreted only has very limited interest because the bug cannot occur in this mode (due to the respective sizes of the different frames). Removing this mode makes the test faster. > > Thumbs up on the fix. > > > This fix is going to need a review from someone on the > Compiler team also. Fred, who did your JEP-270 reviews > from the Compiler team? > I cannot remember. I?ve discussed JEP-270 code with the compiler team before the RFR. I?ll contact them to have a second review. Thank you, Fred >>> >>> This fix has been contributed by Andrew Haley. >>> >>> Thank you, >>> >>> Fred From erik.helin at oracle.com Thu Jul 6 12:55:53 2017 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 6 Jul 2017 14:55:53 +0200 Subject: [PATCH] linux-sparc build fixes In-Reply-To: References: <20170609102041.GA2477@physik.fu-berlin.de> <20170704093304.GD28259@physik.fu-berlin.de> <78dbbda5-f3ea-009e-3989-95a2292a0a46@physik.fu-berlin.de> Message-ID: On 07/05/2017 09:23 PM, Thomas St?fe wrote: > > On Wed 5. Jul 2017 at 19:39, John Paul Adrian Glaubitz > > wrote: > > On 07/05/2017 07:37 PM, Thomas St?fe wrote: > > Nah, I believe you :) Changes are trivial enough. > > OK. So we're good to merge then? > > -- > .''`. John Paul Adrian Glaubitz > : :' : Debian Developer - glaubitz at debian.org > > `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de > > `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 > > Yes, you now have two reviewers. But for these changes (hotspot) you > need a sponsor, which needs to be an Oracle employee, which I am not. > > Maybe Eric could sponsor the change? Yep, I can shepherd the patches in (aka getting them through the build and test system we all know as JPRT). Thanks Thomas for reviewing! Erik > Best regards, Thomas > > From erik.helin at oracle.com Thu Jul 6 13:00:09 2017 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 6 Jul 2017 15:00:09 +0200 Subject: [PATCH] linux-sparc build fixes In-Reply-To: <20170704093304.GD28259@physik.fu-berlin.de> References: <20170609102041.GA2477@physik.fu-berlin.de> <20170704093304.GD28259@physik.fu-berlin.de> Message-ID: <2d442ae6-50bf-6354-08e2-6874118ccfd8@oracle.com> On 07/04/2017 11:33 AM, John Paul Adrian Glaubitz wrote: > On Wed, Jun 14, 2017 at 01:50:06PM +0200, Erik Helin wrote: >> I think the first three patches (hotspot-add-missing-log-header.diff, >> hotspot-fix-checkbytebuffer.diff, rename-sparc-linux-atomic-header.diff) all >> look good, thanks for fixing broken code. Consider them Reviewed by me. >> Every patch needs a corresponding issue in the bug tracker, so I went ahead >> and created: >> - https://bugs.openjdk.java.net/browse/JDK-8182163 >> - https://bugs.openjdk.java.net/browse/JDK-8182164 >> - https://bugs.openjdk.java.net/browse/JDK-8182165 > > Any chance a second maintainer can review those? > > I have run the test suite as mentioned before: > >> https://people.debian.org/~glaubitz/openjdk-9_9~b170-2_sparc64.new.build.gz Although these three patches are correct, it seems like you have some way to go still to make this port solid :) All tests (that are applicable) in hotspot_tier1 should pass (besides the ones in ProblemList.txt). If you run the test via the top-level Makefiles, as in: $ make test TEST=hotspot_tier1 then all tests should pass (because the Makefiles will pass ProblemList.txt to jtreg). After you get the code to compile, this should be your next goal :) Thanks, Erik > Thanks, > Adrian > From erik.osterlund at oracle.com Thu Jul 6 13:08:23 2017 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 6 Jul 2017 15:08:23 +0200 Subject: RFR (M): 8183927: Hotspot needs C++ type_traits metaprogramming utilities Message-ID: <595E3647.1030101@oracle.com> Hi, I'm sure we have all had moments when we wished we could use the C++11 type_traits header and the nice metaprogramming tools in there, but then suddenly remembered that we can not, because we are building libjvm.so with C++03 and are still arguing whether we can use namespaces. Well.. welcome to the future. I am delighted to announce that now at least some of those tools will become available. A bunch of C++11 compliant metaprogramming tools that can be used already in C++03 have been developed. They have been developed with the most conservative and portable C++03 code possible so that even exotic compilers can support it. Special thanks go to Kim Barret whom I have had many long conversations about many paragraphs of the C++ standard throughout this work. It has helped a lot and improved the code. While this change introduces the traits without using it in the code (more than from gtest), it will become used in already prepared patches for refactoring Atomic and OrderAccess to use templates rather than hand picked types like the JNI-specific jint etc types. That serves as the bottom layer of the upcoming GC barrier interface that also requires these utilities. Since they are rather large changes, I hope it is okay that I separate out the metaprogramming utilities. I argue that they are of value in general in hotspot and will make our lives easier. All metaprogramming tools have been placed in a new hotspot/src/share/vm/metaprogramming directory. Tests have been written and placed in a new test/native/metaprogramming directory. A bug ID can be found here: https://bugs.openjdk.java.net/browse/JDK-8183927 A webrev with code can be found here: http://cr.openjdk.java.net/~eosterlund/8183927/webrev.00/ Here is a list of metaprogramming tools that have been introduced with a corresponding short explanation: ============================ template struct IntegralConstant This metafunction takes as parameters a type and a value. The type and value can later be queried from the value_type and value members respectively. Here is an example usage: typedef IntegralConstant TrueType; Now TrueType::value_type is the bool type, and TrueType::value is the true boolean value. Many other metafunctions will later inherit from either TrueType or FalseType to e.g. denote the metafunction returning true or false based on some condition. The following traits that start with Is* belong to this category. They manifest a constant evaluated at compile time that checks some condition of types. ============================ template struct IsSame This metafunction takes as parameters two types, X and Y, and returns true iff they are the same type. Otherwise it returns false. ============================ template struct IsConst This metafunction takes as parameter a type T, and returns true iff T has a const qualifier in it. ============================ template struct IsVolatile This metafunction takes as parameter a type T, and returns true iff T has a volatile qualifier in it. ============================ template struct IsSigned This metafunction takes as parameter a type T, and checks if it is signed. This is equivalent to asking std::numeric_limits<>::is_signed, but additionally disregards CV qualifiers. ============================ template struct IsIntegral This metafunction takes as parameter a type T, and returns true if it is an integral type. This is similar to asking std::numeric_limits<>::is_integer, but additionally disregards CV qualifiers. Note that this returns false for enums. ============================ template struct IsSignedIntegral template struct IsUnsignedIntegral These two metafunctions are helpers that return true for integral types that have corresponding signedness. ============================ template struct IsFloatingPoint This metafunction takes as parameter a type T, and returns true iff T is a floating point type (irrespective of CV qualifiers). ============================ template class IsPointer This metafunction takes as parameter a type T, and returns true iff T is a pointer type (irrespective of CV qualifiers). ============================ That is it as far as metafunctions returning booleans go in these changes. So now we move on to metafunctions that make use of these boolean conditions. template struct Conditional This metafunction constitutes a conditional type, controlled by the condition passed in. Its type evaluates to TrueType iff the condition is true, and otherwise evaluates to FalseType. Here is a contrived toy example: template void example(T base) { typedef Conditional::value, ptrdiff_t, T> ScaleType; ScaleType offset = offset(); do_something_clever(base + offset); } ============================ In this example, the type of the offset is ptrdiff_t iff T is a pointer and otherwise it is T. This allows passing in pointers and integers to the function, and adding offsets into it. template struct EnableIf This metafunction is the key to SFINAE (Substitution Failure Is Not An Error), a C++ metaprogramming idiom that allows explicitly allowing or disallowing function overloads based on precise conditions for types passed in as parameter. The EnableIf overload guard should replace the return value of a function. The type T is the type that is intended to be returned if the overload is allowed based on the passed in condition. By default that is void. Here is a contrived toy example usage: template EnableIf::value> print(T x) {} template EnableIf::value> print(T x) {} ... print(1); // calls the first overload enabled for integral types print("hello SFINAE world"); // call the second overload enabled for non-integral types ============================ Now we move on to metafunctions that remove properties like e.g. CV qualifiers, pointers and references from types, resulting in a new type. template struct RemovePointer This metafunction takes as parameter a type T, and returns the type under the pointer iff T is a pointer, and otherwise returns the same type T. ============================ template struct RemoveReference This metafunction takes as parameter a type T, and returns the type under the reference iff T is a pointer, and otherwise returns the same type T. ============================ template struct RemoveCV This metafunction takes as parameter a type T, and returns the type T without CV qualifiers. ============================ template struct Decay This metafunction takes as parameter a type T, and returns the CV unqualified type under the reference iff T is a reference type, and otherwise returns the CV unqualified type T. ============================ Testing: A bunch of tests have been added to the test/native/metaprogramming folder for all traits (except IntegralConstant, because there is not a whole lot to test there, and it is used by all condition metafunctions in their corresponding tests). They have gone through JPRT, and work well on all oracle supported platforms, and is also known to have recently built with the xlC compiler on AIX. I would like to ask external maintainers though to please verify that this works on your compilers. I have done my best to use the most conservative tricks in the book to make it as portable as possible, and I hope it will work well for us all. Thanks, /Erik From glaubitz at physik.fu-berlin.de Thu Jul 6 13:09:19 2017 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Thu, 6 Jul 2017 15:09:19 +0200 Subject: [PATCH] linux-sparc build fixes In-Reply-To: <2d442ae6-50bf-6354-08e2-6874118ccfd8@oracle.com> References: <20170609102041.GA2477@physik.fu-berlin.de> <20170704093304.GD28259@physik.fu-berlin.de> <2d442ae6-50bf-6354-08e2-6874118ccfd8@oracle.com> Message-ID: <20170706130918.GJ23904@physik.fu-berlin.de> On Thu, Jul 06, 2017 at 03:00:09PM +0200, Erik Helin wrote: > Although these three patches are correct, it seems like you have some way to > go still to make this port solid :) All tests (that are applicable) in > hotspot_tier1 should pass (besides the ones in ProblemList.txt). If you run > the test via the top-level Makefiles, as in: Oh, I agree. But one has to start somewhere, no? ;) > $ make test TEST=hotspot_tier1 > > then all tests should pass (because the Makefiles will pass ProblemList.txt > to jtreg). After you get the code to compile, this should be your next goal > :) Thanks, I will look into this. Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From erik.helin at oracle.com Thu Jul 6 13:09:07 2017 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 6 Jul 2017 15:09:07 +0200 Subject: RFR(XL): 8182299: Enable disabled clang warnings, build on OSX 10 + Xcode 8 In-Reply-To: <901E4351-42D1-44D4-85F6-FBD4C715797A@amazon.com> References: <27FD0413-52BC-42E6-A5B0-3C92A49A2D6F@amazon.com> <1CBD62A9-9B1B-4B05-AAF9-BE2D52DE8C79@amazon.com> <6B296ABE-66C0-4C32-AC4E-8674BE103514@oracle.com> <3ACE75CF-CEA6-4081-952A-BBA138763582@amazon.com> <435C96F2-417E-4A47-8649-28E34E2908AE@amazon.com> <1d06f5fe-c3f8-f599-58bc-09b2099c819c@oracle.com> <035f2cd3-de68-efe7-d32e-bb6f06c58f93@oracle.com> <901E4351-42D1-44D4-85F6-FBD4C715797A@amazon.com> Message-ID: On 07/03/2017 06:47 PM, Hohensee, Paul wrote: > Fixed heapRegionType.cpp per your comment. > > http://cr.openjdk.java.net/~phh/8182299/webrev_hotspot.02/ > > Jesper says he?s ready to push the closed changes. Any other comments on the open ones? The GC changes looks fine to me, but I can't comment on the runtime/compiler changes. Thanks, Erik > Thanks, > > Paul > > On 7/3/17, 12:58 AM, "Erik Helin" wrote: > > On 06/30/2017 06:30 PM, Hohensee, Paul wrote: > > Thanks for picking up the closed work, Jesper, and thanks for the review, Erik. It?s good to be back working on server Java. ( > > > > Yes, I added the G1RootClosures virtual destructor because of the warning. > > > > In heapRegionType.cpp, I just followed along with the existing funky indentation and made the default clause also have a return because all the other clauses do. Clang would indeed be ok with a default clause with just a break. Also, I?ve used the same idiom in other files, so I?d change those too if I change these. Which would you prefer? I?m not invested in any particular format. > > I don't know about the other files, but given the indentation in > heapRegionType.cpp, I would prefer: > > switch (_tag) { > // ... bunch of case statements > case ArchiveTag: return "ARC"; > default: break; > } > ShouldNotReachHere(); > return NULL; // just to keep some compilers happy > } > > My reason for this is mainly that all the other case statements are on > one line (`case` followed by `return`), so I would prefer to the > `default` case to also be on one line (`default` followed by `break`). > > However, for in HeapRegionType::is_valid, I think your changes follows > the existing style in that method: > > 37 case ArchiveTag: > 38 return true; > 39 default: > 40 return false; > 41 } > > I feel like this is quickly approaching bikeshed territory, but the > above is at least my preference :) > > Thanks, > Erik > > > Thanks, > > > > Paul > > > > On 6/30/17, 3:00 AM, "jesper.wilhelmsson at oracle.com" wrote: > > > > I ran the change through JPRT so at least it builds and runs on all platforms we support there. > > I'm currently fixing our closed code to build on Mac with this change. > > /Jesper > > > > > On 30 Jun 2017, at 10:35, Erik Helin wrote: > > > > > > Hi Paul, > > > > > > thanks for contributing! Please see my comments regarding the GC changes below. > > > > > > On 06/29/2017 03:31 PM, Hohensee, Paul wrote: > > >> I now have access to cr.openjdk.java.net, so the latest webrevs are at > > >> > > >> http://cr.openjdk.java.net/~phh/8182299/webrev_jdk.00/ > > >> http://cr.openjdk.java.net/~phh/8182299/webrev_hotspot.00/ > > > > > > gc/g1/g1RootClosures.cpp: > > > - // Closures to process raw oops in the root set. > > > + virtual ~G1RootClosures() {} > > > + > > > +// Closures to process raw oops in the root set. > > > > > > I assume this is added because there is some warning about having only pure virtual methods but not having a virtual destructor. None of the classes inheriting from G1RootClosures needs a destructor, nor does G1RootClosures itself (it is just an "interface"). So there is no problem with correctness here :) > > > > > > However, I like to have lots of warnings enabled, so for me it is fine to an empty virtual destructor here just to please the compiler. > > > > > > gc/g1/heapRegion.cpp: looks correct > > > > > > gc/g1/heapRegionType.cpp: > > > The indentation seems a bit funky to me here :) Also, have you compiled this on some other platforms? I think the last return is outside of the switch just to, as the comment says, "keep some compilers happy". > > > > > > Would clang be ok with having an empty default clause with just a break? And then fall through to the return outside of the switch? > > > > > > gc/g1/ptrQueue.cpp: looks correct > > > > > > gc/parallel/psPromotionManager.cpp: looks correct > > > > > > Thanks, > > > Erik > > > > > >> On 6/28/17, 3:50 PM, "hotspot-dev on behalf of Hohensee, Paul" wrote: > > >> > > >> Thanks for the review, Jesper. > > >> > > >> New webrev sent, has only a change to nativeInst_x86.cpp. > > >> > > >> In nativeInst_x86.cpp, I formatted the expression so I could easily understand it, but you?re right, not everyone does it that way (maybe only me!), so I?ve changed it to > > >> > > >> if (((ubyte_at(0) & NativeTstRegMem::instruction_rex_prefix_mask) == NativeTstRegMem::instruction_rex_prefix && > > >> ubyte_at(1) == NativeTstRegMem::instruction_code_memXregl && > > >> (ubyte_at(2) & NativeTstRegMem::modrm_mask) == NativeTstRegMem::modrm_reg) || > > >> (ubyte_at(0) == NativeTstRegMem::instruction_code_memXregl && > > >> (ubyte_at(1) & NativeTstRegMem::modrm_mask) == NativeTstRegMem::modrm_reg)) { > > >> > > >> In graphKit.cpp, the old code was > > >> > > >> #ifdef ASSERT > > >> case Deoptimization::Action_none: > > >> case Deoptimization::Action_make_not_compilable: > > >> break; > > >> default: > > >> fatal("unknown action %d: %s", action, Deoptimization::trap_action_name(action)); > > >> break; > > >> #endif > > >> > > >> In the non-ASSERT case, the compiler complained about the lack of Action_none, Action_make_not_compilable and default. If the warning had been turned off, the result would have been ?break;? for all three. In the ASSERT case, Action_none and Action_make_not_compilable result in ?break;?, and in the default case ?fatal(); break;? > > >> > > >> The new code is > > >> > > >> case Deoptimization::Action_none: > > >> case Deoptimization::Action_make_not_compilable: > > >> break; > > >> default: > > >> #ifdef ASSERT > > >> fatal("unknown action %d: %s", action, Deoptimization::trap_action_name(action)); > > >> #endif > > >> break; > > >> > > >> The compiler doesn?t complain about Action_none, Action_make_not_compilable or default anymore. In the non-ASSERT case, the result is ?break;? for all three, same as for the old code. In the ASSERT case, Action_none and Action_make_not_compilable result in ?break;?, and in the default case ?fatal(); break;?, again same as for the old code. > > >> > > >> Thanks, > > >> > > >> Paul > > >> > > >> On 6/28/17, 10:36 AM, "jesper.wilhelmsson at oracle.com" wrote: > > >> > > >> Hi Paul, > > >> > > >> Thanks for doing this change! In general everything looks really good, there are a lot of really nice cleanups here. I just have two minor questions/nits: > > >> > > >> * In hotspot/cpu/x86/vm/nativeInst_x86.hpp it seems the expression already have parenthesis around the & operations and the change here is "only" cleaning up the layout of the code which is not a bad thing in it self, but you move the logical operators to the beginning of each line which is a quite different style than the rest of the code in the same function where the operators are at the end of the line. > > >> > > >> * In hotspot/share/vm/opto/graphKit.cpp you moved the #ifdef ASSERT so that Action_none and Action_make_not_compilable are available also when ASSERT is not defined. I don't see this mentioned in your description of the change. Was this change intentional? > > >> > > >> Thanks, > > >> /Jesper > > >> > > >> > > >> > On 27 Jun 2017, at 21:34, Hohensee, Paul wrote: > > >> > > > >> > An attempt at better formatting. > > >> > > > >> > https://bugs.openjdk.java.net/browse/JDK-8182299 > > >> > http://cr.openjdk.java.net/~jwilhelm/8182299/webrev_jdk.00/ > > >> > http://cr.openjdk.java.net/~jwilhelm/8182299/webrev_hotspot.00/ > > >> > > > >> > Jesper has been kind enough to host the webrevs while I get my cr.openjdk.net account set up, and to be the sponsor. > > >> > > > >> > This rfe a combination of enabling disabled clang warnings and getting jdk10 to build on OSX 10 and Xcode 8. At least one enabled warning (delete-non-virtual-dtor) detected what seems to me a real potential bug, with the rest enforcing good code hygiene. > > >> > > > >> > These changes are only in OpenJDK, so I?m looking for a volunteer to make the closed changes. > > >> > > > >> > Thanks, > > >> > > > >> > Paul > > >> > > > >> > > > >> > Here are the jdk file-specific details: > > >> > > > >> > java_md_macosx.c splashscreen_sys.m > > >> > > > >> > Removed objc_registerThreadWithCollector() since it's obsolete and of questionable value in any case. > > >> > > > >> > NSApplicationAWT.m > > >> > > > >> > Use the correct NSEventMask rather than NSUInteger. > > >> > > > >> > jdhuff.c jdphuff.c > > >> > > > >> > Shifting a negative signed value is undefined. > > >> > > > >> > Here are the hotspot notes: > > >> > > > >> > Here are the lists of files affected by enabling a given warning: > > >> > > > >> > switch: all of these are lack of a default clause > > >> > > > >> > c1_LIRAssembler_x86.cpp c1_LIRGenerator_x86.cpp c1_LinearScan_x86.hpp > > >> > jniFastGetField_x86_64.cpp assembler.cpp c1_Canonicalizer.cpp > > >> > c1_GraphBuilder.cpp c1_Instruction.cpp c1_LIR.cpp c1_LIRGenerator.cpp > > >> > c1_LinearScan.cpp c1_ValueStack.hpp c1_ValueType.cpp > > >> > bcEscapeAnalyzer.cpp ciArray.cpp ciEnv.cpp ciInstance.cpp ciMethod.cpp > > >> > ciMethodBlocks.cpp ciMethodData.cpp ciTypeFlow.cpp > > >> > compiledMethod.cpp dependencies.cpp nmethod.cpp compileTask.hpp > > >> > heapRegionType.cpp abstractInterpreter.cpp bytecodes.cpp > > >> > invocationCounter.cpp linkResolver.cpp rewriter.cpp jvmciCompilerToVM.cpp > > >> > jvmciEnv.cpp universe.cpp cpCache.cpp generateOopMap.cpp > > >> > method.cpp methodData.cpp compile.cpp connode.cpp gcm.cpp graphKit.cpp > > >> > ifnode.cpp library_call.cpp memnode.cpp parse1.cpp > > >> > parse2.cpp phaseX.cpp superword.cpp type.cpp vectornode.cpp > > >> > jvmtiClassFileReconstituter.cpp jvmtiEnter.xsl jvmtiEventController.cpp > > >> > jvmtiImpl.cpp jvmtiRedefineClasses.cpp methodComparator.cpp methodHandles.cpp > > >> > advancedThresholdPolicy.cpp reflection.cpp relocator.cpp sharedRuntime.cpp > > >> > simpleThresholdPolicy.cpp writeableFlags.cpp globalDefinitions.hpp > > >> > > > >> > delete-non-virtual-dtor: these may be real latent bugs due to possible failure to execute destructor(s) > > >> > > > >> > decoder_aix.hpp decoder_machO.hpp classLoader.hpp g1RootClosures.hpp > > >> > jvmtiImpl.hpp perfData.hpp decoder.hpp decoder_elf.hpp > > >> > > > >> > dynamic-class-memaccess: obscure use of memcpy > > >> > > > >> > method.cpp > > >> > > > >> > empty-body: ?;? isn?t good enough for clang, it prefers {} > > >> > > > >> > objectMonitor.cpp mallocSiteTable.cpp > > >> > > > >> > format: matches printf format strings against arguments. debug output will be affected by > > >> > incorrect code changes to these. > > >> > > > >> > macroAssembler_x86.cpp os_bsd.cpp os_bsd_x86.cpp ciMethodData.cpp javaClasses.cpp > > >> > debugInfo.cpp logFileOutput.cpp constantPool.cpp jvmtiEnter.xsl jvmtiRedefineClasses.cpp > > >> > safepoint.cpp thread.cpp > > >> > > > >> > logical-op-parentheses: can be tricky to get correct. There are a few very long-winded predicates. > > >> > > > >> > nativeInst_x86.hpp archDesc.cpp output_c.cpp output_h.cpp c1_GraphBuilder.cpp > > >> > c1_LIRGenerator.cpp c1_LinearScan.cpp bcEscapeAnalyzer.cpp ciMethod.cpp > > >> > stackMapTableFormat.hpp compressedStream.cpp dependencies.cpp heapRegion.cpp > > >> > ptrQueue.cpp psPromotionManager.cpp jvmciCompilerToVM.cpp cfgnode.cpp > > >> > chaitin.cpp compile.cpp compile.hpp escape.cpp graphKit.cpp lcm.cpp > > >> > loopTransform.cpp loopnode.cpp loopopts.cpp macro.cpp memnode.cpp > > >> > output.cpp parse1.cpp parseHelper.cpp reg_split.cpp superword.cpp > > >> > superword.hpp jniCheck.cpp jvmtiEventController.cpp arguments.cpp > > >> > javaCalls.cpp sharedRuntime.cpp > > >> > > > >> > parentheses > > >> > > > >> > adlparse.cpp > > >> > > > >> > parentheses-equality > > >> > > > >> > output_c.cpp javaAssertions.cpp gcm.cpp > > >> > > > >> > File-specific details: > > >> > > > >> > GensrcAdlc.gmk CompileJvm.gmk > > >> > Left tautological-compare in place to allow null 'this' pointer checks in methods > > >> > intended to be called from a debugger. > > >> > > > >> > CompileGTest.gmk > > >> > Just an enhanced comment. > > >> > > > >> > MacosxDebuggerLocal.m > > >> > PT_ATTACH has been replaced by PT_ATTACHEXC > > >> > > > >> > ciMethodData.cp > > >> > " 0x%" FORMAT64_MODIFIER "x" reduces to "0x%llx", whereas > > >> > " " INTPTRNZ_FORMAT reduces to "0x%lx", which latter is what clang want. > > >> > > > >> > generateOopMap.cpp > > >> > Refactored duplicate code in print_current_state(). > > >> > > > >> > binaryTreeDictionary.cpp/hpp, hashtable.cpp/hpp > > >> > These provoked ?instantiation of variable required here, > > >> > but no definition is available?. > > >> > > > >> > globalDefinitions_gcc.hpp > > >> > Define FORMAT64_MODIFIER properly for Apple, needed by os.cpp. > > >> > > > >> > globalDefinitions.hpp > > >> > Add INTPTRNZ_FORMAT, needed by ciMethodData.cpp. > > >> > > >> > > >> > > >> > > >> > > > > > > > > From vladimir.x.ivanov at oracle.com Thu Jul 6 13:27:56 2017 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Thu, 6 Jul 2017 16:27:56 +0300 Subject: RFR(XL): 8182299: Enable disabled clang warnings, build on OSX 10 + Xcode 8 In-Reply-To: <901E4351-42D1-44D4-85F6-FBD4C715797A@amazon.com> References: <27FD0413-52BC-42E6-A5B0-3C92A49A2D6F@amazon.com> <1CBD62A9-9B1B-4B05-AAF9-BE2D52DE8C79@amazon.com> <6B296ABE-66C0-4C32-AC4E-8674BE103514@oracle.com> <3ACE75CF-CEA6-4081-952A-BBA138763582@amazon.com> <435C96F2-417E-4A47-8649-28E34E2908AE@amazon.com> <1d06f5fe-c3f8-f599-58bc-09b2099c819c@oracle.com> <035f2cd3-de68-efe7-d32e-bb6f06c58f93@oracle.com> <901E4351-42D1-44D4-85F6-FBD4C715797A@amazon.com> Message-ID: Paul, > http://cr.openjdk.java.net/~phh/8182299/webrev_hotspot.02/ Compiler changes look good. Just a small refactoring suggestion while you on it: src/cpu/x86/vm/macroAssembler_x86.cpp: 0x%016lx can be replaced with INTPTR_FORMAT. Thanks for taking care of the cleanup and welcome back! :-) Best regards, Vladimir Ivanov > > Jesper says he?s ready to push the closed changes. Any other comments on the open ones? > > Thanks, > > Paul > > On 7/3/17, 12:58 AM, "Erik Helin" wrote: > > On 06/30/2017 06:30 PM, Hohensee, Paul wrote: > > Thanks for picking up the closed work, Jesper, and thanks for the review, Erik. It?s good to be back working on server Java. ( > > > > Yes, I added the G1RootClosures virtual destructor because of the warning. > > > > In heapRegionType.cpp, I just followed along with the existing funky indentation and made the default clause also have a return because all the other clauses do. Clang would indeed be ok with a default clause with just a break. Also, I?ve used the same idiom in other files, so I?d change those too if I change these. Which would you prefer? I?m not invested in any particular format. > > I don't know about the other files, but given the indentation in > heapRegionType.cpp, I would prefer: > > switch (_tag) { > // ... bunch of case statements > case ArchiveTag: return "ARC"; > default: break; > } > ShouldNotReachHere(); > return NULL; // just to keep some compilers happy > } > > My reason for this is mainly that all the other case statements are on > one line (`case` followed by `return`), so I would prefer to the > `default` case to also be on one line (`default` followed by `break`). > > However, for in HeapRegionType::is_valid, I think your changes follows > the existing style in that method: > > 37 case ArchiveTag: > 38 return true; > 39 default: > 40 return false; > 41 } > > I feel like this is quickly approaching bikeshed territory, but the > above is at least my preference :) > > Thanks, > Erik > > > Thanks, > > > > Paul > > > > On 6/30/17, 3:00 AM, "jesper.wilhelmsson at oracle.com" wrote: > > > > I ran the change through JPRT so at least it builds and runs on all platforms we support there. > > I'm currently fixing our closed code to build on Mac with this change. > > /Jesper > > > > > On 30 Jun 2017, at 10:35, Erik Helin wrote: > > > > > > Hi Paul, > > > > > > thanks for contributing! Please see my comments regarding the GC changes below. > > > > > > On 06/29/2017 03:31 PM, Hohensee, Paul wrote: > > >> I now have access to cr.openjdk.java.net, so the latest webrevs are at > > >> > > >> http://cr.openjdk.java.net/~phh/8182299/webrev_jdk.00/ > > >> http://cr.openjdk.java.net/~phh/8182299/webrev_hotspot.00/ > > > > > > gc/g1/g1RootClosures.cpp: > > > - // Closures to process raw oops in the root set. > > > + virtual ~G1RootClosures() {} > > > + > > > +// Closures to process raw oops in the root set. > > > > > > I assume this is added because there is some warning about having only pure virtual methods but not having a virtual destructor. None of the classes inheriting from G1RootClosures needs a destructor, nor does G1RootClosures itself (it is just an "interface"). So there is no problem with correctness here :) > > > > > > However, I like to have lots of warnings enabled, so for me it is fine to an empty virtual destructor here just to please the compiler. > > > > > > gc/g1/heapRegion.cpp: looks correct > > > > > > gc/g1/heapRegionType.cpp: > > > The indentation seems a bit funky to me here :) Also, have you compiled this on some other platforms? I think the last return is outside of the switch just to, as the comment says, "keep some compilers happy". > > > > > > Would clang be ok with having an empty default clause with just a break? And then fall through to the return outside of the switch? > > > > > > gc/g1/ptrQueue.cpp: looks correct > > > > > > gc/parallel/psPromotionManager.cpp: looks correct > > > > > > Thanks, > > > Erik > > > > > >> On 6/28/17, 3:50 PM, "hotspot-dev on behalf of Hohensee, Paul" wrote: > > >> > > >> Thanks for the review, Jesper. > > >> > > >> New webrev sent, has only a change to nativeInst_x86.cpp. > > >> > > >> In nativeInst_x86.cpp, I formatted the expression so I could easily understand it, but you?re right, not everyone does it that way (maybe only me!), so I?ve changed it to > > >> > > >> if (((ubyte_at(0) & NativeTstRegMem::instruction_rex_prefix_mask) == NativeTstRegMem::instruction_rex_prefix && > > >> ubyte_at(1) == NativeTstRegMem::instruction_code_memXregl && > > >> (ubyte_at(2) & NativeTstRegMem::modrm_mask) == NativeTstRegMem::modrm_reg) || > > >> (ubyte_at(0) == NativeTstRegMem::instruction_code_memXregl && > > >> (ubyte_at(1) & NativeTstRegMem::modrm_mask) == NativeTstRegMem::modrm_reg)) { > > >> > > >> In graphKit.cpp, the old code was > > >> > > >> #ifdef ASSERT > > >> case Deoptimization::Action_none: > > >> case Deoptimization::Action_make_not_compilable: > > >> break; > > >> default: > > >> fatal("unknown action %d: %s", action, Deoptimization::trap_action_name(action)); > > >> break; > > >> #endif > > >> > > >> In the non-ASSERT case, the compiler complained about the lack of Action_none, Action_make_not_compilable and default. If the warning had been turned off, the result would have been ?break;? for all three. In the ASSERT case, Action_none and Action_make_not_compilable result in ?break;?, and in the default case ?fatal(); break;? > > >> > > >> The new code is > > >> > > >> case Deoptimization::Action_none: > > >> case Deoptimization::Action_make_not_compilable: > > >> break; > > >> default: > > >> #ifdef ASSERT > > >> fatal("unknown action %d: %s", action, Deoptimization::trap_action_name(action)); > > >> #endif > > >> break; > > >> > > >> The compiler doesn?t complain about Action_none, Action_make_not_compilable or default anymore. In the non-ASSERT case, the result is ?break;? for all three, same as for the old code. In the ASSERT case, Action_none and Action_make_not_compilable result in ?break;?, and in the default case ?fatal(); break;?, again same as for the old code. > > >> > > >> Thanks, > > >> > > >> Paul > > >> > > >> On 6/28/17, 10:36 AM, "jesper.wilhelmsson at oracle.com" wrote: > > >> > > >> Hi Paul, > > >> > > >> Thanks for doing this change! In general everything looks really good, there are a lot of really nice cleanups here. I just have two minor questions/nits: > > >> > > >> * In hotspot/cpu/x86/vm/nativeInst_x86.hpp it seems the expression already have parenthesis around the & operations and the change here is "only" cleaning up the layout of the code which is not a bad thing in it self, but you move the logical operators to the beginning of each line which is a quite different style than the rest of the code in the same function where the operators are at the end of the line. > > >> > > >> * In hotspot/share/vm/opto/graphKit.cpp you moved the #ifdef ASSERT so that Action_none and Action_make_not_compilable are available also when ASSERT is not defined. I don't see this mentioned in your description of the change. Was this change intentional? > > >> > > >> Thanks, > > >> /Jesper > > >> > > >> > > >> > On 27 Jun 2017, at 21:34, Hohensee, Paul wrote: > > >> > > > >> > An attempt at better formatting. > > >> > > > >> > https://bugs.openjdk.java.net/browse/JDK-8182299 > > >> > http://cr.openjdk.java.net/~jwilhelm/8182299/webrev_jdk.00/ > > >> > http://cr.openjdk.java.net/~jwilhelm/8182299/webrev_hotspot.00/ > > >> > > > >> > Jesper has been kind enough to host the webrevs while I get my cr.openjdk.net account set up, and to be the sponsor. > > >> > > > >> > This rfe a combination of enabling disabled clang warnings and getting jdk10 to build on OSX 10 and Xcode 8. At least one enabled warning (delete-non-virtual-dtor) detected what seems to me a real potential bug, with the rest enforcing good code hygiene. > > >> > > > >> > These changes are only in OpenJDK, so I?m looking for a volunteer to make the closed changes. > > >> > > > >> > Thanks, > > >> > > > >> > Paul > > >> > > > >> > > > >> > Here are the jdk file-specific details: > > >> > > > >> > java_md_macosx.c splashscreen_sys.m > > >> > > > >> > Removed objc_registerThreadWithCollector() since it's obsolete and of questionable value in any case. > > >> > > > >> > NSApplicationAWT.m > > >> > > > >> > Use the correct NSEventMask rather than NSUInteger. > > >> > > > >> > jdhuff.c jdphuff.c > > >> > > > >> > Shifting a negative signed value is undefined. > > >> > > > >> > Here are the hotspot notes: > > >> > > > >> > Here are the lists of files affected by enabling a given warning: > > >> > > > >> > switch: all of these are lack of a default clause > > >> > > > >> > c1_LIRAssembler_x86.cpp c1_LIRGenerator_x86.cpp c1_LinearScan_x86.hpp > > >> > jniFastGetField_x86_64.cpp assembler.cpp c1_Canonicalizer.cpp > > >> > c1_GraphBuilder.cpp c1_Instruction.cpp c1_LIR.cpp c1_LIRGenerator.cpp > > >> > c1_LinearScan.cpp c1_ValueStack.hpp c1_ValueType.cpp > > >> > bcEscapeAnalyzer.cpp ciArray.cpp ciEnv.cpp ciInstance.cpp ciMethod.cpp > > >> > ciMethodBlocks.cpp ciMethodData.cpp ciTypeFlow.cpp > > >> > compiledMethod.cpp dependencies.cpp nmethod.cpp compileTask.hpp > > >> > heapRegionType.cpp abstractInterpreter.cpp bytecodes.cpp > > >> > invocationCounter.cpp linkResolver.cpp rewriter.cpp jvmciCompilerToVM.cpp > > >> > jvmciEnv.cpp universe.cpp cpCache.cpp generateOopMap.cpp > > >> > method.cpp methodData.cpp compile.cpp connode.cpp gcm.cpp graphKit.cpp > > >> > ifnode.cpp library_call.cpp memnode.cpp parse1.cpp > > >> > parse2.cpp phaseX.cpp superword.cpp type.cpp vectornode.cpp > > >> > jvmtiClassFileReconstituter.cpp jvmtiEnter.xsl jvmtiEventController.cpp > > >> > jvmtiImpl.cpp jvmtiRedefineClasses.cpp methodComparator.cpp methodHandles.cpp > > >> > advancedThresholdPolicy.cpp reflection.cpp relocator.cpp sharedRuntime.cpp > > >> > simpleThresholdPolicy.cpp writeableFlags.cpp globalDefinitions.hpp > > >> > > > >> > delete-non-virtual-dtor: these may be real latent bugs due to possible failure to execute destructor(s) > > >> > > > >> > decoder_aix.hpp decoder_machO.hpp classLoader.hpp g1RootClosures.hpp > > >> > jvmtiImpl.hpp perfData.hpp decoder.hpp decoder_elf.hpp > > >> > > > >> > dynamic-class-memaccess: obscure use of memcpy > > >> > > > >> > method.cpp > > >> > > > >> > empty-body: ?;? isn?t good enough for clang, it prefers {} > > >> > > > >> > objectMonitor.cpp mallocSiteTable.cpp > > >> > > > >> > format: matches printf format strings against arguments. debug output will be affected by > > >> > incorrect code changes to these. > > >> > > > >> > macroAssembler_x86.cpp os_bsd.cpp os_bsd_x86.cpp ciMethodData.cpp javaClasses.cpp > > >> > debugInfo.cpp logFileOutput.cpp constantPool.cpp jvmtiEnter.xsl jvmtiRedefineClasses.cpp > > >> > safepoint.cpp thread.cpp > > >> > > > >> > logical-op-parentheses: can be tricky to get correct. There are a few very long-winded predicates. > > >> > > > >> > nativeInst_x86.hpp archDesc.cpp output_c.cpp output_h.cpp c1_GraphBuilder.cpp > > >> > c1_LIRGenerator.cpp c1_LinearScan.cpp bcEscapeAnalyzer.cpp ciMethod.cpp > > >> > stackMapTableFormat.hpp compressedStream.cpp dependencies.cpp heapRegion.cpp > > >> > ptrQueue.cpp psPromotionManager.cpp jvmciCompilerToVM.cpp cfgnode.cpp > > >> > chaitin.cpp compile.cpp compile.hpp escape.cpp graphKit.cpp lcm.cpp > > >> > loopTransform.cpp loopnode.cpp loopopts.cpp macro.cpp memnode.cpp > > >> > output.cpp parse1.cpp parseHelper.cpp reg_split.cpp superword.cpp > > >> > superword.hpp jniCheck.cpp jvmtiEventController.cpp arguments.cpp > > >> > javaCalls.cpp sharedRuntime.cpp > > >> > > > >> > parentheses > > >> > > > >> > adlparse.cpp > > >> > > > >> > parentheses-equality > > >> > > > >> > output_c.cpp javaAssertions.cpp gcm.cpp > > >> > > > >> > File-specific details: > > >> > > > >> > GensrcAdlc.gmk CompileJvm.gmk > > >> > Left tautological-compare in place to allow null 'this' pointer checks in methods > > >> > intended to be called from a debugger. > > >> > > > >> > CompileGTest.gmk > > >> > Just an enhanced comment. > > >> > > > >> > MacosxDebuggerLocal.m > > >> > PT_ATTACH has been replaced by PT_ATTACHEXC > > >> > > > >> > ciMethodData.cp > > >> > " 0x%" FORMAT64_MODIFIER "x" reduces to "0x%llx", whereas > > >> > " " INTPTRNZ_FORMAT reduces to "0x%lx", which latter is what clang want. > > >> > > > >> > generateOopMap.cpp > > >> > Refactored duplicate code in print_current_state(). > > >> > > > >> > binaryTreeDictionary.cpp/hpp, hashtable.cpp/hpp > > >> > These provoked ?instantiation of variable required here, > > >> > but no definition is available?. > > >> > > > >> > globalDefinitions_gcc.hpp > > >> > Define FORMAT64_MODIFIER properly for Apple, needed by os.cpp. > > >> > > > >> > globalDefinitions.hpp > > >> > Add INTPTRNZ_FORMAT, needed by ciMethodData.cpp. > > >> > > >> > > >> > > >> > > >> > > > > > > > > From volker.simonis at gmail.com Thu Jul 6 14:11:58 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 6 Jul 2017 16:11:58 +0200 Subject: [10] RFR(XS): 8183534: [TEST] Make detection of compilation mode more robust In-Reply-To: References: Message-ID: On Wed, Jul 5, 2017 at 7:36 PM, Thomas St?fe wrote: > Hi Volker, > > this seems fine, but why did you add the error output? None of the other > vmXX() functions seem not to do this. > I didn't wanted to blow up the change but you're probably right so I've updated all the functions which potentially returned 'null' to also print a stack trace in that case: http://cr.openjdk.java.net/~simonis/webrevs/2017/8183534.v1/ The problem is that without error output, JTreg will simply fail with: java.lang.NullPointerException Error: failed to get JDK properties for /sapjvm_9/bin/java ; exit code 1 which gives you no clue where to search for the problem. With my change you'll get a stack trace before the error which clearly indicates the location of the problem: java.lang.Exception: Can't get 'java.vm.info' property at requires.VMProps.nullWithException(VMProps.java:87) at requires.VMProps.vmCompMode(VMProps.java:133) at requires.VMProps.call(VMProps.java:62) at requires.VMProps.call(VMProps.java:48) at com.sun.javatest.regtest.agent.GetJDKProperties.run(GetJDKProperties.java:66) at com.sun.javatest.regtest.agent.GetJDKProperties.main(GetJDKProperties.java:46) java.lang.NullPointerException Error: failed to get JDK properties for /bas/sapjvm/integ/all/gen/optU/linuxx86_64/output/sapjvm_9/bin/java ; exit code 1 Thank you and best regards, Volker > Kind Regards, Thomas > > On Tue, Jul 4, 2017 at 11:05 AM, Volker Simonis > wrote: >> >> Hi, >> >> can you please review the following trivial change which makes the >> detection of compilation mode for JTreg more robust: >> >> http://cr.openjdk.java.net/~simonis/webrevs/2017/8183534/ >> https://bugs.openjdk.java.net/browse/JDK-8183534 >> >> The compilation mode in JTreg is parsed from the "java.vm.info" system >> property. The current implementation expects that "java.vm.info" >> contains exactly one of the strings "mixed mode", "compiled mode" or >> "interpreted mode" otherwise the detection will fail and JTreg will >> quit with an error. >> >> The detection can be done more robust by searching for the substrings >> "mixed mode", "compiled mode" or "interpreted mode" instead, thus >> allowing OpenJDK builders to store additional information in the >> "java.vm.info" system property. >> >> Thank you and best regards, >> Volker > > From christoph.langer at sap.com Thu Jul 6 14:32:35 2017 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 6 Jul 2017 14:32:35 +0000 Subject: [10] RFR(XS): 8183534: [TEST] Make detection of compilation mode more robust In-Reply-To: References: Message-ID: Hi Volker, looks fine, thanks for doing that. Best regards Christoph > -----Original Message----- > From: Volker Simonis [mailto:volker.simonis at gmail.com] > Sent: Donnerstag, 6. Juli 2017 16:12 > To: Thomas St?fe > Cc: HotSpot Open Source Developers ; > Java Core Libs ; Langer, Christoph > > Subject: Re: [10] RFR(XS): 8183534: [TEST] Make detection of compilation > mode more robust > > On Wed, Jul 5, 2017 at 7:36 PM, Thomas St?fe > wrote: > > Hi Volker, > > > > this seems fine, but why did you add the error output? None of the other > > vmXX() functions seem not to do this. > > > > I didn't wanted to blow up the change but you're probably right so > I've updated all the functions which potentially returned 'null' to > also print a stack trace in that case: > > http://cr.openjdk.java.net/~simonis/webrevs/2017/8183534.v1/ > > The problem is that without error output, JTreg will simply fail with: > > java.lang.NullPointerException > Error: failed to get JDK properties for /sapjvm_9/bin/java ; exit code 1 > > which gives you no clue where to search for the problem. With my > change you'll get a stack trace before the error which clearly > indicates the location of the problem: > > java.lang.Exception: Can't get 'java.vm.info' property > at requires.VMProps.nullWithException(VMProps.java:87) > at requires.VMProps.vmCompMode(VMProps.java:133) > at requires.VMProps.call(VMProps.java:62) > at requires.VMProps.call(VMProps.java:48) > at > com.sun.javatest.regtest.agent.GetJDKProperties.run(GetJDKProperties.jav > a:66) > at > com.sun.javatest.regtest.agent.GetJDKProperties.main(GetJDKProperties.ja > va:46) > java.lang.NullPointerException > Error: failed to get JDK properties for > /bas/sapjvm/integ/all/gen/optU/linuxx86_64/output/sapjvm_9/bin/java ; > exit code 1 > > Thank you and best regards, > Volker > > > Kind Regards, Thomas > > > > On Tue, Jul 4, 2017 at 11:05 AM, Volker Simonis > > > wrote: > >> > >> Hi, > >> > >> can you please review the following trivial change which makes the > >> detection of compilation mode for JTreg more robust: > >> > >> http://cr.openjdk.java.net/~simonis/webrevs/2017/8183534/ > >> https://bugs.openjdk.java.net/browse/JDK-8183534 > >> > >> The compilation mode in JTreg is parsed from the "java.vm.info" system > >> property. The current implementation expects that "java.vm.info" > >> contains exactly one of the strings "mixed mode", "compiled mode" or > >> "interpreted mode" otherwise the detection will fail and JTreg will > >> quit with an error. > >> > >> The detection can be done more robust by searching for the substrings > >> "mixed mode", "compiled mode" or "interpreted mode" instead, thus > >> allowing OpenJDK builders to store additional information in the > >> "java.vm.info" system property. > >> > >> Thank you and best regards, > >> Volker > > > > From tobias.hartmann at oracle.com Thu Jul 6 14:45:43 2017 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Thu, 6 Jul 2017 16:45:43 +0200 Subject: RFR: JDK-8172791 fixing issues with Reserved Stack Area In-Reply-To: <3cd1c352-33fa-96fe-d2f6-a78a25c0543e@oracle.com> References: <3cd1c352-33fa-96fe-d2f6-a78a25c0543e@oracle.com> Message-ID: <250d4d1a-0d68-f5b8-2fe1-c6e286a7cfc1@oracle.com> Hi Fred, the changes look reasonable to me. Please run 'hs-precheckin-comp' before pushing. Thanks, Tobias On 18.04.2017 16:47, Frederic Parain wrote: > Greetings, > > Please review this fix which addresses several issues with the ReservedStackArea implementation: > 1 - the method look_for_reserved_stack_annotated_method() fails to detect in-lined > annotated methods, making the annotation ineffective for in-lined methods > 2 - sometime an assertion failure related to reserved area status occurs after incorrect > restoring of guards pages during a return from runtime > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8172791 > > webrev: > http://cr.openjdk.java.net/~fparain/8172791/webrev.00/index.html > > This fix has been contributed by Andrew Haley. > > Thank you, > > Fred From frederic.parain at oracle.com Thu Jul 6 14:59:21 2017 From: frederic.parain at oracle.com (Frederic Parain) Date: Thu, 6 Jul 2017 10:59:21 -0400 Subject: RFR: JDK-8172791 fixing issues with Reserved Stack Area In-Reply-To: <250d4d1a-0d68-f5b8-2fe1-c6e286a7cfc1@oracle.com> References: <3cd1c352-33fa-96fe-d2f6-a78a25c0543e@oracle.com> <250d4d1a-0d68-f5b8-2fe1-c6e286a7cfc1@oracle.com> Message-ID: <19193cae-f8d6-e2ff-eccd-9506cd988daa@oracle.com> Tobias, Thank you for the review. I'll run 'hs-precheckin-comp' before pushing. Regards, Fred On 07/06/2017 10:45 AM, Tobias Hartmann wrote: > Hi Fred, > > the changes look reasonable to me. > > Please run 'hs-precheckin-comp' before pushing. > > Thanks, > Tobias > > On 18.04.2017 16:47, Frederic Parain wrote: >> Greetings, >> >> Please review this fix which addresses several issues with the ReservedStackArea implementation: >> 1 - the method look_for_reserved_stack_annotated_method() fails to detect in-lined >> annotated methods, making the annotation ineffective for in-lined methods >> 2 - sometime an assertion failure related to reserved area status occurs after incorrect >> restoring of guards pages during a return from runtime >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8172791 >> >> webrev: >> http://cr.openjdk.java.net/~fparain/8172791/webrev.00/index.html >> >> This fix has been contributed by Andrew Haley. >> >> Thank you, >> >> Fred From hohensee at amazon.com Thu Jul 6 16:18:05 2017 From: hohensee at amazon.com (Hohensee, Paul) Date: Thu, 6 Jul 2017 16:18:05 +0000 Subject: RFR(XL): 8182299: Enable disabled clang warnings, build on OSX 10 + Xcode 8 In-Reply-To: References: <27FD0413-52BC-42E6-A5B0-3C92A49A2D6F@amazon.com> <1CBD62A9-9B1B-4B05-AAF9-BE2D52DE8C79@amazon.com> <6B296ABE-66C0-4C32-AC4E-8674BE103514@oracle.com> <3ACE75CF-CEA6-4081-952A-BBA138763582@amazon.com> <435C96F2-417E-4A47-8649-28E34E2908AE@amazon.com> <1d06f5fe-c3f8-f599-58bc-09b2099c819c@oracle.com> <035f2cd3-de68-efe7-d32e-bb6f06c58f93@oracle.com> <901E4351-42D1-44D4-85F6-FBD4C715797A@amazon.com> Message-ID: Thank you for the review. Paul On 7/6/17, 6:09 AM, "Erik Helin" wrote: On 07/03/2017 06:47 PM, Hohensee, Paul wrote: > Fixed heapRegionType.cpp per your comment. > > http://cr.openjdk.java.net/~phh/8182299/webrev_hotspot.02/ > > Jesper says he?s ready to push the closed changes. Any other comments on the open ones? The GC changes looks fine to me, but I can't comment on the runtime/compiler changes. Thanks, Erik > Thanks, > > Paul > > On 7/3/17, 12:58 AM, "Erik Helin" wrote: > > On 06/30/2017 06:30 PM, Hohensee, Paul wrote: > > Thanks for picking up the closed work, Jesper, and thanks for the review, Erik. It?s good to be back working on server Java. ( > > > > Yes, I added the G1RootClosures virtual destructor because of the warning. > > > > In heapRegionType.cpp, I just followed along with the existing funky indentation and made the default clause also have a return because all the other clauses do. Clang would indeed be ok with a default clause with just a break. Also, I?ve used the same idiom in other files, so I?d change those too if I change these. Which would you prefer? I?m not invested in any particular format. > > I don't know about the other files, but given the indentation in > heapRegionType.cpp, I would prefer: > > switch (_tag) { > // ... bunch of case statements > case ArchiveTag: return "ARC"; > default: break; > } > ShouldNotReachHere(); > return NULL; // just to keep some compilers happy > } > > My reason for this is mainly that all the other case statements are on > one line (`case` followed by `return`), so I would prefer to the > `default` case to also be on one line (`default` followed by `break`). > > However, for in HeapRegionType::is_valid, I think your changes follows > the existing style in that method: > > 37 case ArchiveTag: > 38 return true; > 39 default: > 40 return false; > 41 } > > I feel like this is quickly approaching bikeshed territory, but the > above is at least my preference :) > > Thanks, > Erik > > > Thanks, > > > > Paul > > > > On 6/30/17, 3:00 AM, "jesper.wilhelmsson at oracle.com" wrote: > > > > I ran the change through JPRT so at least it builds and runs on all platforms we support there. > > I'm currently fixing our closed code to build on Mac with this change. > > /Jesper > > > > > On 30 Jun 2017, at 10:35, Erik Helin wrote: > > > > > > Hi Paul, > > > > > > thanks for contributing! Please see my comments regarding the GC changes below. > > > > > > On 06/29/2017 03:31 PM, Hohensee, Paul wrote: > > >> I now have access to cr.openjdk.java.net, so the latest webrevs are at > > >> > > >> http://cr.openjdk.java.net/~phh/8182299/webrev_jdk.00/ > > >> http://cr.openjdk.java.net/~phh/8182299/webrev_hotspot.00/ > > > > > > gc/g1/g1RootClosures.cpp: > > > - // Closures to process raw oops in the root set. > > > + virtual ~G1RootClosures() {} > > > + > > > +// Closures to process raw oops in the root set. > > > > > > I assume this is added because there is some warning about having only pure virtual methods but not having a virtual destructor. None of the classes inheriting from G1RootClosures needs a destructor, nor does G1RootClosures itself (it is just an "interface"). So there is no problem with correctness here :) > > > > > > However, I like to have lots of warnings enabled, so for me it is fine to an empty virtual destructor here just to please the compiler. > > > > > > gc/g1/heapRegion.cpp: looks correct > > > > > > gc/g1/heapRegionType.cpp: > > > The indentation seems a bit funky to me here :) Also, have you compiled this on some other platforms? I think the last return is outside of the switch just to, as the comment says, "keep some compilers happy". > > > > > > Would clang be ok with having an empty default clause with just a break? And then fall through to the return outside of the switch? > > > > > > gc/g1/ptrQueue.cpp: looks correct > > > > > > gc/parallel/psPromotionManager.cpp: looks correct > > > > > > Thanks, > > > Erik > > > > > >> On 6/28/17, 3:50 PM, "hotspot-dev on behalf of Hohensee, Paul" wrote: > > >> > > >> Thanks for the review, Jesper. > > >> > > >> New webrev sent, has only a change to nativeInst_x86.cpp. > > >> > > >> In nativeInst_x86.cpp, I formatted the expression so I could easily understand it, but you?re right, not everyone does it that way (maybe only me!), so I?ve changed it to > > >> > > >> if (((ubyte_at(0) & NativeTstRegMem::instruction_rex_prefix_mask) == NativeTstRegMem::instruction_rex_prefix && > > >> ubyte_at(1) == NativeTstRegMem::instruction_code_memXregl && > > >> (ubyte_at(2) & NativeTstRegMem::modrm_mask) == NativeTstRegMem::modrm_reg) || > > >> (ubyte_at(0) == NativeTstRegMem::instruction_code_memXregl && > > >> (ubyte_at(1) & NativeTstRegMem::modrm_mask) == NativeTstRegMem::modrm_reg)) { > > >> > > >> In graphKit.cpp, the old code was > > >> > > >> #ifdef ASSERT > > >> case Deoptimization::Action_none: > > >> case Deoptimization::Action_make_not_compilable: > > >> break; > > >> default: > > >> fatal("unknown action %d: %s", action, Deoptimization::trap_action_name(action)); > > >> break; > > >> #endif > > >> > > >> In the non-ASSERT case, the compiler complained about the lack of Action_none, Action_make_not_compilable and default. If the warning had been turned off, the result would have been ?break;? for all three. In the ASSERT case, Action_none and Action_make_not_compilable result in ?break;?, and in the default case ?fatal(); break;? > > >> > > >> The new code is > > >> > > >> case Deoptimization::Action_none: > > >> case Deoptimization::Action_make_not_compilable: > > >> break; > > >> default: > > >> #ifdef ASSERT > > >> fatal("unknown action %d: %s", action, Deoptimization::trap_action_name(action)); > > >> #endif > > >> break; > > >> > > >> The compiler doesn?t complain about Action_none, Action_make_not_compilable or default anymore. In the non-ASSERT case, the result is ?break;? for all three, same as for the old code. In the ASSERT case, Action_none and Action_make_not_compilable result in ?break;?, and in the default case ?fatal(); break;?, again same as for the old code. > > >> > > >> Thanks, > > >> > > >> Paul > > >> > > >> On 6/28/17, 10:36 AM, "jesper.wilhelmsson at oracle.com" wrote: > > >> > > >> Hi Paul, > > >> > > >> Thanks for doing this change! In general everything looks really good, there are a lot of really nice cleanups here. I just have two minor questions/nits: > > >> > > >> * In hotspot/cpu/x86/vm/nativeInst_x86.hpp it seems the expression already have parenthesis around the & operations and the change here is "only" cleaning up the layout of the code which is not a bad thing in it self, but you move the logical operators to the beginning of each line which is a quite different style than the rest of the code in the same function where the operators are at the end of the line. > > >> > > >> * In hotspot/share/vm/opto/graphKit.cpp you moved the #ifdef ASSERT so that Action_none and Action_make_not_compilable are available also when ASSERT is not defined. I don't see this mentioned in your description of the change. Was this change intentional? > > >> > > >> Thanks, > > >> /Jesper > > >> > > >> > > >> > On 27 Jun 2017, at 21:34, Hohensee, Paul wrote: > > >> > > > >> > An attempt at better formatting. > > >> > > > >> > https://bugs.openjdk.java.net/browse/JDK-8182299 > > >> > http://cr.openjdk.java.net/~jwilhelm/8182299/webrev_jdk.00/ > > >> > http://cr.openjdk.java.net/~jwilhelm/8182299/webrev_hotspot.00/ > > >> > > > >> > Jesper has been kind enough to host the webrevs while I get my cr.openjdk.net account set up, and to be the sponsor. > > >> > > > >> > This rfe a combination of enabling disabled clang warnings and getting jdk10 to build on OSX 10 and Xcode 8. At least one enabled warning (delete-non-virtual-dtor) detected what seems to me a real potential bug, with the rest enforcing good code hygiene. > > >> > > > >> > These changes are only in OpenJDK, so I?m looking for a volunteer to make the closed changes. > > >> > > > >> > Thanks, > > >> > > > >> > Paul > > >> > > > >> > > > >> > Here are the jdk file-specific details: > > >> > > > >> > java_md_macosx.c splashscreen_sys.m > > >> > > > >> > Removed objc_registerThreadWithCollector() since it's obsolete and of questionable value in any case. > > >> > > > >> > NSApplicationAWT.m > > >> > > > >> > Use the correct NSEventMask rather than NSUInteger. > > >> > > > >> > jdhuff.c jdphuff.c > > >> > > > >> > Shifting a negative signed value is undefined. > > >> > > > >> > Here are the hotspot notes: > > >> > > > >> > Here are the lists of files affected by enabling a given warning: > > >> > > > >> > switch: all of these are lack of a default clause > > >> > > > >> > c1_LIRAssembler_x86.cpp c1_LIRGenerator_x86.cpp c1_LinearScan_x86.hpp > > >> > jniFastGetField_x86_64.cpp assembler.cpp c1_Canonicalizer.cpp > > >> > c1_GraphBuilder.cpp c1_Instruction.cpp c1_LIR.cpp c1_LIRGenerator.cpp > > >> > c1_LinearScan.cpp c1_ValueStack.hpp c1_ValueType.cpp > > >> > bcEscapeAnalyzer.cpp ciArray.cpp ciEnv.cpp ciInstance.cpp ciMethod.cpp > > >> > ciMethodBlocks.cpp ciMethodData.cpp ciTypeFlow.cpp > > >> > compiledMethod.cpp dependencies.cpp nmethod.cpp compileTask.hpp > > >> > heapRegionType.cpp abstractInterpreter.cpp bytecodes.cpp > > >> > invocationCounter.cpp linkResolver.cpp rewriter.cpp jvmciCompilerToVM.cpp > > >> > jvmciEnv.cpp universe.cpp cpCache.cpp generateOopMap.cpp > > >> > method.cpp methodData.cpp compile.cpp connode.cpp gcm.cpp graphKit.cpp > > >> > ifnode.cpp library_call.cpp memnode.cpp parse1.cpp > > >> > parse2.cpp phaseX.cpp superword.cpp type.cpp vectornode.cpp > > >> > jvmtiClassFileReconstituter.cpp jvmtiEnter.xsl jvmtiEventController.cpp > > >> > jvmtiImpl.cpp jvmtiRedefineClasses.cpp methodComparator.cpp methodHandles.cpp > > >> > advancedThresholdPolicy.cpp reflection.cpp relocator.cpp sharedRuntime.cpp > > >> > simpleThresholdPolicy.cpp writeableFlags.cpp globalDefinitions.hpp > > >> > > > >> > delete-non-virtual-dtor: these may be real latent bugs due to possible failure to execute destructor(s) > > >> > > > >> > decoder_aix.hpp decoder_machO.hpp classLoader.hpp g1RootClosures.hpp > > >> > jvmtiImpl.hpp perfData.hpp decoder.hpp decoder_elf.hpp > > >> > > > >> > dynamic-class-memaccess: obscure use of memcpy > > >> > > > >> > method.cpp > > >> > > > >> > empty-body: ?;? isn?t good enough for clang, it prefers {} > > >> > > > >> > objectMonitor.cpp mallocSiteTable.cpp > > >> > > > >> > format: matches printf format strings against arguments. debug output will be affected by > > >> > incorrect code changes to these. > > >> > > > >> > macroAssembler_x86.cpp os_bsd.cpp os_bsd_x86.cpp ciMethodData.cpp javaClasses.cpp > > >> > debugInfo.cpp logFileOutput.cpp constantPool.cpp jvmtiEnter.xsl jvmtiRedefineClasses.cpp > > >> > safepoint.cpp thread.cpp > > >> > > > >> > logical-op-parentheses: can be tricky to get correct. There are a few very long-winded predicates. > > >> > > > >> > nativeInst_x86.hpp archDesc.cpp output_c.cpp output_h.cpp c1_GraphBuilder.cpp > > >> > c1_LIRGenerator.cpp c1_LinearScan.cpp bcEscapeAnalyzer.cpp ciMethod.cpp > > >> > stackMapTableFormat.hpp compressedStream.cpp dependencies.cpp heapRegion.cpp > > >> > ptrQueue.cpp psPromotionManager.cpp jvmciCompilerToVM.cpp cfgnode.cpp > > >> > chaitin.cpp compile.cpp compile.hpp escape.cpp graphKit.cpp lcm.cpp > > >> > loopTransform.cpp loopnode.cpp loopopts.cpp macro.cpp memnode.cpp > > >> > output.cpp parse1.cpp parseHelper.cpp reg_split.cpp superword.cpp > > >> > superword.hpp jniCheck.cpp jvmtiEventController.cpp arguments.cpp > > >> > javaCalls.cpp sharedRuntime.cpp > > >> > > > >> > parentheses > > >> > > > >> > adlparse.cpp > > >> > > > >> > parentheses-equality > > >> > > > >> > output_c.cpp javaAssertions.cpp gcm.cpp > > >> > > > >> > File-specific details: > > >> > > > >> > GensrcAdlc.gmk CompileJvm.gmk > > >> > Left tautological-compare in place to allow null 'this' pointer checks in methods > > >> > intended to be called from a debugger. > > >> > > > >> > CompileGTest.gmk > > >> > Just an enhanced comment. > > >> > > > >> > MacosxDebuggerLocal.m > > >> > PT_ATTACH has been replaced by PT_ATTACHEXC > > >> > > > >> > ciMethodData.cp > > >> > " 0x%" FORMAT64_MODIFIER "x" reduces to "0x%llx", whereas > > >> > " " INTPTRNZ_FORMAT reduces to "0x%lx", which latter is what clang want. > > >> > > > >> > generateOopMap.cpp > > >> > Refactored duplicate code in print_current_state(). > > >> > > > >> > binaryTreeDictionary.cpp/hpp, hashtable.cpp/hpp > > >> > These provoked ?instantiation of variable required here, > > >> > but no definition is available?. > > >> > > > >> > globalDefinitions_gcc.hpp > > >> > Define FORMAT64_MODIFIER properly for Apple, needed by os.cpp. > > >> > > > >> > globalDefinitions.hpp > > >> > Add INTPTRNZ_FORMAT, needed by ciMethodData.cpp. > > >> > > >> > > >> > > >> > > >> > > > > > > > > From hohensee at amazon.com Thu Jul 6 16:20:45 2017 From: hohensee at amazon.com (Hohensee, Paul) Date: Thu, 6 Jul 2017 16:20:45 +0000 Subject: RFR(XL): 8182299: Enable disabled clang warnings, build on OSX 10 + Xcode 8 In-Reply-To: References: <27FD0413-52BC-42E6-A5B0-3C92A49A2D6F@amazon.com> <1CBD62A9-9B1B-4B05-AAF9-BE2D52DE8C79@amazon.com> <6B296ABE-66C0-4C32-AC4E-8674BE103514@oracle.com> <3ACE75CF-CEA6-4081-952A-BBA138763582@amazon.com> <435C96F2-417E-4A47-8649-28E34E2908AE@amazon.com> <1d06f5fe-c3f8-f599-58bc-09b2099c819c@oracle.com> <035f2cd3-de68-efe7-d32e-bb6f06c58f93@oracle.com> <901E4351-42D1-44D4-85F6-FBD4C715797A@amazon.com> Message-ID: Thanks, Vladimir, it?s good to be back. Jesper?s pushed the change, but your suggestion wasn?t included. I can file another rfe, but that seems like overkill. Let?s just keep it in mind for the next time the file is touched or another cleanup is done. Paul On 7/6/17, 6:27 AM, "Vladimir Ivanov" wrote: Paul, > http://cr.openjdk.java.net/~phh/8182299/webrev_hotspot.02/ Compiler changes look good. Just a small refactoring suggestion while you on it: src/cpu/x86/vm/macroAssembler_x86.cpp: 0x%016lx can be replaced with INTPTR_FORMAT. Thanks for taking care of the cleanup and welcome back! :-) Best regards, Vladimir Ivanov > > Jesper says he?s ready to push the closed changes. Any other comments on the open ones? > > Thanks, > > Paul > > On 7/3/17, 12:58 AM, "Erik Helin" wrote: > > On 06/30/2017 06:30 PM, Hohensee, Paul wrote: > > Thanks for picking up the closed work, Jesper, and thanks for the review, Erik. It?s good to be back working on server Java. ( > > > > Yes, I added the G1RootClosures virtual destructor because of the warning. > > > > In heapRegionType.cpp, I just followed along with the existing funky indentation and made the default clause also have a return because all the other clauses do. Clang would indeed be ok with a default clause with just a break. Also, I?ve used the same idiom in other files, so I?d change those too if I change these. Which would you prefer? I?m not invested in any particular format. > > I don't know about the other files, but given the indentation in > heapRegionType.cpp, I would prefer: > > switch (_tag) { > // ... bunch of case statements > case ArchiveTag: return "ARC"; > default: break; > } > ShouldNotReachHere(); > return NULL; // just to keep some compilers happy > } > > My reason for this is mainly that all the other case statements are on > one line (`case` followed by `return`), so I would prefer to the > `default` case to also be on one line (`default` followed by `break`). > > However, for in HeapRegionType::is_valid, I think your changes follows > the existing style in that method: > > 37 case ArchiveTag: > 38 return true; > 39 default: > 40 return false; > 41 } > > I feel like this is quickly approaching bikeshed territory, but the > above is at least my preference :) > > Thanks, > Erik > > > Thanks, > > > > Paul > > > > On 6/30/17, 3:00 AM, "jesper.wilhelmsson at oracle.com" wrote: > > > > I ran the change through JPRT so at least it builds and runs on all platforms we support there. > > I'm currently fixing our closed code to build on Mac with this change. > > /Jesper > > > > > On 30 Jun 2017, at 10:35, Erik Helin wrote: > > > > > > Hi Paul, > > > > > > thanks for contributing! Please see my comments regarding the GC changes below. > > > > > > On 06/29/2017 03:31 PM, Hohensee, Paul wrote: > > >> I now have access to cr.openjdk.java.net, so the latest webrevs are at > > >> > > >> http://cr.openjdk.java.net/~phh/8182299/webrev_jdk.00/ > > >> http://cr.openjdk.java.net/~phh/8182299/webrev_hotspot.00/ > > > > > > gc/g1/g1RootClosures.cpp: > > > - // Closures to process raw oops in the root set. > > > + virtual ~G1RootClosures() {} > > > + > > > +// Closures to process raw oops in the root set. > > > > > > I assume this is added because there is some warning about having only pure virtual methods but not having a virtual destructor. None of the classes inheriting from G1RootClosures needs a destructor, nor does G1RootClosures itself (it is just an "interface"). So there is no problem with correctness here :) > > > > > > However, I like to have lots of warnings enabled, so for me it is fine to an empty virtual destructor here just to please the compiler. > > > > > > gc/g1/heapRegion.cpp: looks correct > > > > > > gc/g1/heapRegionType.cpp: > > > The indentation seems a bit funky to me here :) Also, have you compiled this on some other platforms? I think the last return is outside of the switch just to, as the comment says, "keep some compilers happy". > > > > > > Would clang be ok with having an empty default clause with just a break? And then fall through to the return outside of the switch? > > > > > > gc/g1/ptrQueue.cpp: looks correct > > > > > > gc/parallel/psPromotionManager.cpp: looks correct > > > > > > Thanks, > > > Erik > > > > > >> On 6/28/17, 3:50 PM, "hotspot-dev on behalf of Hohensee, Paul" wrote: > > >> > > >> Thanks for the review, Jesper. > > >> > > >> New webrev sent, has only a change to nativeInst_x86.cpp. > > >> > > >> In nativeInst_x86.cpp, I formatted the expression so I could easily understand it, but you?re right, not everyone does it that way (maybe only me!), so I?ve changed it to > > >> > > >> if (((ubyte_at(0) & NativeTstRegMem::instruction_rex_prefix_mask) == NativeTstRegMem::instruction_rex_prefix && > > >> ubyte_at(1) == NativeTstRegMem::instruction_code_memXregl && > > >> (ubyte_at(2) & NativeTstRegMem::modrm_mask) == NativeTstRegMem::modrm_reg) || > > >> (ubyte_at(0) == NativeTstRegMem::instruction_code_memXregl && > > >> (ubyte_at(1) & NativeTstRegMem::modrm_mask) == NativeTstRegMem::modrm_reg)) { > > >> > > >> In graphKit.cpp, the old code was > > >> > > >> #ifdef ASSERT > > >> case Deoptimization::Action_none: > > >> case Deoptimization::Action_make_not_compilable: > > >> break; > > >> default: > > >> fatal("unknown action %d: %s", action, Deoptimization::trap_action_name(action)); > > >> break; > > >> #endif > > >> > > >> In the non-ASSERT case, the compiler complained about the lack of Action_none, Action_make_not_compilable and default. If the warning had been turned off, the result would have been ?break;? for all three. In the ASSERT case, Action_none and Action_make_not_compilable result in ?break;?, and in the default case ?fatal(); break;? > > >> > > >> The new code is > > >> > > >> case Deoptimization::Action_none: > > >> case Deoptimization::Action_make_not_compilable: > > >> break; > > >> default: > > >> #ifdef ASSERT > > >> fatal("unknown action %d: %s", action, Deoptimization::trap_action_name(action)); > > >> #endif > > >> break; > > >> > > >> The compiler doesn?t complain about Action_none, Action_make_not_compilable or default anymore. In the non-ASSERT case, the result is ?break;? for all three, same as for the old code. In the ASSERT case, Action_none and Action_make_not_compilable result in ?break;?, and in the default case ?fatal(); break;?, again same as for the old code. > > >> > > >> Thanks, > > >> > > >> Paul > > >> > > >> On 6/28/17, 10:36 AM, "jesper.wilhelmsson at oracle.com" wrote: > > >> > > >> Hi Paul, > > >> > > >> Thanks for doing this change! In general everything looks really good, there are a lot of really nice cleanups here. I just have two minor questions/nits: > > >> > > >> * In hotspot/cpu/x86/vm/nativeInst_x86.hpp it seems the expression already have parenthesis around the & operations and the change here is "only" cleaning up the layout of the code which is not a bad thing in it self, but you move the logical operators to the beginning of each line which is a quite different style than the rest of the code in the same function where the operators are at the end of the line. > > >> > > >> * In hotspot/share/vm/opto/graphKit.cpp you moved the #ifdef ASSERT so that Action_none and Action_make_not_compilable are available also when ASSERT is not defined. I don't see this mentioned in your description of the change. Was this change intentional? > > >> > > >> Thanks, > > >> /Jesper > > >> > > >> > > >> > On 27 Jun 2017, at 21:34, Hohensee, Paul wrote: > > >> > > > >> > An attempt at better formatting. > > >> > > > >> > https://bugs.openjdk.java.net/browse/JDK-8182299 > > >> > http://cr.openjdk.java.net/~jwilhelm/8182299/webrev_jdk.00/ > > >> > http://cr.openjdk.java.net/~jwilhelm/8182299/webrev_hotspot.00/ > > >> > > > >> > Jesper has been kind enough to host the webrevs while I get my cr.openjdk.net account set up, and to be the sponsor. > > >> > > > >> > This rfe a combination of enabling disabled clang warnings and getting jdk10 to build on OSX 10 and Xcode 8. At least one enabled warning (delete-non-virtual-dtor) detected what seems to me a real potential bug, with the rest enforcing good code hygiene. > > >> > > > >> > These changes are only in OpenJDK, so I?m looking for a volunteer to make the closed changes. > > >> > > > >> > Thanks, > > >> > > > >> > Paul > > >> > > > >> > > > >> > Here are the jdk file-specific details: > > >> > > > >> > java_md_macosx.c splashscreen_sys.m > > >> > > > >> > Removed objc_registerThreadWithCollector() since it's obsolete and of questionable value in any case. > > >> > > > >> > NSApplicationAWT.m > > >> > > > >> > Use the correct NSEventMask rather than NSUInteger. > > >> > > > >> > jdhuff.c jdphuff.c > > >> > > > >> > Shifting a negative signed value is undefined. > > >> > > > >> > Here are the hotspot notes: > > >> > > > >> > Here are the lists of files affected by enabling a given warning: > > >> > > > >> > switch: all of these are lack of a default clause > > >> > > > >> > c1_LIRAssembler_x86.cpp c1_LIRGenerator_x86.cpp c1_LinearScan_x86.hpp > > >> > jniFastGetField_x86_64.cpp assembler.cpp c1_Canonicalizer.cpp > > >> > c1_GraphBuilder.cpp c1_Instruction.cpp c1_LIR.cpp c1_LIRGenerator.cpp > > >> > c1_LinearScan.cpp c1_ValueStack.hpp c1_ValueType.cpp > > >> > bcEscapeAnalyzer.cpp ciArray.cpp ciEnv.cpp ciInstance.cpp ciMethod.cpp > > >> > ciMethodBlocks.cpp ciMethodData.cpp ciTypeFlow.cpp > > >> > compiledMethod.cpp dependencies.cpp nmethod.cpp compileTask.hpp > > >> > heapRegionType.cpp abstractInterpreter.cpp bytecodes.cpp > > >> > invocationCounter.cpp linkResolver.cpp rewriter.cpp jvmciCompilerToVM.cpp > > >> > jvmciEnv.cpp universe.cpp cpCache.cpp generateOopMap.cpp > > >> > method.cpp methodData.cpp compile.cpp connode.cpp gcm.cpp graphKit.cpp > > >> > ifnode.cpp library_call.cpp memnode.cpp parse1.cpp > > >> > parse2.cpp phaseX.cpp superword.cpp type.cpp vectornode.cpp > > >> > jvmtiClassFileReconstituter.cpp jvmtiEnter.xsl jvmtiEventController.cpp > > >> > jvmtiImpl.cpp jvmtiRedefineClasses.cpp methodComparator.cpp methodHandles.cpp > > >> > advancedThresholdPolicy.cpp reflection.cpp relocator.cpp sharedRuntime.cpp > > >> > simpleThresholdPolicy.cpp writeableFlags.cpp globalDefinitions.hpp > > >> > > > >> > delete-non-virtual-dtor: these may be real latent bugs due to possible failure to execute destructor(s) > > >> > > > >> > decoder_aix.hpp decoder_machO.hpp classLoader.hpp g1RootClosures.hpp > > >> > jvmtiImpl.hpp perfData.hpp decoder.hpp decoder_elf.hpp > > >> > > > >> > dynamic-class-memaccess: obscure use of memcpy > > >> > > > >> > method.cpp > > >> > > > >> > empty-body: ?;? isn?t good enough for clang, it prefers {} > > >> > > > >> > objectMonitor.cpp mallocSiteTable.cpp > > >> > > > >> > format: matches printf format strings against arguments. debug output will be affected by > > >> > incorrect code changes to these. > > >> > > > >> > macroAssembler_x86.cpp os_bsd.cpp os_bsd_x86.cpp ciMethodData.cpp javaClasses.cpp > > >> > debugInfo.cpp logFileOutput.cpp constantPool.cpp jvmtiEnter.xsl jvmtiRedefineClasses.cpp > > >> > safepoint.cpp thread.cpp > > >> > > > >> > logical-op-parentheses: can be tricky to get correct. There are a few very long-winded predicates. > > >> > > > >> > nativeInst_x86.hpp archDesc.cpp output_c.cpp output_h.cpp c1_GraphBuilder.cpp > > >> > c1_LIRGenerator.cpp c1_LinearScan.cpp bcEscapeAnalyzer.cpp ciMethod.cpp > > >> > stackMapTableFormat.hpp compressedStream.cpp dependencies.cpp heapRegion.cpp > > >> > ptrQueue.cpp psPromotionManager.cpp jvmciCompilerToVM.cpp cfgnode.cpp > > >> > chaitin.cpp compile.cpp compile.hpp escape.cpp graphKit.cpp lcm.cpp > > >> > loopTransform.cpp loopnode.cpp loopopts.cpp macro.cpp memnode.cpp > > >> > output.cpp parse1.cpp parseHelper.cpp reg_split.cpp superword.cpp > > >> > superword.hpp jniCheck.cpp jvmtiEventController.cpp arguments.cpp > > >> > javaCalls.cpp sharedRuntime.cpp > > >> > > > >> > parentheses > > >> > > > >> > adlparse.cpp > > >> > > > >> > parentheses-equality > > >> > > > >> > output_c.cpp javaAssertions.cpp gcm.cpp > > >> > > > >> > File-specific details: > > >> > > > >> > GensrcAdlc.gmk CompileJvm.gmk > > >> > Left tautological-compare in place to allow null 'this' pointer checks in methods > > >> > intended to be called from a debugger. > > >> > > > >> > CompileGTest.gmk > > >> > Just an enhanced comment. > > >> > > > >> > MacosxDebuggerLocal.m > > >> > PT_ATTACH has been replaced by PT_ATTACHEXC > > >> > > > >> > ciMethodData.cp > > >> > " 0x%" FORMAT64_MODIFIER "x" reduces to "0x%llx", whereas > > >> > " " INTPTRNZ_FORMAT reduces to "0x%lx", which latter is what clang want. > > >> > > > >> > generateOopMap.cpp > > >> > Refactored duplicate code in print_current_state(). > > >> > > > >> > binaryTreeDictionary.cpp/hpp, hashtable.cpp/hpp > > >> > These provoked ?instantiation of variable required here, > > >> > but no definition is available?. > > >> > > > >> > globalDefinitions_gcc.hpp > > >> > Define FORMAT64_MODIFIER properly for Apple, needed by os.cpp. > > >> > > > >> > globalDefinitions.hpp > > >> > Add INTPTRNZ_FORMAT, needed by ciMethodData.cpp. > > >> > > >> > > >> > > >> > > >> > > > > > > > > From kim.barrett at oracle.com Thu Jul 6 18:55:32 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 6 Jul 2017 14:55:32 -0400 Subject: RFR (M): 8183927: Hotspot needs C++ type_traits metaprogramming utilities In-Reply-To: <595E3647.1030101@oracle.com> References: <595E3647.1030101@oracle.com> Message-ID: > On Jul 6, 2017, at 9:08 AM, Erik ?sterlund wrote: > > Hi, > > I'm sure we have all had moments when we wished we could use the C++11 type_traits header and the nice metaprogramming tools in there, but then suddenly remembered that we can not, because we are building libjvm.so with C++03 and are still arguing whether we can use namespaces. > Well.. welcome to the future. I am delighted to announce that now at least some of those tools will become available. > > A bunch of C++11 compliant metaprogramming tools that can be used already in C++03 have been developed. > They have been developed with the most conservative and portable C++03 code possible so that even exotic compilers can support it. > Special thanks go to Kim Barret whom I have had many long conversations about many paragraphs of the C++ standard throughout this work. It has helped a lot and improved the code. > > While this change introduces the traits without using it in the code (more than from gtest), it will become used in already prepared patches for refactoring Atomic and OrderAccess to use templates rather than hand picked types like the JNI-specific jint etc types. That serves as the bottom layer of the upcoming GC barrier interface that also requires these utilities. Since they are rather large changes, I hope it is okay that I separate out the metaprogramming utilities. I argue that they are of value in general in hotspot and will make our lives easier. > > All metaprogramming tools have been placed in a new hotspot/src/share/vm/metaprogramming directory. > Tests have been written and placed in a new test/native/metaprogramming directory. > > A bug ID can be found here: > https://bugs.openjdk.java.net/browse/JDK-8183927 > > A webrev with code can be found here: > http://cr.openjdk.java.net/~eosterlund/8183927/webrev.00/ Looks good. From robbin.ehn at oracle.com Thu Jul 6 21:11:02 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Thu, 6 Jul 2017 23:11:02 +0200 Subject: RFR (M): 8183927: Hotspot needs C++ type_traits metaprogramming utilities In-Reply-To: References: <595E3647.1030101@oracle.com> Message-ID: <67b161a1-70be-852e-1f99-e37b58c51bd6@oracle.com> On 07/06/2017 08:55 PM, Kim Barrett wrote: >> On Jul 6, 2017, at 9:08 AM, Erik ?sterlund wrote: >> >> Hi, >> >> I'm sure we have all had moments when we wished we could use the C++11 type_traits header and the nice metaprogramming tools in there, but then suddenly remembered that we can not, because we are building libjvm.so with C++03 and are still arguing whether we can use namespaces. >> Well.. welcome to the future. I am delighted to announce that now at least some of those tools will become available. >> >> A bunch of C++11 compliant metaprogramming tools that can be used already in C++03 have been developed. >> They have been developed with the most conservative and portable C++03 code possible so that even exotic compilers can support it. >> Special thanks go to Kim Barret whom I have had many long conversations about many paragraphs of the C++ standard throughout this work. It has helped a lot and improved the code. >> >> While this change introduces the traits without using it in the code (more than from gtest), it will become used in already prepared patches for refactoring Atomic and OrderAccess to use templates rather than hand picked types like the JNI-specific jint etc types. That serves as the bottom layer of the upcoming GC barrier interface that also requires these utilities. Since they are rather large changes, I hope it is okay that I separate out the metaprogramming utilities. I argue that they are of value in general in hotspot and will make our lives easier. >> >> All metaprogramming tools have been placed in a new hotspot/src/share/vm/metaprogramming directory. >> Tests have been written and placed in a new test/native/metaprogramming directory. >> >> A bug ID can be found here: >> https://bugs.openjdk.java.net/browse/JDK-8183927 >> >> A webrev with code can be found here: >> http://cr.openjdk.java.net/~eosterlund/8183927/webrev.00/ > > Looks good. > +1, ship it! /Robbin From erik.osterlund at oracle.com Fri Jul 7 07:40:12 2017 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 7 Jul 2017 09:40:12 +0200 Subject: RFR (M): 8183927: Hotspot needs C++ type_traits metaprogramming utilities In-Reply-To: References: <595E3647.1030101@oracle.com> Message-ID: <595F3ADC.2020109@oracle.com> Hi Kim, Thanks for the review. /Erik On 2017-07-06 20:55, Kim Barrett wrote: >> On Jul 6, 2017, at 9:08 AM, Erik ?sterlund wrote: >> >> Hi, >> >> I'm sure we have all had moments when we wished we could use the C++11 type_traits header and the nice metaprogramming tools in there, but then suddenly remembered that we can not, because we are building libjvm.so with C++03 and are still arguing whether we can use namespaces. >> Well.. welcome to the future. I am delighted to announce that now at least some of those tools will become available. >> >> A bunch of C++11 compliant metaprogramming tools that can be used already in C++03 have been developed. >> They have been developed with the most conservative and portable C++03 code possible so that even exotic compilers can support it. >> Special thanks go to Kim Barret whom I have had many long conversations about many paragraphs of the C++ standard throughout this work. It has helped a lot and improved the code. >> >> While this change introduces the traits without using it in the code (more than from gtest), it will become used in already prepared patches for refactoring Atomic and OrderAccess to use templates rather than hand picked types like the JNI-specific jint etc types. That serves as the bottom layer of the upcoming GC barrier interface that also requires these utilities. Since they are rather large changes, I hope it is okay that I separate out the metaprogramming utilities. I argue that they are of value in general in hotspot and will make our lives easier. >> >> All metaprogramming tools have been placed in a new hotspot/src/share/vm/metaprogramming directory. >> Tests have been written and placed in a new test/native/metaprogramming directory. >> >> A bug ID can be found here: >> https://bugs.openjdk.java.net/browse/JDK-8183927 >> >> A webrev with code can be found here: >> http://cr.openjdk.java.net/~eosterlund/8183927/webrev.00/ > Looks good. > From erik.osterlund at oracle.com Fri Jul 7 07:40:47 2017 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 7 Jul 2017 09:40:47 +0200 Subject: RFR (M): 8183927: Hotspot needs C++ type_traits metaprogramming utilities In-Reply-To: <67b161a1-70be-852e-1f99-e37b58c51bd6@oracle.com> References: <595E3647.1030101@oracle.com> <67b161a1-70be-852e-1f99-e37b58c51bd6@oracle.com> Message-ID: <595F3AFF.6060307@oracle.com> Hi Robbin, Thanks for the review. /Erik On 2017-07-06 23:11, Robbin Ehn wrote: > On 07/06/2017 08:55 PM, Kim Barrett wrote: >>> On Jul 6, 2017, at 9:08 AM, Erik ?sterlund >>> wrote: >>> >>> Hi, >>> >>> I'm sure we have all had moments when we wished we could use the >>> C++11 type_traits header and the nice metaprogramming tools in >>> there, but then suddenly remembered that we can not, because we are >>> building libjvm.so with C++03 and are still arguing whether we can >>> use namespaces. >>> Well.. welcome to the future. I am delighted to announce that now at >>> least some of those tools will become available. >>> >>> A bunch of C++11 compliant metaprogramming tools that can be used >>> already in C++03 have been developed. >>> They have been developed with the most conservative and portable >>> C++03 code possible so that even exotic compilers can support it. >>> Special thanks go to Kim Barret whom I have had many long >>> conversations about many paragraphs of the C++ standard throughout >>> this work. It has helped a lot and improved the code. >>> >>> While this change introduces the traits without using it in the code >>> (more than from gtest), it will become used in already prepared >>> patches for refactoring Atomic and OrderAccess to use templates >>> rather than hand picked types like the JNI-specific jint etc types. >>> That serves as the bottom layer of the upcoming GC barrier interface >>> that also requires these utilities. Since they are rather large >>> changes, I hope it is okay that I separate out the metaprogramming >>> utilities. I argue that they are of value in general in hotspot and >>> will make our lives easier. >>> >>> All metaprogramming tools have been placed in a new >>> hotspot/src/share/vm/metaprogramming directory. >>> Tests have been written and placed in a new >>> test/native/metaprogramming directory. >>> >>> A bug ID can be found here: >>> https://bugs.openjdk.java.net/browse/JDK-8183927 >>> >>> A webrev with code can be found here: >>> http://cr.openjdk.java.net/~eosterlund/8183927/webrev.00/ >> >> Looks good. >> > > +1, ship it! > > /Robbin From erik.helin at oracle.com Fri Jul 7 13:27:43 2017 From: erik.helin at oracle.com (Erik Helin) Date: Fri, 7 Jul 2017 15:27:43 +0200 Subject: RFR(XL): 8181917: Refactor UL LogStreams to avoid using resource area In-Reply-To: References: <337330d2-512c-bc77-5a55-c05d33e376e5@oracle.com> Message-ID: <3359f071-75c0-595e-e919-35210598bb15@oracle.com> On 07/04/2017 12:05 PM, Thomas St?fe wrote: > On 06/30/2017 08:32 PM, Thomas St?fe wrote: > > - should you add declarations of delete as well to prevent > someone from > calling delete on LogStream pointer? > > > Tried this, but I think I ran into linker errors. I do not think > it is > needed, though. > > > Hmm, that seems strange, a linker error would/could indicate that > some code _is_ calling delete on a LogStream*... > > > Fair enough. I retried and got a ton of unhelpful linkage errors on > Windows, basically almost every file has a reference to > LogStream::operator delete(). Retried on Linux, and gcc refuses to > compile, always with a variant of the following: > > > In file included from > /priv/d031900/openjdk/jdk10-hs/source/hotspot/test/native/logging/logTestUtils.inline.hpp:26:0, > from > /priv/d031900/openjdk/jdk10-hs/source/hotspot/test/native/logging/test_logTagSetDescriptions.cpp:25: > /priv/d031900/openjdk/jdk10-hs/source/hotspot/src/share/vm/logging/logStream.hpp: > In destructor ?virtual LogStreamTemplate<(LogLevel::type)5u, > (LogTag::type)50u, (LogTag::type)0u, (LogTag::type)0u, (LogTag::type)0 > u, (LogTag::type)0u, (LogTag::type)0u>::~LogStreamTemplate()?: > /priv/d031900/openjdk/jdk10-hs/source/hotspot/src/share/vm/logging/logStream.hpp:61:6: > error: ?static void LogStream::operator delete(void*)? is private > void operator delete(void* p); > ^ > In file included from > /priv/d031900/openjdk/jdk10-hs/source/hotspot/test/native/logging/logTestUtils.inline.hpp:26:0, > from > /priv/d031900/openjdk/jdk10-hs/source/hotspot/test/native/logging/test_logTagSetDescriptions.cpp:25: > /priv/d031900/openjdk/jdk10-hs/source/hotspot/src/share/vm/logging/logStream.hpp:99:7: > error: within this context > class LogStreamTemplate : public LogStream { > ^ > > > First I wanted to know if this is a remnant of LogStream being a > ResourceObj, so I removed the inheritance between outputStream and > ResourceObj, but the same errors happen (beside new ones, of course). > > I believe this is caused by ~outputStream() being virtual - so a virtual > ~LogStreamTemplate() gets synthesized, which wants to reference > LogStream::operator delete()? I have the vague notion that this has > something to do with the fact that operator delete is "static but > polymorphic" in the sense that for polymorphic delete calls it needs to > be looked up via the virtual destructor. Something like this: > "https://stackoverflow.com/questions/2273187/why-is-the-delete-operator-required-to-be-static". > However, there are lots of things I do not understand here. > > I still do not think this is a problem, since we disallow new() sucessfully. Ok. On 07/04/2017 12:05 PM, Thomas St?fe wrote: > The key point here is "as one single log call". If we allow > ourselves to pass a more elaborate data structure than a plain > string (such as a linked list of strings), then we can enable this > kind of "chunked expansion". The write call will eventually result in > > flockfile(_stream); > // do stuff > fflush(_stream); > funlockfile(_stream); > > deep down in logFileStreamOutput.cpp. Whatever we do at > `// do stuff` will become atomic wrt to other threads. We could for > example iterate over a linked list of strings and print them (with > space as separator). For an example, see > LogFileStreamOutput::write(LogMessageBuffer::Iterator msg_iterator). > > > I think it can be done, but I am not sure it is worth the trouble. > > Right now, LogOutput is implemented by only one concrete child, writing > to a FILE*. I would not swap a single IO syscall (fprintf) against a > number of them to save some memory reallocations. So, we still call > fprintf() only once, and we would have to assemble all the chunked > buffers beforehand. Granted, that still could be a bit better than > reallocating the LogStream line buffer every 64byte, but that also > depends on the line buffer allocation strategy (back to doubling the > memory for the line buffer?) Yeah, I agree that doing the syscall multiple times is probably more expensive than the copying the memory. > Thinking further ahead: Assuming that we want to implement more > LogOutput childs in the future, I would be careful adding an API which > exposes lockinf behaviour of the underlying implementation. This means I > would not expose the sequence of functions ("lock(), > write(),...,write(), unlock()") in the general LogOutput() API (just as > an example, lets assume you want to have a LogOutput writing to a > lock-free ring buffer in shared memory or some such). > > But I do not think you meant that, right? You meant something like a > single call, "log(struct my_list_of_chunks_t*)" ? I think that could be > done, but if we find that all possible LogOutput implementations we can > think up will re-assemble the chunks anyway before doing something with > it, it may not be worth the trouble. Yes, this is what I was thinking of. Such a log(struct my_list_of_chunks_t*) could then do the final allocation and copy everything over once, then call fprintf. This is probably not worth doing for this patch, but I think we should file an enhancement for this and look into it later. I will continue reviewing the rest of the patch! Thanks, Erik From erik.osterlund at oracle.com Fri Jul 7 15:17:39 2017 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 7 Jul 2017 17:17:39 +0200 Subject: RFR (S): 8182703: Correct G1 barrier queue lock orderings In-Reply-To: <6B5ACCE3-CA0C-41C9-A45A-C79467FB8CE2@oracle.com> References: <59510D5E.10009@oracle.com> <25F423D9-F8D5-4E62-8300-CCE106E70777@oracle.com> <6D1B2CB2-366E-4DBD-9F2E-672325459343@oracle.com> <6B5ACCE3-CA0C-41C9-A45A-C79467FB8CE2@oracle.com> Message-ID: <595FA613.7090306@oracle.com> Hi Kim, Added hotspot-dev as requested. To answer your worries we must first understand what invariant these checks for 'special' locks seek to achieve. The invariant is, AFAIK, that locks with a rank of 'special' and below (now including 'access' as well as 'event' that was already there before) must *not* check for safepoints when grabbed from JavaThreads. Safepoint checks translate to performing ThreadBlockInVM which must *not* be performed in 'special' or below ranked locks. The reason this is necessary, is that those locks must be usable from safepoint-unsafe places, e.g. leaf calls, where we can not yield to the safepoint synchronizer. I believe that must have been what the name 'special' originated from, correct me if I'm wrong. Since locking in safepoint-unsafe places potentially blocks the safepoint synchronizer, a deadlock would arise if a thread, 'Thread A', acquires a special lock with Java/VM thread state. But that special lock is held by another thread, 'Thread B', that has blocked on a non-special lock, because it yielded to the safepoint synchronizer on the VM thread, that is concurrently synchronizing a safepoint. However, the safepoint synchronizer is blocked waiting for the thread acquiring the special lock to yield. But it never will. In the end, 'Thread A' waits for 'Thread B' to release the special lock, and 'Thread B' waits for the VMThread that is safepoint synchronizing, and the VMThread is waiting for 'Thread A' - a deadlock. With the provided invariant however, it is impossible to acquire non-safepoint non-special locks while already holding 'special' and below locks. Therefore, by checking for the condition of the invariant, we will be certain such a deadlock can not happen, as that would violate the usual lock rank ordering that we all know about - one can not acquire a lock that has a rank higher than locks already acquired. First of all, before we delve deeper into this rabbit hole. Note that if our new 'access' locks are used properly, e.g. do not perform safepoint checks or call code that needs to have a safepoint-safe state under the lock, we are good. The G1 barrier code conforms to those restrictions and the code has since forever been called in leaf calls while in Java thread state, without oop maps, making it a not safepoint-safe state. So that is all good. The remaining question to answer is, had we done crazy things while holding those access locks, would the deadlock detection system have detected that? To answer that question we must examine whether that invariant holds or not. There are three paths for locking a Mutex/Monitor, try_lock(), lock() and lock_without_safepoint_check(). Among these, only lock() violates the invariant for 'special' and below locks taken from a JavaThread. It is the only locking operation that performs a safepoint check. try_lock() instead returns false if the lock could not be acquired, and lock_without_safepoint_check does not check for safepoints. MutexLocker and MutexLockerEx are abstractions around Mutex that boil down to calls to either lock() or lock_without_safepoint_check(). So if lock() catches the broken invariant, all locking in hotspot will somehow catch it. Let's examine what happens in lock(). share/vm/runtime/mutex.cpp:933:21: assert(rank() > Mutex::special, "Potential deadlock with special or lesser rank mutex"); This is the check in Monitor::lock() *with* safepoint check. It definitely catches illegal uses of lock() on 'special', 'event' and 'access' ranked locks on JavaThreads. Since we always catch misuse of special and below ranked locks here, the deadlock detection system works correctly even for 'event' and the new 'access' ranked locks. The other checks are mostly redundant and will eventually boil down to this check. Examples: share/vm/runtime/mutexLocker.hpp:165:29: assert(mutex->rank() != Mutex::special share/vm/runtime/mutexLocker.hpp:173:29: assert(mutex->rank() != Mutex::special, These two are constructors for MutexLocker which will call lock(). Therefore, this check is redundant. The 'event' and 'access' ranks will both miss this redundant check, but subsequently run into the check in lock(), which is the one that matters and still catches the broken invariant. share/vm/runtime/mutexLocker.hpp:208:30: assert(mutex->rank() > Mutex::special || no_safepoint_check, This check in MutexLockerEx checks that either the lock should be over special or it must not check for safepoints. This works as intended with 'event' and 'access' locks. They are forced to perform a lock without safepoint check, and hence not enter the lock() path of Mutex. However, if they did, it would still be redundantly sanity checked there too. share/vm/runtime/mutex.cpp:1384:23: || rank() == Mutex::special, "wrong thread state for using locks"); share/vm/runtime/mutex.cpp:1389:30: debug_only(if (rank() != Mutex::special) \ These two check checks are found in Monitor::check_prelock_state(). Which is called from lock() and try_lock(). As for lock, we already concluded that using lock() *at all* on a special lock from a JavaThread will be found out and an assert will trigger. So these checks for special locks seem to be a bit redundant. As for the try_lock() case, they even seem wrong. Checking for valid safepoint state for a locking operation that can't block seems wrong. But that is invariant of whether the lock is special or not. It just should not check for safepoint-safe states on try_lock(). share/vm/runtime/thread.cpp:903:27: cur->rank() == Mutex::special) { This check is in Thread::check_for_valid_safepoint_state() where it walks all the monitors and makes sure that we don't have any of {Threads_lock, Compile_lock, VMOperationRequest_lock, VMOperationQueue_lock} and allow_vm_block() or it contains any special lock. This is performed when e.g. allocating etc. This check should arguably check for special *and below* ranked locks that should not be acquired while in safepoint-safe. However, those access locks return true on allow_vm_block(), and therefore will correctly be detected as dangerous had we done crazy things under these locks. All in all, I believe that the deadlock detecion system has some redundant, and some confusing checks that involve the lock rank Mutex::special. But I do believe that it works and would detect deadlocks, but could do with some reworking to make it more explicit. And that is invariant of the new access rank and applies equally to the event rank. However, since these access locks play well with the current deadlock detection as they do not do anything illegal, and even if use of these locks did indeed do illegal things, it would still be detected by the deadlock detection system, it is reasonable to say that refactoring the deadlock detection system is a separate RFE? Specifically, clarifying the deadlock detection system by removing redundant checks, not checking for safepoint-safe state in try_lock as well as explicitly listing special and below locks as illegal when verifying Thread::check_for_valid_safepoint_state(), regardles of whether allow_vm_block() is true or not. Sounds like a separate RFE to me! Thanks, /Erik On 2017-07-06 22:15, Kim Barrett wrote: >> On Jul 6, 2017, at 4:11 PM, Kim Barrett wrote: >> >>> On Jul 4, 2017, at 10:00 PM, Kim Barrett wrote: >>> The lock ranking changes look good. >> I'm going to retract that. >> >> How does these new lock rankings interact with various assertions that >> rank() == or != Mutex::special? I'm not sure those places handle >> these new ranks properly. (I'm not sure those places handle >> Mutex::event rank properly either.) > And maybe this change needs to be discussed on hotspot-dev rather than hotspot-gc-dev. > From erik.helin at oracle.com Mon Jul 10 09:40:34 2017 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 10 Jul 2017 11:40:34 +0200 Subject: RFR(XL): 8181917: Refactor UL LogStreams to avoid using resource area In-Reply-To: References: <337330d2-512c-bc77-5a55-c05d33e376e5@oracle.com> Message-ID: Hi Thomas, On 06/30/2017 08:32 PM, Thomas St?fe wrote: > Hi Eric, > > thank you for the review! > > New > Version: http://cr.openjdk.java.net/~stuefe/webrevs/8181917-refactor-ul-logstream/all.webrev.03/webrev/ I think you still have a problem with "runaway" loggers in this version: +// try_ensure_cap tries to enlarge the capacity of the internal buffer +// to the given atleast value. May fail if either OOM happens or atleast +// is larger than a reasonable max of 1 M. Caller must not assume +// capacity without checking. +void LogStream::LineBuffer::try_ensure_cap(size_t atleast) { + assert(_cap >= sizeof(_smallbuf), "sanity"); + if (_cap < atleast) { + const size_t reasonable_max = 1 * M; + size_t newcap = align_size_up(atleast + 64, 64); + assert(_cap <= reasonable_max, "sanity"); + // Cap out at a reasonable max to prevent runaway leaks. + if (newcap > reasonable_max) { + newcap = reasonable_max; + } + + char* const newbuf = (char*) os::malloc(newcap, mtLogging); + if (newbuf == NULL) { // OOM. Leave object unchanged. + return; + } + if (_pos > 0) { // preserve old content + memcpy(newbuf, _buf, _pos + 1); // ..including trailing zero + } + if (_buf != _smallbuf) { + os::free(_buf); + } + _buf = newbuf; + _cap = newcap; + } + assert(_cap >= atleast, "sanity"); +} with the above code, even though _cap > reasonable_max, you will always allocate newbuf with os::malloc. For debug builds, we are fine, since the JVM will assert on _cap <= reasonable_max, but for product builds we are not. Would it be better to first check if _cap == reasonable_max? As in the following snippet: void LogStream::LineBuffer::try_ensure_cap(size_t atleast) { if (_cap < atleast) { const size_t reasonable_max = 1 * M; if (_cap == reasonable_max) { // max memory used, "die" in debug builds, log in product builds log_warning(logging)("Max memory used for message: %s\n", _buf); ShouldNotReachHere(); return; } size_t newcap = align_size_up(atleast + 64, 64); if (newcap > reasonable_max) { newcap = reasonable_max; } char* const newbuf = (char*) os::malloc(newcap, mtLogging); // rest of logic } } Thanks, Erik From per.liden at oracle.com Mon Jul 10 09:52:20 2017 From: per.liden at oracle.com (Per Liden) Date: Mon, 10 Jul 2017 11:52:20 +0200 Subject: RFR (M): 8183927: Hotspot needs C++ type_traits metaprogramming utilities In-Reply-To: <595E3647.1030101@oracle.com> References: <595E3647.1030101@oracle.com> Message-ID: Hi Erik, On 2017-07-06 15:08, Erik ?sterlund wrote: > Hi, > > I'm sure we have all had moments when we wished we could use the C++11 > type_traits header and the nice metaprogramming tools in there, but then > suddenly remembered that we can not, because we are building libjvm.so > with C++03 and are still arguing whether we can use namespaces. > Well.. welcome to the future. I am delighted to announce that now at > least some of those tools will become available. > > A bunch of C++11 compliant metaprogramming tools that can be used > already in C++03 have been developed. > They have been developed with the most conservative and portable C++03 > code possible so that even exotic compilers can support it. > Special thanks go to Kim Barret whom I have had many long conversations > about many paragraphs of the C++ standard throughout this work. It has > helped a lot and improved the code. > > While this change introduces the traits without using it in the code > (more than from gtest), it will become used in already prepared patches > for refactoring Atomic and OrderAccess to use templates rather than hand > picked types like the JNI-specific jint etc types. That serves as the > bottom layer of the upcoming GC barrier interface that also requires > these utilities. Since they are rather large changes, I hope it is okay > that I separate out the metaprogramming utilities. I argue that they are > of value in general in hotspot and will make our lives easier. > > All metaprogramming tools have been placed in a new > hotspot/src/share/vm/metaprogramming directory. > Tests have been written and placed in a new test/native/metaprogramming > directory. > > A bug ID can be found here: > https://bugs.openjdk.java.net/browse/JDK-8183927 > > A webrev with code can be found here: > http://cr.openjdk.java.net/~eosterlund/8183927/webrev.00/ Looks good! /Per > > Here is a list of metaprogramming tools that have been introduced with a > corresponding short explanation: > > ============================ > > template > struct IntegralConstant > > This metafunction takes as parameters a type and a value. > The type and value can later be queried from the value_type and value > members respectively. > > Here is an example usage: > typedef IntegralConstant TrueType; > > Now TrueType::value_type is the bool type, and TrueType::value is the > true boolean value. > > Many other metafunctions will later inherit from either TrueType or > FalseType to e.g. denote the metafunction returning true or false based > on some condition. > > The following traits that start with Is* belong to this category. They > manifest a constant evaluated at compile time that checks some condition > of types. > > ============================ > > template > struct IsSame > > This metafunction takes as parameters two types, X and Y, and returns > true iff they are the same type. Otherwise it returns false. > > ============================ > > template > struct IsConst > > This metafunction takes as parameter a type T, and returns true iff T > has a const qualifier in it. > > ============================ > > template > struct IsVolatile > > This metafunction takes as parameter a type T, and returns true iff T > has a volatile qualifier in it. > > ============================ > > template > struct IsSigned > > This metafunction takes as parameter a type T, and checks if it is > signed. This is equivalent to asking std::numeric_limits<>::is_signed, > but additionally disregards CV qualifiers. > > ============================ > > template > struct IsIntegral > > This metafunction takes as parameter a type T, and returns true if it is > an integral type. This is similar to asking > std::numeric_limits<>::is_integer, but additionally disregards CV > qualifiers. Note that this returns false for enums. > > ============================ > > template > struct IsSignedIntegral > > template > struct IsUnsignedIntegral > > These two metafunctions are helpers that return true for integral types > that have corresponding signedness. > > ============================ > > template > struct IsFloatingPoint > > This metafunction takes as parameter a type T, and returns true iff T is > a floating point type (irrespective of CV qualifiers). > > ============================ > > template > class IsPointer > > This metafunction takes as parameter a type T, and returns true iff T is > a pointer type (irrespective of CV qualifiers). > > ============================ > > That is it as far as metafunctions returning booleans go in these > changes. So now we move on to metafunctions that make use of these > boolean conditions. > > template > struct Conditional > > This metafunction constitutes a conditional type, controlled by the > condition passed in. Its type evaluates to TrueType iff the condition is > true, and otherwise evaluates to FalseType. > > Here is a contrived toy example: > > template > void example(T base) { > typedef Conditional::value, ptrdiff_t, T> ScaleType; > ScaleType offset = offset(); > do_something_clever(base + offset); > } > > ============================ > > In this example, the type of the offset is ptrdiff_t iff T is a pointer > and otherwise it is T. This allows passing in pointers and integers to > the function, and adding offsets into it. > > template > struct EnableIf > > This metafunction is the key to SFINAE (Substitution Failure Is Not An > Error), a C++ metaprogramming idiom that allows explicitly allowing or > disallowing function overloads based on precise conditions for types > passed in as parameter. > The EnableIf overload guard should replace the return value of a > function. The type T is the type that is intended to be returned if the > overload is allowed based on the passed in condition. By default that is > void. > > Here is a contrived toy example usage: > > template > EnableIf::value> print(T x) {} > > template > EnableIf::value> print(T x) {} > > ... > > print(1); // calls the first overload enabled for integral types > print("hello SFINAE world"); // call the second overload enabled for > non-integral types > > ============================ > > Now we move on to metafunctions that remove properties like e.g. CV > qualifiers, pointers and references from types, resulting in a new type. > > template > struct RemovePointer > > This metafunction takes as parameter a type T, and returns the type > under the pointer iff T is a pointer, and otherwise returns the same > type T. > > ============================ > > template > struct RemoveReference > > This metafunction takes as parameter a type T, and returns the type > under the reference iff T is a pointer, and otherwise returns the same > type T. > > ============================ > > template > struct RemoveCV > > This metafunction takes as parameter a type T, and returns the type T > without CV qualifiers. > > ============================ > > template > struct Decay > > This metafunction takes as parameter a type T, and returns the CV > unqualified type under the reference iff T is a reference type, and > otherwise returns the CV unqualified type T. > > ============================ > > Testing: A bunch of tests have been added to the > test/native/metaprogramming folder for all traits (except > IntegralConstant, because there is not a whole lot to test there, and it > is used by all condition metafunctions in their corresponding tests). > They have gone through JPRT, and work well on all oracle supported > platforms, and is also known to have recently built with the xlC > compiler on AIX. I would like to ask external maintainers though to > please verify that this works on your compilers. I have done my best to > use the most conservative tricks in the book to make it as portable as > possible, and I hope it will work well for us all. > > Thanks, > /Erik From erik.osterlund at oracle.com Mon Jul 10 10:16:51 2017 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 10 Jul 2017 12:16:51 +0200 Subject: RFR (M): 8183927: Hotspot needs C++ type_traits metaprogramming utilities In-Reply-To: References: <595E3647.1030101@oracle.com> Message-ID: <59635413.4060009@oracle.com> Hi Per, Thanks for the review! /Erik On 2017-07-10 11:52, Per Liden wrote: > Hi Erik, > > On 2017-07-06 15:08, Erik ?sterlund wrote: >> Hi, >> >> I'm sure we have all had moments when we wished we could use the C++11 >> type_traits header and the nice metaprogramming tools in there, but then >> suddenly remembered that we can not, because we are building libjvm.so >> with C++03 and are still arguing whether we can use namespaces. >> Well.. welcome to the future. I am delighted to announce that now at >> least some of those tools will become available. >> >> A bunch of C++11 compliant metaprogramming tools that can be used >> already in C++03 have been developed. >> They have been developed with the most conservative and portable C++03 >> code possible so that even exotic compilers can support it. >> Special thanks go to Kim Barret whom I have had many long conversations >> about many paragraphs of the C++ standard throughout this work. It has >> helped a lot and improved the code. >> >> While this change introduces the traits without using it in the code >> (more than from gtest), it will become used in already prepared patches >> for refactoring Atomic and OrderAccess to use templates rather than hand >> picked types like the JNI-specific jint etc types. That serves as the >> bottom layer of the upcoming GC barrier interface that also requires >> these utilities. Since they are rather large changes, I hope it is okay >> that I separate out the metaprogramming utilities. I argue that they are >> of value in general in hotspot and will make our lives easier. >> >> All metaprogramming tools have been placed in a new >> hotspot/src/share/vm/metaprogramming directory. >> Tests have been written and placed in a new test/native/metaprogramming >> directory. >> >> A bug ID can be found here: >> https://bugs.openjdk.java.net/browse/JDK-8183927 >> >> A webrev with code can be found here: >> http://cr.openjdk.java.net/~eosterlund/8183927/webrev.00/ > > Looks good! > > /Per > >> >> Here is a list of metaprogramming tools that have been introduced with a >> corresponding short explanation: >> >> ============================ >> >> template >> struct IntegralConstant >> >> This metafunction takes as parameters a type and a value. >> The type and value can later be queried from the value_type and value >> members respectively. >> >> Here is an example usage: >> typedef IntegralConstant TrueType; >> >> Now TrueType::value_type is the bool type, and TrueType::value is the >> true boolean value. >> >> Many other metafunctions will later inherit from either TrueType or >> FalseType to e.g. denote the metafunction returning true or false based >> on some condition. >> >> The following traits that start with Is* belong to this category. They >> manifest a constant evaluated at compile time that checks some condition >> of types. >> >> ============================ >> >> template >> struct IsSame >> >> This metafunction takes as parameters two types, X and Y, and returns >> true iff they are the same type. Otherwise it returns false. >> >> ============================ >> >> template >> struct IsConst >> >> This metafunction takes as parameter a type T, and returns true iff T >> has a const qualifier in it. >> >> ============================ >> >> template >> struct IsVolatile >> >> This metafunction takes as parameter a type T, and returns true iff T >> has a volatile qualifier in it. >> >> ============================ >> >> template >> struct IsSigned >> >> This metafunction takes as parameter a type T, and checks if it is >> signed. This is equivalent to asking std::numeric_limits<>::is_signed, >> but additionally disregards CV qualifiers. >> >> ============================ >> >> template >> struct IsIntegral >> >> This metafunction takes as parameter a type T, and returns true if it is >> an integral type. This is similar to asking >> std::numeric_limits<>::is_integer, but additionally disregards CV >> qualifiers. Note that this returns false for enums. >> >> ============================ >> >> template >> struct IsSignedIntegral >> >> template >> struct IsUnsignedIntegral >> >> These two metafunctions are helpers that return true for integral types >> that have corresponding signedness. >> >> ============================ >> >> template >> struct IsFloatingPoint >> >> This metafunction takes as parameter a type T, and returns true iff T is >> a floating point type (irrespective of CV qualifiers). >> >> ============================ >> >> template >> class IsPointer >> >> This metafunction takes as parameter a type T, and returns true iff T is >> a pointer type (irrespective of CV qualifiers). >> >> ============================ >> >> That is it as far as metafunctions returning booleans go in these >> changes. So now we move on to metafunctions that make use of these >> boolean conditions. >> >> template >> struct Conditional >> >> This metafunction constitutes a conditional type, controlled by the >> condition passed in. Its type evaluates to TrueType iff the condition is >> true, and otherwise evaluates to FalseType. >> >> Here is a contrived toy example: >> >> template >> void example(T base) { >> typedef Conditional::value, ptrdiff_t, T> ScaleType; >> ScaleType offset = offset(); >> do_something_clever(base + offset); >> } >> >> ============================ >> >> In this example, the type of the offset is ptrdiff_t iff T is a pointer >> and otherwise it is T. This allows passing in pointers and integers to >> the function, and adding offsets into it. >> >> template >> struct EnableIf >> >> This metafunction is the key to SFINAE (Substitution Failure Is Not An >> Error), a C++ metaprogramming idiom that allows explicitly allowing or >> disallowing function overloads based on precise conditions for types >> passed in as parameter. >> The EnableIf overload guard should replace the return value of a >> function. The type T is the type that is intended to be returned if the >> overload is allowed based on the passed in condition. By default that is >> void. >> >> Here is a contrived toy example usage: >> >> template >> EnableIf::value> print(T x) {} >> >> template >> EnableIf::value> print(T x) {} >> >> ... >> >> print(1); // calls the first overload enabled for integral types >> print("hello SFINAE world"); // call the second overload enabled for >> non-integral types >> >> ============================ >> >> Now we move on to metafunctions that remove properties like e.g. CV >> qualifiers, pointers and references from types, resulting in a new type. >> >> template >> struct RemovePointer >> >> This metafunction takes as parameter a type T, and returns the type >> under the pointer iff T is a pointer, and otherwise returns the same >> type T. >> >> ============================ >> >> template >> struct RemoveReference >> >> This metafunction takes as parameter a type T, and returns the type >> under the reference iff T is a pointer, and otherwise returns the same >> type T. >> >> ============================ >> >> template >> struct RemoveCV >> >> This metafunction takes as parameter a type T, and returns the type T >> without CV qualifiers. >> >> ============================ >> >> template >> struct Decay >> >> This metafunction takes as parameter a type T, and returns the CV >> unqualified type under the reference iff T is a reference type, and >> otherwise returns the CV unqualified type T. >> >> ============================ >> >> Testing: A bunch of tests have been added to the >> test/native/metaprogramming folder for all traits (except >> IntegralConstant, because there is not a whole lot to test there, and it >> is used by all condition metafunctions in their corresponding tests). >> They have gone through JPRT, and work well on all oracle supported >> platforms, and is also known to have recently built with the xlC >> compiler on AIX. I would like to ask external maintainers though to >> please verify that this works on your compilers. I have done my best to >> use the most conservative tricks in the book to make it as portable as >> possible, and I hope it will work well for us all. >> >> Thanks, >> /Erik From adam.petcher at oracle.com Wed Jul 5 20:01:48 2017 From: adam.petcher at oracle.com (Adam Petcher) Date: Wed, 5 Jul 2017 16:01:48 -0400 Subject: AccessController.doPrivileged and default methods In-Reply-To: References: Message-ID: <09f06df1-8547-04a7-36b6-361f9bea1cfd@oracle.com> Very interesting. The exception originates in the VM, so I added the hotspot-dev list to the discussion. It looks like the VM either can't find the method, or it decides that it's not the right kind of method. On 7/4/2017 7:57 AM, Michael Rasmussen wrote: > Hi > > I was playing around with having an easy way to have doPrivileged call > void methods via lambdas/method references, and noticed that if you > use AccessController.doPrivileged(PrivilegedAction), and the run > method is implemented as a default method, then you get the following > exception: java.lang.InternalError: No run method > > I failed finding anywhere in the documentation stating this should not > be supported, or a bug report about it, so writing/asking here. > > See example code below > > Kind regards > Michael Rasmussen > > > //---- > > package com.test; > > import java.security.AccessController; > import java.security.PrivilegedAction; > > public class Test { > interface VoidPrivilegedAction extends PrivilegedAction { > void perform(); > > @Override > default Void run() { > perform(); > return null; > } > } > > static void doPrivileged(VoidPrivilegedAction act) { > AccessController.doPrivileged(act); > } > > public static void main(String[] args) throws Exception { > doPrivileged(() -> System.out.println(System.getProperty("java.home"))); > } > } > > //---- > > Exception in thread "main" java.lang.InternalError: No run method > at java.security.AccessController.doPrivileged(Native Method) > at com.test.Test.doPrivileged(Test.java:18) > at com.test.Test.main(Test.java:22) From sean.mullan at oracle.com Thu Jul 6 14:33:09 2017 From: sean.mullan at oracle.com (Sean Mullan) Date: Thu, 6 Jul 2017 10:33:09 -0400 Subject: AccessController.doPrivileged and default methods In-Reply-To: <09f06df1-8547-04a7-36b6-361f9bea1cfd@oracle.com> References: <09f06df1-8547-04a7-36b6-361f9bea1cfd@oracle.com> Message-ID: On 7/5/17 4:01 PM, Adam Petcher wrote: > > Very interesting. The exception originates in the VM, so I added the > hotspot-dev list to the discussion. It looks like the VM either can't > find the method, or it decides that it's not the right kind of method. Right. The check that is causing this error is in jvm.cpp [1]: 1271 if (m.is_null() || !m->is_method() || !m()->is_public() || m()->is_static()) { 1272 THROW_MSG_0(vmSymbols::java_lang_InternalError(), "No run method"); 1273 } I'll file a bug unless someone has a good explanation for this behavior. Thanks, Sean [1] http://hg.openjdk.java.net/jdk9/jdk9/hotspot/file/1ca8f038fceb/src/share/vm/prims/jvm.cpp#l1271 > On 7/4/2017 7:57 AM, Michael Rasmussen wrote: >> Hi >> >> I was playing around with having an easy way to have doPrivileged call >> void methods via lambdas/method references, and noticed that if you >> use AccessController.doPrivileged(PrivilegedAction), and the run >> method is implemented as a default method, then you get the following >> exception: java.lang.InternalError: No run method >> >> I failed finding anywhere in the documentation stating this should not >> be supported, or a bug report about it, so writing/asking here. >> >> See example code below >> >> Kind regards >> Michael Rasmussen >> >> >> //---- >> >> package com.test; >> >> import java.security.AccessController; >> import java.security.PrivilegedAction; >> >> public class Test { >> interface VoidPrivilegedAction extends PrivilegedAction { >> void perform(); >> >> @Override >> default Void run() { >> perform(); >> return null; >> } >> } >> >> static void doPrivileged(VoidPrivilegedAction act) { >> AccessController.doPrivileged(act); >> } >> >> public static void main(String[] args) throws Exception { >> doPrivileged(() -> >> System.out.println(System.getProperty("java.home"))); >> } >> } >> >> //---- >> >> Exception in thread "main" java.lang.InternalError: No run method >> at java.security.AccessController.doPrivileged(Native Method) >> at com.test.Test.doPrivileged(Test.java:18) >> at com.test.Test.main(Test.java:22) > From sean.mullan at oracle.com Fri Jul 7 20:21:05 2017 From: sean.mullan at oracle.com (Sean Mullan) Date: Fri, 7 Jul 2017 16:21:05 -0400 Subject: AccessController.doPrivileged and default methods In-Reply-To: References: <09f06df1-8547-04a7-36b6-361f9bea1cfd@oracle.com> Message-ID: On 7/6/17 10:33 AM, Sean Mullan wrote: > On 7/5/17 4:01 PM, Adam Petcher wrote: >> >> Very interesting. The exception originates in the VM, so I added the >> hotspot-dev list to the discussion. It looks like the VM either can't >> find the method, or it decides that it's not the right kind of method. > > Right. The check that is causing this error is in jvm.cpp [1]: > > 1271 if (m.is_null() || !m->is_method() || !m()->is_public() || > m()->is_static()) { > 1272 THROW_MSG_0(vmSymbols::java_lang_InternalError(), "No run method"); > 1273 } > > I'll file a bug unless someone has a good explanation for this behavior. Bug now filed: https://bugs.openjdk.java.net/browse/JDK-8183962 --Sean > > Thanks, > Sean > > [1] > http://hg.openjdk.java.net/jdk9/jdk9/hotspot/file/1ca8f038fceb/src/share/vm/prims/jvm.cpp#l1271 > > > >> On 7/4/2017 7:57 AM, Michael Rasmussen wrote: >>> Hi >>> >>> I was playing around with having an easy way to have doPrivileged call >>> void methods via lambdas/method references, and noticed that if you >>> use AccessController.doPrivileged(PrivilegedAction), and the run >>> method is implemented as a default method, then you get the following >>> exception: java.lang.InternalError: No run method >>> >>> I failed finding anywhere in the documentation stating this should not >>> be supported, or a bug report about it, so writing/asking here. >>> >>> See example code below >>> >>> Kind regards >>> Michael Rasmussen >>> >>> >>> //---- >>> >>> package com.test; >>> >>> import java.security.AccessController; >>> import java.security.PrivilegedAction; >>> >>> public class Test { >>> interface VoidPrivilegedAction extends PrivilegedAction { >>> void perform(); >>> >>> @Override >>> default Void run() { >>> perform(); >>> return null; >>> } >>> } >>> >>> static void doPrivileged(VoidPrivilegedAction act) { >>> AccessController.doPrivileged(act); >>> } >>> >>> public static void main(String[] args) throws Exception { >>> doPrivileged(() -> >>> System.out.println(System.getProperty("java.home"))); >>> } >>> } >>> >>> //---- >>> >>> Exception in thread "main" java.lang.InternalError: No run method >>> at java.security.AccessController.doPrivileged(Native Method) >>> at com.test.Test.doPrivileged(Test.java:18) >>> at com.test.Test.main(Test.java:22) >> From jiangli.zhou at oracle.com Mon Jul 10 15:53:58 2017 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Mon, 10 Jul 2017 08:53:58 -0700 Subject: jdk10 : UseSharedSpaces flag and INCLUDE_CDS macro In-Reply-To: <64495e2dba5d4d1788d101eaa6f89a3e@sap.com> References: <64495e2dba5d4d1788d101eaa6f89a3e@sap.com> Message-ID: <36267ACB-03A1-4C37-AA4E-269F244B78D8@oracle.com> Hi Matthias, Thank you for noticing that. Yes, it would be helpful if you can add the #if INCLUDE_CDS for CDS only code. I can help you integrate the changes after they are reviewed. Thanks! Jiangli > On Jul 5, 2017, at 6:36 AM, Baesken, Matthias wrote: > > Hello, when looking into CDS related build options, I noticed that most code-parts that are executed only when UseSharedSpaces is set, > are guarded by the compile-time macro INCLUDE_CDS to support switching off compilation of this coding > when CDS is disabled at compile time : > > > See hotspot/make/lib/JvmFeatures.gmk : > > ifneq ($(call check-jvm-feature, cds), true) > JVM_CFLAGS_FEATURES += -DINCLUDE_CDS=0 > > > > However some places miss the compile-time guarding ( #if INCLUDE_CDS ....) for example in > > > share/vm/prims/jvmtiRedefineClasses.cpp > share/vm/memory/universe.cpp > > (also some other places) > > > Should I prepare a change and add the compile-time guard at the places where missing as well ? > > Best regards, Matthias From kim.barrett at oracle.com Mon Jul 10 21:33:09 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 10 Jul 2017 17:33:09 -0400 Subject: RFR (S): 8182703: Correct G1 barrier queue lock orderings In-Reply-To: <595FA613.7090306@oracle.com> References: <59510D5E.10009@oracle.com> <25F423D9-F8D5-4E62-8300-CCE106E70777@oracle.com> <6D1B2CB2-366E-4DBD-9F2E-672325459343@oracle.com> <6B5ACCE3-CA0C-41C9-A45A-C79467FB8CE2@oracle.com> <595FA613.7090306@oracle.com> Message-ID: <5B0584AA-C49D-4426-AD02-3D23AF45F0CD@oracle.com> On 2017-07-06 22:15, Kim Barrett wrote: >> On Jul 6, 2017, at 4:11 PM, Kim Barrett wrote: >> >>> On Jul 4, 2017, at 10:00 PM, Kim Barrett wrote: >>> The lock ranking changes look good. >> I'm going to retract that. >> >> How does these new lock rankings interact with various assertions that >> rank() == or != Mutex::special? I'm not sure those places handle >> these new ranks properly. (I'm not sure those places handle >> Mutex::event rank properly either.) > On Jul 7, 2017, at 11:17 AM, Erik ?sterlund wrote: > > [?] > All in all, I believe that the deadlock detecion system has some redundant, and some confusing checks that involve the lock rank Mutex::special. But I do believe that it works and would detect deadlocks, but could do with some reworking to make it more explicit. And that is invariant of the new access rank and applies equally to the event rank. > > However, since these access locks play well with the current deadlock detection as they do not do anything illegal, and even if use of these locks did indeed do illegal things, it would still be detected by the deadlock detection system, it is reasonable to say that refactoring the deadlock detection system is a separate RFE? > > Specifically, clarifying the deadlock detection system by removing redundant checks, not checking for safepoint-safe state in try_lock as well as explicitly listing special and below locks as illegal when verifying Thread::check_for_valid_safepoint_state(), regardles of whether allow_vm_block() is true or not. Sounds like a separate RFE to me! Thanks for the additional analysis. I agree that so long as one does what one is supposed to (e.g. these locks always need to avoid safepoint checks), there won't be any undesired assertions. And I also agree there won't be any bad consequences (e.g. incorrect code possibly slipping through) from misuse, though the indicative failures might not always be where one might prefer. I don't think the redundant checks are necessarily bad, as they make it more obvious to future readers what the requirements are at various levels. However, I agree it should be a separate RFE to do some cleanup in this area, particularly where [non-]equality with Mutex::special ought to be an ordered comparison. From erik.osterlund at oracle.com Tue Jul 11 10:28:44 2017 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 11 Jul 2017 12:28:44 +0200 Subject: RFR (S): 8182703: Correct G1 barrier queue lock orderings In-Reply-To: <5B0584AA-C49D-4426-AD02-3D23AF45F0CD@oracle.com> References: <59510D5E.10009@oracle.com> <25F423D9-F8D5-4E62-8300-CCE106E70777@oracle.com> <6D1B2CB2-366E-4DBD-9F2E-672325459343@oracle.com> <6B5ACCE3-CA0C-41C9-A45A-C79467FB8CE2@oracle.com> <595FA613.7090306@oracle.com> <5B0584AA-C49D-4426-AD02-3D23AF45F0CD@oracle.com> Message-ID: <5964A85C.6080807@oracle.com> On 2017-07-10 23:33, Kim Barrett wrote: > On 2017-07-06 22:15, Kim Barrett wrote: >>> On Jul 6, 2017, at 4:11 PM, Kim Barrett wrote: >>> >>>> On Jul 4, 2017, at 10:00 PM, Kim Barrett wrote: >>>> The lock ranking changes look good. >>> I'm going to retract that. >>> >>> How does these new lock rankings interact with various assertions that >>> rank() == or != Mutex::special? I'm not sure those places handle >>> these new ranks properly. (I'm not sure those places handle >>> Mutex::event rank properly either.) >> On Jul 7, 2017, at 11:17 AM, Erik ?sterlund wrote: >> >> [?] >> All in all, I believe that the deadlock detecion system has some redundant, and some confusing checks that involve the lock rank Mutex::special. But I do believe that it works and would detect deadlocks, but could do with some reworking to make it more explicit. And that is invariant of the new access rank and applies equally to the event rank. >> >> However, since these access locks play well with the current deadlock detection as they do not do anything illegal, and even if use of these locks did indeed do illegal things, it would still be detected by the deadlock detection system, it is reasonable to say that refactoring the deadlock detection system is a separate RFE? >> >> Specifically, clarifying the deadlock detection system by removing redundant checks, not checking for safepoint-safe state in try_lock as well as explicitly listing special and below locks as illegal when verifying Thread::check_for_valid_safepoint_state(), regardles of whether allow_vm_block() is true or not. Sounds like a separate RFE to me! > Thanks for the additional analysis. I agree that so long as one does > what one is supposed to (e.g. these locks always need to avoid > safepoint checks), there won't be any undesired assertions. And I > also agree there won't be any bad consequences (e.g. incorrect code > possibly slipping through) from misuse, though the indicative failures > might not always be where one might prefer. > > I don't think the redundant checks are necessarily bad, as they make > it more obvious to future readers what the requirements are at various > levels. However, I agree it should be a separate RFE to do some > cleanup in this area, particularly where [non-]equality with > Mutex::special ought to be an ordered comparison. I am glad we agree in this area. Thanks for reading through the analysis. /Erik From yasuenag at gmail.com Wed Jul 12 01:51:11 2017 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Wed, 12 Jul 2017 10:51:11 +0900 Subject: [10] Buld warnings from GCC 7.1 on Fedora 26 Message-ID: Hi all, I tried to build jdk10-hs on Fedora 26 with gcc-7.1.1-3.fc26.x86_64, then I encountered some build warnings. I want to fix them. Can I file it to JBS and send review request for 10? details: 1-warning: /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/code/dependencies.cpp: In static member function 'static void Dependencies::write_dependency_to(xmlStream*, Dependencies::DepType, GrowableArray*, Klass*)': /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/code/dependencies.cpp:714:6: warning: '%d' directive writing between 1 and 10 bytes into a region of size 9 [-Wformat-overflow=] void Dependencies::write_dependency_to(xmlStream* xtty, ^~~~~~~~~~~~ /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/code/dependencies.cpp:714:6: note: directive argument in the range [0, 2147483647] /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/code/dependencies.cpp:745:27: note: 'sprintf' output between 3 and 12 bytes into a destination of size 10 char xn[10]; sprintf(xn, "x%d", j); ~~~~~~~^~~~~~~~~~~~~~ 1-patch: diff -r 9c54cd2cdf09 src/share/vm/code/dependencies.cpp --- a/src/share/vm/code/dependencies.cpp Mon Jul 10 23:28:25 2017 +0200 +++ b/src/share/vm/code/dependencies.cpp Wed Jul 12 10:32:20 2017 +0900 @@ -742,7 +742,7 @@ xtty->object("x", arg.metadata_value()); } } else { - char xn[10]; sprintf(xn, "x%d", j); + char xn[12]; sprintf(xn, "x%d", j); if (arg.is_oop()) { xtty->object(xn, Handle(thread, arg.oop_value())); } else { 2-warning: /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/compiler/methodMatcher.cpp: In static member function 'static bool MethodMatcher::canonicalize(char*, const char*&)': /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/compiler/methodMatcher.cpp:99:22: warning: comparison between pointer and zero character constant [-Wpointer-compare] if (colon + 2 != '\0') { ^~~~ /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/compiler/methodMatcher.cpp:99:22: note: did you mean to dereference the pointer? 2-patch: diff -r 9c54cd2cdf09 src/share/vm/compiler/methodMatcher.cpp --- a/src/share/vm/compiler/methodMatcher.cpp Mon Jul 10 23:28:25 2017 +0200 +++ b/src/share/vm/compiler/methodMatcher.cpp Wed Jul 12 10:32:20 2017 +0900 @@ -96,7 +96,7 @@ bool have_colon = (colon != NULL); if (have_colon) { // Don't allow multiple '::' - if (colon + 2 != '\0') { + if (colon[2] != '\0') { if (strstr(colon+2, "::")) { error_msg = "Method pattern only allows one '::' allowed"; return false; 3-warning: /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/logging/logFileOutput.cpp: In static member function 'static void LogFileOutput::set_file_name_parameters(jlong)': /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/logging/logFileOutput.cpp:61:99: warning: format not a string literal, format string not checked [-Wformat-nonliteral] res = (int)strftime(_vm_start_time_str, sizeof(_vm_start_time_str), TimestampFormat, &local_time); ^ 3-patch: diff -r 9c54cd2cdf09 src/share/vm/logging/logFileOutput.cpp --- a/src/share/vm/logging/logFileOutput.cpp Mon Jul 10 23:28:25 2017 +0200 +++ b/src/share/vm/logging/logFileOutput.cpp Wed Jul 12 10:32:20 2017 +0900 @@ -51,6 +51,8 @@ _file_name = make_file_name(name + strlen(Prefix), _pid_str, _vm_start_time_str); } +PRAGMA_DIAG_PUSH +PRAGMA_FORMAT_NONLITERAL_IGNORED void LogFileOutput::set_file_name_parameters(jlong vm_start_time) { int res = jio_snprintf(_pid_str, sizeof(_pid_str), "%d", os::current_process_id()); assert(res > 0, "PID buffer too small"); @@ -61,6 +63,7 @@ res = (int)strftime(_vm_start_time_str, sizeof(_vm_start_time_str), TimestampFormat, &local_time); assert(res > 0, "VM start time buffer too small."); } +PRAGMA_DIAG_POP LogFileOutput::~LogFileOutput() { if (_stream != NULL) { Thanks, Yasumasa From vladimir.x.ivanov at oracle.com Wed Jul 12 17:37:19 2017 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 12 Jul 2017 20:37:19 +0300 Subject: [10] Buld warnings from GCC 7.1 on Fedora 26 In-Reply-To: References: Message-ID: > I tried to build jdk10-hs on Fedora 26 with gcc-7.1.1-3.fc26.x86_64, > then I encountered some build warnings. > I want to fix them. Can I file it to JBS and send review request for 10? Sure, feel free to take care of them. Suggested fixes look reasonable. Best regards, Vladimir Ivanov > details: > > 1-warning: > /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/code/dependencies.cpp: > In static member function 'static void > Dependencies::write_dependency_to(xmlStream*, Dependencies::DepType, > GrowableArray*, Klass*)': > /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/code/dependencies.cpp:714:6: > warning: '%d' directive writing between 1 and 10 bytes into a region > of size 9 [-Wformat-overflow=] > void Dependencies::write_dependency_to(xmlStream* xtty, > ^~~~~~~~~~~~ > /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/code/dependencies.cpp:714:6: > note: directive argument in the range [0, 2147483647] > /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/code/dependencies.cpp:745:27: > note: 'sprintf' output between 3 and 12 bytes into a destination of > size 10 > char xn[10]; sprintf(xn, "x%d", j); > ~~~~~~~^~~~~~~~~~~~~~ > > 1-patch: > diff -r 9c54cd2cdf09 src/share/vm/code/dependencies.cpp > --- a/src/share/vm/code/dependencies.cpp Mon Jul 10 23:28:25 2017 +0200 > +++ b/src/share/vm/code/dependencies.cpp Wed Jul 12 10:32:20 2017 +0900 > @@ -742,7 +742,7 @@ > xtty->object("x", arg.metadata_value()); > } > } else { > - char xn[10]; sprintf(xn, "x%d", j); > + char xn[12]; sprintf(xn, "x%d", j); > if (arg.is_oop()) { > xtty->object(xn, Handle(thread, arg.oop_value())); > } else { > > > 2-warning: > /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/compiler/methodMatcher.cpp: > In static member function 'static bool > MethodMatcher::canonicalize(char*, const char*&)': > /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/compiler/methodMatcher.cpp:99:22: > warning: comparison between pointer and zero character constant > [-Wpointer-compare] > if (colon + 2 != '\0') { > ^~~~ > /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/compiler/methodMatcher.cpp:99:22: > note: did you mean to dereference the pointer? > > 2-patch: > diff -r 9c54cd2cdf09 src/share/vm/compiler/methodMatcher.cpp > --- a/src/share/vm/compiler/methodMatcher.cpp Mon Jul 10 23:28:25 2017 +0200 > +++ b/src/share/vm/compiler/methodMatcher.cpp Wed Jul 12 10:32:20 2017 +0900 > @@ -96,7 +96,7 @@ > bool have_colon = (colon != NULL); > if (have_colon) { > // Don't allow multiple '::' > - if (colon + 2 != '\0') { > + if (colon[2] != '\0') { > if (strstr(colon+2, "::")) { > error_msg = "Method pattern only allows one '::' allowed"; > return false; > > > 3-warning: > /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/logging/logFileOutput.cpp: > In static member function 'static void > LogFileOutput::set_file_name_parameters(jlong)': > /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/logging/logFileOutput.cpp:61:99: > warning: format not a string literal, format string not checked > [-Wformat-nonliteral] > res = (int)strftime(_vm_start_time_str, sizeof(_vm_start_time_str), > TimestampFormat, &local_time); > > ^ > > 3-patch: > diff -r 9c54cd2cdf09 src/share/vm/logging/logFileOutput.cpp > --- a/src/share/vm/logging/logFileOutput.cpp Mon Jul 10 23:28:25 2017 +0200 > +++ b/src/share/vm/logging/logFileOutput.cpp Wed Jul 12 10:32:20 2017 +0900 > @@ -51,6 +51,8 @@ > _file_name = make_file_name(name + strlen(Prefix), _pid_str, > _vm_start_time_str); > } > > +PRAGMA_DIAG_PUSH > +PRAGMA_FORMAT_NONLITERAL_IGNORED > void LogFileOutput::set_file_name_parameters(jlong vm_start_time) { > int res = jio_snprintf(_pid_str, sizeof(_pid_str), "%d", > os::current_process_id()); > assert(res > 0, "PID buffer too small"); > @@ -61,6 +63,7 @@ > res = (int)strftime(_vm_start_time_str, sizeof(_vm_start_time_str), > TimestampFormat, &local_time); > assert(res > 0, "VM start time buffer too small."); > } > +PRAGMA_DIAG_POP > > LogFileOutput::~LogFileOutput() { > if (_stream != NULL) { > > > Thanks, > > Yasumasa > From kim.barrett at oracle.com Wed Jul 12 18:30:25 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 12 Jul 2017 14:30:25 -0400 Subject: [10] Buld warnings from GCC 7.1 on Fedora 26 In-Reply-To: References: Message-ID: <2085BC5B-7493-4FA4-A3BD-428CF5A887A1@oracle.com> > On Jul 11, 2017, at 9:51 PM, Yasumasa Suenaga wrote: > > Hi all, > > I tried to build jdk10-hs on Fedora 26 with gcc-7.1.1-3.fc26.x86_64, > then I encountered some build warnings. > I want to fix them. Can I file it to JBS and send review request for 10? Yes to filing issues and submitting patches or RFRs. A couple of comments on the proposed patches. > 2-warning: > /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/compiler/methodMatcher.cpp: > In static member function 'static bool > MethodMatcher::canonicalize(char*, const char*&)': > /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/compiler/methodMatcher.cpp:99:22: > warning: comparison between pointer and zero character constant > [-Wpointer-compare] > if (colon + 2 != '\0') { > ^~~~ > /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/compiler/methodMatcher.cpp:99:22: > note: did you mean to dereference the pointer? > > 2-patch: > diff -r 9c54cd2cdf09 src/share/vm/compiler/methodMatcher.cpp > --- a/src/share/vm/compiler/methodMatcher.cpp Mon Jul 10 23:28:25 2017 +0200 > +++ b/src/share/vm/compiler/methodMatcher.cpp Wed Jul 12 10:32:20 2017 +0900 > @@ -96,7 +96,7 @@ > bool have_colon = (colon != NULL); > if (have_colon) { > // Don't allow multiple '::' > - if (colon + 2 != '\0') { > + if (colon[2] != '\0') { > if (strstr(colon+2, "::")) { > error_msg = "Method pattern only allows one '::' allowed"; > return false; Already reported in JDK-8181503. There?s an RFR out already, though it?s been stalled by discussion of one of the other bullet items in that CR plus vacations. > 3-warning: > /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/logging/logFileOutput.cpp: > In static member function 'static void > LogFileOutput::set_file_name_parameters(jlong)': > /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/logging/logFileOutput.cpp:61:99: > warning: format not a string literal, format string not checked > [-Wformat-nonliteral] > res = (int)strftime(_vm_start_time_str, sizeof(_vm_start_time_str), > TimestampFormat, &local_time); > > ^ > > 3-patch: > diff -r 9c54cd2cdf09 src/share/vm/logging/logFileOutput.cpp > --- a/src/share/vm/logging/logFileOutput.cpp Mon Jul 10 23:28:25 2017 +0200 > +++ b/src/share/vm/logging/logFileOutput.cpp Wed Jul 12 10:32:20 2017 +0900 > @@ -51,6 +51,8 @@ > _file_name = make_file_name(name + strlen(Prefix), _pid_str, > _vm_start_time_str); > } > > +PRAGMA_DIAG_PUSH > +PRAGMA_FORMAT_NONLITERAL_IGNORED > void LogFileOutput::set_file_name_parameters(jlong vm_start_time) { > int res = jio_snprintf(_pid_str, sizeof(_pid_str), "%d", > os::current_process_id()); > assert(res > 0, "PID buffer too small"); > @@ -61,6 +63,7 @@ > res = (int)strftime(_vm_start_time_str, sizeof(_vm_start_time_str), > TimestampFormat, &local_time); > assert(res > 0, "VM start time buffer too small."); > } > +PRAGMA_DIAG_POP > > LogFileOutput::~LogFileOutput() { > if (_stream != NULL) { Could this one instead be fixed by changing the declaration of TimestampFormat from static const char* TimestampFormat; to static const char* const TimestampFormat; with a corresponding update of the definition? Some web searching suggests that should work, and if it does, I?d much prefer it to the warning suppression. If that doesn?t work, then the warning suppression should be commented to indicate exactly what the problem is, e.g. that TimestampFormat is triggering -Wformat-nonliteral. Otherwise, it?s not immediately obvious where the problem lies, and the warning control scope being the entire function doesn?t help. (I don?t remember, can the warning scope be narrowed to just the offending statement?) From yasuenag at gmail.com Thu Jul 13 01:03:13 2017 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Thu, 13 Jul 2017 10:03:13 +0900 Subject: [10] RFR: 8184309: Buld warnings from GCC 7.1 on Fedora 26 Message-ID: Thanks Kim, Vladimir, > Could this one instead be fixed by changing the declaration of TimestampFormat from > > static const char* TimestampFormat; > > to > > static const char* const TimestampFormat; It works fine. So I use it. I filed this issue as JDK-8184309, and uploaded webrev. It does not contain the warning of methodMatcher.cpp . Could you review it? http://cr.openjdk.java.net/~ysuenaga/JDK-8184309/webrev.00/ I cannot access JPRT. So I need a sponsor. Yasumasa 2017-07-13 3:30 GMT+09:00 Kim Barrett : >> On Jul 11, 2017, at 9:51 PM, Yasumasa Suenaga wrote: >> >> Hi all, >> >> I tried to build jdk10-hs on Fedora 26 with gcc-7.1.1-3.fc26.x86_64, >> then I encountered some build warnings. >> I want to fix them. Can I file it to JBS and send review request for 10? > > Yes to filing issues and submitting patches or RFRs. > > A couple of comments on the proposed patches. > >> 2-warning: >> /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/compiler/methodMatcher.cpp: >> In static member function 'static bool >> MethodMatcher::canonicalize(char*, const char*&)': >> /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/compiler/methodMatcher.cpp:99:22: >> warning: comparison between pointer and zero character constant >> [-Wpointer-compare] >> if (colon + 2 != '\0') { >> ^~~~ >> /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/compiler/methodMatcher.cpp:99:22: >> note: did you mean to dereference the pointer? >> >> 2-patch: >> diff -r 9c54cd2cdf09 src/share/vm/compiler/methodMatcher.cpp >> --- a/src/share/vm/compiler/methodMatcher.cpp Mon Jul 10 23:28:25 2017 +0200 >> +++ b/src/share/vm/compiler/methodMatcher.cpp Wed Jul 12 10:32:20 2017 +0900 >> @@ -96,7 +96,7 @@ >> bool have_colon = (colon != NULL); >> if (have_colon) { >> // Don't allow multiple '::' >> - if (colon + 2 != '\0') { >> + if (colon[2] != '\0') { >> if (strstr(colon+2, "::")) { >> error_msg = "Method pattern only allows one '::' allowed"; >> return false; > > Already reported in JDK-8181503. There?s an RFR out already, though it?s been stalled > by discussion of one of the other bullet items in that CR plus vacations. > >> 3-warning: >> /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/logging/logFileOutput.cpp: >> In static member function 'static void >> LogFileOutput::set_file_name_parameters(jlong)': >> /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/logging/logFileOutput.cpp:61:99: >> warning: format not a string literal, format string not checked >> [-Wformat-nonliteral] >> res = (int)strftime(_vm_start_time_str, sizeof(_vm_start_time_str), >> TimestampFormat, &local_time); >> >> ^ >> >> 3-patch: >> diff -r 9c54cd2cdf09 src/share/vm/logging/logFileOutput.cpp >> --- a/src/share/vm/logging/logFileOutput.cpp Mon Jul 10 23:28:25 2017 +0200 >> +++ b/src/share/vm/logging/logFileOutput.cpp Wed Jul 12 10:32:20 2017 +0900 >> @@ -51,6 +51,8 @@ >> _file_name = make_file_name(name + strlen(Prefix), _pid_str, >> _vm_start_time_str); >> } >> >> +PRAGMA_DIAG_PUSH >> +PRAGMA_FORMAT_NONLITERAL_IGNORED >> void LogFileOutput::set_file_name_parameters(jlong vm_start_time) { >> int res = jio_snprintf(_pid_str, sizeof(_pid_str), "%d", >> os::current_process_id()); >> assert(res > 0, "PID buffer too small"); >> @@ -61,6 +63,7 @@ >> res = (int)strftime(_vm_start_time_str, sizeof(_vm_start_time_str), >> TimestampFormat, &local_time); >> assert(res > 0, "VM start time buffer too small."); >> } >> +PRAGMA_DIAG_POP >> >> LogFileOutput::~LogFileOutput() { >> if (_stream != NULL) { > > Could this one instead be fixed by changing the declaration of TimestampFormat from > > static const char* TimestampFormat; > > to > > static const char* const TimestampFormat; > > with a corresponding update of the definition? Some web searching suggests that > should work, and if it does, I?d much prefer it to the warning suppression. > > If that doesn?t work, then the warning suppression should be commented to indicate > exactly what the problem is, e.g. that TimestampFormat is triggering -Wformat-nonliteral. > Otherwise, it?s not immediately obvious where the problem lies, and the warning > control scope being the entire function doesn?t help. (I don?t remember, can the > warning scope be narrowed to just the offending statement?) > From thomas.stuefe at gmail.com Thu Jul 13 06:18:18 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 13 Jul 2017 08:18:18 +0200 Subject: RFR(s): 8179327: gtestLauncher should run tests on a separate thread (optionally) In-Reply-To: References: <438277b7-921c-f4e3-ee12-201ad45e3eef@oracle.com> Message-ID: Ping.. May I please get a second review and a sponsor? Thanks, Thomas On Tue, Jul 4, 2017 at 11:12 AM, Thomas St?fe wrote: > > > On Tue, Jul 4, 2017 at 10:22 AM, Robbin Ehn wrote: > >> Looks reasonable, thanks! >> >> > Thank you! > > >> /Robbin >> >> If you don't mind, please don't wrap lines, links do not work well with >> line breaks in them :) >> >> > Oops, sorry! > > >> On 07/02/2017 09:23 AM, Thomas St?fe wrote: >> >>> Dear all, >>> >>> please review the following change: >>> >>> Bug: JDK-8179327 >>> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/ >>> 8179327-gtestLauncher-should-have-an-option-to-avoid- >>> primordial-thread-for-tests/webrev.00/webrev/ >>> >>> This fix is motivated by the inability of the AIX openjdk vm to run on >>> the >>> primordial thread (see https://bugs.openjdk.java.net/browse/JDK-8171505 >>> ), >>> which we unfortunately cannot do much about. This prevented us until now >>> to >>> run gtests successfully, because the gtestlauncher (in contrast to the >>> standard java launcher) invokes the VM on the primordial thread. >>> >>> To fix this, I gave the gtest launcher the ability to run the tests in an >>> own thread. Instead of sprinkling the source with "#ifdef _AIX", I added >>> a >>> command line option "-new-thread", which allows to start the tests in a >>> new >>> thread, and implemented this feature for all platforms (well, posix and >>> windows). On all platforms save AIX this is by default off, so nothing >>> changes. On AIX, tests will be default be executed in a new thread. >>> >>> Thanks for reviewing, >>> >>> Kind Regards, Thomas >>> >>> > From thomas.stuefe at gmail.com Thu Jul 13 06:46:12 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 13 Jul 2017 08:46:12 +0200 Subject: RFR(XL): 8181917: Refactor UL LogStreams to avoid using resource area In-Reply-To: References: <337330d2-512c-bc77-5a55-c05d33e376e5@oracle.com> Message-ID: Hi Erik! On Mon, Jul 10, 2017 at 11:40 AM, Erik Helin wrote: > Hi Thomas, > > On 06/30/2017 08:32 PM, Thomas St?fe wrote: > >> Hi Eric, >> >> thank you for the review! >> >> New >> Version: http://cr.openjdk.java.net/~stuefe/webrevs/8181917-refactor- >> ul-logstream/all.webrev.03/webrev/ >> > > I think you still have a problem with "runaway" loggers in this version: > > +// try_ensure_cap tries to enlarge the capacity of the internal buffer > +// to the given atleast value. May fail if either OOM happens or atleast > +// is larger than a reasonable max of 1 M. Caller must not assume > +// capacity without checking. > +void LogStream::LineBuffer::try_ensure_cap(size_t atleast) { > + assert(_cap >= sizeof(_smallbuf), "sanity"); > + if (_cap < atleast) { > + const size_t reasonable_max = 1 * M; > + size_t newcap = align_size_up(atleast + 64, 64); > + assert(_cap <= reasonable_max, "sanity"); > + // Cap out at a reasonable max to prevent runaway leaks. > + if (newcap > reasonable_max) { > + newcap = reasonable_max; > + } > + > + char* const newbuf = (char*) os::malloc(newcap, mtLogging); > + if (newbuf == NULL) { // OOM. Leave object unchanged. > + return; > + } > + if (_pos > 0) { // preserve old content > + memcpy(newbuf, _buf, _pos + 1); // ..including trailing zero > + } > + if (_buf != _smallbuf) { > + os::free(_buf); > + } > + _buf = newbuf; > + _cap = newcap; > + } > + assert(_cap >= atleast, "sanity"); > +} > > with the above code, even though _cap > reasonable_max, you will always > allocate newbuf with os::malloc. For debug builds, we are fine, since the > JVM will assert on _cap <= reasonable_max, but for product builds we are > not. Would it be better to first check if _cap == reasonable_max? As in the > following snippet: > > void LogStream::LineBuffer::try_ensure_cap(size_t atleast) { > if (_cap < atleast) { > const size_t reasonable_max = 1 * M; > if (_cap == reasonable_max) { > // max memory used, "die" in debug builds, log in product builds > log_warning(logging)("Max memory used for message: %s\n", _buf); > ShouldNotReachHere(); > return; > } > > size_t newcap = align_size_up(atleast + 64, 64); > if (newcap > reasonable_max) { > newcap = reasonable_max; > } > char* const newbuf = (char*) os::malloc(newcap, mtLogging); > // rest of logic > } > } > > Thanks, > Erik > I'm not sure I understand. Are you concerned that _cap would grow beyond reasonable_max - because I do not see a way this could happen. Or do you want to also treat the *attempt* to grow the _cap beyond reasonable_max as a loggable offence? If the latter, I am not sure about logging. If this is an error which may happen at a customer site without popping up first in inhouse-tests (which it conceivably is, because logging is very content dependent), I would rather hide the error in production code instead of flooding stderr with tons of log warnings or crashing with a guarantee, no? ..Thomas From matthias.baesken at sap.com Thu Jul 13 09:21:12 2017 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 13 Jul 2017 09:21:12 +0000 Subject: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build Message-ID: Hello, when building jdk10 on Suse Linux 11 with default gcc/g++ 4.3.4 installed, I was running into compilation errors because of the missing support for g++ flag -fno-var-tracking-assignments . It seems gcc-4.6 has the -fvar-tracking-assignments / -fnovar-tracking-assignments flags , see https://gcc.gnu.org/onlinedocs/gcc-4.6.4/gcc/Option-Summary.html#Option-Summary I have no gcc-4.6 at hand but could verify that gcc-4.7 and gcc-4.8 "know" the flag. There are various solutions one could do to avoid the compilation error . 1) disallow usage of old gcc versions here : autoconf/toolchain.m4 TOOLCHAIN_MINIMUM_VERSION_gcc="4.3" ( e.g. change to minimum gcc 4.6 or gcc 4.7 supporting the flags used in the build ) 2) remove the flag -fno-var-tracking-assignments for older gcc versions here : (in a similar way it was done : http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/37e693211deb ) hotspot/make/lib/JvmOverrideFiles.gmk-32-ifeq ($(TOOLCHAIN_TYPE), gcc) hotspot/make/lib/JvmOverrideFiles.gmk:33: BUILD_LIBJVM_vmStructs.cpp_CXXFLAGS := -fno-var-tracking-assignments -O0 hotspot/make/lib/JvmOverrideFiles.gmk:34: BUILD_LIBJVM_jvmciCompilerToVM.cpp_CXXFLAGS := -fno-var-tracking-assignments hotspot/make/lib/JvmOverrideFiles.gmk-35-endif What is your preferred solution ? Thanks, Matthias From erik.joelsson at oracle.com Thu Jul 13 09:59:12 2017 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 13 Jul 2017 11:59:12 +0200 Subject: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build In-Reply-To: References: Message-ID: Hello Matthias, AFAIK, the only reason we support GCC versions older than 4.9 is for you guys at SAP, so if you would suggest dropping support, that would of course be the simplest solution. If you want to keep support but make the use of this flag optional, the preferred method is to add a test in flags.m4. We have macros defined for this. FLAGS_COMPILER_CHECK_ARGUMENTS or FLAGS_CXX_COMPILER_CHECK_ARGUMENTS would be suitable in this case. Use it to check if the flag is valid for the current compiler. If so, you can define a variable "CXXFLAGS_GCC_NO_VAR_TRACKING_ASSIGNMENTS=-fno-var-tracking-assignment" and empty otherwise, and in the makefile use this variable in the assignement. We want to avoid static shell expressions in the makefiles if possible and keep that kind of logic in configure. It's also better to explicitly check for features rather than versions. /Erik On 2017-07-13 11:21, Baesken, Matthias wrote: > Hello, when building jdk10 on Suse Linux 11 with default gcc/g++ 4.3.4 installed, I was running into > compilation errors because of the missing support for g++ flag -fno-var-tracking-assignments . > > It seems gcc-4.6 has the -fvar-tracking-assignments / -fnovar-tracking-assignments flags , see > > https://gcc.gnu.org/onlinedocs/gcc-4.6.4/gcc/Option-Summary.html#Option-Summary > > I have no gcc-4.6 at hand but could verify that gcc-4.7 and gcc-4.8 "know" the flag. > There are various solutions one could do to avoid the compilation error . > > > 1) disallow usage of old gcc versions here : > > autoconf/toolchain.m4 > > TOOLCHAIN_MINIMUM_VERSION_gcc="4.3" > > ( e.g. change to minimum gcc 4.6 or gcc 4.7 supporting the flags used in the build ) > > > 2) remove the flag -fno-var-tracking-assignments for older gcc versions here : > (in a similar way it was done : http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/37e693211deb ) > > hotspot/make/lib/JvmOverrideFiles.gmk-32-ifeq ($(TOOLCHAIN_TYPE), gcc) > hotspot/make/lib/JvmOverrideFiles.gmk:33: BUILD_LIBJVM_vmStructs.cpp_CXXFLAGS := -fno-var-tracking-assignments -O0 > hotspot/make/lib/JvmOverrideFiles.gmk:34: BUILD_LIBJVM_jvmciCompilerToVM.cpp_CXXFLAGS := -fno-var-tracking-assignments > hotspot/make/lib/JvmOverrideFiles.gmk-35-endif > > What is your preferred solution ? > > > Thanks, Matthias > > From matthias.baesken at sap.com Thu Jul 13 11:16:43 2017 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 13 Jul 2017 11:16:43 +0000 Subject: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build In-Reply-To: References: Message-ID: <5b01357c0a84470c94d7ebd21d974816@sap.com> Hi Erik, > AFAIK, the only reason we support GCC versions older than 4.9 is for you > guys at SAP, so if you would suggest dropping support, that would of > course be the simplest solution. for jdk10 it is fine for us to use minimum gcc 4.8 . That would be probably the most simple solution to avoid running into the "-fno-var-tracking-assignments" issue ( I now and then run into it when I forget to set the PATH to gcc-4.8 one should use for compiling on the SLES11 machine, so I pick up instead the default gcc 4.3.4 from the machine , configure works nicely and then the build fails in the middle of the HS make ) autoconf/toolchain.m4 TOOLCHAIN_MINIMUM_VERSION_gcc="4.8" Is there right place to adjust I think, I guess we need someone for Oracle to regenerate the generated-configure.sh files ? Best regards, Matthias -----Original Message----- From: Erik Joelsson [mailto:erik.joelsson at oracle.com] Sent: Donnerstag, 13. Juli 2017 11:59 To: Baesken, Matthias ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build Hello Matthias, AFAIK, the only reason we support GCC versions older than 4.9 is for you guys at SAP, so if you would suggest dropping support, that would of course be the simplest solution. If you want to keep support but make the use of this flag optional, the preferred method is to add a test in flags.m4. We have macros defined for this. FLAGS_COMPILER_CHECK_ARGUMENTS or FLAGS_CXX_COMPILER_CHECK_ARGUMENTS would be suitable in this case. Use it to check if the flag is valid for the current compiler. If so, you can define a variable "CXXFLAGS_GCC_NO_VAR_TRACKING_ASSIGNMENTS=-fno-var-tracking-assignment" and empty otherwise, and in the makefile use this variable in the assignement. We want to avoid static shell expressions in the makefiles if possible and keep that kind of logic in configure. It's also better to explicitly check for features rather than versions. /Erik On 2017-07-13 11:21, Baesken, Matthias wrote: > Hello, when building jdk10 on Suse Linux 11 with default gcc/g++ 4.3.4 installed, I was running into > compilation errors because of the missing support for g++ flag -fno-var-tracking-assignments . > > It seems gcc-4.6 has the -fvar-tracking-assignments / -fnovar-tracking-assignments flags , see > > https://gcc.gnu.org/onlinedocs/gcc-4.6.4/gcc/Option-Summary.html#Option-Summary > > I have no gcc-4.6 at hand but could verify that gcc-4.7 and gcc-4.8 "know" the flag. > There are various solutions one could do to avoid the compilation error . > > > 1) disallow usage of old gcc versions here : > > autoconf/toolchain.m4 > > TOOLCHAIN_MINIMUM_VERSION_gcc="4.3" > > ( e.g. change to minimum gcc 4.6 or gcc 4.7 supporting the flags used in the build ) > > > 2) remove the flag -fno-var-tracking-assignments for older gcc versions here : > (in a similar way it was done : http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/37e693211deb ) > > hotspot/make/lib/JvmOverrideFiles.gmk-32-ifeq ($(TOOLCHAIN_TYPE), gcc) > hotspot/make/lib/JvmOverrideFiles.gmk:33: BUILD_LIBJVM_vmStructs.cpp_CXXFLAGS := -fno-var-tracking-assignments -O0 > hotspot/make/lib/JvmOverrideFiles.gmk:34: BUILD_LIBJVM_jvmciCompilerToVM.cpp_CXXFLAGS := -fno-var-tracking-assignments > hotspot/make/lib/JvmOverrideFiles.gmk-35-endif > > What is your preferred solution ? > > > Thanks, Matthias > > From erik.joelsson at oracle.com Thu Jul 13 11:33:38 2017 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 13 Jul 2017 13:33:38 +0200 Subject: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build In-Reply-To: <5b01357c0a84470c94d7ebd21d974816@sap.com> References: <5b01357c0a84470c94d7ebd21d974816@sap.com> Message-ID: <62771b89-5d6a-0bc4-2b38-35278bb7950c@oracle.com> Hello, That would be the correct place. If you prepare the change and send the review I can sponsor the push. /Erik On 2017-07-13 13:16, Baesken, Matthias wrote: > Hi Erik, > >> AFAIK, the only reason we support GCC versions older than 4.9 is for you >> guys at SAP, so if you would suggest dropping support, that would of >> course be the simplest solution. > for jdk10 it is fine for us to use minimum gcc 4.8 . That would be probably the most simple solution to avoid running into the "-fno-var-tracking-assignments" issue > ( I now and then run into it when I forget to set the PATH to gcc-4.8 one should use for compiling on the SLES11 machine, so I pick up instead the default gcc 4.3.4 from the machine , > configure works nicely and then the build fails in the middle of the HS make ) > > autoconf/toolchain.m4 > > TOOLCHAIN_MINIMUM_VERSION_gcc="4.8" > > Is there right place to adjust I think, I guess we need someone for Oracle to regenerate the generated-configure.sh files ? > > Best regards, Matthias > > > > > -----Original Message----- > From: Erik Joelsson [mailto:erik.joelsson at oracle.com] > Sent: Donnerstag, 13. Juli 2017 11:59 > To: Baesken, Matthias ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' > Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build > > Hello Matthias, > > AFAIK, the only reason we support GCC versions older than 4.9 is for you > guys at SAP, so if you would suggest dropping support, that would of > course be the simplest solution. > > If you want to keep support but make the use of this flag optional, the > preferred method is to add a test in flags.m4. We have macros defined > for this. FLAGS_COMPILER_CHECK_ARGUMENTS or > FLAGS_CXX_COMPILER_CHECK_ARGUMENTS would be suitable in this case. Use > it to check if the flag is valid for the current compiler. If so, you > can define a variable > "CXXFLAGS_GCC_NO_VAR_TRACKING_ASSIGNMENTS=-fno-var-tracking-assignment" > and empty otherwise, and in the makefile use this variable in the > assignement. > > We want to avoid static shell expressions in the makefiles if possible > and keep that kind of logic in configure. It's also better to explicitly > check for features rather than versions. > > /Erik > > > On 2017-07-13 11:21, Baesken, Matthias wrote: >> Hello, when building jdk10 on Suse Linux 11 with default gcc/g++ 4.3.4 installed, I was running into >> compilation errors because of the missing support for g++ flag -fno-var-tracking-assignments . >> >> It seems gcc-4.6 has the -fvar-tracking-assignments / -fnovar-tracking-assignments flags , see >> >> https://gcc.gnu.org/onlinedocs/gcc-4.6.4/gcc/Option-Summary.html#Option-Summary >> >> I have no gcc-4.6 at hand but could verify that gcc-4.7 and gcc-4.8 "know" the flag. >> There are various solutions one could do to avoid the compilation error . >> >> >> 1) disallow usage of old gcc versions here : >> >> autoconf/toolchain.m4 >> >> TOOLCHAIN_MINIMUM_VERSION_gcc="4.3" >> >> ( e.g. change to minimum gcc 4.6 or gcc 4.7 supporting the flags used in the build ) >> >> >> 2) remove the flag -fno-var-tracking-assignments for older gcc versions here : >> (in a similar way it was done : http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/37e693211deb ) >> >> hotspot/make/lib/JvmOverrideFiles.gmk-32-ifeq ($(TOOLCHAIN_TYPE), gcc) >> hotspot/make/lib/JvmOverrideFiles.gmk:33: BUILD_LIBJVM_vmStructs.cpp_CXXFLAGS := -fno-var-tracking-assignments -O0 >> hotspot/make/lib/JvmOverrideFiles.gmk:34: BUILD_LIBJVM_jvmciCompilerToVM.cpp_CXXFLAGS := -fno-var-tracking-assignments >> hotspot/make/lib/JvmOverrideFiles.gmk-35-endif >> >> What is your preferred solution ? >> >> >> Thanks, Matthias >> >> From matthias.baesken at sap.com Thu Jul 13 12:18:56 2017 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 13 Jul 2017 12:18:56 +0000 Subject: RFR: compile-time guard some UseSharedSpaces-only coding with the INCLUDE_CDS macro - was :RE: jdk10 : UseSharedSpaces flag and INCLUDE_CDS macro Message-ID: <4d2f3964b0924ffa897074759e508653@sap.com> > Thank you for noticing that. Yes, it would be helpful if you can add the #if INCLUDE_CDS for CDS only code. > I can help you integrate the changes after they are reviewed. Thanks for your feedback ! I created a bug : https://bugs.openjdk.java.net/browse/JDK-8184323 and a webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8184323/ Best regards, Matthias -----Original Message----- From: Jiangli Zhou [mailto:jiangli.zhou at oracle.com] Sent: Montag, 10. Juli 2017 17:54 To: Baesken, Matthias Cc: hotspot-dev at openjdk.java.net; build-dev at openjdk.java.net Subject: Re: jdk10 : UseSharedSpaces flag and INCLUDE_CDS macro Hi Matthias, Thank you for noticing that. Yes, it would be helpful if you can add the #if INCLUDE_CDS for CDS only code. I can help you integrate the changes after they are reviewed. Thanks! Jiangli > On Jul 5, 2017, at 6:36 AM, Baesken, Matthias wrote: > > Hello, when looking into CDS related build options, I noticed that most code-parts that are executed only when UseSharedSpaces is set, > are guarded by the compile-time macro INCLUDE_CDS to support switching off compilation of this coding > when CDS is disabled at compile time : > > > See hotspot/make/lib/JvmFeatures.gmk : > > ifneq ($(call check-jvm-feature, cds), true) > JVM_CFLAGS_FEATURES += -DINCLUDE_CDS=0 > > > > However some places miss the compile-time guarding ( #if INCLUDE_CDS ....) for example in > > > share/vm/prims/jvmtiRedefineClasses.cpp > share/vm/memory/universe.cpp > > (also some other places) > > > Should I prepare a change and add the compile-time guard at the places where missing as well ? > > Best regards, Matthias From vladimir.x.ivanov at oracle.com Thu Jul 13 12:31:50 2017 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Thu, 13 Jul 2017 15:31:50 +0300 Subject: [10] RFR: 8184309: Buld warnings from GCC 7.1 on Fedora 26 In-Reply-To: References: Message-ID: <5abf8d2b-9552-8676-ebfe-13b93edaaf43@oracle.com> > http://cr.openjdk.java.net/~ysuenaga/JDK-8184309/webrev.00/ Looks good. I've submitted a JPRT build for testing. Best regards, Vladimir Ivanov > 2017-07-13 3:30 GMT+09:00 Kim Barrett : >>> On Jul 11, 2017, at 9:51 PM, Yasumasa Suenaga wrote: >>> >>> Hi all, >>> >>> I tried to build jdk10-hs on Fedora 26 with gcc-7.1.1-3.fc26.x86_64, >>> then I encountered some build warnings. >>> I want to fix them. Can I file it to JBS and send review request for 10? >> >> Yes to filing issues and submitting patches or RFRs. >> >> A couple of comments on the proposed patches. >> >>> 2-warning: >>> /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/compiler/methodMatcher.cpp: >>> In static member function 'static bool >>> MethodMatcher::canonicalize(char*, const char*&)': >>> /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/compiler/methodMatcher.cpp:99:22: >>> warning: comparison between pointer and zero character constant >>> [-Wpointer-compare] >>> if (colon + 2 != '\0') { >>> ^~~~ >>> /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/compiler/methodMatcher.cpp:99:22: >>> note: did you mean to dereference the pointer? >>> >>> 2-patch: >>> diff -r 9c54cd2cdf09 src/share/vm/compiler/methodMatcher.cpp >>> --- a/src/share/vm/compiler/methodMatcher.cpp Mon Jul 10 23:28:25 2017 +0200 >>> +++ b/src/share/vm/compiler/methodMatcher.cpp Wed Jul 12 10:32:20 2017 +0900 >>> @@ -96,7 +96,7 @@ >>> bool have_colon = (colon != NULL); >>> if (have_colon) { >>> // Don't allow multiple '::' >>> - if (colon + 2 != '\0') { >>> + if (colon[2] != '\0') { >>> if (strstr(colon+2, "::")) { >>> error_msg = "Method pattern only allows one '::' allowed"; >>> return false; >> >> Already reported in JDK-8181503. There?s an RFR out already, though it?s been stalled >> by discussion of one of the other bullet items in that CR plus vacations. >> >>> 3-warning: >>> /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/logging/logFileOutput.cpp: >>> In static member function 'static void >>> LogFileOutput::set_file_name_parameters(jlong)': >>> /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/logging/logFileOutput.cpp:61:99: >>> warning: format not a string literal, format string not checked >>> [-Wformat-nonliteral] >>> res = (int)strftime(_vm_start_time_str, sizeof(_vm_start_time_str), >>> TimestampFormat, &local_time); >>> >>> ^ >>> >>> 3-patch: >>> diff -r 9c54cd2cdf09 src/share/vm/logging/logFileOutput.cpp >>> --- a/src/share/vm/logging/logFileOutput.cpp Mon Jul 10 23:28:25 2017 +0200 >>> +++ b/src/share/vm/logging/logFileOutput.cpp Wed Jul 12 10:32:20 2017 +0900 >>> @@ -51,6 +51,8 @@ >>> _file_name = make_file_name(name + strlen(Prefix), _pid_str, >>> _vm_start_time_str); >>> } >>> >>> +PRAGMA_DIAG_PUSH >>> +PRAGMA_FORMAT_NONLITERAL_IGNORED >>> void LogFileOutput::set_file_name_parameters(jlong vm_start_time) { >>> int res = jio_snprintf(_pid_str, sizeof(_pid_str), "%d", >>> os::current_process_id()); >>> assert(res > 0, "PID buffer too small"); >>> @@ -61,6 +63,7 @@ >>> res = (int)strftime(_vm_start_time_str, sizeof(_vm_start_time_str), >>> TimestampFormat, &local_time); >>> assert(res > 0, "VM start time buffer too small."); >>> } >>> +PRAGMA_DIAG_POP >>> >>> LogFileOutput::~LogFileOutput() { >>> if (_stream != NULL) { >> >> Could this one instead be fixed by changing the declaration of TimestampFormat from >> >> static const char* TimestampFormat; >> >> to >> >> static const char* const TimestampFormat; >> >> with a corresponding update of the definition? Some web searching suggests that >> should work, and if it does, I?d much prefer it to the warning suppression. >> >> If that doesn?t work, then the warning suppression should be commented to indicate >> exactly what the problem is, e.g. that TimestampFormat is triggering -Wformat-nonliteral. >> Otherwise, it?s not immediately obvious where the problem lies, and the warning >> control scope being the entire function doesn?t help. (I don?t remember, can the >> warning scope be narrowed to just the offending statement?) >> From matthias.baesken at sap.com Thu Jul 13 13:40:52 2017 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 13 Jul 2017 13:40:52 +0000 Subject: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build Message-ID: <6c85b34b225e417788f794a99269e9fb@sap.com> Hi Erik, I prepared the change : http://cr.openjdk.java.net/~mbaesken/webrevs/8184338/ bug : https://bugs.openjdk.java.net/browse/JDK-8184338 Asking for review(s) ... Best regards, Matthias -----Original Message----- From: Erik Joelsson [mailto:erik.joelsson at oracle.com] Sent: Donnerstag, 13. Juli 2017 13:34 To: Baesken, Matthias ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build Hello, That would be the correct place. If you prepare the change and send the review I can sponsor the push. /Erik On 2017-07-13 13:16, Baesken, Matthias wrote: > Hi Erik, > >> AFAIK, the only reason we support GCC versions older than 4.9 is for you >> guys at SAP, so if you would suggest dropping support, that would of >> course be the simplest solution. > for jdk10 it is fine for us to use minimum gcc 4.8 . That would be probably the most simple solution to avoid running into the "-fno-var-tracking-assignments" issue > ( I now and then run into it when I forget to set the PATH to gcc-4.8 one should use for compiling on the SLES11 machine, so I pick up instead the default gcc 4.3.4 from the machine , > configure works nicely and then the build fails in the middle of the HS make ) > > autoconf/toolchain.m4 > > TOOLCHAIN_MINIMUM_VERSION_gcc="4.8" > > Is there right place to adjust I think, I guess we need someone for Oracle to regenerate the generated-configure.sh files ? > > Best regards, Matthias > > > > > -----Original Message----- > From: Erik Joelsson [mailto:erik.joelsson at oracle.com] > Sent: Donnerstag, 13. Juli 2017 11:59 > To: Baesken, Matthias ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' > Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build > > Hello Matthias, > > AFAIK, the only reason we support GCC versions older than 4.9 is for you > guys at SAP, so if you would suggest dropping support, that would of > course be the simplest solution. > > If you want to keep support but make the use of this flag optional, the > preferred method is to add a test in flags.m4. We have macros defined > for this. FLAGS_COMPILER_CHECK_ARGUMENTS or > FLAGS_CXX_COMPILER_CHECK_ARGUMENTS would be suitable in this case. Use > it to check if the flag is valid for the current compiler. If so, you > can define a variable > "CXXFLAGS_GCC_NO_VAR_TRACKING_ASSIGNMENTS=-fno-var-tracking-assignment" > and empty otherwise, and in the makefile use this variable in the > assignement. > > We want to avoid static shell expressions in the makefiles if possible > and keep that kind of logic in configure. It's also better to explicitly > check for features rather than versions. > > /Erik > > > On 2017-07-13 11:21, Baesken, Matthias wrote: >> Hello, when building jdk10 on Suse Linux 11 with default gcc/g++ 4.3.4 installed, I was running into >> compilation errors because of the missing support for g++ flag -fno-var-tracking-assignments . >> >> It seems gcc-4.6 has the -fvar-tracking-assignments / -fnovar-tracking-assignments flags , see >> >> https://gcc.gnu.org/onlinedocs/gcc-4.6.4/gcc/Option-Summary.html#Option-Summary >> >> I have no gcc-4.6 at hand but could verify that gcc-4.7 and gcc-4.8 "know" the flag. >> There are various solutions one could do to avoid the compilation error . >> >> >> 1) disallow usage of old gcc versions here : >> >> autoconf/toolchain.m4 >> >> TOOLCHAIN_MINIMUM_VERSION_gcc="4.3" >> >> ( e.g. change to minimum gcc 4.6 or gcc 4.7 supporting the flags used in the build ) >> >> >> 2) remove the flag -fno-var-tracking-assignments for older gcc versions here : >> (in a similar way it was done : http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/37e693211deb ) >> >> hotspot/make/lib/JvmOverrideFiles.gmk-32-ifeq ($(TOOLCHAIN_TYPE), gcc) >> hotspot/make/lib/JvmOverrideFiles.gmk:33: BUILD_LIBJVM_vmStructs.cpp_CXXFLAGS := -fno-var-tracking-assignments -O0 >> hotspot/make/lib/JvmOverrideFiles.gmk:34: BUILD_LIBJVM_jvmciCompilerToVM.cpp_CXXFLAGS := -fno-var-tracking-assignments >> hotspot/make/lib/JvmOverrideFiles.gmk-35-endif >> >> What is your preferred solution ? >> >> >> Thanks, Matthias >> >> From vladimir.x.ivanov at oracle.com Thu Jul 13 14:32:42 2017 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Thu, 13 Jul 2017 17:32:42 +0300 Subject: [10] RFR: 8184309: Buld warnings from GCC 7.1 on Fedora 26 In-Reply-To: <5abf8d2b-9552-8676-ebfe-13b93edaaf43@oracle.com> References: <5abf8d2b-9552-8676-ebfe-13b93edaaf43@oracle.com> Message-ID: <4b0826d0-9a1d-79ac-5751-c65b35232001@oracle.com> >> http://cr.openjdk.java.net/~ysuenaga/JDK-8184309/webrev.00/ > > Looks good. I've submitted a JPRT build for testing. FTR the job finished successfully. I'll wait for second review before the push. Best regards, Vladimir Ivanov >> 2017-07-13 3:30 GMT+09:00 Kim Barrett : >>>> On Jul 11, 2017, at 9:51 PM, Yasumasa Suenaga >>>> wrote: >>>> >>>> Hi all, >>>> >>>> I tried to build jdk10-hs on Fedora 26 with gcc-7.1.1-3.fc26.x86_64, >>>> then I encountered some build warnings. >>>> I want to fix them. Can I file it to JBS and send review request for >>>> 10? >>> >>> Yes to filing issues and submitting patches or RFRs. >>> >>> A couple of comments on the proposed patches. >>> >>>> 2-warning: >>>> /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/compiler/methodMatcher.cpp: >>>> >>>> In static member function 'static bool >>>> MethodMatcher::canonicalize(char*, const char*&)': >>>> /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/compiler/methodMatcher.cpp:99:22: >>>> >>>> warning: comparison between pointer and zero character constant >>>> [-Wpointer-compare] >>>> if (colon + 2 != '\0') { >>>> ^~~~ >>>> /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/compiler/methodMatcher.cpp:99:22: >>>> >>>> note: did you mean to dereference the pointer? >>>> >>>> 2-patch: >>>> diff -r 9c54cd2cdf09 src/share/vm/compiler/methodMatcher.cpp >>>> --- a/src/share/vm/compiler/methodMatcher.cpp Mon Jul 10 23:28:25 >>>> 2017 +0200 >>>> +++ b/src/share/vm/compiler/methodMatcher.cpp Wed Jul 12 10:32:20 >>>> 2017 +0900 >>>> @@ -96,7 +96,7 @@ >>>> bool have_colon = (colon != NULL); >>>> if (have_colon) { >>>> // Don't allow multiple '::' >>>> - if (colon + 2 != '\0') { >>>> + if (colon[2] != '\0') { >>>> if (strstr(colon+2, "::")) { >>>> error_msg = "Method pattern only allows one '::' allowed"; >>>> return false; >>> >>> Already reported in JDK-8181503. There?s an RFR out already, though >>> it?s been stalled >>> by discussion of one of the other bullet items in that CR plus >>> vacations. >>> >>>> 3-warning: >>>> /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/logging/logFileOutput.cpp: >>>> >>>> In static member function 'static void >>>> LogFileOutput::set_file_name_parameters(jlong)': >>>> /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/logging/logFileOutput.cpp:61:99: >>>> >>>> warning: format not a string literal, format string not checked >>>> [-Wformat-nonliteral] >>>> res = (int)strftime(_vm_start_time_str, sizeof(_vm_start_time_str), >>>> TimestampFormat, &local_time); >>>> >>>> ^ >>>> >>>> 3-patch: >>>> diff -r 9c54cd2cdf09 src/share/vm/logging/logFileOutput.cpp >>>> --- a/src/share/vm/logging/logFileOutput.cpp Mon Jul 10 23:28:25 >>>> 2017 +0200 >>>> +++ b/src/share/vm/logging/logFileOutput.cpp Wed Jul 12 10:32:20 >>>> 2017 +0900 >>>> @@ -51,6 +51,8 @@ >>>> _file_name = make_file_name(name + strlen(Prefix), _pid_str, >>>> _vm_start_time_str); >>>> } >>>> >>>> +PRAGMA_DIAG_PUSH >>>> +PRAGMA_FORMAT_NONLITERAL_IGNORED >>>> void LogFileOutput::set_file_name_parameters(jlong vm_start_time) { >>>> int res = jio_snprintf(_pid_str, sizeof(_pid_str), "%d", >>>> os::current_process_id()); >>>> assert(res > 0, "PID buffer too small"); >>>> @@ -61,6 +63,7 @@ >>>> res = (int)strftime(_vm_start_time_str, sizeof(_vm_start_time_str), >>>> TimestampFormat, &local_time); >>>> assert(res > 0, "VM start time buffer too small."); >>>> } >>>> +PRAGMA_DIAG_POP >>>> >>>> LogFileOutput::~LogFileOutput() { >>>> if (_stream != NULL) { >>> >>> Could this one instead be fixed by changing the declaration of >>> TimestampFormat from >>> >>> static const char* TimestampFormat; >>> >>> to >>> >>> static const char* const TimestampFormat; >>> >>> with a corresponding update of the definition? Some web searching >>> suggests that >>> should work, and if it does, I?d much prefer it to the warning >>> suppression. >>> >>> If that doesn?t work, then the warning suppression should be >>> commented to indicate >>> exactly what the problem is, e.g. that TimestampFormat is triggering >>> -Wformat-nonliteral. >>> Otherwise, it?s not immediately obvious where the problem lies, and >>> the warning >>> control scope being the entire function doesn?t help. (I don?t >>> remember, can the >>> warning scope be narrowed to just the offending statement?) >>> From erik.joelsson at oracle.com Thu Jul 13 14:44:19 2017 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 13 Jul 2017 16:44:19 +0200 Subject: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build In-Reply-To: <6c85b34b225e417788f794a99269e9fb@sap.com> References: <6c85b34b225e417788f794a99269e9fb@sap.com> Message-ID: <4c644405-a75f-1ad0-db7d-be66a6a477a9@oracle.com> Looks good to me. /Erik On 2017-07-13 15:40, Baesken, Matthias wrote: > Hi Erik, I prepared the change : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8184338/ > > bug : > > https://bugs.openjdk.java.net/browse/JDK-8184338 > > Asking for review(s) ... > > > Best regards, Matthias > > > > > -----Original Message----- > From: Erik Joelsson [mailto:erik.joelsson at oracle.com] > Sent: Donnerstag, 13. Juli 2017 13:34 > To: Baesken, Matthias ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' > Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build > > Hello, > > That would be the correct place. If you prepare the change and send the > review I can sponsor the push. > > /Erik > > On 2017-07-13 13:16, Baesken, Matthias wrote: >> Hi Erik, >> >>> AFAIK, the only reason we support GCC versions older than 4.9 is for you >>> guys at SAP, so if you would suggest dropping support, that would of >>> course be the simplest solution. >> for jdk10 it is fine for us to use minimum gcc 4.8 . That would be probably the most simple solution to avoid running into the "-fno-var-tracking-assignments" issue >> ( I now and then run into it when I forget to set the PATH to gcc-4.8 one should use for compiling on the SLES11 machine, so I pick up instead the default gcc 4.3.4 from the machine , >> configure works nicely and then the build fails in the middle of the HS make ) >> >> autoconf/toolchain.m4 >> >> TOOLCHAIN_MINIMUM_VERSION_gcc="4.8" >> >> Is there right place to adjust I think, I guess we need someone for Oracle to regenerate the generated-configure.sh files ? >> >> Best regards, Matthias >> >> >> >> >> -----Original Message----- >> From: Erik Joelsson [mailto:erik.joelsson at oracle.com] >> Sent: Donnerstag, 13. Juli 2017 11:59 >> To: Baesken, Matthias ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' >> Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build >> >> Hello Matthias, >> >> AFAIK, the only reason we support GCC versions older than 4.9 is for you >> guys at SAP, so if you would suggest dropping support, that would of >> course be the simplest solution. >> >> If you want to keep support but make the use of this flag optional, the >> preferred method is to add a test in flags.m4. We have macros defined >> for this. FLAGS_COMPILER_CHECK_ARGUMENTS or >> FLAGS_CXX_COMPILER_CHECK_ARGUMENTS would be suitable in this case. Use >> it to check if the flag is valid for the current compiler. If so, you >> can define a variable >> "CXXFLAGS_GCC_NO_VAR_TRACKING_ASSIGNMENTS=-fno-var-tracking-assignment" >> and empty otherwise, and in the makefile use this variable in the >> assignement. >> >> We want to avoid static shell expressions in the makefiles if possible >> and keep that kind of logic in configure. It's also better to explicitly >> check for features rather than versions. >> >> /Erik >> >> >> On 2017-07-13 11:21, Baesken, Matthias wrote: >>> Hello, when building jdk10 on Suse Linux 11 with default gcc/g++ 4.3.4 installed, I was running into >>> compilation errors because of the missing support for g++ flag -fno-var-tracking-assignments . >>> >>> It seems gcc-4.6 has the -fvar-tracking-assignments / -fnovar-tracking-assignments flags , see >>> >>> https://gcc.gnu.org/onlinedocs/gcc-4.6.4/gcc/Option-Summary.html#Option-Summary >>> >>> I have no gcc-4.6 at hand but could verify that gcc-4.7 and gcc-4.8 "know" the flag. >>> There are various solutions one could do to avoid the compilation error . >>> >>> >>> 1) disallow usage of old gcc versions here : >>> >>> autoconf/toolchain.m4 >>> >>> TOOLCHAIN_MINIMUM_VERSION_gcc="4.3" >>> >>> ( e.g. change to minimum gcc 4.6 or gcc 4.7 supporting the flags used in the build ) >>> >>> >>> 2) remove the flag -fno-var-tracking-assignments for older gcc versions here : >>> (in a similar way it was done : http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/37e693211deb ) >>> >>> hotspot/make/lib/JvmOverrideFiles.gmk-32-ifeq ($(TOOLCHAIN_TYPE), gcc) >>> hotspot/make/lib/JvmOverrideFiles.gmk:33: BUILD_LIBJVM_vmStructs.cpp_CXXFLAGS := -fno-var-tracking-assignments -O0 >>> hotspot/make/lib/JvmOverrideFiles.gmk:34: BUILD_LIBJVM_jvmciCompilerToVM.cpp_CXXFLAGS := -fno-var-tracking-assignments >>> hotspot/make/lib/JvmOverrideFiles.gmk-35-endif >>> >>> What is your preferred solution ? >>> >>> >>> Thanks, Matthias >>> >>> From erik.joelsson at oracle.com Thu Jul 13 14:46:07 2017 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 13 Jul 2017 16:46:07 +0200 Subject: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build In-Reply-To: <4c644405-a75f-1ad0-db7d-be66a6a477a9@oracle.com> References: <6c85b34b225e417788f794a99269e9fb@sap.com> <4c644405-a75f-1ad0-db7d-be66a6a477a9@oracle.com> Message-ID: <9145f8aa-fb2c-00eb-3f52-a03d1d860b2c@oracle.com> Just remembered, please also update the building documentation referencing 4.3 in common/docs/building.md. /Erik On 2017-07-13 16:44, Erik Joelsson wrote: > Looks good to me. > > /Erik > > > On 2017-07-13 15:40, Baesken, Matthias wrote: >> Hi Erik, I prepared the change : >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8184338/ >> >> bug : >> >> https://bugs.openjdk.java.net/browse/JDK-8184338 >> >> Asking for review(s) ... >> >> >> Best regards, Matthias >> >> >> >> >> -----Original Message----- >> From: Erik Joelsson [mailto:erik.joelsson at oracle.com] >> Sent: Donnerstag, 13. Juli 2017 13:34 >> To: Baesken, Matthias ; >> 'build-dev at openjdk.java.net' ; >> 'hotspot-dev at openjdk.java.net' >> Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments g++ >> flag in hotspot build >> >> Hello, >> >> That would be the correct place. If you prepare the change and send the >> review I can sponsor the push. >> >> /Erik >> >> On 2017-07-13 13:16, Baesken, Matthias wrote: >>> Hi Erik, >>> >>>> AFAIK, the only reason we support GCC versions older than 4.9 is >>>> for you >>>> guys at SAP, so if you would suggest dropping support, that would of >>>> course be the simplest solution. >>> for jdk10 it is fine for us to use minimum gcc 4.8 . That >>> would be probably the most simple solution to avoid running into >>> the "-fno-var-tracking-assignments" issue >>> ( I now and then run into it when I forget to set the PATH to >>> gcc-4.8 one should use for compiling on the SLES11 machine, so I >>> pick up instead the default gcc 4.3.4 from the machine , >>> configure works nicely and then the build fails in the middle of the >>> HS make ) >>> >>> autoconf/toolchain.m4 >>> >>> TOOLCHAIN_MINIMUM_VERSION_gcc="4.8" >>> >>> Is there right place to adjust I think, I guess we need someone for >>> Oracle to regenerate the generated-configure.sh files ? >>> >>> Best regards, Matthias >>> >>> >>> >>> >>> -----Original Message----- >>> From: Erik Joelsson [mailto:erik.joelsson at oracle.com] >>> Sent: Donnerstag, 13. Juli 2017 11:59 >>> To: Baesken, Matthias ; >>> 'build-dev at openjdk.java.net' ; >>> 'hotspot-dev at openjdk.java.net' >>> Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments >>> g++ flag in hotspot build >>> >>> Hello Matthias, >>> >>> AFAIK, the only reason we support GCC versions older than 4.9 is for >>> you >>> guys at SAP, so if you would suggest dropping support, that would of >>> course be the simplest solution. >>> >>> If you want to keep support but make the use of this flag optional, the >>> preferred method is to add a test in flags.m4. We have macros defined >>> for this. FLAGS_COMPILER_CHECK_ARGUMENTS or >>> FLAGS_CXX_COMPILER_CHECK_ARGUMENTS would be suitable in this case. Use >>> it to check if the flag is valid for the current compiler. If so, you >>> can define a variable >>> "CXXFLAGS_GCC_NO_VAR_TRACKING_ASSIGNMENTS=-fno-var-tracking-assignment" >>> and empty otherwise, and in the makefile use this variable in the >>> assignement. >>> >>> We want to avoid static shell expressions in the makefiles if possible >>> and keep that kind of logic in configure. It's also better to >>> explicitly >>> check for features rather than versions. >>> >>> /Erik >>> >>> >>> On 2017-07-13 11:21, Baesken, Matthias wrote: >>>> Hello, when building jdk10 on Suse Linux 11 with default gcc/g++ >>>> 4.3.4 installed, I was running into >>>> compilation errors because of the missing support for g++ flag >>>> -fno-var-tracking-assignments . >>>> >>>> It seems gcc-4.6 has the -fvar-tracking-assignments / >>>> -fnovar-tracking-assignments flags , see >>>> >>>> https://gcc.gnu.org/onlinedocs/gcc-4.6.4/gcc/Option-Summary.html#Option-Summary >>>> >>>> >>>> I have no gcc-4.6 at hand but could verify that gcc-4.7 and gcc-4.8 >>>> "know" the flag. >>>> There are various solutions one could do to avoid the compilation >>>> error . >>>> >>>> >>>> 1) disallow usage of old gcc versions here : >>>> >>>> autoconf/toolchain.m4 >>>> >>>> TOOLCHAIN_MINIMUM_VERSION_gcc="4.3" >>>> >>>> ( e.g. change to minimum gcc 4.6 or gcc 4.7 supporting the >>>> flags used in the build ) >>>> >>>> >>>> 2) remove the flag -fno-var-tracking-assignments for older gcc >>>> versions here : >>>> (in a similar way it was done : >>>> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/37e693211deb ) >>>> >>>> hotspot/make/lib/JvmOverrideFiles.gmk-32-ifeq ($(TOOLCHAIN_TYPE), gcc) >>>> hotspot/make/lib/JvmOverrideFiles.gmk:33: >>>> BUILD_LIBJVM_vmStructs.cpp_CXXFLAGS := >>>> -fno-var-tracking-assignments -O0 >>>> hotspot/make/lib/JvmOverrideFiles.gmk:34: >>>> BUILD_LIBJVM_jvmciCompilerToVM.cpp_CXXFLAGS := >>>> -fno-var-tracking-assignments >>>> hotspot/make/lib/JvmOverrideFiles.gmk-35-endif >>>> >>>> What is your preferred solution ? >>>> >>>> >>>> Thanks, Matthias >>>> >>>> > From yasuenag at gmail.com Thu Jul 13 14:46:15 2017 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Thu, 13 Jul 2017 23:46:15 +0900 Subject: [10] RFR: 8184309: Buld warnings from GCC 7.1 on Fedora 26 In-Reply-To: <4b0826d0-9a1d-79ac-5751-c65b35232001@oracle.com> References: <5abf8d2b-9552-8676-ebfe-13b93edaaf43@oracle.com> <4b0826d0-9a1d-79ac-5751-c65b35232001@oracle.com> Message-ID: Thanks Vladimir! Yasumasa On 2017/07/13 23:32, Vladimir Ivanov wrote: > > >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8184309/webrev.00/ >> >> Looks good. I've submitted a JPRT build for testing. > > FTR the job finished successfully. I'll wait for second review before the push. > > Best regards, > Vladimir Ivanov > >>> 2017-07-13 3:30 GMT+09:00 Kim Barrett : >>>>> On Jul 11, 2017, at 9:51 PM, Yasumasa Suenaga wrote: >>>>> >>>>> Hi all, >>>>> >>>>> I tried to build jdk10-hs on Fedora 26 with gcc-7.1.1-3.fc26.x86_64, >>>>> then I encountered some build warnings. >>>>> I want to fix them. Can I file it to JBS and send review request for 10? >>>> >>>> Yes to filing issues and submitting patches or RFRs. >>>> >>>> A couple of comments on the proposed patches. >>>> >>>>> 2-warning: >>>>> /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/compiler/methodMatcher.cpp: >>>>> In static member function 'static bool >>>>> MethodMatcher::canonicalize(char*, const char*&)': >>>>> /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/compiler/methodMatcher.cpp:99:22: >>>>> warning: comparison between pointer and zero character constant >>>>> [-Wpointer-compare] >>>>> if (colon + 2 != '\0') { >>>>> ^~~~ >>>>> /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/compiler/methodMatcher.cpp:99:22: >>>>> note: did you mean to dereference the pointer? >>>>> >>>>> 2-patch: >>>>> diff -r 9c54cd2cdf09 src/share/vm/compiler/methodMatcher.cpp >>>>> --- a/src/share/vm/compiler/methodMatcher.cpp Mon Jul 10 23:28:25 2017 +0200 >>>>> +++ b/src/share/vm/compiler/methodMatcher.cpp Wed Jul 12 10:32:20 2017 +0900 >>>>> @@ -96,7 +96,7 @@ >>>>> bool have_colon = (colon != NULL); >>>>> if (have_colon) { >>>>> // Don't allow multiple '::' >>>>> - if (colon + 2 != '\0') { >>>>> + if (colon[2] != '\0') { >>>>> if (strstr(colon+2, "::")) { >>>>> error_msg = "Method pattern only allows one '::' allowed"; >>>>> return false; >>>> >>>> Already reported in JDK-8181503. There?s an RFR out already, though it?s been stalled >>>> by discussion of one of the other bullet items in that CR plus vacations. >>>> >>>>> 3-warning: >>>>> /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/logging/logFileOutput.cpp: >>>>> In static member function 'static void >>>>> LogFileOutput::set_file_name_parameters(jlong)': >>>>> /home/ysuenaga/OpenJDK/jdk10-hs/hotspot/src/share/vm/logging/logFileOutput.cpp:61:99: >>>>> warning: format not a string literal, format string not checked >>>>> [-Wformat-nonliteral] >>>>> res = (int)strftime(_vm_start_time_str, sizeof(_vm_start_time_str), >>>>> TimestampFormat, &local_time); >>>>> >>>>> ^ >>>>> >>>>> 3-patch: >>>>> diff -r 9c54cd2cdf09 src/share/vm/logging/logFileOutput.cpp >>>>> --- a/src/share/vm/logging/logFileOutput.cpp Mon Jul 10 23:28:25 2017 +0200 >>>>> +++ b/src/share/vm/logging/logFileOutput.cpp Wed Jul 12 10:32:20 2017 +0900 >>>>> @@ -51,6 +51,8 @@ >>>>> _file_name = make_file_name(name + strlen(Prefix), _pid_str, >>>>> _vm_start_time_str); >>>>> } >>>>> >>>>> +PRAGMA_DIAG_PUSH >>>>> +PRAGMA_FORMAT_NONLITERAL_IGNORED >>>>> void LogFileOutput::set_file_name_parameters(jlong vm_start_time) { >>>>> int res = jio_snprintf(_pid_str, sizeof(_pid_str), "%d", >>>>> os::current_process_id()); >>>>> assert(res > 0, "PID buffer too small"); >>>>> @@ -61,6 +63,7 @@ >>>>> res = (int)strftime(_vm_start_time_str, sizeof(_vm_start_time_str), >>>>> TimestampFormat, &local_time); >>>>> assert(res > 0, "VM start time buffer too small."); >>>>> } >>>>> +PRAGMA_DIAG_POP >>>>> >>>>> LogFileOutput::~LogFileOutput() { >>>>> if (_stream != NULL) { >>>> >>>> Could this one instead be fixed by changing the declaration of TimestampFormat from >>>> >>>> static const char* TimestampFormat; >>>> >>>> to >>>> >>>> static const char* const TimestampFormat; >>>> >>>> with a corresponding update of the definition? Some web searching suggests that >>>> should work, and if it does, I?d much prefer it to the warning suppression. >>>> >>>> If that doesn?t work, then the warning suppression should be commented to indicate >>>> exactly what the problem is, e.g. that TimestampFormat is triggering -Wformat-nonliteral. >>>> Otherwise, it?s not immediately obvious where the problem lies, and the warning >>>> control scope being the entire function doesn?t help. (I don?t remember, can the >>>> warning scope be narrowed to just the offending statement?) >>>> From kim.barrett at oracle.com Thu Jul 13 17:13:33 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 13 Jul 2017 13:13:33 -0400 Subject: [10] RFR: 8184309: Buld warnings from GCC 7.1 on Fedora 26 In-Reply-To: References: Message-ID: > On Jul 12, 2017, at 9:03 PM, Yasumasa Suenaga wrote: > > Thanks Kim, Vladimir, > >> Could this one instead be fixed by changing the declaration of TimestampFormat from >> >> static const char* TimestampFormat; >> >> to >> >> static const char* const TimestampFormat; > > It works fine. > So I use it. > > I filed this issue as JDK-8184309, and uploaded webrev. > It does not contain the warning of methodMatcher.cpp . > Could you review it? > > http://cr.openjdk.java.net/~ysuenaga/JDK-8184309/webrev.00/ Thanks for cleaning these up. In context, the addition of the const qualifier to just TimestampFormat looks odd, and made me look at the others nearby. I think all those other nearby "const char*" variables ought to be "const char* const" too. If you want to make that change (and assuming I haven't overlooked something, and it builds ok), I don't need another round of review. Looks good. From simon at cjnash.com Thu Jul 13 19:56:19 2017 From: simon at cjnash.com (Simon Nash) Date: Thu, 13 Jul 2017 20:56:19 +0100 Subject: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build In-Reply-To: <6c85b34b225e417788f794a99269e9fb@sap.com> References: <6c85b34b225e417788f794a99269e9fb@sap.com> Message-ID: <5967D063.1010808@cjnash.com> I am currently building JDK 9 for arm32 (hard float and soft float) using a Linaro GCC 4.7 cross-compiler (binary download). What is the reason that this cannot be supported for JDK 10? Best regards, Simon On 13/07/2017 14:40, Baesken, Matthias wrote: > Hi Erik, I prepared the change : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8184338/ > > bug : > > https://bugs.openjdk.java.net/browse/JDK-8184338 > > Asking for review(s) ... > > > Best regards, Matthias > > > > > -----Original Message----- > From: Erik Joelsson [mailto:erik.joelsson at oracle.com] > Sent: Donnerstag, 13. Juli 2017 13:34 > To: Baesken, Matthias ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' > Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build > > Hello, > > That would be the correct place. If you prepare the change and send the > review I can sponsor the push. > > /Erik > > On 2017-07-13 13:16, Baesken, Matthias wrote: >> Hi Erik, >> >>> AFAIK, the only reason we support GCC versions older than 4.9 is for you >>> guys at SAP, so if you would suggest dropping support, that would of >>> course be the simplest solution. >> for jdk10 it is fine for us to use minimum gcc 4.8 . That would be probably the most simple solution to avoid running into the "-fno-var-tracking-assignments" issue >> ( I now and then run into it when I forget to set the PATH to gcc-4.8 one should use for compiling on the SLES11 machine, so I pick up instead the default gcc 4.3.4 from the machine , >> configure works nicely and then the build fails in the middle of the HS make ) >> >> autoconf/toolchain.m4 >> >> TOOLCHAIN_MINIMUM_VERSION_gcc="4.8" >> >> Is there right place to adjust I think, I guess we need someone for Oracle to regenerate the generated-configure.sh files ? >> >> Best regards, Matthias >> >> >> >> >> -----Original Message----- >> From: Erik Joelsson [mailto:erik.joelsson at oracle.com] >> Sent: Donnerstag, 13. Juli 2017 11:59 >> To: Baesken, Matthias ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' >> Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build >> >> Hello Matthias, >> >> AFAIK, the only reason we support GCC versions older than 4.9 is for you >> guys at SAP, so if you would suggest dropping support, that would of >> course be the simplest solution. >> >> If you want to keep support but make the use of this flag optional, the >> preferred method is to add a test in flags.m4. We have macros defined >> for this. FLAGS_COMPILER_CHECK_ARGUMENTS or >> FLAGS_CXX_COMPILER_CHECK_ARGUMENTS would be suitable in this case. Use >> it to check if the flag is valid for the current compiler. If so, you >> can define a variable >> "CXXFLAGS_GCC_NO_VAR_TRACKING_ASSIGNMENTS=-fno-var-tracking-assignment" >> and empty otherwise, and in the makefile use this variable in the >> assignement. >> >> We want to avoid static shell expressions in the makefiles if possible >> and keep that kind of logic in configure. It's also better to explicitly >> check for features rather than versions. >> >> /Erik >> >> >> On 2017-07-13 11:21, Baesken, Matthias wrote: >>> Hello, when building jdk10 on Suse Linux 11 with default gcc/g++ 4.3.4 installed, I was running into >>> compilation errors because of the missing support for g++ flag -fno-var-tracking-assignments . >>> >>> It seems gcc-4.6 has the -fvar-tracking-assignments / -fnovar-tracking-assignments flags , see >>> >>> https://gcc.gnu.org/onlinedocs/gcc-4.6.4/gcc/Option-Summary.html#Option-Summary >>> >>> I have no gcc-4.6 at hand but could verify that gcc-4.7 and gcc-4.8 "know" the flag. >>> There are various solutions one could do to avoid the compilation error . >>> >>> >>> 1) disallow usage of old gcc versions here : >>> >>> autoconf/toolchain.m4 >>> >>> TOOLCHAIN_MINIMUM_VERSION_gcc="4.3" >>> >>> ( e.g. change to minimum gcc 4.6 or gcc 4.7 supporting the flags used in the build ) >>> >>> >>> 2) remove the flag -fno-var-tracking-assignments for older gcc versions here : >>> (in a similar way it was done : http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/37e693211deb ) >>> >>> hotspot/make/lib/JvmOverrideFiles.gmk-32-ifeq ($(TOOLCHAIN_TYPE), gcc) >>> hotspot/make/lib/JvmOverrideFiles.gmk:33: BUILD_LIBJVM_vmStructs.cpp_CXXFLAGS := -fno-var-tracking-assignments -O0 >>> hotspot/make/lib/JvmOverrideFiles.gmk:34: BUILD_LIBJVM_jvmciCompilerToVM.cpp_CXXFLAGS := -fno-var-tracking-assignments >>> hotspot/make/lib/JvmOverrideFiles.gmk-35-endif >>> >>> What is your preferred solution ? >>> >>> >>> Thanks, Matthias >>> >>> > From glaubitz at physik.fu-berlin.de Thu Jul 13 20:01:17 2017 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Thu, 13 Jul 2017 22:01:17 +0200 Subject: Bug#868255: openjdk-9: Please build with --with-debug-level=slowdebug on Zero-only architectures Message-ID: <149997607774.10222.4353960267175791909.reportbug@stretch64.physik.fu-berlin.de> Source: openjdk-9 Version: 9~b177-3 Severity: normal Hi! openjdk-9 currently FTBFS on architectures which exclusively rely on the Zero VM. This happens because the JVM segfaults during build at some point [1]. The exact reason for the segmentation fault has not been discovered yet, but we know that building with "--with-debug-level=slowdebug" instead of "=release" resolves the issue. Thus, I suggest building openjdk-9 on Zero-only architectures for the time being as a work-around. Thanks, Adrian > [1] http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-June/027117.html -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From yasuenag at gmail.com Fri Jul 14 01:57:37 2017 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Fri, 14 Jul 2017 10:57:37 +0900 Subject: [10] RFR: 8184309: Buld warnings from GCC 7.1 on Fedora 26 In-Reply-To: References: Message-ID: Thanks Kim, I uploaded new webrev for this issue: http://cr.openjdk.java.net/~ysuenaga/JDK-8184309/webrev.01/ Vladimir, could you push this patch? Yasumasa 2017-07-14 2:13 GMT+09:00 Kim Barrett : >> On Jul 12, 2017, at 9:03 PM, Yasumasa Suenaga wrote: >> >> Thanks Kim, Vladimir, >> >>> Could this one instead be fixed by changing the declaration of TimestampFormat from >>> >>> static const char* TimestampFormat; >>> >>> to >>> >>> static const char* const TimestampFormat; >> >> It works fine. >> So I use it. >> >> I filed this issue as JDK-8184309, and uploaded webrev. >> It does not contain the warning of methodMatcher.cpp . >> Could you review it? >> >> http://cr.openjdk.java.net/~ysuenaga/JDK-8184309/webrev.00/ > > Thanks for cleaning these up. > > In context, the addition of the const qualifier to just > TimestampFormat looks odd, and made me look at the others nearby. > I think all those other nearby "const char*" variables ought to > be "const char* const" too. > > If you want to make that change (and assuming I haven't overlooked > something, and it builds ok), I don't need another round of review. > > Looks good. > From kim.barrett at oracle.com Fri Jul 14 03:11:33 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 13 Jul 2017 23:11:33 -0400 Subject: [10] RFR: 8184309: Buld warnings from GCC 7.1 on Fedora 26 In-Reply-To: References: Message-ID: <60251551-F66E-4A2D-B354-399C3C3D1A20@oracle.com> > On Jul 13, 2017, at 9:57 PM, Yasumasa Suenaga wrote: > > Thanks Kim, > > I uploaded new webrev for this issue: > > http://cr.openjdk.java.net/~ysuenaga/JDK-8184309/webrev.01/ > > Vladimir, could you push this patch? Before pushing, please update the copyright in logFileOutput.hpp. > Yasumasa From yasuenag at gmail.com Fri Jul 14 03:49:59 2017 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Fri, 14 Jul 2017 12:49:59 +0900 Subject: [10] RFR: 8184309: Buld warnings from GCC 7.1 on Fedora 26 In-Reply-To: <60251551-F66E-4A2D-B354-399C3C3D1A20@oracle.com> References: <60251551-F66E-4A2D-B354-399C3C3D1A20@oracle.com> Message-ID: Updated: http://cr.openjdk.java.net/~ysuenaga/JDK-8184309/webrev.02/ Yasumasa 2017-07-14 12:11 GMT+09:00 Kim Barrett : >> On Jul 13, 2017, at 9:57 PM, Yasumasa Suenaga wrote: >> >> Thanks Kim, >> >> I uploaded new webrev for this issue: >> >> http://cr.openjdk.java.net/~ysuenaga/JDK-8184309/webrev.01/ >> >> Vladimir, could you push this patch? > > Before pushing, please update the copyright in logFileOutput.hpp. > >> Yasumasa > From matthias.baesken at sap.com Fri Jul 14 06:35:51 2017 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 14 Jul 2017 06:35:51 +0000 Subject: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build In-Reply-To: <5967D063.1010808@cjnash.com> References: <6c85b34b225e417788f794a99269e9fb@sap.com> <5967D063.1010808@cjnash.com> Message-ID: <424e7257bc504fe09d967ce56d64afd1@sap.com> > I am currently building JDK 9 for arm32 (hard float and soft float) using > a Linaro GCC 4.7 cross-compiler (binary download). What is the reason that > this cannot be supported for JDK 10? Hi Simon, reason was that we know gcc-4.8 works nicely because we do a lot of builds (+tests) with this compiler . For gcc 4.7 we just do not know ( but for old 4.3 / 4.4 it was obvious that the current flags do not work any more). If you are using a gcc compiler < 4.8 only a warning is reported , it does not mean that you cannot build any more. My first suggestion was : >>> ( e.g. change to minimum gcc 4.6 or gcc 4.7 supporting the flags used in the build ) So if you see a benefit to test for minimum gcc 4.7 and not 4.8 I am fine with this too (Erik what do you think?). We at SAP just cannot tell that gcc 4.7 still works (because our builds do not use it). Best regards, Matthias -----Original Message----- From: Simon Nash [mailto:simon at cjnash.com] Sent: Donnerstag, 13. Juli 2017 21:56 To: Baesken, Matthias Cc: Erik Joelsson ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' ; Zeller, Arno Subject: Re: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build I am currently building JDK 9 for arm32 (hard float and soft float) using a Linaro GCC 4.7 cross-compiler (binary download). What is the reason that this cannot be supported for JDK 10? Best regards, Simon On 13/07/2017 14:40, Baesken, Matthias wrote: > Hi Erik, I prepared the change : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8184338/ > > bug : > > https://bugs.openjdk.java.net/browse/JDK-8184338 > > Asking for review(s) ... > > > Best regards, Matthias > > > > > -----Original Message----- > From: Erik Joelsson [mailto:erik.joelsson at oracle.com] > Sent: Donnerstag, 13. Juli 2017 13:34 > To: Baesken, Matthias ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' > Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build > > Hello, > > That would be the correct place. If you prepare the change and send the > review I can sponsor the push. > > /Erik > > On 2017-07-13 13:16, Baesken, Matthias wrote: >> Hi Erik, >> >>> AFAIK, the only reason we support GCC versions older than 4.9 is for you >>> guys at SAP, so if you would suggest dropping support, that would of >>> course be the simplest solution. >> for jdk10 it is fine for us to use minimum gcc 4.8 . That would be probably the most simple solution to avoid running into the "-fno-var-tracking-assignments" issue >> ( I now and then run into it when I forget to set the PATH to gcc-4.8 one should use for compiling on the SLES11 machine, so I pick up instead the default gcc 4.3.4 from the machine , >> configure works nicely and then the build fails in the middle of the HS make ) >> >> autoconf/toolchain.m4 >> >> TOOLCHAIN_MINIMUM_VERSION_gcc="4.8" >> >> Is there right place to adjust I think, I guess we need someone for Oracle to regenerate the generated-configure.sh files ? >> >> Best regards, Matthias >> >> >> >> >> -----Original Message----- >> From: Erik Joelsson [mailto:erik.joelsson at oracle.com] >> Sent: Donnerstag, 13. Juli 2017 11:59 >> To: Baesken, Matthias ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' >> Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build >> >> Hello Matthias, >> >> AFAIK, the only reason we support GCC versions older than 4.9 is for you >> guys at SAP, so if you would suggest dropping support, that would of >> course be the simplest solution. >> >> If you want to keep support but make the use of this flag optional, the >> preferred method is to add a test in flags.m4. We have macros defined >> for this. FLAGS_COMPILER_CHECK_ARGUMENTS or >> FLAGS_CXX_COMPILER_CHECK_ARGUMENTS would be suitable in this case. Use >> it to check if the flag is valid for the current compiler. If so, you >> can define a variable >> "CXXFLAGS_GCC_NO_VAR_TRACKING_ASSIGNMENTS=-fno-var-tracking-assignment" >> and empty otherwise, and in the makefile use this variable in the >> assignement. >> >> We want to avoid static shell expressions in the makefiles if possible >> and keep that kind of logic in configure. It's also better to explicitly >> check for features rather than versions. >> >> /Erik >> >> >> On 2017-07-13 11:21, Baesken, Matthias wrote: >>> Hello, when building jdk10 on Suse Linux 11 with default gcc/g++ 4.3.4 installed, I was running into >>> compilation errors because of the missing support for g++ flag -fno-var-tracking-assignments . >>> >>> It seems gcc-4.6 has the -fvar-tracking-assignments / -fnovar-tracking-assignments flags , see >>> >>> https://gcc.gnu.org/onlinedocs/gcc-4.6.4/gcc/Option-Summary.html#Option-Summary >>> >>> I have no gcc-4.6 at hand but could verify that gcc-4.7 and gcc-4.8 "know" the flag. >>> There are various solutions one could do to avoid the compilation error . >>> >>> >>> 1) disallow usage of old gcc versions here : >>> >>> autoconf/toolchain.m4 >>> >>> TOOLCHAIN_MINIMUM_VERSION_gcc="4.3" >>> >>> ( e.g. change to minimum gcc 4.6 or gcc 4.7 supporting the flags used in the build ) >>> >>> >>> 2) remove the flag -fno-var-tracking-assignments for older gcc versions here : >>> (in a similar way it was done : http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/37e693211deb ) >>> >>> hotspot/make/lib/JvmOverrideFiles.gmk-32-ifeq ($(TOOLCHAIN_TYPE), gcc) >>> hotspot/make/lib/JvmOverrideFiles.gmk:33: BUILD_LIBJVM_vmStructs.cpp_CXXFLAGS := -fno-var-tracking-assignments -O0 >>> hotspot/make/lib/JvmOverrideFiles.gmk:34: BUILD_LIBJVM_jvmciCompilerToVM.cpp_CXXFLAGS := -fno-var-tracking-assignments >>> hotspot/make/lib/JvmOverrideFiles.gmk-35-endif >>> >>> What is your preferred solution ? >>> >>> >>> Thanks, Matthias >>> >>> > From erik.joelsson at oracle.com Fri Jul 14 06:45:05 2017 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Fri, 14 Jul 2017 08:45:05 +0200 Subject: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build In-Reply-To: <5967D063.1010808@cjnash.com> References: <6c85b34b225e417788f794a99269e9fb@sap.com> <5967D063.1010808@cjnash.com> Message-ID: <157cc89c-b481-6797-9dc1-b504965fdf1f@oracle.com> From my point of view, the minimum version is a bit arbitrary. We have identified that we likely need 4.6 or newer now, so we could set the limit to 4.6 or 4.7 if needed. Supporting very old compilers have a cost. The setting here will only cause a warning to be printed that essentially means we don't test builds with this old compiler and we won't make any effort in keeping it working. I would expect 4.7 to continue to work for a time anyway. Eventually though, we will upgrade the recommended toolchain and we will start using newer compiler features that will make older compilers obsolete for real. /Erik On 2017-07-13 21:56, Simon Nash wrote: > I am currently building JDK 9 for arm32 (hard float and soft float) using > a Linaro GCC 4.7 cross-compiler (binary download). What is the reason > that > this cannot be supported for JDK 10? > > Best regards, > Simon > > On 13/07/2017 14:40, Baesken, Matthias wrote: >> Hi Erik, I prepared the change : >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8184338/ >> >> bug : >> >> https://bugs.openjdk.java.net/browse/JDK-8184338 >> >> Asking for review(s) ... >> >> >> Best regards, Matthias >> >> >> >> >> -----Original Message----- >> From: Erik Joelsson [mailto:erik.joelsson at oracle.com] Sent: >> Donnerstag, 13. Juli 2017 13:34 >> To: Baesken, Matthias ; >> 'build-dev at openjdk.java.net' ; >> 'hotspot-dev at openjdk.java.net' >> Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments g++ >> flag in hotspot build >> >> Hello, >> >> That would be the correct place. If you prepare the change and send >> the review I can sponsor the push. >> >> /Erik >> >> On 2017-07-13 13:16, Baesken, Matthias wrote: >>> Hi Erik, >>> >>>> AFAIK, the only reason we support GCC versions older than 4.9 is >>>> for you >>>> guys at SAP, so if you would suggest dropping support, that would of >>>> course be the simplest solution. >>> for jdk10 it is fine for us to use minimum gcc 4.8 . That >>> would be probably the most simple solution to avoid running into >>> the "-fno-var-tracking-assignments" issue >>> ( I now and then run into it when I forget to set the PATH to >>> gcc-4.8 one should use for compiling on the SLES11 machine, so I >>> pick up instead the default gcc 4.3.4 from the machine , >>> configure works nicely and then the build fails in the middle of the >>> HS make ) >>> >>> autoconf/toolchain.m4 >>> >>> TOOLCHAIN_MINIMUM_VERSION_gcc="4.8" >>> >>> Is there right place to adjust I think, I guess we need someone for >>> Oracle to regenerate the generated-configure.sh files ? >>> >>> Best regards, Matthias >>> >>> >>> >>> >>> -----Original Message----- >>> From: Erik Joelsson [mailto:erik.joelsson at oracle.com] >>> Sent: Donnerstag, 13. Juli 2017 11:59 >>> To: Baesken, Matthias ; >>> 'build-dev at openjdk.java.net' ; >>> 'hotspot-dev at openjdk.java.net' >>> Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments >>> g++ flag in hotspot build >>> >>> Hello Matthias, >>> >>> AFAIK, the only reason we support GCC versions older than 4.9 is for >>> you >>> guys at SAP, so if you would suggest dropping support, that would of >>> course be the simplest solution. >>> >>> If you want to keep support but make the use of this flag optional, the >>> preferred method is to add a test in flags.m4. We have macros defined >>> for this. FLAGS_COMPILER_CHECK_ARGUMENTS or >>> FLAGS_CXX_COMPILER_CHECK_ARGUMENTS would be suitable in this case. Use >>> it to check if the flag is valid for the current compiler. If so, you >>> can define a variable >>> "CXXFLAGS_GCC_NO_VAR_TRACKING_ASSIGNMENTS=-fno-var-tracking-assignment" >>> and empty otherwise, and in the makefile use this variable in the >>> assignement. >>> >>> We want to avoid static shell expressions in the makefiles if possible >>> and keep that kind of logic in configure. It's also better to >>> explicitly >>> check for features rather than versions. >>> >>> /Erik >>> >>> >>> On 2017-07-13 11:21, Baesken, Matthias wrote: >>>> Hello, when building jdk10 on Suse Linux 11 with default gcc/g++ >>>> 4.3.4 installed, I was running into >>>> compilation errors because of the missing support for g++ flag >>>> -fno-var-tracking-assignments . >>>> >>>> It seems gcc-4.6 has the -fvar-tracking-assignments / >>>> -fnovar-tracking-assignments flags , see >>>> >>>> https://gcc.gnu.org/onlinedocs/gcc-4.6.4/gcc/Option-Summary.html#Option-Summary >>>> >>>> >>>> I have no gcc-4.6 at hand but could verify that gcc-4.7 and gcc-4.8 >>>> "know" the flag. >>>> There are various solutions one could do to avoid the compilation >>>> error . >>>> >>>> >>>> 1) disallow usage of old gcc versions here : >>>> >>>> autoconf/toolchain.m4 >>>> >>>> TOOLCHAIN_MINIMUM_VERSION_gcc="4.3" >>>> >>>> ( e.g. change to minimum gcc 4.6 or gcc 4.7 supporting the >>>> flags used in the build ) >>>> >>>> >>>> 2) remove the flag -fno-var-tracking-assignments for older gcc >>>> versions here : >>>> (in a similar way it was done : >>>> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/37e693211deb ) >>>> >>>> hotspot/make/lib/JvmOverrideFiles.gmk-32-ifeq ($(TOOLCHAIN_TYPE), gcc) >>>> hotspot/make/lib/JvmOverrideFiles.gmk:33: >>>> BUILD_LIBJVM_vmStructs.cpp_CXXFLAGS := >>>> -fno-var-tracking-assignments -O0 >>>> hotspot/make/lib/JvmOverrideFiles.gmk:34: >>>> BUILD_LIBJVM_jvmciCompilerToVM.cpp_CXXFLAGS := >>>> -fno-var-tracking-assignments >>>> hotspot/make/lib/JvmOverrideFiles.gmk-35-endif >>>> >>>> What is your preferred solution ? >>>> >>>> >>>> Thanks, Matthias >>>> >>>> >> > From glaubitz at physik.fu-berlin.de Fri Jul 14 09:23:11 2017 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Fri, 14 Jul 2017 11:23:11 +0200 Subject: [PATCH] linux-sparc build fixes In-Reply-To: References: <20170609102041.GA2477@physik.fu-berlin.de> <20170704093304.GD28259@physik.fu-berlin.de> <78dbbda5-f3ea-009e-3989-95a2292a0a46@physik.fu-berlin.de> Message-ID: <20170714092310.GB26530@physik.fu-berlin.de> Hi Erik! On Thu, Jul 06, 2017 at 02:55:53PM +0200, Erik Helin wrote: > >Yes, you now have two reviewers. But for these changes (hotspot) you > >need a sponsor, which needs to be an Oracle employee, which I am not. > > > >Maybe Eric could sponsor the change? > > Yep, I can shepherd the patches in (aka getting them through the build and > test system we all know as JPRT). Thanks Thomas for reviewing! > Erik Can you tell me what the current status of the patches are? Have they been merged yet? Which repository are they going to get merged? I have searched through the various JDK repositories and couldn't find it last time I checked. Thanks, Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From erik.helin at oracle.com Fri Jul 14 10:36:02 2017 From: erik.helin at oracle.com (Erik Helin) Date: Fri, 14 Jul 2017 12:36:02 +0200 Subject: RFR(XL): 8181917: Refactor UL LogStreams to avoid using resource area In-Reply-To: References: <337330d2-512c-bc77-5a55-c05d33e376e5@oracle.com> Message-ID: On 07/13/2017 08:46 AM, Thomas St?fe wrote: > Hi Erik! > > On Mon, Jul 10, 2017 at 11:40 AM, Erik Helin > wrote: > > Hi Thomas, > > On 06/30/2017 08:32 PM, Thomas St?fe wrote: > > Hi Eric, > > thank you for the review! > > New > Version: > http://cr.openjdk.java.net/~stuefe/webrevs/8181917-refactor-ul-logstream/all.webrev.03/webrev/ > > > > I think you still have a problem with "runaway" loggers in this version: > > +// try_ensure_cap tries to enlarge the capacity of the internal buffer > +// to the given atleast value. May fail if either OOM happens or > atleast > +// is larger than a reasonable max of 1 M. Caller must not assume > +// capacity without checking. > +void LogStream::LineBuffer::try_ensure_cap(size_t atleast) { > + assert(_cap >= sizeof(_smallbuf), "sanity"); > + if (_cap < atleast) { > + const size_t reasonable_max = 1 * M; > + size_t newcap = align_size_up(atleast + 64, 64); > + assert(_cap <= reasonable_max, "sanity"); > + // Cap out at a reasonable max to prevent runaway leaks. > + if (newcap > reasonable_max) { > + newcap = reasonable_max; > + } > + > + char* const newbuf = (char*) os::malloc(newcap, mtLogging); > + if (newbuf == NULL) { // OOM. Leave object unchanged. > + return; > + } > + if (_pos > 0) { // preserve old content > + memcpy(newbuf, _buf, _pos + 1); // ..including trailing zero > + } > + if (_buf != _smallbuf) { > + os::free(_buf); > + } > + _buf = newbuf; > + _cap = newcap; > + } > + assert(_cap >= atleast, "sanity"); > +} > > with the above code, even though _cap > reasonable_max, you will > always allocate newbuf with os::malloc. For debug builds, we are > fine, since the JVM will assert on _cap <= reasonable_max, but for > product builds we are not. Would it be better to first check if _cap > == reasonable_max? As in the following snippet: > > void LogStream::LineBuffer::try_ensure_cap(size_t atleast) { > if (_cap < atleast) { > const size_t reasonable_max = 1 * M; > if (_cap == reasonable_max) { > // max memory used, "die" in debug builds, log in product builds > log_warning(logging)("Max memory used for message: %s\n", _buf); > ShouldNotReachHere(); > return; > } > > size_t newcap = align_size_up(atleast + 64, 64); > if (newcap > reasonable_max) { > newcap = reasonable_max; > } > char* const newbuf = (char*) os::malloc(newcap, mtLogging); > // rest of logic > } > } > > Thanks, > Erik > > > I'm not sure I understand. Are you concerned that _cap would grow beyond > reasonable_max - because I do not see a way this could happen. Or do you > want to also treat the *attempt* to grow the _cap beyond reasonable_max > as a loggable offence? What I am trying to say is that with your current patch, even if _cap == reasonable_max, you will still allocate a new 1M buffer, copy all the chars over, and the free the old one. My proposal simply skips the allocation, copying and freeing if _cap == reasonable_max. The result will be the same, just less stress on malloc. > If the latter, I am not sure about logging. If this is an error which > may happen at a customer site without popping up first in inhouse-tests > (which it conceivably is, because logging is very content dependent), I > would rather hide the error in production code instead of flooding > stderr with tons of log warnings or crashing with a guarantee, no? I see what you mean, but perhaps log_info(logging) at least? I want to be able to discover if this happens in production, for example if a customer complains about truncated logs. A log_info(logging) will only be printed if -Xlog:logging=info is specified on the command-line. Thanks, Erik > ..Thomas > > From matthias.baesken at sap.com Fri Jul 14 11:47:02 2017 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 14 Jul 2017 11:47:02 +0000 Subject: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build In-Reply-To: <424e7257bc504fe09d967ce56d64afd1@sap.com> References: <6c85b34b225e417788f794a99269e9fb@sap.com> <5967D063.1010808@cjnash.com> <424e7257bc504fe09d967ce56d64afd1@sap.com> Message-ID: <2ac4b05d05434369bdeb72f7018d00f8@sap.com> After Simons question , I checked that the jdk10 builds still works with gcc 4.7 on linux x86_64 as well . And created a new webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8184338.1/ - check adjusted to minimum gcc 4.7 - common/doc/building.md adjusted (the part talking about gcc versions) Best regards, Matthias -----Original Message----- From: Baesken, Matthias Sent: Freitag, 14. Juli 2017 08:36 To: 'Simon Nash' Cc: Erik Joelsson ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' ; Zeller, Arno Subject: RE: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build > I am currently building JDK 9 for arm32 (hard float and soft float) using > a Linaro GCC 4.7 cross-compiler (binary download). What is the reason that > this cannot be supported for JDK 10? Hi Simon, reason was that we know gcc-4.8 works nicely because we do a lot of builds (+tests) with this compiler . For gcc 4.7 we just do not know ( but for old 4.3 / 4.4 it was obvious that the current flags do not work any more). If you are using a gcc compiler < 4.8 only a warning is reported , it does not mean that you cannot build any more. My first suggestion was : >>> ( e.g. change to minimum gcc 4.6 or gcc 4.7 supporting the flags used in the build ) So if you see a benefit to test for minimum gcc 4.7 and not 4.8 I am fine with this too (Erik what do you think?). We at SAP just cannot tell that gcc 4.7 still works (because our builds do not use it). Best regards, Matthias -----Original Message----- From: Simon Nash [mailto:simon at cjnash.com] Sent: Donnerstag, 13. Juli 2017 21:56 To: Baesken, Matthias Cc: Erik Joelsson ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' ; Zeller, Arno Subject: Re: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build I am currently building JDK 9 for arm32 (hard float and soft float) using a Linaro GCC 4.7 cross-compiler (binary download). What is the reason that this cannot be supported for JDK 10? Best regards, Simon On 13/07/2017 14:40, Baesken, Matthias wrote: > Hi Erik, I prepared the change : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8184338/ > > bug : > > https://bugs.openjdk.java.net/browse/JDK-8184338 > > Asking for review(s) ... > > > Best regards, Matthias > > > > > -----Original Message----- > From: Erik Joelsson [mailto:erik.joelsson at oracle.com] > Sent: Donnerstag, 13. Juli 2017 13:34 > To: Baesken, Matthias ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' > Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build > > Hello, > > That would be the correct place. If you prepare the change and send the > review I can sponsor the push. > > /Erik > > On 2017-07-13 13:16, Baesken, Matthias wrote: >> Hi Erik, >> >>> AFAIK, the only reason we support GCC versions older than 4.9 is for you >>> guys at SAP, so if you would suggest dropping support, that would of >>> course be the simplest solution. >> for jdk10 it is fine for us to use minimum gcc 4.8 . That would be probably the most simple solution to avoid running into the "-fno-var-tracking-assignments" issue >> ( I now and then run into it when I forget to set the PATH to gcc-4.8 one should use for compiling on the SLES11 machine, so I pick up instead the default gcc 4.3.4 from the machine , >> configure works nicely and then the build fails in the middle of the HS make ) >> >> autoconf/toolchain.m4 >> >> TOOLCHAIN_MINIMUM_VERSION_gcc="4.8" >> >> Is there right place to adjust I think, I guess we need someone for Oracle to regenerate the generated-configure.sh files ? >> >> Best regards, Matthias >> >> >> >> >> -----Original Message----- >> From: Erik Joelsson [mailto:erik.joelsson at oracle.com] >> Sent: Donnerstag, 13. Juli 2017 11:59 >> To: Baesken, Matthias ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' >> Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build >> >> Hello Matthias, >> >> AFAIK, the only reason we support GCC versions older than 4.9 is for you >> guys at SAP, so if you would suggest dropping support, that would of >> course be the simplest solution. >> >> If you want to keep support but make the use of this flag optional, the >> preferred method is to add a test in flags.m4. We have macros defined >> for this. FLAGS_COMPILER_CHECK_ARGUMENTS or >> FLAGS_CXX_COMPILER_CHECK_ARGUMENTS would be suitable in this case. Use >> it to check if the flag is valid for the current compiler. If so, you >> can define a variable >> "CXXFLAGS_GCC_NO_VAR_TRACKING_ASSIGNMENTS=-fno-var-tracking-assignment" >> and empty otherwise, and in the makefile use this variable in the >> assignement. >> >> We want to avoid static shell expressions in the makefiles if possible >> and keep that kind of logic in configure. It's also better to explicitly >> check for features rather than versions. >> >> /Erik >> >> >> On 2017-07-13 11:21, Baesken, Matthias wrote: >>> Hello, when building jdk10 on Suse Linux 11 with default gcc/g++ 4.3.4 installed, I was running into >>> compilation errors because of the missing support for g++ flag -fno-var-tracking-assignments . >>> >>> It seems gcc-4.6 has the -fvar-tracking-assignments / -fnovar-tracking-assignments flags , see >>> >>> https://gcc.gnu.org/onlinedocs/gcc-4.6.4/gcc/Option-Summary.html#Option-Summary >>> >>> I have no gcc-4.6 at hand but could verify that gcc-4.7 and gcc-4.8 "know" the flag. >>> There are various solutions one could do to avoid the compilation error . >>> >>> >>> 1) disallow usage of old gcc versions here : >>> >>> autoconf/toolchain.m4 >>> >>> TOOLCHAIN_MINIMUM_VERSION_gcc="4.3" >>> >>> ( e.g. change to minimum gcc 4.6 or gcc 4.7 supporting the flags used in the build ) >>> >>> >>> 2) remove the flag -fno-var-tracking-assignments for older gcc versions here : >>> (in a similar way it was done : http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/37e693211deb ) >>> >>> hotspot/make/lib/JvmOverrideFiles.gmk-32-ifeq ($(TOOLCHAIN_TYPE), gcc) >>> hotspot/make/lib/JvmOverrideFiles.gmk:33: BUILD_LIBJVM_vmStructs.cpp_CXXFLAGS := -fno-var-tracking-assignments -O0 >>> hotspot/make/lib/JvmOverrideFiles.gmk:34: BUILD_LIBJVM_jvmciCompilerToVM.cpp_CXXFLAGS := -fno-var-tracking-assignments >>> hotspot/make/lib/JvmOverrideFiles.gmk-35-endif >>> >>> What is your preferred solution ? >>> >>> >>> Thanks, Matthias >>> >>> > From erik.helin at oracle.com Fri Jul 14 12:23:14 2017 From: erik.helin at oracle.com (Erik Helin) Date: Fri, 14 Jul 2017 14:23:14 +0200 Subject: [PATCH] linux-sparc build fixes In-Reply-To: <20170714092310.GB26530@physik.fu-berlin.de> References: <20170609102041.GA2477@physik.fu-berlin.de> <20170704093304.GD28259@physik.fu-berlin.de> <78dbbda5-f3ea-009e-3989-95a2292a0a46@physik.fu-berlin.de> <20170714092310.GB26530@physik.fu-berlin.de> Message-ID: On 07/14/2017 11:23 AM, John Paul Adrian Glaubitz wrote: > Hi Erik! > > On Thu, Jul 06, 2017 at 02:55:53PM +0200, Erik Helin wrote: >>> Yes, you now have two reviewers. But for these changes (hotspot) you >>> need a sponsor, which needs to be an Oracle employee, which I am not. >>> >>> Maybe Eric could sponsor the change? >> >> Yep, I can shepherd the patches in (aka getting them through the build and >> test system we all know as JPRT). Thanks Thomas for reviewing! >> Erik > > Can you tell me what the current status of the patches are? Have they > been merged yet? Which repository are they going to get merged? I have > searched through the various JDK repositories and couldn't find it > last time I checked. Sorry, I haven't gotten around to do this until now. The patches are currently running on the build server and assuming everything is fine, will end up in http://hg.openjdk.java.net/jdk10/hs/hotspot. Thanks, Erik > Thanks, > Adrian > From erik.joelsson at oracle.com Fri Jul 14 12:39:47 2017 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Fri, 14 Jul 2017 14:39:47 +0200 Subject: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build In-Reply-To: <2ac4b05d05434369bdeb72f7018d00f8@sap.com> References: <6c85b34b225e417788f794a99269e9fb@sap.com> <5967D063.1010808@cjnash.com> <424e7257bc504fe09d967ce56d64afd1@sap.com> <2ac4b05d05434369bdeb72f7018d00f8@sap.com> Message-ID: <54952e80-0602-7645-6f7c-df5efadaf9bd@oracle.com> Looks good to me. /Erik On 2017-07-14 13:47, Baesken, Matthias wrote: > After Simons question , I checked that the jdk10 builds still works with gcc 4.7 on linux x86_64 as well . > And created a new webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8184338.1/ > > - check adjusted to minimum gcc 4.7 > - common/doc/building.md adjusted (the part talking about gcc versions) > > > Best regards, Matthias > > > > -----Original Message----- > From: Baesken, Matthias > Sent: Freitag, 14. Juli 2017 08:36 > To: 'Simon Nash' > Cc: Erik Joelsson ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' ; Zeller, Arno > Subject: RE: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build > >> I am currently building JDK 9 for arm32 (hard float and soft float) using >> a Linaro GCC 4.7 cross-compiler (binary download). What is the reason that >> this cannot be supported for JDK 10? > Hi Simon, > reason was that we know gcc-4.8 works nicely because we do a lot of builds (+tests) with this compiler . > For gcc 4.7 we just do not know ( but for old 4.3 / 4.4 it was obvious that the current flags do not work any more). > If you are using a gcc compiler < 4.8 only a warning is reported , it does not mean that you cannot build any more. > > My first suggestion was : > >>>> ( e.g. change to minimum gcc 4.6 or gcc 4.7 supporting the flags used in the build ) > So if you see a benefit to test for minimum gcc 4.7 and not 4.8 I am fine with this too (Erik what do you think?). > We at SAP just cannot tell that gcc 4.7 still works (because our builds do not use it). > > Best regards, Matthias > > > > -----Original Message----- > From: Simon Nash [mailto:simon at cjnash.com] > Sent: Donnerstag, 13. Juli 2017 21:56 > To: Baesken, Matthias > Cc: Erik Joelsson ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' ; Zeller, Arno > Subject: Re: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build > > I am currently building JDK 9 for arm32 (hard float and soft float) using > a Linaro GCC 4.7 cross-compiler (binary download). What is the reason that > this cannot be supported for JDK 10? > > Best regards, > Simon > > On 13/07/2017 14:40, Baesken, Matthias wrote: >> Hi Erik, I prepared the change : >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8184338/ >> >> bug : >> >> https://bugs.openjdk.java.net/browse/JDK-8184338 >> >> Asking for review(s) ... >> >> >> Best regards, Matthias >> >> >> >> >> -----Original Message----- >> From: Erik Joelsson [mailto:erik.joelsson at oracle.com] >> Sent: Donnerstag, 13. Juli 2017 13:34 >> To: Baesken, Matthias ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' >> Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build >> >> Hello, >> >> That would be the correct place. If you prepare the change and send the >> review I can sponsor the push. >> >> /Erik >> >> On 2017-07-13 13:16, Baesken, Matthias wrote: >>> Hi Erik, >>> >>>> AFAIK, the only reason we support GCC versions older than 4.9 is for you >>>> guys at SAP, so if you would suggest dropping support, that would of >>>> course be the simplest solution. >>> for jdk10 it is fine for us to use minimum gcc 4.8 . That would be probably the most simple solution to avoid running into the "-fno-var-tracking-assignments" issue >>> ( I now and then run into it when I forget to set the PATH to gcc-4.8 one should use for compiling on the SLES11 machine, so I pick up instead the default gcc 4.3.4 from the machine , >>> configure works nicely and then the build fails in the middle of the HS make ) >>> >>> autoconf/toolchain.m4 >>> >>> TOOLCHAIN_MINIMUM_VERSION_gcc="4.8" >>> >>> Is there right place to adjust I think, I guess we need someone for Oracle to regenerate the generated-configure.sh files ? >>> >>> Best regards, Matthias >>> >>> >>> >>> >>> -----Original Message----- >>> From: Erik Joelsson [mailto:erik.joelsson at oracle.com] >>> Sent: Donnerstag, 13. Juli 2017 11:59 >>> To: Baesken, Matthias ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' >>> Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build >>> >>> Hello Matthias, >>> >>> AFAIK, the only reason we support GCC versions older than 4.9 is for you >>> guys at SAP, so if you would suggest dropping support, that would of >>> course be the simplest solution. >>> >>> If you want to keep support but make the use of this flag optional, the >>> preferred method is to add a test in flags.m4. We have macros defined >>> for this. FLAGS_COMPILER_CHECK_ARGUMENTS or >>> FLAGS_CXX_COMPILER_CHECK_ARGUMENTS would be suitable in this case. Use >>> it to check if the flag is valid for the current compiler. If so, you >>> can define a variable >>> "CXXFLAGS_GCC_NO_VAR_TRACKING_ASSIGNMENTS=-fno-var-tracking-assignment" >>> and empty otherwise, and in the makefile use this variable in the >>> assignement. >>> >>> We want to avoid static shell expressions in the makefiles if possible >>> and keep that kind of logic in configure. It's also better to explicitly >>> check for features rather than versions. >>> >>> /Erik >>> >>> >>> On 2017-07-13 11:21, Baesken, Matthias wrote: >>>> Hello, when building jdk10 on Suse Linux 11 with default gcc/g++ 4.3.4 installed, I was running into >>>> compilation errors because of the missing support for g++ flag -fno-var-tracking-assignments . >>>> >>>> It seems gcc-4.6 has the -fvar-tracking-assignments / -fnovar-tracking-assignments flags , see >>>> >>>> https://gcc.gnu.org/onlinedocs/gcc-4.6.4/gcc/Option-Summary.html#Option-Summary >>>> >>>> I have no gcc-4.6 at hand but could verify that gcc-4.7 and gcc-4.8 "know" the flag. >>>> There are various solutions one could do to avoid the compilation error . >>>> >>>> >>>> 1) disallow usage of old gcc versions here : >>>> >>>> autoconf/toolchain.m4 >>>> >>>> TOOLCHAIN_MINIMUM_VERSION_gcc="4.3" >>>> >>>> ( e.g. change to minimum gcc 4.6 or gcc 4.7 supporting the flags used in the build ) >>>> >>>> >>>> 2) remove the flag -fno-var-tracking-assignments for older gcc versions here : >>>> (in a similar way it was done : http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/37e693211deb ) >>>> >>>> hotspot/make/lib/JvmOverrideFiles.gmk-32-ifeq ($(TOOLCHAIN_TYPE), gcc) >>>> hotspot/make/lib/JvmOverrideFiles.gmk:33: BUILD_LIBJVM_vmStructs.cpp_CXXFLAGS := -fno-var-tracking-assignments -O0 >>>> hotspot/make/lib/JvmOverrideFiles.gmk:34: BUILD_LIBJVM_jvmciCompilerToVM.cpp_CXXFLAGS := -fno-var-tracking-assignments >>>> hotspot/make/lib/JvmOverrideFiles.gmk-35-endif >>>> >>>> What is your preferred solution ? >>>> >>>> >>>> Thanks, Matthias >>>> >>>> From erik.joelsson at oracle.com Fri Jul 14 12:43:08 2017 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Fri, 14 Jul 2017 14:43:08 +0200 Subject: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build In-Reply-To: <2ac4b05d05434369bdeb72f7018d00f8@sap.com> References: <6c85b34b225e417788f794a99269e9fb@sap.com> <5967D063.1010808@cjnash.com> <424e7257bc504fe09d967ce56d64afd1@sap.com> <2ac4b05d05434369bdeb72f7018d00f8@sap.com> Message-ID: <948004c4-63e7-b0e4-1820-bf2351fc2327@oracle.com> This looks good to me. /Erik On 2017-07-14 13:47, Baesken, Matthias wrote: > After Simons question , I checked that the jdk10 builds still works with gcc 4.7 on linux x86_64 as well . > And created a new webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8184338.1/ > > - check adjusted to minimum gcc 4.7 > - common/doc/building.md adjusted (the part talking about gcc versions) > > > Best regards, Matthias > > > > -----Original Message----- > From: Baesken, Matthias > Sent: Freitag, 14. Juli 2017 08:36 > To: 'Simon Nash' > Cc: Erik Joelsson ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' ; Zeller, Arno > Subject: RE: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build > >> I am currently building JDK 9 for arm32 (hard float and soft float) using >> a Linaro GCC 4.7 cross-compiler (binary download). What is the reason that >> this cannot be supported for JDK 10? > Hi Simon, > reason was that we know gcc-4.8 works nicely because we do a lot of builds (+tests) with this compiler . > For gcc 4.7 we just do not know ( but for old 4.3 / 4.4 it was obvious that the current flags do not work any more). > If you are using a gcc compiler < 4.8 only a warning is reported , it does not mean that you cannot build any more. > > My first suggestion was : > >>>> ( e.g. change to minimum gcc 4.6 or gcc 4.7 supporting the flags used in the build ) > So if you see a benefit to test for minimum gcc 4.7 and not 4.8 I am fine with this too (Erik what do you think?). > We at SAP just cannot tell that gcc 4.7 still works (because our builds do not use it). > > Best regards, Matthias > > > > -----Original Message----- > From: Simon Nash [mailto:simon at cjnash.com] > Sent: Donnerstag, 13. Juli 2017 21:56 > To: Baesken, Matthias > Cc: Erik Joelsson ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' ; Zeller, Arno > Subject: Re: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build > > I am currently building JDK 9 for arm32 (hard float and soft float) using > a Linaro GCC 4.7 cross-compiler (binary download). What is the reason that > this cannot be supported for JDK 10? > > Best regards, > Simon > > On 13/07/2017 14:40, Baesken, Matthias wrote: >> Hi Erik, I prepared the change : >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8184338/ >> >> bug : >> >> https://bugs.openjdk.java.net/browse/JDK-8184338 >> >> Asking for review(s) ... >> >> >> Best regards, Matthias >> >> >> >> >> -----Original Message----- >> From: Erik Joelsson [mailto:erik.joelsson at oracle.com] >> Sent: Donnerstag, 13. Juli 2017 13:34 >> To: Baesken, Matthias ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' >> Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build >> >> Hello, >> >> That would be the correct place. If you prepare the change and send the >> review I can sponsor the push. >> >> /Erik >> >> On 2017-07-13 13:16, Baesken, Matthias wrote: >>> Hi Erik, >>> >>>> AFAIK, the only reason we support GCC versions older than 4.9 is for you >>>> guys at SAP, so if you would suggest dropping support, that would of >>>> course be the simplest solution. >>> for jdk10 it is fine for us to use minimum gcc 4.8 . That would be probably the most simple solution to avoid running into the "-fno-var-tracking-assignments" issue >>> ( I now and then run into it when I forget to set the PATH to gcc-4.8 one should use for compiling on the SLES11 machine, so I pick up instead the default gcc 4.3.4 from the machine , >>> configure works nicely and then the build fails in the middle of the HS make ) >>> >>> autoconf/toolchain.m4 >>> >>> TOOLCHAIN_MINIMUM_VERSION_gcc="4.8" >>> >>> Is there right place to adjust I think, I guess we need someone for Oracle to regenerate the generated-configure.sh files ? >>> >>> Best regards, Matthias >>> >>> >>> >>> >>> -----Original Message----- >>> From: Erik Joelsson [mailto:erik.joelsson at oracle.com] >>> Sent: Donnerstag, 13. Juli 2017 11:59 >>> To: Baesken, Matthias ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' >>> Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build >>> >>> Hello Matthias, >>> >>> AFAIK, the only reason we support GCC versions older than 4.9 is for you >>> guys at SAP, so if you would suggest dropping support, that would of >>> course be the simplest solution. >>> >>> If you want to keep support but make the use of this flag optional, the >>> preferred method is to add a test in flags.m4. We have macros defined >>> for this. FLAGS_COMPILER_CHECK_ARGUMENTS or >>> FLAGS_CXX_COMPILER_CHECK_ARGUMENTS would be suitable in this case. Use >>> it to check if the flag is valid for the current compiler. If so, you >>> can define a variable >>> "CXXFLAGS_GCC_NO_VAR_TRACKING_ASSIGNMENTS=-fno-var-tracking-assignment" >>> and empty otherwise, and in the makefile use this variable in the >>> assignement. >>> >>> We want to avoid static shell expressions in the makefiles if possible >>> and keep that kind of logic in configure. It's also better to explicitly >>> check for features rather than versions. >>> >>> /Erik >>> >>> >>> On 2017-07-13 11:21, Baesken, Matthias wrote: >>>> Hello, when building jdk10 on Suse Linux 11 with default gcc/g++ 4.3.4 installed, I was running into >>>> compilation errors because of the missing support for g++ flag -fno-var-tracking-assignments . >>>> >>>> It seems gcc-4.6 has the -fvar-tracking-assignments / -fnovar-tracking-assignments flags , see >>>> >>>> https://gcc.gnu.org/onlinedocs/gcc-4.6.4/gcc/Option-Summary.html#Option-Summary >>>> >>>> I have no gcc-4.6 at hand but could verify that gcc-4.7 and gcc-4.8 "know" the flag. >>>> There are various solutions one could do to avoid the compilation error . >>>> >>>> >>>> 1) disallow usage of old gcc versions here : >>>> >>>> autoconf/toolchain.m4 >>>> >>>> TOOLCHAIN_MINIMUM_VERSION_gcc="4.3" >>>> >>>> ( e.g. change to minimum gcc 4.6 or gcc 4.7 supporting the flags used in the build ) >>>> >>>> >>>> 2) remove the flag -fno-var-tracking-assignments for older gcc versions here : >>>> (in a similar way it was done : http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/37e693211deb ) >>>> >>>> hotspot/make/lib/JvmOverrideFiles.gmk-32-ifeq ($(TOOLCHAIN_TYPE), gcc) >>>> hotspot/make/lib/JvmOverrideFiles.gmk:33: BUILD_LIBJVM_vmStructs.cpp_CXXFLAGS := -fno-var-tracking-assignments -O0 >>>> hotspot/make/lib/JvmOverrideFiles.gmk:34: BUILD_LIBJVM_jvmciCompilerToVM.cpp_CXXFLAGS := -fno-var-tracking-assignments >>>> hotspot/make/lib/JvmOverrideFiles.gmk-35-endif >>>> >>>> What is your preferred solution ? >>>> >>>> >>>> Thanks, Matthias >>>> >>>> From dalibor.topic at oracle.com Fri Jul 14 13:01:36 2017 From: dalibor.topic at oracle.com (dalibor topic) Date: Fri, 14 Jul 2017 15:01:36 +0200 Subject: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build In-Reply-To: <157cc89c-b481-6797-9dc1-b504965fdf1f@oracle.com> References: <6c85b34b225e417788f794a99269e9fb@sap.com> <5967D063.1010808@cjnash.com> <157cc89c-b481-6797-9dc1-b504965fdf1f@oracle.com> Message-ID: Fwiw, gcc 4.x has seen its last release about a year ago with 4.9.4: https://gcc.gnu.org/gcc-4.9/ . It is no longer maintained. cheers, dalibor topic On 14.07.2017 08:45, Erik Joelsson wrote: > From my point of view, the minimum version is a bit arbitrary. We have > identified that we likely need 4.6 or newer now, so we could set the > limit to 4.6 or 4.7 if needed. Supporting very old compilers have a > cost. The setting here will only cause a warning to be printed that > essentially means we don't test builds with this old compiler and we > won't make any effort in keeping it working. I would expect 4.7 to > continue to work for a time anyway. > > Eventually though, we will upgrade the recommended toolchain and we will > start using newer compiler features that will make older compilers > obsolete for real. > > /Erik > > > On 2017-07-13 21:56, Simon Nash wrote: >> I am currently building JDK 9 for arm32 (hard float and soft float) using >> a Linaro GCC 4.7 cross-compiler (binary download). What is the reason >> that >> this cannot be supported for JDK 10? >> >> Best regards, >> Simon >> >> On 13/07/2017 14:40, Baesken, Matthias wrote: >>> Hi Erik, I prepared the change : >>> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8184338/ >>> >>> bug : >>> >>> https://bugs.openjdk.java.net/browse/JDK-8184338 >>> >>> Asking for review(s) ... >>> >>> >>> Best regards, Matthias >>> >>> >>> >>> >>> -----Original Message----- >>> From: Erik Joelsson [mailto:erik.joelsson at oracle.com] Sent: >>> Donnerstag, 13. Juli 2017 13:34 >>> To: Baesken, Matthias ; >>> 'build-dev at openjdk.java.net' ; >>> 'hotspot-dev at openjdk.java.net' >>> Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments g++ >>> flag in hotspot build >>> >>> Hello, >>> >>> That would be the correct place. If you prepare the change and send >>> the review I can sponsor the push. >>> >>> /Erik >>> >>> On 2017-07-13 13:16, Baesken, Matthias wrote: >>>> Hi Erik, >>>> >>>>> AFAIK, the only reason we support GCC versions older than 4.9 is >>>>> for you >>>>> guys at SAP, so if you would suggest dropping support, that would of >>>>> course be the simplest solution. >>>> for jdk10 it is fine for us to use minimum gcc 4.8 . That >>>> would be probably the most simple solution to avoid running into >>>> the "-fno-var-tracking-assignments" issue >>>> ( I now and then run into it when I forget to set the PATH to >>>> gcc-4.8 one should use for compiling on the SLES11 machine, so I >>>> pick up instead the default gcc 4.3.4 from the machine , >>>> configure works nicely and then the build fails in the middle of the >>>> HS make ) >>>> >>>> autoconf/toolchain.m4 >>>> >>>> TOOLCHAIN_MINIMUM_VERSION_gcc="4.8" >>>> >>>> Is there right place to adjust I think, I guess we need someone for >>>> Oracle to regenerate the generated-configure.sh files ? >>>> >>>> Best regards, Matthias >>>> >>>> >>>> >>>> >>>> -----Original Message----- >>>> From: Erik Joelsson [mailto:erik.joelsson at oracle.com] >>>> Sent: Donnerstag, 13. Juli 2017 11:59 >>>> To: Baesken, Matthias ; >>>> 'build-dev at openjdk.java.net' ; >>>> 'hotspot-dev at openjdk.java.net' >>>> Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments >>>> g++ flag in hotspot build >>>> >>>> Hello Matthias, >>>> >>>> AFAIK, the only reason we support GCC versions older than 4.9 is for >>>> you >>>> guys at SAP, so if you would suggest dropping support, that would of >>>> course be the simplest solution. >>>> >>>> If you want to keep support but make the use of this flag optional, the >>>> preferred method is to add a test in flags.m4. We have macros defined >>>> for this. FLAGS_COMPILER_CHECK_ARGUMENTS or >>>> FLAGS_CXX_COMPILER_CHECK_ARGUMENTS would be suitable in this case. Use >>>> it to check if the flag is valid for the current compiler. If so, you >>>> can define a variable >>>> "CXXFLAGS_GCC_NO_VAR_TRACKING_ASSIGNMENTS=-fno-var-tracking-assignment" >>>> and empty otherwise, and in the makefile use this variable in the >>>> assignement. >>>> >>>> We want to avoid static shell expressions in the makefiles if possible >>>> and keep that kind of logic in configure. It's also better to >>>> explicitly >>>> check for features rather than versions. >>>> >>>> /Erik >>>> >>>> >>>> On 2017-07-13 11:21, Baesken, Matthias wrote: >>>>> Hello, when building jdk10 on Suse Linux 11 with default gcc/g++ >>>>> 4.3.4 installed, I was running into >>>>> compilation errors because of the missing support for g++ flag >>>>> -fno-var-tracking-assignments . >>>>> >>>>> It seems gcc-4.6 has the -fvar-tracking-assignments / >>>>> -fnovar-tracking-assignments flags , see >>>>> >>>>> https://gcc.gnu.org/onlinedocs/gcc-4.6.4/gcc/Option-Summary.html#Option-Summary >>>>> >>>>> >>>>> I have no gcc-4.6 at hand but could verify that gcc-4.7 and gcc-4.8 >>>>> "know" the flag. >>>>> There are various solutions one could do to avoid the compilation >>>>> error . >>>>> >>>>> >>>>> 1) disallow usage of old gcc versions here : >>>>> >>>>> autoconf/toolchain.m4 >>>>> >>>>> TOOLCHAIN_MINIMUM_VERSION_gcc="4.3" >>>>> >>>>> ( e.g. change to minimum gcc 4.6 or gcc 4.7 supporting the >>>>> flags used in the build ) >>>>> >>>>> >>>>> 2) remove the flag -fno-var-tracking-assignments for older gcc >>>>> versions here : >>>>> (in a similar way it was done : >>>>> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/37e693211deb ) >>>>> >>>>> hotspot/make/lib/JvmOverrideFiles.gmk-32-ifeq ($(TOOLCHAIN_TYPE), gcc) >>>>> hotspot/make/lib/JvmOverrideFiles.gmk:33: >>>>> BUILD_LIBJVM_vmStructs.cpp_CXXFLAGS := >>>>> -fno-var-tracking-assignments -O0 >>>>> hotspot/make/lib/JvmOverrideFiles.gmk:34: >>>>> BUILD_LIBJVM_jvmciCompilerToVM.cpp_CXXFLAGS := >>>>> -fno-var-tracking-assignments >>>>> hotspot/make/lib/JvmOverrideFiles.gmk-35-endif >>>>> >>>>> What is your preferred solution ? >>>>> >>>>> >>>>> Thanks, Matthias >>>>> >>>>> >>> >> > -- Dalibor Topic | Principal Product Manager Phone: +494089091214 | Mobile: +491737185961 ORACLE Deutschland B.V. & Co. KG | K?hneh?fe 5 | 22761 Hamburg ORACLE Deutschland B.V. & Co. KG Hauptverwaltung: Riesstr. 25, D-80992 M?nchen Registergericht: Amtsgericht M?nchen, HRA 95603 Komplement?rin: ORACLE Deutschland Verwaltung B.V. Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 Gesch?ftsf?hrer: Alexander van der Ven, Jan Schultheiss, Val Maher Oracle is committed to developing practices and products that help protect the environment From simon at cjnash.com Fri Jul 14 13:15:56 2017 From: simon at cjnash.com (Simon Nash) Date: Fri, 14 Jul 2017 14:15:56 +0100 Subject: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build In-Reply-To: <2ac4b05d05434369bdeb72f7018d00f8@sap.com> References: <6c85b34b225e417788f794a99269e9fb@sap.com> <5967D063.1010808@cjnash.com> <424e7257bc504fe09d967ce56d64afd1@sap.com> <2ac4b05d05434369bdeb72f7018d00f8@sap.com> Message-ID: <5968C40C.3010503@cjnash.com> Hi Matthias, Thanks very much for making this change. Best regards, Simon On 14/07/2017 12:47, Baesken, Matthias wrote: > After Simons question , I checked that the jdk10 builds still works with gcc 4.7 on linux x86_64 as well . > And created a new webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8184338.1/ > > - check adjusted to minimum gcc 4.7 > - common/doc/building.md adjusted (the part talking about gcc versions) > > > Best regards, Matthias > > > > -----Original Message----- > From: Baesken, Matthias > Sent: Freitag, 14. Juli 2017 08:36 > To: 'Simon Nash' > Cc: Erik Joelsson ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' ; Zeller, Arno > Subject: RE: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build > >> I am currently building JDK 9 for arm32 (hard float and soft float) using >> a Linaro GCC 4.7 cross-compiler (binary download). What is the reason that >> this cannot be supported for JDK 10? > > Hi Simon, > reason was that we know gcc-4.8 works nicely because we do a lot of builds (+tests) with this compiler . > For gcc 4.7 we just do not know ( but for old 4.3 / 4.4 it was obvious that the current flags do not work any more). > If you are using a gcc compiler < 4.8 only a warning is reported , it does not mean that you cannot build any more. > > My first suggestion was : > >>>> ( e.g. change to minimum gcc 4.6 or gcc 4.7 supporting the flags used in the build ) > > So if you see a benefit to test for minimum gcc 4.7 and not 4.8 I am fine with this too (Erik what do you think?). > We at SAP just cannot tell that gcc 4.7 still works (because our builds do not use it). > > Best regards, Matthias > > > > -----Original Message----- > From: Simon Nash [mailto:simon at cjnash.com] > Sent: Donnerstag, 13. Juli 2017 21:56 > To: Baesken, Matthias > Cc: Erik Joelsson ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' ; Zeller, Arno > Subject: Re: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build > > I am currently building JDK 9 for arm32 (hard float and soft float) using > a Linaro GCC 4.7 cross-compiler (binary download). What is the reason that > this cannot be supported for JDK 10? > > Best regards, > Simon > > On 13/07/2017 14:40, Baesken, Matthias wrote: >> Hi Erik, I prepared the change : >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8184338/ >> >> bug : >> >> https://bugs.openjdk.java.net/browse/JDK-8184338 >> >> Asking for review(s) ... >> >> >> Best regards, Matthias >> >> >> >> >> -----Original Message----- >> From: Erik Joelsson [mailto:erik.joelsson at oracle.com] >> Sent: Donnerstag, 13. Juli 2017 13:34 >> To: Baesken, Matthias ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' >> Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build >> >> Hello, >> >> That would be the correct place. If you prepare the change and send the >> review I can sponsor the push. >> >> /Erik >> >> On 2017-07-13 13:16, Baesken, Matthias wrote: >>> Hi Erik, >>> >>>> AFAIK, the only reason we support GCC versions older than 4.9 is for you >>>> guys at SAP, so if you would suggest dropping support, that would of >>>> course be the simplest solution. >>> for jdk10 it is fine for us to use minimum gcc 4.8 . That would be probably the most simple solution to avoid running into the "-fno-var-tracking-assignments" issue >>> ( I now and then run into it when I forget to set the PATH to gcc-4.8 one should use for compiling on the SLES11 machine, so I pick up instead the default gcc 4.3.4 from the machine , >>> configure works nicely and then the build fails in the middle of the HS make ) >>> >>> autoconf/toolchain.m4 >>> >>> TOOLCHAIN_MINIMUM_VERSION_gcc="4.8" >>> >>> Is there right place to adjust I think, I guess we need someone for Oracle to regenerate the generated-configure.sh files ? >>> >>> Best regards, Matthias >>> >>> >>> >>> >>> -----Original Message----- >>> From: Erik Joelsson [mailto:erik.joelsson at oracle.com] >>> Sent: Donnerstag, 13. Juli 2017 11:59 >>> To: Baesken, Matthias ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' >>> Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build >>> >>> Hello Matthias, >>> >>> AFAIK, the only reason we support GCC versions older than 4.9 is for you >>> guys at SAP, so if you would suggest dropping support, that would of >>> course be the simplest solution. >>> >>> If you want to keep support but make the use of this flag optional, the >>> preferred method is to add a test in flags.m4. We have macros defined >>> for this. FLAGS_COMPILER_CHECK_ARGUMENTS or >>> FLAGS_CXX_COMPILER_CHECK_ARGUMENTS would be suitable in this case. Use >>> it to check if the flag is valid for the current compiler. If so, you >>> can define a variable >>> "CXXFLAGS_GCC_NO_VAR_TRACKING_ASSIGNMENTS=-fno-var-tracking-assignment" >>> and empty otherwise, and in the makefile use this variable in the >>> assignement. >>> >>> We want to avoid static shell expressions in the makefiles if possible >>> and keep that kind of logic in configure. It's also better to explicitly >>> check for features rather than versions. >>> >>> /Erik >>> >>> >>> On 2017-07-13 11:21, Baesken, Matthias wrote: >>>> Hello, when building jdk10 on Suse Linux 11 with default gcc/g++ 4.3.4 installed, I was running into >>>> compilation errors because of the missing support for g++ flag -fno-var-tracking-assignments . >>>> >>>> It seems gcc-4.6 has the -fvar-tracking-assignments / -fnovar-tracking-assignments flags , see >>>> >>>> https://gcc.gnu.org/onlinedocs/gcc-4.6.4/gcc/Option-Summary.html#Option-Summary >>>> >>>> I have no gcc-4.6 at hand but could verify that gcc-4.7 and gcc-4.8 "know" the flag. >>>> There are various solutions one could do to avoid the compilation error . >>>> >>>> >>>> 1) disallow usage of old gcc versions here : >>>> >>>> autoconf/toolchain.m4 >>>> >>>> TOOLCHAIN_MINIMUM_VERSION_gcc="4.3" >>>> >>>> ( e.g. change to minimum gcc 4.6 or gcc 4.7 supporting the flags used in the build ) >>>> >>>> >>>> 2) remove the flag -fno-var-tracking-assignments for older gcc versions here : >>>> (in a similar way it was done : http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/37e693211deb ) >>>> >>>> hotspot/make/lib/JvmOverrideFiles.gmk-32-ifeq ($(TOOLCHAIN_TYPE), gcc) >>>> hotspot/make/lib/JvmOverrideFiles.gmk:33: BUILD_LIBJVM_vmStructs.cpp_CXXFLAGS := -fno-var-tracking-assignments -O0 >>>> hotspot/make/lib/JvmOverrideFiles.gmk:34: BUILD_LIBJVM_jvmciCompilerToVM.cpp_CXXFLAGS := -fno-var-tracking-assignments >>>> hotspot/make/lib/JvmOverrideFiles.gmk-35-endif >>>> >>>> What is your preferred solution ? >>>> >>>> >>>> Thanks, Matthias >>>> >>>> > From bob.vandette at oracle.com Fri Jul 14 14:22:26 2017 From: bob.vandette at oracle.com (Bob Vandette) Date: Fri, 14 Jul 2017 10:22:26 -0400 Subject: JEP [DRAFT]: Container aware Java Message-ID: I?d like to propose the following JEP that will enhance the Java runtime to be more container aware. This will allow the Java runtime to performance better ergonomics and provide better execution time statistics when running in a container. JEP Issue: https://bugs.openjdk.java.net/browse/JDK-8182070 Here?s a Text dump of the JEP contents for your convenience: Summary ------- Container aware Java runtime Goals ----- Enhance the JVM and Core libraries to detect running in a container and adapt to the system resources available to it. This JEP will only support Docker on Linux-x64 although the design should be flexible enough to allow support for other platforms and container technologies. The initial focus will be on Linux low level container technology such as cgroups so that we will be able to easily support other container technologies running on Linux in addition to Docker. Non-Goals --------- It is not a goal of this JEP to support any platform other than Docker container technology running on Linux x64. Success Metrics --------------- Success will be measured by the improved efficiency of running multiple Java containers on a host system with out of the box options. Motivation ---------- Container technology is becoming more and more prevalent in Cloud based applications. This technology provides process isolation and allows the platform vendor to specify limits and alter the behavior of a process running inside a container that the Java runtime is not aware of. This causes the Java runtime to potentially attempt to use more system resources than are available to it causing performance degradation or even termination. Description ----------- This enhancement will be made up of the following work items: A. Detecting if Java is running in a container. The Java runtime, as well as any tests that we might write for this feature, will need to be able to detect that the current Java process is running in a container. I propose that we add a new JVM_ native function that returns a boolean true value if we are running inside a container. JVM_InContainer will return true if the currently running process is running in a container, otherwise false will be returned. B. Exposing container resource limits and configuration. There are several configuration options and limits that can be imposed upon a running container. Not all of these are important to a running Java process. We clearly want to be able to detect how many CPUs have been allocated to our process along with the maximum amount of memory that we be allocated but there are other options that we might want to base runtime decisions on. In addition, since Container typically impose limits on system resources, they also provide the ability to easily access the amount of consumption of these resources. I intent on providing this information in addition to the configuration data. I propose adding a new jdk.internal.Platform class that will allow access to this information. Since some of this information is needed during the startup of the VM, I propose that much of the implementation of the methods in the Platform class be done in the VM and exposed as JVM_xxxxxx functions. In hotspot, the JVM_xxxxxx function will be implemented via the os.hpp interface. Here are the categories of configuration and consumption statistics that will be made available (The exact API is TBD): isContainerized Memory Limit Total Memory Limit Soft Memory Limit Max Memory Usage Current Memory Usage Maximum Kernel Memory CPU Shares CPU Period CPU Quote Number of CPUs CPU Sets CPU Set Memory Nodes CPU Usage CPU Usage Per CPU Block I/O Weight Block I/O Device Weight Device I/O Read Rate Device I/O Write Rate OOM Kill Enabled OOM Score Adjustment Memory Swappiness Shared Memory Size TODO: 1. Need to specify the exact arguments and return format for these accessor functions. C. Adjusting Java runtime configuration based on limits. Java startup normally queries the operating system in order to setup runtime defaults for things such as the number of GC threads and default memory limits. When running in a container, the operating system functions used provide information about the host and does not include the containers configuration and limits. The VM and core libraries will be modified as part of this JEP to first determine if the current running process is running in a container. It will then cause the runtime to use the container values rather than the general operating system functions for configuring and managing the Java process. There have been a few attempts to correct some of these issue in the VM but they are not complete. The CPU detection in the VM currently only handles a container that limits cpu usage via CPU sets. If the Docker --cpu or --cpu-period along with --cpu-quota options are specified, it currently has no effect on the VMs configuration. The experimental memory detection that has been implemented only impacts the Heap selection and does not apply to the os::physical_memory or os::available_memory low level functions. This leaves other parts of the VM and core libraries to believe there is more memory available than there actually is. The Numa support available in the VM is also not correct when running in a container. The number of available memory nodes and enabled nodes as reported by the libnuma library does not take into account the impact of the Docker --cpuset-mems option which restricts which memory nodes the container can use. Inside the container, the file /proc/{pid}/self does report the correct Cpus_allowed and Mems_Allowed but libnuma doesn't respect this. This has been verified via the numactl utility. To correct these shortcomings and make this support more robust, here's a list of the current cgroup subsystems that we be examined in order to update the internal VM and core library configuration. **Number of CPUs** Use a combination of number_of_cpus() and cpu_sets() in order to determine how many processors are available to the process and adjust the JVMs os::active_processor_count appropriately. The number_of_cpus() will be calculated based on the cpu_quota() and cpu_period() using this formula: number_of_cpus() = cpu_quota() / cpu_period(). Since it's not currently possible to understand the relative weight of the running container against all other containers, altering the cpu_shares of a running container will have no affect on altering Java's configuration. **Total available memory** Use the memory_limit() value from the cgroup file system to initialize the os::physical_memory() value in the VM. This value will propagate to all other parts of the Java runtime. We might also consider examining the soft_memory_limit and total_memory_limit in addition to the memory_limit during the ergonomics startup processing in order to fine tuning some of the other VM settings. **CPU Memory Nodes** Use cpu_set_memory_nodes() to configure the os::numa support. **Memory usage** Use memory_usage_in_bytes() for providing os::available_memory() by subtracting the usage from the total available memory allocated to the container. D. Adding container configuration to error crash logs. As as troubleshooting aid, we will dump any available container statistics to the hotspot error log. E. Adding a startup flag to enable/disable this support. Add a -XX:+UseContainerSupport VM option that will be used to enable this support. The default will be off until this feature is proven. Alternatives ------------ There are a few existing RFE's filed that could be used to enhance the current experimental implementation rather than taking the JEP route. Testing ------- Docker/container specific tests should be added in order to validate the functionality being provided with this JEP. Risks and Assumptions --------------------- Docker is currently based on cgroups v1. Cgroups v2 is also available but is incomplete and not yet supported by Docker. It's possible that v2 could replace v1 in an incompatible way rendering this work unusable until it is upgraded. Dependencies ----------- None at this time. From matthias.baesken at sap.com Fri Jul 14 15:05:02 2017 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 14 Jul 2017 15:05:02 +0000 Subject: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build In-Reply-To: <948004c4-63e7-b0e4-1820-bf2351fc2327@oracle.com> References: <6c85b34b225e417788f794a99269e9fb@sap.com> <5967D063.1010808@cjnash.com> <424e7257bc504fe09d967ce56d64afd1@sap.com> <2ac4b05d05434369bdeb72f7018d00f8@sap.com> <948004c4-63e7-b0e4-1820-bf2351fc2327@oracle.com> Message-ID: <738dc2d22854475f9a15e163f7038872@sap.com> Thanks , can you push it ( as far as I know someone from Oracle needs to push it because generated-configure.sh is changed ) ? Best regards, Matthias -----Original Message----- From: Erik Joelsson [mailto:erik.joelsson at oracle.com] Sent: Freitag, 14. Juli 2017 14:43 To: Baesken, Matthias ; Simon Nash Cc: 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' ; Zeller, Arno Subject: Re: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build This looks good to me. /Erik On 2017-07-14 13:47, Baesken, Matthias wrote: > After Simons question , I checked that the jdk10 builds still works with gcc 4.7 on linux x86_64 as well . > And created a new webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8184338.1/ > > - check adjusted to minimum gcc 4.7 > - common/doc/building.md adjusted (the part talking about gcc versions) > > > Best regards, Matthias > > > > -----Original Message----- > From: Baesken, Matthias > Sent: Freitag, 14. Juli 2017 08:36 > To: 'Simon Nash' > Cc: Erik Joelsson ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' ; Zeller, Arno > Subject: RE: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build > >> I am currently building JDK 9 for arm32 (hard float and soft float) using >> a Linaro GCC 4.7 cross-compiler (binary download). What is the reason that >> this cannot be supported for JDK 10? > Hi Simon, > reason was that we know gcc-4.8 works nicely because we do a lot of builds (+tests) with this compiler . > For gcc 4.7 we just do not know ( but for old 4.3 / 4.4 it was obvious that the current flags do not work any more). > If you are using a gcc compiler < 4.8 only a warning is reported , it does not mean that you cannot build any more. > > My first suggestion was : > >>>> ( e.g. change to minimum gcc 4.6 or gcc 4.7 supporting the flags used in the build ) > So if you see a benefit to test for minimum gcc 4.7 and not 4.8 I am fine with this too (Erik what do you think?). > We at SAP just cannot tell that gcc 4.7 still works (because our builds do not use it). > > Best regards, Matthias > > > > -----Original Message----- > From: Simon Nash [mailto:simon at cjnash.com] > Sent: Donnerstag, 13. Juli 2017 21:56 > To: Baesken, Matthias > Cc: Erik Joelsson ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' ; Zeller, Arno > Subject: Re: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build > > I am currently building JDK 9 for arm32 (hard float and soft float) using > a Linaro GCC 4.7 cross-compiler (binary download). What is the reason that > this cannot be supported for JDK 10? > > Best regards, > Simon > > On 13/07/2017 14:40, Baesken, Matthias wrote: >> Hi Erik, I prepared the change : >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8184338/ >> >> bug : >> >> https://bugs.openjdk.java.net/browse/JDK-8184338 >> >> Asking for review(s) ... >> >> >> Best regards, Matthias >> >> >> >> >> -----Original Message----- >> From: Erik Joelsson [mailto:erik.joelsson at oracle.com] >> Sent: Donnerstag, 13. Juli 2017 13:34 >> To: Baesken, Matthias ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' >> Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build >> >> Hello, >> >> That would be the correct place. If you prepare the change and send the >> review I can sponsor the push. >> >> /Erik >> >> On 2017-07-13 13:16, Baesken, Matthias wrote: >>> Hi Erik, >>> >>>> AFAIK, the only reason we support GCC versions older than 4.9 is for you >>>> guys at SAP, so if you would suggest dropping support, that would of >>>> course be the simplest solution. >>> for jdk10 it is fine for us to use minimum gcc 4.8 . That would be probably the most simple solution to avoid running into the "-fno-var-tracking-assignments" issue >>> ( I now and then run into it when I forget to set the PATH to gcc-4.8 one should use for compiling on the SLES11 machine, so I pick up instead the default gcc 4.3.4 from the machine , >>> configure works nicely and then the build fails in the middle of the HS make ) >>> >>> autoconf/toolchain.m4 >>> >>> TOOLCHAIN_MINIMUM_VERSION_gcc="4.8" >>> >>> Is there right place to adjust I think, I guess we need someone for Oracle to regenerate the generated-configure.sh files ? >>> >>> Best regards, Matthias >>> >>> >>> >>> >>> -----Original Message----- >>> From: Erik Joelsson [mailto:erik.joelsson at oracle.com] >>> Sent: Donnerstag, 13. Juli 2017 11:59 >>> To: Baesken, Matthias ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' >>> Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build >>> >>> Hello Matthias, >>> >>> AFAIK, the only reason we support GCC versions older than 4.9 is for you >>> guys at SAP, so if you would suggest dropping support, that would of >>> course be the simplest solution. >>> >>> If you want to keep support but make the use of this flag optional, the >>> preferred method is to add a test in flags.m4. We have macros defined >>> for this. FLAGS_COMPILER_CHECK_ARGUMENTS or >>> FLAGS_CXX_COMPILER_CHECK_ARGUMENTS would be suitable in this case. Use >>> it to check if the flag is valid for the current compiler. If so, you >>> can define a variable >>> "CXXFLAGS_GCC_NO_VAR_TRACKING_ASSIGNMENTS=-fno-var-tracking-assignment" >>> and empty otherwise, and in the makefile use this variable in the >>> assignement. >>> >>> We want to avoid static shell expressions in the makefiles if possible >>> and keep that kind of logic in configure. It's also better to explicitly >>> check for features rather than versions. >>> >>> /Erik >>> >>> >>> On 2017-07-13 11:21, Baesken, Matthias wrote: >>>> Hello, when building jdk10 on Suse Linux 11 with default gcc/g++ 4.3.4 installed, I was running into >>>> compilation errors because of the missing support for g++ flag -fno-var-tracking-assignments . >>>> >>>> It seems gcc-4.6 has the -fvar-tracking-assignments / -fnovar-tracking-assignments flags , see >>>> >>>> https://gcc.gnu.org/onlinedocs/gcc-4.6.4/gcc/Option-Summary.html#Option-Summary >>>> >>>> I have no gcc-4.6 at hand but could verify that gcc-4.7 and gcc-4.8 "know" the flag. >>>> There are various solutions one could do to avoid the compilation error . >>>> >>>> >>>> 1) disallow usage of old gcc versions here : >>>> >>>> autoconf/toolchain.m4 >>>> >>>> TOOLCHAIN_MINIMUM_VERSION_gcc="4.3" >>>> >>>> ( e.g. change to minimum gcc 4.6 or gcc 4.7 supporting the flags used in the build ) >>>> >>>> >>>> 2) remove the flag -fno-var-tracking-assignments for older gcc versions here : >>>> (in a similar way it was done : http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/37e693211deb ) >>>> >>>> hotspot/make/lib/JvmOverrideFiles.gmk-32-ifeq ($(TOOLCHAIN_TYPE), gcc) >>>> hotspot/make/lib/JvmOverrideFiles.gmk:33: BUILD_LIBJVM_vmStructs.cpp_CXXFLAGS := -fno-var-tracking-assignments -O0 >>>> hotspot/make/lib/JvmOverrideFiles.gmk:34: BUILD_LIBJVM_jvmciCompilerToVM.cpp_CXXFLAGS := -fno-var-tracking-assignments >>>> hotspot/make/lib/JvmOverrideFiles.gmk-35-endif >>>> >>>> What is your preferred solution ? >>>> >>>> >>>> Thanks, Matthias >>>> >>>> From volker.simonis at gmail.com Fri Jul 14 15:41:32 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 14 Jul 2017 17:41:32 +0200 Subject: RFR(XS) [10]: 8184672: [ppc64] The fix for '8174749: Use hash table/oops for MemberName table' is broken on ppc64 Message-ID: Hi, can somebody please review the following small, ppc64-only change: http://cr.openjdk.java.net/~simonis/webrevs/2017/8184672 https://bugs.openjdk.java.net/browse/JDK-8184672 The change for "JDK-8174749: Use hash table/oops for MemberName table" introduced a new indirection for the 'vmtarget' field in java.lang.invoke.MemberName. 'vmtarget' was previously directly stored in MemberName and is now a field of the new class ResolvedMethodName, an object of which is now stored in MemberName. The change contained some architecture-specific code which updated the HotSpot to be aware of the new indirection (i.e. instead of loading 'vmtarget' directly from a MemberName object the code now has to first load the ResolvedMethodName object from MemeberName and subsequently the 'vmtarget' field from the ResolvedMethodName object. This code is trivial, but on ppc64, the new version has incorrectly 'misused' the register which contained the MemberName, as storage for the intermediate ResolvedMethodName object. If the MemberName object was stored in an argument register, that register was overwritten in MethodHandles::generate_method_handle_dispatch (when generating the intrinsics for MethodHandle.linkToStatic or MethodHandle.linkToSpecial) and could lead to the following error down the call chain: # Internal Error (jdk10-hs/hotspot/src/cpu/ppc/vm/macroAssembler_ppc.cpp:5493), pid=20194, tid=20202 # guarantee(false) failed: PPC assembly code requires stop: MemberName required for invokeVirtual etc. V [libjvm.so+0x14a0600] stop_on_request(int, char const*)+0x7c J 110 java.lang.invoke.MethodHandle.linkToStatic(Ljava/lang/Object;IILjava/lang/invoke/MemberName;)Ljava/lang/Object; java.base at 10-internal (0 bytes) @ 0x00000fff852d9774 [0x00000fff852d9700+0x0000000000000074] J 309 c2 java.lang.invoke.LambdaForm$MH.invoke(Ljava/lang/Object;J)Ljava/lang/Object; java.base at 10-internal (161 bytes) @ 0x00000fff852fea98 [0x00000fff852fe900+0x0000000000000198] J 308 c2 java.lang.invoke.LambdaForm$MH.linkToTargetMethod(JLjava/lang/Object;)Ljava/lang/Object; java.base at 10-internal (9 bytes) @ 0x00000fff852fe23c [0x00000fff852fe200+0x000000000000003c] j LambdaForm.foo(J)Ljava/lang/String;+86 j LambdaForm.main([Ljava/lang/String;)V+8 The fix for the problem is trivial. Just use the final destination register for 'vmtarget' also as intermediate storage for the ResolvedMethodName object (thus leaving the register holding the MemberName reference untouched). Thank you and best regards, Volker From volker.simonis at gmail.com Fri Jul 14 16:15:24 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 14 Jul 2017 18:15:24 +0200 Subject: JEP [DRAFT]: Container aware Java In-Reply-To: References: Message-ID: Hi Bob, thanks for starting this JEP. I think it is long overdue and will be very helpful for many people. That said, I don't see any mentioning of the targeted Java release. As JEPs usually go into the development version first, I suppose the work for this JEP will be done in JDK10. However, taking into account the current release cycles, the usefulness of this JEP for the Java community will significantly depend on the its availability in Java 9 (at least) and Java 8 (hopefully?). Any thoughts on this? You also mention that this JEP is only about Docker support on Linux/x86_64. I'd argue that we should at least address at least all Docker supported Linux platforms. I'm pretty sure our colleagues from IBM would like to support this on Linux/ppc64 and Linux/s390 as well. Moreover, the suggested API and the new JVM_xxxxxx functions should be at least designed in such a way that other virtualisation environment such as VMWare/VirtualBox/Xen/KVM can take advantage of it. I'm also not sure if a completely new API solely for Docker containers is making sense and is worth the effort? I understand that containers and virtualisation are two different things, but they both impose similar restrictions to the VM (i.e. virtualisation may present a huge number of CPUs to the OS/VM which in reality are mapped to a single, physical CPU) so maybe we should design the new API in such a way that it can be used to tackle both problems. Finally, you mention in section D. that all the information should be logged the to error crash logs. That's good but maybe you could add that all the detection and all the findings should also be made available through the new Unified JVM Logging facility (JEP 158)? Regards, Volker On Fri, Jul 14, 2017 at 4:22 PM, Bob Vandette wrote: > I?d like to propose the following JEP that will enhance the Java runtime to be more container aware. > This will allow the Java runtime to performance better ergonomics and provide better > execution time statistics when running in a container. > > > JEP Issue: > > https://bugs.openjdk.java.net/browse/JDK-8182070 > > Here?s a Text dump of the JEP contents for your convenience: > > > Summary > ------- > > Container aware Java runtime > > Goals > ----- > > Enhance the JVM and Core libraries to detect running in a container and adapt to the system resources available to it. This JEP will only support Docker on Linux-x64 although the design should be flexible enough to allow support for other platforms and container technologies. The initial focus will be on Linux low level container technology such as cgroups so that we will be able to easily support other container technologies running on Linux in addition to Docker. > > Non-Goals > --------- > > It is not a goal of this JEP to support any platform other than Docker container technology running on Linux x64. > > Success Metrics > --------------- > > Success will be measured by the improved efficiency of running multiple Java containers on a host system with out of the box options. > > Motivation > ---------- > > Container technology is becoming more and more prevalent in Cloud based applications. This technology provides process isolation and allows the platform vendor to specify limits and alter the behavior of a process running inside a container that the Java runtime is not aware of. This causes the Java runtime to potentially attempt to use more system resources than are available to it causing performance degradation or even termination. > > > Description > ----------- > > This enhancement will be made up of the following work items: > > A. Detecting if Java is running in a container. > > The Java runtime, as well as any tests that we might write for this feature, will need to be able to detect that the current Java process is running in a container. I propose that we add a new JVM_ native function that returns a boolean true value if we are running inside a container. > > JVM_InContainer will return true if the currently running process is running in a container, otherwise false will be returned. > > B. Exposing container resource limits and configuration. > > There are several configuration options and limits that can be imposed upon a running container. Not all of these > are important to a running Java process. We clearly want to be able to detect how many CPUs have been allocated to our process along with the maximum amount of memory that we be allocated but there are other options that we might want to base runtime decisions on. > > In addition, since Container typically impose limits on system resources, they also provide the ability to easily access the amount of consumption of these resources. I intent on providing this information in addition to the configuration data. > > I propose adding a new jdk.internal.Platform class that will allow access to this information. Since some of this information is needed during the startup of the VM, I propose that much of the implementation of the methods in the Platform class be done in the VM and exposed as JVM_xxxxxx functions. In hotspot, the JVM_xxxxxx function will be implemented via the os.hpp interface. > > Here are the categories of configuration and consumption statistics that will be made available (The exact API is TBD): > > isContainerized > Memory Limit > Total Memory Limit > Soft Memory Limit > Max Memory Usage > Current Memory Usage > Maximum Kernel Memory > CPU Shares > CPU Period > CPU Quote > Number of CPUs > CPU Sets > CPU Set Memory Nodes > CPU Usage > CPU Usage Per CPU > Block I/O Weight > Block I/O Device Weight > Device I/O Read Rate > Device I/O Write Rate > OOM Kill Enabled > OOM Score Adjustment > Memory Swappiness > Shared Memory Size > > TODO: > 1. Need to specify the exact arguments and return format for these accessor functions. > > C. Adjusting Java runtime configuration based on limits. > > Java startup normally queries the operating system in order to setup runtime defaults for things such as the number of GC threads and default memory limits. When running in a container, the operating system functions used provide information about the host and does not include the containers configuration and limits. The VM and core libraries will be modified as part of this JEP to first determine if the current running process is running in a container. It will then cause the runtime to use the container values rather than the general operating system functions for configuring and managing the Java process. There have been a few attempts to correct some of these issue in the VM but they are not complete. The CPU detection in the VM currently only handles a container that limits cpu usage via CPU sets. If the Docker --cpu or --cpu-period along with --cpu-quota options are specified, it currently has no effect on the VMs configuration. > > The experimental memory detection that has been implemented only impacts the Heap selection and does not apply to the os::physical_memory or os::available_memory low level functions. This leaves other parts of the VM and > core libraries to believe there is more memory available than there actually is. > > The Numa support available in the VM is also not correct when running in a container. The number of available memory nodes and enabled nodes as reported by the libnuma library does not take into account the impact of the Docker --cpuset-mems option which restricts which memory nodes the container can use. Inside the container, the file /proc/{pid}/self does report the correct Cpus_allowed and Mems_Allowed but libnuma doesn't respect this. This has been verified via the numactl utility. > > To correct these shortcomings and make this support more robust, here's a list of the current cgroup subsystems that we be examined in order to update the internal VM and core library configuration. > > **Number of CPUs** > > Use a combination of number_of_cpus() and cpu_sets() in order to determine how many processors are available to the process and adjust the JVMs os::active_processor_count appropriately. The number_of_cpus() will be calculated based on the cpu_quota() and cpu_period() using this formula: number_of_cpus() = cpu_quota() / cpu_period(). Since it's not currently possible to understand the relative weight of the running container against all other containers, altering the cpu_shares of a running container will have no affect on altering Java's configuration. > > **Total available memory** > > Use the memory_limit() value from the cgroup file system to initialize the os::physical_memory() value in the VM. This value will propagate to all other parts of the Java runtime. > > We might also consider examining the soft_memory_limit and total_memory_limit in addition to the memory_limit during the ergonomics startup processing in order to fine tuning some of the other VM settings. > > **CPU Memory Nodes** > > Use cpu_set_memory_nodes() to configure the os::numa support. > > **Memory usage** > > Use memory_usage_in_bytes() for providing os::available_memory() by subtracting the usage from the total available memory allocated to the container. > > D. Adding container configuration to error crash logs. > > As as troubleshooting aid, we will dump any available container statistics to the hotspot error log. > > E. Adding a startup flag to enable/disable this support. > > Add a -XX:+UseContainerSupport VM option that will be used to enable this support. The default will be off until this feature is proven. > > Alternatives > ------------ > > There are a few existing RFE's filed that could be used to enhance the current experimental implementation rather than taking the JEP route. > > Testing > ------- > > Docker/container specific tests should be added in order to validate the functionality being provided with this JEP. > > Risks and Assumptions > --------------------- > > Docker is currently based on cgroups v1. Cgroups v2 is also available but is incomplete and not yet supported by Docker. It's possible that v2 could replace v1 in an incompatible way rendering this work unusable until it is upgraded. > > Dependencies > ----------- > > None at this time. > From erik.osterlund at oracle.com Fri Jul 14 16:28:15 2017 From: erik.osterlund at oracle.com (=?utf-8?Q?Erik_=C3=96sterlund?=) Date: Fri, 14 Jul 2017 18:28:15 +0200 Subject: RFR (M): 8184334: Generalizing Atomic with templates Message-ID: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> Hi, Bug: https://bugs.openjdk.java.net/browse/JDK-8184334 Webrev: http://cr.openjdk.java.net/~eosterlund/8184334/webrev.00/ The Atomic class mostly uses the JNI-specific j-types (jbyte, jshort, jint, jlong). This has led to wide-spread pollution of j-types in the C++ code base. Another problem is that when dealing with certain inherently not j-typed types like pointers, hacks were built that allowed them using _ptr member functions that had to be specialized on every architecture supported by Java. Yet another problem is yet another exception, like size_t, that does not quite fit being a pointer and isn't necessarily mapped to any j-type. Then yet more exceptions were built on top. And what about intx? You get the idea by now. As a remedy to the above mentioned problems, I propose to generalize the Atomic implementation to use templates instead. The new Atomic API handles pointers, all integer and floating point types that the underlying architecture supports. To achieve this, a new metaprogramming tool called IntegerTypes was built, that allows us to safely cast between different integral, floating point and pointer types of the same size while preserving the bit pattern of the values. This allows Atomic to take primitive types and canonicalize them to the same integer type that is passed to the platform layer. As for atomic operations on integral types like add/inc/dec, pointer scaling is now performed. That is, the functional behaviour is modelled after the +=, ++ and -- operators correspondingly. The X_ptr member functions have been deprecated, but are still there and can be used with identical behaviour as they had before. But new code should just use the non-ptr member functions instead. The platform layer specializes the specialized_X member functions that take canonicalized signed integers as input, and turn them to some platform-specific instruction sequence. A nice benefit with these changes is that the platform-specific code has gotten a lot of pointless redundancies removed such as how to perform atomic loads and stores on native word or less sized integers, or having inc/dec implementations that just reuse add. As for testing, new gtests were written for IntegerTypes as well as Atomic to make it more convincing that the functional behaviour works. Apart from that, I have run JPRT, JTREG, RBT hs-tier3 and a few local tonga test lists including nsk.jdi, nsk.monitoring, nsk.stress, vm.gc on Solaris SPARC T7-4 and Linux AMD64. As for oracle-external ports, I would appreciate if port maintainers could check if it seems to make sense. I have done my best to try to make the appropriate changes where needed. Special thanks go to Kim Barrett that has worked closely with me on this and whom I exchanged many emails with, and so I would like him to be a co-author. Thanks, /Erik From bob.vandette at oracle.com Fri Jul 14 16:54:36 2017 From: bob.vandette at oracle.com (Bob Vandette) Date: Fri, 14 Jul 2017 12:54:36 -0400 Subject: JEP [DRAFT]: Container aware Java In-Reply-To: References: Message-ID: > On Jul 14, 2017, at 12:15 PM, Volker Simonis wrote: > > Hi Bob, > > thanks for starting this JEP. I think it is long overdue and will be > very helpful for many people. > Thanks for taking a look at the JEP. > That said, I don't see any mentioning of the targeted Java release. As > JEPs usually go into the development version first, I suppose the work > for this JEP will be done in JDK10. However, taking into account the > current release cycles, the usefulness of this JEP for the Java > community will significantly depend on the its availability in Java 9 > (at least) and Java 8 (hopefully?). Any thoughts on this? I am certainly going to do my best to try to get this feature into JDK10 and I understand your concern about supporting Java 8 and 9. The decision to backport this feature will have to be done based on the usual criteria (complexity, usefulness, resource availability and compatibility). It may be possible to add some subset of this functionality in a backport but we?ll have to wait until the design is further along. > > You also mention that this JEP is only about Docker support on > Linux/x86_64. I'd argue that we should at least address at least all > Docker supported Linux platforms. I'm pretty sure our colleagues from > IBM would like to support this on Linux/ppc64 and Linux/s390 as well. Right now I don?t expect to have to write CPU specific logic. Therefore supporting ppc64, s390 or even aarch64 should boil down to testing. > > Moreover, the suggested API and the new JVM_xxxxxx functions should be > at least designed in such a way that other virtualisation environment > such as VMWare/VirtualBox/Xen/KVM can take advantage of it. I'm also > not sure if a completely new API solely for Docker containers is > making sense and is worth the effort? I understand that containers and > virtualisation are two different things, but they both impose similar > restrictions to the VM (i.e. virtualisation may present a huge number > of CPUs to the OS/VM which in reality are mapped to a single, physical > CPU) so maybe we should design the new API in such a way that it can > be used to tackle both problems. I will try to keep that in mind. I will have to look into the feasibility of accessing host configuration from within a container or virtual machine. For containers, it seems to me that would break the intent of isolation. I know that host information does leak into containers through the proc file system but I suspect this was done for simplicity and compatibility reasons. > > Finally, you mention in section D. that all the information should be > logged the to error crash logs. That's good but maybe you could add > that all the detection and all the findings should also be made > available through the new Unified JVM Logging facility (JEP 158)? That?s a good idea. I will add that to the JEP. Thanks, Bob. > > Regards, > Volker > > > On Fri, Jul 14, 2017 at 4:22 PM, Bob Vandette wrote: >> I?d like to propose the following JEP that will enhance the Java runtime to be more container aware. >> This will allow the Java runtime to performance better ergonomics and provide better >> execution time statistics when running in a container. >> >> >> JEP Issue: >> >> https://bugs.openjdk.java.net/browse/JDK-8182070 >> >> Here?s a Text dump of the JEP contents for your convenience: >> >> >> Summary >> ------- >> >> Container aware Java runtime >> >> Goals >> ----- >> >> Enhance the JVM and Core libraries to detect running in a container and adapt to the system resources available to it. This JEP will only support Docker on Linux-x64 although the design should be flexible enough to allow support for other platforms and container technologies. The initial focus will be on Linux low level container technology such as cgroups so that we will be able to easily support other container technologies running on Linux in addition to Docker. >> >> Non-Goals >> --------- >> >> It is not a goal of this JEP to support any platform other than Docker container technology running on Linux x64. >> >> Success Metrics >> --------------- >> >> Success will be measured by the improved efficiency of running multiple Java containers on a host system with out of the box options. >> >> Motivation >> ---------- >> >> Container technology is becoming more and more prevalent in Cloud based applications. This technology provides process isolation and allows the platform vendor to specify limits and alter the behavior of a process running inside a container that the Java runtime is not aware of. This causes the Java runtime to potentially attempt to use more system resources than are available to it causing performance degradation or even termination. >> >> >> Description >> ----------- >> >> This enhancement will be made up of the following work items: >> >> A. Detecting if Java is running in a container. >> >> The Java runtime, as well as any tests that we might write for this feature, will need to be able to detect that the current Java process is running in a container. I propose that we add a new JVM_ native function that returns a boolean true value if we are running inside a container. >> >> JVM_InContainer will return true if the currently running process is running in a container, otherwise false will be returned. >> >> B. Exposing container resource limits and configuration. >> >> There are several configuration options and limits that can be imposed upon a running container. Not all of these >> are important to a running Java process. We clearly want to be able to detect how many CPUs have been allocated to our process along with the maximum amount of memory that we be allocated but there are other options that we might want to base runtime decisions on. >> >> In addition, since Container typically impose limits on system resources, they also provide the ability to easily access the amount of consumption of these resources. I intent on providing this information in addition to the configuration data. >> >> I propose adding a new jdk.internal.Platform class that will allow access to this information. Since some of this information is needed during the startup of the VM, I propose that much of the implementation of the methods in the Platform class be done in the VM and exposed as JVM_xxxxxx functions. In hotspot, the JVM_xxxxxx function will be implemented via the os.hpp interface. >> >> Here are the categories of configuration and consumption statistics that will be made available (The exact API is TBD): >> >> isContainerized >> Memory Limit >> Total Memory Limit >> Soft Memory Limit >> Max Memory Usage >> Current Memory Usage >> Maximum Kernel Memory >> CPU Shares >> CPU Period >> CPU Quote >> Number of CPUs >> CPU Sets >> CPU Set Memory Nodes >> CPU Usage >> CPU Usage Per CPU >> Block I/O Weight >> Block I/O Device Weight >> Device I/O Read Rate >> Device I/O Write Rate >> OOM Kill Enabled >> OOM Score Adjustment >> Memory Swappiness >> Shared Memory Size >> >> TODO: >> 1. Need to specify the exact arguments and return format for these accessor functions. >> >> C. Adjusting Java runtime configuration based on limits. >> >> Java startup normally queries the operating system in order to setup runtime defaults for things such as the number of GC threads and default memory limits. When running in a container, the operating system functions used provide information about the host and does not include the containers configuration and limits. The VM and core libraries will be modified as part of this JEP to first determine if the current running process is running in a container. It will then cause the runtime to use the container values rather than the general operating system functions for configuring and managing the Java process. There have been a few attempts to correct some of these issue in the VM but they are not complete. The CPU detection in the VM currently only handles a container that limits cpu usage via CPU sets. If the Docker --cpu or --cpu-period along with --cpu-quota options are specified, it currently has no effect on the VMs configuration. >> >> The experimental memory detection that has been implemented only impacts the Heap selection and does not apply to the os::physical_memory or os::available_memory low level functions. This leaves other parts of the VM and >> core libraries to believe there is more memory available than there actually is. >> >> The Numa support available in the VM is also not correct when running in a container. The number of available memory nodes and enabled nodes as reported by the libnuma library does not take into account the impact of the Docker --cpuset-mems option which restricts which memory nodes the container can use. Inside the container, the file /proc/{pid}/self does report the correct Cpus_allowed and Mems_Allowed but libnuma doesn't respect this. This has been verified via the numactl utility. >> >> To correct these shortcomings and make this support more robust, here's a list of the current cgroup subsystems that we be examined in order to update the internal VM and core library configuration. >> >> **Number of CPUs** >> >> Use a combination of number_of_cpus() and cpu_sets() in order to determine how many processors are available to the process and adjust the JVMs os::active_processor_count appropriately. The number_of_cpus() will be calculated based on the cpu_quota() and cpu_period() using this formula: number_of_cpus() = cpu_quota() / cpu_period(). Since it's not currently possible to understand the relative weight of the running container against all other containers, altering the cpu_shares of a running container will have no affect on altering Java's configuration. >> >> **Total available memory** >> >> Use the memory_limit() value from the cgroup file system to initialize the os::physical_memory() value in the VM. This value will propagate to all other parts of the Java runtime. >> >> We might also consider examining the soft_memory_limit and total_memory_limit in addition to the memory_limit during the ergonomics startup processing in order to fine tuning some of the other VM settings. >> >> **CPU Memory Nodes** >> >> Use cpu_set_memory_nodes() to configure the os::numa support. >> >> **Memory usage** >> >> Use memory_usage_in_bytes() for providing os::available_memory() by subtracting the usage from the total available memory allocated to the container. >> >> D. Adding container configuration to error crash logs. >> >> As as troubleshooting aid, we will dump any available container statistics to the hotspot error log. >> >> E. Adding a startup flag to enable/disable this support. >> >> Add a -XX:+UseContainerSupport VM option that will be used to enable this support. The default will be off until this feature is proven. >> >> Alternatives >> ------------ >> >> There are a few existing RFE's filed that could be used to enhance the current experimental implementation rather than taking the JEP route. >> >> Testing >> ------- >> >> Docker/container specific tests should be added in order to validate the functionality being provided with this JEP. >> >> Risks and Assumptions >> --------------------- >> >> Docker is currently based on cgroups v1. Cgroups v2 is also available but is incomplete and not yet supported by Docker. It's possible that v2 could replace v1 in an incompatible way rendering this work unusable until it is upgraded. >> >> Dependencies >> ----------- >> >> None at this time. >> From kim.barrett at oracle.com Fri Jul 14 18:43:11 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 14 Jul 2017 14:43:11 -0400 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> Message-ID: <1C895D95-5762-45BB-AE7E-EEC1229765B2@oracle.com> > On Jul 14, 2017, at 12:28 PM, Erik ?sterlund wrote: > > Hi, > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8184334 > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8184334/webrev.00/ A few small things I overlooked in our pre-review discussions. ------------------------------------------------------------------------------ src/share/vm/metaprogramming/integerTypes.hpp 225 template 226 struct IntegerTypes::Cast VALUE_OBJ_CLASS_SPEC { 227 T operator()(U x) const { return (T)((void*)x); } 228 }; One of my pre-review comments seems to have been passed over here. I don't think we need the void* cast at all. And the rest of this file consistently prefers C++ cast operators over C-style casts, suggesting the cast to T should be performed using reinterpret_cast. ------------------------------------------------------------------------------ src/share/vm/runtime/atomic.hpp 29 #include "metaprogramming/conditional.hpp" 30 #include "metaprogramming/enableIf.hpp" ... 34 #include "metaprogramming/isSame.hpp" 35 #include "metaprogramming/removePointer.hpp" I think these are remnents of earlier versions of this code, and have ended up not being used here after all. ------------------------------------------------------------------------------ src/os_cpu/linux_x86/vm/atomic_linux_x86.hpp 143 volatile jlong dest; jlong => int64_t ? ------------------------------------------------------------------------------ src/os_cpu/linux_arm/vm/atomic_linux_arm.hpp 150 jlong rv; jlong => int64_t ? 176 intptr_t val; 192 intptr_t old_val; intptr_t => int64_t ? ------------------------------------------------------------------------------ src/os_cpu/bsd_x86/vm/atomic_bsd_x86.hpp 143 volatile jlong dest; jlong => int64_t ? ------------------------------------------------------------------------------ From kim.barrett at oracle.com Fri Jul 14 18:56:21 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 14 Jul 2017 14:56:21 -0400 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> Message-ID: <85E80DD4-FE40-464A-8C76-44F4A2AA3F05@oracle.com> > On Jul 14, 2017, at 12:28 PM, Erik ?sterlund wrote: > > Hi, > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8184334 > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8184334/webrev.00/ ------------------------------------------------------------------------------ Another pre-review comment that seems to have been missed. (Your reply indicated the suggestion was accepted.) src/share/vm/runtime/atomic.hpp 250 typedef typename IntegerTypes::Signed::type Raw; intptr_t => U* 269 typedef typename IntegerTypes::Signed::type Raw; 286 typedef typename IntegerTypes::Signed::type Raw; intptr_t => T* ------------------------------------------------------------------------------ From coleen.phillimore at oracle.com Fri Jul 14 20:05:37 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 14 Jul 2017 16:05:37 -0400 Subject: RFR(XS) [10]: 8184672: [ppc64] The fix for '8174749: Use hash table/oops for MemberName table' is broken on ppc64 In-Reply-To: References: Message-ID: <29139297-19fe-20ca-df67-e35506aebe90@oracle.com> Hi Volker, This looks good. Sorry for the breakage- I thought my change was trivial and couldn't possibly break :0 Thanks! Coleen On 7/14/17 11:41 AM, Volker Simonis wrote: > Hi, > > can somebody please review the following small, ppc64-only change: > > http://cr.openjdk.java.net/~simonis/webrevs/2017/8184672 > https://bugs.openjdk.java.net/browse/JDK-8184672 > > The change for "JDK-8174749: Use hash table/oops for MemberName table" > introduced a new indirection for the 'vmtarget' field in > java.lang.invoke.MemberName. 'vmtarget' was previously directly stored > in MemberName and is now a field of the new class ResolvedMethodName, > an object of which is now stored in MemberName. > > The change contained some architecture-specific code which updated the > HotSpot to be aware of the new indirection (i.e. instead of loading > 'vmtarget' directly from a MemberName object the code now has to first > load the ResolvedMethodName object from MemeberName and subsequently > the 'vmtarget' field from the ResolvedMethodName object. > > This code is trivial, but on ppc64, the new version has incorrectly > 'misused' the register which contained the MemberName, as storage for > the intermediate ResolvedMethodName object. If the MemberName object > was stored in an argument register, that register was overwritten in > MethodHandles::generate_method_handle_dispatch (when generating the > intrinsics for MethodHandle.linkToStatic or > MethodHandle.linkToSpecial) and could lead to the following error down > the call chain: > > # Internal Error > (jdk10-hs/hotspot/src/cpu/ppc/vm/macroAssembler_ppc.cpp:5493), > pid=20194, tid=20202 > # guarantee(false) failed: PPC assembly code requires stop: MemberName > required for invokeVirtual etc. > > V [libjvm.so+0x14a0600] stop_on_request(int, char const*)+0x7c > J 110 java.lang.invoke.MethodHandle.linkToStatic(Ljava/lang/Object;IILjava/lang/invoke/MemberName;)Ljava/lang/Object; > java.base at 10-internal (0 bytes) @ 0x00000fff852d9774 > [0x00000fff852d9700+0x0000000000000074] > J 309 c2 java.lang.invoke.LambdaForm$MH.invoke(Ljava/lang/Object;J)Ljava/lang/Object; > java.base at 10-internal (161 bytes) @ 0x00000fff852fea98 > [0x00000fff852fe900+0x0000000000000198] > J 308 c2 java.lang.invoke.LambdaForm$MH.linkToTargetMethod(JLjava/lang/Object;)Ljava/lang/Object; > java.base at 10-internal (9 bytes) @ 0x00000fff852fe23c > [0x00000fff852fe200+0x000000000000003c] > j LambdaForm.foo(J)Ljava/lang/String;+86 > j LambdaForm.main([Ljava/lang/String;)V+8 > > The fix for the problem is trivial. Just use the final destination > register for 'vmtarget' also as intermediate storage for the > ResolvedMethodName object (thus leaving the register holding the > MemberName reference untouched). > > Thank you and best regards, > Volker From jiangli.zhou at oracle.com Sat Jul 15 04:37:18 2017 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Fri, 14 Jul 2017 21:37:18 -0700 Subject: RFR: compile-time guard some UseSharedSpaces-only coding with the INCLUDE_CDS macro - was :RE: jdk10 : UseSharedSpaces flag and INCLUDE_CDS macro In-Reply-To: <4d2f3964b0924ffa897074759e508653@sap.com> References: <4d2f3964b0924ffa897074759e508653@sap.com> Message-ID: Hi Baesken, Thank you for making the changes. Related to the change in universe_init() (universe.cpp), the 'if (DumpSharedSpaces)? block at line 715 is also CDS only. To be consistent, you might want to include it under #if INCLUDE_CDS as well. --- 695,716 ---- Universe::_loader_addClass_cache = new LatestMethodCache(); Universe::_pd_implies_cache = new LatestMethodCache(); Universe::_throw_illegal_access_error_cache = new LatestMethodCache(); Universe::_do_stack_walk_cache = new LatestMethodCache(); + #if INCLUDE_CDS if (UseSharedSpaces) { // Read the data structures supporting the shared spaces (shared // system dictionary, symbol table, etc.). After that, access to // the file (other than the mapped regions) is no longer needed, and // the file is closed. Closing the file does not affect the // currently mapped regions. MetaspaceShared::initialize_shared_spaces(); StringTable::create_table(); ! } else ! #endif ! { SymbolTable::create_table(); StringTable::create_table(); if (DumpSharedSpaces) { MetaspaceShared::prepare_for_dumping(); In the past we have been trying to avoid adding too many #if in the code for better readability. For the CDS only code in universe.cpp (and a few other files), since the callee functions (e.g. initialize_shared_spaces()) are already defined with two versions (with and without INCLUDE_CDS). The builds without CDS also works fine even #if INCLUDE_CDS are not added. Thanks, Jiangli > On Jul 13, 2017, at 5:18 AM, Baesken, Matthias wrote: > >> Thank you for noticing that. Yes, it would be helpful if you can add the #if INCLUDE_CDS for CDS only code. >> I can help you integrate the changes after they are reviewed. > > Thanks for your feedback ! > > I created a bug : > > https://bugs.openjdk.java.net/browse/JDK-8184323 > > > and a webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8184323/ > > > Best regards, Matthias > > > -----Original Message----- > From: Jiangli Zhou [mailto:jiangli.zhou at oracle.com] > Sent: Montag, 10. Juli 2017 17:54 > To: Baesken, Matthias > Cc: hotspot-dev at openjdk.java.net; build-dev at openjdk.java.net > Subject: Re: jdk10 : UseSharedSpaces flag and INCLUDE_CDS macro > > Hi Matthias, > > Thank you for noticing that. Yes, it would be helpful if you can add the #if INCLUDE_CDS for CDS only code. I can help you integrate the changes after they are reviewed. > > Thanks! > Jiangli > >> On Jul 5, 2017, at 6:36 AM, Baesken, Matthias wrote: >> >> Hello, when looking into CDS related build options, I noticed that most code-parts that are executed only when UseSharedSpaces is set, >> are guarded by the compile-time macro INCLUDE_CDS to support switching off compilation of this coding >> when CDS is disabled at compile time : >> >> >> See hotspot/make/lib/JvmFeatures.gmk : >> >> ifneq ($(call check-jvm-feature, cds), true) >> JVM_CFLAGS_FEATURES += -DINCLUDE_CDS=0 >> >> >> >> However some places miss the compile-time guarding ( #if INCLUDE_CDS ....) for example in >> >> >> share/vm/prims/jvmtiRedefineClasses.cpp >> share/vm/memory/universe.cpp >> >> (also some other places) >> >> >> Should I prepare a change and add the compile-time guard at the places where missing as well ? >> >> Best regards, Matthias > From thomas.stuefe at gmail.com Mon Jul 17 07:00:01 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 17 Jul 2017 09:00:01 +0200 Subject: RFR(xs): 8184339: Thread::current_or_null() should not assert if Posix TLS is not yet initialized Message-ID: Hi all, may I please have reviews + a sponsor for the following fix: bug: https://bugs.openjdk.java.net/browse/JDK-8184339 webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8184339-Thread-current-or-null-shall-not-assert/webrev.00/webrev/ The problem is caused by the fact that Posix TLS cannot be used before it is initialized. It is initialized in os::init(). If we use Posix TLS (eg via Thread::current()) before, we assert. It is used now (since JDK-8183925 ) before os::init() (see callstack in bug description). There are two functions, Thread::current() and Thread::current_or_null(). The latter should not assert if Posix TLS is not yet available but return NULL instead. This is what callers expect: this function is safe to call, but it might not return a valid Thread*. Note that this issue currently only comes up at AIX, because AIX is the only platform using Posix TLS for Thread::current() - all other platforms use Compiler-based TLS (__thread variables). However, we want to keep the Posix TLS code path alive, so this may also affect other platforms. There have been compiler bugs in the past (e.g. gcc + glibc) leading to errors when using compiler-based TLS, so it is good to have a reliable fallback. Thanks, Thomas From mikael.gerdin at oracle.com Mon Jul 17 08:56:44 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Mon, 17 Jul 2017 10:56:44 +0200 Subject: RFR(s): 8179327: gtestLauncher should run tests on a separate thread (optionally) In-Reply-To: References: <438277b7-921c-f4e3-ee12-201ad45e3eef@oracle.com> Message-ID: Hi Thomas, On 2017-07-13 08:18, Thomas St?fe wrote: > Ping.. > > May I please get a second review and a sponsor? I would prefer if you made a const static bool DEFAULT_SPAWN_IN_NEW_THREAD which is initialized with the proper platform value. The current 5-line return statement in get_spawn_new_main_thread_arg looks a bit weird. If you move the wrapping of argc/argv up to runUnitTests and pass the args_t instead you can make run_in_new_thread completely platform specific instead of having #ifdef _WIN32 in the middle, I think this is the preferred style in the code base overall. In the pthread join error path you have "/rsyn" at the end of the line, is that intentional? Thanks /Mikael > > Thanks, Thomas > > On Tue, Jul 4, 2017 at 11:12 AM, Thomas St?fe > wrote: > >> >> >> On Tue, Jul 4, 2017 at 10:22 AM, Robbin Ehn wrote: >> >>> Looks reasonable, thanks! >>> >>> >> Thank you! >> >> >>> /Robbin >>> >>> If you don't mind, please don't wrap lines, links do not work well with >>> line breaks in them :) >>> >>> >> Oops, sorry! >> >> >>> On 07/02/2017 09:23 AM, Thomas St?fe wrote: >>> >>>> Dear all, >>>> >>>> please review the following change: >>>> >>>> Bug: JDK-8179327 >>>> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/ >>>> 8179327-gtestLauncher-should-have-an-option-to-avoid- >>>> primordial-thread-for-tests/webrev.00/webrev/ >>>> >>>> This fix is motivated by the inability of the AIX openjdk vm to run on >>>> the >>>> primordial thread (see https://bugs.openjdk.java.net/browse/JDK-8171505 >>>> ), >>>> which we unfortunately cannot do much about. This prevented us until now >>>> to >>>> run gtests successfully, because the gtestlauncher (in contrast to the >>>> standard java launcher) invokes the VM on the primordial thread. >>>> >>>> To fix this, I gave the gtest launcher the ability to run the tests in an >>>> own thread. Instead of sprinkling the source with "#ifdef _AIX", I added >>>> a >>>> command line option "-new-thread", which allows to start the tests in a >>>> new >>>> thread, and implemented this feature for all platforms (well, posix and >>>> windows). On all platforms save AIX this is by default off, so nothing >>>> changes. On AIX, tests will be default be executed in a new thread. >>>> >>>> Thanks for reviewing, >>>> >>>> Kind Regards, Thomas >>>> >>>> >> From erik.osterlund at oracle.com Mon Jul 17 09:04:05 2017 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 17 Jul 2017 11:04:05 +0200 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: <85E80DD4-FE40-464A-8C76-44F4A2AA3F05@oracle.com> References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <85E80DD4-FE40-464A-8C76-44F4A2AA3F05@oracle.com> Message-ID: <596C7D85.5060607@oracle.com> Hi Kim, Thank you for the review. I have now fixed all the issues you mentioned. New full webrev: http://cr.openjdk.java.net/~eosterlund/8184334/webrev.01/ New incremental webrev: http://cr.openjdk.java.net/~eosterlund/8184334/webrev.00_01/ Thanks, /Erik On 2017-07-14 20:56, Kim Barrett wrote: >> On Jul 14, 2017, at 12:28 PM, Erik ?sterlund wrote: >> >> Hi, >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8184334 >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8184334/webrev.00/ > ------------------------------------------------------------------------------ > Another pre-review comment that seems to have been missed. (Your > reply indicated the suggestion was accepted.) > > src/share/vm/runtime/atomic.hpp > 250 typedef typename IntegerTypes::Signed::type Raw; > intptr_t => U* > > 269 typedef typename IntegerTypes::Signed::type Raw; > 286 typedef typename IntegerTypes::Signed::type Raw; > intptr_t => T* > > ------------------------------------------------------------------------------ > From rkennke at redhat.com Mon Jul 17 10:00:11 2017 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 17 Jul 2017 12:00:11 +0200 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: <596C7D85.5060607@oracle.com> References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <85E80DD4-FE40-464A-8C76-44F4A2AA3F05@oracle.com> <596C7D85.5060607@oracle.com> Message-ID: Hi Erik, I like the stuff. Cannot find anything wrong in the patch (i.e. ok from me as non-reviewer), but have some questions: - Do you plan to fix+remove the now deprecated methods, e.g. store_ptr() soon? - Have you given any thought about C++11 atomics? I don't know if that will ever be feasible, but part of me would find it nice to just use that in the future? Or at least get/stay as close to it API-wise as possible? Roman Am 17.07.2017 um 11:04 schrieb Erik ?sterlund: > Hi Kim, > > Thank you for the review. I have now fixed all the issues you mentioned. > > New full webrev: > http://cr.openjdk.java.net/~eosterlund/8184334/webrev.01/ > > New incremental webrev: > http://cr.openjdk.java.net/~eosterlund/8184334/webrev.00_01/ > > Thanks, > /Erik > > On 2017-07-14 20:56, Kim Barrett wrote: >>> On Jul 14, 2017, at 12:28 PM, Erik ?sterlund >>> wrote: >>> >>> Hi, >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8184334 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8184334/webrev.00/ >> ------------------------------------------------------------------------------ >> >> Another pre-review comment that seems to have been missed. (Your >> reply indicated the suggestion was accepted.) >> >> src/share/vm/runtime/atomic.hpp >> 250 typedef typename IntegerTypes::Signed::type Raw; >> intptr_t => U* >> >> 269 typedef typename IntegerTypes::Signed::type Raw; >> 286 typedef typename IntegerTypes::Signed::type Raw; >> intptr_t => T* >> >> ------------------------------------------------------------------------------ >> >> > From erik.osterlund at oracle.com Mon Jul 17 12:48:18 2017 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 17 Jul 2017 14:48:18 +0200 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <85E80DD4-FE40-464A-8C76-44F4A2AA3F05@oracle.com> <596C7D85.5060607@oracle.com> Message-ID: <596CB212.600@oracle.com> Hi Roman, Thank you for the review. On 2017-07-17 12:00, Roman Kennke wrote: > Hi Erik, > > I like the stuff. Cannot find anything wrong in the patch (i.e. ok from > me as non-reviewer), but have some questions: > - Do you plan to fix+remove the now deprecated methods, e.g. store_ptr() > soon? Yes, unless anyone else picks it up before me, I will do it. For now I have a bunch of more important stuff though that takes precedence to get the GC barrier interface out. The ones that involve pointer arithmetic need to keep in mind during the conversion that add_ptr uses unscaled pointer arithmetic, whereas add() uses scaled pointer arithmetic. > - Have you given any thought about C++11 atomics? I don't know if that > will ever be feasible, but part of me would find it nice to just use > that in the future? Or at least get/stay as close to it API-wise as > possible? Let's see if I can answer this question without getting into a long argument with Andrew Haley... ;) 1) I think since the frontends of Atomic/OrderAccess are used all over hotspot, it would take a lot of good motivation to change everything. 2) I do like the declarative semantics of C++11 semantics. However, I want declarative semantics to cover more than just the memory ordering semantics. That's why the Access API uses what I refer to as "decorators" to specify both memory ordering related semantics as well as other JVM-specific semantics. So you can do e.g. Atomic::oop_cas(...) for a sequentially consistent CAS on a weak reference, rather than having one system per orthogonal concern w.r.t. access semantics. 3) As a backend, I find C++11 atomics not always reliable as there is no contract as to what fencing convention is being used, e.g. whether leading-sync or trailing-sync is used for non-multiple copy atomic architectures, which may or may not imply "visibility" of stores, which may or may not matter when combined with weaker accesses like acquire. The C++11 atomic API was designed for having concurrent accesses in C++ playing well with other concurrent accesses in C++, where all the accesses involved go through the same atomic API, same convention, same rules. So stores could be designed in a certain way, knowing that corresponding loads have been designed in a certain way. That is not the situation in hotspot. Some accesses come from the C++ runtime, others come from hand written assembly or JITed code that makes subtle assumptions about the fencing conventions used in the VM. As we have no contract with what fencing C++11 atomic will generate, and it has changed over time, I feel hesitant about using C++11 atomic as a backend. The lack of contract about fencing convention makes this behaviour undefined. Perhaps that is okay, but I do not know how I feel about it. The possibility of silent and subtle changes in fencing generated by C++11 not playing well with JITed code seems scary to me. But perhaps I am just paranoid. I hope this answers your questions. Thanks, /Erik > Roman > > Am 17.07.2017 um 11:04 schrieb Erik ?sterlund: >> Hi Kim, >> >> Thank you for the review. I have now fixed all the issues you mentioned. >> >> New full webrev: >> http://cr.openjdk.java.net/~eosterlund/8184334/webrev.01/ >> >> New incremental webrev: >> http://cr.openjdk.java.net/~eosterlund/8184334/webrev.00_01/ >> >> Thanks, >> /Erik >> >> On 2017-07-14 20:56, Kim Barrett wrote: >>>> On Jul 14, 2017, at 12:28 PM, Erik ?sterlund >>>> wrote: >>>> >>>> Hi, >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8184334 >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8184334/webrev.00/ >>> ------------------------------------------------------------------------------ >>> >>> Another pre-review comment that seems to have been missed. (Your >>> reply indicated the suggestion was accepted.) >>> >>> src/share/vm/runtime/atomic.hpp >>> 250 typedef typename IntegerTypes::Signed::type Raw; >>> intptr_t => U* >>> >>> 269 typedef typename IntegerTypes::Signed::type Raw; >>> 286 typedef typename IntegerTypes::Signed::type Raw; >>> intptr_t => T* >>> >>> ------------------------------------------------------------------------------ >>> >>> From thomas.stuefe at gmail.com Mon Jul 17 13:44:40 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 17 Jul 2017 15:44:40 +0200 Subject: RFR(XL): 8181917: Refactor UL LogStreams to avoid using resource area In-Reply-To: References: <337330d2-512c-bc77-5a55-c05d33e376e5@oracle.com> Message-ID: Hi Eric, thank you for your review! new Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8181917-refactor-ul-logstream/all.webrev.04/webrev/ delta to last: http://cr.openjdk.java.net/~stuefe/webrevs/8181917-refactor-ul-logstream/delta-03-to-04/webrev/ Changes: - I changed the allocation function to handle the case where we already reached the reasonable_max cap. You are right, in that case reallocating is unnecessary. - Also, added log_info() for that case as you suggested. - Finally, added a fix to the LogStream gtest which somehow slipped thru the last webrev. Changes are now rebased to the current tip. Thanks! ..Thomas On Fri, Jul 14, 2017 at 12:36 PM, Erik Helin wrote: > On 07/13/2017 08:46 AM, Thomas St?fe wrote: > >> Hi Erik! >> >> >> On Mon, Jul 10, 2017 at 11:40 AM, Erik Helin > > wrote: >> >> Hi Thomas, >> >> On 06/30/2017 08:32 PM, Thomas St?fe wrote: >> >> Hi Eric, >> >> thank you for the review! >> >> New >> Version: >> http://cr.openjdk.java.net/~stuefe/webrevs/8181917-refactor- >> ul-logstream/all.webrev.03/webrev/ >> > -ul-logstream/all.webrev.03/webrev/> >> >> >> I think you still have a problem with "runaway" loggers in this >> version: >> >> +// try_ensure_cap tries to enlarge the capacity of the internal >> buffer >> +// to the given atleast value. May fail if either OOM happens or >> atleast >> +// is larger than a reasonable max of 1 M. Caller must not assume >> +// capacity without checking. >> +void LogStream::LineBuffer::try_ensure_cap(size_t atleast) { >> + assert(_cap >= sizeof(_smallbuf), "sanity"); >> + if (_cap < atleast) { >> + const size_t reasonable_max = 1 * M; >> + size_t newcap = align_size_up(atleast + 64, 64); >> + assert(_cap <= reasonable_max, "sanity"); >> + // Cap out at a reasonable max to prevent runaway leaks. >> + if (newcap > reasonable_max) { >> + newcap = reasonable_max; >> + } >> + >> + char* const newbuf = (char*) os::malloc(newcap, mtLogging); >> + if (newbuf == NULL) { // OOM. Leave object unchanged. >> + return; >> + } >> + if (_pos > 0) { // preserve old content >> + memcpy(newbuf, _buf, _pos + 1); // ..including trailing zero >> + } >> + if (_buf != _smallbuf) { >> + os::free(_buf); >> + } >> + _buf = newbuf; >> + _cap = newcap; >> + } >> + assert(_cap >= atleast, "sanity"); >> +} >> >> with the above code, even though _cap > reasonable_max, you will >> always allocate newbuf with os::malloc. For debug builds, we are >> fine, since the JVM will assert on _cap <= reasonable_max, but for >> product builds we are not. Would it be better to first check if _cap >> == reasonable_max? As in the following snippet: >> >> void LogStream::LineBuffer::try_ensure_cap(size_t atleast) { >> if (_cap < atleast) { >> const size_t reasonable_max = 1 * M; >> if (_cap == reasonable_max) { >> // max memory used, "die" in debug builds, log in product >> builds >> log_warning(logging)("Max memory used for message: %s\n", >> _buf); >> ShouldNotReachHere(); >> return; >> } >> >> size_t newcap = align_size_up(atleast + 64, 64); >> if (newcap > reasonable_max) { >> newcap = reasonable_max; >> } >> char* const newbuf = (char*) os::malloc(newcap, mtLogging); >> // rest of logic >> } >> } >> >> Thanks, >> Erik >> >> >> I'm not sure I understand. Are you concerned that _cap would grow beyond >> reasonable_max - because I do not see a way this could happen. Or do you >> want to also treat the *attempt* to grow the _cap beyond reasonable_max as >> a loggable offence? >> > > What I am trying to say is that with your current patch, even if _cap == > reasonable_max, you will still allocate a new 1M buffer, copy all the chars > over, and the free the old one. My proposal simply skips the allocation, > copying and freeing if _cap == reasonable_max. The result will be the same, > just less stress on malloc. > > If the latter, I am not sure about logging. If this is an error which may >> happen at a customer site without popping up first in inhouse-tests (which >> it conceivably is, because logging is very content dependent), I would >> rather hide the error in production code instead of flooding stderr with >> tons of log warnings or crashing with a guarantee, no? >> > > I see what you mean, but perhaps log_info(logging) at least? I want to be > able to discover if this happens in production, for example if a customer > complains about truncated logs. A log_info(logging) will only be printed if > -Xlog:logging=info is specified on the command-line. > > Thanks, > Erik > > ..Thomas >> >> >> From aph at redhat.com Mon Jul 17 14:21:31 2017 From: aph at redhat.com (Andrew Haley) Date: Mon, 17 Jul 2017 15:21:31 +0100 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> Message-ID: On 14/07/17 17:28, Erik ?sterlund wrote: > > As for oracle-external ports, I would appreciate if port maintainers > could check if it seems to make sense. I have done my best to try to > make the appropriate changes where needed. It seems to, but it looks over-engineered. We're relying on the C++ compiler to get rid of all of the casts and calls, but it's always better not to write code we don't want to execute. So, could you add a little commentary explaining what all this SignedType stuff does? I guess it must be necessary or you wouldn't have written it, but I don't know why. Monitor::TryFast -> Atomic::cmpxchg_ptr -> Atomic::cmpxchg -> Atomic::specialized_cmpxchg -> generic_cmpxchg -> __sync_val_compare_and_swap I wonder if it's intended to avoid undefine behaviour by always guaranteeing that we store a signed type, but on the other hand cmpxchg_ptr doesn't much seem to worry about handing a void** as an intptr_t*, so it can't be that, and besides we compile with -fno-strict-aliasing (or it equivalent on other machines). -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From thomas.stuefe at gmail.com Mon Jul 17 14:33:09 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 17 Jul 2017 16:33:09 +0200 Subject: RFR(s): 8179327: gtestLauncher should run tests on a separate thread (optionally) In-Reply-To: References: <438277b7-921c-f4e3-ee12-201ad45e3eef@oracle.com> Message-ID: Hi Mikael, thanks for your review! New Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8179327-gtestLauncher-should-have-an-option-to-avoid-primordial-thread-for-tests/webrev.01/webrev/ See comments inline. On Mon, Jul 17, 2017 at 10:56 AM, Mikael Gerdin wrote: > Hi Thomas, > > On 2017-07-13 08:18, Thomas St?fe wrote: > >> Ping.. >> >> May I please get a second review and a sponsor? >> > > I would prefer if you made a > const static bool DEFAULT_SPAWN_IN_NEW_THREAD which is initialized with > the proper platform value. The current 5-line return statement in > get_spawn_new_main_thread_arg looks a bit weird. > Good suggestion, fixed. > > If you move the wrapping of argc/argv up to runUnitTests and pass the > args_t instead you can make run_in_new_thread completely platform specific > instead of having #ifdef _WIN32 in the middle, I think this is the > preferred style in the code base overall. > > Okay. I did this and shuffled a bit code around, so now we only have one #ifdef _WIN32 section, should be better readable now. > In the pthread join error path you have > "/rsyn" at the end of the line, is that intentional? > No, good catch, that was a stray console command. Kind Regards, Thomas > > Thanks > /Mikael > > > >> Thanks, Thomas >> >> On Tue, Jul 4, 2017 at 11:12 AM, Thomas St?fe >> wrote: >> >> >>> >>> On Tue, Jul 4, 2017 at 10:22 AM, Robbin Ehn >>> wrote: >>> >>> Looks reasonable, thanks! >>>> >>>> >>>> Thank you! >>> >>> >>> /Robbin >>>> >>>> If you don't mind, please don't wrap lines, links do not work well with >>>> line breaks in them :) >>>> >>>> >>>> Oops, sorry! >>> >>> >>> On 07/02/2017 09:23 AM, Thomas St?fe wrote: >>>> >>>> Dear all, >>>>> >>>>> please review the following change: >>>>> >>>>> Bug: JDK-8179327 >>>>> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/ >>>>> 8179327-gtestLauncher-should-have-an-option-to-avoid- >>>>> primordial-thread-for-tests/webrev.00/webrev/ >>>>> >>>>> This fix is motivated by the inability of the AIX openjdk vm to run on >>>>> the >>>>> primordial thread (see https://bugs.openjdk.java.net/ >>>>> browse/JDK-8171505 >>>>> ), >>>>> which we unfortunately cannot do much about. This prevented us until >>>>> now >>>>> to >>>>> run gtests successfully, because the gtestlauncher (in contrast to the >>>>> standard java launcher) invokes the VM on the primordial thread. >>>>> >>>>> To fix this, I gave the gtest launcher the ability to run the tests in >>>>> an >>>>> own thread. Instead of sprinkling the source with "#ifdef _AIX", I >>>>> added >>>>> a >>>>> command line option "-new-thread", which allows to start the tests in a >>>>> new >>>>> thread, and implemented this feature for all platforms (well, posix and >>>>> windows). On all platforms save AIX this is by default off, so nothing >>>>> changes. On AIX, tests will be default be executed in a new thread. >>>>> >>>>> Thanks for reviewing, >>>>> >>>>> Kind Regards, Thomas >>>>> >>>>> >>>>> >>> From mikael.gerdin at oracle.com Mon Jul 17 14:41:19 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Mon, 17 Jul 2017 16:41:19 +0200 Subject: RFR(s): 8179327: gtestLauncher should run tests on a separate thread (optionally) In-Reply-To: References: <438277b7-921c-f4e3-ee12-201ad45e3eef@oracle.com> Message-ID: <9f27acb0-27bc-ce91-fd3d-3cd6cf638c47@oracle.com> Hi Thomas, On 2017-07-17 16:33, Thomas St?fe wrote: > Hi Mikael, > > thanks for your review! > > New Webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/8179327-gtestLauncher-should-have-an-option-to-avoid-primordial-thread-for-tests/webrev.01/webrev/ > > See comments inline. Thanks for fixing this, it looks a lot nicer now. I can sponsor the change for you. I think Robbin is on vacation but I don't think he'll mind a bit of additional cleanups. /Mikael > > On Mon, Jul 17, 2017 at 10:56 AM, Mikael Gerdin > > wrote: > > Hi Thomas, > > On 2017-07-13 08:18, Thomas St?fe wrote: > > Ping.. > > May I please get a second review and a sponsor? > > > I would prefer if you made a > const static bool DEFAULT_SPAWN_IN_NEW_THREAD which is initialized > with the proper platform value. The current 5-line return statement > in get_spawn_new_main_thread_arg looks a bit weird. > > > Good suggestion, fixed. > > > If you move the wrapping of argc/argv up to runUnitTests and pass > the args_t instead you can make run_in_new_thread completely > platform specific instead of having #ifdef _WIN32 in the middle, I > think this is the preferred style in the code base overall. > > > Okay. I did this and shuffled a bit code around, so now we only have one > #ifdef _WIN32 section, should be better readable now. > > In the pthread join error path you have > "/rsyn" at the end of the line, is that intentional? > > > No, good catch, that was a stray console command. > > Kind Regards, Thomas > > > Thanks > /Mikael > > > > Thanks, Thomas > > On Tue, Jul 4, 2017 at 11:12 AM, Thomas St?fe > > > wrote: > > > > On Tue, Jul 4, 2017 at 10:22 AM, Robbin Ehn > > wrote: > > Looks reasonable, thanks! > > > Thank you! > > > /Robbin > > If you don't mind, please don't wrap lines, links do not > work well with > line breaks in them :) > > > Oops, sorry! > > > On 07/02/2017 09:23 AM, Thomas St?fe wrote: > > Dear all, > > please review the following change: > > Bug: JDK-8179327 > > > Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/ > > 8179327-gtestLauncher-should-have-an-option-to-avoid- > primordial-thread-for-tests/webrev.00/webrev/ > > This fix is motivated by the inability of the AIX > openjdk vm to run on > the > primordial thread (see > https://bugs.openjdk.java.net/browse/JDK-8171505 > > ), > which we unfortunately cannot do much about. This > prevented us until now > to > run gtests successfully, because the gtestlauncher > (in contrast to the > standard java launcher) invokes the VM on the > primordial thread. > > To fix this, I gave the gtest launcher the ability > to run the tests in an > own thread. Instead of sprinkling the source with > "#ifdef _AIX", I added > a > command line option "-new-thread", which allows to > start the tests in a > new > thread, and implemented this feature for all > platforms (well, posix and > windows). On all platforms save AIX this is by > default off, so nothing > changes. On AIX, tests will be default be executed > in a new thread. > > Thanks for reviewing, > > Kind Regards, Thomas > > > > From thomas.stuefe at gmail.com Mon Jul 17 14:42:04 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 17 Jul 2017 16:42:04 +0200 Subject: RFR(s): 8179327: gtestLauncher should run tests on a separate thread (optionally) In-Reply-To: <9f27acb0-27bc-ce91-fd3d-3cd6cf638c47@oracle.com> References: <438277b7-921c-f4e3-ee12-201ad45e3eef@oracle.com> <9f27acb0-27bc-ce91-fd3d-3cd6cf638c47@oracle.com> Message-ID: Hi Mikael, On Mon, Jul 17, 2017 at 4:41 PM, Mikael Gerdin wrote: > Hi Thomas, > > On 2017-07-17 16:33, Thomas St?fe wrote: > >> Hi Mikael, >> >> thanks for your review! >> >> New Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8179327-gtestLaun >> cher-should-have-an-option-to-avoid-primordial-thread-for- >> tests/webrev.01/webrev/ >> >> See comments inline. >> > > Thanks for fixing this, it looks a lot nicer now. > I can sponsor the change for you. I think Robbin is on vacation but I > don't think he'll mind a bit of additional cleanups. > > /Mikael > > Thanks! That would be nice! ..Thomas > >> On Mon, Jul 17, 2017 at 10:56 AM, Mikael Gerdin > > wrote: >> >> Hi Thomas, >> >> On 2017-07-13 08:18, Thomas St?fe wrote: >> >> Ping.. >> >> May I please get a second review and a sponsor? >> >> >> I would prefer if you made a >> const static bool DEFAULT_SPAWN_IN_NEW_THREAD which is initialized >> with the proper platform value. The current 5-line return statement >> in get_spawn_new_main_thread_arg looks a bit weird. >> >> >> Good suggestion, fixed. >> >> >> If you move the wrapping of argc/argv up to runUnitTests and pass >> the args_t instead you can make run_in_new_thread completely >> platform specific instead of having #ifdef _WIN32 in the middle, I >> think this is the preferred style in the code base overall. >> >> >> Okay. I did this and shuffled a bit code around, so now we only have one >> #ifdef _WIN32 section, should be better readable now. >> >> In the pthread join error path you have >> "/rsyn" at the end of the line, is that intentional? >> >> >> No, good catch, that was a stray console command. >> >> Kind Regards, Thomas >> >> >> Thanks >> /Mikael >> >> >> >> Thanks, Thomas >> >> On Tue, Jul 4, 2017 at 11:12 AM, Thomas St?fe >> > >> wrote: >> >> >> >> On Tue, Jul 4, 2017 at 10:22 AM, Robbin Ehn >> > wrote: >> >> Looks reasonable, thanks! >> >> >> Thank you! >> >> >> /Robbin >> >> If you don't mind, please don't wrap lines, links do not >> work well with >> line breaks in them :) >> >> >> Oops, sorry! >> >> >> On 07/02/2017 09:23 AM, Thomas St?fe wrote: >> >> Dear all, >> >> please review the following change: >> >> Bug: JDK-8179327 >> > > >> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/ >> >> 8179327-gtestLauncher-should-have-an-option-to-avoid- >> primordial-thread-for-tests/webrev.00/webrev/ >> >> This fix is motivated by the inability of the AIX >> openjdk vm to run on >> the >> primordial thread (see >> https://bugs.openjdk.java.net/browse/JDK-8171505 >> >> ), >> which we unfortunately cannot do much about. This >> prevented us until now >> to >> run gtests successfully, because the gtestlauncher >> (in contrast to the >> standard java launcher) invokes the VM on the >> primordial thread. >> >> To fix this, I gave the gtest launcher the ability >> to run the tests in an >> own thread. Instead of sprinkling the source with >> "#ifdef _AIX", I added >> a >> command line option "-new-thread", which allows to >> start the tests in a >> new >> thread, and implemented this feature for all >> platforms (well, posix and >> windows). On all platforms save AIX this is by >> default off, so nothing >> changes. On AIX, tests will be default be executed >> in a new thread. >> >> Thanks for reviewing, >> >> Kind Regards, Thomas >> >> >> >> >> From erik.osterlund at oracle.com Mon Jul 17 15:00:54 2017 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 17 Jul 2017 17:00:54 +0200 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> Message-ID: <596CD126.6080100@oracle.com> Hi Andrew, I am glad that it seems to work. Thank you for reviewing. As for the over engineered casting mechanism, the IntegerTypes class in metaprogramming/integerTypes.hpp has lots of commentary about what the different casting mechanisms do. Hope that helps. To explain the reasoning, I will go through an example. Let's take atomic::cmpxchg. Each platform has a specialization for different supported cmpxchg implementations in terms of int32_t and int64_t. Since the Atomic::cmpxchg parameter now takes a generic primitive type T, we need to cast T without changing its binary representation, and without losing type safety, into *exactly* those intX_t types. If even slightly different in any way, the specializations will not match. So for example, we need to be able to perform the following conversions: 1) Floating point to integer without changing bit representaiton. It seems like the only way of doing this without violating the C++03 standard is to use memcpy. (no, the union trick in e.g. jfloat_cast is not standards compliant) 2) Pointer to integer 3) Anything with not matching CV qualifiers to matching CV qualifiers 4) Integer of wrong signedness to matching signedness 5) Integer with same signedness but declared differently, e.g. signed char is not the same as char, and will miss that specialization. For example on Solaris, int8_t is char, but jbyte is signed char, and those types are different. Sometimes even intptr_t is neither int32_t or int64_t as I have seen on clang. Due to these issues, the IntegerTypes class was created to solve the casting problems in one single place while remaining type safe. So it solves the problem of canonicalizing primitive types and casting them into exactly the same canonical integer type that can be safely passed into code that specializes on that canonical integer type, without changing the bit pattern or slipping in type safety. By solving this general casting problem problem in IntegerTypes, it turned the casting issues in Atomic into trivial non-problems, and hopefully can be used elsewhere in hotspot. For example the jfloat_cast with friend functions are not standards compliant and can now use IntegerTypes instead, for that mindblowingly seemless casting experience. I hope this explanation makes sense. Thanks, /Erik On 2017-07-17 16:21, Andrew Haley wrote: > On 14/07/17 17:28, Erik ?sterlund wrote: >> As for oracle-external ports, I would appreciate if port maintainers >> could check if it seems to make sense. I have done my best to try to >> make the appropriate changes where needed. > It seems to, but it looks over-engineered. We're relying on the C++ > compiler to get rid of all of the casts and calls, but it's always > better not to write code we don't want to execute. So, could you add > a little commentary explaining what all this SignedType stuff does? I > guess it must be necessary or you wouldn't have written it, but I > don't know why. > > Monitor::TryFast > -> Atomic::cmpxchg_ptr > -> Atomic::cmpxchg > -> Atomic::specialized_cmpxchg > -> generic_cmpxchg > -> __sync_val_compare_and_swap > > I wonder if it's intended to avoid undefine behaviour by always > guaranteeing that we store a signed type, but on the other hand > cmpxchg_ptr doesn't much seem to worry about handing a void** as an > intptr_t*, so it can't be that, and besides we compile with > -fno-strict-aliasing (or it equivalent on other machines). > From shade at redhat.com Mon Jul 17 15:13:38 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 17 Jul 2017 17:13:38 +0200 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> Message-ID: <171bed59-57bd-0fcd-2cb0-4ab982470314@redhat.com> On 07/17/2017 04:21 PM, Andrew Haley wrote: > On 14/07/17 17:28, Erik ?sterlund wrote: >> >> As for oracle-external ports, I would appreciate if port maintainers >> could check if it seems to make sense. I have done my best to try to >> make the appropriate changes where needed. > > It seems to, but it looks over-engineered. We're relying on the C++ > compiler to get rid of all of the casts and calls, but it's always > better not to write code we don't want to execute. I have a related question: how are fastdebug builds doing? This is important, because we run lots of tests on fastdebug builds, and making them slower would be inconvenient. Something like the Unsafe.{get|put|CAS} -Xint performance test would be nice to have. Thanks, -Aleksey From erik.osterlund at oracle.com Mon Jul 17 15:49:20 2017 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 17 Jul 2017 17:49:20 +0200 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: <171bed59-57bd-0fcd-2cb0-4ab982470314@redhat.com> References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <171bed59-57bd-0fcd-2cb0-4ab982470314@redhat.com> Message-ID: <596CDC80.40006@oracle.com> Hi Aleksey, On 2017-07-17 17:13, Aleksey Shipilev wrote: > On 07/17/2017 04:21 PM, Andrew Haley wrote: >> On 14/07/17 17:28, Erik ?sterlund wrote: >>> As for oracle-external ports, I would appreciate if port maintainers >>> could check if it seems to make sense. I have done my best to try to >>> make the appropriate changes where needed. >> It seems to, but it looks over-engineered. We're relying on the C++ >> compiler to get rid of all of the casts and calls, but it's always >> better not to write code we don't want to execute. > I have a related question: how are fastdebug builds doing? This is important, > because we run lots of tests on fastdebug builds, and making them slower would > be inconvenient. > > Something like the Unsafe.{get|put|CAS} -Xint performance test would be nice to > have. Sorry, but I do not understand what you mean here. 1) The code only boils down to performing casts that do not change any bit pattern, and is by contract forced to be expanded and inlined at compile time, so I would be very surprised about any regressions. 2) This code executes when we are about to perform atomic operations, so any accidental code generated for the casting (which seems odd in the first place), I would be very surprised to see. 3) Even if there was some barely noticeable regression in fastdebug atomics for whatever reason, I do not know if I think it makes sense to pollute the code with ugly hacks to micro optimize fastdebug execution time. I hope you agree. Thanks, /Erik > Thanks, > -Aleksey > From volker.simonis at gmail.com Mon Jul 17 15:55:53 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Mon, 17 Jul 2017 17:55:53 +0200 Subject: JEP [DRAFT]: Container aware Java In-Reply-To: References: Message-ID: On Fri, Jul 14, 2017 at 6:54 PM, Bob Vandette wrote: > >> On Jul 14, 2017, at 12:15 PM, Volker Simonis wrote: >> >> Hi Bob, >> >> thanks for starting this JEP. I think it is long overdue and will be >> very helpful for many people. >> > Thanks for taking a look at the JEP. > >> That said, I don't see any mentioning of the targeted Java release. As >> JEPs usually go into the development version first, I suppose the work >> for this JEP will be done in JDK10. However, taking into account the >> current release cycles, the usefulness of this JEP for the Java >> community will significantly depend on the its availability in Java 9 >> (at least) and Java 8 (hopefully?). Any thoughts on this? > > I am certainly going to do my best to try to get this feature into JDK10 and > I understand your concern about supporting Java 8 and 9. The decision to > backport this feature will have to be done based on the usual criteria (complexity, > usefulness, resource availability and compatibility). It may be possible to > add some subset of this functionality in a backport but we?ll have to wait until > the design is further along. > >> >> You also mention that this JEP is only about Docker support on >> Linux/x86_64. I'd argue that we should at least address at least all >> Docker supported Linux platforms. I'm pretty sure our colleagues from >> IBM would like to support this on Linux/ppc64 and Linux/s390 as well. > > Right now I don?t expect to have to write CPU specific logic. > Therefore supporting ppc64, s390 or even aarch64 should boil down to testing. > >> >> Moreover, the suggested API and the new JVM_xxxxxx functions should be >> at least designed in such a way that other virtualisation environment >> such as VMWare/VirtualBox/Xen/KVM can take advantage of it. I'm also >> not sure if a completely new API solely for Docker containers is >> making sense and is worth the effort? I understand that containers and >> virtualisation are two different things, but they both impose similar >> restrictions to the VM (i.e. virtualisation may present a huge number >> of CPUs to the OS/VM which in reality are mapped to a single, physical >> CPU) so maybe we should design the new API in such a way that it can >> be used to tackle both problems. > > I will try to keep that in mind. I will have to look into the feasibility of accessing > host configuration from within a container or virtual machine. For containers, > it seems to me that would break the intent of isolation. Did you wanted to say "for virtualisation, it seems to me that would break the intent of isolation." here? So you're right, that you can not easily query the host/guest information in a virtualised environment (like for examples VmWare) because that would break isolation. But as far as I know, every virtualisation solution has it's own library/API which can be used to query the host (if there is any) and get information about the host system and the resources allocated for the guest. We've done some basic virtualisation detection for common systems like VmWare or Xen (cc'ed Matthias who may know more details. > I know that host information > does leak into containers through the proc file system but I suspect this was done > for simplicity and compatibility reasons. > >> >> Finally, you mention in section D. that all the information should be >> logged the to error crash logs. That's good but maybe you could add >> that all the detection and all the findings should also be made >> available through the new Unified JVM Logging facility (JEP 158)? > > That?s a good idea. I will add that to the JEP. > > Thanks, > Bob. > >> >> Regards, >> Volker >> >> >> On Fri, Jul 14, 2017 at 4:22 PM, Bob Vandette wrote: >>> I?d like to propose the following JEP that will enhance the Java runtime to be more container aware. >>> This will allow the Java runtime to performance better ergonomics and provide better >>> execution time statistics when running in a container. >>> >>> >>> JEP Issue: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8182070 >>> >>> Here?s a Text dump of the JEP contents for your convenience: >>> >>> >>> Summary >>> ------- >>> >>> Container aware Java runtime >>> >>> Goals >>> ----- >>> >>> Enhance the JVM and Core libraries to detect running in a container and adapt to the system resources available to it. This JEP will only support Docker on Linux-x64 although the design should be flexible enough to allow support for other platforms and container technologies. The initial focus will be on Linux low level container technology such as cgroups so that we will be able to easily support other container technologies running on Linux in addition to Docker. >>> >>> Non-Goals >>> --------- >>> >>> It is not a goal of this JEP to support any platform other than Docker container technology running on Linux x64. >>> >>> Success Metrics >>> --------------- >>> >>> Success will be measured by the improved efficiency of running multiple Java containers on a host system with out of the box options. >>> >>> Motivation >>> ---------- >>> >>> Container technology is becoming more and more prevalent in Cloud based applications. This technology provides process isolation and allows the platform vendor to specify limits and alter the behavior of a process running inside a container that the Java runtime is not aware of. This causes the Java runtime to potentially attempt to use more system resources than are available to it causing performance degradation or even termination. >>> >>> >>> Description >>> ----------- >>> >>> This enhancement will be made up of the following work items: >>> >>> A. Detecting if Java is running in a container. >>> >>> The Java runtime, as well as any tests that we might write for this feature, will need to be able to detect that the current Java process is running in a container. I propose that we add a new JVM_ native function that returns a boolean true value if we are running inside a container. >>> >>> JVM_InContainer will return true if the currently running process is running in a container, otherwise false will be returned. >>> >>> B. Exposing container resource limits and configuration. >>> >>> There are several configuration options and limits that can be imposed upon a running container. Not all of these >>> are important to a running Java process. We clearly want to be able to detect how many CPUs have been allocated to our process along with the maximum amount of memory that we be allocated but there are other options that we might want to base runtime decisions on. >>> >>> In addition, since Container typically impose limits on system resources, they also provide the ability to easily access the amount of consumption of these resources. I intent on providing this information in addition to the configuration data. >>> >>> I propose adding a new jdk.internal.Platform class that will allow access to this information. Since some of this information is needed during the startup of the VM, I propose that much of the implementation of the methods in the Platform class be done in the VM and exposed as JVM_xxxxxx functions. In hotspot, the JVM_xxxxxx function will be implemented via the os.hpp interface. >>> >>> Here are the categories of configuration and consumption statistics that will be made available (The exact API is TBD): >>> >>> isContainerized >>> Memory Limit >>> Total Memory Limit >>> Soft Memory Limit >>> Max Memory Usage >>> Current Memory Usage >>> Maximum Kernel Memory >>> CPU Shares >>> CPU Period >>> CPU Quote >>> Number of CPUs >>> CPU Sets >>> CPU Set Memory Nodes >>> CPU Usage >>> CPU Usage Per CPU >>> Block I/O Weight >>> Block I/O Device Weight >>> Device I/O Read Rate >>> Device I/O Write Rate >>> OOM Kill Enabled >>> OOM Score Adjustment >>> Memory Swappiness >>> Shared Memory Size >>> >>> TODO: >>> 1. Need to specify the exact arguments and return format for these accessor functions. >>> >>> C. Adjusting Java runtime configuration based on limits. >>> >>> Java startup normally queries the operating system in order to setup runtime defaults for things such as the number of GC threads and default memory limits. When running in a container, the operating system functions used provide information about the host and does not include the containers configuration and limits. The VM and core libraries will be modified as part of this JEP to first determine if the current running process is running in a container. It will then cause the runtime to use the container values rather than the general operating system functions for configuring and managing the Java process. There have been a few attempts to correct some of these issue in the VM but they are not complete. The CPU detection in the VM currently only handles a container that limits cpu usage via CPU sets. If the Docker --cpu or --cpu-period along with --cpu-quota options are specified, it currently has no effect on the VMs configuration. >>> >>> The experimental memory detection that has been implemented only impacts the Heap selection and does not apply to the os::physical_memory or os::available_memory low level functions. This leaves other parts of the VM and >>> core libraries to believe there is more memory available than there actually is. >>> >>> The Numa support available in the VM is also not correct when running in a container. The number of available memory nodes and enabled nodes as reported by the libnuma library does not take into account the impact of the Docker --cpuset-mems option which restricts which memory nodes the container can use. Inside the container, the file /proc/{pid}/self does report the correct Cpus_allowed and Mems_Allowed but libnuma doesn't respect this. This has been verified via the numactl utility. >>> >>> To correct these shortcomings and make this support more robust, here's a list of the current cgroup subsystems that we be examined in order to update the internal VM and core library configuration. >>> >>> **Number of CPUs** >>> >>> Use a combination of number_of_cpus() and cpu_sets() in order to determine how many processors are available to the process and adjust the JVMs os::active_processor_count appropriately. The number_of_cpus() will be calculated based on the cpu_quota() and cpu_period() using this formula: number_of_cpus() = cpu_quota() / cpu_period(). Since it's not currently possible to understand the relative weight of the running container against all other containers, altering the cpu_shares of a running container will have no affect on altering Java's configuration. >>> >>> **Total available memory** >>> >>> Use the memory_limit() value from the cgroup file system to initialize the os::physical_memory() value in the VM. This value will propagate to all other parts of the Java runtime. >>> >>> We might also consider examining the soft_memory_limit and total_memory_limit in addition to the memory_limit during the ergonomics startup processing in order to fine tuning some of the other VM settings. >>> >>> **CPU Memory Nodes** >>> >>> Use cpu_set_memory_nodes() to configure the os::numa support. >>> >>> **Memory usage** >>> >>> Use memory_usage_in_bytes() for providing os::available_memory() by subtracting the usage from the total available memory allocated to the container. >>> >>> D. Adding container configuration to error crash logs. >>> >>> As as troubleshooting aid, we will dump any available container statistics to the hotspot error log. >>> >>> E. Adding a startup flag to enable/disable this support. >>> >>> Add a -XX:+UseContainerSupport VM option that will be used to enable this support. The default will be off until this feature is proven. >>> >>> Alternatives >>> ------------ >>> >>> There are a few existing RFE's filed that could be used to enhance the current experimental implementation rather than taking the JEP route. >>> >>> Testing >>> ------- >>> >>> Docker/container specific tests should be added in order to validate the functionality being provided with this JEP. >>> >>> Risks and Assumptions >>> --------------------- >>> >>> Docker is currently based on cgroups v1. Cgroups v2 is also available but is incomplete and not yet supported by Docker. It's possible that v2 could replace v1 in an incompatible way rendering this work unusable until it is upgraded. >>> >>> Dependencies >>> ----------- >>> >>> None at this time. >>> > From shade at redhat.com Mon Jul 17 15:56:07 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 17 Jul 2017 17:56:07 +0200 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: <596CDC80.40006@oracle.com> References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <171bed59-57bd-0fcd-2cb0-4ab982470314@redhat.com> <596CDC80.40006@oracle.com> Message-ID: On 07/17/2017 05:49 PM, Erik ?sterlund wrote: >> Something like the Unsafe.{get|put|CAS} -Xint performance test would be nice to >> have. > > Sorry, but I do not understand what you mean here. > > 1) The code only boils down to performing casts that do not change any bit > pattern, and is by contract forced to be expanded and inlined at compile time, > so I would be very surprised about any regressions. > 2) This code executes when we are about to perform atomic operations, so any > accidental code generated for the casting (which seems odd in the first place), > I would be very surprised to see. > 3) Even if there was some barely noticeable regression in fastdebug atomics for > whatever reason, I do not know if I think it makes sense to pollute the code > with ugly hacks to micro optimize fastdebug execution time. I agree that fastdebug should not be our primary optimization target. I was wondering if all the template magic actually works well in fastdebug builds. Or product builds, for that matter: because while templates are indeed done during compile-time, say, the inliner that has to deal with new code shapes can have opinions. There are some VarHandles perf tests I have lying around, I can check this myself today. -Aleksey From volker.simonis at gmail.com Mon Jul 17 15:58:14 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Mon, 17 Jul 2017 17:58:14 +0200 Subject: RFR(XS) [10]: 8184672: [ppc64] The fix for '8174749: Use hash table/oops for MemberName table' is broken on ppc64 In-Reply-To: <29139297-19fe-20ca-df67-e35506aebe90@oracle.com> References: <29139297-19fe-20ca-df67-e35506aebe90@oracle.com> Message-ID: No problem. Thanks for the quick review and also for applying your changes to our platforms. We really appreciate that! Regards, Volker On Fri, Jul 14, 2017 at 10:05 PM, wrote: > > Hi Volker, > This looks good. Sorry for the breakage- I thought my change was trivial > and couldn't possibly break :0 > Thanks! > Coleen > > > On 7/14/17 11:41 AM, Volker Simonis wrote: >> >> Hi, >> >> can somebody please review the following small, ppc64-only change: >> >> http://cr.openjdk.java.net/~simonis/webrevs/2017/8184672 >> https://bugs.openjdk.java.net/browse/JDK-8184672 >> >> The change for "JDK-8174749: Use hash table/oops for MemberName table" >> introduced a new indirection for the 'vmtarget' field in >> java.lang.invoke.MemberName. 'vmtarget' was previously directly stored >> in MemberName and is now a field of the new class ResolvedMethodName, >> an object of which is now stored in MemberName. >> >> The change contained some architecture-specific code which updated the >> HotSpot to be aware of the new indirection (i.e. instead of loading >> 'vmtarget' directly from a MemberName object the code now has to first >> load the ResolvedMethodName object from MemeberName and subsequently >> the 'vmtarget' field from the ResolvedMethodName object. >> >> This code is trivial, but on ppc64, the new version has incorrectly >> 'misused' the register which contained the MemberName, as storage for >> the intermediate ResolvedMethodName object. If the MemberName object >> was stored in an argument register, that register was overwritten in >> MethodHandles::generate_method_handle_dispatch (when generating the >> intrinsics for MethodHandle.linkToStatic or >> MethodHandle.linkToSpecial) and could lead to the following error down >> the call chain: >> >> # Internal Error >> (jdk10-hs/hotspot/src/cpu/ppc/vm/macroAssembler_ppc.cpp:5493), >> pid=20194, tid=20202 >> # guarantee(false) failed: PPC assembly code requires stop: MemberName >> required for invokeVirtual etc. >> >> V [libjvm.so+0x14a0600] stop_on_request(int, char const*)+0x7c >> J 110 >> java.lang.invoke.MethodHandle.linkToStatic(Ljava/lang/Object;IILjava/lang/invoke/MemberName;)Ljava/lang/Object; >> java.base at 10-internal (0 bytes) @ 0x00000fff852d9774 >> [0x00000fff852d9700+0x0000000000000074] >> J 309 c2 >> java.lang.invoke.LambdaForm$MH.invoke(Ljava/lang/Object;J)Ljava/lang/Object; >> java.base at 10-internal (161 bytes) @ 0x00000fff852fea98 >> [0x00000fff852fe900+0x0000000000000198] >> J 308 c2 >> java.lang.invoke.LambdaForm$MH.linkToTargetMethod(JLjava/lang/Object;)Ljava/lang/Object; >> java.base at 10-internal (9 bytes) @ 0x00000fff852fe23c >> [0x00000fff852fe200+0x000000000000003c] >> j LambdaForm.foo(J)Ljava/lang/String;+86 >> j LambdaForm.main([Ljava/lang/String;)V+8 >> >> The fix for the problem is trivial. Just use the final destination >> register for 'vmtarget' also as intermediate storage for the >> ResolvedMethodName object (thus leaving the register holding the >> MemberName reference untouched). >> >> Thank you and best regards, >> Volker > > From aph at redhat.com Mon Jul 17 15:59:38 2017 From: aph at redhat.com (Andrew Haley) Date: Mon, 17 Jul 2017 16:59:38 +0100 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: <596CD126.6080100@oracle.com> References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <596CD126.6080100@oracle.com> Message-ID: Hi, On 17/07/17 16:00, Erik ?sterlund wrote: > I am glad that it seems to work. Thank you for reviewing. > > As for the over engineered casting mechanism, the IntegerTypes class in > metaprogramming/integerTypes.hpp has lots of commentary about what the > different casting mechanisms do. Hope that helps. It doesn't, because most HotSpot programmers know what this stuff does, but not why you're doing it. > To explain the reasoning, I will go through an example. Let's take > atomic::cmpxchg. > Each platform has a specialization for different supported cmpxchg > implementations in terms of int32_t and int64_t. OK, I get that. Some platforms presumably requires such implementations, but would it not make more sense to restrict the complexity to just those platforms? None of the GCC targets need it. > So for example, we need to be able to perform the following conversions: > 1) Floating point to integer without changing bit representaiton. It > seems like the only way of doing this without violating the C++03 > standard is to use memcpy. (no, the union trick in e.g. jfloat_cast is > not standards compliant) I'm not entirely convinced that memcpy does it either for "truly portable" C++, but perhaps it's as good as we're going to get. > 2) Pointer to integer > 3) Anything with not matching CV qualifiers to matching CV qualifiers > 4) Integer of wrong signedness to matching signedness > 5) Integer with same signedness but declared differently, e.g. signed > char is not the same as char, and will miss that specialization. For > example on Solaris, int8_t is char, but jbyte is signed char, and those > types are different. Sometimes even intptr_t is neither int32_t or > int64_t as I have seen on clang. If you have something like long n; long *p = &n; The you can access the stored value in p with any type compatible with long*. However, this: Atomic::cmpxchg_ptr(0, &p, &n); casts &n to (signed) intptr_t: this is guaranteed to work. Unfortunately, it then calls generic_cmpxchg(0, &p, &n). This accesses the stored pointer in p as a long, which is undefined behaviour. It works on GCC if we compile everything with -fno-strict-aliasing (at least if intptr_t has the same representation in memory as every pointer type, which every machine we care about does) but in that case there's little point casting pointers back and forth to integer types. > Due to these issues, the IntegerTypes class was created to solve the > casting problems in one single place while remaining type safe. So it > solves the problem of canonicalizing primitive types and casting them > into exactly the same canonical integer type that can be safely passed > into code that specializes on that canonical integer type, without > changing the bit pattern or slipping in type safety. It is not type safe. ------------------------------------------------------------------------ 3.10, Lvalues and rvalues If a program attempts to access the stored value of an object through a glvalue of other than one of the following types the behavior is undefined: ? the dynamic type of the object, ? a cv-qualified version of the dynamic type of the object, ? a type similar (as defined in 4.4) to the dynamic type of the object, ? a type that is the signed or unsigned type corresponding to the dynamic type of the object, ? a type that is the signed or unsigned type corresponding to a cv-qualified version of the dynamic type of the object, ? an aggregate or union type that includes one of the aforementioned types among its elements or non- static data members (including, recursively, an element or non-static data member of a subaggregate or contained union), ? a type that is a (possibly cv-qualified) base class type of the dynamic type of the object, ? a char or unsigned char type. ------------------------------------------------------------------------ You only have permission to convert pointers to intptr_t and back: you do not have permission to access the stored value of a pointer an an intptr_t. > By solving this general casting problem problem in IntegerTypes, it > turned the casting issues in Atomic into trivial non-problems, and > hopefully can be used elsewhere in hotspot. For example the jfloat_cast > with friend functions are not standards compliant and can now use > IntegerTypes instead, for that mindblowingly seemless casting experience. > > I hope this explanation makes sense. I don't believe that it entirely does, no. But it is the sort of commentary which ought to be in the source code. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From shade at redhat.com Mon Jul 17 16:00:19 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 17 Jul 2017 18:00:19 +0200 Subject: RFR (S) 8184753: Assert against MinObjectAlignmentMask to avoid integer division Message-ID: <2e2413f4-3eb3-f124-fb77-48e52bd40d41@redhat.com> https://bugs.openjdk.java.net/browse/JDK-8184753 fastdebug builds performance is important to minimize testing time. In GC tests, we have a big hotspot at this line in oop::size_given_klass(): assert(s % MinObjAlignment == 0, "Oop size is not properly aligned: %d", s); The hotspot is because of the integer division. Alas, the divisor is not known at compile time, because it can be set from the JVM command line, so compiler cannot optimize it. There is an easy fix, use the MinObjectAlignmentMask to assert: http://cr.openjdk.java.net/~shade/8184753/webrev.01 This also fixes all uses of MinObjAlignment in similar expressions. Testing: hotspot_tier1 (Linux x86_64/fastdebug) Thanks, -Aleksey From shade at redhat.com Mon Jul 17 16:49:19 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 17 Jul 2017 18:49:19 +0200 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <171bed59-57bd-0fcd-2cb0-4ab982470314@redhat.com> <596CDC80.40006@oracle.com> Message-ID: <7875630a-2ca2-d345-73d0-5653c125f259@redhat.com> On 07/17/2017 05:56 PM, Aleksey Shipilev wrote: > On 07/17/2017 05:49 PM, Erik ?sterlund wrote: >>> Something like the Unsafe.{get|put|CAS} -Xint performance test would be nice to >>> have. >> >> Sorry, but I do not understand what you mean here. >> >> 1) The code only boils down to performing casts that do not change any bit >> pattern, and is by contract forced to be expanded and inlined at compile time, >> so I would be very surprised about any regressions. >> 2) This code executes when we are about to perform atomic operations, so any >> accidental code generated for the casting (which seems odd in the first place), >> I would be very surprised to see. >> 3) Even if there was some barely noticeable regression in fastdebug atomics for >> whatever reason, I do not know if I think it makes sense to pollute the code >> with ugly hacks to micro optimize fastdebug execution time. > > I agree that fastdebug should not be our primary optimization target. I was > wondering if all the template magic actually works well in fastdebug builds. > > Or product builds, for that matter: because while templates are indeed done > during compile-time, say, the inliner that has to deal with new code shapes can > have opinions. > > There are some VarHandles perf tests I have lying around, I can check this > myself today. Checked, times are the same for baseline/patched builds with either fastdebug or release bits. Ditto the instruction counts, number of memory accesses, branches, etc. Seems good! Thanks, -Aleksey From shade at redhat.com Mon Jul 17 17:34:54 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 17 Jul 2017 19:34:54 +0200 Subject: RFR (XS) 8184762: ZapStackSegments should use optimized memset Message-ID: <848320ea-4eed-09dc-95b9-8edb5da2fc56@redhat.com> https://bugs.openjdk.java.net/browse/JDK-8184762 fastdebug builds performance is important to minimize testing time. ZapStackSegments is trueInDebug, and used to zap vm/utilities/stack.* segments as they get recycled. However, that zapping uses the handrolled memset, which shows up in profiles with individual stores. It seems better to use Copy::fill_to_bytes for this, which will call into platform-specific memset. Additionally, this would be a good time to introduce its zapped pattern to globalDefinitions. The choice of 0xCA is arbitrary, and open for discussion. Webrev: http://cr.openjdk.java.net/~shade/8184762/webrev.01/ Testing: hotspot_tier1 (Linux x86_64 / fastdebug) Thanks, -Aleksey From zgu at redhat.com Mon Jul 17 18:16:39 2017 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 17 Jul 2017 14:16:39 -0400 Subject: RFR(XS) [8u backport] 8181055: PPC64: "mbind: Invalid argument" still seen after 8175813 In-Reply-To: References: <5361e308-00e1-3041-c728-b6ebae7586ae@redhat.com> <25ab002d-0728-4aaa-eeb7-d3b9a3ea8254@redhat.com> <62524c62-ded1-6fab-e00c-8666fbff1645@oracle.com> <2239bacf-1760-1f73-20a1-ce792a191741@redhat.com> <593AAB00.4060801@linux.vnet.ibm.com> Message-ID: Hi, This 8u backport of JDK-8181005 has been reviewed by Aleksey [1] and approved by Rob [2]. However, it is not clear to me that the only reviewer has to be 'R'eviewer. If so, can someone with 'R' to review the backport? It is almost the same as JDK9, with very minor conflicts. I will also need sponsor this backport. Bug: https://bugs.openjdk.java.net/browse/JDK-8181055 Webrev: http://cr.openjdk.java.net/~zgu/8181055/8u/webrev.00/ [1] http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-June/027104.html [2] http://mail.openjdk.java.net/pipermail/jdk8u-dev/2017-June/006723.html Thanks, -Zhengyu On 06/09/2017 11:39 AM, Zhengyu Gu wrote: > > > On 06/09/2017 10:04 AM, Gustavo Romero wrote: >> Hi Zhengyu, Dan >> >> On 07-06-2017 17:37, Zhengyu Gu wrote: >>> Thanks for the clarification, Dan. >>> >>> -Zhengyu >>> >>> >>> On 06/07/2017 04:14 PM, Daniel D. Daugherty wrote: >>>> hotspot-dev at ... is the right place for the RFR (which this is). >>>> >>>> jdk8u-dev at ... is the right place for the RFA (Request For Approval) >>>> after the RFR is approved. >>>> >>>> Dan >> >> Does it mean that besides Aleksey's review it's still missing one >> additional >> review in order to proceed with the request for approval in the >> jdk8u-dev ML >> or this bug? Is my understanding correct? >> > Hi Gustavo, > > If backport has the same review process, then it needs a "R"eviwer. > > Thanks, > > -Zhengyu > > >> Thanks! >> >> Regards, >> Gustavo >> >> >>>> >>>> On 6/7/17 12:16 PM, Zhengyu Gu wrote: >>>>> Hi Aleksey, >>>>> >>>>> Thanks for the review. >>>>> >>>>> On 06/07/2017 02:11 PM, Aleksey Shipilev wrote: >>>>>> On 06/07/2017 08:07 PM, Zhengyu Gu wrote: >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8181055 >>>>>>> Webrev: http://cr.openjdk.java.net/~zgu/8181055/8u/webrev.00/ >>>>>> >>>>>> Looks almost the same as 9. Looks good to me. >>>>>> >>>>>> Shouldn't you do this at jdk8-dev? >>>>>> >>>>> There is not jdk8-dev and jdk8 is read-only. I think jdk8u-dev is >>>>> right one. >>>>> >>>>> Thanks, >>>>> >>>>> -Zhengyu >>>>> >>>>> >>>>>> -Aleksey >>>>>> >>>>> >>>> >>> >> From coleen.phillimore at oracle.com Mon Jul 17 18:37:13 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 17 Jul 2017 14:37:13 -0400 Subject: RFR (L) 7133093: Improve system dictionary performance In-Reply-To: References: <89f0b98c-3cbd-7b87-d76b-5e89a5a676fb@oracle.com> Message-ID: <9243da7c-a8e5-4024-c12e-cba9d4f424c8@oracle.com> Hi, I have made changes for Ioi's comments below. The largest change since .01 is that PredictedLoadedClassCount is used to size dictionaries for class loader data, and should be used to specify the average number of classes loaded by each class loader. Since this is an option available via UnlockExperimentalVMOptions, I'm not sure what the process is for changing the meaning of this option. We will work with our customers who have used this option for specifying the total number of classes loaded. We are also working on the change to allow automatic resizing of the dictionaries during safepoints. See: open webrev at http://cr.openjdk.java.net/~coleenp/7133093.02/webrev Tested with the full nightly test set (jdk, nsk, hotspot jtreg tests) on linux x64. Thanks, Coleen On 6/30/17 4:45 PM, coleen.phillimore at oracle.com wrote: > > Ioi, Thank you for looking at this change. > > On 6/30/17 2:39 PM, Ioi Lam wrote: >> Hi Coleen, >> >> Maybe the bug should be renamed to "Use one Dictionary per class >> loader instance"? That way it's more obvious what it is when you look >> at the repo history. > > I can do that. I made it One Dictionary per ClassLoaderData >> >> 1. >> >> Why is assert_locked_or_safepoint(SystemDictionary_lock) necessary in >> SystemDictionary::find_class (line 1826), but not necessary >> SystemDictionary::find (line 951)? Since you removed >> NoSafepointVerifier nosafepoint in the latter, maybe this means it's >> safe to remove the assert_locked_or_safepoint in the former? > > The call to SystemDictionary::find() is the (I believe) usual lock > free lookup and the SystemDictionary::find_class() is used to verify > that a class we're about to add or want to add hasn't been added by > another thread. Or certain cases where we already have a lock to do > something else, like add loader constraints. I took out the > NoSafepointVerifier because it assumes that system dictionary entries > would be moved by GC, which they won't. The old hash function could > safepoint when getting the hash for the class_loader object. > >> >> 2. >> >> 455 static ClassLoaderData* _current_loader_data = NULL; >> 456 static Klass* _current_class_entry = NULL; >> 457 >> 458 InstanceKlass* ClassLoaderDataGraph::try_get_next_class() { >> >> How about moving the static fields into an iterator object. That way >> you don't need to keep track of the globals >> >> ClassLoaderDataGraphIterator { >> ClassLoaderData* _current_loader_data >> Klass* _current_class_entry; >> >> InstanceKlass* try_get_next_class() { ....} >> }; > > Ok, there's a different iterator that iterates over all of the classes > for GC. I will adapt that for this use. That would be better. > >> >> 3. Double check locking in ClassLoaderData::dictionary() -- someone >> else should look at this :-) > > I copied code that David Holmes added for modules. > >> >> 4. We may need a better strategy for deciding the size of each >> dictionary. >> >> 565 const int _primelist[10] = {1, 107, 1009}; >> 571 Dictionary* ClassLoaderData::dictionary() { >> 579 if ((dictionary = _dictionary) == NULL) { >> 580 int size; >> 581 if (this == the_null_class_loader_data() || >> is_system_class_loader_data()) { >> 582 size = _primelist[2]; >> 583 } else if >> (class_loader()->is_a(SystemDictionary::reflect_DelegatingClassLoader_klass())) >> { >> 584 size = _primelist[0]; >> 585 } else { >> 586 size = _primelist[1]; >> 587 } >> 588 dictionary = new Dictionary(this, size); >> >> I'll do some investigation on this issue and get back to you. >> > > How about if someone uses PredictedLoadedClassCount, then we use that > to size all but the reflection and boot class loader? Then if there's > an application that has a class loader with a huge amount of classes > loaded in it, that would help this? It might cost some footprint but > an oversized table would simply be a bigger array of pointers, so it > might not be that bad to oversize. > > I think the long term solution is being able to resize these entries > and make the application provide arguments. Please let me know what > you find in your investigation and if that would work. >> The rest of the changes look good to me. > > Thank you! > Coleen >> >> Thanks >> - Ioi >> >> >> On 6/23/17 4:42 PM, coleen.phillimore at oracle.com wrote: >>> Summary: Implement one dictionary per ClassLoaderData for faster >>> lookup and removal during class unloading >>> >>> See RFE for more details. >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/7133093.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-7133093 >>> >>> Tested with full "nightly" run in rbt, plus locally class loading >>> and unloading tests: >>> >>> jtreg hotspot/test/runtime/ClassUnload >>> >>> jtreg hotspot/test/runtime/modules >>> >>> jtreg hotspot/test/gc/class_unloading >>> >>> make test-hotspot-closed-tonga FILTER=quick TEST_JOBS=4 >>> TEST=vm.parallel_class_loading >>> >>> csh ~/testing/run_jck9 (vm/lang/java_lang) >>> >>> runThese -jck - uses class loader isolation to run each jck test and >>> unloads tests when done (at -gc:5 intervals) >>> >>> >>> Thanks, >>> Coleen >>> >>> >> > From rkennke at redhat.com Mon Jul 17 19:18:13 2017 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 17 Jul 2017 21:18:13 +0200 Subject: RFR (XS) 8184762: ZapStackSegments should use optimized memset In-Reply-To: <848320ea-4eed-09dc-95b9-8edb5da2fc56@redhat.com> References: <848320ea-4eed-09dc-95b9-8edb5da2fc56@redhat.com> Message-ID: <4a27accb-3915-c3fd-134f-6a6842a9cda1@redhat.com> Hi Aleksey, looks good to me. Normally I wouldn't bother optimizing debug code, but this actually makes it more readable (but slightly less funny ;-) ). I'd probably declare the constant char or jubyte or such (fill_to_bytes() takes a jubyte, can't say I like the j*types much). (breaks 'consistency' with other similar constants though... your call) Update copyright header. Roman (not an official reviewer) Am 17.07.2017 um 19:34 schrieb Aleksey Shipilev: > https://bugs.openjdk.java.net/browse/JDK-8184762 > > fastdebug builds performance is important to minimize testing time. > ZapStackSegments is trueInDebug, and used to zap vm/utilities/stack.* segments > as they get recycled. > > However, that zapping uses the handrolled memset, which shows up in profiles > with individual stores. It seems better to use Copy::fill_to_bytes for this, > which will call into platform-specific memset. > > Additionally, this would be a good time to introduce its zapped pattern to > globalDefinitions. The choice of 0xCA is arbitrary, and open for discussion. > > Webrev: > http://cr.openjdk.java.net/~shade/8184762/webrev.01/ > > Testing: hotspot_tier1 (Linux x86_64 / fastdebug) > > Thanks, > -Aleksey > From shade at redhat.com Mon Jul 17 19:40:02 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 17 Jul 2017 21:40:02 +0200 Subject: RFR (S) 8184753: Assert against MinObjectAlignmentMask to avoid integer division In-Reply-To: <2e2413f4-3eb3-f124-fb77-48e52bd40d41@redhat.com> References: <2e2413f4-3eb3-f124-fb77-48e52bd40d41@redhat.com> Message-ID: <4ebdbce4-b1f2-3c82-92f2-cf4b19b7b3b4@redhat.com> On 07/17/2017 06:00 PM, Aleksey Shipilev wrote: > https://bugs.openjdk.java.net/browse/JDK-8184753 > > fastdebug builds performance is important to minimize testing time. In GC tests, > we have a big hotspot at this line in oop::size_given_klass(): > assert(s % MinObjAlignment == 0, "Oop size is not properly aligned: %d", s); > > The hotspot is because of the integer division. Alas, the divisor is not known > at compile time, because it can be set from the JVM command line, so compiler > cannot optimize it. There is an easy fix, use the MinObjectAlignmentMask to assert: > http://cr.openjdk.java.net/~shade/8184753/webrev.01 > > This also fixes all uses of MinObjAlignment in similar expressions. Noticed two new asserts in arguments.cpp have excess whitespace, fixed: http://cr.openjdk.java.net/~shade/8184753/webrev.02 -Aleksey From shade at redhat.com Mon Jul 17 19:42:02 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 17 Jul 2017 21:42:02 +0200 Subject: RFR (XS) 8184762: ZapStackSegments should use optimized memset In-Reply-To: <4a27accb-3915-c3fd-134f-6a6842a9cda1@redhat.com> References: <848320ea-4eed-09dc-95b9-8edb5da2fc56@redhat.com> <4a27accb-3915-c3fd-134f-6a6842a9cda1@redhat.com> Message-ID: <986fbc15-d5ec-4525-3f7e-1248fbba840d@redhat.com> On 07/17/2017 09:18 PM, Roman Kennke wrote: > I'd probably declare the constant char or jubyte or such > (fill_to_bytes() takes a jubyte, can't say I like the j*types much). > (breaks 'consistency' with other similar constants though... your call) Yes, I kept it as "int" for consistency. > Update copyright header. Updated: http://cr.openjdk.java.net/~shade/8184762/webrev.02/ Thanks, -Aleksey From karen.kinnear at oracle.com Mon Jul 17 20:10:12 2017 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Mon, 17 Jul 2017 16:10:12 -0400 Subject: RFR (L) 7133093: Improve system dictionary performance In-Reply-To: <9243da7c-a8e5-4024-c12e-cba9d4f424c8@oracle.com> References: <89f0b98c-3cbd-7b87-d76b-5e89a5a676fb@oracle.com> <9243da7c-a8e5-4024-c12e-cba9d4f424c8@oracle.com> Message-ID: <7814C032-0D4E-4E76-B8AD-62A072E8A625@oracle.com> Feedback on the initial web rev: Thank you for using web rev - that way I can do searches of the overall patch! First - many thanks for doing this exercise so carefully, for following the specification in continuing to record the initiating loader and for splitting this effort into multiple phases to allow further exploration of the current expectations/behavior relative to class unloading. And thank you for all the testing. Just wanted to ensure that I understand some of the assumptions: 1) lock free reads of dictionaries - yes 2) lock free adds of entries to dictionaries - yes 3) removal of entries or entire dictionaries - at safe point only 4) rehashing - none today and none planned 5) resizing of dictionaries - none today, some planned 6) use of SystemDictionary_lock: essentially unchanged 7) atomicity guarantees relative to checking loader constraints and updates to the dictionary: unchanged I did not review the SA/vmStructs - sounds like you had Jini George do that - great. Minor questions/comments: SystemDictionary::verify: "Verify of system dictionary failed" -> looks like "Verify of placeholders failed" thank you for asserts dictionary != NULL Dictionary::print is it worth an assert that loader_data != NULL? Can you please sanity check if there are other cases in which we need to use dictionary_or_null() rather than dictionary()? - are there cases in which we could be using a dictionary from an anonymous class? e.g. there are a lot of places in systemDictionary.cpp where we call class_loader_data(class_loader); loader_data->dictionary(); - was the lazily allocated dictionary allocated yet (see next question) CLD.cpp Would it be less complex to eagerly create the dictionary for a CLD since it is extremely rare to not need one for non-anonymous CLDs? You could create before publishing the CLD Terminology: How are you still using System Dictionary term? It still shows up in some comments and in some printed strings biasedLocking.cpp, line 59 jvmtiGetLoadedClasses: line 298 dictionary.hpp lines 113,114, 120 dictionary.cpp 262-263, 478 dictionary.cpp 491 fix comments e.g. "Iterate the system dictionary" ... - probably worth checking for other comments in the sources too unless these all appear to match new usage model hashTable.hpp line 268: what does this comment mean? "Don't create and use freelist"? I assume this means that this is a direct malloc, not using any block allocation, e.g. metaspace chunks, and this requires explicit freeing? dictionary.hpp line 161 could you possibly clean up comment while you are in here? I presume "a protection" is "a protection domain"? dictionary.cpp why change from Hashtable::free_entry to explicit unlink_entry/FREE_C_HEAP_ARRAY? Future optimization: I think that none of the built-in loaders is ever recorded as an initiating loader for any class in a loader which permits unloading. Note: we need to ensure that built-in includes boot, platform and application class loader (and clean up any confusion with system class loader - which can be set by the user to a custom class loader) Any chance of changing DictionaryEntry* probe->klass() to be probe->instanceKlass()? Thank you for assertion _is_at_safepoint for try_get_next_class CLD.cpp If the names of the classes_do vs. all_entries_do were modified to be defining_loader_classes_do vs. all_entries_classes_do or some way to make the distinction in the name that would be helpful. Otherwise, if you could please add comments to the CLDG::dictionary_classes_do etc. to distinguish these. CLD.cpp lines 1200-1201 Can you please fix the comment? I think you need separate comments for the ModuleEntry reads/PackageEntry's exports which are looking for dead modules. IIUC, the do_unloading is actually looking for classes for which this is the initating loader, but their defining loader is dead. systemDictionary.cpp: 445-447 THANK you for the comments about the java_mirror handle keeping the class and class loader alive Could you clarify that the class_loader handle is not for the class we are checking, but rather for the initiating, i.e. requesting class loader (is that accurate?) So we are keeping both the requestor and the requested class alive You removed the comment that compute_hash could lead to a safepoint/GC Is that not possible? comment in ObjectSynchronizer::FastHashCode says that with UseBiasedLocking we can revoke biases and must be able to handle a safepoint. systemDictionary.cpp: NoSafepointVerifier If we add resizing logic later, and that happens at a safepoint, then the number of buckets change and the d_index is no longer valid so we would need to add back the NSpV? - this is a more general problem if compute_hash can lead to a safepoint/GC systemDictionary.cpp: parse_stream why did you remove the record_dependency? Who calls Dictionary::~Dictionary? Performance: Do you have performance runs of our existing benchmarks? Initial key requirement is that this not slow down throughput through slowing down lookups. I know you are very aware of longer-term goals for reducing safepoint pause times and not increasing footprint significantly. hashtable sizes: future rfe? given that we now have a lot more dictionaries, and they perhaps need individual sizing - would it be helpful to add statistics information about - e.g. lookup_count, lookup_length, _lookup_warning - i.e. number of entries, max lookup depth, etc. to use later for tuning? - might make sense to make this JFR data? Might think about a follow-up with more tuning parameters than a single global PredictedLoadedClassCount. thanks, Karen > On Jul 17, 2017, at 2:37 PM, coleen.phillimore at oracle.com wrote: > > > Hi, I have made changes for Ioi's comments below. The largest change since .01 is that PredictedLoadedClassCount is used to size dictionaries for class loader data, and should be used to specify the average number of classes loaded by each class loader. Since this is an option available via UnlockExperimentalVMOptions, I'm not sure what the process is for changing the meaning of this option. We will work with our customers who have used this option for specifying the total number of classes loaded. We are also working on the change to allow automatic resizing of the dictionaries during safepoints. > > See: > open webrev at http://cr.openjdk.java.net/~coleenp/7133093.02/webrev > > Tested with the full nightly test set (jdk, nsk, hotspot jtreg tests) on linux x64. > > Thanks, > Coleen > > On 6/30/17 4:45 PM, coleen.phillimore at oracle.com wrote: >> >> Ioi, Thank you for looking at this change. >> >> On 6/30/17 2:39 PM, Ioi Lam wrote: >>> Hi Coleen, >>> >>> Maybe the bug should be renamed to "Use one Dictionary per class loader instance"? That way it's more obvious what it is when you look at the repo history. >> >> I can do that. I made it One Dictionary per ClassLoaderData >>> >>> 1. >>> >>> Why is assert_locked_or_safepoint(SystemDictionary_lock) necessary in SystemDictionary::find_class (line 1826), but not necessary SystemDictionary::find (line 951)? Since you removed NoSafepointVerifier nosafepoint in the latter, maybe this means it's safe to remove the assert_locked_or_safepoint in the former? >> >> The call to SystemDictionary::find() is the (I believe) usual lock free lookup and the SystemDictionary::find_class() is used to verify that a class we're about to add or want to add hasn't been added by another thread. Or certain cases where we already have a lock to do something else, like add loader constraints. I took out the NoSafepointVerifier because it assumes that system dictionary entries would be moved by GC, which they won't. The old hash function could safepoint when getting the hash for the class_loader object. >> >>> >>> 2. >>> >>> 455 static ClassLoaderData* _current_loader_data = NULL; >>> 456 static Klass* _current_class_entry = NULL; >>> 457 >>> 458 InstanceKlass* ClassLoaderDataGraph::try_get_next_class() { >>> >>> How about moving the static fields into an iterator object. That way you don't need to keep track of the globals >>> >>> ClassLoaderDataGraphIterator { >>> ClassLoaderData* _current_loader_data >>> Klass* _current_class_entry; >>> >>> InstanceKlass* try_get_next_class() { ....} >>> }; >> >> Ok, there's a different iterator that iterates over all of the classes for GC. I will adapt that for this use. That would be better. >> >>> >>> 3. Double check locking in ClassLoaderData::dictionary() -- someone else should look at this :-) >> >> I copied code that David Holmes added for modules. >> >>> >>> 4. We may need a better strategy for deciding the size of each dictionary. >>> >>> 565 const int _primelist[10] = {1, 107, 1009}; >>> 571 Dictionary* ClassLoaderData::dictionary() { >>> 579 if ((dictionary = _dictionary) == NULL) { >>> 580 int size; >>> 581 if (this == the_null_class_loader_data() || is_system_class_loader_data()) { >>> 582 size = _primelist[2]; >>> 583 } else if (class_loader()->is_a(SystemDictionary::reflect_DelegatingClassLoader_klass())) { >>> 584 size = _primelist[0]; >>> 585 } else { >>> 586 size = _primelist[1]; >>> 587 } >>> 588 dictionary = new Dictionary(this, size); >>> >>> I'll do some investigation on this issue and get back to you. >>> >> >> How about if someone uses PredictedLoadedClassCount, then we use that to size all but the reflection and boot class loader? Then if there's an application that has a class loader with a huge amount of classes loaded in it, that would help this? It might cost some footprint but an oversized table would simply be a bigger array of pointers, so it might not be that bad to oversize. >> >> I think the long term solution is being able to resize these entries and make the application provide arguments. Please let me know what you find in your investigation and if that would work. >>> The rest of the changes look good to me. >> >> Thank you! >> Coleen >>> >>> Thanks >>> - Ioi >>> >>> >>> On 6/23/17 4:42 PM, coleen.phillimore at oracle.com wrote: >>>> Summary: Implement one dictionary per ClassLoaderData for faster lookup and removal during class unloading >>>> >>>> See RFE for more details. >>>> >>>> open webrev at http://cr.openjdk.java.net/~coleenp/7133093.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-7133093 >>>> >>>> Tested with full "nightly" run in rbt, plus locally class loading and unloading tests: >>>> >>>> jtreg hotspot/test/runtime/ClassUnload >>>> >>>> jtreg hotspot/test/runtime/modules >>>> >>>> jtreg hotspot/test/gc/class_unloading >>>> >>>> make test-hotspot-closed-tonga FILTER=quick TEST_JOBS=4 TEST=vm.parallel_class_loading >>>> >>>> csh ~/testing/run_jck9 (vm/lang/java_lang) >>>> >>>> runThese -jck - uses class loader isolation to run each jck test and unloads tests when done (at -gc:5 intervals) >>>> >>>> >>>> Thanks, >>>> Coleen >>>> >>>> >>> >> > From daniel.daugherty at oracle.com Mon Jul 17 20:33:08 2017 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 17 Jul 2017 14:33:08 -0600 Subject: RFR(xs): 8184339: Thread::current_or_null() should not assert if Posix TLS is not yet initialized In-Reply-To: References: Message-ID: <44430144-a6b9-0280-1cd6-c13b3c9aa74d@oracle.com> On 7/17/17 1:00 AM, Thomas St?fe wrote: > Hi all, > > may I please have reviews + a sponsor for the following fix: > > bug: https://bugs.openjdk.java.net/browse/JDK-8184339 > webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/8184339-Thread-current-or-null-shall-not-assert/webrev.00/webrev/ src/share/vm/runtime/thread.hpp No comments. Thumbs up! Thanks for the clear analysis in the bug. Dan > The problem is caused by the fact that Posix TLS cannot be used before it > is initialized. It is initialized in os::init(). If we use Posix TLS (eg > via Thread::current()) before, we assert. It is used now (since JDK-8183925 > ) before os::init() (see > callstack in bug description). > > There are two functions, Thread::current() and Thread::current_or_null(). > The latter should not assert if Posix TLS is not yet available but return > NULL instead. This is what callers expect: this function is safe to call, > but it might not return a valid Thread*. > > Note that this issue currently only comes up at AIX, because AIX is the > only platform using Posix TLS for Thread::current() - all other platforms > use Compiler-based TLS (__thread variables). > > However, we want to keep the Posix TLS code path alive, so this may also > affect other platforms. There have been compiler bugs in the past (e.g. gcc > + glibc) leading to errors when using compiler-based TLS, so it is good to > have a reliable fallback. > > Thanks, > > Thomas From thomas.stuefe at gmail.com Tue Jul 18 05:47:10 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 18 Jul 2017 07:47:10 +0200 Subject: RFR(xs): 8184339: Thread::current_or_null() should not assert if Posix TLS is not yet initialized In-Reply-To: <44430144-a6b9-0280-1cd6-c13b3c9aa74d@oracle.com> References: <44430144-a6b9-0280-1cd6-c13b3c9aa74d@oracle.com> Message-ID: On Mon, Jul 17, 2017 at 10:33 PM, Daniel D. Daugherty < daniel.daugherty at oracle.com> wrote: > On 7/17/17 1:00 AM, Thomas St?fe wrote: > >> Hi all, >> >> may I please have reviews + a sponsor for the following fix: >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8184339 >> webrev: >> http://cr.openjdk.java.net/~stuefe/webrevs/8184339-Thread-cu >> rrent-or-null-shall-not-assert/webrev.00/webrev/ >> > > src/share/vm/runtime/thread.hpp > No comments. > > Thumbs up! Thanks for the clear analysis in the bug. > > Dan > > Thanks Dan! > > The problem is caused by the fact that Posix TLS cannot be used before it >> is initialized. It is initialized in os::init(). If we use Posix TLS (eg >> via Thread::current()) before, we assert. It is used now (since >> JDK-8183925 >> ) before os::init() >> (see >> callstack in bug description). >> >> There are two functions, Thread::current() and Thread::current_or_null(). >> The latter should not assert if Posix TLS is not yet available but return >> NULL instead. This is what callers expect: this function is safe to call, >> but it might not return a valid Thread*. >> >> Note that this issue currently only comes up at AIX, because AIX is the >> only platform using Posix TLS for Thread::current() - all other platforms >> use Compiler-based TLS (__thread variables). >> >> However, we want to keep the Posix TLS code path alive, so this may also >> affect other platforms. There have been compiler bugs in the past (e.g. >> gcc >> + glibc) leading to errors when using compiler-based TLS, so it is good to >> have a reliable fallback. >> >> Thanks, >> >> Thomas >> > > From matthias.baesken at sap.com Tue Jul 18 07:24:37 2017 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 18 Jul 2017 07:24:37 +0000 Subject: RFR: compile-time guard some UseSharedSpaces-only coding with the INCLUDE_CDS macro - was :RE: jdk10 : UseSharedSpaces flag and INCLUDE_CDS macro In-Reply-To: References: <4d2f3964b0924ffa897074759e508653@sap.com> Message-ID: <6f85d82587e349579cc7d5be62914ab5@sap.com> Hi , thanks for the review . >Related to the change in universe_init() (universe.cpp), the 'if (DumpSharedSpaces)? block at line 715 is also CDS only. >To be consistent, you might want to include it under #if INCLUDE_CDS as well. I added the suggested change and created a second webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8184323.1/ Is a second review needed ? Best regards, Matthias From: Jiangli Zhou [mailto:jiangli.zhou at oracle.com] Sent: Samstag, 15. Juli 2017 06:37 To: Baesken, Matthias Cc: hotspot-dev at openjdk.java.net; build-dev at openjdk.java.net Subject: Re: RFR: compile-time guard some UseSharedSpaces-only coding with the INCLUDE_CDS macro - was :RE: jdk10 : UseSharedSpaces flag and INCLUDE_CDS macro Hi Baesken, Thank you for making the changes. Related to the change in universe_init() (universe.cpp), the 'if (DumpSharedSpaces)? block at line 715 is also CDS only. To be consistent, you might want to include it under #if INCLUDE_CDS as well. --- 695,716 ---- Universe::_loader_addClass_cache = new LatestMethodCache(); Universe::_pd_implies_cache = new LatestMethodCache(); Universe::_throw_illegal_access_error_cache = new LatestMethodCache(); Universe::_do_stack_walk_cache = new LatestMethodCache(); + #if INCLUDE_CDS if (UseSharedSpaces) { // Read the data structures supporting the shared spaces (shared // system dictionary, symbol table, etc.). After that, access to // the file (other than the mapped regions) is no longer needed, and // the file is closed. Closing the file does not affect the // currently mapped regions. MetaspaceShared::initialize_shared_spaces(); StringTable::create_table(); ! } else ! #endif ! { SymbolTable::create_table(); StringTable::create_table(); if (DumpSharedSpaces) { MetaspaceShared::prepare_for_dumping(); In the past we have been trying to avoid adding too many #if in the code for better readability. For the CDS only code in universe.cpp (and a few other files), since the callee functions (e.g. initialize_shared_spaces()) are already defined with two versions (with and without INCLUDE_CDS). The builds without CDS also works fine even #if INCLUDE_CDS are not added. Thanks, Jiangli On Jul 13, 2017, at 5:18 AM, Baesken, Matthias > wrote: Thank you for noticing that. Yes, it would be helpful if you can add the #if INCLUDE_CDS for CDS only code. I can help you integrate the changes after they are reviewed. Thanks for your feedback ! I created a bug : https://bugs.openjdk.java.net/browse/JDK-8184323 and a webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8184323/ Best regards, Matthias -----Original Message----- From: Jiangli Zhou [mailto:jiangli.zhou at oracle.com] Sent: Montag, 10. Juli 2017 17:54 To: Baesken, Matthias > Cc: hotspot-dev at openjdk.java.net; build-dev at openjdk.java.net Subject: Re: jdk10 : UseSharedSpaces flag and INCLUDE_CDS macro Hi Matthias, Thank you for noticing that. Yes, it would be helpful if you can add the #if INCLUDE_CDS for CDS only code. I can help you integrate the changes after they are reviewed. Thanks! Jiangli On Jul 5, 2017, at 6:36 AM, Baesken, Matthias > wrote: Hello, when looking into CDS related build options, I noticed that most code-parts that are executed only when UseSharedSpaces is set, are guarded by the compile-time macro INCLUDE_CDS to support switching off compilation of this coding when CDS is disabled at compile time : See hotspot/make/lib/JvmFeatures.gmk : ifneq ($(call check-jvm-feature, cds), true) JVM_CFLAGS_FEATURES += -DINCLUDE_CDS=0 However some places miss the compile-time guarding ( #if INCLUDE_CDS ....) for example in share/vm/prims/jvmtiRedefineClasses.cpp share/vm/memory/universe.cpp (also some other places) Should I prepare a change and add the compile-time guard at the places where missing as well ? Best regards, Matthias From shade at redhat.com Tue Jul 18 07:30:53 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 18 Jul 2017 09:30:53 +0200 Subject: RFR: compile-time guard some UseSharedSpaces-only coding with the INCLUDE_CDS macro - was :RE: jdk10 : UseSharedSpaces flag and INCLUDE_CDS macro In-Reply-To: <6f85d82587e349579cc7d5be62914ab5@sap.com> References: <4d2f3964b0924ffa897074759e508653@sap.com> <6f85d82587e349579cc7d5be62914ab5@sap.com> Message-ID: On 07/18/2017 09:24 AM, Baesken, Matthias wrote: > I created a bug : > https://bugs.openjdk.java.net/browse/JDK-8184323 > > > and a webrev : > http://cr.openjdk.java.net/~mbaesken/webrevs/8184323/ This looks good. -Aleksey From mikael.gerdin at oracle.com Tue Jul 18 07:48:12 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Tue, 18 Jul 2017 09:48:12 +0200 Subject: RFR (S) 8184753: Assert against MinObjectAlignmentMask to avoid integer division In-Reply-To: <4ebdbce4-b1f2-3c82-92f2-cf4b19b7b3b4@redhat.com> References: <2e2413f4-3eb3-f124-fb77-48e52bd40d41@redhat.com> <4ebdbce4-b1f2-3c82-92f2-cf4b19b7b3b4@redhat.com> Message-ID: <430f3473-947f-9b51-a598-09d15922cdf6@oracle.com> Hi Aleksey, On 2017-07-17 21:40, Aleksey Shipilev wrote: > On 07/17/2017 06:00 PM, Aleksey Shipilev wrote: >> https://bugs.openjdk.java.net/browse/JDK-8184753 >> >> fastdebug builds performance is important to minimize testing time. In GC tests, >> we have a big hotspot at this line in oop::size_given_klass(): >> assert(s % MinObjAlignment == 0, "Oop size is not properly aligned: %d", s); >> >> The hotspot is because of the integer division. Alas, the divisor is not known >> at compile time, because it can be set from the JVM command line, so compiler >> cannot optimize it. There is an easy fix, use the MinObjectAlignmentMask to assert: >> http://cr.openjdk.java.net/~shade/8184753/webrev.01 >> >> This also fixes all uses of MinObjAlignment in similar expressions. > > Noticed two new asserts in arguments.cpp have excess whitespace, fixed: > http://cr.openjdk.java.net/~shade/8184753/webrev.02 Did you consider using is_object_aligned in the newly created align.hpp? http://hg.openjdk.java.net/jdk10/hs/hotspot/file/5e9c41536bd2/src/share/vm/utilities/align.hpp#l120 inline bool is_object_aligned(size_t word_size) { return is_aligned(word_size, MinObjAlignment); } Thanks /Mikael > > -Aleksey > From shade at redhat.com Tue Jul 18 07:51:09 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 18 Jul 2017 09:51:09 +0200 Subject: RFR (S) 8184753: Assert against MinObjectAlignmentMask to avoid integer division In-Reply-To: <430f3473-947f-9b51-a598-09d15922cdf6@oracle.com> References: <2e2413f4-3eb3-f124-fb77-48e52bd40d41@redhat.com> <4ebdbce4-b1f2-3c82-92f2-cf4b19b7b3b4@redhat.com> <430f3473-947f-9b51-a598-09d15922cdf6@oracle.com> Message-ID: Hi Mikael, On 07/18/2017 09:48 AM, Mikael Gerdin wrote: >> Noticed two new asserts in arguments.cpp have excess whitespace, fixed: >> http://cr.openjdk.java.net/~shade/8184753/webrev.02 > > Did you consider using is_object_aligned in the newly created align.hpp? > > http://hg.openjdk.java.net/jdk10/hs/hotspot/file/5e9c41536bd2/src/share/vm/utilities/align.hpp#l120 > > inline bool is_object_aligned(size_t word_size) { > return is_aligned(word_size, MinObjAlignment); > } I did, but haven't done this for two reasons: a) The asserts are about object _sizes_, not the objects themselves; b) Doing *Mask fits other uses of MinObjAlignment(InBytes)(Mask) around HS; Thanks, -Aleksey From mikael.gerdin at oracle.com Tue Jul 18 07:52:26 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Tue, 18 Jul 2017 09:52:26 +0200 Subject: RFR (XS) 8184762: ZapStackSegments should use optimized memset In-Reply-To: <986fbc15-d5ec-4525-3f7e-1248fbba840d@redhat.com> References: <848320ea-4eed-09dc-95b9-8edb5da2fc56@redhat.com> <4a27accb-3915-c3fd-134f-6a6842a9cda1@redhat.com> <986fbc15-d5ec-4525-3f7e-1248fbba840d@redhat.com> Message-ID: <728e5746-cddf-8421-5e30-850e45a797f7@oracle.com> Hi Aleksey, On 2017-07-17 21:42, Aleksey Shipilev wrote: > On 07/17/2017 09:18 PM, Roman Kennke wrote: >> I'd probably declare the constant char or jubyte or such >> (fill_to_bytes() takes a jubyte, can't say I like the j*types much). >> (breaks 'consistency' with other similar constants though... your call) > > Yes, I kept it as "int" for consistency. > >> Update copyright header. > > Updated: > http://cr.openjdk.java.net/~shade/8184762/webrev.02/ Looks fine. I can sponsor it for you. /Mikael > > Thanks, > -Aleksey > > > From shade at redhat.com Tue Jul 18 07:55:10 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 18 Jul 2017 09:55:10 +0200 Subject: RFR (XS) 8184762: ZapStackSegments should use optimized memset In-Reply-To: <728e5746-cddf-8421-5e30-850e45a797f7@oracle.com> References: <848320ea-4eed-09dc-95b9-8edb5da2fc56@redhat.com> <4a27accb-3915-c3fd-134f-6a6842a9cda1@redhat.com> <986fbc15-d5ec-4525-3f7e-1248fbba840d@redhat.com> <728e5746-cddf-8421-5e30-850e45a797f7@oracle.com> Message-ID: On 07/18/2017 09:52 AM, Mikael Gerdin wrote: > On 2017-07-17 21:42, Aleksey Shipilev wrote: >> On 07/17/2017 09:18 PM, Roman Kennke wrote: >>> I'd probably declare the constant char or jubyte or such >>> (fill_to_bytes() takes a jubyte, can't say I like the j*types much). >>> (breaks 'consistency' with other similar constants though... your call) >> >> Yes, I kept it as "int" for consistency. >> >>> Update copyright header. >> >> Updated: >> http://cr.openjdk.java.net/~shade/8184762/webrev.02/ > > Looks fine. > I can sponsor it for you. Thank you! Changeset: http://cr.openjdk.java.net/~shade/8184762/hotspot.changeset -Aleksey From mikael.gerdin at oracle.com Tue Jul 18 07:58:27 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Tue, 18 Jul 2017 09:58:27 +0200 Subject: RFR (S) 8184753: Assert against MinObjectAlignmentMask to avoid integer division In-Reply-To: References: <2e2413f4-3eb3-f124-fb77-48e52bd40d41@redhat.com> <4ebdbce4-b1f2-3c82-92f2-cf4b19b7b3b4@redhat.com> <430f3473-947f-9b51-a598-09d15922cdf6@oracle.com> Message-ID: <76b2e8f0-171f-5664-db32-6a9d68a3f10b@oracle.com> Hi Aleksey, On 2017-07-18 09:51, Aleksey Shipilev wrote: > Hi Mikael, > > On 07/18/2017 09:48 AM, Mikael Gerdin wrote: >>> Noticed two new asserts in arguments.cpp have excess whitespace, fixed: >>> http://cr.openjdk.java.net/~shade/8184753/webrev.02 >> >> Did you consider using is_object_aligned in the newly created align.hpp? >> >> http://hg.openjdk.java.net/jdk10/hs/hotspot/file/5e9c41536bd2/src/share/vm/utilities/align.hpp#l120 >> >> inline bool is_object_aligned(size_t word_size) { >> return is_aligned(word_size, MinObjAlignment); >> } > > I did, but haven't done this for two reasons: > a) The asserts are about object _sizes_, not the objects themselves; > b) Doing *Mask fits other uses of MinObjAlignment(InBytes)(Mask) around HS; But the is_object_aligned(size_t) function _is_ about object sizes, it just has a weird naming and a weird overload with the void* variant which asserts that a pointer is correctly aligned. Another approach would be to just get rid of the weird one in align.hpp and use the MinObjAlignmentMask in plab.cpp instead. /Mikael > > Thanks, > -Aleksey > > > From shade at redhat.com Tue Jul 18 08:02:28 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 18 Jul 2017 10:02:28 +0200 Subject: RFR (S) 8184753: Assert against MinObjectAlignmentMask to avoid integer division In-Reply-To: <76b2e8f0-171f-5664-db32-6a9d68a3f10b@oracle.com> References: <2e2413f4-3eb3-f124-fb77-48e52bd40d41@redhat.com> <4ebdbce4-b1f2-3c82-92f2-cf4b19b7b3b4@redhat.com> <430f3473-947f-9b51-a598-09d15922cdf6@oracle.com> <76b2e8f0-171f-5664-db32-6a9d68a3f10b@oracle.com> Message-ID: On 07/18/2017 09:58 AM, Mikael Gerdin wrote: > Hi Aleksey, > > On 2017-07-18 09:51, Aleksey Shipilev wrote: >> Hi Mikael, >> >> On 07/18/2017 09:48 AM, Mikael Gerdin wrote: >>>> Noticed two new asserts in arguments.cpp have excess whitespace, fixed: >>>> http://cr.openjdk.java.net/~shade/8184753/webrev.02 >>> >>> Did you consider using is_object_aligned in the newly created align.hpp? >>> >>> http://hg.openjdk.java.net/jdk10/hs/hotspot/file/5e9c41536bd2/src/share/vm/utilities/align.hpp#l120 >>> >>> >>> inline bool is_object_aligned(size_t word_size) { >>> return is_aligned(word_size, MinObjAlignment); >>> } >> >> I did, but haven't done this for two reasons: >> a) The asserts are about object _sizes_, not the objects themselves; >> b) Doing *Mask fits other uses of MinObjAlignment(InBytes)(Mask) around HS; > > > But the is_object_aligned(size_t) function _is_ about object sizes, it just has > a weird naming and a weird overload with the void* variant which asserts that a > pointer is correctly aligned. Ah, unfortunate naming indeed. Let me try to switch to is_object_aligned. > Another approach would be to just get rid of the weird one in align.hpp and use > the MinObjAlignmentMask in plab.cpp instead. If that was only plab.cpp, I'd try that. But there are more uses around HS now. Thanks, -Aleksey From matthias.baesken at sap.com Tue Jul 18 08:35:20 2017 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 18 Jul 2017 08:35:20 +0000 Subject: RFR: compile-time guard some UseSharedSpaces-only coding with the INCLUDE_CDS macro - was :RE: jdk10 : UseSharedSpaces flag and INCLUDE_CDS macro In-Reply-To: References: <4d2f3964b0924ffa897074759e508653@sap.com> <6f85d82587e349579cc7d5be62914ab5@sap.com> Message-ID: <34d6ade2d4c2428bb4aae85d9527eb65@sap.com> Thanks ! -----Original Message----- From: Aleksey Shipilev [mailto:shade at redhat.com] Sent: Dienstag, 18. Juli 2017 09:31 To: Baesken, Matthias ; Jiangli Zhou Cc: build-dev at openjdk.java.net; hotspot-dev at openjdk.java.net Subject: Re: RFR: compile-time guard some UseSharedSpaces-only coding with the INCLUDE_CDS macro - was :RE: jdk10 : UseSharedSpaces flag and INCLUDE_CDS macro On 07/18/2017 09:24 AM, Baesken, Matthias wrote: > I created a bug : > https://bugs.openjdk.java.net/browse/JDK-8184323 > > > and a webrev : > http://cr.openjdk.java.net/~mbaesken/webrevs/8184323/ This looks good. -Aleksey From shade at redhat.com Tue Jul 18 08:41:39 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 18 Jul 2017 10:41:39 +0200 Subject: RFR (S) 8184753: Asserts against MinObjectAlignment should avoid integer division In-Reply-To: References: <2e2413f4-3eb3-f124-fb77-48e52bd40d41@redhat.com> <4ebdbce4-b1f2-3c82-92f2-cf4b19b7b3b4@redhat.com> <430f3473-947f-9b51-a598-09d15922cdf6@oracle.com> <76b2e8f0-171f-5664-db32-6a9d68a3f10b@oracle.com> Message-ID: <5a3a129e-07ce-9af8-161e-85366f6aedb8@redhat.com> On 07/18/2017 10:02 AM, Aleksey Shipilev wrote: > On 07/18/2017 09:58 AM, Mikael Gerdin wrote: >> Hi Aleksey, >> >> On 2017-07-18 09:51, Aleksey Shipilev wrote: >>> Hi Mikael, >>> >>> On 07/18/2017 09:48 AM, Mikael Gerdin wrote: >>>>> Noticed two new asserts in arguments.cpp have excess whitespace, fixed: >>>>> http://cr.openjdk.java.net/~shade/8184753/webrev.02 >>>> >>>> Did you consider using is_object_aligned in the newly created align.hpp? >>>> >>>> http://hg.openjdk.java.net/jdk10/hs/hotspot/file/5e9c41536bd2/src/share/vm/utilities/align.hpp#l120 >>>> >>>> >>>> inline bool is_object_aligned(size_t word_size) { >>>> return is_aligned(word_size, MinObjAlignment); >>>> } >>> >>> I did, but haven't done this for two reasons: >>> a) The asserts are about object _sizes_, not the objects themselves; >>> b) Doing *Mask fits other uses of MinObjAlignment(InBytes)(Mask) around HS; >> >> >> But the is_object_aligned(size_t) function _is_ about object sizes, it just has >> a weird naming and a weird overload with the void* variant which asserts that a >> pointer is correctly aligned. > > Ah, unfortunate naming indeed. Let me try to switch to is_object_aligned. That seems to work well: http://cr.openjdk.java.net/~shade/8184753/webrev.03/ http://cr.openjdk.java.net/~shade/8184753/hotspot.changeset hotspot_tier1 seems fine. NB: Changed the synopsis for the change. Thanks, -Aleksey From mikael.gerdin at oracle.com Tue Jul 18 08:44:44 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Tue, 18 Jul 2017 10:44:44 +0200 Subject: RFR (S) 8184753: Asserts against MinObjectAlignment should avoid integer division In-Reply-To: <5a3a129e-07ce-9af8-161e-85366f6aedb8@redhat.com> References: <2e2413f4-3eb3-f124-fb77-48e52bd40d41@redhat.com> <4ebdbce4-b1f2-3c82-92f2-cf4b19b7b3b4@redhat.com> <430f3473-947f-9b51-a598-09d15922cdf6@oracle.com> <76b2e8f0-171f-5664-db32-6a9d68a3f10b@oracle.com> <5a3a129e-07ce-9af8-161e-85366f6aedb8@redhat.com> Message-ID: <71c0b741-158c-44e7-6aab-ad06d82d56dd@oracle.com> Hi Aleksey, On 2017-07-18 10:41, Aleksey Shipilev wrote: > On 07/18/2017 10:02 AM, Aleksey Shipilev wrote: >> On 07/18/2017 09:58 AM, Mikael Gerdin wrote: >>> Hi Aleksey, >>> >>> On 2017-07-18 09:51, Aleksey Shipilev wrote: >>>> Hi Mikael, >>>> >>>> On 07/18/2017 09:48 AM, Mikael Gerdin wrote: >>>>>> Noticed two new asserts in arguments.cpp have excess whitespace, fixed: >>>>>> http://cr.openjdk.java.net/~shade/8184753/webrev.02 >>>>> >>>>> Did you consider using is_object_aligned in the newly created align.hpp? >>>>> >>>>> http://hg.openjdk.java.net/jdk10/hs/hotspot/file/5e9c41536bd2/src/share/vm/utilities/align.hpp#l120 >>>>> >>>>> >>>>> inline bool is_object_aligned(size_t word_size) { >>>>> return is_aligned(word_size, MinObjAlignment); >>>>> } >>>> >>>> I did, but haven't done this for two reasons: >>>> a) The asserts are about object _sizes_, not the objects themselves; >>>> b) Doing *Mask fits other uses of MinObjAlignment(InBytes)(Mask) around HS; >>> >>> >>> But the is_object_aligned(size_t) function _is_ about object sizes, it just has >>> a weird naming and a weird overload with the void* variant which asserts that a >>> pointer is correctly aligned. >> >> Ah, unfortunate naming indeed. Let me try to switch to is_object_aligned. > > That seems to work well: > http://cr.openjdk.java.net/~shade/8184753/webrev.03/ > http://cr.openjdk.java.net/~shade/8184753/hotspot.changeset Looks good to me, I'll sponsor it once we get a second pair of eyes on it. /Mikael > > hotspot_tier1 seems fine. > > NB: Changed the synopsis for the change. > > Thanks, > -Aleksey > From thomas.schatzl at oracle.com Tue Jul 18 09:05:50 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 18 Jul 2017 11:05:50 +0200 Subject: RFR (S) 8184753: Asserts against MinObjectAlignment should avoid integer division In-Reply-To: <5a3a129e-07ce-9af8-161e-85366f6aedb8@redhat.com> References: <2e2413f4-3eb3-f124-fb77-48e52bd40d41@redhat.com> <4ebdbce4-b1f2-3c82-92f2-cf4b19b7b3b4@redhat.com> <430f3473-947f-9b51-a598-09d15922cdf6@oracle.com> <76b2e8f0-171f-5664-db32-6a9d68a3f10b@oracle.com> <5a3a129e-07ce-9af8-161e-85366f6aedb8@redhat.com> Message-ID: <1500368750.2815.0.camel@oracle.com> Hi Aleksey, On Tue, 2017-07-18 at 10:41 +0200, Aleksey Shipilev wrote: > On 07/18/2017 10:02 AM, Aleksey Shipilev wrote: > > > > On 07/18/2017 09:58 AM, Mikael Gerdin wrote: > > > > > > Hi Aleksey, > > > > > >?[...] > > > But the is_object_aligned(size_t) function _is_ about object > > > sizes, it just has a weird naming and a weird overload with the > > > void* variant which asserts that a pointer is correctly aligned. > > Ah, unfortunate naming indeed. Let me try to switch to > > is_object_aligned. > That seems to work well: > ? http://cr.openjdk.java.net/~shade/8184753/webrev.03/ > ? http://cr.openjdk.java.net/~shade/8184753/hotspot.changeset > ? latest changeset seems good. Ship it. Thomas From tobias.hartmann at oracle.com Tue Jul 18 09:12:51 2017 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Tue, 18 Jul 2017 11:12:51 +0200 Subject: [10] RFR(XS): 8184009: Missing null pointer check in InterpreterRuntime::update_mdp_for_ret() Message-ID: Hi, please review the following patch that adds a missing null check (guarantee) found by Parfait: https://bugs.openjdk.java.net/browse/JDK-8184009 http://cr.openjdk.java.net/~thartmann/8184009/webrev.00/ Tested with JPRT and RBT (running). Thanks, Tobias From thomas.stuefe at gmail.com Tue Jul 18 09:13:21 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 18 Jul 2017 11:13:21 +0200 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> Message-ID: Hi Eric, Volker, builds fine on AIX 5.3 and 7.1. gtests show no errors. Kind Regards, Thomas On Fri, Jul 14, 2017 at 6:28 PM, Erik ?sterlund wrote: > Hi, > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8184334 < > https://bugs.openjdk.java.net/browse/JDK-8184334> > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8184334/webrev.00/ < > http://cr.openjdk.java.net/~eosterlund/8184334/webrev.00/> > > The Atomic class mostly uses the JNI-specific j-types (jbyte, jshort, > jint, jlong). This has led to wide-spread pollution of j-types in the C++ > code base. > > Another problem is that when dealing with certain inherently not j-typed > types like pointers, hacks were built that allowed them using _ptr member > functions that had to be specialized on every architecture supported by > Java. > > Yet another problem is yet another exception, like size_t, that does not > quite fit being a pointer and isn't necessarily mapped to any j-type. Then > yet more exceptions were built on top. And what about intx? You get the > idea by now. > > As a remedy to the above mentioned problems, I propose to generalize the > Atomic implementation to use templates instead. > The new Atomic API handles pointers, all integer and floating point types > that the underlying architecture supports. > > To achieve this, a new metaprogramming tool called IntegerTypes was built, > that allows us to safely cast between different integral, floating point > and pointer types of the same size while preserving the bit pattern of the > values. This allows Atomic to take primitive types and canonicalize them to > the same integer type that is passed to the platform layer. > > As for atomic operations on integral types like add/inc/dec, pointer > scaling is now performed. That is, the functional behaviour is modelled > after the +=, ++ and -- operators correspondingly. The X_ptr member > functions have been deprecated, but are still there and can be used with > identical behaviour as they had before. But new code should just use the > non-ptr member functions instead. > > The platform layer specializes the specialized_X member functions that > take canonicalized signed integers as input, and turn them to some > platform-specific instruction sequence. > A nice benefit with these changes is that the platform-specific code has > gotten a lot of pointless redundancies removed such as how to perform > atomic loads and stores on native word or less sized integers, or having > inc/dec implementations that just reuse add. > > As for testing, new gtests were written for IntegerTypes as well as Atomic > to make it more convincing that the functional behaviour works. > Apart from that, I have run JPRT, JTREG, RBT hs-tier3 and a few local > tonga test lists including nsk.jdi, nsk.monitoring, nsk.stress, vm.gc on > Solaris SPARC T7-4 and Linux AMD64. > > As for oracle-external ports, I would appreciate if port maintainers could > check if it seems to make sense. I have done my best to try to make the > appropriate changes where needed. > > Special thanks go to Kim Barrett that has worked closely with me on this > and whom I exchanged many emails with, and so I would like him to be a > co-author. > > Thanks, > /Erik From shade at redhat.com Tue Jul 18 09:23:48 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 18 Jul 2017 11:23:48 +0200 Subject: [10] RFR(XS): 8184009: Missing null pointer check in InterpreterRuntime::update_mdp_for_ret() In-Reply-To: References: Message-ID: <8ef35a81-6bb7-6282-692d-e963a3285cce@redhat.com> On 07/18/2017 11:12 AM, Tobias Hartmann wrote: > please review the following patch that adds a missing null check (guarantee) found by Parfait: > https://bugs.openjdk.java.net/browse/JDK-8184009 > http://cr.openjdk.java.net/~thartmann/8184009/webrev.00/ Looks good. -Aleksey From thomas.stuefe at gmail.com Tue Jul 18 09:35:57 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 18 Jul 2017 11:35:57 +0200 Subject: !PRODUCT vs ASSERT? Message-ID: Hi all, a short and maybe stupid question, what is the precise difference between: #ifndef PRODUCT vs #ifdef ASSERT ? I assume: optimized: !PRODUCT && !ASSERT xxxdebug: !PRODUCT && ASSERT release: PRODUCT && !ASSERT is this true? Also, when guarding pieces of performance-costly sanity checks, which one should be used? (I see both used sometimes. For example, MetaChunk::mangle() implementation is guarded by !PRODUCT but its call is guarded once by ASSERT, once by !PRODUCT, is that deliberate?) Thanks, Thomas From erik.osterlund at oracle.com Tue Jul 18 09:38:27 2017 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 18 Jul 2017 11:38:27 +0200 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <596CD126.6080100@oracle.com> Message-ID: <596DD713.6080806@oracle.com> Hi Andrew, On 2017-07-17 17:59, Andrew Haley wrote: > Hi, > > On 17/07/17 16:00, Erik ?sterlund wrote: > >> I am glad that it seems to work. Thank you for reviewing. >> >> As for the over engineered casting mechanism, the IntegerTypes class in >> metaprogramming/integerTypes.hpp has lots of commentary about what the >> different casting mechanisms do. Hope that helps. > It doesn't, because most HotSpot programmers know what this stuff > does, but not why you're doing it. I am not sure what 'this stuff' is in this context, so I am afraid I can not answer why I am doing it at this point. >> To explain the reasoning, I will go through an example. Let's take >> atomic::cmpxchg. >> Each platform has a specialization for different supported cmpxchg >> implementations in terms of int32_t and int64_t. > OK, I get that. Some platforms presumably requires such > implementations, but would it not make more sense to restrict the > complexity to just those platforms? None of the GCC targets need it. The Atomic API in hotspot has a frontend and backend part. The frontend is shared, and mediates the atomic operations to the platform-specific backends. What I have worked on is the frontend. The backend is platform specific. Some platforms might want to use GCC intrinsics, while others might want to do something else. Are you proposing that the shared mediator to the backend should have a GCC-only path? >> So for example, we need to be able to perform the following conversions: >> 1) Floating point to integer without changing bit representaiton. It >> seems like the only way of doing this without violating the C++03 >> standard is to use memcpy. (no, the union trick in e.g. jfloat_cast is >> not standards compliant) > I'm not entirely convinced that memcpy does it either for "truly > portable" C++, but perhaps it's as good as we're going to get. I am curious what you mean here. >> 2) Pointer to integer >> 3) Anything with not matching CV qualifiers to matching CV qualifiers >> 4) Integer of wrong signedness to matching signedness >> 5) Integer with same signedness but declared differently, e.g. signed >> char is not the same as char, and will miss that specialization. For >> example on Solaris, int8_t is char, but jbyte is signed char, and those >> types are different. Sometimes even intptr_t is neither int32_t or >> int64_t as I have seen on clang. > If you have something like > > long n; > long *p = &n; > > The you can access the stored value in p with any type compatible with > long*. However, this: > > Atomic::cmpxchg_ptr(0, &p, &n); > > casts &n to (signed) intptr_t: this is guaranteed to work. Okay, so far so good then. > Unfortunately, it then calls generic_cmpxchg(0, &p, &n). This > accesses the stored pointer in p as a long, which is undefined > behaviour. As generic_cmpxchg is in the backend of Linux AArch64, I assume you are referring to the code in that backend, which should not come as a surprise. Further on, I am assuming you are referring to how for a pointer, previously intptr_t would be sent into generic_cmpxchg, and now we send in int64_t. But you will find that the following assert is true with GCC on AArch64 (the backend you are interested in): #include STATIC_ASSERT(IsSame::value); That is, intptr_t and int64_t is the exact same type by definition, and hence the type of the instantiated generic_cmpxchg is exactly the same. So this particular backend we would previously send in intptr_t, and now send in a type that IsSame as intptr_t. The behaviour looks identical to me. So as long as intptr_t and T* are compatible types (which they are by definition), I do not see how this is undefined behaviour. > It works on GCC if we compile everything with -fno-strict-aliasing (at > least if intptr_t has the same representation in memory as every > pointer type, which every machine we care about does) but in that case > there's little point casting pointers back and forth to integer types. I hope my answer above convinces you that this is not a problem. >> Due to these issues, the IntegerTypes class was created to solve the >> casting problems in one single place while remaining type safe. So it >> solves the problem of canonicalizing primitive types and casting them >> into exactly the same canonical integer type that can be safely passed >> into code that specializes on that canonical integer type, without >> changing the bit pattern or slipping in type safety. > It is not type safe. > > ------------------------------------------------------------------------ > 3.10, Lvalues and rvalues > > If a program attempts to access the stored value of an object through > a glvalue of other than one of the following types the behavior is > undefined: > > ? the dynamic type of the object, > > ? a cv-qualified version of the dynamic type of the object, > > ? a type similar (as defined in 4.4) to the dynamic type of the object, > > ? a type that is the signed or unsigned type corresponding to the > dynamic type of the object, > > ? a type that is the signed or unsigned type corresponding to a > cv-qualified version of the dynamic type of the object, > > ? an aggregate or union type that includes one of the aforementioned > types among its elements or non- static data members (including, > recursively, an element or non-static data member of a subaggregate > or contained union), > > ? a type that is a (possibly cv-qualified) base class type of the > dynamic type of the object, > > ? a char or unsigned char type. > ------------------------------------------------------------------------ > > You only have permission to convert pointers to intptr_t and back: you > do not have permission to access the stored value of a pointer an an > intptr_t. I would say the scenario you describe goes under "the dynamic type of the object" or "a type that is the signed or unsigned type corresponding to the dynamic type of the object", in the quoted section 3.10 of the standard, depending on specific use case. The problem that type aliasing is aimed at is if you store an A* and then load it as a B*, then the dynamic type is A*, yet it is loaded as B*, where B is not compatible with A. This is invariant of whether the value of the store is indirectly performed through intptr_t or loaded indirectly through intptr_t, as the dynamic type of the stored value is still A*. If you now perform a store of A* through Atomic that casts the pointer to intptr_t, and stores it, then the dynamic type is still A*. If a subsequent load of A* observes that store, then the type of that loads needs to be compatible with the dynamic type, which is A*. In this scenario, the observed value is the dynamic type A* and is loaded as A*, so that is all good. Conversely, if you perform a normal store of A*, and observe that value inside of Atomic, then the dynamic type is still A*, and the value observed in atomic is the signed type corresponding to the dynamic type A*, so that is all good. And the signed type corresponding to the dynamic type A* may safely be casted to A*. In summary, my point is that as long as the performed stores and loads in Atomic preserves the bit representation of whatever pointer was passed in, the dynamic type of that pointer is unchanged, invariantly of casting it to/from intptr_t, and hence the aliasing is allowed. Do you have a different interpretation of the standard than I do? > >> By solving this general casting problem problem in IntegerTypes, it >> turned the casting issues in Atomic into trivial non-problems, and >> hopefully can be used elsewhere in hotspot. For example the jfloat_cast >> with friend functions are not standards compliant and can now use >> IntegerTypes instead, for that mindblowingly seemless casting experience. >> >> I hope this explanation makes sense. > I don't believe that it entirely does, no. But it is the sort of > commentary which ought to be in the source code. > I can put some commentary in atomic.hpp describing the use of IntegerTypes for casting if you like. Thanks, /Erik From erik.osterlund at oracle.com Tue Jul 18 09:39:19 2017 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 18 Jul 2017 11:39:19 +0200 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: <7875630a-2ca2-d345-73d0-5653c125f259@redhat.com> References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <171bed59-57bd-0fcd-2cb0-4ab982470314@redhat.com> <596CDC80.40006@oracle.com> <7875630a-2ca2-d345-73d0-5653c125f259@redhat.com> Message-ID: <596DD747.3040500@oracle.com> Hi Aleksey, Thank you for checking this. Glad it turned out as I expected. /Erik On 2017-07-17 18:49, Aleksey Shipilev wrote: > On 07/17/2017 05:56 PM, Aleksey Shipilev wrote: >> On 07/17/2017 05:49 PM, Erik ?sterlund wrote: >>>> Something like the Unsafe.{get|put|CAS} -Xint performance test would be nice to >>>> have. >>> Sorry, but I do not understand what you mean here. >>> >>> 1) The code only boils down to performing casts that do not change any bit >>> pattern, and is by contract forced to be expanded and inlined at compile time, >>> so I would be very surprised about any regressions. >>> 2) This code executes when we are about to perform atomic operations, so any >>> accidental code generated for the casting (which seems odd in the first place), >>> I would be very surprised to see. >>> 3) Even if there was some barely noticeable regression in fastdebug atomics for >>> whatever reason, I do not know if I think it makes sense to pollute the code >>> with ugly hacks to micro optimize fastdebug execution time. >> I agree that fastdebug should not be our primary optimization target. I was >> wondering if all the template magic actually works well in fastdebug builds. >> >> Or product builds, for that matter: because while templates are indeed done >> during compile-time, say, the inliner that has to deal with new code shapes can >> have opinions. >> >> There are some VarHandles perf tests I have lying around, I can check this >> myself today. > Checked, times are the same for baseline/patched builds with either fastdebug or > release bits. Ditto the instruction counts, number of memory accesses, branches, > etc. Seems good! > > Thanks, > -Aleksey > From erik.osterlund at oracle.com Tue Jul 18 09:40:45 2017 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 18 Jul 2017 11:40:45 +0200 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> Message-ID: <596DD79D.1020501@oracle.com> Hi Thomas, Thank you for checking this. /Erik On 2017-07-18 11:13, Thomas St?fe wrote: > Hi Eric, Volker, > > builds fine on AIX 5.3 and 7.1. > > gtests show no errors. > > Kind Regards, Thomas > > > On Fri, Jul 14, 2017 at 6:28 PM, Erik ?sterlund > > wrote: > > Hi, > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8184334 > > > > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8184334/webrev.00/ > > > > > The Atomic class mostly uses the JNI-specific j-types (jbyte, > jshort, jint, jlong). This has led to wide-spread pollution of > j-types in the C++ code base. > > Another problem is that when dealing with certain inherently not > j-typed types like pointers, hacks were built that allowed them > using _ptr member functions that had to be specialized on every > architecture supported by Java. > > Yet another problem is yet another exception, like size_t, that > does not quite fit being a pointer and isn't necessarily mapped to > any j-type. Then yet more exceptions were built on top. And what > about intx? You get the idea by now. > > As a remedy to the above mentioned problems, I propose to > generalize the Atomic implementation to use templates instead. > The new Atomic API handles pointers, all integer and floating > point types that the underlying architecture supports. > > To achieve this, a new metaprogramming tool called IntegerTypes > was built, that allows us to safely cast between different > integral, floating point and pointer types of the same size while > preserving the bit pattern of the values. This allows Atomic to > take primitive types and canonicalize them to the same integer > type that is passed to the platform layer. > > As for atomic operations on integral types like add/inc/dec, > pointer scaling is now performed. That is, the functional > behaviour is modelled after the +=, ++ and -- operators > correspondingly. The X_ptr member functions have been deprecated, > but are still there and can be used with identical behaviour as > they had before. But new code should just use the non-ptr member > functions instead. > > The platform layer specializes the specialized_X member functions > that take canonicalized signed integers as input, and turn them to > some platform-specific instruction sequence. > A nice benefit with these changes is that the platform-specific > code has gotten a lot of pointless redundancies removed such as > how to perform atomic loads and stores on native word or less > sized integers, or having inc/dec implementations that just reuse add. > > As for testing, new gtests were written for IntegerTypes as well > as Atomic to make it more convincing that the functional behaviour > works. > Apart from that, I have run JPRT, JTREG, RBT hs-tier3 and a few > local tonga test lists including nsk.jdi, nsk.monitoring, > nsk.stress, vm.gc on Solaris SPARC T7-4 and Linux AMD64. > > As for oracle-external ports, I would appreciate if port > maintainers could check if it seems to make sense. I have done my > best to try to make the appropriate changes where needed. > > Special thanks go to Kim Barrett that has worked closely with me > on this and whom I exchanged many emails with, and so I would like > him to be a co-author. > > Thanks, > /Erik > > From aph at redhat.com Tue Jul 18 09:57:56 2017 From: aph at redhat.com (Andrew Haley) Date: Tue, 18 Jul 2017 10:57:56 +0100 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: <596DD713.6080806@oracle.com> References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <596CD126.6080100@oracle.com> <596DD713.6080806@oracle.com> Message-ID: <77d5a9e7-b2b0-79e4-effd-903dfe226512@redhat.com> On 18/07/17 10:38, Erik ?sterlund wrote: >> ------------------------------------------------------------------------ >> 3.10, Lvalues and rvalues >> >> If a program attempts to access the stored value of an object through >> a glvalue of other than one of the following types the behavior is >> undefined: >> >> ? the dynamic type of the object, >> >> ? a cv-qualified version of the dynamic type of the object, >> >> ? a type similar (as defined in 4.4) to the dynamic type of the object, >> >> ? a type that is the signed or unsigned type corresponding to the >> dynamic type of the object, >> >> ? a type that is the signed or unsigned type corresponding to a >> cv-qualified version of the dynamic type of the object, >> >> ? an aggregate or union type that includes one of the aforementioned >> types among its elements or non- static data members (including, >> recursively, an element or non-static data member of a subaggregate >> or contained union), >> >> ? a type that is a (possibly cv-qualified) base class type of the >> dynamic type of the object, >> >> ? a char or unsigned char type. >> ------------------------------------------------------------------------ >> >> You only have permission to convert pointers to intptr_t and back: you >> do not have permission to access the stored value of a pointer an an >> intptr_t. > > I would say the scenario you describe goes under "the dynamic type of > the object" or "a type that is the signed or unsigned type corresponding > to the dynamic type of the object", OK. > in the quoted section 3.10 of the standard, depending on specific > use case. The problem that type aliasing is aimed at is if you > store an A* and then load it as a B*, then the dynamic type is A*, > yet it is loaded as B*, where B is not compatible with A. Precisely. That is what is happening in this case. A is, say, void* and B is intptr_t. void* is not compatible with intptr_t. > This is invariant of whether the value of the store is indirectly > performed through intptr_t or loaded indirectly through intptr_t, as > the dynamic type of the stored value is still A*. Exactly so. The stored object is an A*. We're accessing it as an intptr_t. A* and intptr_t are not compatible types. This is true even if the intptr_t was the result of casting from an A*. The compiler doesn't necessarily know that the intptr_t was originally an A*: they might even be in separate compilation units. The compiler is entitled to assume that the object in memory of type A* does not alias with any value of type intptr_t. > If you now perform a store of A* through Atomic that casts the pointer > to intptr_t, and stores it, then the dynamic type is still A*. This isn't how it works. (BTW, in case it helps understand where I'm coming from, although I am not the greatest expert in this area, I am familiar with GCC's alias analysis because I am a GCC author.) > In summary, my point is that as long as the performed stores and loads > in Atomic preserves the bit representation of whatever pointer was > passed in, the dynamic type of that pointer is unchanged, invariantly of > casting it to/from intptr_t, and hence the aliasing is allowed. That is incorrect. Rules about aliasing are nothing to do with the bit representation. Let me reiterate: you may cast from any pointer type to intptr_t. You may not access a pointer in memory as though it were an intptr_t. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From tobias.hartmann at oracle.com Tue Jul 18 10:11:53 2017 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Tue, 18 Jul 2017 12:11:53 +0200 Subject: [10] RFR(XS): 8184009: Missing null pointer check in InterpreterRuntime::update_mdp_for_ret() In-Reply-To: <8ef35a81-6bb7-6282-692d-e963a3285cce@redhat.com> References: <8ef35a81-6bb7-6282-692d-e963a3285cce@redhat.com> Message-ID: Thanks for the review Aleksey! Best regards, Tobias On 18.07.2017 11:23, Aleksey Shipilev wrote: > On 07/18/2017 11:12 AM, Tobias Hartmann wrote: >> please review the following patch that adds a missing null check (guarantee) found by Parfait: >> https://bugs.openjdk.java.net/browse/JDK-8184009 >> http://cr.openjdk.java.net/~thartmann/8184009/webrev.00/ > > Looks good. > > -Aleksey > From matthias.baesken at sap.com Tue Jul 18 10:36:20 2017 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 18 Jul 2017 10:36:20 +0000 Subject: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build In-Reply-To: <738dc2d22854475f9a15e163f7038872@sap.com> References: <6c85b34b225e417788f794a99269e9fb@sap.com> <5967D063.1010808@cjnash.com> <424e7257bc504fe09d967ce56d64afd1@sap.com> <2ac4b05d05434369bdeb72f7018d00f8@sap.com> <948004c4-63e7-b0e4-1820-bf2351fc2327@oracle.com> <738dc2d22854475f9a15e163f7038872@sap.com> Message-ID: <2232715d41bd453abdc339a988bdce3c@sap.com> Ping : could someone from oracle please push it ? (reason is that it touches the generated-configure.sh files ) Thanks, Matthias -----Original Message----- From: Baesken, Matthias Sent: Freitag, 14. Juli 2017 17:05 To: 'Erik Joelsson' ; Simon Nash Cc: 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' ; Zeller, Arno Subject: RE: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build Thanks , can you push it ( as far as I know someone from Oracle needs to push it because generated-configure.sh is changed ) ? Best regards, Matthias -----Original Message----- From: Erik Joelsson [mailto:erik.joelsson at oracle.com] Sent: Freitag, 14. Juli 2017 14:43 To: Baesken, Matthias ; Simon Nash Cc: 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' ; Zeller, Arno Subject: Re: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build This looks good to me. /Erik On 2017-07-14 13:47, Baesken, Matthias wrote: > After Simons question , I checked that the jdk10 builds still works with gcc 4.7 on linux x86_64 as well . > And created a new webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8184338.1/ > > - check adjusted to minimum gcc 4.7 > - common/doc/building.md adjusted (the part talking about gcc versions) > > > Best regards, Matthias > > > > -----Original Message----- > From: Baesken, Matthias > Sent: Freitag, 14. Juli 2017 08:36 > To: 'Simon Nash' > Cc: Erik Joelsson ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' ; Zeller, Arno > Subject: RE: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build > >> I am currently building JDK 9 for arm32 (hard float and soft float) using >> a Linaro GCC 4.7 cross-compiler (binary download). What is the reason that >> this cannot be supported for JDK 10? > Hi Simon, > reason was that we know gcc-4.8 works nicely because we do a lot of builds (+tests) with this compiler . > For gcc 4.7 we just do not know ( but for old 4.3 / 4.4 it was obvious that the current flags do not work any more). > If you are using a gcc compiler < 4.8 only a warning is reported , it does not mean that you cannot build any more. > > My first suggestion was : > >>>> ( e.g. change to minimum gcc 4.6 or gcc 4.7 supporting the flags used in the build ) > So if you see a benefit to test for minimum gcc 4.7 and not 4.8 I am fine with this too (Erik what do you think?). > We at SAP just cannot tell that gcc 4.7 still works (because our builds do not use it). > > Best regards, Matthias > > > > -----Original Message----- > From: Simon Nash [mailto:simon at cjnash.com] > Sent: Donnerstag, 13. Juli 2017 21:56 > To: Baesken, Matthias > Cc: Erik Joelsson ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' ; Zeller, Arno > Subject: Re: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build > > I am currently building JDK 9 for arm32 (hard float and soft float) using > a Linaro GCC 4.7 cross-compiler (binary download). What is the reason that > this cannot be supported for JDK 10? > > Best regards, > Simon > > On 13/07/2017 14:40, Baesken, Matthias wrote: >> Hi Erik, I prepared the change : >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8184338/ >> >> bug : >> >> https://bugs.openjdk.java.net/browse/JDK-8184338 >> >> Asking for review(s) ... >> >> >> Best regards, Matthias >> >> >> >> >> -----Original Message----- >> From: Erik Joelsson [mailto:erik.joelsson at oracle.com] >> Sent: Donnerstag, 13. Juli 2017 13:34 >> To: Baesken, Matthias ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' >> Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build >> >> Hello, >> >> That would be the correct place. If you prepare the change and send the >> review I can sponsor the push. >> >> /Erik >> >> On 2017-07-13 13:16, Baesken, Matthias wrote: >>> Hi Erik, >>> >>>> AFAIK, the only reason we support GCC versions older than 4.9 is for you >>>> guys at SAP, so if you would suggest dropping support, that would of >>>> course be the simplest solution. >>> for jdk10 it is fine for us to use minimum gcc 4.8 . That would be probably the most simple solution to avoid running into the "-fno-var-tracking-assignments" issue >>> ( I now and then run into it when I forget to set the PATH to gcc-4.8 one should use for compiling on the SLES11 machine, so I pick up instead the default gcc 4.3.4 from the machine , >>> configure works nicely and then the build fails in the middle of the HS make ) >>> >>> autoconf/toolchain.m4 >>> >>> TOOLCHAIN_MINIMUM_VERSION_gcc="4.8" >>> >>> Is there right place to adjust I think, I guess we need someone for Oracle to regenerate the generated-configure.sh files ? >>> >>> Best regards, Matthias >>> >>> >>> >>> >>> -----Original Message----- >>> From: Erik Joelsson [mailto:erik.joelsson at oracle.com] >>> Sent: Donnerstag, 13. Juli 2017 11:59 >>> To: Baesken, Matthias ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' >>> Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build >>> >>> Hello Matthias, >>> >>> AFAIK, the only reason we support GCC versions older than 4.9 is for you >>> guys at SAP, so if you would suggest dropping support, that would of >>> course be the simplest solution. >>> >>> If you want to keep support but make the use of this flag optional, the >>> preferred method is to add a test in flags.m4. We have macros defined >>> for this. FLAGS_COMPILER_CHECK_ARGUMENTS or >>> FLAGS_CXX_COMPILER_CHECK_ARGUMENTS would be suitable in this case. Use >>> it to check if the flag is valid for the current compiler. If so, you >>> can define a variable >>> "CXXFLAGS_GCC_NO_VAR_TRACKING_ASSIGNMENTS=-fno-var-tracking-assignment" >>> and empty otherwise, and in the makefile use this variable in the >>> assignement. >>> >>> We want to avoid static shell expressions in the makefiles if possible >>> and keep that kind of logic in configure. It's also better to explicitly >>> check for features rather than versions. >>> >>> /Erik >>> >>> >>> On 2017-07-13 11:21, Baesken, Matthias wrote: >>>> Hello, when building jdk10 on Suse Linux 11 with default gcc/g++ 4.3.4 installed, I was running into >>>> compilation errors because of the missing support for g++ flag -fno-var-tracking-assignments . >>>> >>>> It seems gcc-4.6 has the -fvar-tracking-assignments / -fnovar-tracking-assignments flags , see >>>> >>>> https://gcc.gnu.org/onlinedocs/gcc-4.6.4/gcc/Option-Summary.html#Option-Summary >>>> >>>> I have no gcc-4.6 at hand but could verify that gcc-4.7 and gcc-4.8 "know" the flag. >>>> There are various solutions one could do to avoid the compilation error . >>>> >>>> >>>> 1) disallow usage of old gcc versions here : >>>> >>>> autoconf/toolchain.m4 >>>> >>>> TOOLCHAIN_MINIMUM_VERSION_gcc="4.3" >>>> >>>> ( e.g. change to minimum gcc 4.6 or gcc 4.7 supporting the flags used in the build ) >>>> >>>> >>>> 2) remove the flag -fno-var-tracking-assignments for older gcc versions here : >>>> (in a similar way it was done : http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/37e693211deb ) >>>> >>>> hotspot/make/lib/JvmOverrideFiles.gmk-32-ifeq ($(TOOLCHAIN_TYPE), gcc) >>>> hotspot/make/lib/JvmOverrideFiles.gmk:33: BUILD_LIBJVM_vmStructs.cpp_CXXFLAGS := -fno-var-tracking-assignments -O0 >>>> hotspot/make/lib/JvmOverrideFiles.gmk:34: BUILD_LIBJVM_jvmciCompilerToVM.cpp_CXXFLAGS := -fno-var-tracking-assignments >>>> hotspot/make/lib/JvmOverrideFiles.gmk-35-endif >>>> >>>> What is your preferred solution ? >>>> >>>> >>>> Thanks, Matthias >>>> >>>> From mikael.gerdin at oracle.com Tue Jul 18 12:19:46 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Tue, 18 Jul 2017 14:19:46 +0200 Subject: !PRODUCT vs ASSERT? In-Reply-To: References: Message-ID: Hi Thomas, On 2017-07-18 11:35, Thomas St?fe wrote: > Hi all, > > a short and maybe stupid question, what is the precise difference between: > > #ifndef PRODUCT > > vs > > #ifdef ASSERT > > ? > > I assume: > > optimized: !PRODUCT && !ASSERT > xxxdebug: !PRODUCT && ASSERT > release: PRODUCT && !ASSERT Correct. > > is this true? > > Also, when guarding pieces of performance-costly sanity checks, which one > should be used? My thinking is that the performance of "optimized" should be close to "release" checks should be kept out of the hottest paths. > > (I see both used sometimes. For example, MetaChunk::mangle() implementation > is guarded by !PRODUCT but its call is guarded once by ASSERT, once by > !PRODUCT, is that deliberate?) I think it's a mix between judgment calls and misunderstandings :) I don't consider metaspace allocations such a hot path so it's not completely unreasonable I think. /Mikael > > Thanks, Thomas > From thomas.stuefe at gmail.com Tue Jul 18 12:22:52 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 18 Jul 2017 14:22:52 +0200 Subject: !PRODUCT vs ASSERT? In-Reply-To: References: Message-ID: Hi Mikael, On Tue, Jul 18, 2017 at 2:19 PM, Mikael Gerdin wrote: > Hi Thomas, > > On 2017-07-18 11:35, Thomas St?fe wrote: > >> Hi all, >> >> a short and maybe stupid question, what is the precise difference between: >> >> #ifndef PRODUCT >> >> vs >> >> #ifdef ASSERT >> >> ? >> >> I assume: >> >> optimized: !PRODUCT && !ASSERT >> xxxdebug: !PRODUCT && ASSERT >> release: PRODUCT && !ASSERT >> > > Correct. > > >> is this true? >> >> Also, when guarding pieces of performance-costly sanity checks, which one >> should be used? >> > > My thinking is that the performance of "optimized" should be close to > "release" checks should be kept out of the hottest paths. > > That was my assumption too. So, I'll guard expensive checks with #ifdef ASSERT. > >> (I see both used sometimes. For example, MetaChunk::mangle() >> implementation >> is guarded by !PRODUCT but its call is guarded once by ASSERT, once by >> !PRODUCT, is that deliberate?) >> > > I think it's a mix between judgment calls and misunderstandings :) > I don't consider metaspace allocations such a hot path so it's not > completely unreasonable I think. > /Mikael > > >> Thanks, Thomas >> >> ..Thomas From claes.redestad at oracle.com Tue Jul 18 12:23:27 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 18 Jul 2017 14:23:27 +0200 Subject: !PRODUCT vs ASSERT? In-Reply-To: References: Message-ID: <7bfb78a1-82af-6efd-15e4-7748a0f3d0c5@oracle.com> On 07/18/2017 02:19 PM, Mikael Gerdin wrote: >> > > My thinking is that the performance of "optimized" should be close to > "release" checks should be kept out of the hottest paths. Nit: not sure but this needs a comma somewhere would be nice :-) /Claes From mikael.gerdin at oracle.com Tue Jul 18 12:24:34 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Tue, 18 Jul 2017 14:24:34 +0200 Subject: !PRODUCT vs ASSERT? In-Reply-To: <7bfb78a1-82af-6efd-15e4-7748a0f3d0c5@oracle.com> References: <7bfb78a1-82af-6efd-15e4-7748a0f3d0c5@oracle.com> Message-ID: <7c6d4b30-a035-3b37-09a2-a80a928f5cda@oracle.com> On 2017-07-18 14:23, Claes Redestad wrote: > > > On 07/18/2017 02:19 PM, Mikael Gerdin wrote: >>> >> >> My thinking is that the performance of "optimized" should be close to >> "release" checks should be kept out of the hottest paths. > > Nit: not sure but this needs a comma somewhere would be nice :-) :D > > /Claes From thomas.stuefe at gmail.com Tue Jul 18 13:02:09 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 18 Jul 2017 15:02:09 +0200 Subject: RFR(xs): 8184339: Thread::current_or_null() should not assert if Posix TLS is not yet initialized In-Reply-To: References: Message-ID: Hi Folks, may I please have a second review and a sponsor? This is a bit of an annoying issue, it causes endless recursions on AIX because error reporting stumbles over itself when calling Thread::current(). Thanks! ..Thomas On Mon, Jul 17, 2017 at 9:00 AM, Thomas St?fe wrote: > Hi all, > > may I please have reviews + a sponsor for the following fix: > > bug: https://bugs.openjdk.java.net/browse/JDK-8184339 > webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8184339-Thread- > current-or-null-shall-not-assert/webrev.00/webrev/ > > The problem is caused by the fact that Posix TLS cannot be used before it > is initialized. It is initialized in os::init(). If we use Posix TLS (eg > via Thread::current()) before, we assert. It is used now (since > JDK-8183925 ) before > os::init() (see callstack in bug description). > > There are two functions, Thread::current() and Thread::current_or_null(). > The latter should not assert if Posix TLS is not yet available but return > NULL instead. This is what callers expect: this function is safe to call, > but it might not return a valid Thread*. > > Note that this issue currently only comes up at AIX, because AIX is the > only platform using Posix TLS for Thread::current() - all other platforms > use Compiler-based TLS (__thread variables). > > However, we want to keep the Posix TLS code path alive, so this may also > affect other platforms. There have been compiler bugs in the past (e.g. gcc > + glibc) leading to errors when using compiler-based TLS, so it is good to > have a reliable fallback. > > Thanks, > > Thomas > From gnu.andrew at redhat.com Tue Jul 18 13:31:59 2017 From: gnu.andrew at redhat.com (Andrew Hughes) Date: Tue, 18 Jul 2017 14:31:59 +0100 Subject: RFR(XS) [8u backport] 8181055: PPC64: "mbind: Invalid argument" still seen after 8175813 In-Reply-To: References: <5361e308-00e1-3041-c728-b6ebae7586ae@redhat.com> <25ab002d-0728-4aaa-eeb7-d3b9a3ea8254@redhat.com> <62524c62-ded1-6fab-e00c-8666fbff1645@oracle.com> <2239bacf-1760-1f73-20a1-ce792a191741@redhat.com> <593AAB00.4060801@linux.vnet.ibm.com> Message-ID: On 17 July 2017 at 19:16, Zhengyu Gu wrote: > Hi, > > This 8u backport of JDK-8181005 has been reviewed by Aleksey [1] and > approved by Rob [2]. > > However, it is not clear to me that the only reviewer has to be 'R'eviewer. > If so, can someone with 'R' to review the backport? It is almost the same as > JDK9, with very minor conflicts. > > I will also need sponsor this backport. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8181055 > Webrev: http://cr.openjdk.java.net/~zgu/8181055/8u/webrev.00/ > > > [1] http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-June/027104.html > [2] http://mail.openjdk.java.net/pipermail/jdk8u-dev/2017-June/006723.html > > Thanks, > > -Zhengyu It looks good to me. The 8u patch matches what we included in our packages and matches the patch already in hs/10, bar a slight context difference in one case. It built fine. I'm an 8u reviewer if that's what you're looking for. I can push it too if needed (I've pushed 8u jdk patches before), but not sure if 8u HotSpot patches still need to be run through that JPRT thing. Some clarity on this would be good. Is there a plan to fix this in 9 as well? Thanks, -- Andrew :) Senior Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) Web Site: http://fuseyism.com Twitter: https://twitter.com/gnu_andrew_java PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 From zgu at redhat.com Tue Jul 18 13:39:20 2017 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 18 Jul 2017 09:39:20 -0400 Subject: RFR(XS) [8u backport] 8181055: PPC64: "mbind: Invalid argument" still seen after 8175813 In-Reply-To: References: <5361e308-00e1-3041-c728-b6ebae7586ae@redhat.com> <25ab002d-0728-4aaa-eeb7-d3b9a3ea8254@redhat.com> <62524c62-ded1-6fab-e00c-8666fbff1645@oracle.com> <2239bacf-1760-1f73-20a1-ce792a191741@redhat.com> <593AAB00.4060801@linux.vnet.ibm.com> Message-ID: <30776b56-9521-0632-78cd-997fce1827e8@redhat.com> Hi Andrew, On 07/18/2017 09:31 AM, Andrew Hughes wrote: > On 17 July 2017 at 19:16, Zhengyu Gu wrote: >> Hi, >> >> This 8u backport of JDK-8181005 has been reviewed by Aleksey [1] and >> approved by Rob [2]. >> >> However, it is not clear to me that the only reviewer has to be 'R'eviewer. >> If so, can someone with 'R' to review the backport? It is almost the same as >> JDK9, with very minor conflicts. >> >> I will also need sponsor this backport. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8181055 >> Webrev: http://cr.openjdk.java.net/~zgu/8181055/8u/webrev.00/ >> >> >> [1] http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-June/027104.html >> [2] http://mail.openjdk.java.net/pipermail/jdk8u-dev/2017-June/006723.html >> >> Thanks, >> >> -Zhengyu > > It looks good to me. The 8u patch matches what we included in our packages > and matches the patch already in hs/10, bar a slight context difference in one > case. It built fine. > > I'm an 8u reviewer if that's what you're looking for. Yes, thanks for the reviewing. > > I can push it too if needed (I've pushed 8u jdk patches before), but > not sure if 8u > HotSpot patches still need to be run through that JPRT thing. Some clarity on > this would be good. I assume that it has to go through JPRT, but I am not sure about 8u. > > Is there a plan to fix this in 9 as well? My understanding is that, jdk10 bug will be automatically backported to jdk9, no? Thanks, -Zhengyu > > Thanks, > From bob.vandette at oracle.com Tue Jul 18 13:43:24 2017 From: bob.vandette at oracle.com (Bob Vandette) Date: Tue, 18 Jul 2017 09:43:24 -0400 Subject: JEP [DRAFT]: Container aware Java In-Reply-To: References: Message-ID: > On Jul 17, 2017, at 11:55 AM, Volker Simonis wrote: > > On Fri, Jul 14, 2017 at 6:54 PM, Bob Vandette wrote: >> >>> On Jul 14, 2017, at 12:15 PM, Volker Simonis wrote: >>> >>> Hi Bob, >>> >>> thanks for starting this JEP. I think it is long overdue and will be >>> very helpful for many people. >>> >> Thanks for taking a look at the JEP. >> >>> That said, I don't see any mentioning of the targeted Java release. As >>> JEPs usually go into the development version first, I suppose the work >>> for this JEP will be done in JDK10. However, taking into account the >>> current release cycles, the usefulness of this JEP for the Java >>> community will significantly depend on the its availability in Java 9 >>> (at least) and Java 8 (hopefully?). Any thoughts on this? >> >> I am certainly going to do my best to try to get this feature into JDK10 and >> I understand your concern about supporting Java 8 and 9. The decision to >> backport this feature will have to be done based on the usual criteria (complexity, >> usefulness, resource availability and compatibility). It may be possible to >> add some subset of this functionality in a backport but we?ll have to wait until >> the design is further along. >> >>> >>> You also mention that this JEP is only about Docker support on >>> Linux/x86_64. I'd argue that we should at least address at least all >>> Docker supported Linux platforms. I'm pretty sure our colleagues from >>> IBM would like to support this on Linux/ppc64 and Linux/s390 as well. >> >> Right now I don?t expect to have to write CPU specific logic. >> Therefore supporting ppc64, s390 or even aarch64 should boil down to testing. >> >>> >>> Moreover, the suggested API and the new JVM_xxxxxx functions should be >>> at least designed in such a way that other virtualisation environment >>> such as VMWare/VirtualBox/Xen/KVM can take advantage of it. I'm also >>> not sure if a completely new API solely for Docker containers is >>> making sense and is worth the effort? I understand that containers and >>> virtualisation are two different things, but they both impose similar >>> restrictions to the VM (i.e. virtualisation may present a huge number >>> of CPUs to the OS/VM which in reality are mapped to a single, physical >>> CPU) so maybe we should design the new API in such a way that it can >>> be used to tackle both problems. >> >> I will try to keep that in mind. I will have to look into the feasibility of accessing >> host configuration from within a container or virtual machine. For containers, >> it seems to me that would break the intent of isolation. > > Did you wanted to say "for virtualisation, it seems to me that would > break the intent of isolation." here? Well the comment applies to both technologies but cgroups/containers were specifically designed to isolate or limit processes from accessing host resources. > > So you're right, that you can not easily query the host/guest > information in a virtualised environment (like for examples VmWare) > because that would break isolation. But as far as I know, every > virtualisation solution has it's own library/API which can be used to > query the host (if there is any) and get information about the host > system and the resources allocated for the guest. We've done some > basic virtualisation detection for common systems like VmWare or Xen > (cc'ed Matthias who may know more details. If you can point me at some API docs, I?d appreciate it. Bob. > >> I know that host information >> does leak into containers through the proc file system but I suspect this was done >> for simplicity and compatibility reasons. >> >>> >>> Finally, you mention in section D. that all the information should be >>> logged the to error crash logs. That's good but maybe you could add >>> that all the detection and all the findings should also be made >>> available through the new Unified JVM Logging facility (JEP 158)? >> >> That?s a good idea. I will add that to the JEP. >> >> Thanks, >> Bob. >> >>> >>> Regards, >>> Volker >>> >>> >>> On Fri, Jul 14, 2017 at 4:22 PM, Bob Vandette wrote: >>>> I?d like to propose the following JEP that will enhance the Java runtime to be more container aware. >>>> This will allow the Java runtime to performance better ergonomics and provide better >>>> execution time statistics when running in a container. >>>> >>>> >>>> JEP Issue: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8182070 >>>> >>>> Here?s a Text dump of the JEP contents for your convenience: >>>> >>>> >>>> Summary >>>> ------- >>>> >>>> Container aware Java runtime >>>> >>>> Goals >>>> ----- >>>> >>>> Enhance the JVM and Core libraries to detect running in a container and adapt to the system resources available to it. This JEP will only support Docker on Linux-x64 although the design should be flexible enough to allow support for other platforms and container technologies. The initial focus will be on Linux low level container technology such as cgroups so that we will be able to easily support other container technologies running on Linux in addition to Docker. >>>> >>>> Non-Goals >>>> --------- >>>> >>>> It is not a goal of this JEP to support any platform other than Docker container technology running on Linux x64. >>>> >>>> Success Metrics >>>> --------------- >>>> >>>> Success will be measured by the improved efficiency of running multiple Java containers on a host system with out of the box options. >>>> >>>> Motivation >>>> ---------- >>>> >>>> Container technology is becoming more and more prevalent in Cloud based applications. This technology provides process isolation and allows the platform vendor to specify limits and alter the behavior of a process running inside a container that the Java runtime is not aware of. This causes the Java runtime to potentially attempt to use more system resources than are available to it causing performance degradation or even termination. >>>> >>>> >>>> Description >>>> ----------- >>>> >>>> This enhancement will be made up of the following work items: >>>> >>>> A. Detecting if Java is running in a container. >>>> >>>> The Java runtime, as well as any tests that we might write for this feature, will need to be able to detect that the current Java process is running in a container. I propose that we add a new JVM_ native function that returns a boolean true value if we are running inside a container. >>>> >>>> JVM_InContainer will return true if the currently running process is running in a container, otherwise false will be returned. >>>> >>>> B. Exposing container resource limits and configuration. >>>> >>>> There are several configuration options and limits that can be imposed upon a running container. Not all of these >>>> are important to a running Java process. We clearly want to be able to detect how many CPUs have been allocated to our process along with the maximum amount of memory that we be allocated but there are other options that we might want to base runtime decisions on. >>>> >>>> In addition, since Container typically impose limits on system resources, they also provide the ability to easily access the amount of consumption of these resources. I intent on providing this information in addition to the configuration data. >>>> >>>> I propose adding a new jdk.internal.Platform class that will allow access to this information. Since some of this information is needed during the startup of the VM, I propose that much of the implementation of the methods in the Platform class be done in the VM and exposed as JVM_xxxxxx functions. In hotspot, the JVM_xxxxxx function will be implemented via the os.hpp interface. >>>> >>>> Here are the categories of configuration and consumption statistics that will be made available (The exact API is TBD): >>>> >>>> isContainerized >>>> Memory Limit >>>> Total Memory Limit >>>> Soft Memory Limit >>>> Max Memory Usage >>>> Current Memory Usage >>>> Maximum Kernel Memory >>>> CPU Shares >>>> CPU Period >>>> CPU Quote >>>> Number of CPUs >>>> CPU Sets >>>> CPU Set Memory Nodes >>>> CPU Usage >>>> CPU Usage Per CPU >>>> Block I/O Weight >>>> Block I/O Device Weight >>>> Device I/O Read Rate >>>> Device I/O Write Rate >>>> OOM Kill Enabled >>>> OOM Score Adjustment >>>> Memory Swappiness >>>> Shared Memory Size >>>> >>>> TODO: >>>> 1. Need to specify the exact arguments and return format for these accessor functions. >>>> >>>> C. Adjusting Java runtime configuration based on limits. >>>> >>>> Java startup normally queries the operating system in order to setup runtime defaults for things such as the number of GC threads and default memory limits. When running in a container, the operating system functions used provide information about the host and does not include the containers configuration and limits. The VM and core libraries will be modified as part of this JEP to first determine if the current running process is running in a container. It will then cause the runtime to use the container values rather than the general operating system functions for configuring and managing the Java process. There have been a few attempts to correct some of these issue in the VM but they are not complete. The CPU detection in the VM currently only handles a container that limits cpu usage via CPU sets. If the Docker --cpu or --cpu-period along with --cpu-quota options are specified, it currently has no effect on the VMs configuration. >>>> >>>> The experimental memory detection that has been implemented only impacts the Heap selection and does not apply to the os::physical_memory or os::available_memory low level functions. This leaves other parts of the VM and >>>> core libraries to believe there is more memory available than there actually is. >>>> >>>> The Numa support available in the VM is also not correct when running in a container. The number of available memory nodes and enabled nodes as reported by the libnuma library does not take into account the impact of the Docker --cpuset-mems option which restricts which memory nodes the container can use. Inside the container, the file /proc/{pid}/self does report the correct Cpus_allowed and Mems_Allowed but libnuma doesn't respect this. This has been verified via the numactl utility. >>>> >>>> To correct these shortcomings and make this support more robust, here's a list of the current cgroup subsystems that we be examined in order to update the internal VM and core library configuration. >>>> >>>> **Number of CPUs** >>>> >>>> Use a combination of number_of_cpus() and cpu_sets() in order to determine how many processors are available to the process and adjust the JVMs os::active_processor_count appropriately. The number_of_cpus() will be calculated based on the cpu_quota() and cpu_period() using this formula: number_of_cpus() = cpu_quota() / cpu_period(). Since it's not currently possible to understand the relative weight of the running container against all other containers, altering the cpu_shares of a running container will have no affect on altering Java's configuration. >>>> >>>> **Total available memory** >>>> >>>> Use the memory_limit() value from the cgroup file system to initialize the os::physical_memory() value in the VM. This value will propagate to all other parts of the Java runtime. >>>> >>>> We might also consider examining the soft_memory_limit and total_memory_limit in addition to the memory_limit during the ergonomics startup processing in order to fine tuning some of the other VM settings. >>>> >>>> **CPU Memory Nodes** >>>> >>>> Use cpu_set_memory_nodes() to configure the os::numa support. >>>> >>>> **Memory usage** >>>> >>>> Use memory_usage_in_bytes() for providing os::available_memory() by subtracting the usage from the total available memory allocated to the container. >>>> >>>> D. Adding container configuration to error crash logs. >>>> >>>> As as troubleshooting aid, we will dump any available container statistics to the hotspot error log. >>>> >>>> E. Adding a startup flag to enable/disable this support. >>>> >>>> Add a -XX:+UseContainerSupport VM option that will be used to enable this support. The default will be off until this feature is proven. >>>> >>>> Alternatives >>>> ------------ >>>> >>>> There are a few existing RFE's filed that could be used to enhance the current experimental implementation rather than taking the JEP route. >>>> >>>> Testing >>>> ------- >>>> >>>> Docker/container specific tests should be added in order to validate the functionality being provided with this JEP. >>>> >>>> Risks and Assumptions >>>> --------------------- >>>> >>>> Docker is currently based on cgroups v1. Cgroups v2 is also available but is incomplete and not yet supported by Docker. It's possible that v2 could replace v1 in an incompatible way rendering this work unusable until it is upgraded. >>>> >>>> Dependencies >>>> ----------- >>>> >>>> None at this time. >>>> >> From coleen.phillimore at oracle.com Tue Jul 18 13:45:15 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 18 Jul 2017 09:45:15 -0400 Subject: [10] RFR(XS): 8184009: Missing null pointer check in InterpreterRuntime::update_mdp_for_ret() In-Reply-To: References: Message-ID: <2d498b3a-8158-2925-309e-e884d2739a1b@oracle.com> Looks good. Coleen On 7/18/17 5:12 AM, Tobias Hartmann wrote: > Hi, > > please review the following patch that adds a missing null check (guarantee) found by Parfait: > https://bugs.openjdk.java.net/browse/JDK-8184009 > http://cr.openjdk.java.net/~thartmann/8184009/webrev.00/ > > Tested with JPRT and RBT (running). > > Thanks, > Tobias From erik.osterlund at oracle.com Tue Jul 18 13:46:42 2017 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 18 Jul 2017 15:46:42 +0200 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: <77d5a9e7-b2b0-79e4-effd-903dfe226512@redhat.com> References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <596CD126.6080100@oracle.com> <596DD713.6080806@oracle.com> <77d5a9e7-b2b0-79e4-effd-903dfe226512@redhat.com> Message-ID: <596E1142.6040708@oracle.com> On 2017-07-18 11:57, Andrew Haley wrote: > On 18/07/17 10:38, Erik ?sterlund wrote: > >>> ------------------------------------------------------------------------ >>> 3.10, Lvalues and rvalues >>> >>> If a program attempts to access the stored value of an object through >>> a glvalue of other than one of the following types the behavior is >>> undefined: >>> >>> ? the dynamic type of the object, >>> >>> ? a cv-qualified version of the dynamic type of the object, >>> >>> ? a type similar (as defined in 4.4) to the dynamic type of the object, >>> >>> ? a type that is the signed or unsigned type corresponding to the >>> dynamic type of the object, >>> >>> ? a type that is the signed or unsigned type corresponding to a >>> cv-qualified version of the dynamic type of the object, >>> >>> ? an aggregate or union type that includes one of the aforementioned >>> types among its elements or non- static data members (including, >>> recursively, an element or non-static data member of a subaggregate >>> or contained union), >>> >>> ? a type that is a (possibly cv-qualified) base class type of the >>> dynamic type of the object, >>> >>> ? a char or unsigned char type. >>> ------------------------------------------------------------------------ >>> >>> You only have permission to convert pointers to intptr_t and back: you >>> do not have permission to access the stored value of a pointer an an >>> intptr_t. >> I would say the scenario you describe goes under "the dynamic type of >> the object" or "a type that is the signed or unsigned type corresponding >> to the dynamic type of the object", > OK. > >> in the quoted section 3.10 of the standard, depending on specific >> use case. The problem that type aliasing is aimed at is if you >> store an A* and then load it as a B*, then the dynamic type is A*, >> yet it is loaded as B*, where B is not compatible with A. > Precisely. That is what is happening in this case. A is, say, void* > and B is intptr_t. void* is not compatible with intptr_t. My interpretation is that the aliasing rules are for points-to analysis being able to alias that if somebody stores A* and then other code loads that as B* and accesses B, then it is assumed that the B* does not points-to the A* as they are of incompatible types, and that therefore it is fine to load something (that was stored as A*) as intptr_t and subsequently cast it to A* before the A itself is being accessed. Am I wrong? For example, the following test program compiles and runs with g++ -fstrict-aliasing -Wstrict-aliasing=3 -O3 -std=c++03: #include #include class A{ public: int _x; A() : _x(0) {} }; int main(int argc, char* argv[]) { A a; A b; A* ptr = &a; A** ptr_ptr = &ptr; intptr_t* iptr_ptr = reinterpret_cast(ptr_ptr); *ptr_ptr = &b; intptr_t iptr = *iptr_ptr; A* ptr2 = reinterpret_cast(iptr); printf("iptr = %lx, &a = %lx, &b = %lx, iptr->_x = %d\n", iptr, reinterpret_cast(&a), reinterpret_cast(&b), ptr2->_x); return 0; } The program stores an A*, reads it as intptr_t and casts it to A*, and then dereferences into the A. Even with -Wstrict-aliasing=3 GCC does not complain about this. Is GCC wrong about not complaining about this? The way I interpret the standard, intptr_t is the signed type corresponding to the dynamic type of A*, which seems compliant to me. Of course the way it is stated in the standard is a bit vague (as usual), but the compilers seem to support my interpretation. Is my interpretation and GCC -Wstrict-aliasing=3 wrong here in allowing this? >> This is invariant of whether the value of the store is indirectly >> performed through intptr_t or loaded indirectly through intptr_t, as >> the dynamic type of the stored value is still A*. > Exactly so. The stored object is an A*. We're accessing it as an > intptr_t. A* and intptr_t are not compatible types. This is true > even if the intptr_t was the result of casting from an A*. The > compiler doesn't necessarily know that the intptr_t was originally an > A*: they might even be in separate compilation units. The compiler is > entitled to assume that the object in memory of type A* does not alias > with any value of type intptr_t. > >> If you now perform a store of A* through Atomic that casts the pointer >> to intptr_t, and stores it, then the dynamic type is still A*. > This isn't how it works. (BTW, in case it helps understand where I'm > coming from, although I am not the greatest expert in this area, I am > familiar with GCC's alias analysis because I am a GCC author.) > >> In summary, my point is that as long as the performed stores and loads >> in Atomic preserves the bit representation of whatever pointer was >> passed in, the dynamic type of that pointer is unchanged, invariantly of >> casting it to/from intptr_t, and hence the aliasing is allowed. > That is incorrect. Rules about aliasing are nothing to do with the > bit representation. What I meant is that if the bit representation changed, it would point to something different, hence the dynamic type of whatever is pointed at would not (necessarily) be the same. > Let me reiterate: you may cast from any pointer type to intptr_t. You > may not access a pointer in memory as though it were an intptr_t. > Are you 100% sure about this? Again, I do not know where in the standard this is stated, but -Wstrict-aliasing=3 allows me to do that as long as the intptr_t is appropriately casted to the A* before the A behind the dynamic A* type is used. Perhaps I have missed something? Thanks, /Erik From tobias.hartmann at oracle.com Tue Jul 18 14:07:44 2017 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Tue, 18 Jul 2017 16:07:44 +0200 Subject: [10] RFR(XS): 8184009: Missing null pointer check in InterpreterRuntime::update_mdp_for_ret() In-Reply-To: <2d498b3a-8158-2925-309e-e884d2739a1b@oracle.com> References: <2d498b3a-8158-2925-309e-e884d2739a1b@oracle.com> Message-ID: Thanks Coleen! Best regards, Tobias On 18.07.2017 15:45, coleen.phillimore at oracle.com wrote: > Looks good. > Coleen > > On 7/18/17 5:12 AM, Tobias Hartmann wrote: >> Hi, >> >> please review the following patch that adds a missing null check (guarantee) found by Parfait: >> https://bugs.openjdk.java.net/browse/JDK-8184009 >> http://cr.openjdk.java.net/~thartmann/8184009/webrev.00/ >> >> Tested with JPRT and RBT (running). >> >> Thanks, >> Tobias > From goetz.lindenmaier at sap.com Tue Jul 18 14:07:58 2017 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 18 Jul 2017 14:07:58 +0000 Subject: RFR(XS) [10]: 8184672: [ppc64] The fix for '8174749: Use hash table/oops for MemberName table' is broken on ppc64 In-Reply-To: References: Message-ID: Hi Volker, yeah, this fixes the overwriting of the method register. Looks good. Best regards, Goetz. > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On > Behalf Of Volker Simonis > Sent: Freitag, 14. Juli 2017 17:42 > To: HotSpot Open Source Developers > Subject: RFR(XS) [10]: 8184672: [ppc64] The fix for '8174749: Use hash > table/oops for MemberName table' is broken on ppc64 > > Hi, > > can somebody please review the following small, ppc64-only change: > > http://cr.openjdk.java.net/~simonis/webrevs/2017/8184672 > https://bugs.openjdk.java.net/browse/JDK-8184672 > > The change for "JDK-8174749: Use hash table/oops for MemberName table" > introduced a new indirection for the 'vmtarget' field in > java.lang.invoke.MemberName. 'vmtarget' was previously directly stored > in MemberName and is now a field of the new class ResolvedMethodName, > an object of which is now stored in MemberName. > > The change contained some architecture-specific code which updated the > HotSpot to be aware of the new indirection (i.e. instead of loading > 'vmtarget' directly from a MemberName object the code now has to first > load the ResolvedMethodName object from MemeberName and > subsequently > the 'vmtarget' field from the ResolvedMethodName object. > > This code is trivial, but on ppc64, the new version has incorrectly > 'misused' the register which contained the MemberName, as storage for > the intermediate ResolvedMethodName object. If the MemberName object > was stored in an argument register, that register was overwritten in > MethodHandles::generate_method_handle_dispatch (when generating the > intrinsics for MethodHandle.linkToStatic or > MethodHandle.linkToSpecial) and could lead to the following error down > the call chain: > > # Internal Error > (jdk10-hs/hotspot/src/cpu/ppc/vm/macroAssembler_ppc.cpp:5493), > pid=20194, tid=20202 > # guarantee(false) failed: PPC assembly code requires stop: MemberName > required for invokeVirtual etc. > > V [libjvm.so+0x14a0600] stop_on_request(int, char const*)+0x7c > J 110 > java.lang.invoke.MethodHandle.linkToStatic(Ljava/lang/Object;IILjava/lang/i > nvoke/MemberName;)Ljava/lang/Object; > java.base at 10-internal (0 bytes) @ 0x00000fff852d9774 > [0x00000fff852d9700+0x0000000000000074] > J 309 c2 > java.lang.invoke.LambdaForm$MH.invoke(Ljava/lang/Object;J)Ljava/lang/O > bject; > java.base at 10-internal (161 bytes) @ 0x00000fff852fea98 > [0x00000fff852fe900+0x0000000000000198] > J 308 c2 > java.lang.invoke.LambdaForm$MH.linkToTargetMethod(JLjava/lang/Object;) > Ljava/lang/Object; > java.base at 10-internal (9 bytes) @ 0x00000fff852fe23c > [0x00000fff852fe200+0x000000000000003c] > j LambdaForm.foo(J)Ljava/lang/String;+86 > j LambdaForm.main([Ljava/lang/String;)V+8 > > The fix for the problem is trivial. Just use the final destination > register for 'vmtarget' also as intermediate storage for the > ResolvedMethodName object (thus leaving the register holding the > MemberName reference untouched). > > Thank you and best regards, > Volker From matthias.baesken at sap.com Tue Jul 18 14:11:27 2017 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 18 Jul 2017 14:11:27 +0000 Subject: JEP [DRAFT]: Container aware Java In-Reply-To: References: Message-ID: Hi Bob, VMWare has the option to install additional tools, in case they are installed you get a number of metrics from the guests. Some info can be found here : https://www.vmware.com/support/developer/guest-sdk/ Running in the guests you would load libvmGuestLib.so / vmGuestLib.dll and then you have access to a number of functions for querying metrics. Of course it is all optional , when you run in a guest without the additional tools/ guestlib installed , you get nothing (or not much at least ). For AIX / IBM PowerVM Virtualization there is something similar available via the perfstat API. https://www.ibm.com/support/knowledgecenter/en/ssw_aix_72/com.ibm.aix.prftools/idprftools_perfstat.htm ( libperfstat.a is the related library ) Best regards, Matthias -----Original Message----- From: Bob Vandette [mailto:bob.vandette at oracle.com] Sent: Dienstag, 18. Juli 2017 15:43 To: Volker Simonis Cc: hotspot-dev Source Developers ; core-libs-dev ; Baesken, Matthias Subject: Re: JEP [DRAFT]: Container aware Java > On Jul 17, 2017, at 11:55 AM, Volker Simonis wrote: > > On Fri, Jul 14, 2017 at 6:54 PM, Bob Vandette wrote: >> >>> On Jul 14, 2017, at 12:15 PM, Volker Simonis wrote: >>> >>> Hi Bob, >>> >>> thanks for starting this JEP. I think it is long overdue and will be >>> very helpful for many people. >>> >> Thanks for taking a look at the JEP. >> >>> That said, I don't see any mentioning of the targeted Java release. As >>> JEPs usually go into the development version first, I suppose the work >>> for this JEP will be done in JDK10. However, taking into account the >>> current release cycles, the usefulness of this JEP for the Java >>> community will significantly depend on the its availability in Java 9 >>> (at least) and Java 8 (hopefully?). Any thoughts on this? >> >> I am certainly going to do my best to try to get this feature into JDK10 and >> I understand your concern about supporting Java 8 and 9. The decision to >> backport this feature will have to be done based on the usual criteria (complexity, >> usefulness, resource availability and compatibility). It may be possible to >> add some subset of this functionality in a backport but we?ll have to wait until >> the design is further along. >> >>> >>> You also mention that this JEP is only about Docker support on >>> Linux/x86_64. I'd argue that we should at least address at least all >>> Docker supported Linux platforms. I'm pretty sure our colleagues from >>> IBM would like to support this on Linux/ppc64 and Linux/s390 as well. >> >> Right now I don?t expect to have to write CPU specific logic. >> Therefore supporting ppc64, s390 or even aarch64 should boil down to testing. >> >>> >>> Moreover, the suggested API and the new JVM_xxxxxx functions should be >>> at least designed in such a way that other virtualisation environment >>> such as VMWare/VirtualBox/Xen/KVM can take advantage of it. I'm also >>> not sure if a completely new API solely for Docker containers is >>> making sense and is worth the effort? I understand that containers and >>> virtualisation are two different things, but they both impose similar >>> restrictions to the VM (i.e. virtualisation may present a huge number >>> of CPUs to the OS/VM which in reality are mapped to a single, physical >>> CPU) so maybe we should design the new API in such a way that it can >>> be used to tackle both problems. >> >> I will try to keep that in mind. I will have to look into the feasibility of accessing >> host configuration from within a container or virtual machine. For containers, >> it seems to me that would break the intent of isolation. > > Did you wanted to say "for virtualisation, it seems to me that would > break the intent of isolation." here? Well the comment applies to both technologies but cgroups/containers were specifically designed to isolate or limit processes from accessing host resources. > > So you're right, that you can not easily query the host/guest > information in a virtualised environment (like for examples VmWare) > because that would break isolation. But as far as I know, every > virtualisation solution has it's own library/API which can be used to > query the host (if there is any) and get information about the host > system and the resources allocated for the guest. We've done some > basic virtualisation detection for common systems like VmWare or Xen > (cc'ed Matthias who may know more details. If you can point me at some API docs, I?d appreciate it. Bob. > >> I know that host information >> does leak into containers through the proc file system but I suspect this was done >> for simplicity and compatibility reasons. >> >>> >>> Finally, you mention in section D. that all the information should be >>> logged the to error crash logs. That's good but maybe you could add >>> that all the detection and all the findings should also be made >>> available through the new Unified JVM Logging facility (JEP 158)? >> >> That?s a good idea. I will add that to the JEP. >> >> Thanks, >> Bob. >> >>> >>> Regards, >>> Volker >>> >>> >>> On Fri, Jul 14, 2017 at 4:22 PM, Bob Vandette wrote: >>>> I?d like to propose the following JEP that will enhance the Java runtime to be more container aware. >>>> This will allow the Java runtime to performance better ergonomics and provide better >>>> execution time statistics when running in a container. >>>> >>>> >>>> JEP Issue: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8182070 >>>> >>>> Here?s a Text dump of the JEP contents for your convenience: >>>> >>>> >>>> Summary >>>> ------- >>>> >>>> Container aware Java runtime >>>> >>>> Goals >>>> ----- >>>> >>>> Enhance the JVM and Core libraries to detect running in a container and adapt to the system resources available to it. This JEP will only support Docker on Linux-x64 although the design should be flexible enough to allow support for other platforms and container technologies. The initial focus will be on Linux low level container technology such as cgroups so that we will be able to easily support other container technologies running on Linux in addition to Docker. >>>> >>>> Non-Goals >>>> --------- >>>> >>>> It is not a goal of this JEP to support any platform other than Docker container technology running on Linux x64. >>>> >>>> Success Metrics >>>> --------------- >>>> >>>> Success will be measured by the improved efficiency of running multiple Java containers on a host system with out of the box options. >>>> >>>> Motivation >>>> ---------- >>>> >>>> Container technology is becoming more and more prevalent in Cloud based applications. This technology provides process isolation and allows the platform vendor to specify limits and alter the behavior of a process running inside a container that the Java runtime is not aware of. This causes the Java runtime to potentially attempt to use more system resources than are available to it causing performance degradation or even termination. >>>> >>>> >>>> Description >>>> ----------- >>>> >>>> This enhancement will be made up of the following work items: >>>> >>>> A. Detecting if Java is running in a container. >>>> >>>> The Java runtime, as well as any tests that we might write for this feature, will need to be able to detect that the current Java process is running in a container. I propose that we add a new JVM_ native function that returns a boolean true value if we are running inside a container. >>>> >>>> JVM_InContainer will return true if the currently running process is running in a container, otherwise false will be returned. >>>> >>>> B. Exposing container resource limits and configuration. >>>> >>>> There are several configuration options and limits that can be imposed upon a running container. Not all of these >>>> are important to a running Java process. We clearly want to be able to detect how many CPUs have been allocated to our process along with the maximum amount of memory that we be allocated but there are other options that we might want to base runtime decisions on. >>>> >>>> In addition, since Container typically impose limits on system resources, they also provide the ability to easily access the amount of consumption of these resources. I intent on providing this information in addition to the configuration data. >>>> >>>> I propose adding a new jdk.internal.Platform class that will allow access to this information. Since some of this information is needed during the startup of the VM, I propose that much of the implementation of the methods in the Platform class be done in the VM and exposed as JVM_xxxxxx functions. In hotspot, the JVM_xxxxxx function will be implemented via the os.hpp interface. >>>> >>>> Here are the categories of configuration and consumption statistics that will be made available (The exact API is TBD): >>>> >>>> isContainerized >>>> Memory Limit >>>> Total Memory Limit >>>> Soft Memory Limit >>>> Max Memory Usage >>>> Current Memory Usage >>>> Maximum Kernel Memory >>>> CPU Shares >>>> CPU Period >>>> CPU Quote >>>> Number of CPUs >>>> CPU Sets >>>> CPU Set Memory Nodes >>>> CPU Usage >>>> CPU Usage Per CPU >>>> Block I/O Weight >>>> Block I/O Device Weight >>>> Device I/O Read Rate >>>> Device I/O Write Rate >>>> OOM Kill Enabled >>>> OOM Score Adjustment >>>> Memory Swappiness >>>> Shared Memory Size >>>> >>>> TODO: >>>> 1. Need to specify the exact arguments and return format for these accessor functions. >>>> >>>> C. Adjusting Java runtime configuration based on limits. >>>> >>>> Java startup normally queries the operating system in order to setup runtime defaults for things such as the number of GC threads and default memory limits. When running in a container, the operating system functions used provide information about the host and does not include the containers configuration and limits. The VM and core libraries will be modified as part of this JEP to first determine if the current running process is running in a container. It will then cause the runtime to use the container values rather than the general operating system functions for configuring and managing the Java process. There have been a few attempts to correct some of these issue in the VM but they are not complete. The CPU detection in the VM currently only handles a container that limits cpu usage via CPU sets. If the Docker --cpu or --cpu-period along with --cpu-quota options are specified, it currently has no effect on the VMs configuration. >>>> >>>> The experimental memory detection that has been implemented only impacts the Heap selection and does not apply to the os::physical_memory or os::available_memory low level functions. This leaves other parts of the VM and >>>> core libraries to believe there is more memory available than there actually is. >>>> >>>> The Numa support available in the VM is also not correct when running in a container. The number of available memory nodes and enabled nodes as reported by the libnuma library does not take into account the impact of the Docker --cpuset-mems option which restricts which memory nodes the container can use. Inside the container, the file /proc/{pid}/self does report the correct Cpus_allowed and Mems_Allowed but libnuma doesn't respect this. This has been verified via the numactl utility. >>>> >>>> To correct these shortcomings and make this support more robust, here's a list of the current cgroup subsystems that we be examined in order to update the internal VM and core library configuration. >>>> >>>> **Number of CPUs** >>>> >>>> Use a combination of number_of_cpus() and cpu_sets() in order to determine how many processors are available to the process and adjust the JVMs os::active_processor_count appropriately. The number_of_cpus() will be calculated based on the cpu_quota() and cpu_period() using this formula: number_of_cpus() = cpu_quota() / cpu_period(). Since it's not currently possible to understand the relative weight of the running container against all other containers, altering the cpu_shares of a running container will have no affect on altering Java's configuration. >>>> >>>> **Total available memory** >>>> >>>> Use the memory_limit() value from the cgroup file system to initialize the os::physical_memory() value in the VM. This value will propagate to all other parts of the Java runtime. >>>> >>>> We might also consider examining the soft_memory_limit and total_memory_limit in addition to the memory_limit during the ergonomics startup processing in order to fine tuning some of the other VM settings. >>>> >>>> **CPU Memory Nodes** >>>> >>>> Use cpu_set_memory_nodes() to configure the os::numa support. >>>> >>>> **Memory usage** >>>> >>>> Use memory_usage_in_bytes() for providing os::available_memory() by subtracting the usage from the total available memory allocated to the container. >>>> >>>> D. Adding container configuration to error crash logs. >>>> >>>> As as troubleshooting aid, we will dump any available container statistics to the hotspot error log. >>>> >>>> E. Adding a startup flag to enable/disable this support. >>>> >>>> Add a -XX:+UseContainerSupport VM option that will be used to enable this support. The default will be off until this feature is proven. >>>> >>>> Alternatives >>>> ------------ >>>> >>>> There are a few existing RFE's filed that could be used to enhance the current experimental implementation rather than taking the JEP route. >>>> >>>> Testing >>>> ------- >>>> >>>> Docker/container specific tests should be added in order to validate the functionality being provided with this JEP. >>>> >>>> Risks and Assumptions >>>> --------------------- >>>> >>>> Docker is currently based on cgroups v1. Cgroups v2 is also available but is incomplete and not yet supported by Docker. It's possible that v2 could replace v1 in an incompatible way rendering this work unusable until it is upgraded. >>>> >>>> Dependencies >>>> ----------- >>>> >>>> None at this time. >>>> >> From volker.simonis at gmail.com Tue Jul 18 14:12:17 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 18 Jul 2017 16:12:17 +0200 Subject: RFR: compile-time guard some UseSharedSpaces-only coding with the INCLUDE_CDS macro - was :RE: jdk10 : UseSharedSpaces flag and INCLUDE_CDS macro In-Reply-To: <34d6ade2d4c2428bb4aae85d9527eb65@sap.com> References: <4d2f3964b0924ffa897074759e508653@sap.com> <6f85d82587e349579cc7d5be62914ab5@sap.com> <34d6ade2d4c2428bb4aae85d9527eb65@sap.com> Message-ID: Hi Matthias, thanks for doing this change. It looks good! Can somebody from Oracle (maybe you Jiangli) please sponsor it? Unfortunately I still can't push it because it is in shared code :( Thank you and best regards, Volker On Tue, Jul 18, 2017 at 10:35 AM, Baesken, Matthias wrote: > Thanks ! > > -----Original Message----- > From: Aleksey Shipilev [mailto:shade at redhat.com] > Sent: Dienstag, 18. Juli 2017 09:31 > To: Baesken, Matthias ; Jiangli Zhou > Cc: build-dev at openjdk.java.net; hotspot-dev at openjdk.java.net > Subject: Re: RFR: compile-time guard some UseSharedSpaces-only coding with the INCLUDE_CDS macro - was :RE: jdk10 : UseSharedSpaces flag and INCLUDE_CDS macro > > On 07/18/2017 09:24 AM, Baesken, Matthias wrote: >> I created a bug : >> https://bugs.openjdk.java.net/browse/JDK-8184323 >> >> >> and a webrev : >> http://cr.openjdk.java.net/~mbaesken/webrevs/8184323/ > > This looks good. > > -Aleksey > From volker.simonis at gmail.com Tue Jul 18 14:39:36 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 18 Jul 2017 16:39:36 +0200 Subject: RFR(XS) [10]: 8184672: [ppc64] The fix for '8174749: Use hash table/oops for MemberName table' is broken on ppc64 In-Reply-To: References: Message-ID: Thanks Goetz! I've just pushed the change, Volker On Tue, Jul 18, 2017 at 4:07 PM, Lindenmaier, Goetz wrote: > Hi Volker, > > yeah, this fixes the overwriting of the method register. > Looks good. > > Best regards, > Goetz. > >> -----Original Message----- >> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On >> Behalf Of Volker Simonis >> Sent: Freitag, 14. Juli 2017 17:42 >> To: HotSpot Open Source Developers >> Subject: RFR(XS) [10]: 8184672: [ppc64] The fix for '8174749: Use hash >> table/oops for MemberName table' is broken on ppc64 >> >> Hi, >> >> can somebody please review the following small, ppc64-only change: >> >> http://cr.openjdk.java.net/~simonis/webrevs/2017/8184672 >> https://bugs.openjdk.java.net/browse/JDK-8184672 >> >> The change for "JDK-8174749: Use hash table/oops for MemberName table" >> introduced a new indirection for the 'vmtarget' field in >> java.lang.invoke.MemberName. 'vmtarget' was previously directly stored >> in MemberName and is now a field of the new class ResolvedMethodName, >> an object of which is now stored in MemberName. >> >> The change contained some architecture-specific code which updated the >> HotSpot to be aware of the new indirection (i.e. instead of loading >> 'vmtarget' directly from a MemberName object the code now has to first >> load the ResolvedMethodName object from MemeberName and >> subsequently >> the 'vmtarget' field from the ResolvedMethodName object. >> >> This code is trivial, but on ppc64, the new version has incorrectly >> 'misused' the register which contained the MemberName, as storage for >> the intermediate ResolvedMethodName object. If the MemberName object >> was stored in an argument register, that register was overwritten in >> MethodHandles::generate_method_handle_dispatch (when generating the >> intrinsics for MethodHandle.linkToStatic or >> MethodHandle.linkToSpecial) and could lead to the following error down >> the call chain: >> >> # Internal Error >> (jdk10-hs/hotspot/src/cpu/ppc/vm/macroAssembler_ppc.cpp:5493), >> pid=20194, tid=20202 >> # guarantee(false) failed: PPC assembly code requires stop: MemberName >> required for invokeVirtual etc. >> >> V [libjvm.so+0x14a0600] stop_on_request(int, char const*)+0x7c >> J 110 >> java.lang.invoke.MethodHandle.linkToStatic(Ljava/lang/Object;IILjava/lang/i >> nvoke/MemberName;)Ljava/lang/Object; >> java.base at 10-internal (0 bytes) @ 0x00000fff852d9774 >> [0x00000fff852d9700+0x0000000000000074] >> J 309 c2 >> java.lang.invoke.LambdaForm$MH.invoke(Ljava/lang/Object;J)Ljava/lang/O >> bject; >> java.base at 10-internal (161 bytes) @ 0x00000fff852fea98 >> [0x00000fff852fe900+0x0000000000000198] >> J 308 c2 >> java.lang.invoke.LambdaForm$MH.linkToTargetMethod(JLjava/lang/Object;) >> Ljava/lang/Object; >> java.base at 10-internal (9 bytes) @ 0x00000fff852fe23c >> [0x00000fff852fe200+0x000000000000003c] >> j LambdaForm.foo(J)Ljava/lang/String;+86 >> j LambdaForm.main([Ljava/lang/String;)V+8 >> >> The fix for the problem is trivial. Just use the final destination >> register for 'vmtarget' also as intermediate storage for the >> ResolvedMethodName object (thus leaving the register holding the >> MemberName reference untouched). >> >> Thank you and best regards, >> Volker From aph at redhat.com Tue Jul 18 14:41:14 2017 From: aph at redhat.com (Andrew Haley) Date: Tue, 18 Jul 2017 15:41:14 +0100 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: <596E1142.6040708@oracle.com> References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <596CD126.6080100@oracle.com> <596DD713.6080806@oracle.com> <77d5a9e7-b2b0-79e4-effd-903dfe226512@redhat.com> <596E1142.6040708@oracle.com> Message-ID: <3432d11c-5904-a446-1cb5-419627f1794f@redhat.com> On 18/07/17 14:46, Erik ?sterlund wrote: > > > On 2017-07-18 11:57, Andrew Haley wrote: >> On 18/07/17 10:38, Erik ?sterlund wrote: >> >>>> ------------------------------------------------------------------------ >>>> 3.10, Lvalues and rvalues >>>> >>>> If a program attempts to access the stored value of an object through >>>> a glvalue of other than one of the following types the behavior is >>>> undefined: >>>> >>>> ? the dynamic type of the object, >>>> >>>> ? a cv-qualified version of the dynamic type of the object, >>>> >>>> ? a type similar (as defined in 4.4) to the dynamic type of the object, >>>> >>>> ? a type that is the signed or unsigned type corresponding to the >>>> dynamic type of the object, >>>> >>>> ? a type that is the signed or unsigned type corresponding to a >>>> cv-qualified version of the dynamic type of the object, >>>> >>>> ? an aggregate or union type that includes one of the aforementioned >>>> types among its elements or non- static data members (including, >>>> recursively, an element or non-static data member of a subaggregate >>>> or contained union), >>>> >>>> ? a type that is a (possibly cv-qualified) base class type of the >>>> dynamic type of the object, >>>> >>>> ? a char or unsigned char type. >>>> ------------------------------------------------------------------------ >>>> >>>> You only have permission to convert pointers to intptr_t and back: you >>>> do not have permission to access the stored value of a pointer an an >>>> intptr_t. >>> I would say the scenario you describe goes under "the dynamic type of >>> the object" or "a type that is the signed or unsigned type corresponding >>> to the dynamic type of the object", >> OK. >> >>> in the quoted section 3.10 of the standard, depending on specific >>> use case. The problem that type aliasing is aimed at is if you >>> store an A* and then load it as a B*, then the dynamic type is A*, >>> yet it is loaded as B*, where B is not compatible with A. >> Precisely. That is what is happening in this case. A is, say, void* >> and B is intptr_t. void* is not compatible with intptr_t. > > My interpretation is that the aliasing rules are for points-to analysis > being able to alias that if somebody stores A* and then other code loads > that as B* and accesses B, then it is assumed that the B* does not > points-to the A* as they are of incompatible types, It is more general than that. If you access the stored value of an object through a glvalue of other than one of the allowed types, then your *whole program is undefined*. > and that therefore it is fine to load something (that was stored as > A*) as intptr_t and subsequently cast it to A* before the A itself > is being accessed. Am I wrong? No, that is correct. The question is whether intptr_t is a type compatible with a pointer type, and I don't think you will find any language to the effect that it is. Pointer types are distinct from one another and from all integer types. People sometimes get confused by this: the fact that you can cast from A* to e.g. void*, doesn't tell you that you can cast from A** to void** and use the result: they are different types. > For example, the following test program compiles and runs with g++ > -fstrict-aliasing -Wstrict-aliasing=3 -O3 -std=c++03: > > #include > #include > > class A{ > public: > int _x; > A() : _x(0) {} > }; > > int main(int argc, char* argv[]) { > A a; > A b; > A* ptr = &a; > A** ptr_ptr = &ptr; > intptr_t* iptr_ptr = reinterpret_cast(ptr_ptr); > > *ptr_ptr = &b; > intptr_t iptr = *iptr_ptr; > A* ptr2 = reinterpret_cast(iptr); > > printf("iptr = %lx, &a = %lx, &b = %lx, iptr->_x = %d\n", iptr, > reinterpret_cast(&a), > reinterpret_cast(&b), ptr2->_x); > > return 0; > } > > The program stores an A*, reads it as intptr_t and casts it to A*, and > then dereferences into the A. Even with -Wstrict-aliasing=3 GCC does not > complain about this. Is GCC wrong about not complaining about this? No GCC is not wrong because GCC does not have to complain about this. You are, however, wrong to write it! Try g++ -fsanitize=undefined pp.cc and you'll get iptr = 7ffffcf68ec0, &a = 7ffffcf68ed0, &b = 7ffffcf68ec0, iptr->_x = 0 at one point of undefined behaviour. [ A warning here: don't assume that the sanitizer can detect all UB caused by messing with pointer types, but it can detect this example. ] > The way I interpret the standard, intptr_t is the signed type > corresponding to the dynamic type of A*, which seems compliant to me. But that's not what the standard says. Remember that whatever is allowed is explicitly allowed, and if something is not allowed it is forbidden. > Of course the way it is stated in the standard is a bit vague (as > usual), but the compilers seem to support my interpretation. Is my > interpretation and GCC -Wstrict-aliasing=3 wrong here in allowing > this? Your interpretation is wrong. But a C++ compiler can do anything with undefined behaviour, which includes the possibility that it does what the programmer expected. >> Let me reiterate: you may cast from any pointer type to intptr_t. You >> may not access a pointer in memory as though it were an intptr_t. > > Are you 100% sure about this? Again, I do not know where in the standard > this is stated, It's in my quote above. > but -Wstrict-aliasing=3 allows me to do that as long as the intptr_t > is appropriately casted to the A* before the A behind the dynamic A* > type is used. Perhaps I have missed something? Yes, I am sure. Warnings don't help you here because if you throw in enough casts you can suppress any type-based warnings. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From coleen.phillimore at oracle.com Tue Jul 18 14:56:35 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 18 Jul 2017 10:56:35 -0400 Subject: RFR (L) 7133093: Improve system dictionary performance In-Reply-To: <7814C032-0D4E-4E76-B8AD-62A072E8A625@oracle.com> References: <89f0b98c-3cbd-7b87-d76b-5e89a5a676fb@oracle.com> <9243da7c-a8e5-4024-c12e-cba9d4f424c8@oracle.com> <7814C032-0D4E-4E76-B8AD-62A072E8A625@oracle.com> Message-ID: <233c03e0-8958-7676-8e33-5d8f0ec4edf3@oracle.com> Karen, Thank you for reviewing the code and for the discussions and consulting along the way. On 7/17/17 4:10 PM, Karen Kinnear wrote: > Feedback on the initial web rev: > Thank you for using web rev - that way I can do searches of the > overall patch! > > First - many thanks for doing this exercise so carefully, for > following the specification in continuing > to record the initiating loader and for splitting this effort into > multiple phases to allow further exploration > of the current expectations/behavior relative to class unloading. > > And thank you for all the testing. > > Just wanted to ensure that I understand some of the assumptions: > > 1) lock free reads of dictionaries - yes Yes. > 2) lock free adds of entries to dictionaries - yes No, adds still take out the SystemDictionary_lock. > 3) removal of entries or entire dictionaries - at safe point only Yes. > 4) rehashing - none today and none planned Yes. > 5) resizing of dictionaries - none today, some planned Yes. > 6) use of SystemDictionary_lock: essentially unchanged Yes. > 7) atomicity guarantees relative to checking loader constraints and > updates to the dictionary: unchanged Yes. > > I did not review the SA/vmStructs - sounds like you had Jini George do > that - great. > > Minor questions/comments: > > SystemDictionary::verify: "Verify of system dictionary failed" -> > looks like "Verify of placeholders failed" Fixed. Good find. It's an odd assert: guarantee(placeholders()->number_of_entries() >= 0, "Verify of placeholders failed"); > > thank you for asserts dictionary != NULL > Dictionary::print > is it worth an assert that loader_data != NULL? Added. It's already asserted in Dictionary::verify() > Can you please sanity check if there are other cases in which we need > to use dictionary_or_null() rather than dictionary()? > - are there cases in which we could be using a dictionary from an > anonymous class? > e.g. there are a lot of places in systemDictionary.cpp where we > call class_loader_data(class_loader); loader_data->dictionary(); > - was the lazily allocated dictionary allocated yet (see next > question) There's an assert in dictionary() that the CLD isn't anonymous. In these calls we get class_loader_data from the class_loader which is always the host class loader for anonymous classes, so this doesn't break with the current design. > > CLD.cpp > Would it be less complex to eagerly create the dictionary for a CLD > since it is extremely rare to not need one for non-anonymous CLDs? > You could create before publishing the CLD Yes, I agree it would be better this way. I was looking at some data and it does seem rare to have a CLD with no classes. > > Terminology: > How are you still using System Dictionary term? > It still shows up in some comments and in some printed strings Bad habit, we've been calling it that for years. :) I'll update the comments where it makes sense. I tried to refer to the ClassLoaderDataGraph dictionaries collectively as the "System Dictionary", as well as the loader constraints and placeholder tables, and the other things that SystemDictionary contains. I guess it gets confusing, but it's less awkward than "class loader data dictionaries", and more meaningful than simply "dictionaries". So that's why I didn't change a lot of the comments. > > biasedLocking.cpp, line 59 > jvmtiGetLoadedClasses: line 298 > dictionary.hpp lines 113,114, 120 > dictionary.cpp 262-263, 478 > dictionary.cpp 491 > fix comments e.g. "Iterate the system dictionary" ... > - probably worth checking for other comments in the sources too > unless these all appear to match new usage model Fixed. I've found a couple more and some other comments that are incorrect. > > hashTable.hpp > line 268: what does this comment mean? > "Don't create and use freelist"? > I assume this means that this is a direct malloc, not using any block > allocation, e.g. metaspace chunks, and this requires explicit freeing? > // Don't create and use freelist of HashtableEntry. The default Hashtable<> allocates entries from a block of entries and uses a freelist for freed entries. > > dictionary.hpp line 161 > could you possibly clean up comment while you are in here? > I presume "a protection" is "a protection domain"? Fixed. > > dictionary.cpp > why change from Hashtable::free_entry to explicit > unlink_entry/FREE_C_HEAP_ARRAY? I used the same code as the ModuleTable. The default hashtable doesn't delete the whole thing and left some memory around, if I remember correctly. I'll recheck it. > > Future optimization: > I think that none of the built-in loaders is ever recorded as an > initiating > loader for any class in a loader which permits unloading. > Note: we need to ensure that built-in includes boot, platform and > application class loader (and clean up any confusion with system > class loader - which can be set by the user to a custom class loader) > I think AppClassLoader can initiate loading and is a builtin loader. Dictionary for class loader class loader 0x00007f6abc6e3210 a 'jdk/internal/loader/ClassLoaders$AppClassLoader'{0x0000000101c8df00} Java system dictionary (table_size=107, classes=1) ^ indicates that initiating loader is different from defining loader 106: ^java.util.ArrayList, loader NULL class_loader > > Any chance of changing DictionaryEntry* probe->klass() to be > probe->instanceKlass()? Okay. Changed to instance_klass() as per coding standard. > > Thank you for assertion _is_at_safepoint for try_get_next_class > > CLD.cpp > If the names of the classes_do vs. all_entries_do were modified to be > defining_loader_classes_do vs. all_entries_classes_do or some way > to make the distinction in the name that would be helpful. > Otherwise, if you could please add comments to the > CLDG::dictionary_classes_do > etc. to distinguish these. I added comments to distinguish these. > > CLD.cpp > lines 1200-1201 > Can you please fix the comment? > I think you need separate comments for the ModuleEntry > reads/PackageEntry's > exports which are looking for dead modules. > IIUC, the do_unloading is actually looking for classes for which this > is the initating loader, but their defining loader is dead. // Remove entries in the dictionary of live class loader that have // initiated loading classes in a dead class loader. data->dictionary()->do_unloading(); Moved the module comment to before the module code and added this for dictionary->do_unloading() call. > > systemDictionary.cpp: 445-447 > THANK you for the comments about the java_mirror handle keeping the > class > and class loader alive > Could you clarify that the class_loader handle is not for the class > we are checking, but rather for the initiating, i.e. requesting > class loader > (is that accurate?) So we are keeping both the requestor and the > requested class alive Yes, that is true. I added // The class_loader handle passed in is the initiating loader. > > You removed the comment that compute_hash could lead to a safepoint/GC > Is that not possible? > comment in ObjectSynchronizer::FastHashCode says that with > UseBiasedLocking > we can revoke biases and must be able to handle a safepoint. I made it not possible with this change. The hashcode is now computed by the Symbol* of the Klass name, which is random (os::random plus some bits from the address and first two characters, which we did a study of for bug https://bugs.openjdk.java.net/browse/JDK-8181450). Also, I changed ClassLoaderData::identity_hash() to use the address >> 3 because the addresses of class loader datas are random enough for the one use of this function. Previously, the loader_data identity hash used to go to the class_loader oop and get to ObjectSynchronizer::FastHashCode (which was also os::random but now changed). The dictionary hashing doesn't need to add in the hash for the loader_data now. > > systemDictionary.cpp: > NoSafepointVerifier > If we add resizing logic later, and that happens at a safepoint, > then the number of buckets change and the d_index is no longer valid > so we would need to add back the NSpV? > - this is a more general problem if compute_hash can lead to a > safepoint/GC Yes this is true, except that compute_hash doesn't safepoint. Computing the hash and index should be done together with the dictionary operations. The function resolve_instance_class_or_null computes the hash and index once at the beginning and reuses it for several lookups, which may or may not affect performance. So we need to measure this in isolation before changing this. > > systemDictionary.cpp: parse_stream > why did you remove the record_dependency? I realized it's trivially unnecessary. The class_loader is reached by the ClassLoaderData::_class_loader field, so we don't have to record a dependency from host_klass->class_loader in the anonymous class loader data. (I just checked this again because it's confusing). > > Who calls Dictionary::~Dictionary? The ClassLoaderData::~ClassLoaderData() calls delete _dictionary, which calls the destructor. > > Performance: > Do you have performance runs of our existing benchmarks? Yes, I ran our internal benchmarks and added links to the internal wiki page. I'll rerun them again before checkin. > Initial key requirement is that this not slow down throughput through > slowing down lookups. > I know you are very aware of longer-term goals for reducing > safepoint pause > times and not increasing footprint significantly. > > hashtable sizes: future rfe? > given that we now have a lot more dictionaries, and they perhaps need > individual sizing > - would it be helpful to add statistics information about > - e.g. lookup_count, lookup_length, _lookup_warning > - i.e. number of entries, max lookup depth, etc. to use later for > tuning? All the previous statistics didn't make very much sense to us, so they were removed. There is now logging for all the hashtables: busaa027.us.oracle.com% java -Xlog:hashtables -version java version "10-internal" Java(TM) SE Runtime Environment (fastdebug build 10-internal+0-2017-07-17-1456035.cphillim.10system-dictionary2) Java HotSpot(TM) 64-Bit Server VM (fastdebug build 10-internal+0-2017-07-17-1456035.cphillim.10system-dictionary2, mixed mode) [0.284s][info][hashtables] System Dictionary for jdk/internal/loader/ClassLoaders$AppClassLoader max bucket size 1 bucket 91 element count 1 table size 107 [0.285s][info][hashtables] System Dictionary for max bucket size 4 bucket 263 element count 468 table size 1009 [0.285s][info][hashtables] Placeholder Table max bucket size 0 bucket 0 element count 0 table size 1009 [0.285s][info][hashtables] Protection Domain Table max bucket size 0 bucket 0 element count 0 table size 1009 > - might make sense to make this JFR data? I'll file an RFE to add tracing data with our tools. I'll need some help with this part but the data is there. > > Might think about a follow-up with more tuning parameters than > a single global PredictedLoadedClassCount. > I filed RFE to work on resizing, as this work was almost complete, but didn't show performance improvements at the time. It would be nice to have customers not need tuning parameters, but I guess these things are sometimes necessary. Thank you for the thorough review and all of your help with this! I've made these changes and will rerun testing and send out a new webrev and an incremental webrev. Thanks! Coleen > > thanks, > Karen > > >> On Jul 17, 2017, at 2:37 PM, coleen.phillimore at oracle.com >> wrote: >> >> >> Hi, I have made changes for Ioi's comments below. The largest change >> since .01 is that PredictedLoadedClassCount is used to size >> dictionaries for class loader data, and should be used to specify the >> average number of classes loaded by each class loader. Since this is >> an option available via UnlockExperimentalVMOptions, I'm not sure >> what the process is for changing the meaning of this option. We will >> work with our customers who have used this option for specifying the >> total number of classes loaded. We are also working on the change >> to allow automatic resizing of the dictionaries during safepoints. >> >> See: >> open webrev at http://cr.openjdk.java.net/~coleenp/7133093.02/webrev >> >> >> Tested with the full nightly test set (jdk, nsk, hotspot jtreg tests) >> on linux x64. >> >> Thanks, >> Coleen >> >> On 6/30/17 4:45 PM, coleen.phillimore at oracle.com >> wrote: >>> >>> Ioi, Thank you for looking at this change. >>> >>> On 6/30/17 2:39 PM, Ioi Lam wrote: >>>> Hi Coleen, >>>> >>>> Maybe the bug should be renamed to "Use one Dictionary per class >>>> loader instance"? That way it's more obvious what it is when you >>>> look at the repo history. >>> >>> I can do that. I made it One Dictionary per ClassLoaderData >>>> >>>> 1. >>>> >>>> Why is assert_locked_or_safepoint(SystemDictionary_lock) necessary >>>> in SystemDictionary::find_class (line 1826), but not necessary >>>> SystemDictionary::find (line 951)? Since you removed >>>> NoSafepointVerifier nosafepoint in the latter, maybe this means >>>> it's safe to remove the assert_locked_or_safepoint in the former? >>> >>> The call to SystemDictionary::find() is the (I believe) usual lock >>> free lookup and the SystemDictionary::find_class() is used to verify >>> that a class we're about to add or want to add hasn't been added by >>> another thread. Or certain cases where we already have a lock to >>> do something else, like add loader constraints. I took out the >>> NoSafepointVerifier because it assumes that system dictionary >>> entries would be moved by GC, which they won't. The old hash >>> function could safepoint when getting the hash for the class_loader >>> object. >>> >>>> >>>> 2. >>>> >>>> 455 static ClassLoaderData* _current_loader_data = NULL; >>>> 456 static Klass* _current_class_entry = NULL; >>>> 457 >>>> 458 InstanceKlass* ClassLoaderDataGraph::try_get_next_class() { >>>> >>>> How about moving the static fields into an iterator object. That >>>> way you don't need to keep track of the globals >>>> >>>> ClassLoaderDataGraphIterator { >>>> ClassLoaderData* _current_loader_data >>>> Klass* _current_class_entry; >>>> >>>> InstanceKlass* try_get_next_class() { ....} >>>> }; >>> >>> Ok, there's a different iterator that iterates over all of the >>> classes for GC. I will adapt that for this use. That would be better. >>> >>>> >>>> 3. Double check locking in ClassLoaderData::dictionary() -- someone >>>> else should look at this :-) >>> >>> I copied code that David Holmes added for modules. >>> >>>> >>>> 4. We may need a better strategy for deciding the size of each >>>> dictionary. >>>> >>>> 565 const int _primelist[10] = {1, 107, 1009}; >>>> 571 Dictionary* ClassLoaderData::dictionary() { >>>> 579 if ((dictionary = _dictionary) == NULL) { >>>> 580 int size; >>>> 581 if (this == the_null_class_loader_data() || >>>> is_system_class_loader_data()) { >>>> 582 size = _primelist[2]; >>>> 583 } else if >>>> (class_loader()->is_a(SystemDictionary::reflect_DelegatingClassLoader_klass())) >>>> { >>>> 584 size = _primelist[0]; >>>> 585 } else { >>>> 586 size = _primelist[1]; >>>> 587 } >>>> 588 dictionary = new Dictionary(this, size); >>>> >>>> I'll do some investigation on this issue and get back to you. >>>> >>> >>> How about if someone uses PredictedLoadedClassCount, then we use >>> that to size all but the reflection and boot class loader? Then if >>> there's an application that has a class loader with a huge amount of >>> classes loaded in it, that would help this? It might cost some >>> footprint but an oversized table would simply be a bigger array of >>> pointers, so it might not be that bad to oversize. >>> >>> I think the long term solution is being able to resize these entries >>> and make the application provide arguments. Please let me know >>> what you find in your investigation and if that would work. >>>> The rest of the changes look good to me. >>> >>> Thank you! >>> Coleen >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>> On 6/23/17 4:42 PM, coleen.phillimore at oracle.com >>>> wrote: >>>>> Summary: Implement one dictionary per ClassLoaderData for faster >>>>> lookup and removal during class unloading >>>>> >>>>> See RFE for more details. >>>>> >>>>> open webrev at >>>>> http://cr.openjdk.java.net/~coleenp/7133093.01/webrev >>>>> >>>>> bug link https://bugs.openjdk.java.net/browse/JDK-7133093 >>>>> >>>>> Tested with full "nightly" run in rbt, plus locally class loading >>>>> and unloading tests: >>>>> >>>>> jtreg hotspot/test/runtime/ClassUnload >>>>> >>>>> jtreg hotspot/test/runtime/modules >>>>> >>>>> jtreg hotspot/test/gc/class_unloading >>>>> >>>>> make test-hotspot-closed-tonga FILTER=quick TEST_JOBS=4 >>>>> TEST=vm.parallel_class_loading >>>>> >>>>> csh ~/testing/run_jck9 (vm/lang/java_lang) >>>>> >>>>> runThese -jck - uses class loader isolation to run each jck test >>>>> and unloads tests when done (at -gc:5 intervals) >>>>> >>>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>> >>>> >>> >> > From vladimir.x.ivanov at oracle.com Tue Jul 18 15:19:00 2017 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 18 Jul 2017 18:19:00 +0300 Subject: [10] RFR: 8184309: Buld warnings from GCC 7.1 on Fedora 26 In-Reply-To: References: <60251551-F66E-4A2D-B354-399C3C3D1A20@oracle.com> Message-ID: Pushed [1]. Best regards, Vladimir Ivanov [1] http://hg.openjdk.java.net/jdk10/hs/hotspot/rev/eb11b3f0ae65 On 7/14/17 6:49 AM, Yasumasa Suenaga wrote: > Updated: > http://cr.openjdk.java.net/~ysuenaga/JDK-8184309/webrev.02/ > > > Yasumasa > > > 2017-07-14 12:11 GMT+09:00 Kim Barrett : >>> On Jul 13, 2017, at 9:57 PM, Yasumasa Suenaga wrote: >>> >>> Thanks Kim, >>> >>> I uploaded new webrev for this issue: >>> >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8184309/webrev.01/ >>> >>> Vladimir, could you push this patch? >> >> Before pushing, please update the copyright in logFileOutput.hpp. >> >>> Yasumasa >> From erik.osterlund at oracle.com Tue Jul 18 15:26:04 2017 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 18 Jul 2017 17:26:04 +0200 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: <3432d11c-5904-a446-1cb5-419627f1794f@redhat.com> References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <596CD126.6080100@oracle.com> <596DD713.6080806@oracle.com> <77d5a9e7-b2b0-79e4-effd-903dfe226512@redhat.com> <596E1142.6040708@oracle.com> <3432d11c-5904-a446-1cb5-419627f1794f@redhat.com> Message-ID: <596E288B.90706@oracle.com> On 2017-07-18 16:41, Andrew Haley wrote: > On 18/07/17 14:46, Erik ?sterlund wrote: >> >> On 2017-07-18 11:57, Andrew Haley wrote: >>> On 18/07/17 10:38, Erik ?sterlund wrote: >>> >>>>> ------------------------------------------------------------------------ >>>>> 3.10, Lvalues and rvalues >>>>> >>>>> If a program attempts to access the stored value of an object through >>>>> a glvalue of other than one of the following types the behavior is >>>>> undefined: >>>>> >>>>> ? the dynamic type of the object, >>>>> >>>>> ? a cv-qualified version of the dynamic type of the object, >>>>> >>>>> ? a type similar (as defined in 4.4) to the dynamic type of the object, >>>>> >>>>> ? a type that is the signed or unsigned type corresponding to the >>>>> dynamic type of the object, >>>>> >>>>> ? a type that is the signed or unsigned type corresponding to a >>>>> cv-qualified version of the dynamic type of the object, >>>>> >>>>> ? an aggregate or union type that includes one of the aforementioned >>>>> types among its elements or non- static data members (including, >>>>> recursively, an element or non-static data member of a subaggregate >>>>> or contained union), >>>>> >>>>> ? a type that is a (possibly cv-qualified) base class type of the >>>>> dynamic type of the object, >>>>> >>>>> ? a char or unsigned char type. >>>>> ------------------------------------------------------------------------ >>>>> >>>>> You only have permission to convert pointers to intptr_t and back: you >>>>> do not have permission to access the stored value of a pointer an an >>>>> intptr_t. >>>> I would say the scenario you describe goes under "the dynamic type of >>>> the object" or "a type that is the signed or unsigned type corresponding >>>> to the dynamic type of the object", >>> OK. >>> >>>> in the quoted section 3.10 of the standard, depending on specific >>>> use case. The problem that type aliasing is aimed at is if you >>>> store an A* and then load it as a B*, then the dynamic type is A*, >>>> yet it is loaded as B*, where B is not compatible with A. >>> Precisely. That is what is happening in this case. A is, say, void* >>> and B is intptr_t. void* is not compatible with intptr_t. >> My interpretation is that the aliasing rules are for points-to analysis >> being able to alias that if somebody stores A* and then other code loads >> that as B* and accesses B, then it is assumed that the B* does not >> points-to the A* as they are of incompatible types, > It is more general than that. If you access the stored value of an > object through a glvalue of other than one of the allowed types, then > your *whole program is undefined*. And I think it is in that list of allowed types. Specifically, it is my interpretation of the standard that intptr_t is "a type that is the signed or unsigned type corresponding to the dynamic type of the object" (cf. 5.2.10) if the dynamic type is e.g. void*. >> and that therefore it is fine to load something (that was stored as >> A*) as intptr_t and subsequently cast it to A* before the A itself >> is being accessed. Am I wrong? > No, that is correct. The question is whether intptr_t is a type > compatible with a pointer type, and I don't think you will find any > language to the effect that it is. I think 5.2.10 describing reinterpret_cast seems to suggest that they are compatible by saying "A pointer converted to an integer of sufficient size (if any such exists on the implementation) and back to the same pointer type will have its original value". That is precisely what is done. Conversely, I do not find any language suggesting that intptr_t would not be compatible with pointer types. > Pointer types are distinct from one another and from all integer > types. People sometimes get confused by this: the fact that you can > cast from A* to e.g. void*, doesn't tell you that you can cast from > A** to void** and use the result: they are different types. Agreed. But I do not see any problem with reinterpret_cast from A* to intptr_t back to A* and using the A from that result. That is explicitly allowed as per 5.2.10 as previously described. >> For example, the following test program compiles and runs with g++ >> -fstrict-aliasing -Wstrict-aliasing=3 -O3 -std=c++03: >> >> #include >> #include >> >> class A{ >> public: >> int _x; >> A() : _x(0) {} >> }; >> >> int main(int argc, char* argv[]) { >> A a; >> A b; >> A* ptr = &a; >> A** ptr_ptr = &ptr; >> intptr_t* iptr_ptr = reinterpret_cast(ptr_ptr); >> >> *ptr_ptr = &b; >> intptr_t iptr = *iptr_ptr; >> A* ptr2 = reinterpret_cast(iptr); >> >> printf("iptr = %lx, &a = %lx, &b = %lx, iptr->_x = %d\n", iptr, >> reinterpret_cast(&a), >> reinterpret_cast(&b), ptr2->_x); >> >> return 0; >> } >> >> The program stores an A*, reads it as intptr_t and casts it to A*, and >> then dereferences into the A. Even with -Wstrict-aliasing=3 GCC does not >> complain about this. Is GCC wrong about not complaining about this? > No GCC is not wrong because GCC does not have to complain about this. > You are, however, wrong to write it! Oh dear. My fingers slipped. > Try > > g++ -fsanitize=undefined pp.cc > > and you'll get > > iptr = 7ffffcf68ec0, &a = 7ffffcf68ed0, &b = 7ffffcf68ec0, iptr->_x = 0 > > at one point of undefined behaviour. > > [ A warning here: don't assume that the sanitizer can detect all UB > caused by messing with pointer types, but it can detect this > example. ] That looks like the correct (and indeed expected) output. iptr should be the same as &b, and iptr->_x should be 0, and &a should be different from &b and iptr. I tried this experiment locally too with similar correct result. Am I missing something here? >> The way I interpret the standard, intptr_t is the signed type >> corresponding to the dynamic type of A*, which seems compliant to me. > But that's not what the standard says. Remember that whatever is > allowed is explicitly allowed, and if something is not allowed it is > forbidden. Agreed. But it is my interpretation is that this is explicitly allowed. In the case where A* is stored, and subsequently loaded as intptr_t (and casted to A*), the dynamic type observed by the load is A* and intptr_t is the signed type of A*. This is explicitly allowed w.r.t. aliasing because the type of the load may be: 3.10: "a type that is the signed or unsigned type corresponding to the dynamic type of the object" ...which it is as intptr_t is the signed type corresponding to A*. This intptr_t is then casted back to the dynamic type A* through reinterpret_cast, which is explicitly allowed because: 5.2.10: "A pointer converted to an integer of sufficient size (if any such exists on the implementation) and back to the same pointer type will have its original value" Thanks, /Erik >> Of course the way it is stated in the standard is a bit vague (as >> usual), but the compilers seem to support my interpretation. Is my >> interpretation and GCC -Wstrict-aliasing=3 wrong here in allowing >> this? > Your interpretation is wrong. But a C++ compiler can do anything with > undefined behaviour, which includes the possibility that it does what > the programmer expected. > >>> Let me reiterate: you may cast from any pointer type to intptr_t. You >>> may not access a pointer in memory as though it were an intptr_t. >> Are you 100% sure about this? Again, I do not know where in the standard >> this is stated, > It's in my quote above. > >> but -Wstrict-aliasing=3 allows me to do that as long as the intptr_t >> is appropriately casted to the A* before the A behind the dynamic A* >> type is used. Perhaps I have missed something? > Yes, I am sure. Warnings don't help you here because if you throw in > enough casts you can suppress any type-based warnings. > From coleen.phillimore at oracle.com Tue Jul 18 16:05:00 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 18 Jul 2017 12:05:00 -0400 Subject: RFR(xs): 8184339: Thread::current_or_null() should not assert if Posix TLS is not yet initialized In-Reply-To: References: Message-ID: Seems good. I'll check it in for you. Coleen On 7/18/17 9:02 AM, Thomas St?fe wrote: > Hi Folks, > > may I please have a second review and a sponsor? > > This is a bit of an annoying issue, it causes endless recursions on AIX > because error reporting stumbles over itself when calling Thread::current(). > > Thanks! ..Thomas > > On Mon, Jul 17, 2017 at 9:00 AM, Thomas St?fe > wrote: > >> Hi all, >> >> may I please have reviews + a sponsor for the following fix: >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8184339 >> webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8184339-Thread- >> current-or-null-shall-not-assert/webrev.00/webrev/ >> >> The problem is caused by the fact that Posix TLS cannot be used before it >> is initialized. It is initialized in os::init(). If we use Posix TLS (eg >> via Thread::current()) before, we assert. It is used now (since >> JDK-8183925 ) before >> os::init() (see callstack in bug description). >> >> There are two functions, Thread::current() and Thread::current_or_null(). >> The latter should not assert if Posix TLS is not yet available but return >> NULL instead. This is what callers expect: this function is safe to call, >> but it might not return a valid Thread*. >> >> Note that this issue currently only comes up at AIX, because AIX is the >> only platform using Posix TLS for Thread::current() - all other platforms >> use Compiler-based TLS (__thread variables). >> >> However, we want to keep the Posix TLS code path alive, so this may also >> affect other platforms. There have been compiler bugs in the past (e.g. gcc >> + glibc) leading to errors when using compiler-based TLS, so it is good to >> have a reliable fallback. >> >> Thanks, >> >> Thomas >> From thomas.stuefe at gmail.com Tue Jul 18 18:45:04 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 18 Jul 2017 20:45:04 +0200 Subject: RFR(xs): 8184339: Thread::current_or_null() should not assert if Posix TLS is not yet initialized In-Reply-To: References: Message-ID: Thank you Coleen! On Tue, Jul 18, 2017 at 6:05 PM, wrote: > Seems good. I'll check it in for you. > Coleen > > On 7/18/17 9:02 AM, Thomas St?fe wrote: > >> Hi Folks, >> >> may I please have a second review and a sponsor? >> >> This is a bit of an annoying issue, it causes endless recursions on AIX >> because error reporting stumbles over itself when calling >> Thread::current(). >> >> Thanks! ..Thomas >> >> On Mon, Jul 17, 2017 at 9:00 AM, Thomas St?fe >> wrote: >> >> Hi all, >>> >>> may I please have reviews + a sponsor for the following fix: >>> >>> bug: https://bugs.openjdk.java.net/browse/JDK-8184339 >>> webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8184339-Thread- >>> current-or-null-shall-not-assert/webrev.00/webrev/ >>> >>> The problem is caused by the fact that Posix TLS cannot be used before it >>> is initialized. It is initialized in os::init(). If we use Posix TLS (eg >>> via Thread::current()) before, we assert. It is used now (since >>> JDK-8183925 ) before >>> os::init() (see callstack in bug description). >>> >>> There are two functions, Thread::current() and Thread::current_or_null(). >>> The latter should not assert if Posix TLS is not yet available but return >>> NULL instead. This is what callers expect: this function is safe to call, >>> but it might not return a valid Thread*. >>> >>> Note that this issue currently only comes up at AIX, because AIX is the >>> only platform using Posix TLS for Thread::current() - all other platforms >>> use Compiler-based TLS (__thread variables). >>> >>> However, we want to keep the Posix TLS code path alive, so this may also >>> affect other platforms. There have been compiler bugs in the past (e.g. >>> gcc >>> + glibc) leading to errors when using compiler-based TLS, so it is good >>> to >>> have a reliable fallback. >>> >>> Thanks, >>> >>> Thomas >>> >>> > From coleen.phillimore at oracle.com Tue Jul 18 19:50:28 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 18 Jul 2017 15:50:28 -0400 Subject: RFR: compile-time guard some UseSharedSpaces-only coding with the INCLUDE_CDS macro - was :RE: jdk10 : UseSharedSpaces flag and INCLUDE_CDS macro In-Reply-To: References: <4d2f3964b0924ffa897074759e508653@sap.com> <6f85d82587e349579cc7d5be62914ab5@sap.com> <34d6ade2d4c2428bb4aae85d9527eb65@sap.com> Message-ID: On 7/18/17 10:12 AM, Volker Simonis wrote: > Hi Matthias, > > thanks for doing this change. It looks good! > > Can somebody from Oracle (maybe you Jiangli) please sponsor it? > Unfortunately I still can't push it because it is in shared code :( I can commit it for you. I was testing out my sponsor csh alias. It didn't patch because of changes to systemDictionary so the changeset has a different number. Coleen > > Thank you and best regards, > Volker > > > On Tue, Jul 18, 2017 at 10:35 AM, Baesken, Matthias > wrote: >> Thanks ! >> >> -----Original Message----- >> From: Aleksey Shipilev [mailto:shade at redhat.com] >> Sent: Dienstag, 18. Juli 2017 09:31 >> To: Baesken, Matthias ; Jiangli Zhou >> Cc: build-dev at openjdk.java.net; hotspot-dev at openjdk.java.net >> Subject: Re: RFR: compile-time guard some UseSharedSpaces-only coding with the INCLUDE_CDS macro - was :RE: jdk10 : UseSharedSpaces flag and INCLUDE_CDS macro >> >> On 07/18/2017 09:24 AM, Baesken, Matthias wrote: >>> I created a bug : >>> https://bugs.openjdk.java.net/browse/JDK-8184323 >>> >>> >>> and a webrev : >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8184323/ >> This looks good. >> >> -Aleksey >> From jiangli.zhou at oracle.com Tue Jul 18 21:53:19 2017 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Tue, 18 Jul 2017 14:53:19 -0700 Subject: RFR: compile-time guard some UseSharedSpaces-only coding with the INCLUDE_CDS macro - was :RE: jdk10 : UseSharedSpaces flag and INCLUDE_CDS macro In-Reply-To: <6f85d82587e349579cc7d5be62914ab5@sap.com> References: <4d2f3964b0924ffa897074759e508653@sap.com> <6f85d82587e349579cc7d5be62914ab5@sap.com> Message-ID: <66AA4D76-7526-4C11-BCDC-66EC368900F9@oracle.com> Hi Matthias, > On Jul 18, 2017, at 12:24 AM, Baesken, Matthias wrote: > > Hi , thanks for the review . > > > >Related to the change in universe_init() (universe.cpp), the 'if (DumpSharedSpaces)? block at line 715 is also CDS only. > >To be consistent, you might want to include it under #if INCLUDE_CDS as well. > > I added the suggested change and created a second webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8184323.1/ Looks good. I see Coleen already said she will push the change for you. Thanks, Jiangli > > > Is a second review needed ? > > > Best regards, Matthias > > From: Jiangli Zhou [mailto:jiangli.zhou at oracle.com ] > Sent: Samstag, 15. Juli 2017 06:37 > To: Baesken, Matthias > > Cc: hotspot-dev at openjdk.java.net ; build-dev at openjdk.java.net > Subject: Re: RFR: compile-time guard some UseSharedSpaces-only coding with the INCLUDE_CDS macro - was :RE: jdk10 : UseSharedSpaces flag and INCLUDE_CDS macro > > Hi Baesken, > > Thank you for making the changes. > > Related to the change in universe_init() (universe.cpp), the 'if (DumpSharedSpaces)? block at line 715 is also CDS only. To be consistent, you might want to include it under #if INCLUDE_CDS as well. > --- 695,716 ---- > Universe::_loader_addClass_cache = new LatestMethodCache(); > Universe::_pd_implies_cache = new LatestMethodCache(); > Universe::_throw_illegal_access_error_cache = new LatestMethodCache(); > Universe::_do_stack_walk_cache = new LatestMethodCache(); > > + #if INCLUDE_CDS > if (UseSharedSpaces) { > // Read the data structures supporting the shared spaces (shared > // system dictionary, symbol table, etc.). After that, access to > // the file (other than the mapped regions) is no longer needed, and > // the file is closed. Closing the file does not affect the > // currently mapped regions. > MetaspaceShared::initialize_shared_spaces(); > StringTable::create_table(); > ! } else > ! #endif > ! { > SymbolTable::create_table(); > StringTable::create_table(); > > if (DumpSharedSpaces) { > MetaspaceShared::prepare_for_dumping(); > In the past we have been trying to avoid adding too many #if in the code for better readability. For the CDS only code in universe.cpp (and a few other files), since the callee functions (e.g. initialize_shared_spaces()) are already defined with two versions (with and without INCLUDE_CDS). The builds without CDS also works fine even #if INCLUDE_CDS are not added. > > Thanks, > Jiangli > > On Jul 13, 2017, at 5:18 AM, Baesken, Matthias > wrote: > > Thank you for noticing that. Yes, it would be helpful if you can add the #if INCLUDE_CDS for CDS only code. > I can help you integrate the changes after they are reviewed. > > Thanks for your feedback ! > > I created a bug : > > https://bugs.openjdk.java.net/browse/JDK-8184323 > > > and a webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8184323/ > > > Best regards, Matthias > > > -----Original Message----- > From: Jiangli Zhou [mailto:jiangli.zhou at oracle.com ] > Sent: Montag, 10. Juli 2017 17:54 > To: Baesken, Matthias > > Cc: hotspot-dev at openjdk.java.net ; build-dev at openjdk.java.net > Subject: Re: jdk10 : UseSharedSpaces flag and INCLUDE_CDS macro > > Hi Matthias, > > Thank you for noticing that. Yes, it would be helpful if you can add the #if INCLUDE_CDS for CDS only code. I can help you integrate the changes after they are reviewed. > > Thanks! > Jiangli > > > On Jul 5, 2017, at 6:36 AM, Baesken, Matthias > wrote: > > Hello, when looking into CDS related build options, I noticed that most code-parts that are executed only when UseSharedSpaces is set, > are guarded by the compile-time macro INCLUDE_CDS to support switching off compilation of this coding > when CDS is disabled at compile time : > > > See hotspot/make/lib/JvmFeatures.gmk : > > ifneq ($(call check-jvm-feature, cds), true) > JVM_CFLAGS_FEATURES += -DINCLUDE_CDS=0 > > > > However some places miss the compile-time guarding ( #if INCLUDE_CDS ....) for example in > > > share/vm/prims/jvmtiRedefineClasses.cpp > share/vm/memory/universe.cpp > > (also some other places) > > > Should I prepare a change and add the compile-time guard at the places where missing as well ? > > Best regards, Matthias From yasuenag at gmail.com Tue Jul 18 22:32:36 2017 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Wed, 19 Jul 2017 07:32:36 +0900 Subject: [10] RFR: 8184309: Buld warnings from GCC 7.1 on Fedora 26 In-Reply-To: References: <60251551-F66E-4A2D-B354-399C3C3D1A20@oracle.com> Message-ID: Thank you so much! Yasumasa On 2017/07/19 0:19, Vladimir Ivanov wrote: > Pushed [1]. > > Best regards, > Vladimir Ivanov > > [1] http://hg.openjdk.java.net/jdk10/hs/hotspot/rev/eb11b3f0ae65 > > On 7/14/17 6:49 AM, Yasumasa Suenaga wrote: >> Updated: >> http://cr.openjdk.java.net/~ysuenaga/JDK-8184309/webrev.02/ >> >> >> Yasumasa >> >> >> 2017-07-14 12:11 GMT+09:00 Kim Barrett : >>>> On Jul 13, 2017, at 9:57 PM, Yasumasa Suenaga wrote: >>>> >>>> Thanks Kim, >>>> >>>> I uploaded new webrev for this issue: >>>> >>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8184309/webrev.01/ >>>> >>>> Vladimir, could you push this patch? >>> >>> Before pushing, please update the copyright in logFileOutput.hpp. >>> >>>> Yasumasa >>> From tim.bell at oracle.com Wed Jul 19 01:29:26 2017 From: tim.bell at oracle.com (Tim Bell) Date: Tue, 18 Jul 2017 18:29:26 -0700 Subject: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build In-Reply-To: <2232715d41bd453abdc339a988bdce3c@sap.com> References: <6c85b34b225e417788f794a99269e9fb@sap.com> <5967D063.1010808@cjnash.com> <424e7257bc504fe09d967ce56d64afd1@sap.com> <2ac4b05d05434369bdeb72f7018d00f8@sap.com> <948004c4-63e7-b0e4-1820-bf2351fc2327@oracle.com> <738dc2d22854475f9a15e163f7038872@sap.com> <2232715d41bd453abdc339a988bdce3c@sap.com> Message-ID: <596EB5F6.1000802@oracle.com> Hello - This fix has been pushed to OpenJDK jdk10/jdk10: http://hg.openjdk.java.net/jdk10/jdk10/rev/49a15c503104 Tim On 07/18/17 03:36, Baesken, Matthias wrote: > Ping : could someone from oracle please push it ? > (reason is that it touches the generated-configure.sh files ) > > Thanks, Matthias > > > -----Original Message----- > From: Baesken, Matthias > Sent: Freitag, 14. Juli 2017 17:05 > To: 'Erik Joelsson' ; Simon Nash > Cc: 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' ; Zeller, Arno > Subject: RE: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build > > Thanks , can you push it ( as far as I know someone from Oracle needs to push it because generated-configure.sh is changed ) ? > > Best regards, Matthias > > > > -----Original Message----- > From: Erik Joelsson [mailto:erik.joelsson at oracle.com] > Sent: Freitag, 14. Juli 2017 14:43 > To: Baesken, Matthias ; Simon Nash > Cc: 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' ; Zeller, Arno > Subject: Re: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build > > This looks good to me. > > /Erik > > > On 2017-07-14 13:47, Baesken, Matthias wrote: >> After Simons question , I checked that the jdk10 builds still works with gcc 4.7 on linux x86_64 as well . >> And created a new webrev : >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8184338.1/ >> >> - check adjusted to minimum gcc 4.7 >> - common/doc/building.md adjusted (the part talking about gcc versions) >> >> >> Best regards, Matthias >> >> >> >> -----Original Message----- >> From: Baesken, Matthias >> Sent: Freitag, 14. Juli 2017 08:36 >> To: 'Simon Nash' >> Cc: Erik Joelsson ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' ; Zeller, Arno >> Subject: RE: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build >> >>> I am currently building JDK 9 for arm32 (hard float and soft float) using >>> a Linaro GCC 4.7 cross-compiler (binary download). What is the reason that >>> this cannot be supported for JDK 10? >> Hi Simon, >> reason was that we know gcc-4.8 works nicely because we do a lot of builds (+tests) with this compiler . >> For gcc 4.7 we just do not know ( but for old 4.3 / 4.4 it was obvious that the current flags do not work any more). >> If you are using a gcc compiler < 4.8 only a warning is reported , it does not mean that you cannot build any more. >> >> My first suggestion was : >> >>>>> ( e.g. change to minimum gcc 4.6 or gcc 4.7 supporting the flags used in the build ) >> So if you see a benefit to test for minimum gcc 4.7 and not 4.8 I am fine with this too (Erik what do you think?). >> We at SAP just cannot tell that gcc 4.7 still works (because our builds do not use it). >> >> Best regards, Matthias >> >> >> >> -----Original Message----- >> From: Simon Nash [mailto:simon at cjnash.com] >> Sent: Donnerstag, 13. Juli 2017 21:56 >> To: Baesken, Matthias >> Cc: Erik Joelsson ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' ; Zeller, Arno >> Subject: Re: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build >> >> I am currently building JDK 9 for arm32 (hard float and soft float) using >> a Linaro GCC 4.7 cross-compiler (binary download). What is the reason that >> this cannot be supported for JDK 10? >> >> Best regards, >> Simon >> >> On 13/07/2017 14:40, Baesken, Matthias wrote: >>> Hi Erik, I prepared the change : >>> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8184338/ >>> >>> bug : >>> >>> https://bugs.openjdk.java.net/browse/JDK-8184338 >>> >>> Asking for review(s) ... >>> >>> >>> Best regards, Matthias >>> >>> >>> >>> >>> -----Original Message----- >>> From: Erik Joelsson [mailto:erik.joelsson at oracle.com] >>> Sent: Donnerstag, 13. Juli 2017 13:34 >>> To: Baesken, Matthias ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' >>> Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build >>> >>> Hello, >>> >>> That would be the correct place. If you prepare the change and send the >>> review I can sponsor the push. >>> >>> /Erik >>> >>> On 2017-07-13 13:16, Baesken, Matthias wrote: >>>> Hi Erik, >>>> >>>>> AFAIK, the only reason we support GCC versions older than 4.9 is for you >>>>> guys at SAP, so if you would suggest dropping support, that would of >>>>> course be the simplest solution. >>>> for jdk10 it is fine for us to use minimum gcc 4.8 . That would be probably the most simple solution to avoid running into the "-fno-var-tracking-assignments" issue >>>> ( I now and then run into it when I forget to set the PATH to gcc-4.8 one should use for compiling on the SLES11 machine, so I pick up instead the default gcc 4.3.4 from the machine , >>>> configure works nicely and then the build fails in the middle of the HS make ) >>>> >>>> autoconf/toolchain.m4 >>>> >>>> TOOLCHAIN_MINIMUM_VERSION_gcc="4.8" >>>> >>>> Is there right place to adjust I think, I guess we need someone for Oracle to regenerate the generated-configure.sh files ? >>>> >>>> Best regards, Matthias >>>> >>>> >>>> >>>> >>>> -----Original Message----- >>>> From: Erik Joelsson [mailto:erik.joelsson at oracle.com] >>>> Sent: Donnerstag, 13. Juli 2017 11:59 >>>> To: Baesken, Matthias ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' >>>> Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build >>>> >>>> Hello Matthias, >>>> >>>> AFAIK, the only reason we support GCC versions older than 4.9 is for you >>>> guys at SAP, so if you would suggest dropping support, that would of >>>> course be the simplest solution. >>>> >>>> If you want to keep support but make the use of this flag optional, the >>>> preferred method is to add a test in flags.m4. We have macros defined >>>> for this. FLAGS_COMPILER_CHECK_ARGUMENTS or >>>> FLAGS_CXX_COMPILER_CHECK_ARGUMENTS would be suitable in this case. Use >>>> it to check if the flag is valid for the current compiler. If so, you >>>> can define a variable >>>> "CXXFLAGS_GCC_NO_VAR_TRACKING_ASSIGNMENTS=-fno-var-tracking-assignment" >>>> and empty otherwise, and in the makefile use this variable in the >>>> assignement. >>>> >>>> We want to avoid static shell expressions in the makefiles if possible >>>> and keep that kind of logic in configure. It's also better to explicitly >>>> check for features rather than versions. >>>> >>>> /Erik >>>> >>>> >>>> On 2017-07-13 11:21, Baesken, Matthias wrote: >>>>> Hello, when building jdk10 on Suse Linux 11 with default gcc/g++ 4.3.4 installed, I was running into >>>>> compilation errors because of the missing support for g++ flag -fno-var-tracking-assignments . >>>>> >>>>> It seems gcc-4.6 has the -fvar-tracking-assignments / -fnovar-tracking-assignments flags , see >>>>> >>>>> https://gcc.gnu.org/onlinedocs/gcc-4.6.4/gcc/Option-Summary.html#Option-Summary >>>>> >>>>> I have no gcc-4.6 at hand but could verify that gcc-4.7 and gcc-4.8 "know" the flag. >>>>> There are various solutions one could do to avoid the compilation error . >>>>> >>>>> >>>>> 1) disallow usage of old gcc versions here : >>>>> >>>>> autoconf/toolchain.m4 >>>>> >>>>> TOOLCHAIN_MINIMUM_VERSION_gcc="4.3" >>>>> >>>>> ( e.g. change to minimum gcc 4.6 or gcc 4.7 supporting the flags used in the build ) >>>>> >>>>> >>>>> 2) remove the flag -fno-var-tracking-assignments for older gcc versions here : >>>>> (in a similar way it was done : http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/37e693211deb ) >>>>> >>>>> hotspot/make/lib/JvmOverrideFiles.gmk-32-ifeq ($(TOOLCHAIN_TYPE), gcc) >>>>> hotspot/make/lib/JvmOverrideFiles.gmk:33: BUILD_LIBJVM_vmStructs.cpp_CXXFLAGS := -fno-var-tracking-assignments -O0 >>>>> hotspot/make/lib/JvmOverrideFiles.gmk:34: BUILD_LIBJVM_jvmciCompilerToVM.cpp_CXXFLAGS := -fno-var-tracking-assignments >>>>> hotspot/make/lib/JvmOverrideFiles.gmk-35-endif >>>>> >>>>> What is your preferred solution ? >>>>> >>>>> >>>>> Thanks, Matthias >>>>> >>>>> > From matthias.baesken at sap.com Wed Jul 19 05:29:21 2017 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 19 Jul 2017 05:29:21 +0000 Subject: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build In-Reply-To: <596EB5F6.1000802@oracle.com> References: <6c85b34b225e417788f794a99269e9fb@sap.com> <5967D063.1010808@cjnash.com> <424e7257bc504fe09d967ce56d64afd1@sap.com> <2ac4b05d05434369bdeb72f7018d00f8@sap.com> <948004c4-63e7-b0e4-1820-bf2351fc2327@oracle.com> <738dc2d22854475f9a15e163f7038872@sap.com> <2232715d41bd453abdc339a988bdce3c@sap.com> <596EB5F6.1000802@oracle.com> Message-ID: <27b86a87108d441dad8b7b0d56dba76e@sap.com> Hi Tim , Thanks ! -----Original Message----- From: Tim Bell [mailto:tim.bell at oracle.com] Sent: Mittwoch, 19. Juli 2017 03:29 To: Baesken, Matthias ; Erik Joelsson ; Simon Nash ; Simonis, Volker Cc: 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' ; Zeller, Arno Subject: Re: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build Hello - This fix has been pushed to OpenJDK jdk10/jdk10: http://hg.openjdk.java.net/jdk10/jdk10/rev/49a15c503104 Tim On 07/18/17 03:36, Baesken, Matthias wrote: > Ping : could someone from oracle please push it ? > (reason is that it touches the generated-configure.sh files ) > > Thanks, Matthias > > > -----Original Message----- > From: Baesken, Matthias > Sent: Freitag, 14. Juli 2017 17:05 > To: 'Erik Joelsson' ; Simon Nash > Cc: 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' ; Zeller, Arno > Subject: RE: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build > > Thanks , can you push it ( as far as I know someone from Oracle needs to push it because generated-configure.sh is changed ) ? > > Best regards, Matthias > > > > -----Original Message----- > From: Erik Joelsson [mailto:erik.joelsson at oracle.com] > Sent: Freitag, 14. Juli 2017 14:43 > To: Baesken, Matthias ; Simon Nash > Cc: 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' ; Zeller, Arno > Subject: Re: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build > > This looks good to me. > > /Erik > > > On 2017-07-14 13:47, Baesken, Matthias wrote: >> After Simons question , I checked that the jdk10 builds still works with gcc 4.7 on linux x86_64 as well . >> And created a new webrev : >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8184338.1/ >> >> - check adjusted to minimum gcc 4.7 >> - common/doc/building.md adjusted (the part talking about gcc versions) >> >> >> Best regards, Matthias >> >> >> >> -----Original Message----- >> From: Baesken, Matthias >> Sent: Freitag, 14. Juli 2017 08:36 >> To: 'Simon Nash' >> Cc: Erik Joelsson ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' ; Zeller, Arno >> Subject: RE: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build >> >>> I am currently building JDK 9 for arm32 (hard float and soft float) using >>> a Linaro GCC 4.7 cross-compiler (binary download). What is the reason that >>> this cannot be supported for JDK 10? >> Hi Simon, >> reason was that we know gcc-4.8 works nicely because we do a lot of builds (+tests) with this compiler . >> For gcc 4.7 we just do not know ( but for old 4.3 / 4.4 it was obvious that the current flags do not work any more). >> If you are using a gcc compiler < 4.8 only a warning is reported , it does not mean that you cannot build any more. >> >> My first suggestion was : >> >>>>> ( e.g. change to minimum gcc 4.6 or gcc 4.7 supporting the flags used in the build ) >> So if you see a benefit to test for minimum gcc 4.7 and not 4.8 I am fine with this too (Erik what do you think?). >> We at SAP just cannot tell that gcc 4.7 still works (because our builds do not use it). >> >> Best regards, Matthias >> >> >> >> -----Original Message----- >> From: Simon Nash [mailto:simon at cjnash.com] >> Sent: Donnerstag, 13. Juli 2017 21:56 >> To: Baesken, Matthias >> Cc: Erik Joelsson ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' ; Zeller, Arno >> Subject: Re: RFR: [XS] 8184338 : switch minimum supported gcc version to 4.8 - was : RE: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build >> >> I am currently building JDK 9 for arm32 (hard float and soft float) using >> a Linaro GCC 4.7 cross-compiler (binary download). What is the reason that >> this cannot be supported for JDK 10? >> >> Best regards, >> Simon >> >> On 13/07/2017 14:40, Baesken, Matthias wrote: >>> Hi Erik, I prepared the change : >>> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8184338/ >>> >>> bug : >>> >>> https://bugs.openjdk.java.net/browse/JDK-8184338 >>> >>> Asking for review(s) ... >>> >>> >>> Best regards, Matthias >>> >>> >>> >>> >>> -----Original Message----- >>> From: Erik Joelsson [mailto:erik.joelsson at oracle.com] >>> Sent: Donnerstag, 13. Juli 2017 13:34 >>> To: Baesken, Matthias ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' >>> Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build >>> >>> Hello, >>> >>> That would be the correct place. If you prepare the change and send the >>> review I can sponsor the push. >>> >>> /Erik >>> >>> On 2017-07-13 13:16, Baesken, Matthias wrote: >>>> Hi Erik, >>>> >>>>> AFAIK, the only reason we support GCC versions older than 4.9 is for you >>>>> guys at SAP, so if you would suggest dropping support, that would of >>>>> course be the simplest solution. >>>> for jdk10 it is fine for us to use minimum gcc 4.8 . That would be probably the most simple solution to avoid running into the "-fno-var-tracking-assignments" issue >>>> ( I now and then run into it when I forget to set the PATH to gcc-4.8 one should use for compiling on the SLES11 machine, so I pick up instead the default gcc 4.3.4 from the machine , >>>> configure works nicely and then the build fails in the middle of the HS make ) >>>> >>>> autoconf/toolchain.m4 >>>> >>>> TOOLCHAIN_MINIMUM_VERSION_gcc="4.8" >>>> >>>> Is there right place to adjust I think, I guess we need someone for Oracle to regenerate the generated-configure.sh files ? >>>> >>>> Best regards, Matthias >>>> >>>> >>>> >>>> >>>> -----Original Message----- >>>> From: Erik Joelsson [mailto:erik.joelsson at oracle.com] >>>> Sent: Donnerstag, 13. Juli 2017 11:59 >>>> To: Baesken, Matthias ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' >>>> Subject: Re: jdk10 build : usage of -fno-var-tracking-assignments g++ flag in hotspot build >>>> >>>> Hello Matthias, >>>> >>>> AFAIK, the only reason we support GCC versions older than 4.9 is for you >>>> guys at SAP, so if you would suggest dropping support, that would of >>>> course be the simplest solution. >>>> >>>> If you want to keep support but make the use of this flag optional, the >>>> preferred method is to add a test in flags.m4. We have macros defined >>>> for this. FLAGS_COMPILER_CHECK_ARGUMENTS or >>>> FLAGS_CXX_COMPILER_CHECK_ARGUMENTS would be suitable in this case. Use >>>> it to check if the flag is valid for the current compiler. If so, you >>>> can define a variable >>>> "CXXFLAGS_GCC_NO_VAR_TRACKING_ASSIGNMENTS=-fno-var-tracking-assignment" >>>> and empty otherwise, and in the makefile use this variable in the >>>> assignement. >>>> >>>> We want to avoid static shell expressions in the makefiles if possible >>>> and keep that kind of logic in configure. It's also better to explicitly >>>> check for features rather than versions. >>>> >>>> /Erik >>>> >>>> >>>> On 2017-07-13 11:21, Baesken, Matthias wrote: >>>>> Hello, when building jdk10 on Suse Linux 11 with default gcc/g++ 4.3.4 installed, I was running into >>>>> compilation errors because of the missing support for g++ flag -fno-var-tracking-assignments . >>>>> >>>>> It seems gcc-4.6 has the -fvar-tracking-assignments / -fnovar-tracking-assignments flags , see >>>>> >>>>> https://gcc.gnu.org/onlinedocs/gcc-4.6.4/gcc/Option-Summary.html#Option-Summary >>>>> >>>>> I have no gcc-4.6 at hand but could verify that gcc-4.7 and gcc-4.8 "know" the flag. >>>>> There are various solutions one could do to avoid the compilation error . >>>>> >>>>> >>>>> 1) disallow usage of old gcc versions here : >>>>> >>>>> autoconf/toolchain.m4 >>>>> >>>>> TOOLCHAIN_MINIMUM_VERSION_gcc="4.3" >>>>> >>>>> ( e.g. change to minimum gcc 4.6 or gcc 4.7 supporting the flags used in the build ) >>>>> >>>>> >>>>> 2) remove the flag -fno-var-tracking-assignments for older gcc versions here : >>>>> (in a similar way it was done : http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/37e693211deb ) >>>>> >>>>> hotspot/make/lib/JvmOverrideFiles.gmk-32-ifeq ($(TOOLCHAIN_TYPE), gcc) >>>>> hotspot/make/lib/JvmOverrideFiles.gmk:33: BUILD_LIBJVM_vmStructs.cpp_CXXFLAGS := -fno-var-tracking-assignments -O0 >>>>> hotspot/make/lib/JvmOverrideFiles.gmk:34: BUILD_LIBJVM_jvmciCompilerToVM.cpp_CXXFLAGS := -fno-var-tracking-assignments >>>>> hotspot/make/lib/JvmOverrideFiles.gmk-35-endif >>>>> >>>>> What is your preferred solution ? >>>>> >>>>> >>>>> Thanks, Matthias >>>>> >>>>> > From shafi.s.ahmad at oracle.com Wed Jul 19 07:07:53 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Wed, 19 Jul 2017 00:07:53 -0700 (PDT) Subject: [8u] RFR for backport of JDK-8177958: Possible uninitialized char* in vm_version_solaris_sparc.cpp In-Reply-To: <20170718223710.GE3570@vimes> References: <55a24e4c-3922-4a0e-983d-433701afa68c@default> <20170718121909.GB3570@vimes> <20170718223710.GE3570@vimes> Message-ID: Hi Rob, I am sorry, I should have sent it to hotspot-dev but instead by mistake I sent it to jdk8u-dev. Resending it to correct mailing list. Regards, Shafi > -----Original Message----- > From: Rob McKenna > Sent: Wednesday, July 19, 2017 4:07 AM > To: Shafi Ahmad > Cc: jdk8u-dev at openjdk.java.net > Subject: Re: [8u] RFR for backport of JDK-8177958: Possible uninitialized char* > in vm_version_solaris_sparc.cpp > > Ah, apologies, I took this for a request for approval. > > -Rob > > On 18/07/17 01:19, Rob McKenna wrote: > > Appears to be a line shuffling issue. Approved. > > > > -Rob > > > > On 17/07/17 11:40, Shafi Ahmad wrote: > > > Hi, > > > > > > Please review the backport of ' JDK-8177958: Possible uninitialized char* > in vm_version_solaris_sparc.cpp' to jdk8u. > > > Please note that this is not a clean backport due to single line conflict. > > > > > > Webrev: > http://cr.openjdk.java.net/~shshahma/8177958/jdk8u/webrev.00/ > > > Jdk10 bug: https://bugs.openjdk.java.net/browse/JDK-8177958 > > > Original patch pushed to jdk10: > http://hg.openjdk.java.net/jdk10/hs/hotspot/rev/7295c9c26185 > > > > > > Test: Run jprt. > > > > > > Regards, > > > Shafi From aph at redhat.com Wed Jul 19 16:21:27 2017 From: aph at redhat.com (Andrew Haley) Date: Wed, 19 Jul 2017 17:21:27 +0100 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: <596E288B.90706@oracle.com> References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <596CD126.6080100@oracle.com> <596DD713.6080806@oracle.com> <77d5a9e7-b2b0-79e4-effd-903dfe226512@redhat.com> <596E1142.6040708@oracle.com> <3432d11c-5904-a446-1cb5-419627f1794f@redhat.com> <596E288B.90706@oracle.com> Message-ID: <773a0b64-8273-1a58-a46a-22ae18f927b7@redhat.com> On 18/07/17 16:26, Erik ?sterlund wrote: > > > On 2017-07-18 16:41, Andrew Haley wrote: >> On 18/07/17 14:46, Erik ?sterlund wrote: >>> >>> On 2017-07-18 11:57, Andrew Haley wrote: >>>> On 18/07/17 10:38, Erik ?sterlund wrote: >> It is more general than that. If you access the stored value of an >> object through a glvalue of other than one of the allowed types, then >> your *whole program is undefined*. > > And I think it is in that list of allowed types. Specifically, it is my > interpretation of the standard that intptr_t is "a type that is the > signed or unsigned type corresponding to the dynamic type of the object" > (cf. 5.2.10) if the dynamic type is e.g. void*. > >>> and that therefore it is fine to load something (that was stored as >>> A*) as intptr_t and subsequently cast it to A* before the A itself >>> is being accessed. Am I wrong? >> No, that is correct. The question is whether intptr_t is a type >> compatible with a pointer type, and I don't think you will find any >> language to the effect that it is. > > I think 5.2.10 describing reinterpret_cast seems to suggest that they > are compatible by saying "A pointer converted to an integer of > sufficient size (if any such exists on the implementation) and back to > the same pointer type will have its original value". That is precisely > what is done. This is the core of our disagreement. In a sense, it doesn't matter which of us is right; what really matters is the opinion of the compiler. GCC does not put intptr_t and pointer types into the same alias set, and even if this did turn out to be a bug in GCC it wouldn't help us because we often use unsupported old versions of GCC to build OpenJDK, so it wouldn't get fixed anyway. So what are we to do? Do you believe me when I say that GCC does treat A* and intptr_t to be compatible types? If not, what would it take to convince you? > Conversely, I do not find any language suggesting that intptr_t would > not be compatible with pointer types. > >> Pointer types are distinct from one another and from all integer >> types. People sometimes get confused by this: the fact that you can >> cast from A* to e.g. void*, doesn't tell you that you can cast from >> A** to void** and use the result: they are different types. > > Agreed. But I do not see any problem with reinterpret_cast from A* to > intptr_t back to A* and using the A from that result. That is explicitly > allowed as per 5.2.10 as previously described. I don't think that's what you're doing. I think you're casting from A** to intptr_t*, dereferencing the intptr_t*, and casting the result to A*. > That looks like the correct (and indeed expected) output. iptr should be > the same as &b, and iptr->_x should be 0, and &a should be different > from &b and iptr. I tried this experiment locally too with similar > correct result. Am I missing something here? No, I was; I misinterpreted the result, and it's a red herring. Sorry. I believe this patch to be dangerous. We might well get away with it, given sufficient larding of volatile, but it's still undefined behaviour. We have far too much UB in HotSpot already, and we should not be adding any more. And I still believe that this patch is more complex than is needed, and with many (if not all) targets we could do something much simpler, and which does not need to cast pointer types. It could also map cleanly onto C++11. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From kim.barrett at oracle.com Thu Jul 20 02:08:05 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 19 Jul 2017 22:08:05 -0400 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: <773a0b64-8273-1a58-a46a-22ae18f927b7@redhat.com> References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <596CD126.6080100@oracle.com> <596DD713.6080806@oracle.com> <77d5a9e7-b2b0-79e4-effd-903dfe226512@redhat.com> <596E1142.6040708@oracle.com> <3432d11c-5904-a446-1cb5-419627f1794f@redhat.com> <596E288B.90706@oracle.com> <773a0b64-8273-1a58-a46a-22ae18f927b7@redhat.com> Message-ID: <14A41261-BA5F-43EE-8C31-D08DB22B0112@oracle.com> > On Jul 19, 2017, at 12:21 PM, Andrew Haley wrote: > I believe this patch to be dangerous. We might well get away with it, > given sufficient larding of volatile, but it's still undefined > behaviour. We have far too much UB in HotSpot already, and we should > not be adding any more. > > And I still believe that this patch is more complex than is needed, > and with many (if not all) targets we could do something much simpler, > and which does not need to cast pointer types. It could also map > cleanly onto C++11. I agree with Andrew that we're violating strict aliasing rules [1]. But I disagree with Andrew's assessement of the consequences. Yes, we're violating strict aliasing. Existing code is already doing that. We've been relying on volatile larding (I love that phrasing), opacity of assembly code, and things like -fno-strict-aliasing to get away with that. This change doesn't change any of that. What this change does do is allow us to centralize and isolate where that's happening. Current uses of Atomic are littered with casts, along with some in the existing Atomic implementation. By presenting a general, type-safe, interface, we are eliminating the need for those casts in API uses (or the oft-used alternative of making unnatural type choices in order to match the existing limited API). This new implementation also does a lot of validation to improve code correctness; that's simply missing in the existing casts-in-usage approach. So I think this change makes some things better (in some ways significantly), while not being any worse than we already are on the strict aliasing question. Once we have this new API in place, we can go to work on cleaning up the uses. Independently, we can work on the implementation, which may eventually include using C++11 atomics, at least on some platforms. (Where applicable, that looks like it should be pretty easy with the new API. [2]) The limitations of the present API lead to a strong coupling between usage and implementation that make those kinds of changes difficult. Providing a better interface removes that coupling and makes it easier to work on the various parts in isolation. We also need something like this to support the GC interface work that is in progress. Some mechanism for connecting higher level operations to the low-level platform-specific implementations is needed by that work. Putting that mechanism in the atomics layer (and a forthcoming proposal for changes to orderaccess) makes it available to more than just the GC interface access protocol. I agree with Erik that C++11 atomics cannot be considered a panacea in this domain. As he notes, there are implementation choices that are not exposed, and which require consistency across all uses, but not all of our interacting uses are in C++. There is also a problem with using C++11 atomics that it generally requires enabling C++11 support, which makes it all too easy to unintentionally use other C++11 features. This can easily run into problems because there are some seriously laggard compilers being used by some folks. It would be very easy do some work that unintentionally relies on some set of new language features, test, review, and push it, and only later find out it all needed to be backed out and re-thought due to limitations of one of these exotic platforms. (See discussion on the build-dev list, subject "C++11/C++14 support in XLC ?".) I would like to think we can find our way through that, but it seems unlikely to happen quickly. [1] Erik quoted 3.10: "a type that is the signed or unsigned type corresponding to the dynamic type of the object" and suggested "...which it is as intptr_t is the signed type corresponding to A*." While attractive in this situation, I see no basis in the standard for that interpretation of "corresponding". All relevant uses of "corresponding" are wrto integral types only. My interpretation (which I think matches what Andrew says about gcc's implementation) is that "corresponding" here is about the pairs of signed/unsigned integral types of a given size and rank. That is, one can read a dynamically typed signed long as an unsigned long, or vice versa. No more than that. As Andrew pointed out, in the case of "A* is stored, and subsequently loaded as intptr_t (and casted to A*)", 5.2.10/4 doesn't apply. This is not a cast of A* to intptr_t. Rather, it's a cast of A** to intptr_t* (or in the code code in question maybe some other I* where I is an integral type). The relevant reference is 5.2.10/7, about converting one pointer type to another. There's a C++11 addition there about reinterpret_cast of pointers being equivalent to a two step sequence of static_cast's through void* under certain conditions; that addition codified existing practice. But that doesn't help with the 3.10 dereferencing issue. [2] A couple of months ago I built Linux x86_64 with -std=gnu++11 and ran our nightly tests, and didn't see any surprises. We haven't flipped that switch, in part because of the above mentioned unintended use of new features problem (though with only one platform we shouldn't get to the actual push part). Also, one ad hoc run of a subset of our tests isn't sufficient for such a change. However, I don't think there's anything preventing the aarch64 port from independently doing something like that and reimplementing Atomic using C++11 atomics, so long as the changes to shared code were done in an appropriate manner. I'd review that. I might even find some spare time to help. From coleen.phillimore at oracle.com Thu Jul 20 02:52:43 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 19 Jul 2017 22:52:43 -0400 Subject: RFR (L) 7133093: Improve system dictionary performance In-Reply-To: <233c03e0-8958-7676-8e33-5d8f0ec4edf3@oracle.com> References: <89f0b98c-3cbd-7b87-d76b-5e89a5a676fb@oracle.com> <9243da7c-a8e5-4024-c12e-cba9d4f424c8@oracle.com> <7814C032-0D4E-4E76-B8AD-62A072E8A625@oracle.com> <233c03e0-8958-7676-8e33-5d8f0ec4edf3@oracle.com> Message-ID: <7adf48f1-5908-aeaf-c409-d930895e1715@oracle.com> Hi, I've made the changes in the replies below and retested with both our internal NSK tests, jtreg tests and jck tests. Also I've rerun the performance tests, including SPECjbb2015. An incremental webrev can be found at: http://cr.openjdk.java.net/~coleenp/7133093.03.incremental/webrev/ Latest version at: http://cr.openjdk.java.net/~coleenp/7133093.03/webrev/ Thanks, Coleen On 7/18/17 10:56 AM, coleen.phillimore at oracle.com wrote: > > Karen, Thank you for reviewing the code and for the discussions and > consulting along the way. > > On 7/17/17 4:10 PM, Karen Kinnear wrote: >> Feedback on the initial web rev: >> Thank you for using web rev - that way I can do searches of the >> overall patch! >> >> First - many thanks for doing this exercise so carefully, for >> following the specification in continuing >> to record the initiating loader and for splitting this effort into >> multiple phases to allow further exploration >> of the current expectations/behavior relative to class unloading. >> >> And thank you for all the testing. >> >> Just wanted to ensure that I understand some of the assumptions: >> >> 1) lock free reads of dictionaries - yes > Yes. >> 2) lock free adds of entries to dictionaries - yes > No, adds still take out the SystemDictionary_lock. >> 3) removal of entries or entire dictionaries - at safe point only > Yes. >> 4) rehashing - none today and none planned > Yes. >> 5) resizing of dictionaries - none today, some planned > Yes. >> 6) use of SystemDictionary_lock: essentially unchanged > Yes. >> 7) atomicity guarantees relative to checking loader constraints and >> updates to the dictionary: unchanged > Yes. >> >> I did not review the SA/vmStructs - sounds like you had Jini George >> do that - great. >> >> Minor questions/comments: >> >> SystemDictionary::verify: "Verify of system dictionary failed" -> >> looks like "Verify of placeholders failed" > > Fixed. Good find. It's an odd assert: > > guarantee(placeholders()->number_of_entries() >= 0, > "Verify of placeholders failed"); > >> >> thank you for asserts dictionary != NULL >> Dictionary::print >> is it worth an assert that loader_data != NULL? > > Added. It's already asserted in Dictionary::verify() > >> Can you please sanity check if there are other cases in which we need >> to use dictionary_or_null() rather than dictionary()? >> - are there cases in which we could be using a dictionary from an >> anonymous class? >> e.g. there are a lot of places in systemDictionary.cpp where we >> call class_loader_data(class_loader); loader_data->dictionary(); >> - was the lazily allocated dictionary allocated yet (see next >> question) > > There's an assert in dictionary() that the CLD isn't anonymous. In > these calls we get class_loader_data from the class_loader which is > always the host class loader for anonymous classes, so this doesn't > break with the current design. > >> >> CLD.cpp >> Would it be less complex to eagerly create the dictionary for a CLD >> since it is extremely rare to not need one for non-anonymous CLDs? >> You could create before publishing the CLD > > Yes, I agree it would be better this way. I was looking at some data > and it does seem rare to have a CLD with no classes. >> >> Terminology: >> How are you still using System Dictionary term? >> It still shows up in some comments and in some printed strings > > Bad habit, we've been calling it that for years. :) I'll update the > comments where it makes sense. > I tried to refer to the ClassLoaderDataGraph dictionaries collectively > as the "System Dictionary", as well as the loader constraints and > placeholder tables, and the other things that SystemDictionary contains. > > I guess it gets confusing, but it's less awkward than "class loader > data dictionaries", and more meaningful than simply "dictionaries". So > that's why I didn't change a lot of the comments. > >> >> biasedLocking.cpp, line 59 >> jvmtiGetLoadedClasses: line 298 >> dictionary.hpp lines 113,114, 120 >> dictionary.cpp 262-263, 478 >> dictionary.cpp 491 >> fix comments e.g. "Iterate the system dictionary" ... >> - probably worth checking for other comments in the sources too >> unless these all appear to match new usage model > > Fixed. I've found a couple more and some other comments that are > incorrect. >> >> hashTable.hpp >> line 268: what does this comment mean? >> "Don't create and use freelist"? >> I assume this means that this is a direct malloc, not using any block >> allocation, e.g. metaspace chunks, and this requires explicit freeing? >> > > // Don't create and use freelist of HashtableEntry. > > The default Hashtable<> allocates entries from a block of entries and > uses a freelist for freed entries. > >> >> dictionary.hpp line 161 >> could you possibly clean up comment while you are in here? >> I presume "a protection" is "a protection domain"? > > Fixed. > >> >> dictionary.cpp >> why change from Hashtable::free_entry to explicit >> unlink_entry/FREE_C_HEAP_ARRAY? > > I used the same code as the ModuleTable. The default hashtable > doesn't delete the whole thing and left some memory around, if I > remember correctly. I'll recheck it. >> >> Future optimization: >> I think that none of the built-in loaders is ever recorded as an >> initiating >> loader for any class in a loader which permits unloading. >> Note: we need to ensure that built-in includes boot, platform and >> application class loader (and clean up any confusion with system >> class loader - which can be set by the user to a custom class loader) >> > I think AppClassLoader can initiate loading and is a builtin loader. > > Dictionary for class loader class loader 0x00007f6abc6e3210 a > 'jdk/internal/loader/ClassLoaders$AppClassLoader'{0x0000000101c8df00} > Java system dictionary (table_size=107, classes=1) > ^ indicates that initiating loader is different from defining loader > 106: ^java.util.ArrayList, loader NULL class_loader > >> >> Any chance of changing DictionaryEntry* probe->klass() to be >> probe->instanceKlass()? > > Okay. Changed to instance_klass() as per coding standard. >> >> Thank you for assertion _is_at_safepoint for try_get_next_class >> >> CLD.cpp >> If the names of the classes_do vs. all_entries_do were modified to be >> defining_loader_classes_do vs. all_entries_classes_do or some way >> to make the distinction in the name that would be helpful. >> Otherwise, if you could please add comments to the >> CLDG::dictionary_classes_do >> etc. to distinguish these. > > I added comments to distinguish these. >> >> CLD.cpp >> lines 1200-1201 >> Can you please fix the comment? >> I think you need separate comments for the ModuleEntry >> reads/PackageEntry's >> exports which are looking for dead modules. >> IIUC, the do_unloading is actually looking for classes for which this >> is the initating loader, but their defining loader is dead. > // Remove entries in the dictionary of live class loader that have > // initiated loading classes in a dead class loader. > data->dictionary()->do_unloading(); > > Moved the module comment to before the module code and added this for > dictionary->do_unloading() call. >> >> systemDictionary.cpp: 445-447 >> THANK you for the comments about the java_mirror handle keeping the >> class >> and class loader alive >> Could you clarify that the class_loader handle is not for the class >> we are checking, but rather for the initiating, i.e. requesting >> class loader >> (is that accurate?) So we are keeping both the requestor and the >> requested class alive > > Yes, that is true. I added > > // The class_loader handle passed in is the initiating loader. > >> >> You removed the comment that compute_hash could lead to a safepoint/GC >> Is that not possible? >> comment in ObjectSynchronizer::FastHashCode says that with >> UseBiasedLocking >> we can revoke biases and must be able to handle a safepoint. > > I made it not possible with this change. The hashcode is now > computed by the Symbol* of the Klass name, which is random (os::random > plus some bits from the address and first two characters, which we did > a study of for bug https://bugs.openjdk.java.net/browse/JDK-8181450). > > Also, I changed ClassLoaderData::identity_hash() to use the address >> > 3 because the addresses of class loader datas are random enough for > the one use of this function. Previously, the loader_data identity > hash used to go to the class_loader oop and get to > ObjectSynchronizer::FastHashCode (which was also os::random but now > changed). > > The dictionary hashing doesn't need to add in the hash for the > loader_data now. >> >> systemDictionary.cpp: >> NoSafepointVerifier >> If we add resizing logic later, and that happens at a safepoint, >> then the number of buckets change and the d_index is no longer >> valid >> so we would need to add back the NSpV? >> - this is a more general problem if compute_hash can lead to a >> safepoint/GC > > Yes this is true, except that compute_hash doesn't safepoint. > Computing the hash and index should be done together with the > dictionary operations. The function resolve_instance_class_or_null > computes the hash and index once at the beginning and reuses it for > several lookups, which may or may not affect performance. So we need > to measure this in isolation before changing this. >> >> systemDictionary.cpp: parse_stream >> why did you remove the record_dependency? > > I realized it's trivially unnecessary. The class_loader is reached > by the ClassLoaderData::_class_loader field, so we don't have to > record a dependency from host_klass->class_loader in the anonymous > class loader data. (I just checked this again because it's confusing). > >> >> Who calls Dictionary::~Dictionary? > > The ClassLoaderData::~ClassLoaderData() calls delete _dictionary, > which calls the destructor. >> >> Performance: >> Do you have performance runs of our existing benchmarks? > > Yes, I ran our internal benchmarks and added links to the internal > wiki page. I'll rerun them again before checkin. > >> Initial key requirement is that this not slow down throughput through >> slowing down lookups. >> I know you are very aware of longer-term goals for reducing >> safepoint pause >> times and not increasing footprint significantly. >> >> hashtable sizes: future rfe? >> given that we now have a lot more dictionaries, and they perhaps need >> individual sizing >> - would it be helpful to add statistics information about >> - e.g. lookup_count, lookup_length, _lookup_warning >> - i.e. number of entries, max lookup depth, etc. to use later for >> tuning? > > All the previous statistics didn't make very much sense to us, so they > were removed. There is now logging for all the hashtables: > > busaa027.us.oracle.com% java -Xlog:hashtables -version > java version "10-internal" > Java(TM) SE Runtime Environment (fastdebug build > 10-internal+0-2017-07-17-1456035.cphillim.10system-dictionary2) > Java HotSpot(TM) 64-Bit Server VM (fastdebug build > 10-internal+0-2017-07-17-1456035.cphillim.10system-dictionary2, mixed > mode) > [0.284s][info][hashtables] System Dictionary for > jdk/internal/loader/ClassLoaders$AppClassLoader max bucket size 1 > bucket 91 element count 1 table size 107 > [0.285s][info][hashtables] System Dictionary for max > bucket size 4 bucket 263 element count 468 table size 1009 > [0.285s][info][hashtables] Placeholder Table max bucket size 0 bucket > 0 element count 0 table size 1009 > [0.285s][info][hashtables] Protection Domain Table max bucket size 0 > bucket 0 element count 0 table size 1009 > >> - might make sense to make this JFR data? > > I'll file an RFE to add tracing data with our tools. I'll need some > help with this part but the data is there. >> >> Might think about a follow-up with more tuning parameters than >> a single global PredictedLoadedClassCount. >> > > I filed RFE to work on resizing, as this work was almost complete, but > didn't show performance improvements at the time. It would be nice to > have customers not need tuning parameters, but I guess these things > are sometimes necessary. > > Thank you for the thorough review and all of your help with this! I've > made these changes and will rerun testing and send out a new webrev > and an incremental webrev. > > Thanks! > Coleen > >> >> thanks, >> Karen >> >> >>> On Jul 17, 2017, at 2:37 PM, coleen.phillimore at oracle.com >>> wrote: >>> >>> >>> Hi, I have made changes for Ioi's comments below. The largest >>> change since .01 is that PredictedLoadedClassCount is used to size >>> dictionaries for class loader data, and should be used to specify >>> the average number of classes loaded by each class loader. Since >>> this is an option available via UnlockExperimentalVMOptions, I'm not >>> sure what the process is for changing the meaning of this option. We >>> will work with our customers who have used this option for >>> specifying the total number of classes loaded. We are also working >>> on the change to allow automatic resizing of the dictionaries during >>> safepoints. >>> >>> See: >>> open webrev at http://cr.openjdk.java.net/~coleenp/7133093.02/webrev >>> >>> >>> Tested with the full nightly test set (jdk, nsk, hotspot jtreg >>> tests) on linux x64. >>> >>> Thanks, >>> Coleen >>> >>> On 6/30/17 4:45 PM, coleen.phillimore at oracle.com >>> wrote: >>>> >>>> Ioi, Thank you for looking at this change. >>>> >>>> On 6/30/17 2:39 PM, Ioi Lam wrote: >>>>> Hi Coleen, >>>>> >>>>> Maybe the bug should be renamed to "Use one Dictionary per class >>>>> loader instance"? That way it's more obvious what it is when you >>>>> look at the repo history. >>>> >>>> I can do that. I made it One Dictionary per ClassLoaderData >>>>> >>>>> 1. >>>>> >>>>> Why is assert_locked_or_safepoint(SystemDictionary_lock) necessary >>>>> in SystemDictionary::find_class (line 1826), but not necessary >>>>> SystemDictionary::find (line 951)? Since you removed >>>>> NoSafepointVerifier nosafepoint in the latter, maybe this means >>>>> it's safe to remove the assert_locked_or_safepoint in the former? >>>> >>>> The call to SystemDictionary::find() is the (I believe) usual lock >>>> free lookup and the SystemDictionary::find_class() is used to >>>> verify that a class we're about to add or want to add hasn't been >>>> added by another thread. Or certain cases where we already have a >>>> lock to do something else, like add loader constraints. I took out >>>> the NoSafepointVerifier because it assumes that system dictionary >>>> entries would be moved by GC, which they won't. The old hash >>>> function could safepoint when getting the hash for the class_loader >>>> object. >>>> >>>>> >>>>> 2. >>>>> >>>>> 455 static ClassLoaderData* _current_loader_data = NULL; >>>>> 456 static Klass* _current_class_entry = NULL; >>>>> 457 >>>>> 458 InstanceKlass* ClassLoaderDataGraph::try_get_next_class() { >>>>> >>>>> How about moving the static fields into an iterator object. That >>>>> way you don't need to keep track of the globals >>>>> >>>>> ClassLoaderDataGraphIterator { >>>>> ClassLoaderData* _current_loader_data >>>>> Klass* _current_class_entry; >>>>> >>>>> InstanceKlass* try_get_next_class() { ....} >>>>> }; >>>> >>>> Ok, there's a different iterator that iterates over all of the >>>> classes for GC. I will adapt that for this use. That would be >>>> better. >>>> >>>>> >>>>> 3. Double check locking in ClassLoaderData::dictionary() -- >>>>> someone else should look at this :-) >>>> >>>> I copied code that David Holmes added for modules. >>>> >>>>> >>>>> 4. We may need a better strategy for deciding the size of each >>>>> dictionary. >>>>> >>>>> 565 const int _primelist[10] = {1, 107, 1009}; >>>>> 571 Dictionary* ClassLoaderData::dictionary() { >>>>> 579 if ((dictionary = _dictionary) == NULL) { >>>>> 580 int size; >>>>> 581 if (this == the_null_class_loader_data() || >>>>> is_system_class_loader_data()) { >>>>> 582 size = _primelist[2]; >>>>> 583 } else if >>>>> (class_loader()->is_a(SystemDictionary::reflect_DelegatingClassLoader_klass())) >>>>> { >>>>> 584 size = _primelist[0]; >>>>> 585 } else { >>>>> 586 size = _primelist[1]; >>>>> 587 } >>>>> 588 dictionary = new Dictionary(this, size); >>>>> >>>>> I'll do some investigation on this issue and get back to you. >>>>> >>>> >>>> How about if someone uses PredictedLoadedClassCount, then we use >>>> that to size all but the reflection and boot class loader? Then if >>>> there's an application that has a class loader with a huge amount >>>> of classes loaded in it, that would help this? It might cost some >>>> footprint but an oversized table would simply be a bigger array of >>>> pointers, so it might not be that bad to oversize. >>>> >>>> I think the long term solution is being able to resize these >>>> entries and make the application provide arguments. Please let me >>>> know what you find in your investigation and if that would work. >>>>> The rest of the changes look good to me. >>>> >>>> Thank you! >>>> Coleen >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> >>>>> On 6/23/17 4:42 PM, coleen.phillimore at oracle.com >>>>> wrote: >>>>>> Summary: Implement one dictionary per ClassLoaderData for faster >>>>>> lookup and removal during class unloading >>>>>> >>>>>> See RFE for more details. >>>>>> >>>>>> open webrev at >>>>>> http://cr.openjdk.java.net/~coleenp/7133093.01/webrev >>>>>> >>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-7133093 >>>>>> >>>>>> Tested with full "nightly" run in rbt, plus locally class loading >>>>>> and unloading tests: >>>>>> >>>>>> jtreg hotspot/test/runtime/ClassUnload >>>>>> >>>>>> jtreg hotspot/test/runtime/modules >>>>>> >>>>>> jtreg hotspot/test/gc/class_unloading >>>>>> >>>>>> make test-hotspot-closed-tonga FILTER=quick TEST_JOBS=4 >>>>>> TEST=vm.parallel_class_loading >>>>>> >>>>>> csh ~/testing/run_jck9 (vm/lang/java_lang) >>>>>> >>>>>> runThese -jck - uses class loader isolation to run each jck test >>>>>> and unloads tests when done (at -gc:5 intervals) >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Coleen >>>>>> >>>>>> >>>>> >>>> >>> >> > From coleen.phillimore at oracle.com Thu Jul 20 02:54:44 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 19 Jul 2017 22:54:44 -0400 Subject: RFR (L) 7133093: Improve system dictionary performance (closed part) In-Reply-To: <89f0b98c-3cbd-7b87-d76b-5e89a5a676fb@oracle.com> References: <89f0b98c-3cbd-7b87-d76b-5e89a5a676fb@oracle.com> Message-ID: Hi, The closed part of this change is: http://oklahoma.us.oracle.com/~cphillim/webrev/7133093.src.closed.01/webrev/ Which is a relatively smaller change. This was tested with all the AppCDS tests and was the result of discussions with Ioi and the AppCDS team. Thanks! Coleen On 6/23/17 7:42 PM, coleen.phillimore at oracle.com wrote: > Summary: Implement one dictionary per ClassLoaderData for faster > lookup and removal during class unloading > > See RFE for more details. > > open webrev at http://cr.openjdk.java.net/~coleenp/7133093.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-7133093 > > Tested with full "nightly" run in rbt, plus locally class loading and > unloading tests: > > jtreg hotspot/test/runtime/ClassUnload > > jtreg hotspot/test/runtime/modules > > jtreg hotspot/test/gc/class_unloading > > make test-hotspot-closed-tonga FILTER=quick TEST_JOBS=4 > TEST=vm.parallel_class_loading > > csh ~/testing/run_jck9 (vm/lang/java_lang) > > runThese -jck - uses class loader isolation to run each jck test and > unloads tests when done (at -gc:5 intervals) > > > Thanks, > Coleen > > From coleen.phillimore at oracle.com Thu Jul 20 02:55:33 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 19 Jul 2017 22:55:33 -0400 Subject: RFR (L) 7133093: Improve system dictionary performance (closed part) In-Reply-To: References: <89f0b98c-3cbd-7b87-d76b-5e89a5a676fb@oracle.com> Message-ID: PLEASE ignore this email. Coleen On 7/19/17 10:54 PM, coleen.phillimore at oracle.com wrote: > > Hi, The closed part of this change is: > > http://oklahoma.us.oracle.com/~cphillim/webrev/7133093.src.closed.01/webrev/ > > > Which is a relatively smaller change. This was tested with all the > AppCDS tests and was the result of discussions with Ioi and the AppCDS > team. > > Thanks! > Coleen > > On 6/23/17 7:42 PM, coleen.phillimore at oracle.com wrote: >> Summary: Implement one dictionary per ClassLoaderData for faster >> lookup and removal during class unloading >> >> See RFE for more details. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/7133093.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-7133093 >> >> Tested with full "nightly" run in rbt, plus locally class loading and >> unloading tests: >> >> jtreg hotspot/test/runtime/ClassUnload >> >> jtreg hotspot/test/runtime/modules >> >> jtreg hotspot/test/gc/class_unloading >> >> make test-hotspot-closed-tonga FILTER=quick TEST_JOBS=4 >> TEST=vm.parallel_class_loading >> >> csh ~/testing/run_jck9 (vm/lang/java_lang) >> >> runThese -jck - uses class loader isolation to run each jck test and >> unloads tests when done (at -gc:5 intervals) >> >> >> Thanks, >> Coleen >> >> > From kim.barrett at oracle.com Thu Jul 20 03:35:22 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 19 Jul 2017 23:35:22 -0400 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: <14A41261-BA5F-43EE-8C31-D08DB22B0112@oracle.com> References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <596CD126.6080100@oracle.com> <596DD713.6080806@oracle.com> <77d5a9e7-b2b0-79e4-effd-903dfe226512@redhat.com> <596E1142.6040708@oracle.com> <3432d11c-5904-a446-1cb5-419627f1794f@redhat.com> <596E288B.90706@oracle.com> <773a0b64-8273-1a58-a46a-22ae18f927b7@redhat.com> <14A41261-BA5F-43EE-8C31-D08DB22B0112@oracle.com> Message-ID: <5A2B4D64-BB3A-48DA-BA5B-A8D1F127F38E@oracle.com> > On Jul 19, 2017, at 10:08 PM, Kim Barrett wrote: > Independently, we can work on the implementation, which may > eventually include using C++11 atomics, at least on some platforms. > (Where applicable, that looks like it should be pretty easy with the > new API. [2]) I forgot; one part of using C++11 atomics seems to be changing the types of a large number of variables to atomic_T or atomic or some Hotspot synonym. That?s not quite so easy :( From shafi.s.ahmad at oracle.com Thu Jul 20 05:49:57 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Wed, 19 Jul 2017 22:49:57 -0700 (PDT) Subject: [8u] RFR for backport of JDK-8177958: Possible uninitialized char* in vm_version_solaris_sparc.cpp In-Reply-To: References: <55a24e4c-3922-4a0e-983d-433701afa68c@default> Message-ID: <1a472440-d292-4269-bbc8-7e9310ce8866@default> May I get the second review for this small change. Regards, Shafi > -----Original Message----- > From: Vladimir Kozlov > Sent: Tuesday, July 18, 2017 9:27 PM > To: Shafi Ahmad ; jdk8u-dev at openjdk.java.net > Subject: Re: [8u] RFR for backport of JDK-8177958: Possible uninitialized char* > in vm_version_solaris_sparc.cpp > > Looks good. > > Thanks, > Vladimir > > On 7/17/17 11:40 PM, Shafi Ahmad wrote: > > Hi, > > > > Please review the backport of ' JDK-8177958: Possible uninitialized char* in > vm_version_solaris_sparc.cpp' to jdk8u. > > Please note that this is not a clean backport due to single line conflict. > > > > Webrev: > http://cr.openjdk.java.net/~shshahma/8177958/jdk8u/webrev.00/ > > Jdk10 bug: https://bugs.openjdk.java.net/browse/JDK-8177958 > > Original patch pushed to jdk10: > http://hg.openjdk.java.net/jdk10/hs/hotspot/rev/7295c9c26185 > > > > Test: Run jprt. > > > > Regards, > > Shafi > > From shade at redhat.com Thu Jul 20 06:39:01 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 20 Jul 2017 08:39:01 +0200 Subject: [8u] RFR for backport of JDK-8177958: Possible uninitialized char* in vm_version_solaris_sparc.cpp In-Reply-To: <1a472440-d292-4269-bbc8-7e9310ce8866@default> References: <55a24e4c-3922-4a0e-983d-433701afa68c@default> <1a472440-d292-4269-bbc8-7e9310ce8866@default> Message-ID: On 07/20/2017 07:49 AM, Shafi Ahmad wrote: > May I get the second review for this small change. > > Regards, > Shafi > >> -----Original Message----- >> From: Vladimir Kozlov >> Sent: Tuesday, July 18, 2017 9:27 PM >> To: Shafi Ahmad ; jdk8u-dev at openjdk.java.net >> Subject: Re: [8u] RFR for backport of JDK-8177958: Possible uninitialized char* >> in vm_version_solaris_sparc.cpp >> >> Looks good. >> >> Thanks, >> Vladimir >> >> On 7/17/17 11:40 PM, Shafi Ahmad wrote: >>> Hi, >>> >>> Please review the backport of ' JDK-8177958: Possible uninitialized char* in >> vm_version_solaris_sparc.cpp' to jdk8u. >>> Please note that this is not a clean backport due to single line conflict. >>> >>> Webrev: >> http://cr.openjdk.java.net/~shshahma/8177958/jdk8u/webrev.00/ This looks good. -Aleksey From shafi.s.ahmad at oracle.com Thu Jul 20 06:40:20 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Wed, 19 Jul 2017 23:40:20 -0700 (PDT) Subject: [8u] RFR for backport of JDK-8177958: Possible uninitialized char* in vm_version_solaris_sparc.cpp In-Reply-To: References: <55a24e4c-3922-4a0e-983d-433701afa68c@default> <1a472440-d292-4269-bbc8-7e9310ce8866@default> Message-ID: <15923490-67b7-41cf-87c0-b375a91c8f6a@default> Thank you Aleksey for reviewing it. Regards, Shafi > -----Original Message----- > From: Aleksey Shipilev [mailto:shade at redhat.com] > Sent: Thursday, July 20, 2017 12:09 PM > To: Shafi Ahmad ; hotspot- > dev at openjdk.java.net > Subject: Re: [8u] RFR for backport of JDK-8177958: Possible uninitialized char* > in vm_version_solaris_sparc.cpp > > On 07/20/2017 07:49 AM, Shafi Ahmad wrote: > > May I get the second review for this small change. > > > > Regards, > > Shafi > > > >> -----Original Message----- > >> From: Vladimir Kozlov > >> Sent: Tuesday, July 18, 2017 9:27 PM > >> To: Shafi Ahmad ; > >> jdk8u-dev at openjdk.java.net > >> Subject: Re: [8u] RFR for backport of JDK-8177958: Possible > >> uninitialized char* in vm_version_solaris_sparc.cpp > >> > >> Looks good. > >> > >> Thanks, > >> Vladimir > >> > >> On 7/17/17 11:40 PM, Shafi Ahmad wrote: > >>> Hi, > >>> > >>> Please review the backport of ' JDK-8177958: Possible uninitialized > >>> char* in > >> vm_version_solaris_sparc.cpp' to jdk8u. > >>> Please note that this is not a clean backport due to single line conflict. > >>> > >>> Webrev: > >> http://cr.openjdk.java.net/~shshahma/8177958/jdk8u/webrev.00/ > > This looks good. > > -Aleksey > From thomas.stuefe at gmail.com Thu Jul 20 07:11:07 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 20 Jul 2017 09:11:07 +0200 Subject: RFR(XL): 8181917: Refactor UL LogStreams to avoid using resource area In-Reply-To: References: <337330d2-512c-bc77-5a55-c05d33e376e5@oracle.com> Message-ID: Hi all, after the last irc discussion with Erik here the new webrev with the changes he requested: Full: http://cr.openjdk.java.net/~stuefe/webrevs/8181917-refactor-ul-logstream/all.webrev.05/webrev/ Delta: http://cr.openjdk.java.net/~stuefe/webrevs/8181917-refactor-ul-logstream/delta-04-to-05/webrev/ I built the change on windows (32bit too :P), Linux and AIX. I am currently re-running jtreg tests on Linux, no issues so far. The changes: - I removed all stray "Unconditional write?" comments I had used before to mark sites which did logging unconditionally. Marcus had opened a follow up bug report for that issue (https://bugs.openjdk.java.net/browse/JDK-8182466), and I added the affected sites I found to the comment section of that bug. - In g1GCPhaseTimes.cpp, I changed the logging code in G1GCPhaseTimes::debug_phase() to use LogTarget instead of Log to be consistent with G1GCPhaseTimes::trace_phase() - In the new LogStream implementation (logStream.cpp), I added temporary variables in LogStream::LineBuffer::append() and LogStream::LineBuffer::try_ensure_cap(). - I also enlarge the line buffer a bit more aggressive. We now, once the initial 64byte buffer is exhausted - enlarge by 256 bytes steps. - Finally, I renamed LineBuffer::ptr() to LineBuffer::buffer() A note about memory consumption of the LogStream line buffer: I do not think we need to overly worry about memory consumption, at least not if we think the old solution was fine. Old solution was using stringStream, which uses NEW_RESOURCE_ARRAY to enlarge the line buffer when exhausted. This is resource area, so it *always* leaks the previous allocation, which is not deallocated until the resource area stack is reset. Even worse, for larger line lengths the growth strategy in stringStream() seems to be to just enlarge by the requested amount, which means that every single write would cause a reallocation and hence a new allocation of the underlying buffer (see stringStream::write()). I am not 100% sure, this is just from dry-reading the code. Kind Regards, Thomas On Mon, Jul 17, 2017 at 3:44 PM, Thomas St?fe wrote: > Hi Eric, > > thank you for your review! > > new Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8181917- > refactor-ul-logstream/all.webrev.04/webrev/ > delta to last: http://cr.openjdk.java.net/~stuefe/webrevs/8181917- > refactor-ul-logstream/delta-03-to-04/webrev/ > > Changes: > > - I changed the allocation function to handle the case where we already > reached the reasonable_max cap. You are right, in that case reallocating is > unnecessary. > - Also, added log_info() for that case as you suggested. > - Finally, added a fix to the LogStream gtest which somehow slipped thru > the last webrev. > > Changes are now rebased to the current tip. > > Thanks! > > ..Thomas > > On Fri, Jul 14, 2017 at 12:36 PM, Erik Helin > wrote: > >> On 07/13/2017 08:46 AM, Thomas St?fe wrote: >> >>> Hi Erik! >>> >>> >>> On Mon, Jul 10, 2017 at 11:40 AM, Erik Helin >> > wrote: >>> >>> Hi Thomas, >>> >>> On 06/30/2017 08:32 PM, Thomas St?fe wrote: >>> >>> Hi Eric, >>> >>> thank you for the review! >>> >>> New >>> Version: >>> http://cr.openjdk.java.net/~stuefe/webrevs/8181917-refactor- >>> ul-logstream/all.webrev.03/webrev/ >>> >> -ul-logstream/all.webrev.03/webrev/> >>> >>> >>> I think you still have a problem with "runaway" loggers in this >>> version: >>> >>> +// try_ensure_cap tries to enlarge the capacity of the internal >>> buffer >>> +// to the given atleast value. May fail if either OOM happens or >>> atleast >>> +// is larger than a reasonable max of 1 M. Caller must not assume >>> +// capacity without checking. >>> +void LogStream::LineBuffer::try_ensure_cap(size_t atleast) { >>> + assert(_cap >= sizeof(_smallbuf), "sanity"); >>> + if (_cap < atleast) { >>> + const size_t reasonable_max = 1 * M; >>> + size_t newcap = align_size_up(atleast + 64, 64); >>> + assert(_cap <= reasonable_max, "sanity"); >>> + // Cap out at a reasonable max to prevent runaway leaks. >>> + if (newcap > reasonable_max) { >>> + newcap = reasonable_max; >>> + } >>> + >>> + char* const newbuf = (char*) os::malloc(newcap, mtLogging); >>> + if (newbuf == NULL) { // OOM. Leave object unchanged. >>> + return; >>> + } >>> + if (_pos > 0) { // preserve old content >>> + memcpy(newbuf, _buf, _pos + 1); // ..including trailing zero >>> + } >>> + if (_buf != _smallbuf) { >>> + os::free(_buf); >>> + } >>> + _buf = newbuf; >>> + _cap = newcap; >>> + } >>> + assert(_cap >= atleast, "sanity"); >>> +} >>> >>> with the above code, even though _cap > reasonable_max, you will >>> always allocate newbuf with os::malloc. For debug builds, we are >>> fine, since the JVM will assert on _cap <= reasonable_max, but for >>> product builds we are not. Would it be better to first check if _cap >>> == reasonable_max? As in the following snippet: >>> >>> void LogStream::LineBuffer::try_ensure_cap(size_t atleast) { >>> if (_cap < atleast) { >>> const size_t reasonable_max = 1 * M; >>> if (_cap == reasonable_max) { >>> // max memory used, "die" in debug builds, log in product >>> builds >>> log_warning(logging)("Max memory used for message: %s\n", >>> _buf); >>> ShouldNotReachHere(); >>> return; >>> } >>> >>> size_t newcap = align_size_up(atleast + 64, 64); >>> if (newcap > reasonable_max) { >>> newcap = reasonable_max; >>> } >>> char* const newbuf = (char*) os::malloc(newcap, mtLogging); >>> // rest of logic >>> } >>> } >>> >>> Thanks, >>> Erik >>> >>> >>> I'm not sure I understand. Are you concerned that _cap would grow beyond >>> reasonable_max - because I do not see a way this could happen. Or do you >>> want to also treat the *attempt* to grow the _cap beyond reasonable_max as >>> a loggable offence? >>> >> >> What I am trying to say is that with your current patch, even if _cap == >> reasonable_max, you will still allocate a new 1M buffer, copy all the chars >> over, and the free the old one. My proposal simply skips the allocation, >> copying and freeing if _cap == reasonable_max. The result will be the same, >> just less stress on malloc. >> >> If the latter, I am not sure about logging. If this is an error which may >>> happen at a customer site without popping up first in inhouse-tests (which >>> it conceivably is, because logging is very content dependent), I would >>> rather hide the error in production code instead of flooding stderr with >>> tons of log warnings or crashing with a guarantee, no? >>> >> >> I see what you mean, but perhaps log_info(logging) at least? I want to be >> able to discover if this happens in production, for example if a customer >> complains about truncated logs. A log_info(logging) will only be printed if >> -Xlog:logging=info is specified on the command-line. >> >> Thanks, >> Erik >> >> ..Thomas >>> >>> >>> > From muthusamy.chinnathambi at oracle.com Thu Jul 20 07:24:15 2017 From: muthusamy.chinnathambi at oracle.com (Muthusamy Chinnathambi) Date: Thu, 20 Jul 2017 00:24:15 -0700 (PDT) Subject: [8u] RFR for backport of JDK-8158012: Use SW prefetch instructions instead of BIS for allocation prefetches on SPARC Core C4 Message-ID: <4768675b-cf31-4351-902a-51ab0a4bd5c7@default> Hi, Please review the backport of bug: "JDK-8158012: Use SW prefetch instructions instead of BIS for allocation prefetches on SPARC Core C4" to jdk8u-dev Please note that this is not a clean backport due to conflict in comments and slight re-order of code. Webrev: http://cr.openjdk.java.net/~shshahma/Muthu/8158012/webrev.00/ jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-8158012 Original patch pushed to jdk9: http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/eebdc7acebd4 Test: Ran jprt and jtreg. Regards, Muthusamy C From erik.osterlund at oracle.com Thu Jul 20 13:45:33 2017 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 20 Jul 2017 15:45:33 +0200 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: <773a0b64-8273-1a58-a46a-22ae18f927b7@redhat.com> References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <596CD126.6080100@oracle.com> <596DD713.6080806@oracle.com> <77d5a9e7-b2b0-79e4-effd-903dfe226512@redhat.com> <596E1142.6040708@oracle.com> <3432d11c-5904-a446-1cb5-419627f1794f@redhat.com> <596E288B.90706@oracle.com> <773a0b64-8273-1a58-a46a-22ae18f927b7@redhat.com> Message-ID: <5970B3FD.607@oracle.com> Hi Andrew, On 2017-07-19 18:21, Andrew Haley wrote: > On 18/07/17 16:26, Erik ?sterlund wrote: >> >> On 2017-07-18 16:41, Andrew Haley wrote: >>> On 18/07/17 14:46, Erik ?sterlund wrote: >>>> On 2017-07-18 11:57, Andrew Haley wrote: >>>>> On 18/07/17 10:38, Erik ?sterlund wrote: >>> It is more general than that. If you access the stored value of an >>> object through a glvalue of other than one of the allowed types, then >>> your *whole program is undefined*. >> And I think it is in that list of allowed types. Specifically, it is my >> interpretation of the standard that intptr_t is "a type that is the >> signed or unsigned type corresponding to the dynamic type of the object" >> (cf. 5.2.10) if the dynamic type is e.g. void*. >> >>>> and that therefore it is fine to load something (that was stored as >>>> A*) as intptr_t and subsequently cast it to A* before the A itself >>>> is being accessed. Am I wrong? >>> No, that is correct. The question is whether intptr_t is a type >>> compatible with a pointer type, and I don't think you will find any >>> language to the effect that it is. >> I think 5.2.10 describing reinterpret_cast seems to suggest that they >> are compatible by saying "A pointer converted to an integer of >> sufficient size (if any such exists on the implementation) and back to >> the same pointer type will have its original value". That is precisely >> what is done. > This is the core of our disagreement. > > In a sense, it doesn't matter which of us is right; what really > matters is the opinion of the compiler. GCC does not put intptr_t and > pointer types into the same alias set, and even if this did turn out > to be a bug in GCC it wouldn't help us because we often use > unsupported old versions of GCC to build OpenJDK, so it wouldn't get > fixed anyway. > > So what are we to do? Do you believe me when I say that GCC does > treat A* and intptr_t to be compatible types? If not, what would it > take to convince you? It seems like the aliasing problem has multiple levels at which we can debate. I will try to put my thoughts down in a reasonably structured way... let's see how that goes. These are the core questions we are debating (and I will answer them in order): 1) What does the standard say about aliasing rules? 2) What did concrete implementations like GCC implement the aliasing rules? 3) What are the implications of those aliasing rules being violated? Does it matter? 4) Have these changes actually changed anything relating to those aliasing rules? 5) How do we proceed? Regarding #1: We seem to agree that there are various possible interpretations of the standard (wouldn't be the first time...); at least one interpretation that violates treating intptr_t as compatible with A* and at least one interpretation that permits it. I do not know if there is any one true interpretation of the C++ standard, so I will assume the possibility that either one of us could be right here, and hence that a compiler writer might write a compiler in which it is either permitted aliasing or not. Regarding #2: While the simplest most contrived example of what we are arguing about being a strict aliasing violation is not caught as a strict aliasing violation using the strictest strict aliasing checks available in GCC, I feel dubious, but will take your word for it that intptr_t and A* are not compatible types. So for the rest of the discussion, I will assume this is true. Regarding #3: We already inherently rely on passed in intptr_t or even jlong aliasing A*, possibly today enforced through -fno-strict-aliasing. I do not think the JVM will ever be able to turn such optimizations on. Our C++ runtime lives in the border lands between Java and C++ that I will refer to as "shady land". C++ standards are relatively successful at describing well-behaved semantics for a program where C++ code interacts with other C++ code. Unfortunately, we are working in "shady land" where C++ code also interacts with JIT-compiled machine code from Java land which not surprisingly has poorly defined meaning in the context of C++ standards. For example, we take pointers from C/C++, send them as jlong via JNI into Java that may pass it around and ultimately store it in some Java long field of an object (with a store or perhaps an Unsafe CompareAndSetLong call that may or may not be implemented via an intrinsic), and ultimately be passed in to C++ code that loads that long as jlong, and casts it into a pointer and starts poking around in values behind that pointer (e.g. Unsafe_SetMemory0(), Copy::conjoint_jlongs_atomic(), jni_NewDirectByteBuffer(), etc). In fact, the type for "native pointers" in Java is long, a.k.a. jlong, and I am fairly certain that the code base is sufficiently full of this that pointer sized integers and pointers *have* to be aliased by the compiler or we are in deep trouble. I do not think we can ever allow them not to be aliased, especially not in "shady land". So if compilers ever started assuming that there is no way anything loaded as an intptr_t, or jlong for that matter, could point at any C++ types (despite being casted to do so), and perform aggressive optimizations utilizing points-to analysis, relying on those assumptions, then that would be so fundamentally broken that we could arguably not recover. This goes without appreciating just how much other code in hotspot already relies on this. While it is possible to try to isolate code in "shady land" from the rest of the C++ code, it is precisely in "shady land" that Atomic and OrderAccess live. They are first class citizens of "shady land". They poke around in memory that is being concurrently accessed either by other C++ code or by JIT-compiled Java code (possibly performing atomics with jlongs that are in fact native pointers passed in from JNI). Here the aliasing must be as conservative as possible. My conclusion is that if intptr_t/jlong is not considered compatible with A* by the compiler, the compiler has to be tamed to think so anyway, or the JVM will break horribly. Especially for Atomic and OrderAccess that lives in "shady land". Regarding #4: I have made changes in the connection between the frontend and the backend. But I would like to maintain that the backends retain the same behaviour as they did before. For example, your xchg_ptr for void* casts (C-style) the exchange_value to intptr_t and use the intptr_t variant of xchg_ptr that passes intptr_t (a.k.a. int64_t on your architecture) into _sync_lock_test_and_set. So whatever hypothetical aliasing problem the new Atomic implementation class were still there before these changes. I have not introduced any of those aliasing issues, only improved the mediation between the frontend to the platform dependent backend. Regarding #5: Now as for how to move on with this, I hope we can agree that for code in "shady land", the compiler simply has to treat intptr_t as possible aliases to A* and that if it does not, then we need to tame it to do so. Because the C++ compiler simply does not know the full story of what is going on in our program and can not safely make any meaningful points-to analysis and data flow optimizations crossing dynamically generated JIT-compiled machine code of a different language. I propose three different solutions that we can discuss: Solution A: Do nothing. With this solution we recognize that a) we have actually not introduced any undefined behaviour that was not already there - the backend used intptr_t before and continues to do so, and b) doing so is safe (and inherently has to be safe) due to other levels of safeguarding such as turning off strict aliasing and using volatile. Solution B: If intptr_t does indeed not alias A*, we could find a pointer sized type, let's call it CanonicalPointer, that does indeed alias a generic A*, and pass pointer types from the frontend as CanonicalPointer to the backend. For example char* is explicitly described in the standard as an exception of the rules that is explicitly aliased to a generic A* (hopefully we agree about that). I have put together a small example of how xchg might look with this model here: http://cr.openjdk.java.net/~eosterlund/8184334/webrev.01_alias_pointer_A/ The frontend canonicalizes types and sends them to the backend. The canonical type of any pointer is char*, which explicitly aliases to all other pointers. Passing pointers through jlong (which is actually done) is possibly still not safe in this variant. Solution C: Go even further (yet with less code) and continue sending canonical integer types into the backend, and leave it to the backend to decide what measures need to be taken to trick the compiler into not performing dangerous optimizations (in the hypothetical scenario that we turn on strict aliasing). For example on AArch64 linux treating int64_t as char* that explicitly aliases all pointers (3.10), relying on the roundtrip guarantees of reinterpret_cast guarantees (5.2.10) between pointers and intptr_t. A code example, again for xchg, can be found here: http://cr.openjdk.java.net/~eosterlund/8184334/webrev.01_alias_pointer_B/ This solution also allows atomically loading a jlong and subsequently casting it to a pointer (because its backing load access used char*), which I argue inherently needs to be allowed. My preference is solution A for now (as no new violations are actually introduced with these changes, conversely the part of the pipeline from the C++ runtime to the backend in "shady land" has been purged from evil), and possibly looking into C later, on a platform by platform basis, to further strengthen aliasing in "shady land". I am curious to hear your thoughts about this. Thanks, /Erik >> Conversely, I do not find any language suggesting that intptr_t would >> not be compatible with pointer types. >> >>> Pointer types are distinct from one another and from all integer >>> types. People sometimes get confused by this: the fact that you can >>> cast from A* to e.g. void*, doesn't tell you that you can cast from >>> A** to void** and use the result: they are different types. >> Agreed. But I do not see any problem with reinterpret_cast from A* to >> intptr_t back to A* and using the A from that result. That is explicitly >> allowed as per 5.2.10 as previously described. > I don't think that's what you're doing. I think you're casting from > A** to intptr_t*, dereferencing the intptr_t*, and casting the result > to A*. > >> That looks like the correct (and indeed expected) output. iptr should be >> the same as &b, and iptr->_x should be 0, and &a should be different >> from &b and iptr. I tried this experiment locally too with similar >> correct result. Am I missing something here? > No, I was; I misinterpreted the result, and it's a red herring. > Sorry. > > I believe this patch to be dangerous. We might well get away with it, > given sufficient larding of volatile, but it's still undefined > behaviour. We have far too much UB in HotSpot already, and we should > not be adding any more. > > And I still believe that this patch is more complex than is needed, > and with many (if not all) targets we could do something much simpler, > and which does not need to cast pointer types. It could also map > cleanly onto C++11. > From alexander.harlap at oracle.com Thu Jul 20 15:31:20 2017 From: alexander.harlap at oracle.com (Alexander Harlap) Date: Thu, 20 Jul 2017 11:31:20 -0400 Subject: Request for review for JDK-8184045 - TestSystemGCWithG1.java times out on Solaris SPARC Message-ID: Please review changes forJDK-8184045 - TestSystemGCWithG1.java times out on Solaris SPARC (labeled as integration blocker) Proposed change will avoid timeout failure for new stress tests TestSystemGCWithG1.java, TestSystemGCWithCMS.java and TestSystemGCWithSerial.java regularly observed on Solaris Sparc Change is located at http://cr.openjdk.java.net/~aharlap/8184045/webrev.00/ Alex From coleen.phillimore at oracle.com Thu Jul 20 15:47:33 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 20 Jul 2017 11:47:33 -0400 Subject: RFR (L) 7133093: Improve system dictionary performance In-Reply-To: <233c03e0-8958-7676-8e33-5d8f0ec4edf3@oracle.com> References: <89f0b98c-3cbd-7b87-d76b-5e89a5a676fb@oracle.com> <9243da7c-a8e5-4024-c12e-cba9d4f424c8@oracle.com> <7814C032-0D4E-4E76-B8AD-62A072E8A625@oracle.com> <233c03e0-8958-7676-8e33-5d8f0ec4edf3@oracle.com> Message-ID: <8919091c-a896-c8fc-f966-68cf2348fe93@oracle.com> Hi, I was wrong about logging below. We only log the size of the hashtables when verifying them, but not in production where you'd want them. I added RFE to do this: https://bugs.openjdk.java.net/browse/JDK-8184994 Thanks, Coleen On 7/18/17 10:56 AM, coleen.phillimore at oracle.com wrote: > > Karen, Thank you for reviewing the code and for the discussions and > consulting along the way. > > On 7/17/17 4:10 PM, Karen Kinnear wrote: >> Feedback on the initial web rev: >> Thank you for using web rev - that way I can do searches of the >> overall patch! >> >> First - many thanks for doing this exercise so carefully, for >> following the specification in continuing >> to record the initiating loader and for splitting this effort into >> multiple phases to allow further exploration >> of the current expectations/behavior relative to class unloading. >> >> And thank you for all the testing. >> >> Just wanted to ensure that I understand some of the assumptions: >> >> 1) lock free reads of dictionaries - yes > Yes. >> 2) lock free adds of entries to dictionaries - yes > No, adds still take out the SystemDictionary_lock. >> 3) removal of entries or entire dictionaries - at safe point only > Yes. >> 4) rehashing - none today and none planned > Yes. >> 5) resizing of dictionaries - none today, some planned > Yes. >> 6) use of SystemDictionary_lock: essentially unchanged > Yes. >> 7) atomicity guarantees relative to checking loader constraints and >> updates to the dictionary: unchanged > Yes. >> >> I did not review the SA/vmStructs - sounds like you had Jini George >> do that - great. >> >> Minor questions/comments: >> >> SystemDictionary::verify: "Verify of system dictionary failed" -> >> looks like "Verify of placeholders failed" > > Fixed. Good find. It's an odd assert: > > guarantee(placeholders()->number_of_entries() >= 0, > "Verify of placeholders failed"); > >> >> thank you for asserts dictionary != NULL >> Dictionary::print >> is it worth an assert that loader_data != NULL? > > Added. It's already asserted in Dictionary::verify() > >> Can you please sanity check if there are other cases in which we need >> to use dictionary_or_null() rather than dictionary()? >> - are there cases in which we could be using a dictionary from an >> anonymous class? >> e.g. there are a lot of places in systemDictionary.cpp where we >> call class_loader_data(class_loader); loader_data->dictionary(); >> - was the lazily allocated dictionary allocated yet (see next >> question) > > There's an assert in dictionary() that the CLD isn't anonymous. In > these calls we get class_loader_data from the class_loader which is > always the host class loader for anonymous classes, so this doesn't > break with the current design. > >> >> CLD.cpp >> Would it be less complex to eagerly create the dictionary for a CLD >> since it is extremely rare to not need one for non-anonymous CLDs? >> You could create before publishing the CLD > > Yes, I agree it would be better this way. I was looking at some data > and it does seem rare to have a CLD with no classes. >> >> Terminology: >> How are you still using System Dictionary term? >> It still shows up in some comments and in some printed strings > > Bad habit, we've been calling it that for years. :) I'll update the > comments where it makes sense. > I tried to refer to the ClassLoaderDataGraph dictionaries collectively > as the "System Dictionary", as well as the loader constraints and > placeholder tables, and the other things that SystemDictionary contains. > > I guess it gets confusing, but it's less awkward than "class loader > data dictionaries", and more meaningful than simply "dictionaries". So > that's why I didn't change a lot of the comments. > >> >> biasedLocking.cpp, line 59 >> jvmtiGetLoadedClasses: line 298 >> dictionary.hpp lines 113,114, 120 >> dictionary.cpp 262-263, 478 >> dictionary.cpp 491 >> fix comments e.g. "Iterate the system dictionary" ... >> - probably worth checking for other comments in the sources too >> unless these all appear to match new usage model > > Fixed. I've found a couple more and some other comments that are > incorrect. >> >> hashTable.hpp >> line 268: what does this comment mean? >> "Don't create and use freelist"? >> I assume this means that this is a direct malloc, not using any block >> allocation, e.g. metaspace chunks, and this requires explicit freeing? >> > > // Don't create and use freelist of HashtableEntry. > > The default Hashtable<> allocates entries from a block of entries and > uses a freelist for freed entries. > >> >> dictionary.hpp line 161 >> could you possibly clean up comment while you are in here? >> I presume "a protection" is "a protection domain"? > > Fixed. > >> >> dictionary.cpp >> why change from Hashtable::free_entry to explicit >> unlink_entry/FREE_C_HEAP_ARRAY? > > I used the same code as the ModuleTable. The default hashtable > doesn't delete the whole thing and left some memory around, if I > remember correctly. I'll recheck it. >> >> Future optimization: >> I think that none of the built-in loaders is ever recorded as an >> initiating >> loader for any class in a loader which permits unloading. >> Note: we need to ensure that built-in includes boot, platform and >> application class loader (and clean up any confusion with system >> class loader - which can be set by the user to a custom class loader) >> > I think AppClassLoader can initiate loading and is a builtin loader. > > Dictionary for class loader class loader 0x00007f6abc6e3210 a > 'jdk/internal/loader/ClassLoaders$AppClassLoader'{0x0000000101c8df00} > Java system dictionary (table_size=107, classes=1) > ^ indicates that initiating loader is different from defining loader > 106: ^java.util.ArrayList, loader NULL class_loader > >> >> Any chance of changing DictionaryEntry* probe->klass() to be >> probe->instanceKlass()? > > Okay. Changed to instance_klass() as per coding standard. >> >> Thank you for assertion _is_at_safepoint for try_get_next_class >> >> CLD.cpp >> If the names of the classes_do vs. all_entries_do were modified to be >> defining_loader_classes_do vs. all_entries_classes_do or some way >> to make the distinction in the name that would be helpful. >> Otherwise, if you could please add comments to the >> CLDG::dictionary_classes_do >> etc. to distinguish these. > > I added comments to distinguish these. >> >> CLD.cpp >> lines 1200-1201 >> Can you please fix the comment? >> I think you need separate comments for the ModuleEntry >> reads/PackageEntry's >> exports which are looking for dead modules. >> IIUC, the do_unloading is actually looking for classes for which this >> is the initating loader, but their defining loader is dead. > // Remove entries in the dictionary of live class loader that have > // initiated loading classes in a dead class loader. > data->dictionary()->do_unloading(); > > Moved the module comment to before the module code and added this for > dictionary->do_unloading() call. >> >> systemDictionary.cpp: 445-447 >> THANK you for the comments about the java_mirror handle keeping the >> class >> and class loader alive >> Could you clarify that the class_loader handle is not for the class >> we are checking, but rather for the initiating, i.e. requesting >> class loader >> (is that accurate?) So we are keeping both the requestor and the >> requested class alive > > Yes, that is true. I added > > // The class_loader handle passed in is the initiating loader. > >> >> You removed the comment that compute_hash could lead to a safepoint/GC >> Is that not possible? >> comment in ObjectSynchronizer::FastHashCode says that with >> UseBiasedLocking >> we can revoke biases and must be able to handle a safepoint. > > I made it not possible with this change. The hashcode is now > computed by the Symbol* of the Klass name, which is random (os::random > plus some bits from the address and first two characters, which we did > a study of for bug https://bugs.openjdk.java.net/browse/JDK-8181450). > > Also, I changed ClassLoaderData::identity_hash() to use the address >> > 3 because the addresses of class loader datas are random enough for > the one use of this function. Previously, the loader_data identity > hash used to go to the class_loader oop and get to > ObjectSynchronizer::FastHashCode (which was also os::random but now > changed). > > The dictionary hashing doesn't need to add in the hash for the > loader_data now. >> >> systemDictionary.cpp: >> NoSafepointVerifier >> If we add resizing logic later, and that happens at a safepoint, >> then the number of buckets change and the d_index is no longer >> valid >> so we would need to add back the NSpV? >> - this is a more general problem if compute_hash can lead to a >> safepoint/GC > > Yes this is true, except that compute_hash doesn't safepoint. > Computing the hash and index should be done together with the > dictionary operations. The function resolve_instance_class_or_null > computes the hash and index once at the beginning and reuses it for > several lookups, which may or may not affect performance. So we need > to measure this in isolation before changing this. >> >> systemDictionary.cpp: parse_stream >> why did you remove the record_dependency? > > I realized it's trivially unnecessary. The class_loader is reached > by the ClassLoaderData::_class_loader field, so we don't have to > record a dependency from host_klass->class_loader in the anonymous > class loader data. (I just checked this again because it's confusing). > >> >> Who calls Dictionary::~Dictionary? > > The ClassLoaderData::~ClassLoaderData() calls delete _dictionary, > which calls the destructor. >> >> Performance: >> Do you have performance runs of our existing benchmarks? > > Yes, I ran our internal benchmarks and added links to the internal > wiki page. I'll rerun them again before checkin. > >> Initial key requirement is that this not slow down throughput through >> slowing down lookups. >> I know you are very aware of longer-term goals for reducing >> safepoint pause >> times and not increasing footprint significantly. >> >> hashtable sizes: future rfe? >> given that we now have a lot more dictionaries, and they perhaps need >> individual sizing >> - would it be helpful to add statistics information about >> - e.g. lookup_count, lookup_length, _lookup_warning >> - i.e. number of entries, max lookup depth, etc. to use later for >> tuning? > > All the previous statistics didn't make very much sense to us, so they > were removed. There is now logging for all the hashtables: > > busaa027.us.oracle.com% java -Xlog:hashtables -version > java version "10-internal" > Java(TM) SE Runtime Environment (fastdebug build > 10-internal+0-2017-07-17-1456035.cphillim.10system-dictionary2) > Java HotSpot(TM) 64-Bit Server VM (fastdebug build > 10-internal+0-2017-07-17-1456035.cphillim.10system-dictionary2, mixed > mode) > [0.284s][info][hashtables] System Dictionary for > jdk/internal/loader/ClassLoaders$AppClassLoader max bucket size 1 > bucket 91 element count 1 table size 107 > [0.285s][info][hashtables] System Dictionary for max > bucket size 4 bucket 263 element count 468 table size 1009 > [0.285s][info][hashtables] Placeholder Table max bucket size 0 bucket > 0 element count 0 table size 1009 > [0.285s][info][hashtables] Protection Domain Table max bucket size 0 > bucket 0 element count 0 table size 1009 > >> - might make sense to make this JFR data? > > I'll file an RFE to add tracing data with our tools. I'll need some > help with this part but the data is there. >> >> Might think about a follow-up with more tuning parameters than >> a single global PredictedLoadedClassCount. >> > > I filed RFE to work on resizing, as this work was almost complete, but > didn't show performance improvements at the time. It would be nice to > have customers not need tuning parameters, but I guess these things > are sometimes necessary. > > Thank you for the thorough review and all of your help with this! I've > made these changes and will rerun testing and send out a new webrev > and an incremental webrev. > > Thanks! > Coleen > >> >> thanks, >> Karen >> >> >>> On Jul 17, 2017, at 2:37 PM, coleen.phillimore at oracle.com >>> wrote: >>> >>> >>> Hi, I have made changes for Ioi's comments below. The largest >>> change since .01 is that PredictedLoadedClassCount is used to size >>> dictionaries for class loader data, and should be used to specify >>> the average number of classes loaded by each class loader. Since >>> this is an option available via UnlockExperimentalVMOptions, I'm not >>> sure what the process is for changing the meaning of this option. We >>> will work with our customers who have used this option for >>> specifying the total number of classes loaded. We are also working >>> on the change to allow automatic resizing of the dictionaries during >>> safepoints. >>> >>> See: >>> open webrev at http://cr.openjdk.java.net/~coleenp/7133093.02/webrev >>> >>> >>> Tested with the full nightly test set (jdk, nsk, hotspot jtreg >>> tests) on linux x64. >>> >>> Thanks, >>> Coleen >>> >>> On 6/30/17 4:45 PM, coleen.phillimore at oracle.com >>> wrote: >>>> >>>> Ioi, Thank you for looking at this change. >>>> >>>> On 6/30/17 2:39 PM, Ioi Lam wrote: >>>>> Hi Coleen, >>>>> >>>>> Maybe the bug should be renamed to "Use one Dictionary per class >>>>> loader instance"? That way it's more obvious what it is when you >>>>> look at the repo history. >>>> >>>> I can do that. I made it One Dictionary per ClassLoaderData >>>>> >>>>> 1. >>>>> >>>>> Why is assert_locked_or_safepoint(SystemDictionary_lock) necessary >>>>> in SystemDictionary::find_class (line 1826), but not necessary >>>>> SystemDictionary::find (line 951)? Since you removed >>>>> NoSafepointVerifier nosafepoint in the latter, maybe this means >>>>> it's safe to remove the assert_locked_or_safepoint in the former? >>>> >>>> The call to SystemDictionary::find() is the (I believe) usual lock >>>> free lookup and the SystemDictionary::find_class() is used to >>>> verify that a class we're about to add or want to add hasn't been >>>> added by another thread. Or certain cases where we already have a >>>> lock to do something else, like add loader constraints. I took out >>>> the NoSafepointVerifier because it assumes that system dictionary >>>> entries would be moved by GC, which they won't. The old hash >>>> function could safepoint when getting the hash for the class_loader >>>> object. >>>> >>>>> >>>>> 2. >>>>> >>>>> 455 static ClassLoaderData* _current_loader_data = NULL; >>>>> 456 static Klass* _current_class_entry = NULL; >>>>> 457 >>>>> 458 InstanceKlass* ClassLoaderDataGraph::try_get_next_class() { >>>>> >>>>> How about moving the static fields into an iterator object. That >>>>> way you don't need to keep track of the globals >>>>> >>>>> ClassLoaderDataGraphIterator { >>>>> ClassLoaderData* _current_loader_data >>>>> Klass* _current_class_entry; >>>>> >>>>> InstanceKlass* try_get_next_class() { ....} >>>>> }; >>>> >>>> Ok, there's a different iterator that iterates over all of the >>>> classes for GC. I will adapt that for this use. That would be >>>> better. >>>> >>>>> >>>>> 3. Double check locking in ClassLoaderData::dictionary() -- >>>>> someone else should look at this :-) >>>> >>>> I copied code that David Holmes added for modules. >>>> >>>>> >>>>> 4. We may need a better strategy for deciding the size of each >>>>> dictionary. >>>>> >>>>> 565 const int _primelist[10] = {1, 107, 1009}; >>>>> 571 Dictionary* ClassLoaderData::dictionary() { >>>>> 579 if ((dictionary = _dictionary) == NULL) { >>>>> 580 int size; >>>>> 581 if (this == the_null_class_loader_data() || >>>>> is_system_class_loader_data()) { >>>>> 582 size = _primelist[2]; >>>>> 583 } else if >>>>> (class_loader()->is_a(SystemDictionary::reflect_DelegatingClassLoader_klass())) >>>>> { >>>>> 584 size = _primelist[0]; >>>>> 585 } else { >>>>> 586 size = _primelist[1]; >>>>> 587 } >>>>> 588 dictionary = new Dictionary(this, size); >>>>> >>>>> I'll do some investigation on this issue and get back to you. >>>>> >>>> >>>> How about if someone uses PredictedLoadedClassCount, then we use >>>> that to size all but the reflection and boot class loader? Then if >>>> there's an application that has a class loader with a huge amount >>>> of classes loaded in it, that would help this? It might cost some >>>> footprint but an oversized table would simply be a bigger array of >>>> pointers, so it might not be that bad to oversize. >>>> >>>> I think the long term solution is being able to resize these >>>> entries and make the application provide arguments. Please let me >>>> know what you find in your investigation and if that would work. >>>>> The rest of the changes look good to me. >>>> >>>> Thank you! >>>> Coleen >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> >>>>> On 6/23/17 4:42 PM, coleen.phillimore at oracle.com >>>>> wrote: >>>>>> Summary: Implement one dictionary per ClassLoaderData for faster >>>>>> lookup and removal during class unloading >>>>>> >>>>>> See RFE for more details. >>>>>> >>>>>> open webrev at >>>>>> http://cr.openjdk.java.net/~coleenp/7133093.01/webrev >>>>>> >>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-7133093 >>>>>> >>>>>> Tested with full "nightly" run in rbt, plus locally class loading >>>>>> and unloading tests: >>>>>> >>>>>> jtreg hotspot/test/runtime/ClassUnload >>>>>> >>>>>> jtreg hotspot/test/runtime/modules >>>>>> >>>>>> jtreg hotspot/test/gc/class_unloading >>>>>> >>>>>> make test-hotspot-closed-tonga FILTER=quick TEST_JOBS=4 >>>>>> TEST=vm.parallel_class_loading >>>>>> >>>>>> csh ~/testing/run_jck9 (vm/lang/java_lang) >>>>>> >>>>>> runThese -jck - uses class loader isolation to run each jck test >>>>>> and unloads tests when done (at -gc:5 intervals) >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Coleen >>>>>> >>>>>> >>>>> >>>> >>> >> > From daniel.daugherty at oracle.com Thu Jul 20 16:37:34 2017 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 20 Jul 2017 10:37:34 -0600 Subject: Request for review for JDK-8184045 - TestSystemGCWithG1.java times out on Solaris SPARC In-Reply-To: References: Message-ID: <3635f59f-142b-1241-7dbf-2617ac420f9d@oracle.com> On 7/20/17 9:31 AM, Alexander Harlap wrote: > Please review changes forJDK-8184045 > - > TestSystemGCWithG1.java times out on Solaris SPARC (labeled as > integration blocker) > > Proposed change will avoid timeout failure for new stress tests > TestSystemGCWithG1.java, TestSystemGCWithCMS.java and > TestSystemGCWithSerial.java regularly observed on Solaris Sparc > > Change is located at > http://cr.openjdk.java.net/~aharlap/8184045/webrev.00/ test/gc/stress/systemgc/TestSystemGC.java L32: import jdk.test.lib.Platform; Not sure why this was added. (old) L187: runAllPhases(); So your theory is that calling runAllPhases() twice takes too much time on the Solaris SPARC machines. What are the average times before and after making this change on Solaris SPARC? Thumbs up! Don't need to see a new webrev if you decide to delete the new import. Dan > > > Alex > From alexander.harlap at oracle.com Thu Jul 20 17:28:48 2017 From: alexander.harlap at oracle.com (Alexander Harlap) Date: Thu, 20 Jul 2017 13:28:48 -0400 Subject: Request for review for JDK-8184045 - TestSystemGCWithG1.java times out on Solaris SPARC In-Reply-To: <3635f59f-142b-1241-7dbf-2617ac420f9d@oracle.com> References: <3635f59f-142b-1241-7dbf-2617ac420f9d@oracle.com> Message-ID: <294d8fb0-70b5-6483-ecdd-7078afe4f251@oracle.com> Hi Dan, Thank you for your review. I will remove erroneous extra import. Running jtreg test TestSystemGCWithG1.java (outside of aurora) on Solaris Sparc takes about 2900 sec (before change) and is twice faster after change. With aurora it will be slower, but should be enough to eliminate timeout. Alex On 7/20/2017 12:37 PM, Daniel D. Daugherty wrote: > On 7/20/17 9:31 AM, Alexander Harlap wrote: >> Please review changes forJDK-8184045 >> - >> TestSystemGCWithG1.java times out on Solaris SPARC (labeled as >> integration blocker) >> >> Proposed change will avoid timeout failure for new stress tests >> TestSystemGCWithG1.java, TestSystemGCWithCMS.java and >> TestSystemGCWithSerial.java regularly observed on Solaris Sparc >> >> Change is located at >> http://cr.openjdk.java.net/~aharlap/8184045/webrev.00/ > > test/gc/stress/systemgc/TestSystemGC.java > L32: import jdk.test.lib.Platform; > Not sure why this was added. > > (old) L187: runAllPhases(); > So your theory is that calling runAllPhases() twice > takes too much time on the Solaris SPARC machines. > > What are the average times before and after making > this change on Solaris SPARC? > > Thumbs up! Don't need to see a new webrev if you decide > to delete the new import. > > Dan > >> >> >> Alex >> > From daniel.daugherty at oracle.com Thu Jul 20 17:34:07 2017 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 20 Jul 2017 11:34:07 -0600 Subject: Request for review for JDK-8184045 - TestSystemGCWithG1.java times out on Solaris SPARC In-Reply-To: <294d8fb0-70b5-6483-ecdd-7078afe4f251@oracle.com> References: <3635f59f-142b-1241-7dbf-2617ac420f9d@oracle.com> <294d8fb0-70b5-6483-ecdd-7078afe4f251@oracle.com> Message-ID: On 7/20/17 11:28 AM, Alexander Harlap wrote: > Hi Dan, > > Thank you for your review. > > I will remove erroneous extra import. > > Running jtreg test TestSystemGCWithG1.java (outside of aurora) on > Solaris Sparc takes about 2900 sec (before change) and is twice faster > after change. > > With aurora it will be slower, but should be enough to eliminate timeout. Thanks for the info. All four variants are already run with: * @run main/othervm/timeout=300 ... so they aren't run in parallel with anything else... I think this is the best you can do... Dan > > Alex > > On 7/20/2017 12:37 PM, Daniel D. Daugherty wrote: >> On 7/20/17 9:31 AM, Alexander Harlap wrote: >>> Please review changes forJDK-8184045 >>> - >>> TestSystemGCWithG1.java times out on Solaris SPARC (labeled as >>> integration blocker) >>> >>> Proposed change will avoid timeout failure for new stress tests >>> TestSystemGCWithG1.java, TestSystemGCWithCMS.java and >>> TestSystemGCWithSerial.java regularly observed on Solaris Sparc >>> >>> Change is located at >>> http://cr.openjdk.java.net/~aharlap/8184045/webrev.00/ >> >> test/gc/stress/systemgc/TestSystemGC.java >> L32: import jdk.test.lib.Platform; >> Not sure why this was added. >> >> (old) L187: runAllPhases(); >> So your theory is that calling runAllPhases() twice >> takes too much time on the Solaris SPARC machines. >> >> What are the average times before and after making >> this change on Solaris SPARC? >> >> Thumbs up! Don't need to see a new webrev if you decide >> to delete the new import. >> >> Dan >> >>> >>> >>> Alex >>> >> > From vladimir.kozlov at oracle.com Thu Jul 20 22:44:56 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 20 Jul 2017 15:44:56 -0700 Subject: [10] RFR(XS) 8185013: Disable subword vector analysis optimization until 8184995 is fixed Message-ID: https://bugs.openjdk.java.net/browse/JDK-8185013 Switch UseSubwordForMaxVector flag off by default. src/share/vm/opto/c2_globals.hpp @@ -192,7 +192,7 @@ "of rounds of unroll,optimize,..") \ range(0, max_jint) \ \ - product(bool, UseSubwordForMaxVector, true, \ + product(bool, UseSubwordForMaxVector, false, \ "Use Subword Analysis to set maximum vector size") \ \ develop(intx, UnrollLimitForProfileCheck, 1, \ Thanks, Vladimir From daniel.daugherty at oracle.com Thu Jul 20 22:47:27 2017 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 20 Jul 2017 16:47:27 -0600 Subject: [10] RFR(XS) 8185013: Disable subword vector analysis optimization until 8184995 is fixed In-Reply-To: References: Message-ID: Thumbs up! Dan On 7/20/17 4:44 PM, Vladimir Kozlov wrote: > https://bugs.openjdk.java.net/browse/JDK-8185013 > > Switch UseSubwordForMaxVector flag off by default. > > src/share/vm/opto/c2_globals.hpp > @@ -192,7 +192,7 @@ > "of rounds of unroll,optimize,..") \ > range(0, max_jint) \ > \ > - product(bool, UseSubwordForMaxVector, true, \ > + product(bool, UseSubwordForMaxVector, false, \ > "Use Subword Analysis to set maximum vector size") \ > \ > develop(intx, UnrollLimitForProfileCheck, 1, \ > > Thanks, > Vladimir From vladimir.kozlov at oracle.com Thu Jul 20 22:51:54 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 20 Jul 2017 15:51:54 -0700 Subject: [10] RFR(XS) 8185013: Disable subword vector analysis optimization until 8184995 is fixed In-Reply-To: References: Message-ID: Thank you, Dan. I think one reviewer is enough for this. I am pushing it. Vladimir On 7/20/17 3:47 PM, Daniel D. Daugherty wrote: > Thumbs up! > > Dan > > > On 7/20/17 4:44 PM, Vladimir Kozlov wrote: >> https://bugs.openjdk.java.net/browse/JDK-8185013 >> >> Switch UseSubwordForMaxVector flag off by default. >> >> src/share/vm/opto/c2_globals.hpp >> @@ -192,7 +192,7 @@ >> "of rounds of unroll,optimize,..") \ >> range(0, max_jint) \ >> \ >> - product(bool, UseSubwordForMaxVector, true, \ >> + product(bool, UseSubwordForMaxVector, false, \ >> "Use Subword Analysis to set maximum vector size") \ >> \ >> develop(intx, UnrollLimitForProfileCheck, 1, \ >> >> Thanks, >> Vladimir > From vladimir.kozlov at oracle.com Thu Jul 20 23:07:39 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 20 Jul 2017 16:07:39 -0700 Subject: [8u] RFR for backport of JDK-8158012: Use SW prefetch instructions instead of BIS for allocation prefetches on SPARC Core C4 In-Reply-To: <4768675b-cf31-4351-902a-51ab0a4bd5c7@default> References: <4768675b-cf31-4351-902a-51ab0a4bd5c7@default> Message-ID: The indent for next comment line in vm_version_sparc.cpp is off: // Use BIS instr And AllocatePrefetchStyle set to 3 should be outside new check (should be set for both cases). In JDK 9 there is following code which set it for AllocatePrefetchInstr == 1 which JDK8 does not have: http://hg.openjdk.java.net/jdk9/jdk9/hotspot/file/eebdc7acebd4/src/cpu/sparc/vm/vm_version_sparc.cpp#l126 thanks, Vladimir On 7/20/17 12:24 AM, Muthusamy Chinnathambi wrote: > Hi, > > Please review the backport of bug: "JDK-8158012: Use SW prefetch instructions instead of BIS for allocation prefetches on SPARC Core C4" to jdk8u-dev > > Please note that this is not a clean backport due to conflict in comments and slight re-order of code. > > > Webrev: http://cr.openjdk.java.net/~shshahma/Muthu/8158012/webrev.00/ > jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-8158012 > Original patch pushed to jdk9: http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/eebdc7acebd4 > > Test: Ran jprt and jtreg. > > Regards, > Muthusamy C > From coleen.phillimore at oracle.com Thu Jul 20 23:44:44 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 20 Jul 2017 19:44:44 -0400 Subject: RFR (S) 8182397: Race in field updates when creating ArrayKlasses can lead to crash Message-ID: Summary: Update array_klass field in component mirror after klass.java_mirror field for concurrent readers in compiled code. See bug for details. open webrev at http://cr.openjdk.java.net/~coleenp/8182397.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8182397 Ran tier1 testing with -Xcomp and without. Thank you to Tom for the test and diagnosis. Thanks, Coleen From vladimir.kozlov at oracle.com Thu Jul 20 23:59:22 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 20 Jul 2017 16:59:22 -0700 Subject: [10] RFR(S) 8185018: Put AOT tests on problem list for OS-X and Windows Message-ID: <1ccaa9d3-da06-4cae-3ba0-af0c09043b76@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8185018 webrev: http://cr.openjdk.java.net/~kvn/8185018/webrev.00/ Put AOT tests on problem list for OS-X and Windows until we solve the problem JDK-8183337 with missing tools (linker) on tests machines. Thanks, Vladimir From daniel.daugherty at oracle.com Fri Jul 21 00:10:28 2017 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 20 Jul 2017 18:10:28 -0600 Subject: [10] RFR(XS) 8185013: Disable subword vector analysis optimization until 8184995 is fixed In-Reply-To: References: Message-ID: <21c8c128-15c3-4359-8946-6e6e3f5d4e8c@oracle.com> Agreed. Sorry, I usually say up front when I think a change qualifies for the "HotSpot Trivial Change" rule... :-) Dan On 7/20/17 4:51 PM, Vladimir Kozlov wrote: > Thank you, Dan. > > I think one reviewer is enough for this. I am pushing it. > > Vladimir > > On 7/20/17 3:47 PM, Daniel D. Daugherty wrote: >> Thumbs up! >> >> Dan >> >> >> On 7/20/17 4:44 PM, Vladimir Kozlov wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8185013 >>> >>> Switch UseSubwordForMaxVector flag off by default. >>> >>> src/share/vm/opto/c2_globals.hpp >>> @@ -192,7 +192,7 @@ >>> "of rounds of >>> unroll,optimize,..") \ >>> range(0, >>> max_jint) \ >>> >>> \ >>> - product(bool, UseSubwordForMaxVector, >>> true, \ >>> + product(bool, UseSubwordForMaxVector, >>> false, \ >>> "Use Subword Analysis to set maximum vector >>> size") \ >>> >>> \ >>> develop(intx, UnrollLimitForProfileCheck, >>> 1, \ >>> >>> Thanks, >>> Vladimir >> > From daniel.daugherty at oracle.com Fri Jul 21 00:12:23 2017 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 20 Jul 2017 18:12:23 -0600 Subject: [10] RFR(S) 8185018: Put AOT tests on problem list for OS-X and Windows In-Reply-To: <1ccaa9d3-da06-4cae-3ba0-af0c09043b76@oracle.com> References: <1ccaa9d3-da06-4cae-3ba0-af0c09043b76@oracle.com> Message-ID: <83035c9f-cd1b-2100-e165-129134693f11@oracle.com> On 7/20/17 5:59 PM, Vladimir Kozlov wrote: > https://bugs.openjdk.java.net/browse/JDK-8185018 > webrev: http://cr.openjdk.java.net/~kvn/8185018/webrev.00/ test/ProblemList.txt No comments. Looks like you figured out the magic incantation! Thanks! I think this qualifies for the "HotSpot Trivial Change" rule. Thumbs up! Dan > > Put AOT tests on problem list for OS-X and Windows until we solve the > problem JDK-8183337 with missing tools (linker) on tests machines. > > Thanks, > Vladimir From vladimir.kozlov at oracle.com Fri Jul 21 00:13:37 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 20 Jul 2017 17:13:37 -0700 Subject: [10] RFR(S) 8185018: Put AOT tests on problem list for OS-X and Windows In-Reply-To: <83035c9f-cd1b-2100-e165-129134693f11@oracle.com> References: <1ccaa9d3-da06-4cae-3ba0-af0c09043b76@oracle.com> <83035c9f-cd1b-2100-e165-129134693f11@oracle.com> Message-ID: Thank you, Dan On 7/20/17 5:12 PM, Daniel D. Daugherty wrote: > On 7/20/17 5:59 PM, Vladimir Kozlov wrote: >> https://bugs.openjdk.java.net/browse/JDK-8185018 >> webrev: http://cr.openjdk.java.net/~kvn/8185018/webrev.00/ > > test/ProblemList.txt > No comments. > > Looks like you figured out the magic incantation! Thanks! > > I think this qualifies for the "HotSpot Trivial Change" rule. Yes. > > Thumbs up! Thanks, Vladimir > > Dan > > >> >> Put AOT tests on problem list for OS-X and Windows until we solve the >> problem JDK-8183337 with missing tools (linker) on tests machines. >> >> Thanks, >> Vladimir > From daniel.daugherty at oracle.com Fri Jul 21 00:14:59 2017 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 20 Jul 2017 18:14:59 -0600 Subject: [10] RFR(S) 8185018: Put AOT tests on problem list for OS-X and Windows In-Reply-To: References: <1ccaa9d3-da06-4cae-3ba0-af0c09043b76@oracle.com> <83035c9f-cd1b-2100-e165-129134693f11@oracle.com> Message-ID: Kim noticed that there are '//' in the file... Dan On 7/20/17 6:13 PM, Vladimir Kozlov wrote: > Thank you, Dan > > On 7/20/17 5:12 PM, Daniel D. Daugherty wrote: >> On 7/20/17 5:59 PM, Vladimir Kozlov wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8185018 >>> webrev: http://cr.openjdk.java.net/~kvn/8185018/webrev.00/ >> >> test/ProblemList.txt >> No comments. >> >> Looks like you figured out the magic incantation! Thanks! >> >> I think this qualifies for the "HotSpot Trivial Change" rule. > > Yes. > >> >> Thumbs up! > > Thanks, > Vladimir > >> >> Dan >> >> >>> >>> Put AOT tests on problem list for OS-X and Windows until we solve >>> the problem JDK-8183337 with missing tools (linker) on tests machines. >>> >>> Thanks, >>> Vladimir >> From vladimir.kozlov at oracle.com Fri Jul 21 00:19:16 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 20 Jul 2017 17:19:16 -0700 Subject: [10] RFR(S) 8185018: Put AOT tests on problem list for OS-X and Windows In-Reply-To: References: <1ccaa9d3-da06-4cae-3ba0-af0c09043b76@oracle.com> <83035c9f-cd1b-2100-e165-129134693f11@oracle.com> Message-ID: <5e88f6a4-964d-dea5-2336-e069f9f6cba8@oracle.com> Thank you for catching this. I updated webrev (same webrev.00). I will add Kim to reviewer list. Vladimir On 7/20/17 5:14 PM, Daniel D. Daugherty wrote: > Kim noticed that there are '//' in the file... > > Dan > > > On 7/20/17 6:13 PM, Vladimir Kozlov wrote: >> Thank you, Dan >> >> On 7/20/17 5:12 PM, Daniel D. Daugherty wrote: >>> On 7/20/17 5:59 PM, Vladimir Kozlov wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8185018 >>>> webrev: http://cr.openjdk.java.net/~kvn/8185018/webrev.00/ >>> >>> test/ProblemList.txt >>> No comments. >>> >>> Looks like you figured out the magic incantation! Thanks! >>> >>> I think this qualifies for the "HotSpot Trivial Change" rule. >> >> Yes. >> >>> >>> Thumbs up! >> >> Thanks, >> Vladimir >> >>> >>> Dan >>> >>> >>>> >>>> Put AOT tests on problem list for OS-X and Windows until we solve >>>> the problem JDK-8183337 with missing tools (linker) on tests machines. >>>> >>>> Thanks, >>>> Vladimir >>> > From kim.barrett at oracle.com Fri Jul 21 00:24:46 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 20 Jul 2017 20:24:46 -0400 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <596CD126.6080100@oracle.com> Message-ID: <275B9A64-23C1-44AC-A5C2-5ED81CFFAE47@oracle.com> > On Jul 17, 2017, at 11:59 AM, Andrew Haley wrote: >> So for example, we need to be able to perform the following conversions: >> 1) Floating point to integer without changing bit representaiton. It >> seems like the only way of doing this without violating the C++03 >> standard is to use memcpy. (no, the union trick in e.g. jfloat_cast is >> not standards compliant) > > I'm not entirely convinced that memcpy does it either for "truly > portable" C++, but perhaps it's as good as we're going to get. See 3.9, where it talks about using memcpy to copy the "value representation". That dodges the 3.10/15 restrictions by not accessing the stored value through an lvalue. From kim.barrett at oracle.com Fri Jul 21 00:26:07 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 20 Jul 2017 20:26:07 -0400 Subject: [10] RFR(S) 8185018: Put AOT tests on problem list for OS-X and Windows In-Reply-To: <5e88f6a4-964d-dea5-2336-e069f9f6cba8@oracle.com> References: <1ccaa9d3-da06-4cae-3ba0-af0c09043b76@oracle.com> <83035c9f-cd1b-2100-e165-129134693f11@oracle.com> <5e88f6a4-964d-dea5-2336-e069f9f6cba8@oracle.com> Message-ID: > On Jul 20, 2017, at 8:19 PM, Vladimir Kozlov wrote: > > Thank you for catching this. I updated webrev (same webrev.00). > I will add Kim to reviewer list. Looks good now. > > Vladimir > > On 7/20/17 5:14 PM, Daniel D. Daugherty wrote: >> Kim noticed that there are '//' in the file... >> Dan >> On 7/20/17 6:13 PM, Vladimir Kozlov wrote: >>> Thank you, Dan >>> >>> On 7/20/17 5:12 PM, Daniel D. Daugherty wrote: >>>> On 7/20/17 5:59 PM, Vladimir Kozlov wrote: >>>>> https://bugs.openjdk.java.net/browse/JDK-8185018 >>>>> webrev: http://cr.openjdk.java.net/~kvn/8185018/webrev.00/ >>>> >>>> test/ProblemList.txt >>>> No comments. >>>> >>>> Looks like you figured out the magic incantation! Thanks! >>>> >>>> I think this qualifies for the "HotSpot Trivial Change" rule. >>> >>> Yes. >>> >>>> >>>> Thumbs up! >>> >>> Thanks, >>> Vladimir >>> >>>> >>>> Dan >>>> >>>> >>>>> >>>>> Put AOT tests on problem list for OS-X and Windows until we solve the problem JDK-8183337 with missing tools (linker) on tests machines. >>>>> >>>>> Thanks, >>>>> Vladimir From thomas.stuefe at gmail.com Fri Jul 21 06:54:51 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 21 Jul 2017 08:54:51 +0200 Subject: RFR(xs): 8185033: On Metaspace OOM, ChunkManager composition should be logged. Message-ID: Dear all, May I please get reviews and a sponsor for the following small enhancement. Issue: https://bugs.openjdk.java.net/browse/JDK-8185033 Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8185033-On-Metaspace-OOM-chunkManager-composition-should-be-logged/webrev.00/webrev/ This patch adds a bit of logging if a Metaspace OOM happens. We will now get a printout of the composition of the global chunk managers (non-class and, if necessary, class). Before it looked like this: [272.727s][info][gc,metaspace,freelist] All Metaspace: [272.728s][info][gc,metaspace,freelist] data space: Chunk accounting: used in chunks 100752K + unused in chunks 650K + capacity in free chunks 8293K = 109696K capacity in allocated chunks 101403K [272.728s][info][gc,metaspace,freelist] class space: Chunk accounting: used in chunks 4047K + unused in chunks 97K + capacity in free chunks 6085K = 10230K capacity in allocated chunks 4145K [272.729s][info][gc,metaspace,freelist] Total fragmentation waste (words) doesn't count free space [272.729s][info][gc,metaspace,freelist] data: 183 specialized(s) 960, 169 small(s) 41, 1507 medium(s) 4480, large count 1 [272.729s][info][gc,metaspace,freelist] class: 183 specialized(s) 0, 13 small(s) 12, 111 medium(s) 8, large count 1 Now: [250.548s][info][gc,metaspace,freelist] All Metaspace: [250.549s][info][gc,metaspace,freelist] data space: Chunk accounting: used in chunks 100748K + unused in chunks 638K + capacity in free chunks 10357K = 111744K capacity in allocated chunks 101387K [250.550s][info][gc,metaspace,freelist] class space: Chunk accounting: used in chunks 4047K + unused in chunks 97K + capacity in free chunks 6085K = 10230K capacity in allocated chunks 4145K [250.550s][info][gc,metaspace,freelist] Total fragmentation waste (words) doesn't count free space [250.550s][info][gc,metaspace,freelist] data: 183 specialized(s) 960, 165 small(s) 27, 1507 medium(s) 4504, large count 1 [250.550s][info][gc,metaspace,freelist] class: 183 specialized(s) 0, 13 small(s) 12, 111 medium(s) 14, large count 1 [250.550s][info][gc,metaspace,freelist] Chunkmanager (non-class): [250.550s][info][gc,metaspace,freelist] 89 specialized (128 bytes) chunks, total 91136 bytes [250.550s][info][gc,metaspace,freelist] 2567 small (512 bytes) chunks, total 10514432 bytes [250.550s][info][gc,metaspace,freelist] 0 medium (8192 bytes) chunks, total 0 bytes [250.550s][info][gc,metaspace,freelist] 0 humongous chunks, total 0 bytes [250.550s][info][gc,metaspace,freelist] total size: 10605568. [250.550s][info][gc,metaspace,freelist] Chunkmanager (class): [250.550s][info][gc,metaspace,freelist] 87 specialized (128 bytes) chunks, total 89088 bytes [250.550s][info][gc,metaspace,freelist] 2999 small (256 bytes) chunks, total 6141952 bytes [250.550s][info][gc,metaspace,freelist] 0 medium (4096 bytes) chunks, total 0 bytes [250.550s][info][gc,metaspace,freelist] 0 humongous chunks, total 0 bytes [250.550s][info][gc,metaspace,freelist] total size: 6231040. This helps in understanding the underlying cause of the OOM, if that cause is related to chunks lingering around in the freelist. See also [1], a Jep proposing to improve the metaspace chunk allocation to increase the chance of chunk reuse. This patch here is also a preparation for the prototype work of [1]. Note that admittedly, metaspace.cpp will again gain a bit of complexity by this patch. When examining the coding, I was tempted to refactor but refrained from doing so to keep the patch focused. But I think metaspace.cpp (especially the coding associated with statistics) could be cleaned up a bit. I opened a follow up item [2] to collect cleanup-issues. I am willing to do that myself, but would prefer it to be done after functional patches - which still have a slight chance of being backported to jdk9 - are all being added. Patch was built and tested on Windows x64, Linux x64 and AIX. I also ran jtreg hotspot/runtime/Metaspace tests, no issues. Kind Regards, Thomas ---- [1] https://bugs.openjdk.java.net/browse/JDK-8166690 [2] https://bugs.openjdk.java.net/browse/JDK-8185034 From erik.helin at oracle.com Fri Jul 21 08:24:34 2017 From: erik.helin at oracle.com (Erik Helin) Date: Fri, 21 Jul 2017 10:24:34 +0200 Subject: RFR(XL): 8181917: Refactor UL LogStreams to avoid using resource area In-Reply-To: References: <337330d2-512c-bc77-5a55-c05d33e376e5@oracle.com> Message-ID: <9b09acae-5cf6-f7dd-5195-99d65588b943@oracle.com> On 07/20/2017 09:11 AM, Thomas St?fe wrote: > Hi all, > > after the last irc discussion with Erik here the new webrev with the > changes he requested: > > Full: > http://cr.openjdk.java.net/~stuefe/webrevs/8181917-refactor-ul-logstream/all.webrev.05/webrev/ > Delta: > http://cr.openjdk.java.net/~stuefe/webrevs/8181917-refactor-ul-logstream/delta-04-to-05/webrev/ This looks good to me, Reviewed. Thank you Thomas for "hanging in there" with this big patch, really solid work! > I built the change on windows (32bit too :P), Linux and AIX. I am > currently re-running jtreg tests on Linux, no issues so far. I took the patch for a spin in our build system and it worked fine. I will sponsor this patch. Thanks, Erik > The changes: > > - I removed all stray "Unconditional write?" comments I had used before > to mark sites which did logging unconditionally. Marcus had opened a > follow up bug report for that issue > (https://bugs.openjdk.java.net/browse/JDK-8182466), and I added the > affected sites I found to the comment section of that bug. > > - In g1GCPhaseTimes.cpp, I changed the logging code in > G1GCPhaseTimes::debug_phase() to use LogTarget instead of Log to be > consistent with G1GCPhaseTimes::trace_phase() > > - In the new LogStream implementation (logStream.cpp), I added temporary > variables in LogStream::LineBuffer::append() and > LogStream::LineBuffer::try_ensure_cap(). > > - I also enlarge the line buffer a bit more aggressive. We now, once the > initial 64byte buffer is exhausted - enlarge by 256 bytes steps. > > - Finally, I renamed LineBuffer::ptr() to LineBuffer::buffer() > A note about memory consumption of the LogStream line buffer: I do not > think we need to overly worry about memory consumption, at least not if > we think the old solution was fine. Old solution was using stringStream, > which uses NEW_RESOURCE_ARRAY to enlarge the line buffer when exhausted. > This is resource area, so it *always* leaks the previous allocation, > which is not deallocated until the resource area stack is reset. Even > worse, for larger line lengths the growth strategy in stringStream() > seems to be to just enlarge by the requested amount, which means that > every single write would cause a reallocation and hence a new allocation > of the underlying buffer (see stringStream::write()). I am not 100% > sure, this is just from dry-reading the code. > > Kind Regards, Thomas > > > > On Mon, Jul 17, 2017 at 3:44 PM, Thomas St?fe > wrote: > > Hi Eric, > > thank you for your review! > > new Webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/8181917-refactor-ul-logstream/all.webrev.04/webrev/ > > delta to last: > http://cr.openjdk.java.net/~stuefe/webrevs/8181917-refactor-ul-logstream/delta-03-to-04/webrev/ > > > Changes: > > - I changed the allocation function to handle the case where we > already reached the reasonable_max cap. You are right, in that case > reallocating is unnecessary. > - Also, added log_info() for that case as you suggested. > - Finally, added a fix to the LogStream gtest which somehow slipped > thru the last webrev. > > Changes are now rebased to the current tip. > > Thanks! > > ..Thomas > > On Fri, Jul 14, 2017 at 12:36 PM, Erik Helin > wrote: > > On 07/13/2017 08:46 AM, Thomas St?fe wrote: > > Hi Erik! > > > On Mon, Jul 10, 2017 at 11:40 AM, Erik Helin > > >> wrote: > > Hi Thomas, > > On 06/30/2017 08:32 PM, Thomas St?fe wrote: > > Hi Eric, > > thank you for the review! > > New > Version: > http://cr.openjdk.java.net/~stuefe/webrevs/8181917-refactor-ul-logstream/all.webrev.03/webrev/ > > > > > > > I think you still have a problem with "runaway" loggers > in this version: > > +// try_ensure_cap tries to enlarge the capacity of the > internal buffer > +// to the given atleast value. May fail if either OOM > happens or > atleast > +// is larger than a reasonable max of 1 M. Caller must > not assume > +// capacity without checking. > +void LogStream::LineBuffer::try_ensure_cap(size_t > atleast) { > + assert(_cap >= sizeof(_smallbuf), "sanity"); > + if (_cap < atleast) { > + const size_t reasonable_max = 1 * M; > + size_t newcap = align_size_up(atleast + 64, 64); > + assert(_cap <= reasonable_max, "sanity"); > + // Cap out at a reasonable max to prevent runaway > leaks. > + if (newcap > reasonable_max) { > + newcap = reasonable_max; > + } > + > + char* const newbuf = (char*) os::malloc(newcap, > mtLogging); > + if (newbuf == NULL) { // OOM. Leave object unchanged. > + return; > + } > + if (_pos > 0) { // preserve old content > + memcpy(newbuf, _buf, _pos + 1); // ..including > trailing zero > + } > + if (_buf != _smallbuf) { > + os::free(_buf); > + } > + _buf = newbuf; > + _cap = newcap; > + } > + assert(_cap >= atleast, "sanity"); > +} > > with the above code, even though _cap > reasonable_max, > you will > always allocate newbuf with os::malloc. For debug > builds, we are > fine, since the JVM will assert on _cap <= > reasonable_max, but for > product builds we are not. Would it be better to first > check if _cap > == reasonable_max? As in the following snippet: > > void LogStream::LineBuffer::try_ensure_cap(size_t > atleast) { > if (_cap < atleast) { > const size_t reasonable_max = 1 * M; > if (_cap == reasonable_max) { > // max memory used, "die" in debug builds, log > in product builds > log_warning(logging)("Max memory used for > message: %s\n", _buf); > ShouldNotReachHere(); > return; > } > > size_t newcap = align_size_up(atleast + 64, 64); > if (newcap > reasonable_max) { > newcap = reasonable_max; > } > char* const newbuf = (char*) os::malloc(newcap, > mtLogging); > // rest of logic > } > } > > Thanks, > Erik > > > I'm not sure I understand. Are you concerned that _cap would > grow beyond reasonable_max - because I do not see a way this > could happen. Or do you want to also treat the *attempt* to > grow the _cap beyond reasonable_max as a loggable offence? > > > What I am trying to say is that with your current patch, even if > _cap == reasonable_max, you will still allocate a new 1M buffer, > copy all the chars over, and the free the old one. My proposal > simply skips the allocation, copying and freeing if _cap == > reasonable_max. The result will be the same, just less stress on > malloc. > > If the latter, I am not sure about logging. If this is an > error which may happen at a customer site without popping up > first in inhouse-tests (which it conceivably is, because > logging is very content dependent), I would rather hide the > error in production code instead of flooding stderr with > tons of log warnings or crashing with a guarantee, no? > > > I see what you mean, but perhaps log_info(logging) at least? I > want to be able to discover if this happens in production, for > example if a customer complains about truncated logs. A > log_info(logging) will only be printed if -Xlog:logging=info is > specified on the command-line. > > Thanks, > Erik > > ..Thomas > > > > From mikael.gerdin at oracle.com Fri Jul 21 08:46:09 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Fri, 21 Jul 2017 10:46:09 +0200 Subject: RFR(xs): 8185033: On Metaspace OOM, ChunkManager composition should be logged. In-Reply-To: References: Message-ID: <94a12182-d318-e423-644e-56ff918d717c@oracle.com> Hi Thomas, On 2017-07-21 08:54, Thomas St?fe wrote: > Dear all, > > May I please get reviews and a sponsor for the following small enhancement. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8185033 > Webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/8185033-On-Metaspace-OOM-chunkManager-composition-should-be-logged/webrev.00/webrev/ > > This patch adds a bit of logging if a Metaspace OOM happens. We will now > get a printout of the composition of the global chunk managers (non-class > and, if necessary, class). > > Before it looked like this: > > [272.727s][info][gc,metaspace,freelist] All Metaspace: > [272.728s][info][gc,metaspace,freelist] data space: Chunk accounting: > used in chunks 100752K + unused in chunks 650K + capacity in free chunks > 8293K = 109696K capacity in allocated chunks 101403K > [272.728s][info][gc,metaspace,freelist] class space: Chunk accounting: > used in chunks 4047K + unused in chunks 97K + capacity in free chunks > 6085K = 10230K capacity in allocated chunks 4145K > [272.729s][info][gc,metaspace,freelist] Total fragmentation waste (words) > doesn't count free space > [272.729s][info][gc,metaspace,freelist] data: 183 specialized(s) 960, 169 > small(s) 41, 1507 medium(s) 4480, large count 1 > [272.729s][info][gc,metaspace,freelist] class: 183 specialized(s) 0, 13 > small(s) 12, 111 medium(s) 8, large count 1 > > Now: > [250.548s][info][gc,metaspace,freelist] All Metaspace: > [250.549s][info][gc,metaspace,freelist] data space: Chunk accounting: > used in chunks 100748K + unused in chunks 638K + capacity in free chunks > 10357K = 111744K capacity in allocated chunks 101387K > [250.550s][info][gc,metaspace,freelist] class space: Chunk accounting: > used in chunks 4047K + unused in chunks 97K + capacity in free chunks > 6085K = 10230K capacity in allocated chunks 4145K > [250.550s][info][gc,metaspace,freelist] Total fragmentation waste (words) > doesn't count free space > [250.550s][info][gc,metaspace,freelist] data: 183 specialized(s) 960, 165 > small(s) 27, 1507 medium(s) 4504, large count 1 > [250.550s][info][gc,metaspace,freelist] class: 183 specialized(s) 0, 13 > small(s) 12, 111 medium(s) 14, large count 1 > [250.550s][info][gc,metaspace,freelist] Chunkmanager (non-class): > [250.550s][info][gc,metaspace,freelist] 89 specialized (128 bytes) > chunks, total 91136 bytes > [250.550s][info][gc,metaspace,freelist] 2567 small (512 bytes) chunks, > total 10514432 bytes > [250.550s][info][gc,metaspace,freelist] 0 medium (8192 bytes) chunks, > total 0 bytes > [250.550s][info][gc,metaspace,freelist] 0 humongous chunks, total 0 bytes > [250.550s][info][gc,metaspace,freelist] total size: 10605568. > [250.550s][info][gc,metaspace,freelist] Chunkmanager (class): > [250.550s][info][gc,metaspace,freelist] 87 specialized (128 bytes) > chunks, total 89088 bytes > [250.550s][info][gc,metaspace,freelist] 2999 small (256 bytes) chunks, > total 6141952 bytes > [250.550s][info][gc,metaspace,freelist] 0 medium (4096 bytes) chunks, > total 0 bytes > [250.550s][info][gc,metaspace,freelist] 0 humongous chunks, total 0 bytes > [250.550s][info][gc,metaspace,freelist] total size: 6231040. > > This helps in understanding the underlying cause of the OOM, if that cause > is related to chunks lingering around in the freelist. See also [1], a Jep > proposing to improve the metaspace chunk allocation to increase the chance > of chunk reuse. This patch here is also a preparation for the prototype > work of [1]. > > Note that admittedly, metaspace.cpp will again gain a bit of complexity by > this patch. When examining the coding, I was tempted to refactor but > refrained from doing so to keep the patch focused. But I think > metaspace.cpp (especially the coding associated with statistics) could be > cleaned up a bit. I opened a follow up item [2] to collect cleanup-issues. > I am willing to do that myself, but would prefer it to be done after > functional patches - which still have a slight chance of being backported > to jdk9 - are all being added. I know this is a bit counter to what you said about keeping the patch small but what do you think about doing this on top of your change? http://cr.openjdk.java.net/~mgerdin/8185033/webrev.00/src/share/vm/memory/metaspace.cpp.udiff.html I'm not sure why you went with uint32_t for the lengths since it's just a couple of bytes and you can get rid of the casts. I also took the liberty to squash out two const_casts which were making my eyes twitch... :) /Mikael > > Patch was built and tested on Windows x64, Linux x64 and AIX. I also ran > jtreg hotspot/runtime/Metaspace tests, no issues. > > Kind Regards, Thomas > > ---- > > [1] https://bugs.openjdk.java.net/browse/JDK-8166690 > [2] https://bugs.openjdk.java.net/browse/JDK-8185034 > From erik.helin at oracle.com Fri Jul 21 11:50:06 2017 From: erik.helin at oracle.com (Erik Helin) Date: Fri, 21 Jul 2017 13:50:06 +0200 Subject: Request for review for JDK-8184045 - TestSystemGCWithG1.java times out on Solaris SPARC In-Reply-To: References: Message-ID: <98794b3a-ac78-26b5-8647-9f8f99aef048@oracle.com> On 07/20/2017 05:31 PM, Alexander Harlap wrote: > Please review changes forJDK-8184045 > - > TestSystemGCWithG1.java times out on Solaris SPARC (labeled as > integration blocker) > > Proposed change will avoid timeout failure for new stress tests > TestSystemGCWithG1.java, TestSystemGCWithCMS.java and > TestSystemGCWithSerial.java regularly observed on Solaris Sparc > > Change is located at http://cr.openjdk.java.net/~aharlap/8184045/webrev.00/ Instead of removing the second pass, can you check if the args[0] == "long"? The code can look like: public static void main(String[] args) { populateLongLived(); runAllPhases(); if (args.length > 0 && args[0] == "long") { runAllPhases(); } } This way the stress test can be used in "long" and "short" mode. Thanks, Erik > Alex > From thomas.stuefe at gmail.com Fri Jul 21 12:21:34 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 21 Jul 2017 14:21:34 +0200 Subject: RFR(xs): 8185033: On Metaspace OOM, ChunkManager composition should be logged. In-Reply-To: <94a12182-d318-e423-644e-56ff918d717c@oracle.com> References: <94a12182-d318-e423-644e-56ff918d717c@oracle.com> Message-ID: Hi Mikael, thanks for looking at this! the const correctness stuff makes sense, I started to do this myself but reverted that part and added it as a Todo to https://bugs.openjdk.java.net/browse/JDK-8185034. But sure, we can do this in this change. As for making the counters size_t, I am not sure. I do not think that size_t is a good fit for numbers-of-things, size_t, to me, always means memory size or address range. I think counters are better represented by int or unsigned, if you do not like uint32_t. I also think that the original decision to make the chunk counters size_t is debatable. But I do not have strong emotions here, if you prefer I use size_t, I can use size_t. Thanks, Thomas On Fri, Jul 21, 2017 at 10:46 AM, Mikael Gerdin wrote: > Hi Thomas, > > > On 2017-07-21 08:54, Thomas St?fe wrote: > >> Dear all, >> >> May I please get reviews and a sponsor for the following small >> enhancement. >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8185033 >> Webrev: >> http://cr.openjdk.java.net/~stuefe/webrevs/8185033-On-Metasp >> ace-OOM-chunkManager-composition-should-be-logged/webrev.00/webrev/ >> >> This patch adds a bit of logging if a Metaspace OOM happens. We will now >> get a printout of the composition of the global chunk managers (non-class >> and, if necessary, class). >> >> Before it looked like this: >> >> [272.727s][info][gc,metaspace,freelist] All Metaspace: >> [272.728s][info][gc,metaspace,freelist] data space: Chunk accounting: >> used in chunks 100752K + unused in chunks 650K + capacity in free chunks >> 8293K = 109696K capacity in allocated chunks 101403K >> [272.728s][info][gc,metaspace,freelist] class space: Chunk accounting: >> used in chunks 4047K + unused in chunks 97K + capacity in free chunks >> 6085K = 10230K capacity in allocated chunks 4145K >> [272.729s][info][gc,metaspace,freelist] Total fragmentation waste (words) >> doesn't count free space >> [272.729s][info][gc,metaspace,freelist] data: 183 specialized(s) 960, >> 169 >> small(s) 41, 1507 medium(s) 4480, large count 1 >> [272.729s][info][gc,metaspace,freelist] class: 183 specialized(s) 0, 13 >> small(s) 12, 111 medium(s) 8, large count 1 >> >> Now: >> [250.548s][info][gc,metaspace,freelist] All Metaspace: >> [250.549s][info][gc,metaspace,freelist] data space: Chunk accounting: >> used in chunks 100748K + unused in chunks 638K + capacity in free chunks >> 10357K = 111744K capacity in allocated chunks 101387K >> [250.550s][info][gc,metaspace,freelist] class space: Chunk accounting: >> used in chunks 4047K + unused in chunks 97K + capacity in free chunks >> 6085K = 10230K capacity in allocated chunks 4145K >> [250.550s][info][gc,metaspace,freelist] Total fragmentation waste (words) >> doesn't count free space >> [250.550s][info][gc,metaspace,freelist] data: 183 specialized(s) 960, >> 165 >> small(s) 27, 1507 medium(s) 4504, large count 1 >> [250.550s][info][gc,metaspace,freelist] class: 183 specialized(s) 0, 13 >> small(s) 12, 111 medium(s) 14, large count 1 >> [250.550s][info][gc,metaspace,freelist] Chunkmanager (non-class): >> [250.550s][info][gc,metaspace,freelist] 89 specialized (128 bytes) >> chunks, total 91136 bytes >> [250.550s][info][gc,metaspace,freelist] 2567 small (512 bytes) chunks, >> total 10514432 bytes >> [250.550s][info][gc,metaspace,freelist] 0 medium (8192 bytes) chunks, >> total 0 bytes >> [250.550s][info][gc,metaspace,freelist] 0 humongous chunks, total 0 >> bytes >> [250.550s][info][gc,metaspace,freelist] total size: 10605568. >> [250.550s][info][gc,metaspace,freelist] Chunkmanager (class): >> [250.550s][info][gc,metaspace,freelist] 87 specialized (128 bytes) >> chunks, total 89088 bytes >> [250.550s][info][gc,metaspace,freelist] 2999 small (256 bytes) chunks, >> total 6141952 bytes >> [250.550s][info][gc,metaspace,freelist] 0 medium (4096 bytes) chunks, >> total 0 bytes >> [250.550s][info][gc,metaspace,freelist] 0 humongous chunks, total 0 >> bytes >> [250.550s][info][gc,metaspace,freelist] total size: 6231040. >> >> This helps in understanding the underlying cause of the OOM, if that cause >> is related to chunks lingering around in the freelist. See also [1], a Jep >> proposing to improve the metaspace chunk allocation to increase the chance >> of chunk reuse. This patch here is also a preparation for the prototype >> work of [1]. >> >> Note that admittedly, metaspace.cpp will again gain a bit of complexity by >> this patch. When examining the coding, I was tempted to refactor but >> refrained from doing so to keep the patch focused. But I think >> metaspace.cpp (especially the coding associated with statistics) could be >> cleaned up a bit. I opened a follow up item [2] to collect cleanup-issues. >> I am willing to do that myself, but would prefer it to be done after >> functional patches - which still have a slight chance of being backported >> to jdk9 - are all being added. >> > > I know this is a bit counter to what you said about keeping the patch > small but what do you think about doing this on top of your change? > http://cr.openjdk.java.net/~mgerdin/8185033/webrev.00/src/sh > are/vm/memory/metaspace.cpp.udiff.html > > I'm not sure why you went with uint32_t for the lengths since it's just a > couple of bytes and you can get rid of the casts. > I also took the liberty to squash out two const_casts which were making my > eyes twitch... :) > > /Mikael > > > >> Patch was built and tested on Windows x64, Linux x64 and AIX. I also ran >> jtreg hotspot/runtime/Metaspace tests, no issues. >> >> Kind Regards, Thomas >> >> ---- >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8166690 >> [2] https://bugs.openjdk.java.net/browse/JDK-8185034 >> >> From thomas.stuefe at gmail.com Fri Jul 21 12:23:46 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 21 Jul 2017 14:23:46 +0200 Subject: RFR(XL): 8181917: Refactor UL LogStreams to avoid using resource area In-Reply-To: <9b09acae-5cf6-f7dd-5195-99d65588b943@oracle.com> References: <337330d2-512c-bc77-5a55-c05d33e376e5@oracle.com> <9b09acae-5cf6-f7dd-5195-99d65588b943@oracle.com> Message-ID: Great, thanks a lot Eric and Marcus for your review work! Note I still have not yet time to run current iteration of the patch fully thru jtreg. I did full jtreg tests on one of the earlier versions, so I do not expect any surprises. Thank you for sponsoring! Kind Regards, Thomas On Fri, Jul 21, 2017 at 10:24 AM, Erik Helin wrote: > On 07/20/2017 09:11 AM, Thomas St?fe wrote: > >> Hi all, >> >> after the last irc discussion with Erik here the new webrev with the >> changes he requested: >> >> Full: http://cr.openjdk.java.net/~stuefe/webrevs/8181917-refactor- >> ul-logstream/all.webrev.05/webrev/ >> Delta: http://cr.openjdk.java.net/~stuefe/webrevs/8181917-refactor- >> ul-logstream/delta-04-to-05/webrev/ >> > > This looks good to me, Reviewed. Thank you Thomas for "hanging in there" > with this big patch, really solid work! > > I built the change on windows (32bit too :P), Linux and AIX. I am >> currently re-running jtreg tests on Linux, no issues so far. >> > > I took the patch for a spin in our build system and it worked fine. I will > sponsor this patch. > > Thanks, > Erik > > The changes: >> >> - I removed all stray "Unconditional write?" comments I had used before >> to mark sites which did logging unconditionally. Marcus had opened a follow >> up bug report for that issue (https://bugs.openjdk.java.net >> /browse/JDK-8182466), and I added the affected sites I found to the >> comment section of that bug. >> >> - In g1GCPhaseTimes.cpp, I changed the logging code in >> G1GCPhaseTimes::debug_phase() to use LogTarget instead of Log to be >> consistent with G1GCPhaseTimes::trace_phase() >> >> - In the new LogStream implementation (logStream.cpp), I added temporary >> variables in LogStream::LineBuffer::append() and >> LogStream::LineBuffer::try_ensure_cap(). >> >> - I also enlarge the line buffer a bit more aggressive. We now, once the >> initial 64byte buffer is exhausted - enlarge by 256 bytes steps. >> >> - Finally, I renamed LineBuffer::ptr() to LineBuffer::buffer() >> A note about memory consumption of the LogStream line buffer: I do not >> think we need to overly worry about memory consumption, at least not if we >> think the old solution was fine. Old solution was using stringStream, which >> uses NEW_RESOURCE_ARRAY to enlarge the line buffer when exhausted. This is >> resource area, so it *always* leaks the previous allocation, which is not >> deallocated until the resource area stack is reset. Even worse, for larger >> line lengths the growth strategy in stringStream() seems to be to just >> enlarge by the requested amount, which means that every single write would >> cause a reallocation and hence a new allocation of the underlying buffer >> (see stringStream::write()). I am not 100% sure, this is just from >> dry-reading the code. >> >> Kind Regards, Thomas >> >> >> >> On Mon, Jul 17, 2017 at 3:44 PM, Thomas St?fe > > wrote: >> >> Hi Eric, >> >> thank you for your review! >> >> new Webrev: >> http://cr.openjdk.java.net/~stuefe/webrevs/8181917-refactor- >> ul-logstream/all.webrev.04/webrev/ >> > -ul-logstream/all.webrev.04/webrev/> >> delta to last: >> http://cr.openjdk.java.net/~stuefe/webrevs/8181917-refactor- >> ul-logstream/delta-03-to-04/webrev/ >> > -ul-logstream/delta-03-to-04/webrev/> >> >> Changes: >> >> - I changed the allocation function to handle the case where we >> already reached the reasonable_max cap. You are right, in that case >> reallocating is unnecessary. >> - Also, added log_info() for that case as you suggested. >> - Finally, added a fix to the LogStream gtest which somehow slipped >> thru the last webrev. >> >> Changes are now rebased to the current tip. >> >> Thanks! >> >> ..Thomas >> >> On Fri, Jul 14, 2017 at 12:36 PM, Erik Helin > > wrote: >> >> On 07/13/2017 08:46 AM, Thomas St?fe wrote: >> >> Hi Erik! >> >> >> On Mon, Jul 10, 2017 at 11:40 AM, Erik Helin >> >> > >> >> wrote: >> >> Hi Thomas, >> >> On 06/30/2017 08:32 PM, Thomas St?fe wrote: >> >> Hi Eric, >> >> thank you for the review! >> >> New >> Version: >> http://cr.openjdk.java.net/~stuefe/webrevs/8181917-refactor- >> ul-logstream/all.webrev.03/webrev/ >> > -ul-logstream/all.webrev.03/webrev/> >> > tuefe/webrevs/8181917-refactor-ul-logstream/all.webrev.03/webrev/ >> > -ul-logstream/all.webrev.03/webrev/>> >> >> >> I think you still have a problem with "runaway" loggers >> in this version: >> >> +// try_ensure_cap tries to enlarge the capacity of the >> internal buffer >> +// to the given atleast value. May fail if either OOM >> happens or >> atleast >> +// is larger than a reasonable max of 1 M. Caller must >> not assume >> +// capacity without checking. >> +void LogStream::LineBuffer::try_ensure_cap(size_t >> atleast) { >> + assert(_cap >= sizeof(_smallbuf), "sanity"); >> + if (_cap < atleast) { >> + const size_t reasonable_max = 1 * M; >> + size_t newcap = align_size_up(atleast + 64, 64); >> + assert(_cap <= reasonable_max, "sanity"); >> + // Cap out at a reasonable max to prevent runaway >> leaks. >> + if (newcap > reasonable_max) { >> + newcap = reasonable_max; >> + } >> + >> + char* const newbuf = (char*) os::malloc(newcap, >> mtLogging); >> + if (newbuf == NULL) { // OOM. Leave object >> unchanged. >> + return; >> + } >> + if (_pos > 0) { // preserve old content >> + memcpy(newbuf, _buf, _pos + 1); // ..including >> trailing zero >> + } >> + if (_buf != _smallbuf) { >> + os::free(_buf); >> + } >> + _buf = newbuf; >> + _cap = newcap; >> + } >> + assert(_cap >= atleast, "sanity"); >> +} >> >> with the above code, even though _cap > reasonable_max, >> you will >> always allocate newbuf with os::malloc. For debug >> builds, we are >> fine, since the JVM will assert on _cap <= >> reasonable_max, but for >> product builds we are not. Would it be better to first >> check if _cap >> == reasonable_max? As in the following snippet: >> >> void LogStream::LineBuffer::try_ensure_cap(size_t >> atleast) { >> if (_cap < atleast) { >> const size_t reasonable_max = 1 * M; >> if (_cap == reasonable_max) { >> // max memory used, "die" in debug builds, log >> in product builds >> log_warning(logging)("Max memory used for >> message: %s\n", _buf); >> ShouldNotReachHere(); >> return; >> } >> >> size_t newcap = align_size_up(atleast + 64, 64); >> if (newcap > reasonable_max) { >> newcap = reasonable_max; >> } >> char* const newbuf = (char*) os::malloc(newcap, >> mtLogging); >> // rest of logic >> } >> } >> >> Thanks, >> Erik >> >> >> I'm not sure I understand. Are you concerned that _cap would >> grow beyond reasonable_max - because I do not see a way this >> could happen. Or do you want to also treat the *attempt* to >> grow the _cap beyond reasonable_max as a loggable offence? >> >> >> What I am trying to say is that with your current patch, even if >> _cap == reasonable_max, you will still allocate a new 1M buffer, >> copy all the chars over, and the free the old one. My proposal >> simply skips the allocation, copying and freeing if _cap == >> reasonable_max. The result will be the same, just less stress on >> malloc. >> >> If the latter, I am not sure about logging. If this is an >> error which may happen at a customer site without popping up >> first in inhouse-tests (which it conceivably is, because >> logging is very content dependent), I would rather hide the >> error in production code instead of flooding stderr with >> tons of log warnings or crashing with a guarantee, no? >> >> >> I see what you mean, but perhaps log_info(logging) at least? I >> want to be able to discover if this happens in production, for >> example if a customer complains about truncated logs. A >> log_info(logging) will only be printed if -Xlog:logging=info is >> specified on the command-line. >> >> Thanks, >> Erik >> >> ..Thomas >> >> >> >> >> From mikael.gerdin at oracle.com Fri Jul 21 12:33:11 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Fri, 21 Jul 2017 14:33:11 +0200 Subject: RFR(xs): 8185033: On Metaspace OOM, ChunkManager composition should be logged. In-Reply-To: References: <94a12182-d318-e423-644e-56ff918d717c@oracle.com> Message-ID: <6e4f97c6-550e-306b-123d-cec667064f39@oracle.com> Hi Thomas, On 2017-07-21 14:21, Thomas St?fe wrote: > Hi Mikael, > > thanks for looking at this! > > the const correctness stuff makes sense, I started to do this myself but > reverted that part and added it as a Todo to > https://bugs.openjdk.java.net/browse/JDK-8185034. But sure, we can do > this in this change. > > As for making the counters size_t, I am not sure. I do not think that > size_t is a good fit for numbers-of-things, size_t, to me, always means > memory size or address range. I think counters are better represented by > int or unsigned, if you do not like uint32_t. I also think that the > original decision to make the chunk counters size_t is debatable. > > But I do not have strong emotions here, if you prefer I use size_t, I > can use size_t. Since the accessors you get the values from return size_t I would prefer if we keep the stats as size_t as well. Then we can change the type later if we feel like it. /Mikael > > Thanks, Thomas > > > On Fri, Jul 21, 2017 at 10:46 AM, Mikael Gerdin > > wrote: > > Hi Thomas, > > > On 2017-07-21 08:54, Thomas St?fe wrote: > > Dear all, > > May I please get reviews and a sponsor for the following small > enhancement. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8185033 > > Webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/8185033-On-Metaspace-OOM-chunkManager-composition-should-be-logged/webrev.00/webrev/ > > > This patch adds a bit of logging if a Metaspace OOM happens. We > will now > get a printout of the composition of the global chunk managers > (non-class > and, if necessary, class). > > Before it looked like this: > > [272.727s][info][gc,metaspace,freelist] All Metaspace: > [272.728s][info][gc,metaspace,freelist] data space: Chunk > accounting: > used in chunks 100752K + unused in chunks 650K + capacity in > free chunks > 8293K = 109696K capacity in allocated chunks 101403K > [272.728s][info][gc,metaspace,freelist] class space: Chunk > accounting: > used in chunks 4047K + unused in chunks 97K + capacity in free > chunks > 6085K = 10230K capacity in allocated chunks 4145K > [272.729s][info][gc,metaspace,freelist] Total fragmentation > waste (words) > doesn't count free space > [272.729s][info][gc,metaspace,freelist] data: 183 > specialized(s) 960, 169 > small(s) 41, 1507 medium(s) 4480, large count 1 > [272.729s][info][gc,metaspace,freelist] class: 183 > specialized(s) 0, 13 > small(s) 12, 111 medium(s) 8, large count 1 > > Now: > [250.548s][info][gc,metaspace,freelist] All Metaspace: > [250.549s][info][gc,metaspace,freelist] data space: Chunk > accounting: > used in chunks 100748K + unused in chunks 638K + capacity in > free chunks > 10357K = 111744K capacity in allocated chunks 101387K > [250.550s][info][gc,metaspace,freelist] class space: Chunk > accounting: > used in chunks 4047K + unused in chunks 97K + capacity in free > chunks > 6085K = 10230K capacity in allocated chunks 4145K > [250.550s][info][gc,metaspace,freelist] Total fragmentation > waste (words) > doesn't count free space > [250.550s][info][gc,metaspace,freelist] data: 183 > specialized(s) 960, 165 > small(s) 27, 1507 medium(s) 4504, large count 1 > [250.550s][info][gc,metaspace,freelist] class: 183 > specialized(s) 0, 13 > small(s) 12, 111 medium(s) 14, large count 1 > [250.550s][info][gc,metaspace,freelist] Chunkmanager (non-class): > [250.550s][info][gc,metaspace,freelist] 89 specialized (128 bytes) > chunks, total 91136 bytes > [250.550s][info][gc,metaspace,freelist] 2567 small (512 bytes) > chunks, > total 10514432 bytes > [250.550s][info][gc,metaspace,freelist] 0 medium (8192 bytes) > chunks, > total 0 bytes > [250.550s][info][gc,metaspace,freelist] 0 humongous chunks, > total 0 bytes > [250.550s][info][gc,metaspace,freelist] total size: 10605568. > [250.550s][info][gc,metaspace,freelist] Chunkmanager (class): > [250.550s][info][gc,metaspace,freelist] 87 specialized (128 bytes) > chunks, total 89088 bytes > [250.550s][info][gc,metaspace,freelist] 2999 small (256 bytes) > chunks, > total 6141952 bytes > [250.550s][info][gc,metaspace,freelist] 0 medium (4096 bytes) > chunks, > total 0 bytes > [250.550s][info][gc,metaspace,freelist] 0 humongous chunks, > total 0 bytes > [250.550s][info][gc,metaspace,freelist] total size: 6231040. > > This helps in understanding the underlying cause of the OOM, if > that cause > is related to chunks lingering around in the freelist. See also > [1], a Jep > proposing to improve the metaspace chunk allocation to increase > the chance > of chunk reuse. This patch here is also a preparation for the > prototype > work of [1]. > > Note that admittedly, metaspace.cpp will again gain a bit of > complexity by > this patch. When examining the coding, I was tempted to refactor but > refrained from doing so to keep the patch focused. But I think > metaspace.cpp (especially the coding associated with statistics) > could be > cleaned up a bit. I opened a follow up item [2] to collect > cleanup-issues. > I am willing to do that myself, but would prefer it to be done after > functional patches - which still have a slight chance of being > backported > to jdk9 - are all being added. > > > I know this is a bit counter to what you said about keeping the > patch small but what do you think about doing this on top of your > change? > http://cr.openjdk.java.net/~mgerdin/8185033/webrev.00/src/share/vm/memory/metaspace.cpp.udiff.html > > > I'm not sure why you went with uint32_t for the lengths since it's > just a couple of bytes and you can get rid of the casts. > I also took the liberty to squash out two const_casts which were > making my eyes twitch... :) > > /Mikael > > > > Patch was built and tested on Windows x64, Linux x64 and AIX. I > also ran > jtreg hotspot/runtime/Metaspace tests, no issues. > > Kind Regards, Thomas > > ---- > > [1] https://bugs.openjdk.java.net/browse/JDK-8166690 > > [2] https://bugs.openjdk.java.net/browse/JDK-8185034 > > > From erik.helin at oracle.com Fri Jul 21 12:44:26 2017 From: erik.helin at oracle.com (Erik Helin) Date: Fri, 21 Jul 2017 14:44:26 +0200 Subject: Request for review for JDK-8184045 - TestSystemGCWithG1.java times out on Solaris SPARC In-Reply-To: <98794b3a-ac78-26b5-8647-9f8f99aef048@oracle.com> References: <98794b3a-ac78-26b5-8647-9f8f99aef048@oracle.com> Message-ID: <455156a7-d4b2-364c-e5b6-e0ed804ae93b@oracle.com> On 07/21/2017 01:50 PM, Erik Helin wrote: > On 07/20/2017 05:31 PM, Alexander Harlap wrote: >> Please review changes forJDK-8184045 >> - >> TestSystemGCWithG1.java times out on Solaris SPARC (labeled as >> integration blocker) >> >> Proposed change will avoid timeout failure for new stress tests >> TestSystemGCWithG1.java, TestSystemGCWithCMS.java and >> TestSystemGCWithSerial.java regularly observed on Solaris Sparc >> >> Change is located at >> http://cr.openjdk.java.net/~aharlap/8184045/webrev.00/ > > Instead of removing the second pass, can you check if the args[0] == > "long"? The code can look like: > > public static void main(String[] args) { > populateLongLived(); > runAllPhases(); > if (args.length > 0 && args[0] == "long") { ...aaaand as couple of my colleagues pointed out while smiling, you should of course write args[0].equals("long") :) Thanks, Erik > runAllPhases(); > } > } > > This way the stress test can be used in "long" and "short" mode. > > Thanks, > Erik > >> Alex >> From erik.helin at oracle.com Fri Jul 21 12:48:40 2017 From: erik.helin at oracle.com (Erik Helin) Date: Fri, 21 Jul 2017 14:48:40 +0200 Subject: RFR(XL): 8181917: Refactor UL LogStreams to avoid using resource area In-Reply-To: References: <337330d2-512c-bc77-5a55-c05d33e376e5@oracle.com> <9b09acae-5cf6-f7dd-5195-99d65588b943@oracle.com> Message-ID: On 07/21/2017 02:23 PM, Thomas St?fe wrote: > Great, thanks a lot Eric and Marcus for your review work! > > Note I still have not yet time to run current iteration of the patch > fully thru jtreg. I did full jtreg tests on one of the earlier versions, > so I do not expect any surprises. Thank you for sponsoring! I did run quite a bit on testing on the patch and it looked good :) I have already pushed the patch: http://hg.openjdk.java.net/jdk10/hs/hotspot/rev/ff28370e679e Thanks, Erik > Kind Regards, Thomas > > On Fri, Jul 21, 2017 at 10:24 AM, Erik Helin > wrote: > > On 07/20/2017 09:11 AM, Thomas St?fe wrote: > > Hi all, > > after the last irc discussion with Erik here the new webrev with > the changes he requested: > > Full: > http://cr.openjdk.java.net/~stuefe/webrevs/8181917-refactor-ul-logstream/all.webrev.05/webrev/ > > Delta: > http://cr.openjdk.java.net/~stuefe/webrevs/8181917-refactor-ul-logstream/delta-04-to-05/webrev/ > > > > This looks good to me, Reviewed. Thank you Thomas for "hanging in > there" with this big patch, really solid work! > > I built the change on windows (32bit too :P), Linux and AIX. I > am currently re-running jtreg tests on Linux, no issues so far. > > > I took the patch for a spin in our build system and it worked fine. > I will sponsor this patch. > > Thanks, > Erik > > The changes: > > - I removed all stray "Unconditional write?" comments I had used > before to mark sites which did logging unconditionally. Marcus > had opened a follow up bug report for that issue > (https://bugs.openjdk.java.net/browse/JDK-8182466 > ), and I added > the affected sites I found to the comment section of that bug. > > - In g1GCPhaseTimes.cpp, I changed the logging code in > G1GCPhaseTimes::debug_phase() to use LogTarget instead of Log to > be consistent with G1GCPhaseTimes::trace_phase() > > - In the new LogStream implementation (logStream.cpp), I added > temporary variables in LogStream::LineBuffer::append() and > LogStream::LineBuffer::try_ensure_cap(). > > - I also enlarge the line buffer a bit more aggressive. We now, > once the initial 64byte buffer is exhausted - enlarge by 256 > bytes steps. > > - Finally, I renamed LineBuffer::ptr() to LineBuffer::buffer() > A note about memory consumption of the LogStream line buffer: I > do not think we need to overly worry about memory consumption, > at least not if we think the old solution was fine. Old solution > was using stringStream, which uses NEW_RESOURCE_ARRAY to enlarge > the line buffer when exhausted. This is resource area, so it > *always* leaks the previous allocation, which is not deallocated > until the resource area stack is reset. Even worse, for larger > line lengths the growth strategy in stringStream() seems to be > to just enlarge by the requested amount, which means that every > single write would cause a reallocation and hence a new > allocation of the underlying buffer (see stringStream::write()). > I am not 100% sure, this is just from dry-reading the code. > > Kind Regards, Thomas > > > > On Mon, Jul 17, 2017 at 3:44 PM, Thomas St?fe > > >> wrote: > > Hi Eric, > > thank you for your review! > > new Webrev: > > http://cr.openjdk.java.net/~stuefe/webrevs/8181917-refactor-ul-logstream/all.webrev.04/webrev/ > > > > > delta to last: > > http://cr.openjdk.java.net/~stuefe/webrevs/8181917-refactor-ul-logstream/delta-03-to-04/webrev/ > > > > > > Changes: > > - I changed the allocation function to handle the case where we > already reached the reasonable_max cap. You are right, in > that case > reallocating is unnecessary. > - Also, added log_info() for that case as you suggested. > - Finally, added a fix to the LogStream gtest which somehow > slipped > thru the last webrev. > > Changes are now rebased to the current tip. > > Thanks! > > ..Thomas > > On Fri, Jul 14, 2017 at 12:36 PM, Erik Helin > > >> wrote: > > On 07/13/2017 08:46 AM, Thomas St?fe wrote: > > Hi Erik! > > > On Mon, Jul 10, 2017 at 11:40 AM, Erik Helin > > > > > >>> wrote: > > Hi Thomas, > > On 06/30/2017 08:32 PM, Thomas St?fe wrote: > > Hi Eric, > > thank you for the review! > > New > Version: > > http://cr.openjdk.java.net/~stuefe/webrevs/8181917-refactor-ul-logstream/all.webrev.03/webrev/ > > > > > > > > >> > > > I think you still have a problem with "runaway" > loggers > in this version: > > +// try_ensure_cap tries to enlarge the > capacity of the > internal buffer > +// to the given atleast value. May fail if > either OOM > happens or > atleast > +// is larger than a reasonable max of 1 M. > Caller must > not assume > +// capacity without checking. > +void LogStream::LineBuffer::try_ensure_cap(size_t > atleast) { > + assert(_cap >= sizeof(_smallbuf), "sanity"); > + if (_cap < atleast) { > + const size_t reasonable_max = 1 * M; > + size_t newcap = align_size_up(atleast + > 64, 64); > + assert(_cap <= reasonable_max, "sanity"); > + // Cap out at a reasonable max to prevent > runaway > leaks. > + if (newcap > reasonable_max) { > + newcap = reasonable_max; > + } > + > + char* const newbuf = (char*) > os::malloc(newcap, > mtLogging); > + if (newbuf == NULL) { // OOM. Leave object > unchanged. > + return; > + } > + if (_pos > 0) { // preserve old content > + memcpy(newbuf, _buf, _pos + 1); // > ..including > trailing zero > + } > + if (_buf != _smallbuf) { > + os::free(_buf); > + } > + _buf = newbuf; > + _cap = newcap; > + } > + assert(_cap >= atleast, "sanity"); > +} > > with the above code, even though _cap > > reasonable_max, > you will > always allocate newbuf with os::malloc. For debug > builds, we are > fine, since the JVM will assert on _cap <= > reasonable_max, but for > product builds we are not. Would it be better > to first > check if _cap > == reasonable_max? As in the following snippet: > > void LogStream::LineBuffer::try_ensure_cap(size_t > atleast) { > if (_cap < atleast) { > const size_t reasonable_max = 1 * M; > if (_cap == reasonable_max) { > // max memory used, "die" in debug > builds, log > in product builds > log_warning(logging)("Max memory used for > message: %s\n", _buf); > ShouldNotReachHere(); > return; > } > > size_t newcap = align_size_up(atleast + > 64, 64); > if (newcap > reasonable_max) { > newcap = reasonable_max; > } > char* const newbuf = (char*) > os::malloc(newcap, > mtLogging); > // rest of logic > } > } > > Thanks, > Erik > > > I'm not sure I understand. Are you concerned that > _cap would > grow beyond reasonable_max - because I do not see a > way this > could happen. Or do you want to also treat the > *attempt* to > grow the _cap beyond reasonable_max as a loggable > offence? > > > What I am trying to say is that with your current patch, > even if > _cap == reasonable_max, you will still allocate a new 1M > buffer, > copy all the chars over, and the free the old one. My > proposal > simply skips the allocation, copying and freeing if _cap == > reasonable_max. The result will be the same, just less > stress on > malloc. > > If the latter, I am not sure about logging. If this > is an > error which may happen at a customer site without > popping up > first in inhouse-tests (which it conceivably is, because > logging is very content dependent), I would rather > hide the > error in production code instead of flooding stderr with > tons of log warnings or crashing with a guarantee, no? > > > I see what you mean, but perhaps log_info(logging) at > least? I > want to be able to discover if this happens in > production, for > example if a customer complains about truncated logs. A > log_info(logging) will only be printed if > -Xlog:logging=info is > specified on the command-line. > > Thanks, > Erik > > ..Thomas > > > > > From aph at redhat.com Fri Jul 21 13:43:43 2017 From: aph at redhat.com (Andrew Haley) Date: Fri, 21 Jul 2017 14:43:43 +0100 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: <5970B3FD.607@oracle.com> References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <596CD126.6080100@oracle.com> <596DD713.6080806@oracle.com> <77d5a9e7-b2b0-79e4-effd-903dfe226512@redhat.com> <596E1142.6040708@oracle.com> <3432d11c-5904-a446-1cb5-419627f1794f@redhat.com> <596E288B.90706@oracle.com> <773a0b64-8273-1a58-a46a-22ae18f927b7@redhat.com> <5970B3FD.607@oracle.com> Message-ID: <8e1d9a32-7e01-b917-df1d-595c4000e4b2@redhat.com> Hi, On 20/07/17 14:45, Erik ?sterlund wrote: > It seems like the aliasing problem has multiple levels at which we can > debate. I will try to put my thoughts down in a reasonably structured > way... let's see how that goes. > > These are the core questions we are debating (and I will answer them in > order): > > 1) What does the standard say about aliasing rules? > 2) What did concrete implementations like GCC implement the aliasing rules? > 3) What are the implications of those aliasing rules being violated? > Does it matter? > 4) Have these changes actually changed anything relating to those > aliasing rules? > 5) How do we proceed? > > Regarding #1: We seem to agree that there are various possible > interpretations of the standard (wouldn't be the first time...); at > least one interpretation that violates treating intptr_t as > compatible with A* and at least one interpretation that permits > it. I do not know if there is any one true interpretation of the C++ > standard, so I will assume the possibility that either one of us > could be right here, and hence that a compiler writer might write a > compiler in which it is either permitted aliasing or not. > > Regarding #2: While the simplest most contrived example of what we > are arguing about being a strict aliasing violation is not caught as > a strict aliasing violation using the strictest strict aliasing > checks available in GCC, I feel dubious, but will take your word for > it that intptr_t and A* are not compatible types. So for the rest of > the discussion, I will assume this is true. OK. For the sake of the discussion I'll go along with this. > Regarding #3: We already inherently rely on passed in intptr_t or even > jlong aliasing A*, possibly today enforced through -fno-strict-aliasing. > I do not think the JVM will ever be able to turn such optimizations on. I completely agree. However, if we are using -fno-strict-aliasing, then I do not believe that there is any need for the complexity of the casting that your patch does. We should be able to do the operations that we need with no type casting at all, with suitable use of template functions. > Our C++ runtime lives in the border lands between Java and C++ that I > will refer to as "shady land". Yep. > Regarding #4: I have made changes in the connection between the > frontend and the backend. But I would like to maintain that the > backends retain the same behaviour as they did before. For example, > your xchg_ptr for void* casts (C-style) the exchange_value to > intptr_t and use the intptr_t variant of xchg_ptr that passes > intptr_t (a.k.a. int64_t on your architecture) into > _sync_lock_test_and_set. So whatever hypothetical aliasing problem > the new Atomic implementation class were still there before these > changes. I have not introduced any of those aliasing issues, only > improved the mediation between the frontend to the platform > dependent backend. True. But this is new code, and IMO should do better, and also IMO does not need to be significantly more complex than the already complex code we have. I assume that the complexity is due to some compilers I don't know about, though, so I admit that I am at something of a disadvantage. I suspect that none of it is necessary for a complete and correct implementation on GCC. By "the complexity" I am referring to a 4k line patch. I fear that not only is it complex, it may also inhibit useful compiler optimizations. > Regarding #5: Now as for how to move on with this, I hope we can > agree that for code in "shady land", the compiler simply has to > treat intptr_t as possible aliases to A* and that if it does not, > then we need to tame it to do so. Because the C++ compiler simply > does not know the full story of what is going on in our program and > can not safely make any meaningful points-to analysis and data flow > optimizations crossing dynamically generated JIT-compiled machine > code of a different language. > > I propose three different solutions that we can discuss: > > Solution A: Do nothing. With this solution we recognize that a) we > have actually not introduced any undefined behaviour that was not > already there - the backend used intptr_t before and continues to do > so, and b) doing so is safe (and inherently has to be safe) due to > other levels of safeguarding such as turning off strict aliasing and > using volatile. > > Solution B: If intptr_t does indeed not alias A*, we could find a > pointer sized type, let's call it CanonicalPointer, that does indeed > alias a generic A*, and pass pointer types from the frontend as > CanonicalPointer to the backend. For example char* is explicitly > described in the standard as an exception of the rules that is > explicitly aliased to a generic A* (hopefully we agree about > that). We don't. A char* can point to the bytes of any type, but this rule does not mean that an object of type char* can be accessed by an lvalue of some other pointer type. They aren't the same thing at all. The questions "what can a char* point to?" and "what can point to a char* ?" are quite different. An int** can't point to a char*. The only permission we have is that any pointer can be cast to a character pointer, and that character pointer points to the first byte of the object. With regard to a solution, I feel there may be a much simpler answer. But in order to sketch out that simpler answer, I need to know exactly what problem is being solved. I presume that there is a need to turn a call to a generalized function such as template inline static void store(T store_value, volatile U* dest); into a call to one of a number of specialized functions such as template <> inline void Atomic::specialized_store(int64_t store_value, volatile int64_t* dest) { _Atomic_move_long(&store_value, dest); } and also perform some suitable magic for floating-point types. But do we ever use atomic floats for anything? Will we ever? -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From adinn at redhat.com Fri Jul 21 13:47:02 2017 From: adinn at redhat.com (Andrew Dinn) Date: Fri, 21 Jul 2017 14:47:02 +0100 Subject: RFR (S) 8182397: Race in field updates when creating ArrayKlasses can lead to crash In-Reply-To: References: Message-ID: Hi Coleen, On 21/07/17 00:44, coleen.phillimore at oracle.com wrote: > Summary: Update array_klass field in component mirror after > klass.java_mirror field for concurrent readers in compiled code. > > See bug for details. > > open webrev at http://cr.openjdk.java.net/~coleenp/8182397.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8182397 > > Ran tier1 testing with -Xcomp and without. Thank you to Tom for the > test and diagnosis. The re-ordering of the writes to k->set_java_mirror(mirror()); . . . set_array_klass(comp_mirror(), k); is all that is needed, assuming that the second write has release semantics. I discussed this with Andrew Haley and he confirmed my assumption (also Tom's) that on AArch64 there is no need for a load acquire on the reader side because of the address dependency between any load of the component mirror's array klass k and ensuing load of k's mirror. However, I think your implementation of method metadata_field_put_volatile() is overkill. You have void oopDesc::metadata_field_put_volatile(int offset, Metadata* value) { OrderAccess::release(); *metadata_field_addr(offset) = value; OrderAccess::fence(); } That fence() will impose a full memory barrier which is more than is actually needed. It should be ok just to use another release. void oopDesc::metadata_field_put_volatile(int offset, Metadata* value) { OrderAccess::release(); *metadata_field_addr(offset) = value; OrderAccess::release(); } Alternatively, I believe you could implement the assign of using a call to OrderAccess::store_release_ptr which would achieve the same outcome on AArch64. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From coleen.phillimore at oracle.com Fri Jul 21 14:23:01 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 21 Jul 2017 10:23:01 -0400 Subject: RFR (S) 8182397: Race in field updates when creating ArrayKlasses can lead to crash In-Reply-To: References: Message-ID: <7a44ced1-40f9-3717-8403-e920aad607ac@oracle.com> Andrew, Thank you for your comments and suggestions. On 7/21/17 9:47 AM, Andrew Dinn wrote: > Hi Coleen, > > On 21/07/17 00:44, coleen.phillimore at oracle.com wrote: >> Summary: Update array_klass field in component mirror after >> klass.java_mirror field for concurrent readers in compiled code. >> >> See bug for details. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8182397.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8182397 >> >> Ran tier1 testing with -Xcomp and without. Thank you to Tom for the >> test and diagnosis. > The re-ordering of the writes to > > k->set_java_mirror(mirror()); > . . . > set_array_klass(comp_mirror(), k); > > is all that is needed, assuming that the second write has release > semantics. I discussed this with Andrew Haley and he confirmed my > assumption (also Tom's) that on AArch64 there is no need for a load > acquire on the reader side because of the address dependency between any > load of the component mirror's array klass k and ensuing load of k's mirror. > > However, I think your implementation of method > metadata_field_put_volatile() is overkill. You have > > void oopDesc::metadata_field_put_volatile(int offset, Metadata* value) { > OrderAccess::release(); > *metadata_field_addr(offset) = value; > OrderAccess::fence(); > } I copied this from void oopDesc::obj_field_put_volatile(int offset, oop value) { OrderAccess::release(); obj_field_put(offset, value); OrderAccess::fence(); } Is it overkill also in the obj_field_put_volatile case? > > That fence() will impose a full memory barrier which is more than is > actually needed. It should be ok just to use another release. > > void oopDesc::metadata_field_put_volatile(int offset, Metadata* value) { > OrderAccess::release(); > *metadata_field_addr(offset) = value; > OrderAccess::release(); > } > > Alternatively, I believe you could implement the assign of using a call > to OrderAccess::store_release_ptr which would achieve the same outcome > on AArch64. This is nicer. Is this what you mean? void oopDesc::metadata_field_put_volatile(int offset, Metadata* value) { OrderAccess::release_store_ptr(metadata_field_addr(offset), value); } Thanks! Coleen > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From adinn at redhat.com Fri Jul 21 15:23:40 2017 From: adinn at redhat.com (Andrew Dinn) Date: Fri, 21 Jul 2017 16:23:40 +0100 Subject: RFR (S) 8182397: Race in field updates when creating ArrayKlasses can lead to crash In-Reply-To: <7a44ced1-40f9-3717-8403-e920aad607ac@oracle.com> References: <7a44ced1-40f9-3717-8403-e920aad607ac@oracle.com> Message-ID: <4a8327a1-a6a3-8d16-b24a-646785fe7b0a@redhat.com> On 21/07/17 15:23, coleen.phillimore at oracle.com wrote: > Andrew, Thank you for your comments and suggestions. All jus' part of the job, ma'am ... :-) > I copied this from > > void oopDesc::obj_field_put_volatile(int offset, oop value) { > OrderAccess::release(); > obj_field_put(offset, value); > OrderAccess::fence(); > } > > Is it overkill also in the obj_field_put_volatile case? Well, I would assume from the name that obj_field_put_volatile is used to do a volatile write internally from the JVM with the intention that it be equivalent to doing a volatile write from JDK code. If so then, in general, I /guess/ that might perhaps demand a full fence even though on some platforms (like AArch64) we /can/ and /do/ implement volatile writes in JITted code using only a releasing store. Andrew Haley might know better on this score. That said, the method you have defined is only (currently) used to ensure that the two stores for the mirror and array klass are ordered correctly as regards a read through from klass to mirror. That usage definitely only needs a releasing store (assuming the memory system architecture respects address dependencies). In which case it might be more appropriate to rename the method you inserted to metadata_field_put_release -- highlighting the fact that this is all the order guarantee it provides. >> Alternatively, I believe you could implement the assign of using a call >> to OrderAccess::store_release_ptr which would achieve the same outcome >> on AArch64. > > This is nicer. Is this what you mean? > > void oopDesc::metadata_field_put_volatile(int offset, Metadata* value) { > OrderAccess::release_store_ptr(metadata_field_addr(offset), value); > } Yes, that was what I was thinking of. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From coleen.phillimore at oracle.com Fri Jul 21 15:34:23 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 21 Jul 2017 11:34:23 -0400 Subject: RFR(xs): 8185033: On Metaspace OOM, ChunkManager composition should be logged. In-Reply-To: References: Message-ID: Hi Thomas, Thank you for working on this. On 7/21/17 2:54 AM, Thomas St?fe wrote: > Dear all, > > May I please get reviews and a sponsor for the following small enhancement. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8185033 > Webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/8185033-On-Metaspace-OOM-chunkManager-composition-should-be-logged/webrev.00/webrev/ > > This patch adds a bit of logging if a Metaspace OOM happens. We will now > get a printout of the composition of the global chunk managers (non-class > and, if necessary, class). > > Before it looked like this: > > [272.727s][info][gc,metaspace,freelist] All Metaspace: > [272.728s][info][gc,metaspace,freelist] data space: Chunk accounting: > used in chunks 100752K + unused in chunks 650K + capacity in free chunks > 8293K = 109696K capacity in allocated chunks 101403K > [272.728s][info][gc,metaspace,freelist] class space: Chunk accounting: > used in chunks 4047K + unused in chunks 97K + capacity in free chunks > 6085K = 10230K capacity in allocated chunks 4145K > [272.729s][info][gc,metaspace,freelist] Total fragmentation waste (words) > doesn't count free space > [272.729s][info][gc,metaspace,freelist] data: 183 specialized(s) 960, 169 > small(s) 41, 1507 medium(s) 4480, large count 1 > [272.729s][info][gc,metaspace,freelist] class: 183 specialized(s) 0, 13 > small(s) 12, 111 medium(s) 8, large count 1 > > Now: > [250.548s][info][gc,metaspace,freelist] All Metaspace: > [250.549s][info][gc,metaspace,freelist] data space: Chunk accounting: > used in chunks 100748K + unused in chunks 638K + capacity in free chunks > 10357K = 111744K capacity in allocated chunks 101387K > [250.550s][info][gc,metaspace,freelist] class space: Chunk accounting: > used in chunks 4047K + unused in chunks 97K + capacity in free chunks > 6085K = 10230K capacity in allocated chunks 4145K > [250.550s][info][gc,metaspace,freelist] Total fragmentation waste (words) > doesn't count free space > [250.550s][info][gc,metaspace,freelist] data: 183 specialized(s) 960, 165 > small(s) 27, 1507 medium(s) 4504, large count 1 > [250.550s][info][gc,metaspace,freelist] class: 183 specialized(s) 0, 13 > small(s) 12, 111 medium(s) 14, large count 1 > [250.550s][info][gc,metaspace,freelist] Chunkmanager (non-class): > [250.550s][info][gc,metaspace,freelist] 89 specialized (128 bytes) > chunks, total 91136 bytes > [250.550s][info][gc,metaspace,freelist] 2567 small (512 bytes) chunks, > total 10514432 bytes > [250.550s][info][gc,metaspace,freelist] 0 medium (8192 bytes) chunks, > total 0 bytes > [250.550s][info][gc,metaspace,freelist] 0 humongous chunks, total 0 bytes > [250.550s][info][gc,metaspace,freelist] total size: 10605568. > [250.550s][info][gc,metaspace,freelist] Chunkmanager (class): > [250.550s][info][gc,metaspace,freelist] 87 specialized (128 bytes) > chunks, total 89088 bytes > [250.550s][info][gc,metaspace,freelist] 2999 small (256 bytes) chunks, > total 6141952 bytes > [250.550s][info][gc,metaspace,freelist] 0 medium (4096 bytes) chunks, > total 0 bytes > [250.550s][info][gc,metaspace,freelist] 0 humongous chunks, total 0 bytes > [250.550s][info][gc,metaspace,freelist] total size: 6231040. Yes, this looks like it'll help. Your numbers are less mystifying than the existing ones. > > This helps in understanding the underlying cause of the OOM, if that cause > is related to chunks lingering around in the freelist. See also [1], a Jep > proposing to improve the metaspace chunk allocation to increase the chance > of chunk reuse. This patch here is also a preparation for the prototype > work of [1]. > > Note that admittedly, metaspace.cpp will again gain a bit of complexity by > this patch. When examining the coding, I was tempted to refactor but > refrained from doing so to keep the patch focused. But I think > metaspace.cpp (especially the coding associated with statistics) could be > cleaned up a bit. I opened a follow up item [2] to collect cleanup-issues. > I am willing to do that myself, but would prefer it to be done after > functional patches - which still have a slight chance of being backported > to jdk9 - are all being added. My first impression was that get_statistics() and print_statistics() should be member functions of ChunkManagerStatistics because of all the stat->lines. But ChunkManager has to own these also. I can't think of anything better. Yes, the metaspace code could definitely use refactoring with respect to statistics collection. Please! Also I added to your RFE to rename MetaspaceAux. I agree with Mikael about adding the code to avoid casting. You could change the type to 'int' with another patch (or this one, I don't mind looking at it again). I could sponsor this when it's ready. Thanks! Coleen > > Patch was built and tested on Windows x64, Linux x64 and AIX. I also ran > jtreg hotspot/runtime/Metaspace tests, no issues. > > Kind Regards, Thomas > > ---- > > [1] https://bugs.openjdk.java.net/browse/JDK-8166690 > [2] https://bugs.openjdk.java.net/browse/JDK-8185034 From erik.osterlund at oracle.com Fri Jul 21 16:08:02 2017 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 21 Jul 2017 18:08:02 +0200 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: <8e1d9a32-7e01-b917-df1d-595c4000e4b2@redhat.com> References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <596CD126.6080100@oracle.com> <596DD713.6080806@oracle.com> <77d5a9e7-b2b0-79e4-effd-903dfe226512@redhat.com> <596E1142.6040708@oracle.com> <3432d11c-5904-a446-1cb5-419627f1794f@redhat.com> <596E288B.90706@oracle.com> <773a0b64-8273-1a58-a46a-22ae18f927b7@redhat.com> <5970B3FD.607@oracle.com> <8e1d9a32-7e01-b917-df1d-595c4000e4b2@redhat.com> Message-ID: <597226E2.6090803@oracle.com> Hi Andrew, On 2017-07-21 15:43, Andrew Haley wrote: > Hi, > > On 20/07/17 14:45, Erik ?sterlund wrote: > >> It seems like the aliasing problem has multiple levels at which we can >> debate. I will try to put my thoughts down in a reasonably structured >> way... let's see how that goes. >> >> These are the core questions we are debating (and I will answer them in >> order): >> >> 1) What does the standard say about aliasing rules? >> 2) What did concrete implementations like GCC implement the aliasing rules? >> 3) What are the implications of those aliasing rules being violated? >> Does it matter? >> 4) Have these changes actually changed anything relating to those >> aliasing rules? >> 5) How do we proceed? >> >> Regarding #1: We seem to agree that there are various possible >> interpretations of the standard (wouldn't be the first time...); at >> least one interpretation that violates treating intptr_t as >> compatible with A* and at least one interpretation that permits >> it. I do not know if there is any one true interpretation of the C++ >> standard, so I will assume the possibility that either one of us >> could be right here, and hence that a compiler writer might write a >> compiler in which it is either permitted aliasing or not. >> >> Regarding #2: While the simplest most contrived example of what we >> are arguing about being a strict aliasing violation is not caught as >> a strict aliasing violation using the strictest strict aliasing >> checks available in GCC, I feel dubious, but will take your word for >> it that intptr_t and A* are not compatible types. So for the rest of >> the discussion, I will assume this is true. > OK. For the sake of the discussion I'll go along with this. > >> Regarding #3: We already inherently rely on passed in intptr_t or even >> jlong aliasing A*, possibly today enforced through -fno-strict-aliasing. >> I do not think the JVM will ever be able to turn such optimizations on. > I completely agree. However, if we are using -fno-strict-aliasing, > then I do not believe that there is any need for the complexity of the > casting that your patch does. We should be able to do the operations > that we need with no type casting at all, with suitable use of > template functions. > >> Our C++ runtime lives in the border lands between Java and C++ that I >> will refer to as "shady land". > Yep. > >> Regarding #4: I have made changes in the connection between the >> frontend and the backend. But I would like to maintain that the >> backends retain the same behaviour as they did before. For example, >> your xchg_ptr for void* casts (C-style) the exchange_value to >> intptr_t and use the intptr_t variant of xchg_ptr that passes >> intptr_t (a.k.a. int64_t on your architecture) into >> _sync_lock_test_and_set. So whatever hypothetical aliasing problem >> the new Atomic implementation class were still there before these >> changes. I have not introduced any of those aliasing issues, only >> improved the mediation between the frontend to the platform >> dependent backend. > True. But this is new code, and IMO should do better, and also IMO > does not need to be significantly more complex than the already > complex code we have. I assume that the complexity is due to some > compilers I don't know about, though, so I admit that I am at > something of a disadvantage. I suspect that none of it is necessary > for a complete and correct implementation on GCC. By "the complexity" > I am referring to a 4k line patch. I fear that not only is it > complex, it may also inhibit useful compiler optimizations. > >> Regarding #5: Now as for how to move on with this, I hope we can >> agree that for code in "shady land", the compiler simply has to >> treat intptr_t as possible aliases to A* and that if it does not, >> then we need to tame it to do so. Because the C++ compiler simply >> does not know the full story of what is going on in our program and >> can not safely make any meaningful points-to analysis and data flow >> optimizations crossing dynamically generated JIT-compiled machine >> code of a different language. >> >> I propose three different solutions that we can discuss: >> >> Solution A: Do nothing. With this solution we recognize that a) we >> have actually not introduced any undefined behaviour that was not >> already there - the backend used intptr_t before and continues to do >> so, and b) doing so is safe (and inherently has to be safe) due to >> other levels of safeguarding such as turning off strict aliasing and >> using volatile. >> >> Solution B: If intptr_t does indeed not alias A*, we could find a >> pointer sized type, let's call it CanonicalPointer, that does indeed >> alias a generic A*, and pass pointer types from the frontend as >> CanonicalPointer to the backend. For example char* is explicitly >> described in the standard as an exception of the rules that is >> explicitly aliased to a generic A* (hopefully we agree about >> that). > We don't. A char* can point to the bytes of any type, but this rule > does not mean that an object of type char* can be accessed by an > lvalue of some other pointer type. They aren't the same thing at all. > > The questions "what can a char* point to?" and "what can point to a > char* ?" are quite different. An int** can't point to a char*. The > only permission we have is that any pointer can be cast to a character > pointer, and that character pointer points to the first byte of the > object. I'm not sure if I was clear enough, pardon me if I was not, but the loophole I was referring to is section 3.10 where it says: ================================================ If a program attempts to access the stored value of an object through a glvalue of other than one of the following types the behavior is undefined: ? the dynamic type of the object, ? a cv-qualified version of the dynamic type of the object, ? a type similar (as defined in 4.4) to the dynamic type of the object, ? a type that is the signed or unsigned type corresponding to the dynamic type of the object, ? a type that is the signed or unsigned type corresponding to a cv-qualified version of the dynamic type of the object, ? an aggregate or union type that includes one of the aforementioned types among its elements or non- static data members (including, recursively, an element or non-static data member of a subaggregate or contained union), ? a type that is a (possibly cv-qualified) base class type of the dynamic type of the object, ? a char or unsigned char type. ================================================ I will make a store load pair of A* example split into three relevant cases: 1) A non-Atomic store is observed by an Atomic load 2) An Atomic store is observed by a non-Atomic load 3) An Atomic store is observed by an Atomic load Case 1: A non-Atomic store is observed by an Atomic load A non-Atomic store (possibly initialization code) stores an A* that is observed as char* and subsequently reinterpret_casted to A*. I claim that A* and char* are compatible types in this context. It is not undefined behaviour w.r.t. 3.10 if the type behind the pointer in a pointer load is a char, regardless of the dynamic type it actually points at. The load then has to assume conservatively that it does not know what those bytes are referring to - it might be anything (including the dynamic type behind the stored pointer: A). In other words, char aliases all dynamic types that the pointer could point at. It seems like we both agree this case is fine. Case 2: An Atomic store is observed by a non-Atomic load Conversely, if the Atomic API was to store something as a char* that actually has a dynamic type of A*, and a normal load observes this as an A*, that is also fine as the dynamic type is still A* despite being stored as char*. In other words, the load is invariant of what type the store had, w.r.t. 3.10, if its type matches the dynamic type. And it does. It seems to me like this is the case where you made a point that despite being stored as char*, that does not mean that a non-Atomic load of A* will understand the aliasing. And my point is that this is still not undefined behavior w.r.t. 3.10 as the non-Atomic load is true to the dynamic type. I hope we agree here. But then apart from the standard there is the problem of aliasing optimizations of specific compilers, like -fstrict-aliasing on GCC. Let's remember that in C++03, "strict aliasing" is not a thing. And a concrete points-to analysis might not know what the true dynamic type of a pointer is, so depending on how conservative the data flow analysis, points-to analysis and type-based aliasing analysis is, an aggressive compiler might make further assumptions than what the standard outlines as undefined behaviour, and as a result mess up the program. But then again, we turn such optimizations off, and they are arguably not supported by the standard. If we were to support that some day, perhaps CanonicalPointer would be better off as the union of A* and char*. But then as I mentioned, that is not enough because C++ programs can send, via JNI, a pointer as a jlong into Java that will try to perform a CAS through var handles and unsafe, and ultimately end up in a possible runtime call to Atomic performing that CAS with Atomic::cmpxchg of jlong. Therefore, we must inherently be able to deal with the dynamic type being destroyed in "shady land" long before it reaches Atomic. It will seemingly just never work reliably with -fstrict-aliasing, as far as I can tell. Case 3: An Atomic store is observed by an Atomic load This seems trivially compatible as the store and load pair are communicating through the same type, char* that are compatible and aliased. > With regard to a solution, I feel there may be a much simpler answer. > But in order to sketch out that simpler answer, I need to know exactly > what problem is being solved. I presume that there is a need to turn > a call to a generalized function such as > > template > inline static void store(T store_value, volatile U* dest); > > into a call to one of a number of specialized functions such as > > template <> > inline void Atomic::specialized_store(int64_t store_value, volatile int64_t* dest) { > _Atomic_move_long(&store_value, dest); > } Yes that's about it. Without messing things up of course w.r.t T and U having different signedness, size and type. > and also perform some suitable magic for floating-point types. But do > we ever use atomic floats for anything? Will we ever? > That is a good point. I am not certain. But the current API supports it, so it seemed like a good idea to continue supporting it. Thanks, /Erik From aph at redhat.com Fri Jul 21 16:26:35 2017 From: aph at redhat.com (Andrew Haley) Date: Fri, 21 Jul 2017 17:26:35 +0100 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: <597226E2.6090803@oracle.com> References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <596CD126.6080100@oracle.com> <596DD713.6080806@oracle.com> <77d5a9e7-b2b0-79e4-effd-903dfe226512@redhat.com> <596E1142.6040708@oracle.com> <3432d11c-5904-a446-1cb5-419627f1794f@redhat.com> <596E288B.90706@oracle.com> <773a0b64-8273-1a58-a46a-22ae18f927b7@redhat.com> <5970B3FD.607@oracle.com> <8e1d9a32-7e01-b917-df1d-595c4000e4b2@redhat.com> <597226E2.6090803@oracle.com> Message-ID: <2c637e28-8d95-cbaf-c29e-d01867ed9157@redhat.com> On 21/07/17 17:08, Erik ?sterlund wrote: > I will make a store load pair of A* example split into three relevant cases: > 1) A non-Atomic store is observed by an Atomic load > 2) An Atomic store is observed by a non-Atomic load > 3) An Atomic store is observed by an Atomic load > > Case 1: A non-Atomic store is observed by an Atomic load > A non-Atomic store (possibly initialization code) stores an A* that is > observed as char* and subsequently reinterpret_casted to A*. I claim > that A* and char* are compatible types in this context. It depends on exactly what this language means. What is the type of the lvalue? And the type of the stored object? If the type of the lvalue is any character type, it's fine. If the type of the lvalue is a pointer type other than the type of the stored object, it is not. > It is not undefined behaviour w.r.t. 3.10 if the type behind the pointer > in a pointer load is a char, regardless of the dynamic type it actually > points at. The load then has to assume conservatively that it does not > know what those bytes are referring to - it might be anything (including > the dynamic type behind the stored pointer: A). In other words, char > aliases all dynamic types that the pointer could point at. It seems like > we both agree this case is fine. > > Case 2: An Atomic store is observed by a non-Atomic load > Conversely, if the Atomic API was to store something as a char* that > actually has a dynamic type of A*, and a normal load observes this as an > A*, that is also fine as the dynamic type is still A* despite being > stored as char*. In other words, the load is invariant of what type the > store had, w.r.t. 3.10, if its type matches the dynamic type. And it does. > It seems to me like this is the case where you made a point that despite > being stored as char*, that does not mean that a non-Atomic load of A* > will understand the aliasing. And my point is that this is still not > undefined behavior w.r.t. 3.10 as the non-Atomic load is true to the > dynamic type. I hope we agree here. > > But then apart from the standard there is the problem of aliasing > optimizations of specific compilers, like -fstrict-aliasing on GCC. Well, they're not really separate: GCC does what the standard says it may do. > Let's remember that in C++03, "strict aliasing" is not a thing. And a > concrete points-to analysis might not know what the true dynamic type of > a pointer is, so depending on how conservative the data flow analysis, > points-to analysis and type-based aliasing analysis is, an aggressive > compiler might make further assumptions than what the standard outlines > as undefined behaviour, and as a result mess up the program. But then > again, we turn such optimizations off, and they are arguably not > supported by the standard. If we were to support that some day, perhaps > CanonicalPointer would be better off as the union of A* and char*. But > then as I mentioned, that is not enough because C++ programs can send, > via JNI, a pointer as a jlong into Java that will try to perform a CAS > through var handles and unsafe, and ultimately end up in a possible > runtime call to Atomic performing that CAS with Atomic::cmpxchg of > jlong. Therefore, we must inherently be able to deal with the dynamic > type being destroyed in "shady land" long before it reaches Atomic. It > will seemingly just never work reliably with -fstrict-aliasing, as far > as I can tell. I think it can, but it would require a sophisticated analysis of what we really mean. Having said that, I have a lot of far more interesting work to do, and I'm sure the same goes for you and everyone else whi is a HotSpot author, so it won't get done. > Case 3: An Atomic store is observed by an Atomic load > This seems trivially compatible as the store and load pair are > communicating through the same type, char* that are compatible and aliased. I may have misinterpreted what you were suggesting. It seemed to me that you said you'd have a canonical pointer type, and all of your pointer stores would go through something like void store_ptr(char **addr, char *ptr) { *addr = ptr; } and we could do something like long a_long; long *long_ptr; store_ptr((char**)&long_ptr, (char*)&a_long); to make long_ptr point to a_long. But I'm happy to drop this conversation, because we're agreed we're going to use -fno-strict-aliasing everywhere, so this kind of correctness doesn't matter. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Fri Jul 21 16:31:29 2017 From: aph at redhat.com (Andrew Haley) Date: Fri, 21 Jul 2017 17:31:29 +0100 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: <597226E2.6090803@oracle.com> References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <596CD126.6080100@oracle.com> <596DD713.6080806@oracle.com> <77d5a9e7-b2b0-79e4-effd-903dfe226512@redhat.com> <596E1142.6040708@oracle.com> <3432d11c-5904-a446-1cb5-419627f1794f@redhat.com> <596E288B.90706@oracle.com> <773a0b64-8273-1a58-a46a-22ae18f927b7@redhat.com> <5970B3FD.607@oracle.com> <8e1d9a32-7e01-b917-df1d-595c4000e4b2@redhat.com> <597226E2.6090803@oracle.com> Message-ID: On 21/07/17 17:08, Erik ?sterlund wrote: >> With regard to a solution, I feel there may be a much simpler answer. >> But in order to sketch out that simpler answer, I need to know exactly >> what problem is being solved. I presume that there is a need to turn >> a call to a generalized function such as >> >> template >> inline static void store(T store_value, volatile U* dest); >> >> into a call to one of a number of specialized functions such as >> >> template <> >> inline void Atomic::specialized_store(int64_t store_value, volatile int64_t* dest) { >> _Atomic_move_long(&store_value, dest); >> } > Yes that's about it. Without messing things up of course w.r.t T and U > having different signedness, size and type. OK. I'll sketch out a solution. It might not work, but I'll try. Thanks for your patience: I don't usually push back so hard when reviewing stuff, but I think this code is very important. >> and also perform some suitable magic for floating-point types. But do >> we ever use atomic floats for anything? Will we ever? >> > That is a good point. I am not certain. But the current API supports it, > so it seemed like a good idea to continue supporting it. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From alexander.harlap at oracle.com Fri Jul 21 17:17:50 2017 From: alexander.harlap at oracle.com (Alexander Harlap) Date: Fri, 21 Jul 2017 13:17:50 -0400 Subject: Request for review for JDK-8184045 - TestSystemGCWithG1.java times out on Solaris SPARC In-Reply-To: <455156a7-d4b2-364c-e5b6-e0ed804ae93b@oracle.com> References: <98794b3a-ac78-26b5-8647-9f8f99aef048@oracle.com> <455156a7-d4b2-364c-e5b6-e0ed804ae93b@oracle.com> Message-ID: <4afe0501-a927-7ecf-b229-63aeeecf76be@oracle.com> Hi Erik, Thank you for your suggestion. New version is at http://cr.openjdk.java.net/~aharlap/8184045/webrev.01/ Alex On 7/21/2017 8:44 AM, Erik Helin wrote: > On 07/21/2017 01:50 PM, Erik Helin wrote: >> On 07/20/2017 05:31 PM, Alexander Harlap wrote: >>> Please review changes forJDK-8184045 >>> - >>> TestSystemGCWithG1.java times out on Solaris SPARC (labeled as >>> integration blocker) >>> >>> Proposed change will avoid timeout failure for new stress tests >>> TestSystemGCWithG1.java, TestSystemGCWithCMS.java and >>> TestSystemGCWithSerial.java regularly observed on Solaris Sparc >>> >>> Change is located at >>> http://cr.openjdk.java.net/~aharlap/8184045/webrev.00/ >> >> Instead of removing the second pass, can you check if the args[0] == >> "long"? The code can look like: >> >> public static void main(String[] args) { >> populateLongLived(); >> runAllPhases(); >> if (args.length > 0 && args[0] == "long") { > > ...aaaand as couple of my colleagues pointed out while smiling, you > should of course write args[0].equals("long") :) > > Thanks, > Erik > >> runAllPhases(); >> } >> } >> >> This way the stress test can be used in "long" and "short" mode. >> >> Thanks, >> Erik >> >>> Alex >>> From coleen.phillimore at oracle.com Fri Jul 21 17:30:58 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 21 Jul 2017 13:30:58 -0400 Subject: RFR (S) 8182397: Race in field updates when creating ArrayKlasses can lead to crash In-Reply-To: <4a8327a1-a6a3-8d16-b24a-646785fe7b0a@redhat.com> References: <7a44ced1-40f9-3717-8403-e920aad607ac@oracle.com> <4a8327a1-a6a3-8d16-b24a-646785fe7b0a@redhat.com> Message-ID: On 7/21/17 11:23 AM, Andrew Dinn wrote: > On 21/07/17 15:23, coleen.phillimore at oracle.com wrote: >> Andrew, Thank you for your comments and suggestions. > All jus' part of the job, ma'am ... :-) > >> I copied this from >> >> void oopDesc::obj_field_put_volatile(int offset, oop value) { >> OrderAccess::release(); >> obj_field_put(offset, value); >> OrderAccess::fence(); >> } >> >> Is it overkill also in the obj_field_put_volatile case? > Well, I would assume from the name that obj_field_put_volatile is used > to do a volatile write internally from the JVM with the intention that > it be equivalent to doing a volatile write from JDK code. > > If so then, in general, I /guess/ that might perhaps demand a full fence > even though on some platforms (like AArch64) we /can/ and /do/ implement > volatile writes in JITted code using only a releasing store. Andrew > Haley might know better on this score. > > That said, the method you have defined is only (currently) used to > ensure that the two stores for the mirror and array klass are ordered > correctly as regards a read through from klass to mirror. That usage > definitely only needs a releasing store (assuming the memory system > architecture respects address dependencies). > > In which case it might be more appropriate to rename the method you > inserted to metadata_field_put_release -- highlighting the fact that > this is all the order guarantee it provides. Thank you. I'll rename it to release to be more clear. I'm not comfortable messing around with the other function, I was just curious. > >>> Alternatively, I believe you could implement the assign of using a call >>> to OrderAccess::store_release_ptr which would achieve the same outcome >>> on AArch64. >> This is nicer. Is this what you mean? >> >> void oopDesc::metadata_field_put_volatile(int offset, Metadata* value) { >> OrderAccess::release_store_ptr(metadata_field_addr(offset), value); >> } > Yes, that was what I was thinking of. Thank you for the code review. Coleen > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From thomas.stuefe at gmail.com Fri Jul 21 18:06:34 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 21 Jul 2017 20:06:34 +0200 Subject: RFR(xs): 8185033: On Metaspace OOM, ChunkManager composition should be logged. In-Reply-To: <6e4f97c6-550e-306b-123d-cec667064f39@oracle.com> References: <94a12182-d318-e423-644e-56ff918d717c@oracle.com> <6e4f97c6-550e-306b-123d-cec667064f39@oracle.com> Message-ID: Hi Mikael, On Fri, Jul 21, 2017 at 2:33 PM, Mikael Gerdin wrote: > Hi Thomas, > > On 2017-07-21 14:21, Thomas St?fe wrote: > >> Hi Mikael, >> >> thanks for looking at this! >> >> the const correctness stuff makes sense, I started to do this myself but >> reverted that part and added it as a Todo to >> https://bugs.openjdk.java.net/browse/JDK-8185034. But sure, we can do >> this in this change. >> >> As for making the counters size_t, I am not sure. I do not think that >> size_t is a good fit for numbers-of-things, size_t, to me, always means >> memory size or address range. I think counters are better represented by >> int or unsigned, if you do not like uint32_t. I also think that the >> original decision to make the chunk counters size_t is debatable. >> >> But I do not have strong emotions here, if you prefer I use size_t, I can >> use size_t. >> > > Since the accessors you get the values from return size_t I would prefer > if we keep the stats as size_t as well. Then we can change the type later > if we feel like it. > > /Mikael > > Okay, I will change the type to size_t! ..Thomas > >> Thanks, Thomas >> >> >> >> On Fri, Jul 21, 2017 at 10:46 AM, Mikael Gerdin > > wrote: >> >> Hi Thomas, >> >> >> On 2017-07-21 08:54, Thomas St?fe wrote: >> >> Dear all, >> >> May I please get reviews and a sponsor for the following small >> enhancement. >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8185033 >> >> Webrev: >> http://cr.openjdk.java.net/~stuefe/webrevs/8185033-On-Metasp >> ace-OOM-chunkManager-composition-should-be-logged/webrev.00/webrev/ >> > pace-OOM-chunkManager-composition-should-be-logged/webrev.00/webrev/> >> >> This patch adds a bit of logging if a Metaspace OOM happens. We >> will now >> get a printout of the composition of the global chunk managers >> (non-class >> and, if necessary, class). >> >> Before it looked like this: >> >> [272.727s][info][gc,metaspace,freelist] All Metaspace: >> [272.728s][info][gc,metaspace,freelist] data space: Chunk >> accounting: >> used in chunks 100752K + unused in chunks 650K + capacity in >> free chunks >> 8293K = 109696K capacity in allocated chunks 101403K >> [272.728s][info][gc,metaspace,freelist] class space: Chunk >> accounting: >> used in chunks 4047K + unused in chunks 97K + capacity in free >> chunks >> 6085K = 10230K capacity in allocated chunks 4145K >> [272.729s][info][gc,metaspace,freelist] Total fragmentation >> waste (words) >> doesn't count free space >> [272.729s][info][gc,metaspace,freelist] data: 183 >> specialized(s) 960, 169 >> small(s) 41, 1507 medium(s) 4480, large count 1 >> [272.729s][info][gc,metaspace,freelist] class: 183 >> specialized(s) 0, 13 >> small(s) 12, 111 medium(s) 8, large count 1 >> >> Now: >> [250.548s][info][gc,metaspace,freelist] All Metaspace: >> [250.549s][info][gc,metaspace,freelist] data space: Chunk >> accounting: >> used in chunks 100748K + unused in chunks 638K + capacity in >> free chunks >> 10357K = 111744K capacity in allocated chunks 101387K >> [250.550s][info][gc,metaspace,freelist] class space: Chunk >> accounting: >> used in chunks 4047K + unused in chunks 97K + capacity in free >> chunks >> 6085K = 10230K capacity in allocated chunks 4145K >> [250.550s][info][gc,metaspace,freelist] Total fragmentation >> waste (words) >> doesn't count free space >> [250.550s][info][gc,metaspace,freelist] data: 183 >> specialized(s) 960, 165 >> small(s) 27, 1507 medium(s) 4504, large count 1 >> [250.550s][info][gc,metaspace,freelist] class: 183 >> specialized(s) 0, 13 >> small(s) 12, 111 medium(s) 14, large count 1 >> [250.550s][info][gc,metaspace,freelist] Chunkmanager (non-class): >> [250.550s][info][gc,metaspace,freelist] 89 specialized (128 >> bytes) >> chunks, total 91136 bytes >> [250.550s][info][gc,metaspace,freelist] 2567 small (512 bytes) >> chunks, >> total 10514432 bytes >> [250.550s][info][gc,metaspace,freelist] 0 medium (8192 bytes) >> chunks, >> total 0 bytes >> [250.550s][info][gc,metaspace,freelist] 0 humongous chunks, >> total 0 bytes >> [250.550s][info][gc,metaspace,freelist] total size: 10605568. >> [250.550s][info][gc,metaspace,freelist] Chunkmanager (class): >> [250.550s][info][gc,metaspace,freelist] 87 specialized (128 >> bytes) >> chunks, total 89088 bytes >> [250.550s][info][gc,metaspace,freelist] 2999 small (256 bytes) >> chunks, >> total 6141952 bytes >> [250.550s][info][gc,metaspace,freelist] 0 medium (4096 bytes) >> chunks, >> total 0 bytes >> [250.550s][info][gc,metaspace,freelist] 0 humongous chunks, >> total 0 bytes >> [250.550s][info][gc,metaspace,freelist] total size: 6231040. >> >> This helps in understanding the underlying cause of the OOM, if >> that cause >> is related to chunks lingering around in the freelist. See also >> [1], a Jep >> proposing to improve the metaspace chunk allocation to increase >> the chance >> of chunk reuse. This patch here is also a preparation for the >> prototype >> work of [1]. >> >> Note that admittedly, metaspace.cpp will again gain a bit of >> complexity by >> this patch. When examining the coding, I was tempted to refactor >> but >> refrained from doing so to keep the patch focused. But I think >> metaspace.cpp (especially the coding associated with statistics) >> could be >> cleaned up a bit. I opened a follow up item [2] to collect >> cleanup-issues. >> I am willing to do that myself, but would prefer it to be done >> after >> functional patches - which still have a slight chance of being >> backported >> to jdk9 - are all being added. >> >> >> I know this is a bit counter to what you said about keeping the >> patch small but what do you think about doing this on top of your >> change? >> http://cr.openjdk.java.net/~mgerdin/8185033/webrev.00/src/sh >> are/vm/memory/metaspace.cpp.udiff.html >> > hare/vm/memory/metaspace.cpp.udiff.html> >> >> I'm not sure why you went with uint32_t for the lengths since it's >> just a couple of bytes and you can get rid of the casts. >> I also took the liberty to squash out two const_casts which were >> making my eyes twitch... :) >> >> /Mikael >> >> >> >> Patch was built and tested on Windows x64, Linux x64 and AIX. I >> also ran >> jtreg hotspot/runtime/Metaspace tests, no issues. >> >> Kind Regards, Thomas >> >> ---- >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8166690 >> >> [2] https://bugs.openjdk.java.net/browse/JDK-8185034 >> >> >> >> From thomas.stuefe at gmail.com Fri Jul 21 18:16:55 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 21 Jul 2017 20:16:55 +0200 Subject: RFR(xs): 8185033: On Metaspace OOM, ChunkManager composition should be logged. In-Reply-To: References: Message-ID: Hi Coleen, On Fri, Jul 21, 2017 at 5:34 PM, wrote: > > Hi Thomas, > Thank you for working on this. > > thanks for reviewing! > > On 7/21/17 2:54 AM, Thomas St?fe wrote: > >> Dear all, >> >> May I please get reviews and a sponsor for the following small >> enhancement. >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8185033 >> Webrev: >> http://cr.openjdk.java.net/~stuefe/webrevs/8185033-On-Metasp >> ace-OOM-chunkManager-composition-should-be-logged/webrev.00/webrev/ >> >> This patch adds a bit of logging if a Metaspace OOM happens. We will now >> get a printout of the composition of the global chunk managers (non-class >> and, if necessary, class). >> >> Before it looked like this: >> >> [272.727s][info][gc,metaspace,freelist] All Metaspace: >> [272.728s][info][gc,metaspace,freelist] data space: Chunk accounting: >> used in chunks 100752K + unused in chunks 650K + capacity in free chunks >> 8293K = 109696K capacity in allocated chunks 101403K >> [272.728s][info][gc,metaspace,freelist] class space: Chunk accounting: >> used in chunks 4047K + unused in chunks 97K + capacity in free chunks >> 6085K = 10230K capacity in allocated chunks 4145K >> [272.729s][info][gc,metaspace,freelist] Total fragmentation waste (words) >> doesn't count free space >> [272.729s][info][gc,metaspace,freelist] data: 183 specialized(s) 960, >> 169 >> small(s) 41, 1507 medium(s) 4480, large count 1 >> [272.729s][info][gc,metaspace,freelist] class: 183 specialized(s) 0, 13 >> small(s) 12, 111 medium(s) 8, large count 1 >> >> Now: >> [250.548s][info][gc,metaspace,freelist] All Metaspace: >> [250.549s][info][gc,metaspace,freelist] data space: Chunk accounting: >> used in chunks 100748K + unused in chunks 638K + capacity in free chunks >> 10357K = 111744K capacity in allocated chunks 101387K >> [250.550s][info][gc,metaspace,freelist] class space: Chunk accounting: >> used in chunks 4047K + unused in chunks 97K + capacity in free chunks >> 6085K = 10230K capacity in allocated chunks 4145K >> [250.550s][info][gc,metaspace,freelist] Total fragmentation waste (words) >> doesn't count free space >> [250.550s][info][gc,metaspace,freelist] data: 183 specialized(s) 960, >> 165 >> small(s) 27, 1507 medium(s) 4504, large count 1 >> [250.550s][info][gc,metaspace,freelist] class: 183 specialized(s) 0, 13 >> small(s) 12, 111 medium(s) 14, large count 1 >> [250.550s][info][gc,metaspace,freelist] Chunkmanager (non-class): >> [250.550s][info][gc,metaspace,freelist] 89 specialized (128 bytes) >> chunks, total 91136 bytes >> [250.550s][info][gc,metaspace,freelist] 2567 small (512 bytes) chunks, >> total 10514432 bytes >> [250.550s][info][gc,metaspace,freelist] 0 medium (8192 bytes) chunks, >> total 0 bytes >> [250.550s][info][gc,metaspace,freelist] 0 humongous chunks, total 0 >> bytes >> [250.550s][info][gc,metaspace,freelist] total size: 10605568. >> [250.550s][info][gc,metaspace,freelist] Chunkmanager (class): >> [250.550s][info][gc,metaspace,freelist] 87 specialized (128 bytes) >> chunks, total 89088 bytes >> [250.550s][info][gc,metaspace,freelist] 2999 small (256 bytes) chunks, >> total 6141952 bytes >> [250.550s][info][gc,metaspace,freelist] 0 medium (4096 bytes) chunks, >> total 0 bytes >> [250.550s][info][gc,metaspace,freelist] 0 humongous chunks, total 0 >> bytes >> [250.550s][info][gc,metaspace,freelist] total size: 6231040. >> > > Yes, this looks like it'll help. Your numbers are less mystifying than > the existing ones. > >> >> This helps in understanding the underlying cause of the OOM, if that cause >> is related to chunks lingering around in the freelist. See also [1], a Jep >> proposing to improve the metaspace chunk allocation to increase the chance >> of chunk reuse. This patch here is also a preparation for the prototype >> work of [1]. >> >> Note that admittedly, metaspace.cpp will again gain a bit of complexity by >> this patch. When examining the coding, I was tempted to refactor but >> refrained from doing so to keep the patch focused. But I think >> metaspace.cpp (especially the coding associated with statistics) could be >> cleaned up a bit. I opened a follow up item [2] to collect cleanup-issues. >> I am willing to do that myself, but would prefer it to be done after >> functional patches - which still have a slight chance of being backported >> to jdk9 - are all being added. >> > > My first impression was that get_statistics() and print_statistics() > should be member functions of ChunkManagerStatistics because of all the > stat->lines. But ChunkManager has to own these also. I can't think of > anything better. > > I wanted to keep the structure a dumb data holder (I actually would have preferred a more C-ish name like cmstat_t or similar but I kept the name in line with existing name conventions). But I could make "ChunkManager::print_statistics" a member function of ChunkManagerStatistics, if you prefer it that way. I think it does not matter much, just a question of aesthetics. > Yes, the metaspace code could definitely use refactoring with respect to > statistics collection. Please! Also I added to your RFE to rename > MetaspaceAux. > > Great! Lets use that RFE to collect more thoughts. > I agree with Mikael about adding the code to avoid casting. You could > change the type to 'int' with another patch (or this one, I don't mind > looking at it again). > > I will do as Mikael requested, his additions make sense. Changing the numbers to "int" can be part of that cleanup to come. > I could sponsor this when it's ready. > > Thanks! I'll post an updated webrev soon. > Thanks! > Coleen > > Kind Regards Thomas > > > >> Patch was built and tested on Windows x64, Linux x64 and AIX. I also ran >> jtreg hotspot/runtime/Metaspace tests, no issues. >> >> Kind Regards, Thomas >> >> ---- >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8166690 >> [2] https://bugs.openjdk.java.net/browse/JDK-8185034 >> > > From coleen.phillimore at oracle.com Fri Jul 21 18:28:05 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 21 Jul 2017 14:28:05 -0400 Subject: RFR (L) 7133093: Improve system dictionary performance In-Reply-To: <7adf48f1-5908-aeaf-c409-d930895e1715@oracle.com> References: <89f0b98c-3cbd-7b87-d76b-5e89a5a676fb@oracle.com> <9243da7c-a8e5-4024-c12e-cba9d4f424c8@oracle.com> <7814C032-0D4E-4E76-B8AD-62A072E8A625@oracle.com> <233c03e0-8958-7676-8e33-5d8f0ec4edf3@oracle.com> <7adf48f1-5908-aeaf-c409-d930895e1715@oracle.com> Message-ID: From internal discussions, the only sizing of the dictionary for the java.system.class.loader will follow PredictedLoadedClassCount until dictionary resizing is done. The latest full webrev is here: open webrev at http://cr.openjdk.java.net/~coleenp/7133093.04/webrev The only change from .03 is http://cr.openjdk.java.net/~coleenp/7133093.04/webrev/src/share/vm/classfile/classLoaderData.cpp.frames.html lines 595-630. Please review! Thanks, Coleen On 7/19/17 10:52 PM, coleen.phillimore at oracle.com wrote: > > Hi, I've made the changes in the replies below and retested with both > our internal NSK tests, jtreg tests and jck tests. Also I've rerun > the performance tests, including SPECjbb2015. > > An incremental webrev can be found at: > http://cr.openjdk.java.net/~coleenp/7133093.03.incremental/webrev/ > > Latest version at: > > http://cr.openjdk.java.net/~coleenp/7133093.03/webrev/ > > Thanks, > Coleen > > > On 7/18/17 10:56 AM, coleen.phillimore at oracle.com wrote: >> >> Karen, Thank you for reviewing the code and for the discussions and >> consulting along the way. >> >> On 7/17/17 4:10 PM, Karen Kinnear wrote: >>> Feedback on the initial web rev: >>> Thank you for using web rev - that way I can do searches of the >>> overall patch! >>> >>> First - many thanks for doing this exercise so carefully, for >>> following the specification in continuing >>> to record the initiating loader and for splitting this effort into >>> multiple phases to allow further exploration >>> of the current expectations/behavior relative to class unloading. >>> >>> And thank you for all the testing. >>> >>> Just wanted to ensure that I understand some of the assumptions: >>> >>> 1) lock free reads of dictionaries - yes >> Yes. >>> 2) lock free adds of entries to dictionaries - yes >> No, adds still take out the SystemDictionary_lock. >>> 3) removal of entries or entire dictionaries - at safe point only >> Yes. >>> 4) rehashing - none today and none planned >> Yes. >>> 5) resizing of dictionaries - none today, some planned >> Yes. >>> 6) use of SystemDictionary_lock: essentially unchanged >> Yes. >>> 7) atomicity guarantees relative to checking loader constraints and >>> updates to the dictionary: unchanged >> Yes. >>> >>> I did not review the SA/vmStructs - sounds like you had Jini George >>> do that - great. >>> >>> Minor questions/comments: >>> >>> SystemDictionary::verify: "Verify of system dictionary failed" -> >>> looks like "Verify of placeholders failed" >> >> Fixed. Good find. It's an odd assert: >> >> guarantee(placeholders()->number_of_entries() >= 0, >> "Verify of placeholders failed"); >> >>> >>> thank you for asserts dictionary != NULL >>> Dictionary::print >>> is it worth an assert that loader_data != NULL? >> >> Added. It's already asserted in Dictionary::verify() >> >>> Can you please sanity check if there are other cases in which we need >>> to use dictionary_or_null() rather than dictionary()? >>> - are there cases in which we could be using a dictionary from an >>> anonymous class? >>> e.g. there are a lot of places in systemDictionary.cpp where we >>> call class_loader_data(class_loader); loader_data->dictionary(); >>> - was the lazily allocated dictionary allocated yet (see next >>> question) >> >> There's an assert in dictionary() that the CLD isn't anonymous. In >> these calls we get class_loader_data from the class_loader which is >> always the host class loader for anonymous classes, so this doesn't >> break with the current design. >> >>> >>> CLD.cpp >>> Would it be less complex to eagerly create the dictionary for a CLD >>> since it is extremely rare to not need one for non-anonymous CLDs? >>> You could create before publishing the CLD >> >> Yes, I agree it would be better this way. I was looking at some data >> and it does seem rare to have a CLD with no classes. >>> >>> Terminology: >>> How are you still using System Dictionary term? >>> It still shows up in some comments and in some printed strings >> >> Bad habit, we've been calling it that for years. :) I'll update the >> comments where it makes sense. >> I tried to refer to the ClassLoaderDataGraph dictionaries >> collectively as the "System Dictionary", as well as the loader >> constraints and placeholder tables, and the other things that >> SystemDictionary contains. >> >> I guess it gets confusing, but it's less awkward than "class loader >> data dictionaries", and more meaningful than simply "dictionaries". >> So that's why I didn't change a lot of the comments. >> >>> >>> biasedLocking.cpp, line 59 >>> jvmtiGetLoadedClasses: line 298 >>> dictionary.hpp lines 113,114, 120 >>> dictionary.cpp 262-263, 478 >>> dictionary.cpp 491 >>> fix comments e.g. "Iterate the system dictionary" ... >>> - probably worth checking for other comments in the sources too >>> unless these all appear to match new usage model >> >> Fixed. I've found a couple more and some other comments that are >> incorrect. >>> >>> hashTable.hpp >>> line 268: what does this comment mean? >>> "Don't create and use freelist"? >>> I assume this means that this is a direct malloc, not using any block >>> allocation, e.g. metaspace chunks, and this requires explicit >>> freeing? >>> >> >> // Don't create and use freelist of HashtableEntry. >> >> The default Hashtable<> allocates entries from a block of entries and >> uses a freelist for freed entries. >> >>> >>> dictionary.hpp line 161 >>> could you possibly clean up comment while you are in here? >>> I presume "a protection" is "a protection domain"? >> >> Fixed. >> >>> >>> dictionary.cpp >>> why change from Hashtable::free_entry to explicit >>> unlink_entry/FREE_C_HEAP_ARRAY? >> >> I used the same code as the ModuleTable. The default hashtable >> doesn't delete the whole thing and left some memory around, if I >> remember correctly. I'll recheck it. >>> >>> Future optimization: >>> I think that none of the built-in loaders is ever recorded as an >>> initiating >>> loader for any class in a loader which permits unloading. >>> Note: we need to ensure that built-in includes boot, platform and >>> application class loader (and clean up any confusion with system >>> class loader - which can be set by the user to a custom class loader) >>> >> I think AppClassLoader can initiate loading and is a builtin loader. >> >> Dictionary for class loader class loader 0x00007f6abc6e3210 a >> 'jdk/internal/loader/ClassLoaders$AppClassLoader'{0x0000000101c8df00} >> Java system dictionary (table_size=107, classes=1) >> ^ indicates that initiating loader is different from defining loader >> 106: ^java.util.ArrayList, loader NULL class_loader >> >>> >>> Any chance of changing DictionaryEntry* probe->klass() to be >>> probe->instanceKlass()? >> >> Okay. Changed to instance_klass() as per coding standard. >>> >>> Thank you for assertion _is_at_safepoint for try_get_next_class >>> >>> CLD.cpp >>> If the names of the classes_do vs. all_entries_do were modified to be >>> defining_loader_classes_do vs. all_entries_classes_do or some way >>> to make the distinction in the name that would be helpful. >>> Otherwise, if you could please add comments to the >>> CLDG::dictionary_classes_do >>> etc. to distinguish these. >> >> I added comments to distinguish these. >>> >>> CLD.cpp >>> lines 1200-1201 >>> Can you please fix the comment? >>> I think you need separate comments for the ModuleEntry >>> reads/PackageEntry's >>> exports which are looking for dead modules. >>> IIUC, the do_unloading is actually looking for classes for which this >>> is the initating loader, but their defining loader is dead. >> // Remove entries in the dictionary of live class loader that have >> // initiated loading classes in a dead class loader. >> data->dictionary()->do_unloading(); >> >> Moved the module comment to before the module code and added this for >> dictionary->do_unloading() call. >>> >>> systemDictionary.cpp: 445-447 >>> THANK you for the comments about the java_mirror handle keeping >>> the class >>> and class loader alive >>> Could you clarify that the class_loader handle is not for the class >>> we are checking, but rather for the initiating, i.e. requesting >>> class loader >>> (is that accurate?) So we are keeping both the requestor and the >>> requested class alive >> >> Yes, that is true. I added >> >> // The class_loader handle passed in is the initiating loader. >> >>> >>> You removed the comment that compute_hash could lead to a safepoint/GC >>> Is that not possible? >>> comment in ObjectSynchronizer::FastHashCode says that with >>> UseBiasedLocking >>> we can revoke biases and must be able to handle a safepoint. >> >> I made it not possible with this change. The hashcode is now >> computed by the Symbol* of the Klass name, which is random >> (os::random plus some bits from the address and first two characters, >> which we did a study of for bug >> https://bugs.openjdk.java.net/browse/JDK-8181450). >> >> Also, I changed ClassLoaderData::identity_hash() to use the address >> >> 3 because the addresses of class loader datas are random enough >> for the one use of this function. Previously, the loader_data >> identity hash used to go to the class_loader oop and get to >> ObjectSynchronizer::FastHashCode (which was also os::random but now >> changed). >> >> The dictionary hashing doesn't need to add in the hash for the >> loader_data now. >>> >>> systemDictionary.cpp: >>> NoSafepointVerifier >>> If we add resizing logic later, and that happens at a safepoint, >>> then the number of buckets change and the d_index is no longer >>> valid >>> so we would need to add back the NSpV? >>> - this is a more general problem if compute_hash can lead to a >>> safepoint/GC >> >> Yes this is true, except that compute_hash doesn't safepoint. >> Computing the hash and index should be done together with the >> dictionary operations. The function resolve_instance_class_or_null >> computes the hash and index once at the beginning and reuses it for >> several lookups, which may or may not affect performance. So we need >> to measure this in isolation before changing this. >>> >>> systemDictionary.cpp: parse_stream >>> why did you remove the record_dependency? >> >> I realized it's trivially unnecessary. The class_loader is reached >> by the ClassLoaderData::_class_loader field, so we don't have to >> record a dependency from host_klass->class_loader in the anonymous >> class loader data. (I just checked this again because it's confusing). >> >>> >>> Who calls Dictionary::~Dictionary? >> >> The ClassLoaderData::~ClassLoaderData() calls delete _dictionary, >> which calls the destructor. >>> >>> Performance: >>> Do you have performance runs of our existing benchmarks? >> >> Yes, I ran our internal benchmarks and added links to the internal >> wiki page. I'll rerun them again before checkin. >> >>> Initial key requirement is that this not slow down throughput through >>> slowing down lookups. >>> I know you are very aware of longer-term goals for reducing >>> safepoint pause >>> times and not increasing footprint significantly. >>> >>> hashtable sizes: future rfe? >>> given that we now have a lot more dictionaries, and they perhaps need >>> individual sizing >>> - would it be helpful to add statistics information about >>> - e.g. lookup_count, lookup_length, _lookup_warning >>> - i.e. number of entries, max lookup depth, etc. to use later for >>> tuning? >> >> All the previous statistics didn't make very much sense to us, so >> they were removed. There is now logging for all the hashtables: >> >> busaa027.us.oracle.com% java -Xlog:hashtables -version >> java version "10-internal" >> Java(TM) SE Runtime Environment (fastdebug build >> 10-internal+0-2017-07-17-1456035.cphillim.10system-dictionary2) >> Java HotSpot(TM) 64-Bit Server VM (fastdebug build >> 10-internal+0-2017-07-17-1456035.cphillim.10system-dictionary2, mixed >> mode) >> [0.284s][info][hashtables] System Dictionary for >> jdk/internal/loader/ClassLoaders$AppClassLoader max bucket size 1 >> bucket 91 element count 1 table size 107 >> [0.285s][info][hashtables] System Dictionary for max >> bucket size 4 bucket 263 element count 468 table size 1009 >> [0.285s][info][hashtables] Placeholder Table max bucket size 0 bucket >> 0 element count 0 table size 1009 >> [0.285s][info][hashtables] Protection Domain Table max bucket size 0 >> bucket 0 element count 0 table size 1009 >> >>> - might make sense to make this JFR data? >> >> I'll file an RFE to add tracing data with our tools. I'll need some >> help with this part but the data is there. >>> >>> Might think about a follow-up with more tuning parameters than >>> a single global PredictedLoadedClassCount. >>> >> >> I filed RFE to work on resizing, as this work was almost complete, >> but didn't show performance improvements at the time. It would be >> nice to have customers not need tuning parameters, but I guess these >> things are sometimes necessary. >> >> Thank you for the thorough review and all of your help with this! >> I've made these changes and will rerun testing and send out a new >> webrev and an incremental webrev. >> >> Thanks! >> Coleen >> >>> >>> thanks, >>> Karen >>> >>> >>>> On Jul 17, 2017, at 2:37 PM, coleen.phillimore at oracle.com >>>> wrote: >>>> >>>> >>>> Hi, I have made changes for Ioi's comments below. The largest >>>> change since .01 is that PredictedLoadedClassCount is used to size >>>> dictionaries for class loader data, and should be used to specify >>>> the average number of classes loaded by each class loader. Since >>>> this is an option available via UnlockExperimentalVMOptions, I'm >>>> not sure what the process is for changing the meaning of this >>>> option. We will work with our customers who have used this option >>>> for specifying the total number of classes loaded. We are also >>>> working on the change to allow automatic resizing of the >>>> dictionaries during safepoints. >>>> >>>> See: >>>> open webrev at >>>> http://cr.openjdk.java.net/~coleenp/7133093.02/webrev >>>> >>>> >>>> Tested with the full nightly test set (jdk, nsk, hotspot jtreg >>>> tests) on linux x64. >>>> >>>> Thanks, >>>> Coleen >>>> >>>> On 6/30/17 4:45 PM, coleen.phillimore at oracle.com >>>> wrote: >>>>> >>>>> Ioi, Thank you for looking at this change. >>>>> >>>>> On 6/30/17 2:39 PM, Ioi Lam wrote: >>>>>> Hi Coleen, >>>>>> >>>>>> Maybe the bug should be renamed to "Use one Dictionary per class >>>>>> loader instance"? That way it's more obvious what it is when you >>>>>> look at the repo history. >>>>> >>>>> I can do that. I made it One Dictionary per ClassLoaderData >>>>>> >>>>>> 1. >>>>>> >>>>>> Why is assert_locked_or_safepoint(SystemDictionary_lock) >>>>>> necessary in SystemDictionary::find_class (line 1826), but not >>>>>> necessary SystemDictionary::find (line 951)? Since you removed >>>>>> NoSafepointVerifier nosafepoint in the latter, maybe this means >>>>>> it's safe to remove the assert_locked_or_safepoint in the former? >>>>> >>>>> The call to SystemDictionary::find() is the (I believe) usual lock >>>>> free lookup and the SystemDictionary::find_class() is used to >>>>> verify that a class we're about to add or want to add hasn't been >>>>> added by another thread. Or certain cases where we already have >>>>> a lock to do something else, like add loader constraints. I took >>>>> out the NoSafepointVerifier because it assumes that system >>>>> dictionary entries would be moved by GC, which they won't. The >>>>> old hash function could safepoint when getting the hash for the >>>>> class_loader object. >>>>> >>>>>> >>>>>> 2. >>>>>> >>>>>> 455 static ClassLoaderData* _current_loader_data = NULL; >>>>>> 456 static Klass* _current_class_entry = NULL; >>>>>> 457 >>>>>> 458 InstanceKlass* ClassLoaderDataGraph::try_get_next_class() { >>>>>> >>>>>> How about moving the static fields into an iterator object. That >>>>>> way you don't need to keep track of the globals >>>>>> >>>>>> ClassLoaderDataGraphIterator { >>>>>> ClassLoaderData* _current_loader_data >>>>>> Klass* _current_class_entry; >>>>>> >>>>>> InstanceKlass* try_get_next_class() { ....} >>>>>> }; >>>>> >>>>> Ok, there's a different iterator that iterates over all of the >>>>> classes for GC. I will adapt that for this use. That would be >>>>> better. >>>>> >>>>>> >>>>>> 3. Double check locking in ClassLoaderData::dictionary() -- >>>>>> someone else should look at this :-) >>>>> >>>>> I copied code that David Holmes added for modules. >>>>> >>>>>> >>>>>> 4. We may need a better strategy for deciding the size of each >>>>>> dictionary. >>>>>> >>>>>> 565 const int _primelist[10] = {1, 107, 1009}; >>>>>> 571 Dictionary* ClassLoaderData::dictionary() { >>>>>> 579 if ((dictionary = _dictionary) == NULL) { >>>>>> 580 int size; >>>>>> 581 if (this == the_null_class_loader_data() || >>>>>> is_system_class_loader_data()) { >>>>>> 582 size = _primelist[2]; >>>>>> 583 } else if >>>>>> (class_loader()->is_a(SystemDictionary::reflect_DelegatingClassLoader_klass())) >>>>>> { >>>>>> 584 size = _primelist[0]; >>>>>> 585 } else { >>>>>> 586 size = _primelist[1]; >>>>>> 587 } >>>>>> 588 dictionary = new Dictionary(this, size); >>>>>> >>>>>> I'll do some investigation on this issue and get back to you. >>>>>> >>>>> >>>>> How about if someone uses PredictedLoadedClassCount, then we use >>>>> that to size all but the reflection and boot class loader? Then >>>>> if there's an application that has a class loader with a huge >>>>> amount of classes loaded in it, that would help this? It might >>>>> cost some footprint but an oversized table would simply be a >>>>> bigger array of pointers, so it might not be that bad to oversize. >>>>> >>>>> I think the long term solution is being able to resize these >>>>> entries and make the application provide arguments. Please let >>>>> me know what you find in your investigation and if that would work. >>>>>> The rest of the changes look good to me. >>>>> >>>>> Thank you! >>>>> Coleen >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>> >>>>>> On 6/23/17 4:42 PM, coleen.phillimore at oracle.com >>>>>> wrote: >>>>>>> Summary: Implement one dictionary per ClassLoaderData for faster >>>>>>> lookup and removal during class unloading >>>>>>> >>>>>>> See RFE for more details. >>>>>>> >>>>>>> open webrev at >>>>>>> http://cr.openjdk.java.net/~coleenp/7133093.01/webrev >>>>>>> >>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-7133093 >>>>>>> >>>>>>> Tested with full "nightly" run in rbt, plus locally class >>>>>>> loading and unloading tests: >>>>>>> >>>>>>> jtreg hotspot/test/runtime/ClassUnload >>>>>>> >>>>>>> jtreg hotspot/test/runtime/modules >>>>>>> >>>>>>> jtreg hotspot/test/gc/class_unloading >>>>>>> >>>>>>> make test-hotspot-closed-tonga FILTER=quick TEST_JOBS=4 >>>>>>> TEST=vm.parallel_class_loading >>>>>>> >>>>>>> csh ~/testing/run_jck9 (vm/lang/java_lang) >>>>>>> >>>>>>> runThese -jck - uses class loader isolation to run each jck test >>>>>>> and unloads tests when done (at -gc:5 intervals) >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Coleen >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From coleen.phillimore at oracle.com Fri Jul 21 18:54:15 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 21 Jul 2017 14:54:15 -0400 Subject: RFR(xs): 8185033: On Metaspace OOM, ChunkManager composition should be logged. In-Reply-To: References: Message-ID: <74885bac-a0f9-52ea-cc90-b8b6ef39edee@oracle.com> On 7/21/17 2:16 PM, Thomas St?fe wrote: > Hi Coleen, > > On Fri, Jul 21, 2017 at 5:34 PM, > wrote: > > > Hi Thomas, > Thank you for working on this. > > > thanks for reviewing! > > > On 7/21/17 2:54 AM, Thomas St?fe wrote: > > Dear all, > > May I please get reviews and a sponsor for the following small > enhancement. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8185033 > > Webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/8185033-On-Metaspace-OOM-chunkManager-composition-should-be-logged/webrev.00/webrev/ > > > This patch adds a bit of logging if a Metaspace OOM happens. > We will now > get a printout of the composition of the global chunk managers > (non-class > and, if necessary, class). > > Before it looked like this: > > [272.727s][info][gc,metaspace,freelist] All Metaspace: > [272.728s][info][gc,metaspace,freelist] data space: Chunk > accounting: > used in chunks 100752K + unused in chunks 650K + capacity in > free chunks > 8293K = 109696K capacity in allocated chunks 101403K > [272.728s][info][gc,metaspace,freelist] class space: Chunk > accounting: > used in chunks 4047K + unused in chunks 97K + capacity in > free chunks > 6085K = 10230K capacity in allocated chunks 4145K > [272.729s][info][gc,metaspace,freelist] Total fragmentation > waste (words) > doesn't count free space > [272.729s][info][gc,metaspace,freelist] data: 183 > specialized(s) 960, 169 > small(s) 41, 1507 medium(s) 4480, large count 1 > [272.729s][info][gc,metaspace,freelist] class: 183 > specialized(s) 0, 13 > small(s) 12, 111 medium(s) 8, large count 1 > > Now: > [250.548s][info][gc,metaspace,freelist] All Metaspace: > [250.549s][info][gc,metaspace,freelist] data space: Chunk > accounting: > used in chunks 100748K + unused in chunks 638K + capacity in > free chunks > 10357K = 111744K capacity in allocated chunks 101387K > [250.550s][info][gc,metaspace,freelist] class space: Chunk > accounting: > used in chunks 4047K + unused in chunks 97K + capacity in > free chunks > 6085K = 10230K capacity in allocated chunks 4145K > [250.550s][info][gc,metaspace,freelist] Total fragmentation > waste (words) > doesn't count free space > [250.550s][info][gc,metaspace,freelist] data: 183 > specialized(s) 960, 165 > small(s) 27, 1507 medium(s) 4504, large count 1 > [250.550s][info][gc,metaspace,freelist] class: 183 > specialized(s) 0, 13 > small(s) 12, 111 medium(s) 14, large count 1 > [250.550s][info][gc,metaspace,freelist] Chunkmanager (non-class): > [250.550s][info][gc,metaspace,freelist] 89 specialized (128 > bytes) > chunks, total 91136 bytes > [250.550s][info][gc,metaspace,freelist] 2567 small (512 > bytes) chunks, > total 10514432 bytes > [250.550s][info][gc,metaspace,freelist] 0 medium (8192 > bytes) chunks, > total 0 bytes > [250.550s][info][gc,metaspace,freelist] 0 humongous chunks, > total 0 bytes > [250.550s][info][gc,metaspace,freelist] total size: 10605568. > [250.550s][info][gc,metaspace,freelist] Chunkmanager (class): > [250.550s][info][gc,metaspace,freelist] 87 specialized (128 > bytes) > chunks, total 89088 bytes > [250.550s][info][gc,metaspace,freelist] 2999 small (256 > bytes) chunks, > total 6141952 bytes > [250.550s][info][gc,metaspace,freelist] 0 medium (4096 > bytes) chunks, > total 0 bytes > [250.550s][info][gc,metaspace,freelist] 0 humongous chunks, > total 0 bytes > [250.550s][info][gc,metaspace,freelist] total size: 6231040. > > > Yes, this looks like it'll help. Your numbers are less > mystifying than the existing ones. > > > This helps in understanding the underlying cause of the OOM, > if that cause > is related to chunks lingering around in the freelist. See > also [1], a Jep > proposing to improve the metaspace chunk allocation to > increase the chance > of chunk reuse. This patch here is also a preparation for the > prototype > work of [1]. > > Note that admittedly, metaspace.cpp will again gain a bit of > complexity by > this patch. When examining the coding, I was tempted to > refactor but > refrained from doing so to keep the patch focused. But I think > metaspace.cpp (especially the coding associated with > statistics) could be > cleaned up a bit. I opened a follow up item [2] to collect > cleanup-issues. > I am willing to do that myself, but would prefer it to be done > after > functional patches - which still have a slight chance of being > backported > to jdk9 - are all being added. > > > My first impression was that get_statistics() and > print_statistics() should be member functions of > ChunkManagerStatistics because of all the stat->lines. But > ChunkManager has to own these also. I can't think of anything better. > > > I wanted to keep the structure a dumb data holder (I actually would > have preferred a more C-ish name like cmstat_t or similar but I kept > the name in line with existing name conventions). But I could make > "ChunkManager::print_statistics" a member function of > ChunkManagerStatistics, if you prefer it that way. I think it does not > matter much, just a question of aesthetics. Yes, it seemed that you were trying to encapsulate the data and I don't see a better way. I'm glad you kept the name of the class with the existing naming conventions because you may find an improvement to make it aesthetic at some point. It's not worth changing now. > Yes, the metaspace code could definitely use refactoring with > respect to statistics collection. Please! Also I added to your > RFE to rename MetaspaceAux. > > > Great! Lets use that RFE to collect more thoughts. > > I agree with Mikael about adding the code to avoid casting. You > could change the type to 'int' with another patch (or this one, I > don't mind looking at it again). > > > I will do as Mikael requested, his additions make sense. Changing the > numbers to "int" can be part of that cleanup to come. > > I could sponsor this when it's ready. > > > Thanks! I'll post an updated webrev soon. I'll watch for it. Thanks, Coleen > Thanks! > Coleen > > > Kind Regards Thomas > > > > > Patch was built and tested on Windows x64, Linux x64 and AIX. > I also ran > jtreg hotspot/runtime/Metaspace tests, no issues. > > Kind Regards, Thomas > > ---- > > [1] https://bugs.openjdk.java.net/browse/JDK-8166690 > > [2] https://bugs.openjdk.java.net/browse/JDK-8185034 > > > > From daniel.daugherty at oracle.com Fri Jul 21 19:57:05 2017 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 21 Jul 2017 13:57:05 -0600 Subject: Request for review for JDK-8184045 - TestSystemGCWithG1.java times out on Solaris SPARC In-Reply-To: <4afe0501-a927-7ecf-b229-63aeeecf76be@oracle.com> References: <98794b3a-ac78-26b5-8647-9f8f99aef048@oracle.com> <455156a7-d4b2-364c-e5b6-e0ed804ae93b@oracle.com> <4afe0501-a927-7ecf-b229-63aeeecf76be@oracle.com> Message-ID: <2eee43f3-e92b-015b-f087-dfdce0b2229d@oracle.com> On 7/21/17 11:17 AM, Alexander Harlap wrote: > Hi Erik, > > Thank you for your suggestion. > > New version is at http://cr.openjdk.java.net/~aharlap/8184045/webrev.01/ test/gc/stress/systemgc/TestSystemGC.java No comments. Thumbs up. Dan > > Alex > > > On 7/21/2017 8:44 AM, Erik Helin wrote: >> On 07/21/2017 01:50 PM, Erik Helin wrote: >>> On 07/20/2017 05:31 PM, Alexander Harlap wrote: >>>> Please review changes forJDK-8184045 >>>> - >>>> TestSystemGCWithG1.java times out on Solaris SPARC (labeled as >>>> integration blocker) >>>> >>>> Proposed change will avoid timeout failure for new stress tests >>>> TestSystemGCWithG1.java, TestSystemGCWithCMS.java and >>>> TestSystemGCWithSerial.java regularly observed on Solaris Sparc >>>> >>>> Change is located at >>>> http://cr.openjdk.java.net/~aharlap/8184045/webrev.00/ >>> >>> Instead of removing the second pass, can you check if the args[0] == >>> "long"? The code can look like: >>> >>> public static void main(String[] args) { >>> populateLongLived(); >>> runAllPhases(); >>> if (args.length > 0 && args[0] == "long") { >> >> ...aaaand as couple of my colleagues pointed out while smiling, you >> should of course write args[0].equals("long") :) >> >> Thanks, >> Erik >> >>> runAllPhases(); >>> } >>> } >>> >>> This way the stress test can be used in "long" and "short" mode. >>> >>> Thanks, >>> Erik >>> >>>> Alex >>>> > From ioi.lam at oracle.com Fri Jul 21 20:37:34 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 21 Jul 2017 13:37:34 -0700 Subject: RFR (L) 7133093: Improve system dictionary performance In-Reply-To: References: <89f0b98c-3cbd-7b87-d76b-5e89a5a676fb@oracle.com> <9243da7c-a8e5-4024-c12e-cba9d4f424c8@oracle.com> <7814C032-0D4E-4E76-B8AD-62A072E8A625@oracle.com> <233c03e0-8958-7676-8e33-5d8f0ec4edf3@oracle.com> <7adf48f1-5908-aeaf-c409-d930895e1715@oracle.com> Message-ID: <26A04A99-5E1A-4F19-8BD1-B2D8764D1C50@oracle.com> Hi Coleen, The new changes look good to me. Thanks! Ioi > On Jul 21, 2017, at 11:28 AM, coleen.phillimore at oracle.com wrote: > > > From internal discussions, the only sizing of the dictionary for the java.system.class.loader will follow PredictedLoadedClassCount until dictionary resizing is done. The latest full webrev is here: > > open webrev at http://cr.openjdk.java.net/~coleenp/7133093.04/webrev > > The only change from .03 is > > http://cr.openjdk.java.net/~coleenp/7133093.04/webrev/src/share/vm/classfile/classLoaderData.cpp.frames.html > > lines 595-630. Please review! > > Thanks, > Coleen > > >> On 7/19/17 10:52 PM, coleen.phillimore at oracle.com wrote: >> >> Hi, I've made the changes in the replies below and retested with both our internal NSK tests, jtreg tests and jck tests. Also I've rerun the performance tests, including SPECjbb2015. >> >> An incremental webrev can be found at: >> http://cr.openjdk.java.net/~coleenp/7133093.03.incremental/webrev/ >> >> Latest version at: >> >> http://cr.openjdk.java.net/~coleenp/7133093.03/webrev/ >> >> Thanks, >> Coleen >> >> >>> On 7/18/17 10:56 AM, coleen.phillimore at oracle.com wrote: >>> >>> Karen, Thank you for reviewing the code and for the discussions and consulting along the way. >>> >>>> On 7/17/17 4:10 PM, Karen Kinnear wrote: >>>> Feedback on the initial web rev: >>>> Thank you for using web rev - that way I can do searches of the overall patch! >>>> >>>> First - many thanks for doing this exercise so carefully, for following the specification in continuing >>>> to record the initiating loader and for splitting this effort into multiple phases to allow further exploration >>>> of the current expectations/behavior relative to class unloading. >>>> >>>> And thank you for all the testing. >>>> >>>> Just wanted to ensure that I understand some of the assumptions: >>>> >>>> 1) lock free reads of dictionaries - yes >>> Yes. >>>> 2) lock free adds of entries to dictionaries - yes >>> No, adds still take out the SystemDictionary_lock. >>>> 3) removal of entries or entire dictionaries - at safe point only >>> Yes. >>>> 4) rehashing - none today and none planned >>> Yes. >>>> 5) resizing of dictionaries - none today, some planned >>> Yes. >>>> 6) use of SystemDictionary_lock: essentially unchanged >>> Yes. >>>> 7) atomicity guarantees relative to checking loader constraints and updates to the dictionary: unchanged >>> Yes. >>>> >>>> I did not review the SA/vmStructs - sounds like you had Jini George do that - great. >>>> >>>> Minor questions/comments: >>>> >>>> SystemDictionary::verify: "Verify of system dictionary failed" -> >>>> looks like "Verify of placeholders failed" >>> >>> Fixed. Good find. It's an odd assert: >>> >>> guarantee(placeholders()->number_of_entries() >= 0, >>> "Verify of placeholders failed"); >>> >>>> >>>> thank you for asserts dictionary != NULL >>>> Dictionary::print >>>> is it worth an assert that loader_data != NULL? >>> >>> Added. It's already asserted in Dictionary::verify() >>> >>>> Can you please sanity check if there are other cases in which we need >>>> to use dictionary_or_null() rather than dictionary()? >>>> - are there cases in which we could be using a dictionary from an anonymous class? >>>> e.g. there are a lot of places in systemDictionary.cpp where we >>>> call class_loader_data(class_loader); loader_data->dictionary(); >>>> - was the lazily allocated dictionary allocated yet (see next question) >>> >>> There's an assert in dictionary() that the CLD isn't anonymous. In these calls we get class_loader_data from the class_loader which is always the host class loader for anonymous classes, so this doesn't break with the current design. >>> >>>> >>>> CLD.cpp >>>> Would it be less complex to eagerly create the dictionary for a CLD >>>> since it is extremely rare to not need one for non-anonymous CLDs? >>>> You could create before publishing the CLD >>> >>> Yes, I agree it would be better this way. I was looking at some data and it does seem rare to have a CLD with no classes. >>>> >>>> Terminology: >>>> How are you still using System Dictionary term? >>>> It still shows up in some comments and in some printed strings >>> >>> Bad habit, we've been calling it that for years. :) I'll update the comments where it makes sense. >>> I tried to refer to the ClassLoaderDataGraph dictionaries collectively as the "System Dictionary", as well as the loader constraints and placeholder tables, and the other things that SystemDictionary contains. >>> >>> I guess it gets confusing, but it's less awkward than "class loader data dictionaries", and more meaningful than simply "dictionaries". So that's why I didn't change a lot of the comments. >>> >>>> >>>> biasedLocking.cpp, line 59 >>>> jvmtiGetLoadedClasses: line 298 >>>> dictionary.hpp lines 113,114, 120 >>>> dictionary.cpp 262-263, 478 >>>> dictionary.cpp 491 >>>> fix comments e.g. "Iterate the system dictionary" ... >>>> - probably worth checking for other comments in the sources too >>>> unless these all appear to match new usage model >>> >>> Fixed. I've found a couple more and some other comments that are incorrect. >>>> >>>> hashTable.hpp >>>> line 268: what does this comment mean? >>>> "Don't create and use freelist"? >>>> I assume this means that this is a direct malloc, not using any block >>>> allocation, e.g. metaspace chunks, and this requires explicit freeing? >>>> >>> >>> // Don't create and use freelist of HashtableEntry. >>> >>> The default Hashtable<> allocates entries from a block of entries and uses a freelist for freed entries. >>> >>>> >>>> dictionary.hpp line 161 >>>> could you possibly clean up comment while you are in here? >>>> I presume "a protection" is "a protection domain"? >>> >>> Fixed. >>> >>>> >>>> dictionary.cpp >>>> why change from Hashtable::free_entry to explicit >>>> unlink_entry/FREE_C_HEAP_ARRAY? >>> >>> I used the same code as the ModuleTable. The default hashtable doesn't delete the whole thing and left some memory around, if I remember correctly. I'll recheck it. >>>> >>>> Future optimization: >>>> I think that none of the built-in loaders is ever recorded as an initiating >>>> loader for any class in a loader which permits unloading. >>>> Note: we need to ensure that built-in includes boot, platform and >>>> application class loader (and clean up any confusion with system >>>> class loader - which can be set by the user to a custom class loader) >>>> >>> I think AppClassLoader can initiate loading and is a builtin loader. >>> >>> Dictionary for class loader class loader 0x00007f6abc6e3210 a 'jdk/internal/loader/ClassLoaders$AppClassLoader'{0x0000000101c8df00} >>> Java system dictionary (table_size=107, classes=1) >>> ^ indicates that initiating loader is different from defining loader >>> 106: ^java.util.ArrayList, loader NULL class_loader >>> >>>> >>>> Any chance of changing DictionaryEntry* probe->klass() to be >>>> probe->instanceKlass()? >>> >>> Okay. Changed to instance_klass() as per coding standard. >>>> >>>> Thank you for assertion _is_at_safepoint for try_get_next_class >>>> >>>> CLD.cpp >>>> If the names of the classes_do vs. all_entries_do were modified to be >>>> defining_loader_classes_do vs. all_entries_classes_do or some way >>>> to make the distinction in the name that would be helpful. >>>> Otherwise, if you could please add comments to the CLDG::dictionary_classes_do >>>> etc. to distinguish these. >>> >>> I added comments to distinguish these. >>>> >>>> CLD.cpp >>>> lines 1200-1201 >>>> Can you please fix the comment? >>>> I think you need separate comments for the ModuleEntry reads/PackageEntry's >>>> exports which are looking for dead modules. >>>> IIUC, the do_unloading is actually looking for classes for which this >>>> is the initating loader, but their defining loader is dead. >>> // Remove entries in the dictionary of live class loader that have >>> // initiated loading classes in a dead class loader. >>> data->dictionary()->do_unloading(); >>> >>> Moved the module comment to before the module code and added this for dictionary->do_unloading() call. >>>> >>>> systemDictionary.cpp: 445-447 >>>> THANK you for the comments about the java_mirror handle keeping the class >>>> and class loader alive >>>> Could you clarify that the class_loader handle is not for the class >>>> we are checking, but rather for the initiating, i.e. requesting class loader >>>> (is that accurate?) So we are keeping both the requestor and the >>>> requested class alive >>> >>> Yes, that is true. I added >>> >>> // The class_loader handle passed in is the initiating loader. >>> >>>> >>>> You removed the comment that compute_hash could lead to a safepoint/GC >>>> Is that not possible? >>>> comment in ObjectSynchronizer::FastHashCode says that with UseBiasedLocking >>>> we can revoke biases and must be able to handle a safepoint. >>> >>> I made it not possible with this change. The hashcode is now computed by the Symbol* of the Klass name, which is random (os::random plus some bits from the address and first two characters, which we did a study of for bug https://bugs.openjdk.java.net/browse/JDK-8181450). >>> >>> Also, I changed ClassLoaderData::identity_hash() to use the address >> 3 because the addresses of class loader datas are random enough for the one use of this function. Previously, the loader_data identity hash used to go to the class_loader oop and get to ObjectSynchronizer::FastHashCode (which was also os::random but now changed). >>> >>> The dictionary hashing doesn't need to add in the hash for the loader_data now. >>>> >>>> systemDictionary.cpp: >>>> NoSafepointVerifier >>>> If we add resizing logic later, and that happens at a safepoint, >>>> then the number of buckets change and the d_index is no longer valid >>>> so we would need to add back the NSpV? >>>> - this is a more general problem if compute_hash can lead to a safepoint/GC >>> >>> Yes this is true, except that compute_hash doesn't safepoint. Computing the hash and index should be done together with the dictionary operations. The function resolve_instance_class_or_null computes the hash and index once at the beginning and reuses it for several lookups, which may or may not affect performance. So we need to measure this in isolation before changing this. >>>> >>>> systemDictionary.cpp: parse_stream >>>> why did you remove the record_dependency? >>> >>> I realized it's trivially unnecessary. The class_loader is reached by the ClassLoaderData::_class_loader field, so we don't have to record a dependency from host_klass->class_loader in the anonymous class loader data. (I just checked this again because it's confusing). >>> >>>> >>>> Who calls Dictionary::~Dictionary? >>> >>> The ClassLoaderData::~ClassLoaderData() calls delete _dictionary, which calls the destructor. >>>> >>>> Performance: >>>> Do you have performance runs of our existing benchmarks? >>> >>> Yes, I ran our internal benchmarks and added links to the internal wiki page. I'll rerun them again before checkin. >>> >>>> Initial key requirement is that this not slow down throughput through >>>> slowing down lookups. >>>> I know you are very aware of longer-term goals for reducing safepoint pause >>>> times and not increasing footprint significantly. >>>> >>>> hashtable sizes: future rfe? >>>> given that we now have a lot more dictionaries, and they perhaps need >>>> individual sizing >>>> - would it be helpful to add statistics information about >>>> - e.g. lookup_count, lookup_length, _lookup_warning >>>> - i.e. number of entries, max lookup depth, etc. to use later for tuning? >>> >>> All the previous statistics didn't make very much sense to us, so they were removed. There is now logging for all the hashtables: >>> >>> busaa027.us.oracle.com% java -Xlog:hashtables -version >>> java version "10-internal" >>> Java(TM) SE Runtime Environment (fastdebug build 10-internal+0-2017-07-17-1456035.cphillim.10system-dictionary2) >>> Java HotSpot(TM) 64-Bit Server VM (fastdebug build 10-internal+0-2017-07-17-1456035.cphillim.10system-dictionary2, mixed mode) >>> [0.284s][info][hashtables] System Dictionary for jdk/internal/loader/ClassLoaders$AppClassLoader max bucket size 1 bucket 91 element count 1 table size 107 >>> [0.285s][info][hashtables] System Dictionary for max bucket size 4 bucket 263 element count 468 table size 1009 >>> [0.285s][info][hashtables] Placeholder Table max bucket size 0 bucket 0 element count 0 table size 1009 >>> [0.285s][info][hashtables] Protection Domain Table max bucket size 0 bucket 0 element count 0 table size 1009 >>> >>>> - might make sense to make this JFR data? >>> >>> I'll file an RFE to add tracing data with our tools. I'll need some help with this part but the data is there. >>>> >>>> Might think about a follow-up with more tuning parameters than >>>> a single global PredictedLoadedClassCount. >>>> >>> >>> I filed RFE to work on resizing, as this work was almost complete, but didn't show performance improvements at the time. It would be nice to have customers not need tuning parameters, but I guess these things are sometimes necessary. >>> >>> Thank you for the thorough review and all of your help with this! I've made these changes and will rerun testing and send out a new webrev and an incremental webrev. >>> >>> Thanks! >>> Coleen >>> >>>> >>>> thanks, >>>> Karen >>>> >>>> >>>>> On Jul 17, 2017, at 2:37 PM, coleen.phillimore at oracle.com wrote: >>>>> >>>>> >>>>> Hi, I have made changes for Ioi's comments below. The largest change since .01 is that PredictedLoadedClassCount is used to size dictionaries for class loader data, and should be used to specify the average number of classes loaded by each class loader. Since this is an option available via UnlockExperimentalVMOptions, I'm not sure what the process is for changing the meaning of this option. We will work with our customers who have used this option for specifying the total number of classes loaded. We are also working on the change to allow automatic resizing of the dictionaries during safepoints. >>>>> >>>>> See: >>>>> open webrev at http://cr.openjdk.java.net/~coleenp/7133093.02/webrev >>>>> >>>>> Tested with the full nightly test set (jdk, nsk, hotspot jtreg tests) on linux x64. >>>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>>> On 6/30/17 4:45 PM, coleen.phillimore at oracle.com wrote: >>>>>> >>>>>> Ioi, Thank you for looking at this change. >>>>>> >>>>>>> On 6/30/17 2:39 PM, Ioi Lam wrote: >>>>>>> Hi Coleen, >>>>>>> >>>>>>> Maybe the bug should be renamed to "Use one Dictionary per class loader instance"? That way it's more obvious what it is when you look at the repo history. >>>>>> >>>>>> I can do that. I made it One Dictionary per ClassLoaderData >>>>>>> >>>>>>> 1. >>>>>>> >>>>>>> Why is assert_locked_or_safepoint(SystemDictionary_lock) necessary in SystemDictionary::find_class (line 1826), but not necessary SystemDictionary::find (line 951)? Since you removed NoSafepointVerifier nosafepoint in the latter, maybe this means it's safe to remove the assert_locked_or_safepoint in the former? >>>>>> >>>>>> The call to SystemDictionary::find() is the (I believe) usual lock free lookup and the SystemDictionary::find_class() is used to verify that a class we're about to add or want to add hasn't been added by another thread. Or certain cases where we already have a lock to do something else, like add loader constraints. I took out the NoSafepointVerifier because it assumes that system dictionary entries would be moved by GC, which they won't. The old hash function could safepoint when getting the hash for the class_loader object. >>>>>> >>>>>>> >>>>>>> 2. >>>>>>> >>>>>>> 455 static ClassLoaderData* _current_loader_data = NULL; >>>>>>> 456 static Klass* _current_class_entry = NULL; >>>>>>> 457 >>>>>>> 458 InstanceKlass* ClassLoaderDataGraph::try_get_next_class() { >>>>>>> >>>>>>> How about moving the static fields into an iterator object. That way you don't need to keep track of the globals >>>>>>> >>>>>>> ClassLoaderDataGraphIterator { >>>>>>> ClassLoaderData* _current_loader_data >>>>>>> Klass* _current_class_entry; >>>>>>> >>>>>>> InstanceKlass* try_get_next_class() { ....} >>>>>>> }; >>>>>> >>>>>> Ok, there's a different iterator that iterates over all of the classes for GC. I will adapt that for this use. That would be better. >>>>>> >>>>>>> >>>>>>> 3. Double check locking in ClassLoaderData::dictionary() -- someone else should look at this :-) >>>>>> >>>>>> I copied code that David Holmes added for modules. >>>>>> >>>>>>> >>>>>>> 4. We may need a better strategy for deciding the size of each dictionary. >>>>>>> >>>>>>> 565 const int _primelist[10] = {1, 107, 1009}; >>>>>>> 571 Dictionary* ClassLoaderData::dictionary() { >>>>>>> 579 if ((dictionary = _dictionary) == NULL) { >>>>>>> 580 int size; >>>>>>> 581 if (this == the_null_class_loader_data() || is_system_class_loader_data()) { >>>>>>> 582 size = _primelist[2]; >>>>>>> 583 } else if (class_loader()->is_a(SystemDictionary::reflect_DelegatingClassLoader_klass())) { >>>>>>> 584 size = _primelist[0]; >>>>>>> 585 } else { >>>>>>> 586 size = _primelist[1]; >>>>>>> 587 } >>>>>>> 588 dictionary = new Dictionary(this, size); >>>>>>> >>>>>>> I'll do some investigation on this issue and get back to you. >>>>>>> >>>>>> >>>>>> How about if someone uses PredictedLoadedClassCount, then we use that to size all but the reflection and boot class loader? Then if there's an application that has a class loader with a huge amount of classes loaded in it, that would help this? It might cost some footprint but an oversized table would simply be a bigger array of pointers, so it might not be that bad to oversize. >>>>>> >>>>>> I think the long term solution is being able to resize these entries and make the application provide arguments. Please let me know what you find in your investigation and if that would work. >>>>>>> The rest of the changes look good to me. >>>>>> >>>>>> Thank you! >>>>>> Coleen >>>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>>> >>>>>>> >>>>>>>> On 6/23/17 4:42 PM, coleen.phillimore at oracle.com wrote: >>>>>>>> Summary: Implement one dictionary per ClassLoaderData for faster lookup and removal during class unloading >>>>>>>> >>>>>>>> See RFE for more details. >>>>>>>> >>>>>>>> open webrev at http://cr.openjdk.java.net/~coleenp/7133093.01/webrev >>>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-7133093 >>>>>>>> >>>>>>>> Tested with full "nightly" run in rbt, plus locally class loading and unloading tests: >>>>>>>> >>>>>>>> jtreg hotspot/test/runtime/ClassUnload >>>>>>>> >>>>>>>> jtreg hotspot/test/runtime/modules >>>>>>>> >>>>>>>> jtreg hotspot/test/gc/class_unloading >>>>>>>> >>>>>>>> make test-hotspot-closed-tonga FILTER=quick TEST_JOBS=4 TEST=vm.parallel_class_loading >>>>>>>> >>>>>>>> csh ~/testing/run_jck9 (vm/lang/java_lang) >>>>>>>> >>>>>>>> runThese -jck - uses class loader isolation to run each jck test and unloads tests when done (at -gc:5 intervals) >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Coleen >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From kim.barrett at oracle.com Sat Jul 22 02:26:20 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 21 Jul 2017 22:26:20 -0400 Subject: RFR (S) 8182397: Race in field updates when creating ArrayKlasses can lead to crash In-Reply-To: References: Message-ID: <166602DE-F4BA-40A8-93C8-7C1E74679D77@oracle.com> > On Jul 20, 2017, at 7:44 PM, coleen.phillimore at oracle.com wrote: > > Summary: Update array_klass field in component mirror after klass.java_mirror field for concurrent readers in compiled code. > > See bug for details. > > open webrev at http://cr.openjdk.java.net/~coleenp/8182397.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8182397 > > Ran tier1 testing with -Xcomp and without. Thank you to Tom for the test and diagnosis. > > Thanks, > Coleen ------------------------------------------------------------------------------ src/share/vm/classfile/javaClasses.cpp 888 k->set_java_mirror(mirror()); This was protected by "if (k != NULL)", but you removed that. That removal seems correct, as there are lots of preceeding uses of k that are wrong if it's NULL. However, the similar "if" on line 837 also seems unnecessary and even more confusing now. ------------------------------------------------------------------------------ src/share/vm/classfile/javaClasses.cpp 862 // Set after k->java_mirror() is published, because compiled code running 863 // concurrently doesn't expect a k to have a null java_mirror. 864 // set_array_klass(comp_mirror, k); Rather than this new comment and the commented out set_array_klass, I would rather there be a comment here directing to the later set_java_mirror and set_array_klass pair, and a comment with that pair discussing the importance of the order. ------------------------------------------------------------------------------ src/share/vm/oops/oop.inline.hpp 449 void oopDesc::metadata_field_put_volatile(int offset, Metadata* value) { 450 OrderAccess::release(); 451 *metadata_field_addr(offset) = value; 452 OrderAccess::fence(); 453 } Why not use OrderAccess::release_store_ptr_fence? While I see why we need the release, I don't see any explanation for why we need the trailing fence. Oh, I see. I'm looking at a webrev that hasn't incorporated some of the later discussion. Note that the usual naming convention seems to be to to use a "release_" prefix for this sort of thing. See, for example, release_XXX_field_put. Also, renaming set_array_klass to release_set_array_klass seems called for. There are only two callers, one of which I think doesn't care. ------------------------------------------------------------------------------ test/runtime/CreateMirror/ArraysNewInstanceBug.java I'm guessing this test is subject to false negatives? That is, it may or may not crash when the bug exists, but won't crash if the bug is fixed? If so, that seems worth mentioning in a comment, or maybe the @summary. ------------------------------------------------------------------------------ From kim.barrett at oracle.com Sat Jul 22 02:43:54 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 21 Jul 2017 22:43:54 -0400 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: <8e1d9a32-7e01-b917-df1d-595c4000e4b2@redhat.com> References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <596CD126.6080100@oracle.com> <596DD713.6080806@oracle.com> <77d5a9e7-b2b0-79e4-effd-903dfe226512@redhat.com> <596E1142.6040708@oracle.com> <3432d11c-5904-a446-1cb5-419627f1794f@redhat.com> <596E288B.90706@oracle.com> <773a0b64-8273-1a58-a46a-22ae18f927b7@redhat.com> <5970B3FD.607@oracle.com> <8e1d9a32-7e01-b917-df1d-595c4000e4b2@redhat.com> Message-ID: > On Jul 21, 2017, at 9:43 AM, Andrew Haley wrote: > However, if we are using -fno-strict-aliasing, > then I do not believe that there is any need for the complexity of the > casting that your patch does. We should be able to do the operations > that we need with no type casting at all, with suitable use of > template functions. Some of the casting exists in order to target the (new) generic API at the existing specialized_xxx operations. Erik and I discussed making changes to the specialized layer, but decided not to at this time. That's the layer where all the platform-specific assembly code is located, and having to make significant changes across all platforms in that layer seemed too risky for the immediate goal of a cleaner client level API. As it is, it still got touched quite a bit, but it could have been a lot worse. I think that with a different API for the specialized layer it would be pretty easy to eliminate the pointer casts for some gcc-based targets. For example, I think something like the following (completely untested code) could work for linux_x64, and doesn't involve any casts between pointer types. Instead, it relies on implicit type erasure at the interface between C++ and inline assembly. class Atomic ... { ... template class SpecializedAdd; ... }; // *d += i, for integral *d template inline D Atomic::add(I i, D volatile* d) { STATIC_ASSERT(IsInteger::value); STATIC_ASSERT(IsInteger::value); D v = i; // Let -Wconversion or similar complain if unsafe return SpecializedAdd()(v, d); } // *d += i, for pointer *d template inline D* Atomic::add(I i, D* volatile* d) { STATIC_ASSERT(IsInteger::value); STATIC_ASSERT(sizeof(ptrdiff_t) == sizeof(D*)); ptrdiff_t v = i; // Let -Wconversion or similar complain if unsafe v *= sizeof(D); return SpecializedAdd()(v, d); } template<> struct Atomic::SpecializedAdd<4> { template D operator()(I i, D volatile* d) const { I addend = i; __asm__ volatile("lock xaddl %0,(%2)" : "=r" (addend) : "0" (addend), "r" (d) : "cc", "memory"); // For integer case, I == D, and cast is a nop. // For pointer case (D is P*), cast from I => P*. return reinterpret_cast(addend + i); } }; #ifdef _LP64 template<> struct Atomic::SpecializedAdd<8> { template D operator()(I i, D volatile* d) const { I addend = i; __asm__ volatile("lock xaddq %0,(%2)" : "=r" (addend) : "0" (addend), "r" (d) : "cc", "memory"); // For integer case, I == D, and cast is a nop. // For pointer case (D is P*), cast from I => P*. return reinterpret_cast(addend + i); } }; #endif Maybe this is the kind of thing you are suggesting? But what do we do when we need a helper stub function, like _Atomic_cmpxchg_long for linux_x86. That can't be dealt with using templates. And for some targets, that's what we have for pretty much everything. From thomas.stuefe at gmail.com Sat Jul 22 08:37:06 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 22 Jul 2017 10:37:06 +0200 Subject: RFR(xs): 8185033: On Metaspace OOM, ChunkManager composition should be logged. In-Reply-To: <74885bac-a0f9-52ea-cc90-b8b6ef39edee@oracle.com> References: <74885bac-a0f9-52ea-cc90-b8b6ef39edee@oracle.com> Message-ID: Hi Coleen, Mikael, new webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8185033-On-Metaspace-OOM-chunkManager-composition-should-be-logged/webrev.01/webrev/ I only applied Mikaels patch atop of mine, rebased to top (after the UL refactoring change Eric pushed yesterday) and fixed merge errors. Thanks, Thomas On Fri, Jul 21, 2017 at 8:54 PM, wrote: > > > On 7/21/17 2:16 PM, Thomas St?fe wrote: > > Hi Coleen, > > On Fri, Jul 21, 2017 at 5:34 PM, wrote: > >> >> Hi Thomas, >> Thank you for working on this. >> >> > thanks for reviewing! > > >> >> On 7/21/17 2:54 AM, Thomas St?fe wrote: >> >>> Dear all, >>> >>> May I please get reviews and a sponsor for the following small >>> enhancement. >>> >>> Issue: https://bugs.openjdk.java.net/browse/JDK-8185033 >>> Webrev: >>> http://cr.openjdk.java.net/~stuefe/webrevs/8185033-On-Metasp >>> ace-OOM-chunkManager-composition-should-be-logged/webrev.00/webrev/ >>> >>> This patch adds a bit of logging if a Metaspace OOM happens. We will now >>> get a printout of the composition of the global chunk managers (non-class >>> and, if necessary, class). >>> >>> Before it looked like this: >>> >>> [272.727s][info][gc,metaspace,freelist] All Metaspace: >>> [272.728s][info][gc,metaspace,freelist] data space: Chunk accounting: >>> used in chunks 100752K + unused in chunks 650K + capacity in free >>> chunks >>> 8293K = 109696K capacity in allocated chunks 101403K >>> [272.728s][info][gc,metaspace,freelist] class space: Chunk accounting: >>> used in chunks 4047K + unused in chunks 97K + capacity in free chunks >>> 6085K = 10230K capacity in allocated chunks 4145K >>> [272.729s][info][gc,metaspace,freelist] Total fragmentation waste >>> (words) >>> doesn't count free space >>> [272.729s][info][gc,metaspace,freelist] data: 183 specialized(s) 960, >>> 169 >>> small(s) 41, 1507 medium(s) 4480, large count 1 >>> [272.729s][info][gc,metaspace,freelist] class: 183 specialized(s) 0, 13 >>> small(s) 12, 111 medium(s) 8, large count 1 >>> >>> Now: >>> [250.548s][info][gc,metaspace,freelist] All Metaspace: >>> [250.549s][info][gc,metaspace,freelist] data space: Chunk accounting: >>> used in chunks 100748K + unused in chunks 638K + capacity in free >>> chunks >>> 10357K = 111744K capacity in allocated chunks 101387K >>> [250.550s][info][gc,metaspace,freelist] class space: Chunk accounting: >>> used in chunks 4047K + unused in chunks 97K + capacity in free chunks >>> 6085K = 10230K capacity in allocated chunks 4145K >>> [250.550s][info][gc,metaspace,freelist] Total fragmentation waste >>> (words) >>> doesn't count free space >>> [250.550s][info][gc,metaspace,freelist] data: 183 specialized(s) 960, >>> 165 >>> small(s) 27, 1507 medium(s) 4504, large count 1 >>> [250.550s][info][gc,metaspace,freelist] class: 183 specialized(s) 0, 13 >>> small(s) 12, 111 medium(s) 14, large count 1 >>> [250.550s][info][gc,metaspace,freelist] Chunkmanager (non-class): >>> [250.550s][info][gc,metaspace,freelist] 89 specialized (128 bytes) >>> chunks, total 91136 bytes >>> [250.550s][info][gc,metaspace,freelist] 2567 small (512 bytes) chunks, >>> total 10514432 bytes >>> [250.550s][info][gc,metaspace,freelist] 0 medium (8192 bytes) chunks, >>> total 0 bytes >>> [250.550s][info][gc,metaspace,freelist] 0 humongous chunks, total 0 >>> bytes >>> [250.550s][info][gc,metaspace,freelist] total size: 10605568. >>> [250.550s][info][gc,metaspace,freelist] Chunkmanager (class): >>> [250.550s][info][gc,metaspace,freelist] 87 specialized (128 bytes) >>> chunks, total 89088 bytes >>> [250.550s][info][gc,metaspace,freelist] 2999 small (256 bytes) chunks, >>> total 6141952 bytes >>> [250.550s][info][gc,metaspace,freelist] 0 medium (4096 bytes) chunks, >>> total 0 bytes >>> [250.550s][info][gc,metaspace,freelist] 0 humongous chunks, total 0 >>> bytes >>> [250.550s][info][gc,metaspace,freelist] total size: 6231040. >>> >> >> Yes, this looks like it'll help. Your numbers are less mystifying than >> the existing ones. >> >>> >>> This helps in understanding the underlying cause of the OOM, if that >>> cause >>> is related to chunks lingering around in the freelist. See also [1], a >>> Jep >>> proposing to improve the metaspace chunk allocation to increase the >>> chance >>> of chunk reuse. This patch here is also a preparation for the prototype >>> work of [1]. >>> >>> Note that admittedly, metaspace.cpp will again gain a bit of complexity >>> by >>> this patch. When examining the coding, I was tempted to refactor but >>> refrained from doing so to keep the patch focused. But I think >>> metaspace.cpp (especially the coding associated with statistics) could be >>> cleaned up a bit. I opened a follow up item [2] to collect >>> cleanup-issues. >>> I am willing to do that myself, but would prefer it to be done after >>> functional patches - which still have a slight chance of being backported >>> to jdk9 - are all being added. >>> >> >> My first impression was that get_statistics() and print_statistics() >> should be member functions of ChunkManagerStatistics because of all the >> stat->lines. But ChunkManager has to own these also. I can't think of >> anything better. >> >> > I wanted to keep the structure a dumb data holder (I actually would have > preferred a more C-ish name like cmstat_t or similar but I kept the name in > line with existing name conventions). But I could make "ChunkManager::print_statistics" > a member function of ChunkManagerStatistics, if you prefer it that way. I > think it does not matter much, just a question of aesthetics. > > > Yes, it seemed that you were trying to encapsulate the data and I don't > see a better way. I'm glad you kept the name of the class with the > existing naming conventions because you may find an improvement to make it > aesthetic at some point. It's not worth changing now. > > > >> Yes, the metaspace code could definitely use refactoring with respect to >> statistics collection. Please! Also I added to your RFE to rename >> MetaspaceAux. >> >> > Great! Lets use that RFE to collect more thoughts. > > >> I agree with Mikael about adding the code to avoid casting. You could >> change the type to 'int' with another patch (or this one, I don't mind >> looking at it again). >> >> > I will do as Mikael requested, his additions make sense. Changing the > numbers to "int" can be part of that cleanup to come. > > >> I could sponsor this when it's ready. >> >> > Thanks! I'll post an updated webrev soon. > > > I'll watch for it. > Thanks, > Coleen > > > >> Thanks! >> Coleen >> >> > Kind Regards Thomas > >> >> >> >>> Patch was built and tested on Windows x64, Linux x64 and AIX. I also ran >>> jtreg hotspot/runtime/Metaspace tests, no issues. >>> >>> Kind Regards, Thomas >>> >>> ---- >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8166690 >>> [2] https://bugs.openjdk.java.net/browse/JDK-8185034 >>> >> >> > > From kim.barrett at oracle.com Sat Jul 22 20:55:41 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 22 Jul 2017 16:55:41 -0400 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <596CD126.6080100@oracle.com> <596DD713.6080806@oracle.com> <77d5a9e7-b2b0-79e4-effd-903dfe226512@redhat.com> <596E1142.6040708@oracle.com> <3432d11c-5904-a446-1cb5-419627f1794f@redhat.com> <596E288B.90706@oracle.com> <773a0b64-8273-1a58-a46a-22ae18f927b7@redhat.com> <5970B3FD.607@oracle.com> <8e1d9a32-7e01-b917-df1d-595c4000e4b2@redhat.com> Message-ID: <6A285E44-1374-4ECE-9802-4B3E164407E5@oracle.com> > On Jul 21, 2017, at 10:43 PM, Kim Barrett wrote: > For example, I think something like the following > (completely untested code) could work for linux_x64, and doesn't > involve any casts between pointer types. Instead, it relies on > implicit type erasure at the interface between C++ and inline > assembly. > > [?] > template<> > struct Atomic::SpecializedAdd<4> { > template > D operator()(I i, D volatile* d) const { > I addend = i; > __asm__ volatile("lock xaddl %0,(%2)" > : "=r" (addend) > : "0" (addend), "r" (d) > : "cc", "memory"); > // For integer case, I == D, and cast is a nop. > // For pointer case (D is P*), cast from I => P*. > return reinterpret_cast(addend + i); > } > }; > [?] Sorry, I was in a hurry and forgot to fix the cast to a new pointer of the result of arithmetic on the integer representation of an existing pointer. That?s easily dealt with: template<> struct Atomic::SpecializedAdd<4> { template D operator()(I i, I scaled_i, D volatile* d) const { I addend = scaled_i; __asm__ volatile("lock xaddl %0,(%2)" : "=r" (addend) : "0" (addend), "r" (d) : "cc", "memory"); // For integer case, I == D, and cast is a nop. // For pointer case (D is P*), cast from I => P*. return reinterpret_cast(addend) + i; } }; and fix the callers to pass in the appropriately scaled value for scaled_i. From muthusamy.chinnathambi at oracle.com Mon Jul 24 06:56:55 2017 From: muthusamy.chinnathambi at oracle.com (Muthusamy Chinnathambi) Date: Sun, 23 Jul 2017 23:56:55 -0700 (PDT) Subject: [8u] RFR for backport of JDK-8158012: Use SW prefetch instructions instead of BIS for allocation prefetches on SPARC Core C4 In-Reply-To: References: <4768675b-cf31-4351-902a-51ab0a4bd5c7@default> Message-ID: Hi Vladimir, Thanks for the review. Please find the updated webrev at http://cr.openjdk.java.net/~shshahma/Muthu/8158012/webrev.01/ Regards, Muthusamy C -----Original Message----- From: Vladimir Kozlov Sent: Friday, July 21, 2017 4:38 AM To: Muthusamy Chinnathambi ; hotspot-dev at openjdk.java.net Subject: Re: [8u] RFR for backport of JDK-8158012: Use SW prefetch instructions instead of BIS for allocation prefetches on SPARC Core C4 The indent for next comment line in vm_version_sparc.cpp is off: // Use BIS instr And AllocatePrefetchStyle set to 3 should be outside new check (should be set for both cases). In JDK 9 there is following code which set it for AllocatePrefetchInstr == 1 which JDK8 does not have: http://hg.openjdk.java.net/jdk9/jdk9/hotspot/file/eebdc7acebd4/src/cpu/sparc/vm/vm_version_sparc.cpp#l126 thanks, Vladimir On 7/20/17 12:24 AM, Muthusamy Chinnathambi wrote: > Hi, > > Please review the backport of bug: "JDK-8158012: Use SW prefetch instructions instead of BIS for allocation prefetches on SPARC Core C4" to jdk8u-dev > > Please note that this is not a clean backport due to conflict in comments and slight re-order of code. > > > Webrev: http://cr.openjdk.java.net/~shshahma/Muthu/8158012/webrev.00/ > jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-8158012 > Original patch pushed to jdk9: http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/eebdc7acebd4 > > Test: Ran jprt and jtreg. > > Regards, > Muthusamy C > From erik.helin at oracle.com Mon Jul 24 07:46:58 2017 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 24 Jul 2017 09:46:58 +0200 Subject: Request for review for JDK-8184045 - TestSystemGCWithG1.java times out on Solaris SPARC In-Reply-To: <4afe0501-a927-7ecf-b229-63aeeecf76be@oracle.com> References: <98794b3a-ac78-26b5-8647-9f8f99aef048@oracle.com> <455156a7-d4b2-364c-e5b6-e0ed804ae93b@oracle.com> <4afe0501-a927-7ecf-b229-63aeeecf76be@oracle.com> Message-ID: <91535736-46c5-7434-ced8-73da24384333@oracle.com> On 07/21/2017 07:17 PM, Alexander Harlap wrote: > Hi Erik, > > Thank you for your suggestion. > > New version is at http://cr.openjdk.java.net/~aharlap/8184045/webrev.01/ Looks good, Reviewed. Thanks, Erik > Alex > > > On 7/21/2017 8:44 AM, Erik Helin wrote: >> On 07/21/2017 01:50 PM, Erik Helin wrote: >>> On 07/20/2017 05:31 PM, Alexander Harlap wrote: >>>> Please review changes forJDK-8184045 >>>> - >>>> TestSystemGCWithG1.java times out on Solaris SPARC (labeled as >>>> integration blocker) >>>> >>>> Proposed change will avoid timeout failure for new stress tests >>>> TestSystemGCWithG1.java, TestSystemGCWithCMS.java and >>>> TestSystemGCWithSerial.java regularly observed on Solaris Sparc >>>> >>>> Change is located at >>>> http://cr.openjdk.java.net/~aharlap/8184045/webrev.00/ >>> >>> Instead of removing the second pass, can you check if the args[0] == >>> "long"? The code can look like: >>> >>> public static void main(String[] args) { >>> populateLongLived(); >>> runAllPhases(); >>> if (args.length > 0 && args[0] == "long") { >> >> ...aaaand as couple of my colleagues pointed out while smiling, you >> should of course write args[0].equals("long") :) >> >> Thanks, >> Erik >> >>> runAllPhases(); >>> } >>> } >>> >>> This way the stress test can be used in "long" and "short" mode. >>> >>> Thanks, >>> Erik >>> >>>> Alex >>>> > From alexander.harlap at oracle.com Mon Jul 24 16:01:05 2017 From: alexander.harlap at oracle.com (Alexander Harlap) Date: Mon, 24 Jul 2017 12:01:05 -0400 Subject: Request for review for JDK-8184045 - TestSystemGCWithG1.java times out on Solaris SPARC In-Reply-To: <91535736-46c5-7434-ced8-73da24384333@oracle.com> References: <98794b3a-ac78-26b5-8647-9f8f99aef048@oracle.com> <455156a7-d4b2-364c-e5b6-e0ed804ae93b@oracle.com> <4afe0501-a927-7ecf-b229-63aeeecf76be@oracle.com> <91535736-46c5-7434-ced8-73da24384333@oracle.com> Message-ID: <08b6e965-7ebe-787a-302b-8361271ab300@oracle.com> Thank you very much! Alex On 7/24/2017 3:46 AM, Erik Helin wrote: > On 07/21/2017 07:17 PM, Alexander Harlap wrote: >> Hi Erik, >> >> Thank you for your suggestion. >> >> New version is at http://cr.openjdk.java.net/~aharlap/8184045/webrev.01/ > > Looks good, Reviewed. > > Thanks, > Erik > >> Alex >> >> >> On 7/21/2017 8:44 AM, Erik Helin wrote: >>> On 07/21/2017 01:50 PM, Erik Helin wrote: >>>> On 07/20/2017 05:31 PM, Alexander Harlap wrote: >>>>> Please review changes forJDK-8184045 >>>>> - >>>>> TestSystemGCWithG1.java times out on Solaris SPARC (labeled as >>>>> integration blocker) >>>>> >>>>> Proposed change will avoid timeout failure for new stress tests >>>>> TestSystemGCWithG1.java, TestSystemGCWithCMS.java and >>>>> TestSystemGCWithSerial.java regularly observed on Solaris Sparc >>>>> >>>>> Change is located at >>>>> http://cr.openjdk.java.net/~aharlap/8184045/webrev.00/ >>>> >>>> Instead of removing the second pass, can you check if the args[0] == >>>> "long"? The code can look like: >>>> >>>> public static void main(String[] args) { >>>> populateLongLived(); >>>> runAllPhases(); >>>> if (args.length > 0 && args[0] == "long") { >>> >>> ...aaaand as couple of my colleagues pointed out while smiling, you >>> should of course write args[0].equals("long") :) >>> >>> Thanks, >>> Erik >>> >>>> runAllPhases(); >>>> } >>>> } >>>> >>>> This way the stress test can be used in "long" and "short" mode. >>>> >>>> Thanks, >>>> Erik >>>> >>>>> Alex >>>>> >> From vladimir.kozlov at oracle.com Mon Jul 24 16:21:57 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 24 Jul 2017 09:21:57 -0700 Subject: [8u] RFR for backport of JDK-8158012: Use SW prefetch instructions instead of BIS for allocation prefetches on SPARC Core C4 In-Reply-To: References: <4768675b-cf31-4351-902a-51ab0a4bd5c7@default> Message-ID: <2d0ca8d5-139e-3b8b-a96d-6daa04d94c7b@oracle.com> This looks good. Thanks, Vladimir On 7/23/17 11:56 PM, Muthusamy Chinnathambi wrote: > Hi Vladimir, > > Thanks for the review. > Please find the updated webrev at http://cr.openjdk.java.net/~shshahma/Muthu/8158012/webrev.01/ > > Regards, > Muthusamy C > > -----Original Message----- > From: Vladimir Kozlov > Sent: Friday, July 21, 2017 4:38 AM > To: Muthusamy Chinnathambi ; hotspot-dev at openjdk.java.net > Subject: Re: [8u] RFR for backport of JDK-8158012: Use SW prefetch instructions instead of BIS for allocation prefetches on SPARC Core C4 > > The indent for next comment line in vm_version_sparc.cpp is off: > > // Use BIS instr > > And AllocatePrefetchStyle set to 3 should be outside new check (should > be set for both cases). In JDK 9 there is following code which set it > for AllocatePrefetchInstr == 1 which JDK8 does not have: > > http://hg.openjdk.java.net/jdk9/jdk9/hotspot/file/eebdc7acebd4/src/cpu/sparc/vm/vm_version_sparc.cpp#l126 > > thanks, > Vladimir > > On 7/20/17 12:24 AM, Muthusamy Chinnathambi wrote: >> Hi, >> >> Please review the backport of bug: "JDK-8158012: Use SW prefetch instructions instead of BIS for allocation prefetches on SPARC Core C4" to jdk8u-dev >> >> Please note that this is not a clean backport due to conflict in comments and slight re-order of code. >> >> >> Webrev: http://cr.openjdk.java.net/~shshahma/Muthu/8158012/webrev.00/ >> jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-8158012 >> Original patch pushed to jdk9: http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/eebdc7acebd4 >> >> Test: Ran jprt and jtreg. >> >> Regards, >> Muthusamy C >> From erik.osterlund at oracle.com Mon Jul 24 16:44:02 2017 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Mon, 24 Jul 2017 18:44:02 +0200 Subject: RFR (S) 8182397: Race in field updates when creating ArrayKlasses can lead to crash In-Reply-To: References: Message-ID: <807A38E7-692E-4849-9258-75CFCFDA404A@oracle.com> Hi Andrew, > On 21 Jul 2017, at 15:47, Andrew Dinn wrote: > > Hi Coleen, > >> On 21/07/17 00:44, coleen.phillimore at oracle.com wrote: >> Summary: Update array_klass field in component mirror after >> klass.java_mirror field for concurrent readers in compiled code. >> >> See bug for details. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8182397.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8182397 >> >> Ran tier1 testing with -Xcomp and without. Thank you to Tom for the >> test and diagnosis. > The re-ordering of the writes to > > k->set_java_mirror(mirror()); > . . . > set_array_klass(comp_mirror(), k); > > is all that is needed, assuming that the second write has release > semantics. I discussed this with Andrew Haley and he confirmed my > assumption (also Tom's) that on AArch64 there is no need for a load > acquire on the reader side because of the address dependency between any > load of the component mirror's array klass k and ensuing load of k's mirror. Sorry for jumping in to the conversation a bit late. Since David Holmes is on vacation, I have to take his place in questioning the elision of acquire on the reader side being okay. Because I know he would if he was here. In hotspot we almost *never* elide the acquire on the reader side, relying on dependent loads. This would be an exception to that rule, and I do not see why this is warranted. In order to utilize the dependent load properties in shared code, we need memory ordering semantics for this. That memory ordering semantics is called consume in C++11, and has arguably turned out to be a terrible idea. Hotspot does not have a consume memory ordering like C++11, that allows utilizing dependent loads, and I do not think we intend to introduce it any time soon unless there are very compelling reasons to do so. In its absence, we should use load_acquire instead, and not micro optimize it away. AFAIK, compiler implementers are not happy with C++11 consume memory ordering semantics and implement it with acquire instead as there are too many problems with consume. To elide acquire in an acquire release pair in shared code, very strong motivation is needed. Like for example how constructors publish final object references with release but relies on dependent load in JIT compiled code to elide acquire, because using acquire semantics on all reference loads is too expensive, so it is warranted here. Therefore, I am wondering if perhaps we want to add an acquire on the load side anyway, so we do not have to introduce consume memory ordering semantics or something similar in the shared code base. Using a normal load without any memory ordering semantics in shared code to act like a load consume rings alarm clocks in my head. Thanks, /Erik > However, I think your implementation of method > metadata_field_put_volatile() is overkill. You have > > void oopDesc::metadata_field_put_volatile(int offset, Metadata* value) { > OrderAccess::release(); > *metadata_field_addr(offset) = value; > OrderAccess::fence(); > } > > That fence() will impose a full memory barrier which is more than is > actually needed. It should be ok just to use another release. > > void oopDesc::metadata_field_put_volatile(int offset, Metadata* value) { > OrderAccess::release(); > *metadata_field_addr(offset) = value; > OrderAccess::release(); > } > > Alternatively, I believe you could implement the assign of using a call > to OrderAccess::store_release_ptr which would achieve the same outcome > on AArch64. > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From aph at redhat.com Mon Jul 24 17:40:26 2017 From: aph at redhat.com (Andrew Haley) Date: Mon, 24 Jul 2017 18:40:26 +0100 Subject: RFR (S) 8182397: Race in field updates when creating ArrayKlasses can lead to crash In-Reply-To: <807A38E7-692E-4849-9258-75CFCFDA404A@oracle.com> References: <807A38E7-692E-4849-9258-75CFCFDA404A@oracle.com> Message-ID: <0eb805af-7ac5-874f-eaa4-33ce1788d427@redhat.com> On 24/07/17 17:44, Erik Osterlund wrote: > Sorry for jumping in to the conversation a bit late. > Since David Holmes is on vacation, I have to take his place in > questioning the elision of acquire on the reader side being > okay. Because I know he would if he was here. > > In hotspot we almost *never* elide the acquire on the reader side, > relying on dependent loads. I'm sure we do in plenty of places, it's just undocumented or maybe even unknown to the original author. We certainly do this a lot in the code that the JIT compilers generate, as you note below. > This would be an exception to that rule, and I do not see why this > is warranted. In order to utilize the dependent load properties in > shared code, we need memory ordering semantics for this. That memory > ordering semantics is called consume in C++11, and has arguably > turned out to be a terrible idea. Hotspot does not have a consume > memory ordering like C++11, that allows utilizing dependent loads, > and I do not think we intend to introduce it any time soon unless > there are very compelling reasons to do so. In its absence, we > should use load_acquire instead, and not micro optimize it away. > > AFAIK, compiler implementers are not happy with C++11 consume memory > ordering semantics and implement it with acquire instead as there > are too many problems with consume. I strongly disagree. Consume is fine now: I think you're dealing with out-of-date information. It took a while to figure out how to specify it, that's all. > To elide acquire in an acquire release pair in shared code, very > strong motivation is needed. Like for example how constructors > publish final object references with release but relies on dependent > load in JIT compiled code to elide acquire, because using acquire > semantics on all reference loads is too expensive, so it is > warranted here. Right. So we already depend on that property, so... > Therefore, I am wondering if perhaps we want to add an acquire on > the load side anyway, so we do not have to introduce consume memory > ordering semantics or something similar in the shared code > base. Using a normal load without any memory ordering semantics in > shared code to act like a load consume rings alarm clocks in my > head. I don't think it's that much of a big deal, really. Putting in an acquire fence every time we read a java mirror strikes me as madness when consume is all we need on any target. Even Power PC doesn't need a fence instruction there. At least make sure that targets can turn that off: we should not pay the price for something we don't need. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From coleen.phillimore at oracle.com Mon Jul 24 18:52:22 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 24 Jul 2017 14:52:22 -0400 Subject: RFR (S) 8182397: Race in field updates when creating ArrayKlasses can lead to crash In-Reply-To: <166602DE-F4BA-40A8-93C8-7C1E74679D77@oracle.com> References: <166602DE-F4BA-40A8-93C8-7C1E74679D77@oracle.com> Message-ID: <7bec60a2-d23c-f72c-5bd0-026c158bc3d6@oracle.com> New webrev, see below: http://cr.openjdk.java.net/~coleenp/8182397.02/webrev/index.html On 7/21/17 10:26 PM, Kim Barrett wrote: >> On Jul 20, 2017, at 7:44 PM, coleen.phillimore at oracle.com wrote: >> >> Summary: Update array_klass field in component mirror after klass.java_mirror field for concurrent readers in compiled code. >> >> See bug for details. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8182397.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8182397 >> >> Ran tier1 testing with -Xcomp and without. Thank you to Tom for the test and diagnosis. >> >> Thanks, >> Coleen > ------------------------------------------------------------------------------ > src/share/vm/classfile/javaClasses.cpp > 888 k->set_java_mirror(mirror()); > > This was protected by "if (k != NULL)", but you removed that. That > removal seems correct, as there are lots of preceeding uses of k that > are wrong if it's NULL. However, the similar "if" on line 837 also > seems unnecessary and even more confusing now. Interesting. k can be NULL for primitive types but these mirrors are created in create_basic_type_mirror now. The conditionals weren't taken out when that function was added. Added at the beginning of create_mirror: assert(k != NULL, "Use create_basic_type_mirror for primitive types"); > > ------------------------------------------------------------------------------ > src/share/vm/classfile/javaClasses.cpp > 862 // Set after k->java_mirror() is published, because compiled code running > 863 // concurrently doesn't expect a k to have a null java_mirror. > 864 // set_array_klass(comp_mirror, k); > > Rather than this new comment and the commented out set_array_klass, I > would rather there be a comment here directing to the later > set_java_mirror and set_array_klass pair, and a comment with that pair > discussing the importance of the order. How about: set_component_mirror(mirror(), comp_mirror()); // See below for ordering dependencies between field array_klass in component mirror // and java_mirror in this klass. and this comment is moved to: // Setup indirection from klass->mirror // after any exceptions can happen during allocations. k->set_java_mirror(mirror()); if (comp_mirror() != NULL) { // Set after k->java_mirror() is published, because compiled code running // concurrently doesn't expect a k to have a null java_mirror. set_array_klass(comp_mirror(), k); } > > ------------------------------------------------------------------------------ > src/share/vm/oops/oop.inline.hpp > 449 void oopDesc::metadata_field_put_volatile(int offset, Metadata* value) { > 450 OrderAccess::release(); > 451 *metadata_field_addr(offset) = value; > 452 OrderAccess::fence(); > 453 } > > Why not use OrderAccess::release_store_ptr_fence? > > While I see why we need the release, I don't see any explanation for > why we need the trailing fence. > > Oh, I see. I'm looking at a webrev that hasn't incorporated some of > the later discussion. Note that the usual naming convention seems to > be to to use a "release_" prefix for this sort of thing. See, for > example, release_XXX_field_put. Yes, I made the minor change without a new webrev. There are both conventions. There's obj_field_put_volatile, which I used this convention. There's also release_obj_field_put, which seems more appropriate for this metadata_field_put function, ie. changed to release_metadata_field_put. > > Also, renaming set_array_klass to release_set_array_klass seems called > for. There are only two callers, one of which I think doesn't care. Sure, fixed. > > ------------------------------------------------------------------------------ > test/runtime/CreateMirror/ArraysNewInstanceBug.java > > I'm guessing this test is subject to false negatives? That is, it may > or may not crash when the bug exists, but won't crash if the bug is > fixed? If so, that seems worth mentioning in a comment, or maybe the > @summary. > > ------------------------------------------------------------------------------ > The test always crashes with the bug present. Thanks, Coleen From coleen.phillimore at oracle.com Mon Jul 24 19:39:09 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 24 Jul 2017 15:39:09 -0400 Subject: RFR (S) 8182397: Race in field updates when creating ArrayKlasses can lead to crash In-Reply-To: <0eb805af-7ac5-874f-eaa4-33ce1788d427@redhat.com> References: <807A38E7-692E-4849-9258-75CFCFDA404A@oracle.com> <0eb805af-7ac5-874f-eaa4-33ce1788d427@redhat.com> Message-ID: Thank you for your comments, Erik and Andrew. I like the idea of rules for innocent coders (ones who aren't versed in the complexities of architecture ordering semantics like myself), that say things like "if you have a store-release, you should have a load-acquire on the other side". I would follow this if the load were in the C++ code. Since it's in the JIT compiled code, I think taking advantage of the knowledge that this is a dependent load (if I understand this correctly) seems acceptable and recommended because performance counts. Thanks, Coleen On 7/24/17 1:40 PM, Andrew Haley wrote: > On 24/07/17 17:44, Erik Osterlund wrote: >> Sorry for jumping in to the conversation a bit late. >> Since David Holmes is on vacation, I have to take his place in >> questioning the elision of acquire on the reader side being >> okay. Because I know he would if he was here. >> >> In hotspot we almost *never* elide the acquire on the reader side, >> relying on dependent loads. > I'm sure we do in plenty of places, it's just undocumented or maybe > even unknown to the original author. We certainly do this a lot in > the code that the JIT compilers generate, as you note below. > >> This would be an exception to that rule, and I do not see why this >> is warranted. In order to utilize the dependent load properties in >> shared code, we need memory ordering semantics for this. That memory >> ordering semantics is called consume in C++11, and has arguably >> turned out to be a terrible idea. Hotspot does not have a consume >> memory ordering like C++11, that allows utilizing dependent loads, >> and I do not think we intend to introduce it any time soon unless >> there are very compelling reasons to do so. In its absence, we >> should use load_acquire instead, and not micro optimize it away. >> >> AFAIK, compiler implementers are not happy with C++11 consume memory >> ordering semantics and implement it with acquire instead as there >> are too many problems with consume. > I strongly disagree. Consume is fine now: I think you're dealing with > out-of-date information. It took a while to figure out how to specify > it, that's all. > >> To elide acquire in an acquire release pair in shared code, very >> strong motivation is needed. Like for example how constructors >> publish final object references with release but relies on dependent >> load in JIT compiled code to elide acquire, because using acquire >> semantics on all reference loads is too expensive, so it is >> warranted here. > Right. So we already depend on that property, so... > >> Therefore, I am wondering if perhaps we want to add an acquire on >> the load side anyway, so we do not have to introduce consume memory >> ordering semantics or something similar in the shared code >> base. Using a normal load without any memory ordering semantics in >> shared code to act like a load consume rings alarm clocks in my >> head. > I don't think it's that much of a big deal, really. Putting in an > acquire fence every time we read a java mirror strikes me as madness > when consume is all we need on any target. Even Power PC doesn't need > a fence instruction there. At least make sure that targets can turn > that off: we should not pay the price for something we don't need. > From kim.barrett at oracle.com Mon Jul 24 20:36:30 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 24 Jul 2017 16:36:30 -0400 Subject: RFR (S) 8182397: Race in field updates when creating ArrayKlasses can lead to crash In-Reply-To: <7bec60a2-d23c-f72c-5bd0-026c158bc3d6@oracle.com> References: <166602DE-F4BA-40A8-93C8-7C1E74679D77@oracle.com> <7bec60a2-d23c-f72c-5bd0-026c158bc3d6@oracle.com> Message-ID: <002E88AA-1462-436B-B921-531908E6D30C@oracle.com> > On Jul 24, 2017, at 2:52 PM, coleen.phillimore at oracle.com wrote: > > > New webrev, see below: > http://cr.openjdk.java.net/~coleenp/8182397.02/webrev/index.html Looks good. From coleen.phillimore at oracle.com Mon Jul 24 21:35:55 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 24 Jul 2017 17:35:55 -0400 Subject: RFR (S) 8182397: Race in field updates when creating ArrayKlasses can lead to crash In-Reply-To: <002E88AA-1462-436B-B921-531908E6D30C@oracle.com> References: <166602DE-F4BA-40A8-93C8-7C1E74679D77@oracle.com> <7bec60a2-d23c-f72c-5bd0-026c158bc3d6@oracle.com> <002E88AA-1462-436B-B921-531908E6D30C@oracle.com> Message-ID: Thank you, Kim. If we want to dicuss this further, we can open a bug against hotspot/compiler to add the load aquires. thanks, Coleen On 7/24/17 4:36 PM, Kim Barrett wrote: >> On Jul 24, 2017, at 2:52 PM, coleen.phillimore at oracle.com wrote: >> >> >> New webrev, see below: >> http://cr.openjdk.java.net/~coleenp/8182397.02/webrev/index.html > Looks good. > From coleen.phillimore at oracle.com Mon Jul 24 23:37:59 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 24 Jul 2017 19:37:59 -0400 Subject: RFR (L) 7133093: Improve system dictionary performance In-Reply-To: <26A04A99-5E1A-4F19-8BD1-B2D8764D1C50@oracle.com> References: <89f0b98c-3cbd-7b87-d76b-5e89a5a676fb@oracle.com> <9243da7c-a8e5-4024-c12e-cba9d4f424c8@oracle.com> <7814C032-0D4E-4E76-B8AD-62A072E8A625@oracle.com> <233c03e0-8958-7676-8e33-5d8f0ec4edf3@oracle.com> <7adf48f1-5908-aeaf-c409-d930895e1715@oracle.com> <26A04A99-5E1A-4F19-8BD1-B2D8764D1C50@oracle.com> Message-ID: <119ec3eb-4b2e-1a5c-1d4e-b4ffc46661f4@oracle.com> Thanks, Ioi. Coleen On 7/21/17 4:37 PM, Ioi Lam wrote: > Hi Coleen, > > The new changes look good to me. Thanks! > Ioi > >> On Jul 21, 2017, at 11:28 AM, coleen.phillimore at oracle.com wrote: >> >> >> From internal discussions, the only sizing of the dictionary for the java.system.class.loader will follow PredictedLoadedClassCount until dictionary resizing is done. The latest full webrev is here: >> >> open webrev at http://cr.openjdk.java.net/~coleenp/7133093.04/webrev >> >> The only change from .03 is >> >> http://cr.openjdk.java.net/~coleenp/7133093.04/webrev/src/share/vm/classfile/classLoaderData.cpp.frames.html >> >> lines 595-630. Please review! >> >> Thanks, >> Coleen >> >> >>> On 7/19/17 10:52 PM, coleen.phillimore at oracle.com wrote: >>> >>> Hi, I've made the changes in the replies below and retested with both our internal NSK tests, jtreg tests and jck tests. Also I've rerun the performance tests, including SPECjbb2015. >>> >>> An incremental webrev can be found at: >>> http://cr.openjdk.java.net/~coleenp/7133093.03.incremental/webrev/ >>> >>> Latest version at: >>> >>> http://cr.openjdk.java.net/~coleenp/7133093.03/webrev/ >>> >>> Thanks, >>> Coleen >>> >>> >>>> On 7/18/17 10:56 AM, coleen.phillimore at oracle.com wrote: >>>> >>>> Karen, Thank you for reviewing the code and for the discussions and consulting along the way. >>>> >>>>> On 7/17/17 4:10 PM, Karen Kinnear wrote: >>>>> Feedback on the initial web rev: >>>>> Thank you for using web rev - that way I can do searches of the overall patch! >>>>> >>>>> First - many thanks for doing this exercise so carefully, for following the specification in continuing >>>>> to record the initiating loader and for splitting this effort into multiple phases to allow further exploration >>>>> of the current expectations/behavior relative to class unloading. >>>>> >>>>> And thank you for all the testing. >>>>> >>>>> Just wanted to ensure that I understand some of the assumptions: >>>>> >>>>> 1) lock free reads of dictionaries - yes >>>> Yes. >>>>> 2) lock free adds of entries to dictionaries - yes >>>> No, adds still take out the SystemDictionary_lock. >>>>> 3) removal of entries or entire dictionaries - at safe point only >>>> Yes. >>>>> 4) rehashing - none today and none planned >>>> Yes. >>>>> 5) resizing of dictionaries - none today, some planned >>>> Yes. >>>>> 6) use of SystemDictionary_lock: essentially unchanged >>>> Yes. >>>>> 7) atomicity guarantees relative to checking loader constraints and updates to the dictionary: unchanged >>>> Yes. >>>>> I did not review the SA/vmStructs - sounds like you had Jini George do that - great. >>>>> >>>>> Minor questions/comments: >>>>> >>>>> SystemDictionary::verify: "Verify of system dictionary failed" -> >>>>> looks like "Verify of placeholders failed" >>>> Fixed. Good find. It's an odd assert: >>>> >>>> guarantee(placeholders()->number_of_entries() >= 0, >>>> "Verify of placeholders failed"); >>>> >>>>> thank you for asserts dictionary != NULL >>>>> Dictionary::print >>>>> is it worth an assert that loader_data != NULL? >>>> Added. It's already asserted in Dictionary::verify() >>>> >>>>> Can you please sanity check if there are other cases in which we need >>>>> to use dictionary_or_null() rather than dictionary()? >>>>> - are there cases in which we could be using a dictionary from an anonymous class? >>>>> e.g. there are a lot of places in systemDictionary.cpp where we >>>>> call class_loader_data(class_loader); loader_data->dictionary(); >>>>> - was the lazily allocated dictionary allocated yet (see next question) >>>> There's an assert in dictionary() that the CLD isn't anonymous. In these calls we get class_loader_data from the class_loader which is always the host class loader for anonymous classes, so this doesn't break with the current design. >>>> >>>>> CLD.cpp >>>>> Would it be less complex to eagerly create the dictionary for a CLD >>>>> since it is extremely rare to not need one for non-anonymous CLDs? >>>>> You could create before publishing the CLD >>>> Yes, I agree it would be better this way. I was looking at some data and it does seem rare to have a CLD with no classes. >>>>> Terminology: >>>>> How are you still using System Dictionary term? >>>>> It still shows up in some comments and in some printed strings >>>> Bad habit, we've been calling it that for years. :) I'll update the comments where it makes sense. >>>> I tried to refer to the ClassLoaderDataGraph dictionaries collectively as the "System Dictionary", as well as the loader constraints and placeholder tables, and the other things that SystemDictionary contains. >>>> >>>> I guess it gets confusing, but it's less awkward than "class loader data dictionaries", and more meaningful than simply "dictionaries". So that's why I didn't change a lot of the comments. >>>> >>>>> biasedLocking.cpp, line 59 >>>>> jvmtiGetLoadedClasses: line 298 >>>>> dictionary.hpp lines 113,114, 120 >>>>> dictionary.cpp 262-263, 478 >>>>> dictionary.cpp 491 >>>>> fix comments e.g. "Iterate the system dictionary" ... >>>>> - probably worth checking for other comments in the sources too >>>>> unless these all appear to match new usage model >>>> Fixed. I've found a couple more and some other comments that are incorrect. >>>>> hashTable.hpp >>>>> line 268: what does this comment mean? >>>>> "Don't create and use freelist"? >>>>> I assume this means that this is a direct malloc, not using any block >>>>> allocation, e.g. metaspace chunks, and this requires explicit freeing? >>>>> >>>> // Don't create and use freelist of HashtableEntry. >>>> >>>> The default Hashtable<> allocates entries from a block of entries and uses a freelist for freed entries. >>>> >>>>> dictionary.hpp line 161 >>>>> could you possibly clean up comment while you are in here? >>>>> I presume "a protection" is "a protection domain"? >>>> Fixed. >>>> >>>>> dictionary.cpp >>>>> why change from Hashtable::free_entry to explicit >>>>> unlink_entry/FREE_C_HEAP_ARRAY? >>>> I used the same code as the ModuleTable. The default hashtable doesn't delete the whole thing and left some memory around, if I remember correctly. I'll recheck it. >>>>> Future optimization: >>>>> I think that none of the built-in loaders is ever recorded as an initiating >>>>> loader for any class in a loader which permits unloading. >>>>> Note: we need to ensure that built-in includes boot, platform and >>>>> application class loader (and clean up any confusion with system >>>>> class loader - which can be set by the user to a custom class loader) >>>>> >>>> I think AppClassLoader can initiate loading and is a builtin loader. >>>> >>>> Dictionary for class loader class loader 0x00007f6abc6e3210 a 'jdk/internal/loader/ClassLoaders$AppClassLoader'{0x0000000101c8df00} >>>> Java system dictionary (table_size=107, classes=1) >>>> ^ indicates that initiating loader is different from defining loader >>>> 106: ^java.util.ArrayList, loader NULL class_loader >>>> >>>>> Any chance of changing DictionaryEntry* probe->klass() to be >>>>> probe->instanceKlass()? >>>> Okay. Changed to instance_klass() as per coding standard. >>>>> Thank you for assertion _is_at_safepoint for try_get_next_class >>>>> >>>>> CLD.cpp >>>>> If the names of the classes_do vs. all_entries_do were modified to be >>>>> defining_loader_classes_do vs. all_entries_classes_do or some way >>>>> to make the distinction in the name that would be helpful. >>>>> Otherwise, if you could please add comments to the CLDG::dictionary_classes_do >>>>> etc. to distinguish these. >>>> I added comments to distinguish these. >>>>> CLD.cpp >>>>> lines 1200-1201 >>>>> Can you please fix the comment? >>>>> I think you need separate comments for the ModuleEntry reads/PackageEntry's >>>>> exports which are looking for dead modules. >>>>> IIUC, the do_unloading is actually looking for classes for which this >>>>> is the initating loader, but their defining loader is dead. >>>> // Remove entries in the dictionary of live class loader that have >>>> // initiated loading classes in a dead class loader. >>>> data->dictionary()->do_unloading(); >>>> >>>> Moved the module comment to before the module code and added this for dictionary->do_unloading() call. >>>>> systemDictionary.cpp: 445-447 >>>>> THANK you for the comments about the java_mirror handle keeping the class >>>>> and class loader alive >>>>> Could you clarify that the class_loader handle is not for the class >>>>> we are checking, but rather for the initiating, i.e. requesting class loader >>>>> (is that accurate?) So we are keeping both the requestor and the >>>>> requested class alive >>>> Yes, that is true. I added >>>> >>>> // The class_loader handle passed in is the initiating loader. >>>> >>>>> You removed the comment that compute_hash could lead to a safepoint/GC >>>>> Is that not possible? >>>>> comment in ObjectSynchronizer::FastHashCode says that with UseBiasedLocking >>>>> we can revoke biases and must be able to handle a safepoint. >>>> I made it not possible with this change. The hashcode is now computed by the Symbol* of the Klass name, which is random (os::random plus some bits from the address and first two characters, which we did a study of for bug https://bugs.openjdk.java.net/browse/JDK-8181450). >>>> >>>> Also, I changed ClassLoaderData::identity_hash() to use the address >> 3 because the addresses of class loader datas are random enough for the one use of this function. Previously, the loader_data identity hash used to go to the class_loader oop and get to ObjectSynchronizer::FastHashCode (which was also os::random but now changed). >>>> >>>> The dictionary hashing doesn't need to add in the hash for the loader_data now. >>>>> systemDictionary.cpp: >>>>> NoSafepointVerifier >>>>> If we add resizing logic later, and that happens at a safepoint, >>>>> then the number of buckets change and the d_index is no longer valid >>>>> so we would need to add back the NSpV? >>>>> - this is a more general problem if compute_hash can lead to a safepoint/GC >>>> Yes this is true, except that compute_hash doesn't safepoint. Computing the hash and index should be done together with the dictionary operations. The function resolve_instance_class_or_null computes the hash and index once at the beginning and reuses it for several lookups, which may or may not affect performance. So we need to measure this in isolation before changing this. >>>>> systemDictionary.cpp: parse_stream >>>>> why did you remove the record_dependency? >>>> I realized it's trivially unnecessary. The class_loader is reached by the ClassLoaderData::_class_loader field, so we don't have to record a dependency from host_klass->class_loader in the anonymous class loader data. (I just checked this again because it's confusing). >>>> >>>>> Who calls Dictionary::~Dictionary? >>>> The ClassLoaderData::~ClassLoaderData() calls delete _dictionary, which calls the destructor. >>>>> Performance: >>>>> Do you have performance runs of our existing benchmarks? >>>> Yes, I ran our internal benchmarks and added links to the internal wiki page. I'll rerun them again before checkin. >>>> >>>>> Initial key requirement is that this not slow down throughput through >>>>> slowing down lookups. >>>>> I know you are very aware of longer-term goals for reducing safepoint pause >>>>> times and not increasing footprint significantly. >>>>> >>>>> hashtable sizes: future rfe? >>>>> given that we now have a lot more dictionaries, and they perhaps need >>>>> individual sizing >>>>> - would it be helpful to add statistics information about >>>>> - e.g. lookup_count, lookup_length, _lookup_warning >>>>> - i.e. number of entries, max lookup depth, etc. to use later for tuning? >>>> All the previous statistics didn't make very much sense to us, so they were removed. There is now logging for all the hashtables: >>>> >>>> busaa027.us.oracle.com% java -Xlog:hashtables -version >>>> java version "10-internal" >>>> Java(TM) SE Runtime Environment (fastdebug build 10-internal+0-2017-07-17-1456035.cphillim.10system-dictionary2) >>>> Java HotSpot(TM) 64-Bit Server VM (fastdebug build 10-internal+0-2017-07-17-1456035.cphillim.10system-dictionary2, mixed mode) >>>> [0.284s][info][hashtables] System Dictionary for jdk/internal/loader/ClassLoaders$AppClassLoader max bucket size 1 bucket 91 element count 1 table size 107 >>>> [0.285s][info][hashtables] System Dictionary for max bucket size 4 bucket 263 element count 468 table size 1009 >>>> [0.285s][info][hashtables] Placeholder Table max bucket size 0 bucket 0 element count 0 table size 1009 >>>> [0.285s][info][hashtables] Protection Domain Table max bucket size 0 bucket 0 element count 0 table size 1009 >>>> >>>>> - might make sense to make this JFR data? >>>> I'll file an RFE to add tracing data with our tools. I'll need some help with this part but the data is there. >>>>> Might think about a follow-up with more tuning parameters than >>>>> a single global PredictedLoadedClassCount. >>>>> >>>> I filed RFE to work on resizing, as this work was almost complete, but didn't show performance improvements at the time. It would be nice to have customers not need tuning parameters, but I guess these things are sometimes necessary. >>>> >>>> Thank you for the thorough review and all of your help with this! I've made these changes and will rerun testing and send out a new webrev and an incremental webrev. >>>> >>>> Thanks! >>>> Coleen >>>> >>>>> thanks, >>>>> Karen >>>>> >>>>> >>>>>> On Jul 17, 2017, at 2:37 PM, coleen.phillimore at oracle.com wrote: >>>>>> >>>>>> >>>>>> Hi, I have made changes for Ioi's comments below. The largest change since .01 is that PredictedLoadedClassCount is used to size dictionaries for class loader data, and should be used to specify the average number of classes loaded by each class loader. Since this is an option available via UnlockExperimentalVMOptions, I'm not sure what the process is for changing the meaning of this option. We will work with our customers who have used this option for specifying the total number of classes loaded. We are also working on the change to allow automatic resizing of the dictionaries during safepoints. >>>>>> >>>>>> See: >>>>>> open webrev at http://cr.openjdk.java.net/~coleenp/7133093.02/webrev >>>>>> >>>>>> Tested with the full nightly test set (jdk, nsk, hotspot jtreg tests) on linux x64. >>>>>> >>>>>> Thanks, >>>>>> Coleen >>>>>> >>>>>>> On 6/30/17 4:45 PM, coleen.phillimore at oracle.com wrote: >>>>>>> >>>>>>> Ioi, Thank you for looking at this change. >>>>>>> >>>>>>>> On 6/30/17 2:39 PM, Ioi Lam wrote: >>>>>>>> Hi Coleen, >>>>>>>> >>>>>>>> Maybe the bug should be renamed to "Use one Dictionary per class loader instance"? That way it's more obvious what it is when you look at the repo history. >>>>>>> I can do that. I made it One Dictionary per ClassLoaderData >>>>>>>> 1. >>>>>>>> >>>>>>>> Why is assert_locked_or_safepoint(SystemDictionary_lock) necessary in SystemDictionary::find_class (line 1826), but not necessary SystemDictionary::find (line 951)? Since you removed NoSafepointVerifier nosafepoint in the latter, maybe this means it's safe to remove the assert_locked_or_safepoint in the former? >>>>>>> The call to SystemDictionary::find() is the (I believe) usual lock free lookup and the SystemDictionary::find_class() is used to verify that a class we're about to add or want to add hasn't been added by another thread. Or certain cases where we already have a lock to do something else, like add loader constraints. I took out the NoSafepointVerifier because it assumes that system dictionary entries would be moved by GC, which they won't. The old hash function could safepoint when getting the hash for the class_loader object. >>>>>>> >>>>>>>> 2. >>>>>>>> >>>>>>>> 455 static ClassLoaderData* _current_loader_data = NULL; >>>>>>>> 456 static Klass* _current_class_entry = NULL; >>>>>>>> 457 >>>>>>>> 458 InstanceKlass* ClassLoaderDataGraph::try_get_next_class() { >>>>>>>> >>>>>>>> How about moving the static fields into an iterator object. That way you don't need to keep track of the globals >>>>>>>> >>>>>>>> ClassLoaderDataGraphIterator { >>>>>>>> ClassLoaderData* _current_loader_data >>>>>>>> Klass* _current_class_entry; >>>>>>>> >>>>>>>> InstanceKlass* try_get_next_class() { ....} >>>>>>>> }; >>>>>>> Ok, there's a different iterator that iterates over all of the classes for GC. I will adapt that for this use. That would be better. >>>>>>> >>>>>>>> 3. Double check locking in ClassLoaderData::dictionary() -- someone else should look at this :-) >>>>>>> I copied code that David Holmes added for modules. >>>>>>> >>>>>>>> 4. We may need a better strategy for deciding the size of each dictionary. >>>>>>>> >>>>>>>> 565 const int _primelist[10] = {1, 107, 1009}; >>>>>>>> 571 Dictionary* ClassLoaderData::dictionary() { >>>>>>>> 579 if ((dictionary = _dictionary) == NULL) { >>>>>>>> 580 int size; >>>>>>>> 581 if (this == the_null_class_loader_data() || is_system_class_loader_data()) { >>>>>>>> 582 size = _primelist[2]; >>>>>>>> 583 } else if (class_loader()->is_a(SystemDictionary::reflect_DelegatingClassLoader_klass())) { >>>>>>>> 584 size = _primelist[0]; >>>>>>>> 585 } else { >>>>>>>> 586 size = _primelist[1]; >>>>>>>> 587 } >>>>>>>> 588 dictionary = new Dictionary(this, size); >>>>>>>> >>>>>>>> I'll do some investigation on this issue and get back to you. >>>>>>>> >>>>>>> How about if someone uses PredictedLoadedClassCount, then we use that to size all but the reflection and boot class loader? Then if there's an application that has a class loader with a huge amount of classes loaded in it, that would help this? It might cost some footprint but an oversized table would simply be a bigger array of pointers, so it might not be that bad to oversize. >>>>>>> >>>>>>> I think the long term solution is being able to resize these entries and make the application provide arguments. Please let me know what you find in your investigation and if that would work. >>>>>>>> The rest of the changes look good to me. >>>>>>> Thank you! >>>>>>> Coleen >>>>>>>> Thanks >>>>>>>> - Ioi >>>>>>>> >>>>>>>> >>>>>>>>> On 6/23/17 4:42 PM, coleen.phillimore at oracle.com wrote: >>>>>>>>> Summary: Implement one dictionary per ClassLoaderData for faster lookup and removal during class unloading >>>>>>>>> >>>>>>>>> See RFE for more details. >>>>>>>>> >>>>>>>>> open webrev at http://cr.openjdk.java.net/~coleenp/7133093.01/webrev >>>>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-7133093 >>>>>>>>> >>>>>>>>> Tested with full "nightly" run in rbt, plus locally class loading and unloading tests: >>>>>>>>> >>>>>>>>> jtreg hotspot/test/runtime/ClassUnload >>>>>>>>> >>>>>>>>> jtreg hotspot/test/runtime/modules >>>>>>>>> >>>>>>>>> jtreg hotspot/test/gc/class_unloading >>>>>>>>> >>>>>>>>> make test-hotspot-closed-tonga FILTER=quick TEST_JOBS=4 TEST=vm.parallel_class_loading >>>>>>>>> >>>>>>>>> csh ~/testing/run_jck9 (vm/lang/java_lang) >>>>>>>>> >>>>>>>>> runThese -jck - uses class loader isolation to run each jck test and unloads tests when done (at -gc:5 intervals) >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Coleen >>>>>>>>> >>>>>>>>> From erik.osterlund at oracle.com Tue Jul 25 08:29:15 2017 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 25 Jul 2017 10:29:15 +0200 Subject: RFR (S) 8182397: Race in field updates when creating ArrayKlasses can lead to crash In-Reply-To: <0eb805af-7ac5-874f-eaa4-33ce1788d427@redhat.com> References: <807A38E7-692E-4849-9258-75CFCFDA404A@oracle.com> <0eb805af-7ac5-874f-eaa4-33ce1788d427@redhat.com> Message-ID: <5977015B.2080102@oracle.com> Hi Andrew, On 2017-07-24 19:40, Andrew Haley wrote: > On 24/07/17 17:44, Erik Osterlund wrote: >> Sorry for jumping in to the conversation a bit late. >> Since David Holmes is on vacation, I have to take his place in >> questioning the elision of acquire on the reader side being >> okay. Because I know he would if he was here. >> >> In hotspot we almost *never* elide the acquire on the reader side, >> relying on dependent loads. > I'm sure we do in plenty of places, it's just undocumented or maybe > even unknown to the original author. That should be purged from the code. With fire. > We certainly do this a lot in > the code that the JIT compilers generate, as you note below. That is more okay when we are in control of the code generation. Consume-like semantics have the following levels of support: 1) No required support: Machines with strong enough memory ordering to guarantee all normal loads and stores have acquire/release semantics require no support. 2) Compiler support: Some machines can rely on dependent loads, but what looks like a dependent load in the C++ source code might not translate to dependent loads in the compiled machine code. In order to enforce that, the data dependency chains from the source code must be retained in the generated machine code. The compiler needs to be explicitly told that it needs to keep them and not perform some otherwise perfectly valid compiler optimizations that would break the dependencies from the source code in the generated machine code. That is what consume semantics do in C++11 atomics. They tell the compiler to keep the dependency chain from the source code in the generated machine code. Without corresponding compiler support, we do not have any guarantees that the generated machine code will retain the data dependencies. It probably will in this case, but "probably" is not a particularly attractive guarantee, and not something I would ever rely upon. Without a contract with the compiler that it will retain the data dependency like C++11 consume semantics, we can not rely on such data dependencies, and must therefore use acquire instead until we start supporting such semantics in OrderAccess. And we certainly do not support it today. 3) Hardware reordering: Some machines like the alpha also requires fencing even on dependent loads, and should use that instead. As for the JIT-compiled code - it is fine to use consume-like semantics if we control the code generation and know that the generated machine code will respect the data dependencies that we wanted it to. But since we do not control the C++ code generation, and do not have access to tools like C++11 consume semantics, we can not just elide acquire and hope for the best. Instead we have to use acquire until OrderAccess starts supporting consume (if it ever will). What we perhaps could do is to add shared code to OrderAccess for load_consume, implement them as load_acquire, so that we at least remember where in the code we could have gotten away with dependent loads and exploit it, and later on plug it into C++11 consume so that C++ compilers that support such contracts can utilize it on a platform by platform basis. >> This would be an exception to that rule, and I do not see why this >> is warranted. In order to utilize the dependent load properties in >> shared code, we need memory ordering semantics for this. That memory >> ordering semantics is called consume in C++11, and has arguably >> turned out to be a terrible idea. Hotspot does not have a consume >> memory ordering like C++11, that allows utilizing dependent loads, >> and I do not think we intend to introduce it any time soon unless >> there are very compelling reasons to do so. In its absence, we >> should use load_acquire instead, and not micro optimize it away. >> >> AFAIK, compiler implementers are not happy with C++11 consume memory >> ordering semantics and implement it with acquire instead as there >> are too many problems with consume. > I strongly disagree. Consume is fine now: I think you're dealing with > out-of-date information. It took a while to figure out how to specify > it, that's all. Perhaps. Last time I looked, compilers would conservatively emit an acquire when asked for consume. Perhaps that has been improved upon since the specification was improved. Nevertheless, I doubt "just treat it like a normal load" was the answer to that, which is what we are doing here. >> To elide acquire in an acquire release pair in shared code, very >> strong motivation is needed. Like for example how constructors >> publish final object references with release but relies on dependent >> load in JIT compiled code to elide acquire, because using acquire >> semantics on all reference loads is too expensive, so it is >> warranted here. > Right. So we already depend on that property, so... Yes - where we can enforce the generated code respects the data dependency. Where we can not enforce it, we use acquire. >> Therefore, I am wondering if perhaps we want to add an acquire on >> the load side anyway, so we do not have to introduce consume memory >> ordering semantics or something similar in the shared code >> base. Using a normal load without any memory ordering semantics in >> shared code to act like a load consume rings alarm clocks in my >> head. > I don't think it's that much of a big deal, really. Putting in an > acquire fence every time we read a java mirror strikes me as madness > when consume is all we need on any target. Even Power PC doesn't need > a fence instruction there. At least make sure that targets can turn > that off: we should not pay the price for something we don't need. Conversely, those targets would pay the price of compromised correctness - a price higher than performance regressions. Performance can only be optimized within the legal bounds of correctness. Therefore we can only elide fences if we have well defined memory ordering semantics for doing that and somehow control that the generated code adheres to those semantics and respects them. We simply do not have consume semantics today, and therefore to ensure correctness, we must use acquire, until the day comes when we support acquire. Doing anything else jeopardizes correctness. And doing that for a premature optimization seems like madness to me. If we are going to knowingly compromise the correctness of the synchronization, I want to at least know that this would cause a significant enough regression to motivate doing anything about it and finding ways to safely elide it, rather than assuming by default that eliding acquire (without anything well defined like compiler consume support to take its place) in the name of performance is the right thing to do, without measuring what that performance impact would be. And that was a surprisingly long sentence. Thanks, /Erik From aph at redhat.com Tue Jul 25 09:12:18 2017 From: aph at redhat.com (Andrew Haley) Date: Tue, 25 Jul 2017 10:12:18 +0100 Subject: RFR (S) 8182397: Race in field updates when creating ArrayKlasses can lead to crash In-Reply-To: <5977015B.2080102@oracle.com> References: <807A38E7-692E-4849-9258-75CFCFDA404A@oracle.com> <0eb805af-7ac5-874f-eaa4-33ce1788d427@redhat.com> <5977015B.2080102@oracle.com> Message-ID: On 25/07/17 09:29, Erik ?sterlund wrote: > > On 2017-07-24 19:40, Andrew Haley wrote: >> On 24/07/17 17:44, Erik Osterlund wrote: >>> Sorry for jumping in to the conversation a bit late. >>> Since David Holmes is on vacation, I have to take his place in >>> questioning the elision of acquire on the reader side being >>> okay. Because I know he would if he was here. >>> >>> In hotspot we almost *never* elide the acquire on the reader side, >>> relying on dependent loads. >> >> I'm sure we do in plenty of places, it's just undocumented or maybe >> even unknown to the original author. > > That should be purged from the code. With fire. Possibly, but let us not burn ourselves in the process. >> We certainly do this a lot in the code that the JIT compilers >> generate, as you note below. > > That is more okay when we are in control of the code generation. > Consume-like semantics have the following levels of support: > > 1) No required support: Machines with strong enough memory ordering to > guarantee all normal loads and stores have acquire/release semantics > require no support. > > 2) Compiler support: Some machines can rely on dependent loads, but what > looks like a dependent load in the C++ source code might not translate > to dependent loads in the compiled machine code. In order to enforce > that, the data dependency chains from the source code must be retained > in the generated machine code. Yes, yes, we know. :-) > As for the JIT-compiled code - it is fine to use consume-like semantics > if we control the code generation and know that the generated machine > code will respect the data dependencies that we wanted it to. But since > we do not control the C++ code generation, and do not have access to > tools like C++11 consume semantics, we can not just elide acquire and > hope for the best. Instead we have to use acquire until OrderAccess > starts supporting consume (if it ever will). OrderAccess could support consume tomorrow: that's up to us. > What we perhaps could do is to add shared code to OrderAccess for > load_consume, implement them as load_acquire, so that we at least > remember where in the code we could have gotten away with dependent > loads and exploit it, and later on plug it into C++11 consume so that > C++ compilers that support such contracts can utilize it on a platform > by platform basis. I suspect we could make it work immediately, because we know how to emit compiler fences on all platforms (do we?) and almost all of them respect data dependencies in hardware anyway. Those that don't will have to use acquire. >> I don't think it's that much of a big deal, really. Putting in an >> acquire fence every time we read a java mirror strikes me as madness >> when consume is all we need on any target. Even Power PC doesn't need >> a fence instruction there. At least make sure that targets can turn >> that off: we should not pay the price for something we don't need. > > Conversely, those targets would pay the price of compromised > correctness - a price higher than performance regressions. > Performance can only be optimized within the legal bounds of > correctness. Please, spare us the lecture. No-one was sugesting compromising correctness. > Therefore we can only elide fences if we have well defined memory > ordering semantics for doing that and somehow control that the > generated code adheres to those semantics and respects them. We > simply do not have consume semantics today, and therefore to ensure > correctness, we must use acquire, until the day comes when we > support acquire. I guess you meant "consume" here? Anyway, all we need is a compiler fence. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From muthusamy.chinnathambi at oracle.com Tue Jul 25 09:14:21 2017 From: muthusamy.chinnathambi at oracle.com (Muthusamy Chinnathambi) Date: Tue, 25 Jul 2017 02:14:21 -0700 (PDT) Subject: [8u] RFR for backport of JDK-8158012: Use SW prefetch instructions instead of BIS for allocation prefetches on SPARC Core C4 In-Reply-To: <2d0ca8d5-139e-3b8b-a96d-6daa04d94c7b@oracle.com> References: <4768675b-cf31-4351-902a-51ab0a4bd5c7@default> <2d0ca8d5-139e-3b8b-a96d-6daa04d94c7b@oracle.com> Message-ID: <3f0b16ad-2eeb-4326-bfd2-f94e17422527@default> May I please get a second review for this change. Thanks, Muthusamy C -----Original Message----- From: Vladimir Kozlov Sent: Monday, July 24, 2017 9:52 PM To: Muthusamy Chinnathambi ; hotspot-dev at openjdk.java.net Subject: Re: [8u] RFR for backport of JDK-8158012: Use SW prefetch instructions instead of BIS for allocation prefetches on SPARC Core C4 This looks good. Thanks, Vladimir On 7/23/17 11:56 PM, Muthusamy Chinnathambi wrote: > Hi Vladimir, > > Thanks for the review. > Please find the updated webrev at http://cr.openjdk.java.net/~shshahma/Muthu/8158012/webrev.01/ > > Regards, > Muthusamy C > > -----Original Message----- > From: Vladimir Kozlov > Sent: Friday, July 21, 2017 4:38 AM > To: Muthusamy Chinnathambi ; hotspot-dev at openjdk.java.net > Subject: Re: [8u] RFR for backport of JDK-8158012: Use SW prefetch instructions instead of BIS for allocation prefetches on SPARC Core C4 > > The indent for next comment line in vm_version_sparc.cpp is off: > > // Use BIS instr > > And AllocatePrefetchStyle set to 3 should be outside new check (should > be set for both cases). In JDK 9 there is following code which set it > for AllocatePrefetchInstr == 1 which JDK8 does not have: > > http://hg.openjdk.java.net/jdk9/jdk9/hotspot/file/eebdc7acebd4/src/cpu/sparc/vm/vm_version_sparc.cpp#l126 > > thanks, > Vladimir > > On 7/20/17 12:24 AM, Muthusamy Chinnathambi wrote: >> Hi, >> >> Please review the backport of bug: "JDK-8158012: Use SW prefetch instructions instead of BIS for allocation prefetches on SPARC Core C4" to jdk8u-dev >> >> Please note that this is not a clean backport due to conflict in comments and slight re-order of code. >> >> >> Webrev: http://cr.openjdk.java.net/~shshahma/Muthu/8158012/webrev.00/ >> jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-8158012 >> Original patch pushed to jdk9: http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/eebdc7acebd4 >> >> Test: Ran jprt and jtreg. >> >> Regards, >> Muthusamy C >> From erik.osterlund at oracle.com Tue Jul 25 11:13:29 2017 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 25 Jul 2017 13:13:29 +0200 Subject: RFR (S) 8182397: Race in field updates when creating ArrayKlasses can lead to crash In-Reply-To: References: <807A38E7-692E-4849-9258-75CFCFDA404A@oracle.com> <0eb805af-7ac5-874f-eaa4-33ce1788d427@redhat.com> <5977015B.2080102@oracle.com> Message-ID: <597727D9.4070502@oracle.com> On 2017-07-25 11:12, Andrew Haley wrote: > On 25/07/17 09:29, Erik ?sterlund wrote: >> On 2017-07-24 19:40, Andrew Haley wrote: >>> On 24/07/17 17:44, Erik Osterlund wrote: >>>> Sorry for jumping in to the conversation a bit late. >>>> Since David Holmes is on vacation, I have to take his place in >>>> questioning the elision of acquire on the reader side being >>>> okay. Because I know he would if he was here. >>>> >>>> In hotspot we almost *never* elide the acquire on the reader side, >>>> relying on dependent loads. >>> I'm sure we do in plenty of places, it's just undocumented or maybe >>> even unknown to the original author. >> That should be purged from the code. With fire. > Possibly, but let us not burn ourselves in the process. > >>> We certainly do this a lot in the code that the JIT compilers >>> generate, as you note below. >> That is more okay when we are in control of the code generation. >> Consume-like semantics have the following levels of support: >> >> 1) No required support: Machines with strong enough memory ordering to >> guarantee all normal loads and stores have acquire/release semantics >> require no support. >> >> 2) Compiler support: Some machines can rely on dependent loads, but what >> looks like a dependent load in the C++ source code might not translate >> to dependent loads in the compiled machine code. In order to enforce >> that, the data dependency chains from the source code must be retained >> in the generated machine code. > Yes, yes, we know. :-) > >> As for the JIT-compiled code - it is fine to use consume-like semantics >> if we control the code generation and know that the generated machine >> code will respect the data dependencies that we wanted it to. But since >> we do not control the C++ code generation, and do not have access to >> tools like C++11 consume semantics, we can not just elide acquire and >> hope for the best. Instead we have to use acquire until OrderAccess >> starts supporting consume (if it ever will). > OrderAccess could support consume tomorrow: that's up to us. We could indeed. But then we got to do that first, and replace load_acquire with load_consume, rather than with normal loads. >> What we perhaps could do is to add shared code to OrderAccess for >> load_consume, implement them as load_acquire, so that we at least >> remember where in the code we could have gotten away with dependent >> loads and exploit it, and later on plug it into C++11 consume so that >> C++ compilers that support such contracts can utilize it on a platform >> by platform basis. > I suspect we could make it work immediately, because we know how to > emit compiler fences on all platforms (do we?) and almost all of them > respect data dependencies in hardware anyway. Those that don't will > have to use acquire. I think how we would implement consume warrants a new RFE and discussion. I suspect this thread could become long off-topic discussion otherwise. I am happy for now if we just agree that we got to have some kind of consume implementation before we start removing acquire and replacing it with that today non-existent load_consume. Having said that... I do not know if I agree compiler fences is not enough to ensure that data dependencies in the generated machine code is preserved from the source code. For example, take this example pseudo code for performing what I refer to as a stable load between two fields modified concurrently with potential ABA issues: loop { x_start = load_relaxed(field_A) y = load_consume(field_B) x = load_consume(field_A) if (x_start == x) break; } // use x->foo In this example, the loop finishes when the loaded value from field_A remains stable across a load of field_B. But it could have changed in-between. But the choice of register for x used in x->foo after the loop could be either the one corresponding to the x_start or x variables in the pseudo code due to the arithmetic equality. One choice will retain the intended data dependency to the load_consume(field_A), and the other choice will not. If the choice of base register in x->foo is equal to x_start (which seems perfectly valid), then the data dependency is to a possibly stale load that survived an ABA check but might have pointed to a different object that was concurrently recycled, whereas if the base register is the same as x in the loop, then the data dependency correctly corresponds to the load that had consume semantics, and is not a stale value. As a result, the generated machine code does not necessarily retain the data dependencies from the source code if we only used compiler barriers. I think we need C++11 consume specifically (or acquire) to be safe. >>> I don't think it's that much of a big deal, really. Putting in an >>> acquire fence every time we read a java mirror strikes me as madness >>> when consume is all we need on any target. Even Power PC doesn't need >>> a fence instruction there. At least make sure that targets can turn >>> that off: we should not pay the price for something we don't need. >> Conversely, those targets would pay the price of compromised >> correctness - a price higher than performance regressions. >> Performance can only be optimized within the legal bounds of >> correctness. > Please, spare us the lecture. No-one was sugesting compromising > correctness. Okay. It seemed to me like treating load_consume as just a normal load would compromise correctness (due to lack of guarantees the dependency chain is retained in generated code) and violate current hotspot practices. And it seemed to me like this is precisely what was suggested. I am sorry if I misunderstood something here... Thanks, /Erik > >> Therefore we can only elide fences if we have well defined memory >> ordering semantics for doing that and somehow control that the >> generated code adheres to those semantics and respects them. We >> simply do not have consume semantics today, and therefore to ensure >> correctness, we must use acquire, until the day comes when we >> support acquire. > I guess you meant "consume" here? Anyway, all we need is a compiler > fence. > From aph at redhat.com Tue Jul 25 12:42:23 2017 From: aph at redhat.com (Andrew Haley) Date: Tue, 25 Jul 2017 13:42:23 +0100 Subject: RFR (S) 8182397: Race in field updates when creating ArrayKlasses can lead to crash In-Reply-To: <597727D9.4070502@oracle.com> References: <807A38E7-692E-4849-9258-75CFCFDA404A@oracle.com> <0eb805af-7ac5-874f-eaa4-33ce1788d427@redhat.com> <5977015B.2080102@oracle.com> <597727D9.4070502@oracle.com> Message-ID: <1043bdc9-53c5-f893-c2e2-338df7b343d8@redhat.com> On 25/07/17 12:13, Erik ?sterlund wrote: > For example, take this example pseudo code for performing what I refer > to as a stable load between two fields modified concurrently with > potential ABA issues: > > loop { > x_start = load_relaxed(field_A) > y = load_consume(field_B) > x = load_consume(field_A) > if (x_start == x) break; > } > > // use x->foo I don't understand this pseudocode. What is the base address for field_A and field_B ? -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From dl at cs.oswego.edu Tue Jul 25 13:10:12 2017 From: dl at cs.oswego.edu (Doug Lea) Date: Tue, 25 Jul 2017 09:10:12 -0400 Subject: RFR (S) 8182397: Race in field updates when creating ArrayKlasses can lead to crash In-Reply-To: <597727D9.4070502@oracle.com> References: <807A38E7-692E-4849-9258-75CFCFDA404A@oracle.com> <0eb805af-7ac5-874f-eaa4-33ce1788d427@redhat.com> <5977015B.2080102@oracle.com> <597727D9.4070502@oracle.com> Message-ID: On 07/25/2017 07:13 AM, Erik ?sterlund wrote: >> OrderAccess could support consume tomorrow: that's up to us. > > We could indeed. But then we got to do that first, and replace > load_acquire with load_consume, rather than with normal loads. A full spec of C++ consume mode is challenging (and still in progress). But it is possible to take the easier path underlying JDK9 modes, that (implicitly) deal only with pointer dereferences: As in: T* p = x; V v = p.field; where x is some shared variable. If a reader cannot have a valid value for the field of an object (or struct etc) without knowing its address, then it cannot reorder these statements. The usual reason is that the reader cannot have ever seen that address before, at least not since some other fence at least as strong as acquire. (Another way to say this is that neither compilers not processors may act as if they speculate addresses that they have never seen before. If this were not true, then secure computing would be just about impossible.) Correctness is usually easy to check in particular cases, but nearly impossible in the general case (you may need global information-flow analysis). This is close to the linux "ACCESS_ONCE" approach, and is also the idea underlying Java final fields. See http://gee.cs.oswego.edu/dl/html/j9mm.html Using C++-atomic-relaxed/Java-opaque can be used to express this, but in the vastly most common case where all subsequent reads will return the same value (at least up until some other fenced event), just plain loads suffice. Are there any other cases of interest where full consume mode is needed/desired? -Doug From erik.osterlund at oracle.com Tue Jul 25 13:41:59 2017 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 25 Jul 2017 15:41:59 +0200 Subject: RFR (S) 8182397: Race in field updates when creating ArrayKlasses can lead to crash In-Reply-To: <1043bdc9-53c5-f893-c2e2-338df7b343d8@redhat.com> References: <807A38E7-692E-4849-9258-75CFCFDA404A@oracle.com> <0eb805af-7ac5-874f-eaa4-33ce1788d427@redhat.com> <5977015B.2080102@oracle.com> <597727D9.4070502@oracle.com> <1043bdc9-53c5-f893-c2e2-338df7b343d8@redhat.com> Message-ID: <59774AA7.8080006@oracle.com> On 2017-07-25 14:42, Andrew Haley wrote: > On 25/07/17 12:13, Erik ?sterlund wrote: >> For example, take this example pseudo code for performing what I refer >> to as a stable load between two fields modified concurrently with >> potential ABA issues: >> >> loop { >> x_start = load_relaxed(field_A) >> y = load_consume(field_B) >> x = load_consume(field_A) >> if (x_start == x) break; >> } >> >> // use x->foo > I don't understand this pseudocode. What is the base address for field_A > and field_B ? > field_A and field_B could be two different registers pointing at different addresses - i.e. they are arbitrary pointers. The key in this example is that field_A is reloaded, and then we compare if the reloaded value is equal to the original value (with a possible ABA problem), and stop the loop then. But the original and reloaded value could reside in different registers, and when we continue using x->foo afterwards, the compiler could elect to use either one of the two registers as base pointers in the dereference - either the one from the reloaded value of field_A or for the original value, as they are equal to each other. Normally that is totally fine, but had the compiler known that the reload had consume semantics, it would have known that it definitely has to use that register that the reload went into when dereferencing x->foo after the loop, despite the fact that it is arithmetically equal to the other register. Hope that makes more sense. Thanks, /Erik From adinn at redhat.com Tue Jul 25 14:42:13 2017 From: adinn at redhat.com (Andrew Dinn) Date: Tue, 25 Jul 2017 15:42:13 +0100 Subject: RFR (S) 8182397: Race in field updates when creating ArrayKlasses can lead to crash In-Reply-To: References: <807A38E7-692E-4849-9258-75CFCFDA404A@oracle.com> <0eb805af-7ac5-874f-eaa4-33ce1788d427@redhat.com> <5977015B.2080102@oracle.com> <597727D9.4070502@oracle.com> Message-ID: <3a754d65-3ea4-af4b-435e-782e6fc3be8b@redhat.com> On 25/07/17 14:10, Doug Lea wrote: > A full spec of C++ consume mode is challenging (and still in progress). > But it is possible to take the easier path underlying JDK9 modes, > that (implicitly) deal only with pointer dereferences: > As in: T* p = x; V v = p.field; where x is some shared variable. > > If a reader cannot have a valid value for the field of an > object (or struct etc) without knowing its address, then it > cannot reorder these statements. The usual reason is that the > reader cannot have ever seen that address before, at least not since > some other fence at least as strong as acquire. (Another way > to say this is that neither compilers not processors may act as > if they speculate addresses that they have never seen before. > If this were not true, then secure computing would be just about > impossible.) Correctness is usually easy to check in particular cases, > but nearly impossible in the general case (you may need global > information-flow analysis). > > This is close to the linux "ACCESS_ONCE" approach, and is > also the idea underlying Java final fields. See > http://gee.cs.oswego.edu/dl/html/j9mm.html > > Using C++-atomic-relaxed/Java-opaque can be used to express this, > but in the vastly most common case where all subsequent reads will > return the same value (at least up until some other fenced event), > just plain loads suffice. Thanks, Doug, for clarifying what is at stake in this specific case vs the more general problem that might be faced. > Are there any other cases of interest where full consume mode > is needed/desired? I cannot answer your last question. However, it seems clear to me that the point in question in Coleen's proposed fix falls under the specific category of problem you describe above i.e. the two addresses whose visibility is ordered by the release barrier are such that the second cannot be accessed without loading and dereferencing the first. I can perhaps imagine a processor which fails to respect that sort of dependency existing on a Research Lab blackboard but I doubt we will ever see the like embodied in silicon. So, I don't think we need to fear any consequences if we let this change set go through without a load barrier. I'll defer to other reviewers if they wish to disagree. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From adinn at redhat.com Tue Jul 25 14:44:05 2017 From: adinn at redhat.com (Andrew Dinn) Date: Tue, 25 Jul 2017 15:44:05 +0100 Subject: RFR (S) 8182397: Race in field updates when creating ArrayKlasses can lead to crash In-Reply-To: References: <166602DE-F4BA-40A8-93C8-7C1E74679D77@oracle.com> <7bec60a2-d23c-f72c-5bd0-026c158bc3d6@oracle.com> <002E88AA-1462-436B-B921-531908E6D30C@oracle.com> Message-ID: On 24/07/17 22:35, coleen.phillimore at oracle.com wrote: > Thank you, Kim. If we want to dicuss this further, we can open a bug > against hotspot/compiler to add the load aquires. > thanks, New change set is also ok by me. If an acquire is needed on the compiler side then we should definitely take that up in a separate bug/fix. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From coleen.phillimore at oracle.com Tue Jul 25 14:58:59 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 25 Jul 2017 10:58:59 -0400 Subject: RFR (S) 8182397: Race in field updates when creating ArrayKlasses can lead to crash In-Reply-To: References: <166602DE-F4BA-40A8-93C8-7C1E74679D77@oracle.com> <7bec60a2-d23c-f72c-5bd0-026c158bc3d6@oracle.com> <002E88AA-1462-436B-B921-531908E6D30C@oracle.com> Message-ID: <39f38ac9-52c4-215f-92e2-de0f113cfdce@oracle.com> On 7/25/17 10:44 AM, Andrew Dinn wrote: > On 24/07/17 22:35, coleen.phillimore at oracle.com wrote: >> Thank you, Kim. If we want to dicuss this further, we can open a bug >> against hotspot/compiler to add the load aquires. >> thanks, > New change set is also ok by me. > > If an acquire is needed on the compiler side then we should definitely > take that up in a separate bug/fix. Thank you, Andrew. Coleen > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From alexander.harlap at oracle.com Tue Jul 25 15:32:44 2017 From: alexander.harlap at oracle.com (Alexander Harlap) Date: Tue, 25 Jul 2017 11:32:44 -0400 Subject: Request for review JDK-8183973 - gc/TestFullGCALot.java fails in JDK10-hs nightly Message-ID: Please review change for JDK-8183973 - gc/TestFullGCALot.java fails in JDK10-hs nightly Proposed change is located at http://cr.openjdk.java.net/~aharlap/8183973/webrev.00/ Change is to avoid failure failure due to insufficient NewSize while running test with UseDeterministicG1GC option. Changed is suggested by Erik Helin. I reproduced failure on machine where nightly failure happened. Suggested change eliminated failure. Thank you, Alex From erik.osterlund at oracle.com Tue Jul 25 15:36:39 2017 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 25 Jul 2017 17:36:39 +0200 Subject: RFR (S) 8182397: Race in field updates when creating ArrayKlasses can lead to crash In-Reply-To: References: <807A38E7-692E-4849-9258-75CFCFDA404A@oracle.com> <0eb805af-7ac5-874f-eaa4-33ce1788d427@redhat.com> Message-ID: <59776587.7020903@oracle.com> Hi Coleen, I agree that if this is accessed purely in JIT compiled code that we control, and the value is stable, i.e. goes from NULL to not NULL, and then never changes, then this seems safe without acquire in this specific case. Having said that, the corresponding java_lang_Class::array_klass() function loads the klass with a non-volatile load, rather than a load_acquire. Now, it seems like the only use of ths function is in InstanceKlass::oop_print_on(), so I guess it is not crucial performance wise. It would be great if the java_lang_Class::array_klass() was safe to use in our code base so that new users that accidentally use it for more than printing will not be surprised by this piece of code and how it inconsistent to other hotspot code violates our synchronization practices of linking release_store with load_acquire, in favor of a non-volatile load. Would you agree it would be beneficial to use load_acquire at least in this not so hot path in the C++ code? Then we have indisputably correct synchronization in the C++ code, can dodge the whole consume discussion (we do what every other acquire/release pair does in hotspot - business as usual), and leave the optimized case to our own JIT compilers that we trust retain the data dependency. I would be happy with that. Thanks, /Erik On 2017-07-24 21:39, coleen.phillimore at oracle.com wrote: > > Thank you for your comments, Erik and Andrew. I like the idea of > rules for innocent coders (ones who aren't versed in the complexities > of architecture ordering semantics like myself), that say things like > "if you have a store-release, you should have a load-acquire on the > other side". I would follow this if the load were in the C++ code. > Since it's in the JIT compiled code, I think taking advantage of the > knowledge that this is a dependent load (if I understand this > correctly) seems acceptable and recommended because performance counts. > > Thanks, > Coleen > > On 7/24/17 1:40 PM, Andrew Haley wrote: >> On 24/07/17 17:44, Erik Osterlund wrote: >>> Sorry for jumping in to the conversation a bit late. >>> Since David Holmes is on vacation, I have to take his place in >>> questioning the elision of acquire on the reader side being >>> okay. Because I know he would if he was here. >>> >>> In hotspot we almost *never* elide the acquire on the reader side, >>> relying on dependent loads. >> I'm sure we do in plenty of places, it's just undocumented or maybe >> even unknown to the original author. We certainly do this a lot in >> the code that the JIT compilers generate, as you note below. >> >>> This would be an exception to that rule, and I do not see why this >>> is warranted. In order to utilize the dependent load properties in >>> shared code, we need memory ordering semantics for this. That memory >>> ordering semantics is called consume in C++11, and has arguably >>> turned out to be a terrible idea. Hotspot does not have a consume >>> memory ordering like C++11, that allows utilizing dependent loads, >>> and I do not think we intend to introduce it any time soon unless >>> there are very compelling reasons to do so. In its absence, we >>> should use load_acquire instead, and not micro optimize it away. >>> >>> AFAIK, compiler implementers are not happy with C++11 consume memory >>> ordering semantics and implement it with acquire instead as there >>> are too many problems with consume. >> I strongly disagree. Consume is fine now: I think you're dealing with >> out-of-date information. It took a while to figure out how to specify >> it, that's all. >> >>> To elide acquire in an acquire release pair in shared code, very >>> strong motivation is needed. Like for example how constructors >>> publish final object references with release but relies on dependent >>> load in JIT compiled code to elide acquire, because using acquire >>> semantics on all reference loads is too expensive, so it is >>> warranted here. >> Right. So we already depend on that property, so... >> >>> Therefore, I am wondering if perhaps we want to add an acquire on >>> the load side anyway, so we do not have to introduce consume memory >>> ordering semantics or something similar in the shared code >>> base. Using a normal load without any memory ordering semantics in >>> shared code to act like a load consume rings alarm clocks in my >>> head. >> I don't think it's that much of a big deal, really. Putting in an >> acquire fence every time we read a java mirror strikes me as madness >> when consume is all we need on any target. Even Power PC doesn't need >> a fence instruction there. At least make sure that targets can turn >> that off: we should not pay the price for something we don't need. >> > From mikael.gerdin at oracle.com Tue Jul 25 15:54:58 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Tue, 25 Jul 2017 17:54:58 +0200 Subject: Request for review JDK-8183973 - gc/TestFullGCALot.java fails in JDK10-hs nightly In-Reply-To: References: Message-ID: Hi Alex, On 2017-07-25 17:32, Alexander Harlap wrote: > Please review change for JDK-8183973 > - > gc/TestFullGCALot.java fails in JDK10-hs nightly > > Proposed change is located at > http://cr.openjdk.java.net/~aharlap/8183973/webrev.00/ Fix looks good to me. /Mikael > > Change is to avoid failure failure due to insufficient NewSize while > running test with UseDeterministicG1GC option. > > Changed is suggested by Erik Helin. > > I reproduced failure on machine where nightly failure happened. > Suggested change eliminated failure. > > Thank you, > > Alex > From erik.osterlund at oracle.com Tue Jul 25 16:21:26 2017 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Tue, 25 Jul 2017 18:21:26 +0200 Subject: Request for review JDK-8183973 - gc/TestFullGCALot.java fails in JDK10-hs nightly In-Reply-To: References: Message-ID: Hi Alex, Looks good. Thanks, /Erik > On 25 Jul 2017, at 17:32, Alexander Harlap wrote: > > Please review change for JDK-8183973 - gc/TestFullGCALot.java fails in JDK10-hs nightly > > Proposed change is located at http://cr.openjdk.java.net/~aharlap/8183973/webrev.00/ > > Change is to avoid failure failure due to insufficient NewSize while running test with UseDeterministicG1GC option. > > Changed is suggested by Erik Helin. > > I reproduced failure on machine where nightly failure happened. Suggested change eliminated failure. > > Thank you, > > Alex > From alexander.harlap at oracle.com Tue Jul 25 16:30:56 2017 From: alexander.harlap at oracle.com (Alexander Harlap) Date: Tue, 25 Jul 2017 12:30:56 -0400 Subject: Request for review JDK-8183973 - gc/TestFullGCALot.java fails in JDK10-hs nightly In-Reply-To: References: Message-ID: <0b1fba3e-fd55-574e-2610-bff632a43854@oracle.com> Thank you for review! Alex On 7/25/2017 12:21 PM, Erik Osterlund wrote: > Hi Alex, > > Looks good. > > Thanks, > /Erik > >> On 25 Jul 2017, at 17:32, Alexander Harlap wrote: >> >> Please review change for JDK-8183973 - gc/TestFullGCALot.java fails in JDK10-hs nightly >> >> Proposed change is located at http://cr.openjdk.java.net/~aharlap/8183973/webrev.00/ >> >> Change is to avoid failure failure due to insufficient NewSize while running test with UseDeterministicG1GC option. >> >> Changed is suggested by Erik Helin. >> >> I reproduced failure on machine where nightly failure happened. Suggested change eliminated failure. >> >> Thank you, >> >> Alex >> From aph at redhat.com Tue Jul 25 16:47:49 2017 From: aph at redhat.com (Andrew Haley) Date: Tue, 25 Jul 2017 17:47:49 +0100 Subject: RFR (S) 8182397: Race in field updates when creating ArrayKlasses can lead to crash In-Reply-To: <59776587.7020903@oracle.com> References: <807A38E7-692E-4849-9258-75CFCFDA404A@oracle.com> <0eb805af-7ac5-874f-eaa4-33ce1788d427@redhat.com> <59776587.7020903@oracle.com> Message-ID: <623c1d8e-06e0-2d90-0a69-be9b940287d0@redhat.com> On 25/07/17 16:36, Erik ?sterlund wrote: > Would you agree it would be beneficial to use load_acquire at least in > this not so hot path in the C++ code? Then we have indisputably correct > synchronization in the C++ code, can dodge the whole consume discussion > (we do what every other acquire/release pair does in hotspot - business > as usual), and leave the optimized case to our own JIT compilers that we > trust retain the data dependency. I would be happy with that. I would too. If acquire gets used in places where a data dependency could be used at some point in the future, we can have this argu^H^H^H^H discussion again. :-) -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Tue Jul 25 16:57:28 2017 From: aph at redhat.com (Andrew Haley) Date: Tue, 25 Jul 2017 17:57:28 +0100 Subject: RFR (S) 8182397: Race in field updates when creating ArrayKlasses can lead to crash In-Reply-To: <59774AA7.8080006@oracle.com> References: <807A38E7-692E-4849-9258-75CFCFDA404A@oracle.com> <0eb805af-7ac5-874f-eaa4-33ce1788d427@redhat.com> <5977015B.2080102@oracle.com> <597727D9.4070502@oracle.com> <1043bdc9-53c5-f893-c2e2-338df7b343d8@redhat.com> <59774AA7.8080006@oracle.com> Message-ID: <61341c53-c9e6-fdc1-bfe3-58ddf34ee9fc@redhat.com> On 25/07/17 14:41, Erik ?sterlund wrote: > > On 2017-07-25 14:42, Andrew Haley wrote: >> On 25/07/17 12:13, Erik ?sterlund wrote: >>> For example, take this example pseudo code for performing what I refer >>> to as a stable load between two fields modified concurrently with >>> potential ABA issues: >>> >>> loop { >>> x_start = load_relaxed(field_A) >>> y = load_consume(field_B) >>> x = load_consume(field_A) >>> if (x_start == x) break; >>> } >>> >>> // use x->foo >> I don't understand this pseudocode. What is the base address for field_A >> and field_B ? > > field_A and field_B could be two different registers pointing at > different addresses - i.e. they are arbitrary pointers. The key in this > example is that field_A is reloaded, and then we compare if the reloaded > value is equal to the original value (with a possible ABA problem), and > stop the loop then. But the original and reloaded value could reside in > different registers, and when we continue using x->foo afterwards, the > compiler could elect to use either one of the two registers as base > pointers in the dereference - either the one from the reloaded value of > field_A or for the original value, as they are equal to each other. OK, I see what you're getting at. Compilers have to be pretty smart to make consume work properly. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From thomas.stuefe at gmail.com Tue Jul 25 17:07:45 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 25 Jul 2017 19:07:45 +0200 Subject: RFR(xxs): 8185262: Keep default logging for Metaspace OOM short and concise Message-ID: Hi all, may I have please reviews for this very tiny change: bug: https://bugs.openjdk.java.net/browse/JDK-8185262 webrev: http://cr.openjdk.java.net/~stuefe/webrevs/ 8185262-keep-metaspace-oom-log-short-for-default/webrev.00/webrev/ On Metaspace OOM, we dump all metaspace chunks in use. This can be very lengthy. It would be nice if this part of logging were restricted to a higher loglevel. In this change, I restrict it to debug. Kind Regards, Thomas From ioi.lam at oracle.com Tue Jul 25 17:12:35 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 25 Jul 2017 10:12:35 -0700 Subject: RFR(xxs): 8185262: Keep default logging for Metaspace OOM short and concise In-Reply-To: References: Message-ID: Hi Thomas, how about switching lines 3739 and 3740? 3738 if (log.is_debug()) { 3739 LogStream ls(log.debug()); 3740 if (loader_data->metaspace_or_null() != NULL) { 3741 loader_data->dump(&ls); 3742 } 3743 } What kind of tests have you run? Thanks - Ioi On 7/25/17 10:07 AM, Thomas St?fe wrote: > Hi all, > > may I have please reviews for this very tiny change: > > bug: https://bugs.openjdk.java.net/browse/JDK-8185262 > webrev: http://cr.openjdk.java.net/~stuefe/webrevs/ > 8185262-keep-metaspace-oom-log-short-for-default/webrev.00/webrev/ > > On Metaspace OOM, we dump all metaspace chunks in use. This can be very > lengthy. It would be nice if this part of logging were restricted to a > higher loglevel. In this change, I restrict it to debug. > > Kind Regards, Thomas From thomas.stuefe at gmail.com Tue Jul 25 17:13:40 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 25 Jul 2017 19:13:40 +0200 Subject: RFR(xs): 8185263: Fix zero build after 8169881, 8175318, 8178350 Message-ID: Hi all, may I please have reviews and sponsor for these zero-related build fixes? Issue: https://bugs.openjdk.java.net/browse/JDK-8185263 webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8185263-fix-zero-build/webrev.00/webrev/index.html Thanks! Thomas From coleen.phillimore at oracle.com Tue Jul 25 17:18:45 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 25 Jul 2017 13:18:45 -0400 Subject: RFR (S) 8182397: Race in field updates when creating ArrayKlasses can lead to crash In-Reply-To: <59776587.7020903@oracle.com> References: <807A38E7-692E-4849-9258-75CFCFDA404A@oracle.com> <0eb805af-7ac5-874f-eaa4-33ce1788d427@redhat.com> <59776587.7020903@oracle.com> Message-ID: On 7/25/17 11:36 AM, Erik ?sterlund wrote: > Hi Coleen, > > I agree that if this is accessed purely in JIT compiled code that we > control, and the value is stable, i.e. goes from NULL to not NULL, and > then never changes, then this seems safe without acquire in this > specific case. > Having said that, the corresponding java_lang_Class::array_klass() > function loads the klass with a non-volatile load, rather than a > load_acquire. Now, it seems like the only use of ths function is in > InstanceKlass::oop_print_on(), so I guess it is not crucial > performance wise. It would be great if the > java_lang_Class::array_klass() was safe to use in our code base so > that new users that accidentally use it for more than printing will > not be surprised by this piece of code and how it inconsistent to > other hotspot code violates our synchronization practices of linking > release_store with load_acquire, in favor of a non-volatile load. > > Would you agree it would be beneficial to use load_acquire at least in > this not so hot path in the C++ code? Then we have indisputably > correct synchronization in the C++ code, can dodge the whole consume > discussion (we do what every other acquire/release pair does in > hotspot - business as usual), and leave the optimized case to our own > JIT compilers that we trust retain the data dependency. I would be > happy with that. Yes, I checked this in but I should have done this. I mistakenly thought the only accesses for this field were through compiled code and thought there wasn't a java_lang_Class::array_klass() accessor. But there is but it's only used for printing. I'll file a bug to make the array_klass() use load acquire for cleanliness and send it out in a bit. Sorry to jump the gun on pushing. thanks, Coleen > > Thanks, > /Erik > > On 2017-07-24 21:39, coleen.phillimore at oracle.com wrote: >> >> Thank you for your comments, Erik and Andrew. I like the idea of >> rules for innocent coders (ones who aren't versed in the complexities >> of architecture ordering semantics like myself), that say things like >> "if you have a store-release, you should have a load-acquire on the >> other side". I would follow this if the load were in the C++ code. >> Since it's in the JIT compiled code, I think taking advantage of the >> knowledge that this is a dependent load (if I understand this >> correctly) seems acceptable and recommended because performance counts. >> >> Thanks, >> Coleen >> >> On 7/24/17 1:40 PM, Andrew Haley wrote: >>> On 24/07/17 17:44, Erik Osterlund wrote: >>>> Sorry for jumping in to the conversation a bit late. >>>> Since David Holmes is on vacation, I have to take his place in >>>> questioning the elision of acquire on the reader side being >>>> okay. Because I know he would if he was here. >>>> >>>> In hotspot we almost *never* elide the acquire on the reader side, >>>> relying on dependent loads. >>> I'm sure we do in plenty of places, it's just undocumented or maybe >>> even unknown to the original author. We certainly do this a lot in >>> the code that the JIT compilers generate, as you note below. >>> >>>> This would be an exception to that rule, and I do not see why this >>>> is warranted. In order to utilize the dependent load properties in >>>> shared code, we need memory ordering semantics for this. That memory >>>> ordering semantics is called consume in C++11, and has arguably >>>> turned out to be a terrible idea. Hotspot does not have a consume >>>> memory ordering like C++11, that allows utilizing dependent loads, >>>> and I do not think we intend to introduce it any time soon unless >>>> there are very compelling reasons to do so. In its absence, we >>>> should use load_acquire instead, and not micro optimize it away. >>>> >>>> AFAIK, compiler implementers are not happy with C++11 consume memory >>>> ordering semantics and implement it with acquire instead as there >>>> are too many problems with consume. >>> I strongly disagree. Consume is fine now: I think you're dealing with >>> out-of-date information. It took a while to figure out how to specify >>> it, that's all. >>> >>>> To elide acquire in an acquire release pair in shared code, very >>>> strong motivation is needed. Like for example how constructors >>>> publish final object references with release but relies on dependent >>>> load in JIT compiled code to elide acquire, because using acquire >>>> semantics on all reference loads is too expensive, so it is >>>> warranted here. >>> Right. So we already depend on that property, so... >>> >>>> Therefore, I am wondering if perhaps we want to add an acquire on >>>> the load side anyway, so we do not have to introduce consume memory >>>> ordering semantics or something similar in the shared code >>>> base. Using a normal load without any memory ordering semantics in >>>> shared code to act like a load consume rings alarm clocks in my >>>> head. >>> I don't think it's that much of a big deal, really. Putting in an >>> acquire fence every time we read a java mirror strikes me as madness >>> when consume is all we need on any target. Even Power PC doesn't need >>> a fence instruction there. At least make sure that targets can turn >>> that off: we should not pay the price for something we don't need. >>> >> > From thomas.stuefe at gmail.com Tue Jul 25 17:25:25 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 25 Jul 2017 19:25:25 +0200 Subject: RFR(xxs): 8185262: Keep default logging for Metaspace OOM short and concise In-Reply-To: References: Message-ID: Hi Ioi, On Tue, Jul 25, 2017 at 7:12 PM, Ioi Lam wrote: > Hi Thomas, > > how about switching lines 3739 and 3740? > > 3738 if (log.is_debug()) { > 3739 LogStream ls(log.debug()); > 3740 if (loader_data->metaspace_or_null() != NULL) { > 3741 loader_data->dump(&ls); > 3742 } > 3743 } > > sure. > What kind of tests have you run? > > Besides my tiny test case causing metaspace oom? gtests on Windows x64, Linux x64/ppc and AIX, and, on Linux x64, jtreg tests from hotspot/runtime/Metaspace. All with fastdebug builds. I also looked around whether any test parses the log output, but could not find any code setting gc+metaspace+freelist. Thanks, Thomas Thanks > - Ioi > > > On 7/25/17 10:07 AM, Thomas St?fe wrote: > >> Hi all, >> >> may I have please reviews for this very tiny change: >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8185262 >> webrev: http://cr.openjdk.java.net/~stuefe/webrevs/ >> 8185262-keep-metaspace-oom-log-short-for-default/webrev.00/webrev/ >> >> On Metaspace OOM, we dump all metaspace chunks in use. This can be very >> lengthy. It would be nice if this part of logging were restricted to a >> higher loglevel. In this change, I restrict it to debug. >> >> Kind Regards, Thomas >> > > From ioi.lam at oracle.com Tue Jul 25 17:29:09 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 25 Jul 2017 10:29:09 -0700 Subject: RFR(xxs): 8185262: Keep default logging for Metaspace OOM short and concise In-Reply-To: References: Message-ID: <3916fc10-38fd-a121-98c7-43dd12078453@oracle.com> Sounds good. Thanks Thomas. - Ioi On 7/25/17 10:25 AM, Thomas St?fe wrote: > Hi Ioi, > > On Tue, Jul 25, 2017 at 7:12 PM, Ioi Lam > wrote: > > Hi Thomas, > > how about switching lines 3739 and 3740? > > 3738 if (log.is_debug()) { > 3739 LogStream ls(log.debug()); > 3740 if (loader_data->metaspace_or_null() != NULL) { > 3741 loader_data->dump(&ls); > 3742 } > 3743 } > > > sure. > > What kind of tests have you run? > > > Besides my tiny test case causing metaspace oom? gtests on Windows > x64, Linux x64/ppc and AIX, and, on Linux x64, jtreg tests from > hotspot/runtime/Metaspace. All with fastdebug builds. > > I also looked around whether any test parses the log output, but could > not find any code setting gc+metaspace+freelist. > > Thanks, Thomas > > Thanks > - Ioi > > > On 7/25/17 10:07 AM, Thomas St?fe wrote: > > Hi all, > > may I have please reviews for this very tiny change: > > bug: https://bugs.openjdk.java.net/browse/JDK-8185262 > > webrev: http://cr.openjdk.java.net/~stuefe/webrevs/ > > 8185262-keep-metaspace-oom-log-short-for-default/webrev.00/webrev/ > > On Metaspace OOM, we dump all metaspace chunks in use. This > can be very > lengthy. It would be nice if this part of logging were > restricted to a > higher loglevel. In this change, I restrict it to debug. > > Kind Regards, Thomas > > > From ioi.lam at oracle.com Tue Jul 25 17:31:28 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 25 Jul 2017 10:31:28 -0700 Subject: RFR(xs): 8185263: Fix zero build after 8169881, 8175318, 8178350 In-Reply-To: References: Message-ID: Hi Thomas, Looks good. I can sponsor it. Thanks - Ioi On 7/25/17 10:13 AM, Thomas St?fe wrote: > Hi all, > > may I please have reviews and sponsor for these zero-related build fixes? > > Issue: https://bugs.openjdk.java.net/browse/JDK-8185263 > webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/8185263-fix-zero-build/webrev.00/webrev/index.html > > Thanks! Thomas From coleen.phillimore at oracle.com Tue Jul 25 17:51:29 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 25 Jul 2017 13:51:29 -0400 Subject: RFR (S) 8185296: java_lang_Class::array_klass should be array_klass_acquire Message-ID: <985f60b9-8481-715e-2b31-f0007968f962@oracle.com> For consistency with https://bugs.openjdk.java.net/browse/JDK-8185296 should have made array_klass use load_acquire. open webrev at http://cr.openjdk.java.net/~coleenp/8185296.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8185296 Tested with inserted (now removed) print for the component mirror. Eg: printing mirror for component mirror: java.lang.Class {0x00000000f3a25be8} - klass: 'java/lang/Class' - ---- fields (total size 15 words): - private volatile transient strict 'cachedConstructor' 'Ljava/lang/reflect/Constructor;' @12 NULL (0 0) - private volatile transient strict 'newInstanceCallerCache' 'Ljava/lang/Class;' @16 NULL (0 f3a26948) - private transient 'name' 'Ljava/lang/String;' @20 "jdk.tools.jlink.internal.ImageLocationWriter"{0x00000000f3a26948} (f3a26948 eab3f768) - private transient 'module' 'Ljava/lang/Module;' @24 a 'java/lang/Module'{0x00000000eab3f768} (eab3f768 eaaa0000) - private final 'classLoader' 'Ljava/lang/ClassLoader;' @28 a 'jdk/internal/loader/ClassLoaders$AppClassLoader'{0x00000000eaaa0000} (eaaa0000 eab4d438) - private transient 'packageName' 'Ljava/lang/String;' @32 "jdk.tools.jlink.internal"{0x00000000eab4d438} (eab4d438 0) - private final strict 'componentType' 'Ljava/lang/Class;' @36 NULL (0 0) - private volatile transient strict 'reflectionData' 'Ljava/lang/ref/SoftReference;' @40 NULL (0 0) - private volatile transient 'genericInfo' 'Lsun/reflect/generics/repository/ClassRepository;' @44 NULL (0 0) - private volatile transient strict 'enumConstants' '[Ljava/lang/Object;' @48 NULL (0 0) - private volatile transient strict 'enumConstantDirectory' 'Ljava/util/Map;' @52 NULL (0 0) - private volatile transient 'annotationData' 'Ljava/lang/Class$AnnotationData;' @56 NULL (0 0) - private volatile transient 'annotationType' 'Lsun/reflect/annotation/AnnotationType;' @60 NULL (0 0) - transient 'classValueMap' 'Ljava/lang/ClassValue$ClassValueMap;' @64 NULL (0 eab49d08) - private volatile transient 'classRedefinedCount' 'I' @96 0 - signature: Ljdk/tools/jlink/internal/ImageLocationWriter; - fake entry for mirror: 'jdk/tools/jlink/internal/ImageLocationWriter' - fake entry for array: 'jdk/tools/jlink/internal/ImageLocationWriter'[] <=== HERE - fake entry for oop_size: 15 - fake entry for static_oop_field_count: 0 - static final synthetic '$assertionsDisabled' 'Z' @112 true Thanks, Coleen From erik.osterlund at oracle.com Tue Jul 25 18:12:44 2017 From: erik.osterlund at oracle.com (=?utf-8?Q?Erik_=C3=96sterlund?=) Date: Tue, 25 Jul 2017 20:12:44 +0200 Subject: RFR (S) 8182397: Race in field updates when creating ArrayKlasses can lead to crash In-Reply-To: <623c1d8e-06e0-2d90-0a69-be9b940287d0@redhat.com> References: <807A38E7-692E-4849-9258-75CFCFDA404A@oracle.com> <0eb805af-7ac5-874f-eaa4-33ce1788d427@redhat.com> <59776587.7020903@oracle.com> <623c1d8e-06e0-2d90-0a69-be9b940287d0@redhat.com> Message-ID: > On 25 Jul 2017, at 18:47, Andrew Haley wrote: > > On 25/07/17 16:36, Erik ?sterlund wrote: >> Would you agree it would be beneficial to use load_acquire at least in >> this not so hot path in the C++ code? Then we have indisputably correct >> synchronization in the C++ code, can dodge the whole consume discussion >> (we do what every other acquire/release pair does in hotspot - business >> as usual), and leave the optimized case to our own JIT compilers that we >> trust retain the data dependency. I would be happy with that. > > I would too. If acquire gets used in places where a data dependency could > be used at some point in the future, we can have this argu^H^H^H^H discussion > again. :-) Yes - I will be happy to have that conversation again in the future. :) Thanks, /Erik > > -- > Andrew Haley > Java Platform Lead Engineer > Red Hat UK Ltd. > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From erik.osterlund at oracle.com Tue Jul 25 18:13:30 2017 From: erik.osterlund at oracle.com (=?utf-8?Q?Erik_=C3=96sterlund?=) Date: Tue, 25 Jul 2017 20:13:30 +0200 Subject: RFR (S) 8182397: Race in field updates when creating ArrayKlasses can lead to crash In-Reply-To: <61341c53-c9e6-fdc1-bfe3-58ddf34ee9fc@redhat.com> References: <807A38E7-692E-4849-9258-75CFCFDA404A@oracle.com> <0eb805af-7ac5-874f-eaa4-33ce1788d427@redhat.com> <5977015B.2080102@oracle.com> <597727D9.4070502@oracle.com> <1043bdc9-53c5-f893-c2e2-338df7b343d8@redhat.com> <59774AA7.8080006@oracle.com> <61341c53-c9e6-fdc1-bfe3-58ddf34ee9fc@redhat.com> Message-ID: <54910434-DB13-4A65-93F2-A2DAE27BB0C7@oracle.com> > On 25 Jul 2017, at 18:57, Andrew Haley wrote: > > On 25/07/17 14:41, Erik ?sterlund wrote: >> >> On 2017-07-25 14:42, Andrew Haley wrote: >>> On 25/07/17 12:13, Erik ?sterlund wrote: >>>> For example, take this example pseudo code for performing what I refer >>>> to as a stable load between two fields modified concurrently with >>>> potential ABA issues: >>>> >>>> loop { >>>> x_start = load_relaxed(field_A) >>>> y = load_consume(field_B) >>>> x = load_consume(field_A) >>>> if (x_start == x) break; >>>> } >>>> >>>> // use x->foo >>> I don't understand this pseudocode. What is the base address for field_A >>> and field_B ? >> >> field_A and field_B could be two different registers pointing at >> different addresses - i.e. they are arbitrary pointers. The key in this >> example is that field_A is reloaded, and then we compare if the reloaded >> value is equal to the original value (with a possible ABA problem), and >> stop the loop then. But the original and reloaded value could reside in >> different registers, and when we continue using x->foo afterwards, the >> compiler could elect to use either one of the two registers as base >> pointers in the dereference - either the one from the reloaded value of >> field_A or for the original value, as they are equal to each other. > > OK, I see what you're getting at. Compilers have to be pretty > smart to make consume work properly. Precisely. /Erik > -- > Andrew Haley > Java Platform Lead Engineer > Red Hat UK Ltd. > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From erik.osterlund at oracle.com Tue Jul 25 18:14:54 2017 From: erik.osterlund at oracle.com (=?utf-8?Q?Erik_=C3=96sterlund?=) Date: Tue, 25 Jul 2017 20:14:54 +0200 Subject: RFR (S) 8182397: Race in field updates when creating ArrayKlasses can lead to crash In-Reply-To: References: <807A38E7-692E-4849-9258-75CFCFDA404A@oracle.com> <0eb805af-7ac5-874f-eaa4-33ce1788d427@redhat.com> <59776587.7020903@oracle.com> Message-ID: > On 25 Jul 2017, at 19:18, coleen.phillimore at oracle.com wrote: > > > > On 7/25/17 11:36 AM, Erik ?sterlund wrote: >> Hi Coleen, >> >> I agree that if this is accessed purely in JIT compiled code that we control, and the value is stable, i.e. goes from NULL to not NULL, and then never changes, then this seems safe without acquire in this specific case. >> Having said that, the corresponding java_lang_Class::array_klass() function loads the klass with a non-volatile load, rather than a load_acquire. Now, it seems like the only use of ths function is in InstanceKlass::oop_print_on(), so I guess it is not crucial performance wise. It would be great if the java_lang_Class::array_klass() was safe to use in our code base so that new users that accidentally use it for more than printing will not be surprised by this piece of code and how it inconsistent to other hotspot code violates our synchronization practices of linking release_store with load_acquire, in favor of a non-volatile load. >> >> Would you agree it would be beneficial to use load_acquire at least in this not so hot path in the C++ code? Then we have indisputably correct synchronization in the C++ code, can dodge the whole consume discussion (we do what every other acquire/release pair does in hotspot - business as usual), and leave the optimized case to our own JIT compilers that we trust retain the data dependency. I would be happy with that. > > Yes, I checked this in but I should have done this. I mistakenly thought the only accesses for this field were through compiled code and thought there wasn't a java_lang_Class::array_klass() accessor. But there is but it's only used for printing. > > I'll file a bug to make the array_klass() use load acquire for cleanliness and send it out in a bit. Sorry to jump the gun on pushing. No problem, as long as we fix it. ;) Thanks, /Erik > thanks, > Coleen > >> >> Thanks, >> /Erik >> >> On 2017-07-24 21:39, coleen.phillimore at oracle.com wrote: >>> >>> Thank you for your comments, Erik and Andrew. I like the idea of rules for innocent coders (ones who aren't versed in the complexities of architecture ordering semantics like myself), that say things like "if you have a store-release, you should have a load-acquire on the other side". I would follow this if the load were in the C++ code. Since it's in the JIT compiled code, I think taking advantage of the knowledge that this is a dependent load (if I understand this correctly) seems acceptable and recommended because performance counts. >>> >>> Thanks, >>> Coleen >>> >>> On 7/24/17 1:40 PM, Andrew Haley wrote: >>>> On 24/07/17 17:44, Erik Osterlund wrote: >>>>> Sorry for jumping in to the conversation a bit late. >>>>> Since David Holmes is on vacation, I have to take his place in >>>>> questioning the elision of acquire on the reader side being >>>>> okay. Because I know he would if he was here. >>>>> >>>>> In hotspot we almost *never* elide the acquire on the reader side, >>>>> relying on dependent loads. >>>> I'm sure we do in plenty of places, it's just undocumented or maybe >>>> even unknown to the original author. We certainly do this a lot in >>>> the code that the JIT compilers generate, as you note below. >>>> >>>>> This would be an exception to that rule, and I do not see why this >>>>> is warranted. In order to utilize the dependent load properties in >>>>> shared code, we need memory ordering semantics for this. That memory >>>>> ordering semantics is called consume in C++11, and has arguably >>>>> turned out to be a terrible idea. Hotspot does not have a consume >>>>> memory ordering like C++11, that allows utilizing dependent loads, >>>>> and I do not think we intend to introduce it any time soon unless >>>>> there are very compelling reasons to do so. In its absence, we >>>>> should use load_acquire instead, and not micro optimize it away. >>>>> >>>>> AFAIK, compiler implementers are not happy with C++11 consume memory >>>>> ordering semantics and implement it with acquire instead as there >>>>> are too many problems with consume. >>>> I strongly disagree. Consume is fine now: I think you're dealing with >>>> out-of-date information. It took a while to figure out how to specify >>>> it, that's all. >>>> >>>>> To elide acquire in an acquire release pair in shared code, very >>>>> strong motivation is needed. Like for example how constructors >>>>> publish final object references with release but relies on dependent >>>>> load in JIT compiled code to elide acquire, because using acquire >>>>> semantics on all reference loads is too expensive, so it is >>>>> warranted here. >>>> Right. So we already depend on that property, so... >>>> >>>>> Therefore, I am wondering if perhaps we want to add an acquire on >>>>> the load side anyway, so we do not have to introduce consume memory >>>>> ordering semantics or something similar in the shared code >>>>> base. Using a normal load without any memory ordering semantics in >>>>> shared code to act like a load consume rings alarm clocks in my >>>>> head. >>>> I don't think it's that much of a big deal, really. Putting in an >>>> acquire fence every time we read a java mirror strikes me as madness >>>> when consume is all we need on any target. Even Power PC doesn't need >>>> a fence instruction there. At least make sure that targets can turn >>>> that off: we should not pay the price for something we don't need. >>>> >>> >> > From erik.osterlund at oracle.com Tue Jul 25 18:16:14 2017 From: erik.osterlund at oracle.com (=?utf-8?Q?Erik_=C3=96sterlund?=) Date: Tue, 25 Jul 2017 20:16:14 +0200 Subject: RFR (S) 8185296: java_lang_Class::array_klass should be array_klass_acquire In-Reply-To: <985f60b9-8481-715e-2b31-f0007968f962@oracle.com> References: <985f60b9-8481-715e-2b31-f0007968f962@oracle.com> Message-ID: Mmm yes - safe printing is the best kind of printing. Looks good. Thank you for fixing this. /Erik > On 25 Jul 2017, at 19:51, coleen.phillimore at oracle.com wrote: > > For consistency with https://bugs.openjdk.java.net/browse/JDK-8185296 should have made array_klass use load_acquire. > > open webrev at http://cr.openjdk.java.net/~coleenp/8185296.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8185296 > > Tested with inserted (now removed) print for the component mirror. Eg: > > printing mirror for component mirror: > java.lang.Class > {0x00000000f3a25be8} - klass: 'java/lang/Class' > - ---- fields (total size 15 words): > - private volatile transient strict 'cachedConstructor' 'Ljava/lang/reflect/Constructor;' @12 NULL (0 0) > - private volatile transient strict 'newInstanceCallerCache' 'Ljava/lang/Class;' @16 NULL (0 f3a26948) > - private transient 'name' 'Ljava/lang/String;' @20 "jdk.tools.jlink.internal.ImageLocationWriter"{0x00000000f3a26948} (f3a26948 eab3f768) > - private transient 'module' 'Ljava/lang/Module;' @24 a 'java/lang/Module'{0x00000000eab3f768} (eab3f768 eaaa0000) > - private final 'classLoader' 'Ljava/lang/ClassLoader;' @28 a 'jdk/internal/loader/ClassLoaders$AppClassLoader'{0x00000000eaaa0000} (eaaa0000 eab4d438) > - private transient 'packageName' 'Ljava/lang/String;' @32 "jdk.tools.jlink.internal"{0x00000000eab4d438} (eab4d438 0) > - private final strict 'componentType' 'Ljava/lang/Class;' @36 NULL (0 0) > - private volatile transient strict 'reflectionData' 'Ljava/lang/ref/SoftReference;' @40 NULL (0 0) > - private volatile transient 'genericInfo' 'Lsun/reflect/generics/repository/ClassRepository;' @44 NULL (0 0) > - private volatile transient strict 'enumConstants' '[Ljava/lang/Object;' @48 NULL (0 0) > - private volatile transient strict 'enumConstantDirectory' 'Ljava/util/Map;' @52 NULL (0 0) > - private volatile transient 'annotationData' 'Ljava/lang/Class$AnnotationData;' @56 NULL (0 0) > - private volatile transient 'annotationType' 'Lsun/reflect/annotation/AnnotationType;' @60 NULL (0 0) > - transient 'classValueMap' 'Ljava/lang/ClassValue$ClassValueMap;' @64 NULL (0 eab49d08) > - private volatile transient 'classRedefinedCount' 'I' @96 0 > - signature: Ljdk/tools/jlink/internal/ImageLocationWriter; > - fake entry for mirror: 'jdk/tools/jlink/internal/ImageLocationWriter' > - fake entry for array: 'jdk/tools/jlink/internal/ImageLocationWriter'[] <=== HERE > - fake entry for oop_size: 15 > - fake entry for static_oop_field_count: 0 > - static final synthetic '$assertionsDisabled' 'Z' @112 true > > > Thanks, > Coleen > > From coleen.phillimore at oracle.com Tue Jul 25 18:23:13 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 25 Jul 2017 14:23:13 -0400 Subject: RFR (S) 8185296: java_lang_Class::array_klass should be array_klass_acquire In-Reply-To: References: <985f60b9-8481-715e-2b31-f0007968f962@oracle.com> Message-ID: <7938bab3-3b0f-7da1-f38d-6606031eaf37@oracle.com> On 7/25/17 2:16 PM, Erik ?sterlund wrote: > Mmm yes - safe printing is the best kind of printing. > > Looks good. Thank you for fixing this. Thanks Erik! Coleen > > /Erik > >> On 25 Jul 2017, at 19:51, coleen.phillimore at oracle.com wrote: >> >> For consistency with https://bugs.openjdk.java.net/browse/JDK-8185296 should have made array_klass use load_acquire. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8185296.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8185296 >> >> Tested with inserted (now removed) print for the component mirror. Eg: >> >> printing mirror for component mirror: >> java.lang.Class >> {0x00000000f3a25be8} - klass: 'java/lang/Class' >> - ---- fields (total size 15 words): >> - private volatile transient strict 'cachedConstructor' 'Ljava/lang/reflect/Constructor;' @12 NULL (0 0) >> - private volatile transient strict 'newInstanceCallerCache' 'Ljava/lang/Class;' @16 NULL (0 f3a26948) >> - private transient 'name' 'Ljava/lang/String;' @20 "jdk.tools.jlink.internal.ImageLocationWriter"{0x00000000f3a26948} (f3a26948 eab3f768) >> - private transient 'module' 'Ljava/lang/Module;' @24 a 'java/lang/Module'{0x00000000eab3f768} (eab3f768 eaaa0000) >> - private final 'classLoader' 'Ljava/lang/ClassLoader;' @28 a 'jdk/internal/loader/ClassLoaders$AppClassLoader'{0x00000000eaaa0000} (eaaa0000 eab4d438) >> - private transient 'packageName' 'Ljava/lang/String;' @32 "jdk.tools.jlink.internal"{0x00000000eab4d438} (eab4d438 0) >> - private final strict 'componentType' 'Ljava/lang/Class;' @36 NULL (0 0) >> - private volatile transient strict 'reflectionData' 'Ljava/lang/ref/SoftReference;' @40 NULL (0 0) >> - private volatile transient 'genericInfo' 'Lsun/reflect/generics/repository/ClassRepository;' @44 NULL (0 0) >> - private volatile transient strict 'enumConstants' '[Ljava/lang/Object;' @48 NULL (0 0) >> - private volatile transient strict 'enumConstantDirectory' 'Ljava/util/Map;' @52 NULL (0 0) >> - private volatile transient 'annotationData' 'Ljava/lang/Class$AnnotationData;' @56 NULL (0 0) >> - private volatile transient 'annotationType' 'Lsun/reflect/annotation/AnnotationType;' @60 NULL (0 0) >> - transient 'classValueMap' 'Ljava/lang/ClassValue$ClassValueMap;' @64 NULL (0 eab49d08) >> - private volatile transient 'classRedefinedCount' 'I' @96 0 >> - signature: Ljdk/tools/jlink/internal/ImageLocationWriter; >> - fake entry for mirror: 'jdk/tools/jlink/internal/ImageLocationWriter' >> - fake entry for array: 'jdk/tools/jlink/internal/ImageLocationWriter'[] <=== HERE >> - fake entry for oop_size: 15 >> - fake entry for static_oop_field_count: 0 >> - static final synthetic '$assertionsDisabled' 'Z' @112 true >> >> >> Thanks, >> Coleen >> >> From ysr1729 at gmail.com Tue Jul 25 22:35:10 2017 From: ysr1729 at gmail.com (Srinivas Ramakrishna) Date: Tue, 25 Jul 2017 15:35:10 -0700 Subject: "product rw" vs "manageable" Message-ID: Today I had occasion to want to switch on what I thought was a "manageable" flag in a running JVM (TraceClass{Unl,L}oading), and I found that the flags were declared "product_rw" which is documented as: // product_rw flags are writeable internal product flags. // They are like "manageable" flags but for internal/private use. // The list of product_rw flags are internal/private flags which // may be changed/removed in a future release. It can be set // through the management interface to get/set value // when the name of flag is supplied. // // A flag can be made as "product_rw" only if // - the VM implementation supports dynamic setting of the flag. // This implies that the VM must *always* query the flag variable // and not reuse state related to the flag state at any given time. I assumed this just meant that the flags were like "manageable" but not "supported" (in the sense of constituting a stable interface). However, I was surprised to find that I wasn't able to modify these flags via jinfo, which instead elicited the following error: $ jinfo -flag +TraceClassUnloading 26136 -XX:-TraceClassUnloading $ jinfo -flag +TraceClassUnloading 26136 Exception in thread "main" com.sun.tools.attach.AttachOperationFailedException: flag 'TraceClassUnloading' cannot be changed at sun.tools.attach.LinuxVirtualMachine.execute(LinuxVirtualMachine.java:269) at sun.tools.attach.HotSpotVirtualMachine.executeCommand(HotSpotVirtualMachine.java:261) at sun.tools.attach.HotSpotVirtualMachine.setFlag(HotSpotVirtualMachine.java:234) at sun.tools.jinfo.JInfo.flag(JInfo.java:140) at sun.tools.jinfo.JInfo.main(JInfo.java:81) So, what do "product_rw" flags represent, and can they be changed (using jinfo or similar)? thanks! -- ramki From kim.barrett at oracle.com Tue Jul 25 23:07:22 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 25 Jul 2017 19:07:22 -0400 Subject: RFR (S) 8185296: java_lang_Class::array_klass should be array_klass_acquire In-Reply-To: <985f60b9-8481-715e-2b31-f0007968f962@oracle.com> References: <985f60b9-8481-715e-2b31-f0007968f962@oracle.com> Message-ID: > On Jul 25, 2017, at 1:51 PM, coleen.phillimore at oracle.com wrote: > > For consistency with https://bugs.openjdk.java.net/browse/JDK-8185296 should have made array_klass use load_acquire. > > open webrev at http://cr.openjdk.java.net/~coleenp/8185296.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8185296 Looks good. From coleen.phillimore at oracle.com Tue Jul 25 23:10:42 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 25 Jul 2017 19:10:42 -0400 Subject: RFR (S) 8185296: java_lang_Class::array_klass should be array_klass_acquire In-Reply-To: References: <985f60b9-8481-715e-2b31-f0007968f962@oracle.com> Message-ID: <9d62a7de-2755-e5f0-0520-5b233defffd3@oracle.com> On 7/25/17 7:07 PM, Kim Barrett wrote: >> On Jul 25, 2017, at 1:51 PM, coleen.phillimore at oracle.com wrote: >> >> For consistency with https://bugs.openjdk.java.net/browse/JDK-8185296 should have made array_klass use load_acquire. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8185296.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8185296 > Looks good. Thanks, Kim! Coleen > From rednaxelafx at gmail.com Wed Jul 26 00:20:50 2017 From: rednaxelafx at gmail.com (Krystal Mok) Date: Tue, 25 Jul 2017 17:20:50 -0700 Subject: "product rw" vs "manageable" In-Reply-To: References: Message-ID: Hi Ramki, In the current JDK8u HotSpot, jinfo -flag is implemented in the attach listener as: 344 // Implementation of "setflag" command 345 static jint set_flag(AttachOperation* op, outputStream* out) { 346 347 const char* name = NULL; 348 if ((name = op->arg(0)) == NULL) { 349 out->print_cr("flag name is missing"); 350 return JNI_ERR; 351 } 352 353 Flag* f = Flag::find_flag((char*)name, strlen(name)); 354 if (f && f->is_external() && f->is_writeable()) { 355 if (f->is_bool()) { 356 return set_bool_flag(name, op, out); 357 } else if (f->is_intx()) { 358 return set_intx_flag(name, op, out); 359 } else if (f->is_uintx()) { 360 return set_uintx_flag(name, op, out); 361 } else if (f->is_uint64_t()) { 362 return set_uint64_t_flag(name, op, out); 363 } else if (f->is_ccstr()) { 364 return set_ccstr_flag(name, op, out); 365 } else { 366 ShouldNotReachHere(); 367 return JNI_ERR; 368 } 369 } else { 370 return AttachListener::pd_set_flag(op, out); 371 } 372 } So the reason why you're not able to set a product_rw flag through jinfo is because: 282 // All flags except "manageable" are assumed to be internal flags. 283 // Long term, we need to define a mechanism to specify which flags 284 // are external/stable and change this function accordingly. 285 bool Flag::is_external() const { 286 return is_manageable() || is_external_ext(); 287 } But through MXBean "HotSpotDiagnosticMXBean", you can actually set product_rw flags without any problems. Try setting the flag with JConsole or VisualVM or any other JMX client and you'll see. That's because jmm_SetVMGlobal() only checks whether or not a flag is writeable, but doesn't check if it's external. - Kris On Tue, Jul 25, 2017 at 3:35 PM, Srinivas Ramakrishna wrote: > Today I had occasion to want to switch on what I thought was a "manageable" > flag in a running JVM (TraceClass{Unl,L}oading), and I found that the flags > were declared "product_rw" which is documented as: > > // product_rw flags are writeable internal product flags. > // They are like "manageable" flags but for internal/private use. > // The list of product_rw flags are internal/private flags which > // may be changed/removed in a future release. It can be set > // through the management interface to get/set value > // when the name of flag is supplied. > // > // A flag can be made as "product_rw" only if > // - the VM implementation supports dynamic setting of the flag. > // This implies that the VM must *always* query the flag variable > // and not reuse state related to the flag state at any given time. > > I assumed this just meant that the flags were like "manageable" but not > "supported" (in the sense of constituting a stable interface). > However, I was surprised to find that I wasn't able to modify these flags > via jinfo, which instead elicited the following error: > > $ jinfo -flag +TraceClassUnloading 26136 > -XX:-TraceClassUnloading > > $ jinfo -flag +TraceClassUnloading 26136 > Exception in thread "main" > com.sun.tools.attach.AttachOperationFailedException: flag > 'TraceClassUnloading' cannot be changed > > at > sun.tools.attach.LinuxVirtualMachine.execute(LinuxVirtualMachine.java:269) > at > sun.tools.attach.HotSpotVirtualMachine.executeCommand( > HotSpotVirtualMachine.java:261) > at > sun.tools.attach.HotSpotVirtualMachine.setFlag(HotSpotVirtualMachine.java: > 234) > at sun.tools.jinfo.JInfo.flag(JInfo.java:140) > at sun.tools.jinfo.JInfo.main(JInfo.java:81) > > So, what do "product_rw" flags represent, and can they be changed (using > jinfo or similar)? > > thanks! > -- ramki > From ysr1729 at gmail.com Wed Jul 26 04:02:40 2017 From: ysr1729 at gmail.com (Srinivas Ramakrishna) Date: Tue, 25 Jul 2017 21:02:40 -0700 Subject: "product rw" vs "manageable" In-Reply-To: References: Message-ID: Ah, Thanks Kris! I'll assume there was no real reason to make it difficult to set the flag through jinfo, then, as I don't see why the logic of setting these flags needs to be so convoluted. If anyone recalls any reasoned history for this difference in behaviour, please let me know. (It's OK to say, "that's a bug in jinfo" or "that's a bug in jmm"; else i'll assume it's the former :-) and the flags should be settable from jinfo, except that the flags themselves do not constitute a stable api. thanks, -- ramki On Tue, Jul 25, 2017 at 5:20 PM, Krystal Mok wrote: > Hi Ramki, > > In the current JDK8u HotSpot, jinfo -flag is implemented in the attach > listener as: > > 344 // Implementation of "setflag" command > 345 static jint set_flag(AttachOperation* op, outputStream* out) { > 346 > 347 const char* name = NULL; > 348 if ((name = op->arg(0)) == NULL) { > 349 out->print_cr("flag name is missing"); > 350 return JNI_ERR; > 351 } > 352 > 353 Flag* f = Flag::find_flag((char*)name, strlen(name)); > 354 if (f && f->is_external() && f->is_writeable()) { > 355 if (f->is_bool()) { > 356 return set_bool_flag(name, op, out); > 357 } else if (f->is_intx()) { > 358 return set_intx_flag(name, op, out); > 359 } else if (f->is_uintx()) { > 360 return set_uintx_flag(name, op, out); > 361 } else if (f->is_uint64_t()) { > 362 return set_uint64_t_flag(name, op, out); > 363 } else if (f->is_ccstr()) { > 364 return set_ccstr_flag(name, op, out); > 365 } else { > 366 ShouldNotReachHere(); > 367 return JNI_ERR; > 368 } > 369 } else { > 370 return AttachListener::pd_set_flag(op, out); > 371 } > 372 } > > So the reason why you're not able to set a product_rw flag through jinfo > is because: > > 282 // All flags except "manageable" are assumed to be internal flags. > 283 // Long term, we need to define a mechanism to specify which flags > 284 // are external/stable and change this function accordingly. > 285 bool Flag::is_external() const { > 286 return is_manageable() || is_external_ext(); > 287 } > > But through MXBean "HotSpotDiagnosticMXBean", you can actually set > product_rw flags without any problems. Try setting the flag with JConsole > or VisualVM or any other JMX client and you'll see. > > That's because jmm_SetVMGlobal() only checks whether or not a flag is > writeable, but doesn't check if it's external. > > - Kris > > On Tue, Jul 25, 2017 at 3:35 PM, Srinivas Ramakrishna > wrote: > >> Today I had occasion to want to switch on what I thought was a >> "manageable" >> flag in a running JVM (TraceClass{Unl,L}oading), and I found that the >> flags >> were declared "product_rw" which is documented as: >> >> // product_rw flags are writeable internal product flags. >> // They are like "manageable" flags but for internal/private use. >> // The list of product_rw flags are internal/private flags which >> // may be changed/removed in a future release. It can be set >> // through the management interface to get/set value >> // when the name of flag is supplied. >> // >> // A flag can be made as "product_rw" only if >> // - the VM implementation supports dynamic setting of the flag. >> // This implies that the VM must *always* query the flag variable >> // and not reuse state related to the flag state at any given time. >> >> I assumed this just meant that the flags were like "manageable" but not >> "supported" (in the sense of constituting a stable interface). >> However, I was surprised to find that I wasn't able to modify these flags >> via jinfo, which instead elicited the following error: >> >> $ jinfo -flag +TraceClassUnloading 26136 >> -XX:-TraceClassUnloading >> >> $ jinfo -flag +TraceClassUnloading 26136 >> Exception in thread "main" >> com.sun.tools.attach.AttachOperationFailedException: flag >> 'TraceClassUnloading' cannot be changed >> >> at >> sun.tools.attach.LinuxVirtualMachine.execute(LinuxVirtualMac >> hine.java:269) >> at >> sun.tools.attach.HotSpotVirtualMachine.executeCommand(HotSpo >> tVirtualMachine.java:261) >> at >> sun.tools.attach.HotSpotVirtualMachine.setFlag(HotSpotVirtua >> lMachine.java:234) >> at sun.tools.jinfo.JInfo.flag(JInfo.java:140) >> at sun.tools.jinfo.JInfo.main(JInfo.java:81) >> >> So, what do "product_rw" flags represent, and can they be changed (using >> jinfo or similar)? >> >> thanks! >> -- ramki >> > > From david.holmes at oracle.com Wed Jul 26 06:15:51 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 26 Jul 2017 16:15:51 +1000 Subject: "product rw" vs "manageable" In-Reply-To: References: Message-ID: <9a7ca984-8d71-21ee-9fd1-95c7e5202db7@oracle.com> On 26/07/2017 2:02 PM, Srinivas Ramakrishna wrote: > Ah, Thanks Kris! I'll assume there was no real reason to make it difficult > to set the flag through jinfo, then, as I don't see why the logic of > setting these flags needs to be so convoluted. > > If anyone recalls any reasoned history for this difference in behaviour, > please let me know. (It's OK to say, "that's a bug in jinfo" or "that's a > bug in jmm"; else i'll assume it's the former :-) and the flags should be > settable from jinfo, except that the flags themselves do not constitute a > stable api. That's really a question for serviceability folk - cc'd - but I think jinfo should only allow setting of the exported external flags ie manageable ones. For hotspot internal flags (product_rw) you use a hotspot specific mechanism - i.e HotspotMXBean. So no bug here IMHO. But these lines are blurrier now than they used to be I think :) David > thanks, > -- ramki > > On Tue, Jul 25, 2017 at 5:20 PM, Krystal Mok wrote: > >> Hi Ramki, >> >> In the current JDK8u HotSpot, jinfo -flag is implemented in the attach >> listener as: >> >> 344 // Implementation of "setflag" command >> 345 static jint set_flag(AttachOperation* op, outputStream* out) { >> 346 >> 347 const char* name = NULL; >> 348 if ((name = op->arg(0)) == NULL) { >> 349 out->print_cr("flag name is missing"); >> 350 return JNI_ERR; >> 351 } >> 352 >> 353 Flag* f = Flag::find_flag((char*)name, strlen(name)); >> 354 if (f && f->is_external() && f->is_writeable()) { >> 355 if (f->is_bool()) { >> 356 return set_bool_flag(name, op, out); >> 357 } else if (f->is_intx()) { >> 358 return set_intx_flag(name, op, out); >> 359 } else if (f->is_uintx()) { >> 360 return set_uintx_flag(name, op, out); >> 361 } else if (f->is_uint64_t()) { >> 362 return set_uint64_t_flag(name, op, out); >> 363 } else if (f->is_ccstr()) { >> 364 return set_ccstr_flag(name, op, out); >> 365 } else { >> 366 ShouldNotReachHere(); >> 367 return JNI_ERR; >> 368 } >> 369 } else { >> 370 return AttachListener::pd_set_flag(op, out); >> 371 } >> 372 } >> >> So the reason why you're not able to set a product_rw flag through jinfo >> is because: >> >> 282 // All flags except "manageable" are assumed to be internal flags. >> 283 // Long term, we need to define a mechanism to specify which flags >> 284 // are external/stable and change this function accordingly. >> 285 bool Flag::is_external() const { >> 286 return is_manageable() || is_external_ext(); >> 287 } >> >> But through MXBean "HotSpotDiagnosticMXBean", you can actually set >> product_rw flags without any problems. Try setting the flag with JConsole >> or VisualVM or any other JMX client and you'll see. >> >> That's because jmm_SetVMGlobal() only checks whether or not a flag is >> writeable, but doesn't check if it's external. >> >> - Kris >> >> On Tue, Jul 25, 2017 at 3:35 PM, Srinivas Ramakrishna >> wrote: >> >>> Today I had occasion to want to switch on what I thought was a >>> "manageable" >>> flag in a running JVM (TraceClass{Unl,L}oading), and I found that the >>> flags >>> were declared "product_rw" which is documented as: >>> >>> // product_rw flags are writeable internal product flags. >>> // They are like "manageable" flags but for internal/private use. >>> // The list of product_rw flags are internal/private flags which >>> // may be changed/removed in a future release. It can be set >>> // through the management interface to get/set value >>> // when the name of flag is supplied. >>> // >>> // A flag can be made as "product_rw" only if >>> // - the VM implementation supports dynamic setting of the flag. >>> // This implies that the VM must *always* query the flag variable >>> // and not reuse state related to the flag state at any given time. >>> >>> I assumed this just meant that the flags were like "manageable" but not >>> "supported" (in the sense of constituting a stable interface). >>> However, I was surprised to find that I wasn't able to modify these flags >>> via jinfo, which instead elicited the following error: >>> >>> $ jinfo -flag +TraceClassUnloading 26136 >>> -XX:-TraceClassUnloading >>> >>> $ jinfo -flag +TraceClassUnloading 26136 >>> Exception in thread "main" >>> com.sun.tools.attach.AttachOperationFailedException: flag >>> 'TraceClassUnloading' cannot be changed >>> >>> at >>> sun.tools.attach.LinuxVirtualMachine.execute(LinuxVirtualMac >>> hine.java:269) >>> at >>> sun.tools.attach.HotSpotVirtualMachine.executeCommand(HotSpo >>> tVirtualMachine.java:261) >>> at >>> sun.tools.attach.HotSpotVirtualMachine.setFlag(HotSpotVirtua >>> lMachine.java:234) >>> at sun.tools.jinfo.JInfo.flag(JInfo.java:140) >>> at sun.tools.jinfo.JInfo.main(JInfo.java:81) >>> >>> So, what do "product_rw" flags represent, and can they be changed (using >>> jinfo or similar)? >>> >>> thanks! >>> -- ramki >>> >> >> From thomas.stuefe at gmail.com Wed Jul 26 12:48:35 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 26 Jul 2017 14:48:35 +0200 Subject: RFR(xxs): 8185262: Keep default logging for Metaspace OOM short and concise In-Reply-To: <3916fc10-38fd-a121-98c7-43dd12078453@oracle.com> References: <3916fc10-38fd-a121-98c7-43dd12078453@oracle.com> Message-ID: Hi Ioi, new webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8185262-keep-metaspace-oom-log-short-for-default/webrev.01/webrev/ I moved the initialization of the debug LogStream object into the "loader_data->metaspace_or_null()" condition, as you suggested. Thanks! Thomas On Tue, Jul 25, 2017 at 7:29 PM, Ioi Lam wrote: > Sounds good. Thanks Thomas. > > - Ioi > > On 7/25/17 10:25 AM, Thomas St?fe wrote: > > Hi Ioi, > > On Tue, Jul 25, 2017 at 7:12 PM, Ioi Lam wrote: > >> Hi Thomas, >> >> how about switching lines 3739 and 3740? >> >> 3738 if (log.is_debug()) { >> 3739 LogStream ls(log.debug()); >> 3740 if (loader_data->metaspace_or_null() != NULL) { >> 3741 loader_data->dump(&ls); >> 3742 } >> 3743 } >> >> > sure. > > >> What kind of tests have you run? >> >> > Besides my tiny test case causing metaspace oom? gtests on Windows x64, > Linux x64/ppc and AIX, and, on Linux x64, jtreg tests from > hotspot/runtime/Metaspace. All with fastdebug builds. > > I also looked around whether any test parses the log output, but could not > find any code setting gc+metaspace+freelist. > > Thanks, Thomas > > Thanks >> - Ioi >> >> >> On 7/25/17 10:07 AM, Thomas St?fe wrote: >> >>> Hi all, >>> >>> may I have please reviews for this very tiny change: >>> >>> bug: https://bugs.openjdk.java.net/browse/JDK-8185262 >>> webrev: http://cr.openjdk.java.net/~stuefe/webrevs/ >>> 8185262-keep-metaspace-oom-log-short-for-default/webrev.00/webrev/ >>> >>> On Metaspace OOM, we dump all metaspace chunks in use. This can be very >>> lengthy. It would be nice if this part of logging were restricted to a >>> higher loglevel. In this change, I restrict it to debug. >>> >>> Kind Regards, Thomas >>> >> >> > > From aph at redhat.com Wed Jul 26 13:22:37 2017 From: aph at redhat.com (Andrew Haley) Date: Wed, 26 Jul 2017 14:22:37 +0100 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: <2c637e28-8d95-cbaf-c29e-d01867ed9157@redhat.com> References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <596CD126.6080100@oracle.com> <596DD713.6080806@oracle.com> <77d5a9e7-b2b0-79e4-effd-903dfe226512@redhat.com> <596E1142.6040708@oracle.com> <3432d11c-5904-a446-1cb5-419627f1794f@redhat.com> <596E288B.90706@oracle.com> <773a0b64-8273-1a58-a46a-22ae18f927b7@redhat.com> <5970B3FD.607@oracle.com> <8e1d9a32-7e01-b917-df1d-595c4000e4b2@redhat.com> <597226E2.6090803@oracle.com> <2c637e28-8d95-cbaf-c29e-d01867ed9157@redhat.com> Message-ID: I've been sketching out an alternative way to handle this. I've got my flak jacket ready, so here goes... My goals are: Simplicity; with this comes, hopefully, clarity. Within reason, we should minimize the extra code this brings in. Type safety where possible. I'm aware that we compile HotSpot with -fno-strict-aliasing or its equivalent, but I think we can do this job without relying on that. The more information we expose to the compiler, the better code it can generate. Something that plays nicely with compiler builtins, where appropriate. It must also be able to handle legacy assembly-language definitions. [ Correctness, hopefully, goes without saying. ] I've done all the work on cmpxchg, because that is the most complex of the operations: I believe that if cmpxchg can work well, the rest will be relatively simple. ---------------------------------------------------------------------- We should expose the type information of the original operands all the way to the back ends, and not cast pointer types if we can possibly avoid it. So, in this design every back end defines a template function to match this declaration: template inline static U cmpxchg(T exchange_value, volatile U* dest, V compare_value, cmpxchg_memory_order order = memory_order_conservative); [ Note: the types of the operand, the compare value, and the exchange value are distinct. This is a design decision I took from Erik's patch, but I don't like it: it seems to me that we should insist on the same types for the operands. This requires a few changes in HotSpot, but arguably these are bugs anyway. I tried this, and had to fix eight places, all trivial. We should do it as a cleanup. ] First, the simplest possible case. In some modern GCC targets (e.g Aarch64) we can trivially define our generic cmpxchg using a compiler builtin, like so: template inline U Atomic::cmpxchg(T exchange_value, volatile U* dest, V compare_value, cmpxchg_memory_order order) { return __sync_val_compare_and_swap(dest, compare_value, exchange_value); } That's everything we need, although I'd do some optimization for the memory_order. With older targets and those where we need to support assembly language, we'll have to use some template magic to select an appropriate assembly fragment: template inline U Atomic::cmpxchg(T exchange_value, volatile U* dest, V compare_value, cmpxchg_memory_order order) { return CmpxchgHelper()(exchange_value, dest, compare_value, order); } Here's one such fragment, for the 64-bit case: template struct Atomic::CmpxchgHelper { U operator()(T exchange_value, volatile U* dest, V compare_value, cmpxchg_memory_order order) { U result; __asm__ __volatile__ ("lock cmpxchgq %1,(%3)" : "=a" (result) : "r" ((U)exchange_value), "a" ((U)compare_value), "r" (dest) : "cc", "memory"); return result; } }; The other cases are very similar, and I won't describe them here. Note that we're using plain casts to convert the compare and exchange values: this allows trivial cases such as extending a plain integer constant to a 64-bit type. [ Note: The new Helper Classes are necessary because the antique version of C++ we use to build HotSpot doesn't permit the partial specialization of templates. This is merely annoying syntax and doesn't affect the generated code. ] Where there are back-end functions that cannot be genericized in this way (perhaps because they always take a void * or an intptr_t *) we can do whatever casting is necessary in the back-end fragments. There doesn't need to be any casting in the shared code. This proposal does not handle floating-point operands because none are currently needed, but I believe that the correct way to do it involves explicit specializations for the types double and float. These can be in the shared code in atomic.h if we ever need them. [ Note: Supporting floating-point operands is problematic because it's hard to get the types right. If T, U, and V can all be different types then we need to know exactly what is meant by cmpxchg() where the object in memory is an integer type and the compare or exchange values are of floating-point type. IMO, we should reject nonsense like that at compile time. Forcing T, U, and V to be the same type allows us to do that. ] ---------------------------------------------------------------------- This solution doesn't require any new helper classes. It does, however, require changes to the back ends which convert methods to helper classes. This is additional work now, but the result IMO will be simpler, easier to maintain, and more future proof. Do you know of any operating systems, compilers, or targets where this won't work? Or of any particular difficulty doing this? I know we'll have to get buy-in from the back end maintainers, but right now in the JDK 10 release cycle is the time to get things like this done. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From ysr1729 at gmail.com Wed Jul 26 15:38:24 2017 From: ysr1729 at gmail.com (Srinivas Ramakrishna) Date: Wed, 26 Jul 2017 08:38:24 -0700 Subject: "product rw" vs "manageable" In-Reply-To: <9a7ca984-8d71-21ee-9fd1-95c7e5202db7@oracle.com> References: <9a7ca984-8d71-21ee-9fd1-95c7e5202db7@oracle.com> Message-ID: Thanks for your comments, David! I'll wait for further comments, if any, from serviceability folk. -- ramki On Tue, Jul 25, 2017 at 11:15 PM, David Holmes wrote: > On 26/07/2017 2:02 PM, Srinivas Ramakrishna wrote: > >> Ah, Thanks Kris! I'll assume there was no real reason to make it difficult >> to set the flag through jinfo, then, as I don't see why the logic of >> setting these flags needs to be so convoluted. >> >> If anyone recalls any reasoned history for this difference in behaviour, >> please let me know. (It's OK to say, "that's a bug in jinfo" or "that's a >> bug in jmm"; else i'll assume it's the former :-) and the flags should be >> settable from jinfo, except that the flags themselves do not constitute a >> stable api. >> > > That's really a question for serviceability folk - cc'd - but I think > jinfo should only allow setting of the exported external flags ie > manageable ones. For hotspot internal flags (product_rw) you use a hotspot > specific mechanism - i.e HotspotMXBean. So no bug here IMHO. But these > lines are blurrier now than they used to be I think :) > > David > > > > thanks, >> -- ramki >> >> On Tue, Jul 25, 2017 at 5:20 PM, Krystal Mok >> wrote: >> >> Hi Ramki, >>> >>> In the current JDK8u HotSpot, jinfo -flag is implemented in the attach >>> listener as: >>> >>> 344 // Implementation of "setflag" command >>> 345 static jint set_flag(AttachOperation* op, outputStream* out) { >>> 346 >>> 347 const char* name = NULL; >>> 348 if ((name = op->arg(0)) == NULL) { >>> 349 out->print_cr("flag name is missing"); >>> 350 return JNI_ERR; >>> 351 } >>> 352 >>> 353 Flag* f = Flag::find_flag((char*)name, strlen(name)); >>> 354 if (f && f->is_external() && f->is_writeable()) { >>> 355 if (f->is_bool()) { >>> 356 return set_bool_flag(name, op, out); >>> 357 } else if (f->is_intx()) { >>> 358 return set_intx_flag(name, op, out); >>> 359 } else if (f->is_uintx()) { >>> 360 return set_uintx_flag(name, op, out); >>> 361 } else if (f->is_uint64_t()) { >>> 362 return set_uint64_t_flag(name, op, out); >>> 363 } else if (f->is_ccstr()) { >>> 364 return set_ccstr_flag(name, op, out); >>> 365 } else { >>> 366 ShouldNotReachHere(); >>> 367 return JNI_ERR; >>> 368 } >>> 369 } else { >>> 370 return AttachListener::pd_set_flag(op, out); >>> 371 } >>> 372 } >>> >>> So the reason why you're not able to set a product_rw flag through jinfo >>> is because: >>> >>> 282 // All flags except "manageable" are assumed to be internal flags. >>> 283 // Long term, we need to define a mechanism to specify which flags >>> 284 // are external/stable and change this function accordingly. >>> 285 bool Flag::is_external() const { >>> 286 return is_manageable() || is_external_ext(); >>> 287 } >>> >>> But through MXBean "HotSpotDiagnosticMXBean", you can actually set >>> product_rw flags without any problems. Try setting the flag with JConsole >>> or VisualVM or any other JMX client and you'll see. >>> >>> That's because jmm_SetVMGlobal() only checks whether or not a flag is >>> writeable, but doesn't check if it's external. >>> >>> - Kris >>> >>> On Tue, Jul 25, 2017 at 3:35 PM, Srinivas Ramakrishna >> > >>> wrote: >>> >>> Today I had occasion to want to switch on what I thought was a >>>> "manageable" >>>> flag in a running JVM (TraceClass{Unl,L}oading), and I found that the >>>> flags >>>> were declared "product_rw" which is documented as: >>>> >>>> // product_rw flags are writeable internal product flags. >>>> // They are like "manageable" flags but for internal/private use. >>>> // The list of product_rw flags are internal/private flags which >>>> // may be changed/removed in a future release. It can be set >>>> // through the management interface to get/set value >>>> // when the name of flag is supplied. >>>> // >>>> // A flag can be made as "product_rw" only if >>>> // - the VM implementation supports dynamic setting of the flag. >>>> // This implies that the VM must *always* query the flag variable >>>> // and not reuse state related to the flag state at any given time. >>>> >>>> I assumed this just meant that the flags were like "manageable" but not >>>> "supported" (in the sense of constituting a stable interface). >>>> However, I was surprised to find that I wasn't able to modify these >>>> flags >>>> via jinfo, which instead elicited the following error: >>>> >>>> $ jinfo -flag +TraceClassUnloading 26136 >>>> -XX:-TraceClassUnloading >>>> >>>> $ jinfo -flag +TraceClassUnloading 26136 >>>> Exception in thread "main" >>>> com.sun.tools.attach.AttachOperationFailedException: flag >>>> 'TraceClassUnloading' cannot be changed >>>> >>>> at >>>> sun.tools.attach.LinuxVirtualMachine.execute(LinuxVirtualMac >>>> hine.java:269) >>>> at >>>> sun.tools.attach.HotSpotVirtualMachine.executeCommand(HotSpo >>>> tVirtualMachine.java:261) >>>> at >>>> sun.tools.attach.HotSpotVirtualMachine.setFlag(HotSpotVirtua >>>> lMachine.java:234) >>>> at sun.tools.jinfo.JInfo.flag(JInfo.java:140) >>>> at sun.tools.jinfo.JInfo.main(JInfo.java:81) >>>> >>>> So, what do "product_rw" flags represent, and can they be changed (using >>>> jinfo or similar)? >>>> >>>> thanks! >>>> -- ramki >>>> >>>> >>> >>> From coleen.phillimore at oracle.com Wed Jul 26 16:09:17 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 26 Jul 2017 12:09:17 -0400 Subject: RFR(xs): 8185263: Fix zero build after 8169881, 8175318, 8178350 In-Reply-To: References: Message-ID: <8ae9387c-d503-8a05-325a-2c0a9a7a2a6b@oracle.com> Looks good. Thanks Ioi for sponsoring. Coleen On 7/25/17 1:31 PM, Ioi Lam wrote: > Hi Thomas, > > Looks good. I can sponsor it. > > Thanks > > - Ioi > > > On 7/25/17 10:13 AM, Thomas St?fe wrote: >> Hi all, >> >> may I please have reviews and sponsor for these zero-related build >> fixes? >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8185263 >> webrev: >> http://cr.openjdk.java.net/~stuefe/webrevs/8185263-fix-zero-build/webrev.00/webrev/index.html >> >> >> Thanks! Thomas > From coleen.phillimore at oracle.com Wed Jul 26 16:11:56 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 26 Jul 2017 12:11:56 -0400 Subject: RFR(xxs): 8185262: Keep default logging for Metaspace OOM short and concise In-Reply-To: <3916fc10-38fd-a121-98c7-43dd12078453@oracle.com> References: <3916fc10-38fd-a121-98c7-43dd12078453@oracle.com> Message-ID: <1f260c72-826f-5773-b4f1-74ccbf60aa95@oracle.com> Hi Thomas, version .01 looks good with Ioi's suggestion. I'll sponsor it. thanks, Coleen On 7/25/17 1:29 PM, Ioi Lam wrote: > Sounds good. Thanks Thomas. > > - Ioi > > > On 7/25/17 10:25 AM, Thomas St?fe wrote: >> Hi Ioi, >> >> On Tue, Jul 25, 2017 at 7:12 PM, Ioi Lam > > wrote: >> >> Hi Thomas, >> >> how about switching lines 3739 and 3740? >> >> 3738 if (log.is_debug()) { >> 3739 LogStream ls(log.debug()); >> 3740 if (loader_data->metaspace_or_null() != NULL) { >> 3741 loader_data->dump(&ls); >> 3742 } >> 3743 } >> >> >> sure. >> >> What kind of tests have you run? >> >> >> Besides my tiny test case causing metaspace oom? gtests on Windows >> x64, Linux x64/ppc and AIX, and, on Linux x64, jtreg tests from >> hotspot/runtime/Metaspace. All with fastdebug builds. >> >> I also looked around whether any test parses the log output, but >> could not find any code setting gc+metaspace+freelist. >> >> Thanks, Thomas >> >> Thanks >> - Ioi >> >> >> On 7/25/17 10:07 AM, Thomas St?fe wrote: >> >> Hi all, >> >> may I have please reviews for this very tiny change: >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8185262 >> >> webrev: http://cr.openjdk.java.net/~stuefe/webrevs/ >> >> 8185262-keep-metaspace-oom-log-short-for-default/webrev.00/webrev/ >> >> On Metaspace OOM, we dump all metaspace chunks in use. This >> can be very >> lengthy. It would be nice if this part of logging were >> restricted to a >> higher loglevel. In this change, I restrict it to debug. >> >> Kind Regards, Thomas >> >> >> > From thomas.stuefe at gmail.com Wed Jul 26 17:51:39 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 26 Jul 2017 19:51:39 +0200 Subject: RFR(xxs): 8185262: Keep default logging for Metaspace OOM short and concise In-Reply-To: <1f260c72-826f-5773-b4f1-74ccbf60aa95@oracle.com> References: <3916fc10-38fd-a121-98c7-43dd12078453@oracle.com> <1f260c72-826f-5773-b4f1-74ccbf60aa95@oracle.com> Message-ID: Thanks Coleen! On Wed, Jul 26, 2017 at 6:11 PM, wrote: > > Hi Thomas, > version .01 looks good with Ioi's suggestion. I'll sponsor it. > thanks, > Coleen > > On 7/25/17 1:29 PM, Ioi Lam wrote: > >> Sounds good. Thanks Thomas. >> >> - Ioi >> >> >> On 7/25/17 10:25 AM, Thomas St?fe wrote: >> >>> Hi Ioi, >>> >>> On Tue, Jul 25, 2017 at 7:12 PM, Ioi Lam >> ioi.lam at oracle.com>> wrote: >>> >>> Hi Thomas, >>> >>> how about switching lines 3739 and 3740? >>> >>> 3738 if (log.is_debug()) { >>> 3739 LogStream ls(log.debug()); >>> 3740 if (loader_data->metaspace_or_null() != NULL) { >>> 3741 loader_data->dump(&ls); >>> 3742 } >>> 3743 } >>> >>> >>> sure. >>> >>> What kind of tests have you run? >>> >>> >>> Besides my tiny test case causing metaspace oom? gtests on Windows x64, >>> Linux x64/ppc and AIX, and, on Linux x64, jtreg tests from >>> hotspot/runtime/Metaspace. All with fastdebug builds. >>> >>> I also looked around whether any test parses the log output, but could >>> not find any code setting gc+metaspace+freelist. >>> >>> Thanks, Thomas >>> >>> Thanks >>> - Ioi >>> >>> >>> On 7/25/17 10:07 AM, Thomas St?fe wrote: >>> >>> Hi all, >>> >>> may I have please reviews for this very tiny change: >>> >>> bug: https://bugs.openjdk.java.net/browse/JDK-8185262 >>> >>> webrev: http://cr.openjdk.java.net/~stuefe/webrevs/ >>> >>> 8185262-keep-metaspace-oom-log-short-for-default/webrev.00/webrev/ >>> >>> On Metaspace OOM, we dump all metaspace chunks in use. This >>> can be very >>> lengthy. It would be nice if this part of logging were >>> restricted to a >>> higher loglevel. In this change, I restrict it to debug. >>> >>> Kind Regards, Thomas >>> >>> >>> >>> >> > From mark.reinhold at oracle.com Wed Jul 26 21:09:45 2017 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Wed, 26 Jul 2017 14:09:45 -0700 (PDT) Subject: JEP 306: Restore Always-Strict Floating-Point Semantics Message-ID: <20170726210945.AE001983AE@eggemoggin.niobe.net> New JEP Candidate: http://openjdk.java.net/jeps/306 - Mark From andrey.x.nazarov at oracle.com Thu Jul 20 01:21:01 2017 From: andrey.x.nazarov at oracle.com (Andrey Nazarov) Date: Wed, 19 Jul 2017 18:21:01 -0700 Subject: RFR 8184961: jdk.test.lib.util.FileUtils.deleteFileWithRetry0 should wait for absence of a file Message-ID: Hi, Could you review fix in test library which should help to resolve intermittent issues on Windows machines during directories clean up. Bug: https://bugs.openjdk.java.net/browse/JDK-8184961 Review: http://cr.openjdk.java.net/~anazarov/JDK-8184961/webrev.00/webrev/ ?Andrei From andrey.x.nazarov at oracle.com Tue Jul 25 22:59:25 2017 From: andrey.x.nazarov at oracle.com (Andrey Nazarov) Date: Tue, 25 Jul 2017 15:59:25 -0700 Subject: RFR 8184961: jdk.test.lib.util.FileUtils.deleteFileWithRetry0 should wait for absence of a file In-Reply-To: References: Message-ID: Can anyone look? ?Thanks, Andrei > On 19 Jul 2017, at 18:21, Andrey Nazarov wrote: > > Hi, > > Could you review fix in test library which should help to resolve intermittent issues on Windows machines during directories clean up. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8184961 > Review: http://cr.openjdk.java.net/~anazarov/JDK-8184961/webrev.00/webrev/ > > > ?Andrei From brian.burkhalter at oracle.com Tue Jul 25 23:41:07 2017 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 25 Jul 2017 16:41:07 -0700 Subject: RFR 8184961: jdk.test.lib.util.FileUtils.deleteFileWithRetry0 should wait for absence of a file In-Reply-To: References: Message-ID: <9EDF7199-8FA3-4C2E-8792-E36E456B8778@oracle.com> Hi Andrei, I think this looks good. Thanks, Brian On Jul 25, 2017, at 3:59 PM, Andrey Nazarov wrote: > Can anyone look? > > ?Thanks, > Andrei >> On 19 Jul 2017, at 18:21, Andrey Nazarov wrote: >> >> Hi, >> >> Could you review fix in test library which should help to resolve intermittent issues on Windows machines during directories clean up. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8184961 >> Review: http://cr.openjdk.java.net/~anazarov/JDK-8184961/webrev.00/webrev/ >> >> >> ?Andrei From sgehwolf at redhat.com Thu Jul 27 07:30:07 2017 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Thu, 27 Jul 2017 09:30:07 +0200 Subject: RFR(xs): 8185263: Fix zero build after 8169881, 8175318, 8178350 In-Reply-To: References: Message-ID: <1501140607.3963.0.camel@redhat.com> Hi, On Tue, 2017-07-25 at 19:13 +0200, Thomas St?fe wrote: > Hi all, > > may I please have reviews and sponsor for these zero-related build fixes? > > Issue: https://bugs.openjdk.java.net/browse/JDK-8185263 > webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/8185263-fix-zero-build/webrev.00/webrev/index.html I can confirm Zero builds fine with this for me on x86_64 Linux release/fastdebug/slowdebug variants. Thanks, Severin From aph at redhat.com Thu Jul 27 08:58:58 2017 From: aph at redhat.com (Andrew Haley) Date: Thu, 27 Jul 2017 09:58:58 +0100 Subject: RFR (S) 8182397: Race in field updates when creating ArrayKlasses can lead to crash In-Reply-To: <54910434-DB13-4A65-93F2-A2DAE27BB0C7@oracle.com> References: <807A38E7-692E-4849-9258-75CFCFDA404A@oracle.com> <0eb805af-7ac5-874f-eaa4-33ce1788d427@redhat.com> <5977015B.2080102@oracle.com> <597727D9.4070502@oracle.com> <1043bdc9-53c5-f893-c2e2-338df7b343d8@redhat.com> <59774AA7.8080006@oracle.com> <61341c53-c9e6-fdc1-bfe3-58ddf34ee9fc@redhat.com> <54910434-DB13-4A65-93F2-A2DAE27BB0C7@oracle.com> Message-ID: <731283b3-e4dd-4a59-3134-c2692bef7cd2@redhat.com> On 25/07/17 19:13, Erik ?sterlund wrote: > >> On 25 Jul 2017, at 18:57, Andrew Haley wrote: >> >> On 25/07/17 14:41, Erik ?sterlund wrote: >>> >>> On 2017-07-25 14:42, Andrew Haley wrote: >>>> On 25/07/17 12:13, Erik ?sterlund wrote: >>>>> For example, take this example pseudo code for performing what I refer >>>>> to as a stable load between two fields modified concurrently with >>>>> potential ABA issues: >>>>> >>>>> loop { >>>>> x_start = load_relaxed(field_A) >>>>> y = load_consume(field_B) >>>>> x = load_consume(field_A) >>>>> if (x_start == x) break; >>>>> } >>>>> >>>>> // use x->foo >>>> I don't understand this pseudocode. What is the base address for field_A >>>> and field_B ? >>> >>> field_A and field_B could be two different registers pointing at >>> different addresses - i.e. they are arbitrary pointers. The key in this >>> example is that field_A is reloaded, and then we compare if the reloaded >>> value is equal to the original value (with a possible ABA problem), and >>> stop the loop then. But the original and reloaded value could reside in >>> different registers, and when we continue using x->foo afterwards, the >>> compiler could elect to use either one of the two registers as base >>> pointers in the dereference - either the one from the reloaded value of >>> field_A or for the original value, as they are equal to each other. >> >> OK, I see what you're getting at. Compilers have to be pretty >> smart to make consume work properly. > > Precisely. I've been thinking about this some more, and I think this example does not have a problem after all. loop { x_start = load_relaxed(field_A) y = load_consume(field_B) x = load_consume(field_A) if (x_start == x) break; } // use x->foo It is true that if the compiler uses x_start instead of x to load x->foo then there is no data dependency between the load of x and the use of x->foo. However, this does not matter, because the load of x->foo is *control dependent* on load_consume(field_A), and this is enough to keep things in order. Sure, this would fail on some mis-specified architecture which recognizes data dependencies but not control dependencies, but that's their problem and they'll have to use load acquire until their hardware spec is fixed. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From erik.osterlund at oracle.com Thu Jul 27 12:56:54 2017 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 27 Jul 2017 14:56:54 +0200 Subject: RFR (S) 8182397: Race in field updates when creating ArrayKlasses can lead to crash In-Reply-To: <731283b3-e4dd-4a59-3134-c2692bef7cd2@redhat.com> References: <807A38E7-692E-4849-9258-75CFCFDA404A@oracle.com> <0eb805af-7ac5-874f-eaa4-33ce1788d427@redhat.com> <5977015B.2080102@oracle.com> <597727D9.4070502@oracle.com> <1043bdc9-53c5-f893-c2e2-338df7b343d8@redhat.com> <59774AA7.8080006@oracle.com> <61341c53-c9e6-fdc1-bfe3-58ddf34ee9fc@redhat.com> <54910434-DB13-4A65-93F2-A2DAE27BB0C7@oracle.com> <731283b3-e4dd-4a59-3134-c2692bef7cd2@redhat.com> Message-ID: <5979E316.3040009@oracle.com> Hi Andrew, On 2017-07-27 10:58, Andrew Haley wrote: > On 25/07/17 19:13, Erik ?sterlund wrote: >>> On 25 Jul 2017, at 18:57, Andrew Haley wrote: >>> >>> On 25/07/17 14:41, Erik ?sterlund wrote: >>>> On 2017-07-25 14:42, Andrew Haley wrote: >>>>> On 25/07/17 12:13, Erik ?sterlund wrote: >>>>>> For example, take this example pseudo code for performing what I refer >>>>>> to as a stable load between two fields modified concurrently with >>>>>> potential ABA issues: >>>>>> >>>>>> loop { >>>>>> x_start = load_relaxed(field_A) >>>>>> y = load_consume(field_B) >>>>>> x = load_consume(field_A) >>>>>> if (x_start == x) break; >>>>>> } >>>>>> >>>>>> // use x->foo >>>>> I don't understand this pseudocode. What is the base address for field_A >>>>> and field_B ? >>>> field_A and field_B could be two different registers pointing at >>>> different addresses - i.e. they are arbitrary pointers. The key in this >>>> example is that field_A is reloaded, and then we compare if the reloaded >>>> value is equal to the original value (with a possible ABA problem), and >>>> stop the loop then. But the original and reloaded value could reside in >>>> different registers, and when we continue using x->foo afterwards, the >>>> compiler could elect to use either one of the two registers as base >>>> pointers in the dereference - either the one from the reloaded value of >>>> field_A or for the original value, as they are equal to each other. >>> OK, I see what you're getting at. Compilers have to be pretty >>> smart to make consume work properly. >> Precisely. > I've been thinking about this some more, and I think this example does > not have a problem after all. > > loop { > x_start = load_relaxed(field_A) > y = load_consume(field_B) > x = load_consume(field_A) > if (x_start == x) break; > } > > // use x->foo > > It is true that if the compiler uses x_start instead of x to load > x->foo then there is no data dependency between the load of x and the > use of x->foo. However, this does not matter, because the load of > x->foo is *control dependent* on load_consume(field_A), and this is > enough to keep things in order. > > Sure, this would fail on some mis-specified architecture which > recognizes data dependencies but not control dependencies, but that's > their problem and they'll have to use load acquire until their > hardware spec is fixed. > The problem I described was that data dependent loads on source level are not necessarily mapped to data dependent loads in generated machine code, because that is what we were talking about. I am glad we seem to agree about this. Now whether that is a problem or not on a given architecture or not is a totally different question. I see you are arguing that it shouldn't be and that if it is, then that is their problem. I don't know if I see it the same way. At the very least, I think we agree that we can not leave normal loads around without at the very least wrapping them in an API with some well defined semantics that at the very least allows certain platforms to choose what to do in such a scenario. Having said that, I do believe that e.g. PPC requires an isync to respect that loads before a conditional branch are consistent with loads after the branch, unless there is a data dependency between a load before the branch and a load after the branch to force the issue across that branch. And in this case there is not. Because it is totally fine to speculate across the branch unless a data dependency ties loads before the branch with loads after the branch. Then once the branch is committed and x_start == x as speculated, the already calculated outcome of that branch is perfectly valid unless either isync prevented such speculation or an explicit data dependency made the speculation illegal. That is precisely why some implementations of load acquire on PPC use ld; cmp; bc; isync instead of ld; lwsync to force the load before the conditional branch to complete first. Because it would certainly not be forced to complete unless isync was there, and therefore not data dependent loads would not have to be consistent. In a very similar fashion, I believe that at least according to specification (not necessarily implementations of it - don't know about that), ARMv7 will similarly require an isb after that conditional branch. Thanks, /Erik From coleen.phillimore at oracle.com Thu Jul 27 13:19:43 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 27 Jul 2017 09:19:43 -0400 Subject: RFR (L but tedious) 8183039: Re-examine methodHandle methods uninlined by 8144256 Message-ID: <09dd7fda-80f0-1060-8516-7b8e96039868@oracle.com> Summary: Add more const references so out-of-line methodHandle destructor and copy ctor called infrequently See RFE for more details. In short, use const methodHandle& when possible. open webrev at http://cr.openjdk.java.net/~coleenp/8183039.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8183039 Tested with RBT on linux, solaris and windows (still running) with tier2 testing and all tier1-5 tests on linux x64. Thanks, Coleen From harold.seigel at oracle.com Thu Jul 27 14:16:13 2017 From: harold.seigel at oracle.com (harold seigel) Date: Thu, 27 Jul 2017 10:16:13 -0400 Subject: RFR (L but tedious) 8183039: Re-examine methodHandle methods uninlined by 8144256 In-Reply-To: <09dd7fda-80f0-1060-8516-7b8e96039868@oracle.com> References: <09dd7fda-80f0-1060-8516-7b8e96039868@oracle.com> Message-ID: <7e66f126-b2e6-d5e6-6eab-fd235a6af9e1@oracle.com> Hi Coleen, The changes look good! One small nit, in rewriter.cpp, Rewriter::rewrite_jsrs(), could this: + methodHandle new_method = method; + new_method = romc.do_potential_rewrite(CHECK_(methodHandle())); just be: + methodHandle new_method = romc.do_potential_rewrite(CHECK_(methodHandle())); No need for a new webrev. Thanks, Harold On 7/27/2017 9:19 AM, coleen.phillimore at oracle.com wrote: > Summary: Add more const references so out-of-line methodHandle > destructor and copy ctor called infrequently > > See RFE for more details. In short, use const methodHandle& when > possible. > > open webrev at http://cr.openjdk.java.net/~coleenp/8183039.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8183039 > > Tested with RBT on linux, solaris and windows (still running) with > tier2 testing and all tier1-5 tests on linux x64. > > Thanks, > Coleen From claes.redestad at oracle.com Thu Jul 27 14:35:18 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 27 Jul 2017 16:35:18 +0200 Subject: RFR (L but tedious) 8183039: Re-examine methodHandle methods uninlined by 8144256 In-Reply-To: <09dd7fda-80f0-1060-8516-7b8e96039868@oracle.com> References: <09dd7fda-80f0-1060-8516-7b8e96039868@oracle.com> Message-ID: <698071a3-ed67-14e9-888e-9ff912333516@oracle.com> Hi, this reduces number of invocations of the copy constructor by ~87% on a hello world, adding up to a small but measurable improvement to startup. The gain is similar to what we would get from inlining the methods again, but without any of the static footprint increase. Good! In classFileParser.cpp you change a constantPoolHandle to const constantPoolHandle&, which is likely OK but seems somewhat out of place for this patch. Rest looks good. Thanks! /Claes On 07/27/2017 03:19 PM, coleen.phillimore at oracle.com wrote: > Summary: Add more const references so out-of-line methodHandle > destructor and copy ctor called infrequently > > See RFE for more details. In short, use const methodHandle& when > possible. > > open webrev at http://cr.openjdk.java.net/~coleenp/8183039.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8183039 > > Tested with RBT on linux, solaris and windows (still running) with > tier2 testing and all tier1-5 tests on linux x64. > > Thanks, > Coleen From erik.osterlund at oracle.com Thu Jul 27 14:54:46 2017 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 27 Jul 2017 16:54:46 +0200 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <596CD126.6080100@oracle.com> <596DD713.6080806@oracle.com> <77d5a9e7-b2b0-79e4-effd-903dfe226512@redhat.com> <596E1142.6040708@oracle.com> <3432d11c-5904-a446-1cb5-419627f1794f@redhat.com> <596E288B.90706@oracle.com> <773a0b64-8273-1a58-a46a-22ae18f927b7@redhat.com> <5970B3FD.607@oracle.com> <8e1d9a32-7e01-b917-df1d-595c4000e4b2@redhat.com> <597226E2.6090803@oracle.com> <2c637e28-8d95-cbaf-c29e-d01867ed9157@redhat.com> Message-ID: <5979FEB6.10506@oracle.com> Hi Andrew, Thank you for having another look at this. I do see appeal with allowing platforms to specialize before truncating types. I also recognize that your implementation is simpler. So thank you for your efforts helping out here. However... I do think that the simplification comes at a cost. Since we are now introducing templates that accept any types, I would like to protect the user of the API from doing things that are not a good idea. That was one of the key reasons why the IntegerTypes casting mechanisms were employed. To safely cast and sanity check the passed in types to make sense. Take for example a user passing in to your cmpxchg method a new_value of float, a destination of volatile int*, and an expected value of float. This clearly makes no sense, and I would like the Atomic API to not allow passing in things that clearly do not make sense. The proposed mechanism will explicitly convert the expected and new values to ints, which will truncate their values under the hood, resulting in a different value being written to what was expected. If the assembly used a load link + store conditional pair like e.g. PPC linux does, then the passed in truncated expected value will lead to false negatives where the CAS fails due to the passed in expected value being seen due to being truncated in the explicit conversion, and similarly it will lead to false positives where a CAS succeeds despite not having the expected value in memory. Similarly, I believe similar problems may apply when different integer types of with different sizes are passed in that would not be implicitly convertible, but sure are explicitly convertible (with possibly unintentional truncation). You mentioned the case of extending a 32 bit integer to a 64 bit integer, which indeed is fine. Unfortunately, the reverse is not fine yet possible in this API. For example, if the destination is of type volatile jint* and the passed in expected value is a jlong, then that is clearly wrong. Yet the API will accept these types and explicitly convert the jlong to jint, truncating it in the process without warning. This is the kind of thing that the casting mechanism me and Kim worked on prevents you from doing accidentally. So while my solution truncated type information that this solution does not, it seems like this solution would instead silently truncate values passed in where my solution would not, making it less safe to use. Only at runtime would you (if lucky) find out that the types used were not compatible, and silently messed up your data. All in all, I do like the idea of allowing platforms that do not want to rely on -fno-strict-aliasing to keep as much type information as possible, and give it to the compiler. Having said that, I do think the API should protect users from messing up instead of silently accepting not compliant types and truncating their values. How would you feel about a mechanism where we allow each platform to specialize on the top level with all type information available, but by default send everything in to the Atomic implementation I have proposed that canonicalizes and sanity checks all values. Then we have solved a bunch of existing problems, not introduced any new problems, yet are future proof for allowing arbitrary specialization on each platform to allow further improving the solution on platforms that have cooperative compilers that allow such improvements, on a platform by platform basis. What do you think? Thanks, /Erik On 2017-07-26 15:22, Andrew Haley wrote: > I've been sketching out an alternative way to handle this. I've got > my flak jacket ready, so here goes... > > My goals are: > > Simplicity; with this comes, hopefully, clarity. Within reason, we > should minimize the extra code this brings in. > > Type safety where possible. I'm aware that we compile HotSpot with > -fno-strict-aliasing or its equivalent, but I think we can do this > job without relying on that. The more information we expose to the > compiler, the better code it can generate. > > Something that plays nicely with compiler builtins, where > appropriate. It must also be able to handle legacy > assembly-language definitions. > > [ Correctness, hopefully, goes without saying. ] > > I've done all the work on cmpxchg, because that is the most complex of > the operations: I believe that if cmpxchg can work well, the rest will > be relatively simple. > > ---------------------------------------------------------------------- > > We should expose the type information of the original operands all the > way to the back ends, and not cast pointer types if we can possibly > avoid it. So, in this design every back end defines a template function > to match this declaration: > > template > inline static U cmpxchg(T exchange_value, volatile U* dest, V compare_value, > cmpxchg_memory_order order = memory_order_conservative); > > [ Note: the types of the operand, the compare value, and the exchange > value are distinct. This is a design decision I took from Erik's > patch, but I don't like it: it seems to me that we should insist on > the same types for the operands. This requires a few changes in > HotSpot, but arguably these are bugs anyway. I tried this, and had to > fix eight places, all trivial. We should do it as a cleanup. ] > > First, the simplest possible case. In some modern GCC targets (e.g > Aarch64) we can trivially define our generic cmpxchg using a > compiler builtin, like so: > > template > inline U Atomic::cmpxchg(T exchange_value, volatile U* dest, V compare_value, > cmpxchg_memory_order order) { > return __sync_val_compare_and_swap(dest, compare_value, exchange_value); > } > > That's everything we need, although I'd do some optimization for > the memory_order. > > With older targets and those where we need to support assembly > language, we'll have to use some template magic to select an > appropriate assembly fragment: > > template > inline U Atomic::cmpxchg(T exchange_value, volatile U* dest, V compare_value, > cmpxchg_memory_order order) { > return CmpxchgHelper()(exchange_value, dest, > compare_value, order); > } > > Here's one such fragment, for the 64-bit case: > > template > struct Atomic::CmpxchgHelper { > U operator()(T exchange_value, volatile U* dest, V compare_value, > cmpxchg_memory_order order) { > U result; > __asm__ __volatile__ ("lock cmpxchgq %1,(%3)" > : "=a" (result) > : "r" ((U)exchange_value), "a" ((U)compare_value), "r" (dest) > : "cc", "memory"); > return result; > } > }; > > The other cases are very similar, and I won't describe them here. > > Note that we're using plain casts to convert the compare and exchange > values: this allows trivial cases such as extending a plain integer > constant to a 64-bit type. > > [ Note: The new Helper Classes are necessary because the antique > version of C++ we use to build HotSpot doesn't permit the partial > specialization of templates. This is merely annoying syntax and > doesn't affect the generated code. ] > > Where there are back-end functions that cannot be genericized in this > way (perhaps because they always take a void * or an intptr_t *) we > can do whatever casting is necessary in the back-end fragments. There > doesn't need to be any casting in the shared code. > > This proposal does not handle floating-point operands because none are > currently needed, but I believe that the correct way to do it involves > explicit specializations for the types double and float. These can be > in the shared code in atomic.h if we ever need them. > > [ Note: Supporting floating-point operands is problematic because it's > hard to get the types right. If T, U, and V can all be different > types then we need to know exactly what is meant by cmpxchg() where > the object in memory is an integer type and the compare or exchange > values are of floating-point type. IMO, we should reject nonsense > like that at compile time. Forcing T, U, and V to be the same type > allows us to do that. ] > > ---------------------------------------------------------------------- > > This solution doesn't require any new helper classes. It does, > however, require changes to the back ends which convert methods to > helper classes. This is additional work now, but the result IMO will > be simpler, easier to maintain, and more future proof. > > Do you know of any operating systems, compilers, or targets where this > won't work? Or of any particular difficulty doing this? I know we'll > have to get buy-in from the back end maintainers, but right now in the > JDK 10 release cycle is the time to get things like this done. > From coleen.phillimore at oracle.com Thu Jul 27 15:13:24 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 27 Jul 2017 11:13:24 -0400 Subject: RFR (L but tedious) 8183039: Re-examine methodHandle methods uninlined by 8144256 In-Reply-To: <698071a3-ed67-14e9-888e-9ff912333516@oracle.com> References: <09dd7fda-80f0-1060-8516-7b8e96039868@oracle.com> <698071a3-ed67-14e9-888e-9ff912333516@oracle.com> Message-ID: <9e0b997f-a4ae-a507-14f1-c79352ca0be5@oracle.com> On 7/27/17 10:35 AM, Claes Redestad wrote: > Hi, > > this reduces number of invocations of the copy constructor by ~87% on > a hello world, adding up to a > small but measurable improvement to startup. The gain is similar to > what we would get from inlining > the methods again, but without any of the static footprint increase. > Good! > > In classFileParser.cpp you change a constantPoolHandle to const > constantPoolHandle&, which is likely OK > but seems somewhat out of place for this patch. Rest looks good. Thanks Claes for the performance testing! Since constantPoolHandle is the same as methodHandle fundamentally. The code is defined as macros for the two. I thought that change belonged here too. thanks! Coleen > > Thanks! > > /Claes > > On 07/27/2017 03:19 PM, coleen.phillimore at oracle.com wrote: >> Summary: Add more const references so out-of-line methodHandle >> destructor and copy ctor called infrequently >> >> See RFE for more details. In short, use const methodHandle& when >> possible. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8183039.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8183039 >> >> Tested with RBT on linux, solaris and windows (still running) with >> tier2 testing and all tier1-5 tests on linux x64. >> >> Thanks, >> Coleen > From coleen.phillimore at oracle.com Thu Jul 27 15:16:32 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 27 Jul 2017 11:16:32 -0400 Subject: RFR (L but tedious) 8183039: Re-examine methodHandle methods uninlined by 8144256 In-Reply-To: <7e66f126-b2e6-d5e6-6eab-fd235a6af9e1@oracle.com> References: <09dd7fda-80f0-1060-8516-7b8e96039868@oracle.com> <7e66f126-b2e6-d5e6-6eab-fd235a6af9e1@oracle.com> Message-ID: On 7/27/17 10:16 AM, harold seigel wrote: > Hi Coleen, > > The changes look good! > > One small nit, in rewriter.cpp, Rewriter::rewrite_jsrs(), could this: > > + methodHandle new_method = method; > + new_method = romc.do_potential_rewrite(CHECK_(methodHandle())); > > just be: > > + methodHandle new_method = > romc.do_potential_rewrite(CHECK_(methodHandle())); > > No need for a new webrev. Fixed. Thanks for reviewing! Wondering if this is essentially a trivial change, since I just changed mostly type of parameters. Coleen > > Thanks, Harold > > > On 7/27/2017 9:19 AM, coleen.phillimore at oracle.com wrote: >> Summary: Add more const references so out-of-line methodHandle >> destructor and copy ctor called infrequently >> >> See RFE for more details. In short, use const methodHandle& when >> possible. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8183039.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8183039 >> >> Tested with RBT on linux, solaris and windows (still running) with >> tier2 testing and all tier1-5 tests on linux x64. >> >> Thanks, >> Coleen > From aph at redhat.com Thu Jul 27 15:35:52 2017 From: aph at redhat.com (Andrew Haley) Date: Thu, 27 Jul 2017 16:35:52 +0100 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: <5979FEB6.10506@oracle.com> References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <596CD126.6080100@oracle.com> <596DD713.6080806@oracle.com> <77d5a9e7-b2b0-79e4-effd-903dfe226512@redhat.com> <596E1142.6040708@oracle.com> <3432d11c-5904-a446-1cb5-419627f1794f@redhat.com> <596E288B.90706@oracle.com> <773a0b64-8273-1a58-a46a-22ae18f927b7@redhat.com> <5970B3FD.607@oracle.com> <8e1d9a32-7e01-b917-df1d-595c4000e4b2@redhat.com> <597226E2.6090803@oracle.com> <2c637e28-8d95-cbaf-c29e-d01867ed9157@redhat.com> <5979FEB6.10506@oracle.com> Message-ID: Hi, On 27/07/17 15:54, Erik ?sterlund wrote: > Thank you for having another look at this. I do see appeal with allowing > platforms to specialize before truncating types. I also recognize that > your implementation is simpler. So thank you for your efforts helping > out here. Thank you. You're being very gracious about this. > However... I do think that the simplification comes at a cost. Since we > are now introducing templates that accept any types, I would like to > protect the user of the API from doing things that are not a good idea. Well, I think that was you, not me: it's simpler for everyone if we insist that all of the arguments to these functions are the same type. Some of the callers will have to change, sure. > That was one of the key reasons why the IntegerTypes casting mechanisms > were employed. To safely cast and sanity check the passed in types to > make sense. > > Take for example a user passing in to your cmpxchg method a new_value of > float, a destination of volatile int*, and an expected value of float. Perhaps I wasn't clear enough. I think I implied that that there should be specializations which would do the bitwise mapping from e.g. double to uint64_t and call the generic cmpxchg. I didn't describe these for the sake of brevity and because they aren't used anywhere in HotSpot. > This clearly makes no sense, and I would like the Atomic API to not > allow passing in things that clearly do not make sense. I agree. > The proposed mechanism will explicitly convert the expected and new > values to ints, which will truncate their values under the hood, > resulting in a different value being written to what was > expected. Sure, I get that. > Similarly, I believe similar problems may apply when different > integer types of with different sizes are passed in that would not > be implicitly convertible, but sure are explicitly convertible (with > possibly unintentional truncation). You mentioned the case of > extending a 32 bit integer to a 64 bit integer, which indeed is > fine. Unfortunately, the reverse is not fine yet possible in this > API. For example, if the destination is of type volatile jint* and > the passed in expected value is a jlong, then that is clearly > wrong. Yet the API will accept these types and explicitly convert > the jlong to jint, truncating it in the process without > warning. This is the kind of thing that the casting mechanism me and > Kim worked on prevents you from doing accidentally. OK, I see. But it's a heck of a lot of code just to do that. Instead, we could simply insist that the types must be the same, and fix the callers. This makes everything clearer, including the point where the Atomic API is called. Mismatched types at this level are either unintentional oversights or plain bugs. I have made this change, and it took a few minutes to fix all eight of the callers in HotSpot which have mismatched types. > All in all, I do like the idea of allowing platforms that do not > want to rely on -fno-strict-aliasing to keep as much type > information as possible, and give it to the compiler. Having said > that, I do think the API should protect users from messing up > instead of silently accepting not compliant types and truncating > their values. OK, good. We're on the same page. > How would you feel about a mechanism where we allow each platform to > specialize on the top level with all type information available, but > by default send everything in to the Atomic implementation I have > proposed that canonicalizes and sanity checks all values. Then we > have solved a bunch of existing problems, not introduced any new > problems, yet are future proof for allowing arbitrary specialization > on each platform to allow further improving the solution on > platforms that have cooperative compilers that allow such > improvements, on a platform by platform basis. > > What do you think? It's a better solution, and it is much more well suited to the future, when we can move over to compiler intrinsics. That's really important, IMO: it gives us an escape hatch from the past. But I think it's not really necessary: if we insist on something like template inline static T cmpxchg(T exchange_value, volatile T* dest, T compare_value, cmpxchg_memory_order order = memory_order_conservative); then all of the casts go away because they're unneeded. We'd have e.g. template struct Atomic::CmpxchgHelper { T operator()(T exchange_value, volatile T* dest, T compare_value, cmpxchg_memory_order order) { T result; __asm__ __volatile__ ("lock cmpxchgq %1,(%3)" : "=a" (result) : "r" (exchange_value), "a" (compare_value), "r" (dest) : "cc", "memory"); return result; } }; (Look ma, no casts!) However, this would certainly fail to compile if anyone passed a double. To handle floating-point operands I would do one of two things: Create explicit specializations for doubles and floats: these do bitwise copies into integer values when calling the assembly-language sinippets, or Make all of the snippets copy their arguments bitwise into integer values and back. I believe that the best-supported way of doing that is the "union trick", which is widely supported by C++ compilers, even when using strict aliasing. Andrew. From erik.osterlund at oracle.com Thu Jul 27 17:27:52 2017 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Thu, 27 Jul 2017 19:27:52 +0200 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <596CD126.6080100@oracle.com> <596DD713.6080806@oracle.com> <77d5a9e7-b2b0-79e4-effd-903dfe226512@redhat.com> <596E1142.6040708@oracle.com> <3432d11c-5904-a446-1cb5-419627f1794f@redhat.com> <596E288B.90706@oracle.com> <773a0b64-8273-1a58-a46a-22ae18f927b7@redhat.com> <5970B3FD.607@oracle.com> <8e1d9a32-7e01-b917-df1d-595c4000e4b2@redhat.com> <597226E2.6090803@oracle.com> <2c637e28-8d95-cbaf-c29e-d01867ed9157@redhat.com> <5979FEB6.10506@oracle.com> Message-ID: <3C8BDDFA-1044-47F6-B15C-6DE2085ACF7C@oracle.com> Hi Andrew, > On 27 Jul 2017, at 17:35, Andrew Haley wrote: > > Hi, > >> On 27/07/17 15:54, Erik ?sterlund wrote: >> >> Thank you for having another look at this. I do see appeal with allowing >> platforms to specialize before truncating types. I also recognize that >> your implementation is simpler. So thank you for your efforts helping >> out here. > > Thank you. You're being very gracious about this. > >> However... I do think that the simplification comes at a cost. Since we >> are now introducing templates that accept any types, I would like to >> protect the user of the API from doing things that are not a good idea. > > Well, I think that was you, not me: it's simpler for everyone if we > insist that all of the arguments to these functions are the same type. > Some of the callers will have to change, sure. I did indeed introduce the ability to match any types, but with applied sanity checks protecting users from being able to compile if incompatible types are passed in. I did try earlier on to see what would happen if we forced the types to be identical as you propose. That turned out to be too strict. It made it impossible to pass in perfectly fine literals like NULL or 0 where it should be fine (among other things). This would force too many call sites to have to be changed to explicitly cast passed in values to types that they could perfectly safely convert to implicitly. Therefore I decided against it. > >> That was one of the key reasons why the IntegerTypes casting mechanisms >> were employed. To safely cast and sanity check the passed in types to >> make sense. >> >> Take for example a user passing in to your cmpxchg method a new_value of >> float, a destination of volatile int*, and an expected value of float. > > Perhaps I wasn't clear enough. I think I implied that that there > should be specializations which would do the bitwise mapping from > e.g. double to uint64_t and call the generic cmpxchg. I didn't > describe these for the sake of brevity and because they aren't used > anywhere in HotSpot. Okay, sorry I did not catch that. > >> This clearly makes no sense, and I would like the Atomic API to not >> allow passing in things that clearly do not make sense. > > I agree. > >> The proposed mechanism will explicitly convert the expected and new >> values to ints, which will truncate their values under the hood, >> resulting in a different value being written to what was >> expected. > > Sure, I get that. > >> Similarly, I believe similar problems may apply when different >> integer types of with different sizes are passed in that would not >> be implicitly convertible, but sure are explicitly convertible (with >> possibly unintentional truncation). You mentioned the case of >> extending a 32 bit integer to a 64 bit integer, which indeed is >> fine. Unfortunately, the reverse is not fine yet possible in this >> API. For example, if the destination is of type volatile jint* and >> the passed in expected value is a jlong, then that is clearly >> wrong. Yet the API will accept these types and explicitly convert >> the jlong to jint, truncating it in the process without >> warning. This is the kind of thing that the casting mechanism me and >> Kim worked on prevents you from doing accidentally. > > OK, I see. But it's a heck of a lot of code just to do that. Perhaps. But IntegerTypes is a general reusable tool that might find further use later on. > Instead, we could simply insist that the types must be the same, and > fix the callers. This makes everything clearer, including the point > where the Atomic API is called. Mismatched types at this level are > either unintentional oversights or plain bugs. I have made this > change, and it took a few minutes to fix all eight of the callers > in HotSpot which have mismatched types. I think I already covered why this was not done. Also, that is also a "heck of a lot of code" invested in being able to reduce the anount of code in Atomic. I think you will come to the same conclusion once you apply that pattern to all atomic operations as well as OrderAccess (coming up after we are done here as it builds on Atomic). > >> All in all, I do like the idea of allowing platforms that do not >> want to rely on -fno-strict-aliasing to keep as much type >> information as possible, and give it to the compiler. Having said >> that, I do think the API should protect users from messing up >> instead of silently accepting not compliant types and truncating >> their values. > > OK, good. We're on the same page. Excellent. > >> How would you feel about a mechanism where we allow each platform to >> specialize on the top level with all type information available, but >> by default send everything in to the Atomic implementation I have >> proposed that canonicalizes and sanity checks all values. Then we >> have solved a bunch of existing problems, not introduced any new >> problems, yet are future proof for allowing arbitrary specialization >> on each platform to allow further improving the solution on >> platforms that have cooperative compilers that allow such >> improvements, on a platform by platform basis. >> >> What do you think? > > It's a better solution, and it is much more well suited to the future, > when we can move over to compiler intrinsics. That's really > important, IMO: it gives us an escape hatch from the past. Okay. I am currently on my way home, but here is an untested prototype: http://cr.openjdk.java.net/~eosterlund/8184334/webrev.02/ This variant allows you to declare an AllStatic PlatformAtomic class in your favourite platfoem file that it should automatically pick up (unless I messed up - I have not tried it yet) as an override for AtomicImpl which Atomic forwards all operations to too. This allows overriding Atomic completely on a per platform basis. That is, when no such PlatformAtomic declaration is found from the platform specific files, there is only a forward declaration of PlatformAtomic, which is detected, and the AtomicImpl will redirect its type to GeneralizedAtomic instead, which canonicalizes the types safely according to current mechanisms. Now this webrev is only a sneak preview of that concept for quicker turnaround of the idea. If we decide we like the concept, then I will send out a new webrev that has been tested and fixed indentation issues that I know are present after the mechanical change from Atomic:: to GeneralizedAtomic:: in platform specific files. > But I think it's not really necessary: if we insist on something like > > template > inline static T cmpxchg(T exchange_value, volatile T* dest, T compare_value, > cmpxchg_memory_order order = memory_order_conservative); > > then all of the casts go away because they're unneeded. We'd have > e.g. > > template > struct Atomic::CmpxchgHelper { > T operator()(T exchange_value, volatile T* dest, T compare_value, > cmpxchg_memory_order order) { > T result; > __asm__ __volatile__ ("lock cmpxchgq %1,(%3)" > : "=a" (result) > : "r" (exchange_value), "a" (compare_value), "r" (dest) > : "cc", "memory"); > return result; > } > }; > > (Look ma, no casts!) This does indeed not use casts. However, I believe that would either suffer from the problem I mentioned earlier about not accepting perfectly reasonable literals eithout explicit casting, propagating to a lot of call sites and being generally annoying, or miss e.g. required sign extension in places if the types are allowed to be different and passed straight into the inline assembly. > However, this would certainly fail to compile if anyone passed a > double. To handle floating-point operands I would do one of two > things: > > Create explicit specializations for doubles and floats: these do > bitwise copies into integer values when calling the assembly-language > sinippets, or > > Make all of the snippets copy their arguments bitwise into integer > values and back. I believe that the best-supported way of doing that > is the "union trick", which is widely supported by C++ compilers, even > when using strict aliasing. That makes sense. Unfortunately though, that would still violate section 3.10 and remain undefined behaviour. That is, a double field could be initialized as 0.0, and then be loaded as an int64_t through e.g. Atomic::xchg. It would be unfortunate if a mechanism was built with the sole purpose of cutting reliance on 3.10 if it does not actually solve that problem. But I am happy to defer that discussion about a possible new specialization to when we actually get there. But perhaps we could do that in a subsequent RFR. Hope we are converging! Thanks, /Erik > Andrew. From ecki at zusammenkunft.net Mon Jul 17 21:57:39 2017 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Mon, 17 Jul 2017 21:57:39 +0000 Subject: JEP [DRAFT]: Container aware Java In-Reply-To: References: , Message-ID: For a virtualized solution you can use the normal methods for getting the number of virtual CPUs or the RAM size (and I am quite sure nobody expects auto tuning based on host resources). JVM already does that. This fails for soft partitioning, especially cgroups, CPU sets (and NUMA Zones). I am not sure how important auto tuning is, but it does make sense to observe soft partitioning limits. None of them are Docker specific, however. Besides the maximum heap it also affects parallelity for GC and the default fork/join pool. Anything else in scope for this JEP? Gruss Bernd -- http://bernd.eckenfels.net ________________________________ From: core-libs-dev on behalf of Volker Simonis Sent: Monday, July 17, 2017 5:55:53 PM To: Bob Vandette Cc: hotspot-dev Source Developers; core-libs-dev; Baesken, Matthias Subject: Re: JEP [DRAFT]: Container aware Java On Fri, Jul 14, 2017 at 6:54 PM, Bob Vandette wrote: > >> On Jul 14, 2017, at 12:15 PM, Volker Simonis wrote: >> >> Hi Bob, >> >> thanks for starting this JEP. I think it is long overdue and will be >> very helpful for many people. >> > Thanks for taking a look at the JEP. > >> That said, I don't see any mentioning of the targeted Java release. As >> JEPs usually go into the development version first, I suppose the work >> for this JEP will be done in JDK10. However, taking into account the >> current release cycles, the usefulness of this JEP for the Java >> community will significantly depend on the its availability in Java 9 >> (at least) and Java 8 (hopefully?). Any thoughts on this? > > I am certainly going to do my best to try to get this feature into JDK10 and > I understand your concern about supporting Java 8 and 9. The decision to > backport this feature will have to be done based on the usual criteria (complexity, > usefulness, resource availability and compatibility). It may be possible to > add some subset of this functionality in a backport but we?ll have to wait until > the design is further along. > >> >> You also mention that this JEP is only about Docker support on >> Linux/x86_64. I'd argue that we should at least address at least all >> Docker supported Linux platforms. I'm pretty sure our colleagues from >> IBM would like to support this on Linux/ppc64 and Linux/s390 as well. > > Right now I don?t expect to have to write CPU specific logic. > Therefore supporting ppc64, s390 or even aarch64 should boil down to testing. > >> >> Moreover, the suggested API and the new JVM_xxxxxx functions should be >> at least designed in such a way that other virtualisation environment >> such as VMWare/VirtualBox/Xen/KVM can take advantage of it. I'm also >> not sure if a completely new API solely for Docker containers is >> making sense and is worth the effort? I understand that containers and >> virtualisation are two different things, but they both impose similar >> restrictions to the VM (i.e. virtualisation may present a huge number >> of CPUs to the OS/VM which in reality are mapped to a single, physical >> CPU) so maybe we should design the new API in such a way that it can >> be used to tackle both problems. > > I will try to keep that in mind. I will have to look into the feasibility of accessing > host configuration from within a container or virtual machine. For containers, > it seems to me that would break the intent of isolation. Did you wanted to say "for virtualisation, it seems to me that would break the intent of isolation." here? So you're right, that you can not easily query the host/guest information in a virtualised environment (like for examples VmWare) because that would break isolation. But as far as I know, every virtualisation solution has it's own library/API which can be used to query the host (if there is any) and get information about the host system and the resources allocated for the guest. We've done some basic virtualisation detection for common systems like VmWare or Xen (cc'ed Matthias who may know more details. > I know that host information > does leak into containers through the proc file system but I suspect this was done > for simplicity and compatibility reasons. > >> >> Finally, you mention in section D. that all the information should be >> logged the to error crash logs. That's good but maybe you could add >> that all the detection and all the findings should also be made >> available through the new Unified JVM Logging facility (JEP 158)? > > That?s a good idea. I will add that to the JEP. > > Thanks, > Bob. > >> >> Regards, >> Volker >> >> >> On Fri, Jul 14, 2017 at 4:22 PM, Bob Vandette wrote: >>> I?d like to propose the following JEP that will enhance the Java runtime to be more container aware. >>> This will allow the Java runtime to performance better ergonomics and provide better >>> execution time statistics when running in a container. >>> >>> >>> JEP Issue: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8182070 >>> >>> Here?s a Text dump of the JEP contents for your convenience: >>> >>> >>> Summary >>> ------- >>> >>> Container aware Java runtime >>> >>> Goals >>> ----- >>> >>> Enhance the JVM and Core libraries to detect running in a container and adapt to the system resources available to it. This JEP will only support Docker on Linux-x64 although the design should be flexible enough to allow support for other platforms and container technologies. The initial focus will be on Linux low level container technology such as cgroups so that we will be able to easily support other container technologies running on Linux in addition to Docker. >>> >>> Non-Goals >>> --------- >>> >>> It is not a goal of this JEP to support any platform other than Docker container technology running on Linux x64. >>> >>> Success Metrics >>> --------------- >>> >>> Success will be measured by the improved efficiency of running multiple Java containers on a host system with out of the box options. >>> >>> Motivation >>> ---------- >>> >>> Container technology is becoming more and more prevalent in Cloud based applications. This technology provides process isolation and allows the platform vendor to specify limits and alter the behavior of a process running inside a container that the Java runtime is not aware of. This causes the Java runtime to potentially attempt to use more system resources than are available to it causing performance degradation or even termination. >>> >>> >>> Description >>> ----------- >>> >>> This enhancement will be made up of the following work items: >>> >>> A. Detecting if Java is running in a container. >>> >>> The Java runtime, as well as any tests that we might write for this feature, will need to be able to detect that the current Java process is running in a container. I propose that we add a new JVM_ native function that returns a boolean true value if we are running inside a container. >>> >>> JVM_InContainer will return true if the currently running process is running in a container, otherwise false will be returned. >>> >>> B. Exposing container resource limits and configuration. >>> >>> There are several configuration options and limits that can be imposed upon a running container. Not all of these >>> are important to a running Java process. We clearly want to be able to detect how many CPUs have been allocated to our process along with the maximum amount of memory that we be allocated but there are other options that we might want to base runtime decisions on. >>> >>> In addition, since Container typically impose limits on system resources, they also provide the ability to easily access the amount of consumption of these resources. I intent on providing this information in addition to the configuration data. >>> >>> I propose adding a new jdk.internal.Platform class that will allow access to this information. Since some of this information is needed during the startup of the VM, I propose that much of the implementation of the methods in the Platform class be done in the VM and exposed as JVM_xxxxxx functions. In hotspot, the JVM_xxxxxx function will be implemented via the os.hpp interface. >>> >>> Here are the categories of configuration and consumption statistics that will be made available (The exact API is TBD): >>> >>> isContainerized >>> Memory Limit >>> Total Memory Limit >>> Soft Memory Limit >>> Max Memory Usage >>> Current Memory Usage >>> Maximum Kernel Memory >>> CPU Shares >>> CPU Period >>> CPU Quote >>> Number of CPUs >>> CPU Sets >>> CPU Set Memory Nodes >>> CPU Usage >>> CPU Usage Per CPU >>> Block I/O Weight >>> Block I/O Device Weight >>> Device I/O Read Rate >>> Device I/O Write Rate >>> OOM Kill Enabled >>> OOM Score Adjustment >>> Memory Swappiness >>> Shared Memory Size >>> >>> TODO: >>> 1. Need to specify the exact arguments and return format for these accessor functions. >>> >>> C. Adjusting Java runtime configuration based on limits. >>> >>> Java startup normally queries the operating system in order to setup runtime defaults for things such as the number of GC threads and default memory limits. When running in a container, the operating system functions used provide information about the host and does not include the containers configuration and limits. The VM and core libraries will be modified as part of this JEP to first determine if the current running process is running in a container. It will then cause the runtime to use the container values rather than the general operating system functions for configuring and managing the Java process. There have been a few attempts to correct some of these issue in the VM but they are not complete. The CPU detection in the VM currently only handles a container that limits cpu usage via CPU sets. If the Docker --cpu or --cpu-period along with --cpu-quota options are specified, it currently has no effect on the VMs configuration. >>> >>> The experimental memory detection that has been implemented only impacts the Heap selection and does not apply to the os::physical_memory or os::available_memory low level functions. This leaves other parts of the VM and >>> core libraries to believe there is more memory available than there actually is. >>> >>> The Numa support available in the VM is also not correct when running in a container. The number of available memory nodes and enabled nodes as reported by the libnuma library does not take into account the impact of the Docker --cpuset-mems option which restricts which memory nodes the container can use. Inside the container, the file /proc/{pid}/self does report the correct Cpus_allowed and Mems_Allowed but libnuma doesn't respect this. This has been verified via the numactl utility. >>> >>> To correct these shortcomings and make this support more robust, here's a list of the current cgroup subsystems that we be examined in order to update the internal VM and core library configuration. >>> >>> **Number of CPUs** >>> >>> Use a combination of number_of_cpus() and cpu_sets() in order to determine how many processors are available to the process and adjust the JVMs os::active_processor_count appropriately. The number_of_cpus() will be calculated based on the cpu_quota() and cpu_period() using this formula: number_of_cpus() = cpu_quota() / cpu_period(). Since it's not currently possible to understand the relative weight of the running container against all other containers, altering the cpu_shares of a running container will have no affect on altering Java's configuration. >>> >>> **Total available memory** >>> >>> Use the memory_limit() value from the cgroup file system to initialize the os::physical_memory() value in the VM. This value will propagate to all other parts of the Java runtime. >>> >>> We might also consider examining the soft_memory_limit and total_memory_limit in addition to the memory_limit during the ergonomics startup processing in order to fine tuning some of the other VM settings. >>> >>> **CPU Memory Nodes** >>> >>> Use cpu_set_memory_nodes() to configure the os::numa support. >>> >>> **Memory usage** >>> >>> Use memory_usage_in_bytes() for providing os::available_memory() by subtracting the usage from the total available memory allocated to the container. >>> >>> D. Adding container configuration to error crash logs. >>> >>> As as troubleshooting aid, we will dump any available container statistics to the hotspot error log. >>> >>> E. Adding a startup flag to enable/disable this support. >>> >>> Add a -XX:+UseContainerSupport VM option that will be used to enable this support. The default will be off until this feature is proven. >>> >>> Alternatives >>> ------------ >>> >>> There are a few existing RFE's filed that could be used to enhance the current experimental implementation rather than taking the JEP route. >>> >>> Testing >>> ------- >>> >>> Docker/container specific tests should be added in order to validate the functionality being provided with this JEP. >>> >>> Risks and Assumptions >>> --------------------- >>> >>> Docker is currently based on cgroups v1. Cgroups v2 is also available but is incomplete and not yet supported by Docker. It's possible that v2 could replace v1 in an incompatible way rendering this work unusable until it is upgraded. >>> >>> Dependencies >>> ----------- >>> >>> None at this time. >>> > From hzzengyijie at corp.netease.com Wed Jul 5 09:20:57 2017 From: hzzengyijie at corp.netease.com (zengyijie) Date: Wed, 05 Jul 2017 09:20:57 -0000 Subject: Where in OpenJDK repository can I find Hotspot VM source codes? Message-ID: <65511B52-BA7A-4F1B-AB3B-4217E25499FB@corp.netease.com> Hi, we are running an outdated Oracle Java JDK6 with Hotspot VM, I have spent quite a while browsing openjdk repository, but I can?t find the source codes for the specific Hotspot VM version. The repository tip of http://hg.openjdk.java.net/jdk6/jdk6/hotspot/ is only jdk6-b43, there?s no 1.6.0_45. Could someone help tell where to find it? The Oracle JDK version is as follows: java version "1.6.0_45" Java(TM) SE Runtime Environment (build 1.6.0_45-b06) Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01, mixed mode) From rednaxelafx at gmail.com Thu Jul 27 18:44:13 2017 From: rednaxelafx at gmail.com (Krystal Mok) Date: Thu, 27 Jul 2017 11:44:13 -0700 Subject: Where in OpenJDK repository can I find Hotspot VM source codes? In-Reply-To: <65511B52-BA7A-4F1B-AB3B-4217E25499FB@corp.netease.com> References: <65511B52-BA7A-4F1B-AB3B-4217E25499FB@corp.netease.com> Message-ID: Hi Yijie, I don't think there is one that exactly matches what you want. I wrote a doc [1] long ago about the correspondence of Sun/Oracle JDK 6 versus OpenJDK 6 in terms of the JVM used. Unfortunately, because of the history behind OpenJDK 6 (it's a backport of OpenJDK 7 to be a JDK6 release, rather than the open sourced version of Sun/Oracle JDK 6), there was no exact correspondence to begin with. Back then, the HotSpot Expression release model was still in place, so the Oracle JDK 6 and the then still-in-development JDK 7 would mostly share the same JVM, that's how the OpenJDK 6 HotSpot VM still very closely match the Oracle JDK 6 version -- less some proprietary features like UseCompressedStrings, and a bit of delay in releasing the new tiered compilation features, etc. After Oracle JDK6u32 or something, I don't think the OpenJDK 6 HotSpot VM reflected changes to the Oracle JDK 6 any more, as JDK6 was going towards EOL'd and less changes from Oracle JDK 6 were sync'd to OpenJDK 6. Red Hat later took over the maintenance of the OpenJDK 6 release, and it further diverged from the Oracle internal one. What's in Oracle JDK6u45 specifically that you wanted to check the sources for, though? - Kris [1]: https://gist.github.com/rednaxelafx/925323 On Wed, Jul 5, 2017 at 2:18 AM, zengyijie wrote: > Hi, we are running an outdated Oracle Java JDK6 with Hotspot VM, I have > spent quite a while browsing openjdk repository, > but I can?t find the source codes for the specific Hotspot VM version. The > repository tip of http://hg.openjdk.java.net/jdk6/jdk6/hotspot/ < > http://hg.openjdk.java.net/jdk6/jdk6/hotspot/> is only jdk6-b43, there?s > no 1.6.0_45. > Could someone help tell where to find it? > The Oracle JDK version is as follows: > > java version "1.6.0_45" > Java(TM) SE Runtime Environment (build 1.6.0_45-b06) > Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01, mixed mode) From jiangli.zhou at oracle.com Fri Jul 28 01:06:28 2017 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Thu, 27 Jul 2017 18:06:28 -0700 Subject: RFR (L) 7133093: Improve system dictionary performance In-Reply-To: References: <89f0b98c-3cbd-7b87-d76b-5e89a5a676fb@oracle.com> <9243da7c-a8e5-4024-c12e-cba9d4f424c8@oracle.com> <7814C032-0D4E-4E76-B8AD-62A072E8A625@oracle.com> <233c03e0-8958-7676-8e33-5d8f0ec4edf3@oracle.com> <7adf48f1-5908-aeaf-c409-d930895e1715@oracle.com> Message-ID: <4ABB23CC-844C-437E-9416-957DCA525533@oracle.com> Hi Coleen, The changes look really good to me. I have one minor question/comment. - src/share/vm/classfile/classLoaderData.cpp Line 468, in try_get_next_class(), if the ?_current_loader_data? reaches the end, it restarts from the ClassLoaderDataGraph::_head (line 490). So a caller could see the same klass being returned from try_get_next_class(). Is that an intended behavior? Also, if none of the loader data contains a ?loaded' class, it could loop forever. In reality, it probably would never happen. Thanks, Jiangli > On Jul 21, 2017, at 11:28 AM, coleen.phillimore at oracle.com wrote: > > > From internal discussions, the only sizing of the dictionary for the java.system.class.loader will follow PredictedLoadedClassCount until dictionary resizing is done. The latest full webrev is here: > > open webrev at http://cr.openjdk.java.net/~coleenp/7133093.04/webrev > > The only change from .03 is > > http://cr.openjdk.java.net/~coleenp/7133093.04/webrev/src/share/vm/classfile/classLoaderData.cpp.frames.html > > lines 595-630. Please review! > > Thanks, > Coleen > > > On 7/19/17 10:52 PM, coleen.phillimore at oracle.com wrote: >> >> Hi, I've made the changes in the replies below and retested with both our internal NSK tests, jtreg tests and jck tests. Also I've rerun the performance tests, including SPECjbb2015. >> >> An incremental webrev can be found at: >> http://cr.openjdk.java.net/~coleenp/7133093.03.incremental/webrev/ >> >> Latest version at: >> >> http://cr.openjdk.java.net/~coleenp/7133093.03/webrev/ >> >> Thanks, >> Coleen >> >> >> On 7/18/17 10:56 AM, coleen.phillimore at oracle.com wrote: >>> >>> Karen, Thank you for reviewing the code and for the discussions and consulting along the way. >>> >>> On 7/17/17 4:10 PM, Karen Kinnear wrote: >>>> Feedback on the initial web rev: >>>> Thank you for using web rev - that way I can do searches of the overall patch! >>>> >>>> First - many thanks for doing this exercise so carefully, for following the specification in continuing >>>> to record the initiating loader and for splitting this effort into multiple phases to allow further exploration >>>> of the current expectations/behavior relative to class unloading. >>>> >>>> And thank you for all the testing. >>>> >>>> Just wanted to ensure that I understand some of the assumptions: >>>> >>>> 1) lock free reads of dictionaries - yes >>> Yes. >>>> 2) lock free adds of entries to dictionaries - yes >>> No, adds still take out the SystemDictionary_lock. >>>> 3) removal of entries or entire dictionaries - at safe point only >>> Yes. >>>> 4) rehashing - none today and none planned >>> Yes. >>>> 5) resizing of dictionaries - none today, some planned >>> Yes. >>>> 6) use of SystemDictionary_lock: essentially unchanged >>> Yes. >>>> 7) atomicity guarantees relative to checking loader constraints and updates to the dictionary: unchanged >>> Yes. >>>> >>>> I did not review the SA/vmStructs - sounds like you had Jini George do that - great. >>>> >>>> Minor questions/comments: >>>> >>>> SystemDictionary::verify: "Verify of system dictionary failed" -> >>>> looks like "Verify of placeholders failed" >>> >>> Fixed. Good find. It's an odd assert: >>> >>> guarantee(placeholders()->number_of_entries() >= 0, >>> "Verify of placeholders failed"); >>> >>>> >>>> thank you for asserts dictionary != NULL >>>> Dictionary::print >>>> is it worth an assert that loader_data != NULL? >>> >>> Added. It's already asserted in Dictionary::verify() >>> >>>> Can you please sanity check if there are other cases in which we need >>>> to use dictionary_or_null() rather than dictionary()? >>>> - are there cases in which we could be using a dictionary from an anonymous class? >>>> e.g. there are a lot of places in systemDictionary.cpp where we >>>> call class_loader_data(class_loader); loader_data->dictionary(); >>>> - was the lazily allocated dictionary allocated yet (see next question) >>> >>> There's an assert in dictionary() that the CLD isn't anonymous. In these calls we get class_loader_data from the class_loader which is always the host class loader for anonymous classes, so this doesn't break with the current design. >>> >>>> >>>> CLD.cpp >>>> Would it be less complex to eagerly create the dictionary for a CLD >>>> since it is extremely rare to not need one for non-anonymous CLDs? >>>> You could create before publishing the CLD >>> >>> Yes, I agree it would be better this way. I was looking at some data and it does seem rare to have a CLD with no classes. >>>> >>>> Terminology: >>>> How are you still using System Dictionary term? >>>> It still shows up in some comments and in some printed strings >>> >>> Bad habit, we've been calling it that for years. :) I'll update the comments where it makes sense. >>> I tried to refer to the ClassLoaderDataGraph dictionaries collectively as the "System Dictionary", as well as the loader constraints and placeholder tables, and the other things that SystemDictionary contains. >>> >>> I guess it gets confusing, but it's less awkward than "class loader data dictionaries", and more meaningful than simply "dictionaries". So that's why I didn't change a lot of the comments. >>> >>>> >>>> biasedLocking.cpp, line 59 >>>> jvmtiGetLoadedClasses: line 298 >>>> dictionary.hpp lines 113,114, 120 >>>> dictionary.cpp 262-263, 478 >>>> dictionary.cpp 491 >>>> fix comments e.g. "Iterate the system dictionary" ... >>>> - probably worth checking for other comments in the sources too >>>> unless these all appear to match new usage model >>> >>> Fixed. I've found a couple more and some other comments that are incorrect. >>>> >>>> hashTable.hpp >>>> line 268: what does this comment mean? >>>> "Don't create and use freelist"? >>>> I assume this means that this is a direct malloc, not using any block >>>> allocation, e.g. metaspace chunks, and this requires explicit freeing? >>>> >>> >>> // Don't create and use freelist of HashtableEntry. >>> >>> The default Hashtable<> allocates entries from a block of entries and uses a freelist for freed entries. >>> >>>> >>>> dictionary.hpp line 161 >>>> could you possibly clean up comment while you are in here? >>>> I presume "a protection" is "a protection domain"? >>> >>> Fixed. >>> >>>> >>>> dictionary.cpp >>>> why change from Hashtable::free_entry to explicit >>>> unlink_entry/FREE_C_HEAP_ARRAY? >>> >>> I used the same code as the ModuleTable. The default hashtable doesn't delete the whole thing and left some memory around, if I remember correctly. I'll recheck it. >>>> >>>> Future optimization: >>>> I think that none of the built-in loaders is ever recorded as an initiating >>>> loader for any class in a loader which permits unloading. >>>> Note: we need to ensure that built-in includes boot, platform and >>>> application class loader (and clean up any confusion with system >>>> class loader - which can be set by the user to a custom class loader) >>>> >>> I think AppClassLoader can initiate loading and is a builtin loader. >>> >>> Dictionary for class loader class loader 0x00007f6abc6e3210 a 'jdk/internal/loader/ClassLoaders$AppClassLoader'{0x0000000101c8df00} >>> Java system dictionary (table_size=107, classes=1) >>> ^ indicates that initiating loader is different from defining loader >>> 106: ^java.util.ArrayList, loader NULL class_loader >>> >>>> >>>> Any chance of changing DictionaryEntry* probe->klass() to be >>>> probe->instanceKlass()? >>> >>> Okay. Changed to instance_klass() as per coding standard. >>>> >>>> Thank you for assertion _is_at_safepoint for try_get_next_class >>>> >>>> CLD.cpp >>>> If the names of the classes_do vs. all_entries_do were modified to be >>>> defining_loader_classes_do vs. all_entries_classes_do or some way >>>> to make the distinction in the name that would be helpful. >>>> Otherwise, if you could please add comments to the CLDG::dictionary_classes_do >>>> etc. to distinguish these. >>> >>> I added comments to distinguish these. >>>> >>>> CLD.cpp >>>> lines 1200-1201 >>>> Can you please fix the comment? >>>> I think you need separate comments for the ModuleEntry reads/PackageEntry's >>>> exports which are looking for dead modules. >>>> IIUC, the do_unloading is actually looking for classes for which this >>>> is the initating loader, but their defining loader is dead. >>> // Remove entries in the dictionary of live class loader that have >>> // initiated loading classes in a dead class loader. >>> data->dictionary()->do_unloading(); >>> >>> Moved the module comment to before the module code and added this for dictionary->do_unloading() call. >>>> >>>> systemDictionary.cpp: 445-447 >>>> THANK you for the comments about the java_mirror handle keeping the class >>>> and class loader alive >>>> Could you clarify that the class_loader handle is not for the class >>>> we are checking, but rather for the initiating, i.e. requesting class loader >>>> (is that accurate?) So we are keeping both the requestor and the >>>> requested class alive >>> >>> Yes, that is true. I added >>> >>> // The class_loader handle passed in is the initiating loader. >>> >>>> >>>> You removed the comment that compute_hash could lead to a safepoint/GC >>>> Is that not possible? >>>> comment in ObjectSynchronizer::FastHashCode says that with UseBiasedLocking >>>> we can revoke biases and must be able to handle a safepoint. >>> >>> I made it not possible with this change. The hashcode is now computed by the Symbol* of the Klass name, which is random (os::random plus some bits from the address and first two characters, which we did a study of for bug https://bugs.openjdk.java.net/browse/JDK-8181450). >>> >>> Also, I changed ClassLoaderData::identity_hash() to use the address >> 3 because the addresses of class loader datas are random enough for the one use of this function. Previously, the loader_data identity hash used to go to the class_loader oop and get to ObjectSynchronizer::FastHashCode (which was also os::random but now changed). >>> >>> The dictionary hashing doesn't need to add in the hash for the loader_data now. >>>> >>>> systemDictionary.cpp: >>>> NoSafepointVerifier >>>> If we add resizing logic later, and that happens at a safepoint, >>>> then the number of buckets change and the d_index is no longer valid >>>> so we would need to add back the NSpV? >>>> - this is a more general problem if compute_hash can lead to a safepoint/GC >>> >>> Yes this is true, except that compute_hash doesn't safepoint. Computing the hash and index should be done together with the dictionary operations. The function resolve_instance_class_or_null computes the hash and index once at the beginning and reuses it for several lookups, which may or may not affect performance. So we need to measure this in isolation before changing this. >>>> >>>> systemDictionary.cpp: parse_stream >>>> why did you remove the record_dependency? >>> >>> I realized it's trivially unnecessary. The class_loader is reached by the ClassLoaderData::_class_loader field, so we don't have to record a dependency from host_klass->class_loader in the anonymous class loader data. (I just checked this again because it's confusing). >>> >>>> >>>> Who calls Dictionary::~Dictionary? >>> >>> The ClassLoaderData::~ClassLoaderData() calls delete _dictionary, which calls the destructor. >>>> >>>> Performance: >>>> Do you have performance runs of our existing benchmarks? >>> >>> Yes, I ran our internal benchmarks and added links to the internal wiki page. I'll rerun them again before checkin. >>> >>>> Initial key requirement is that this not slow down throughput through >>>> slowing down lookups. >>>> I know you are very aware of longer-term goals for reducing safepoint pause >>>> times and not increasing footprint significantly. >>>> >>>> hashtable sizes: future rfe? >>>> given that we now have a lot more dictionaries, and they perhaps need >>>> individual sizing >>>> - would it be helpful to add statistics information about >>>> - e.g. lookup_count, lookup_length, _lookup_warning >>>> - i.e. number of entries, max lookup depth, etc. to use later for tuning? >>> >>> All the previous statistics didn't make very much sense to us, so they were removed. There is now logging for all the hashtables: >>> >>> busaa027.us.oracle.com% java -Xlog:hashtables -version >>> java version "10-internal" >>> Java(TM) SE Runtime Environment (fastdebug build 10-internal+0-2017-07-17-1456035.cphillim.10system-dictionary2) >>> Java HotSpot(TM) 64-Bit Server VM (fastdebug build 10-internal+0-2017-07-17-1456035.cphillim.10system-dictionary2, mixed mode) >>> [0.284s][info][hashtables] System Dictionary for jdk/internal/loader/ClassLoaders$AppClassLoader max bucket size 1 bucket 91 element count 1 table size 107 >>> [0.285s][info][hashtables] System Dictionary for max bucket size 4 bucket 263 element count 468 table size 1009 >>> [0.285s][info][hashtables] Placeholder Table max bucket size 0 bucket 0 element count 0 table size 1009 >>> [0.285s][info][hashtables] Protection Domain Table max bucket size 0 bucket 0 element count 0 table size 1009 >>> >>>> - might make sense to make this JFR data? >>> >>> I'll file an RFE to add tracing data with our tools. I'll need some help with this part but the data is there. >>>> >>>> Might think about a follow-up with more tuning parameters than >>>> a single global PredictedLoadedClassCount. >>>> >>> >>> I filed RFE to work on resizing, as this work was almost complete, but didn't show performance improvements at the time. It would be nice to have customers not need tuning parameters, but I guess these things are sometimes necessary. >>> >>> Thank you for the thorough review and all of your help with this! I've made these changes and will rerun testing and send out a new webrev and an incremental webrev. >>> >>> Thanks! >>> Coleen >>> >>>> >>>> thanks, >>>> Karen >>>> >>>> >>>>> On Jul 17, 2017, at 2:37 PM, coleen.phillimore at oracle.com wrote: >>>>> >>>>> >>>>> Hi, I have made changes for Ioi's comments below. The largest change since .01 is that PredictedLoadedClassCount is used to size dictionaries for class loader data, and should be used to specify the average number of classes loaded by each class loader. Since this is an option available via UnlockExperimentalVMOptions, I'm not sure what the process is for changing the meaning of this option. We will work with our customers who have used this option for specifying the total number of classes loaded. We are also working on the change to allow automatic resizing of the dictionaries during safepoints. >>>>> >>>>> See: >>>>> open webrev at http://cr.openjdk.java.net/~coleenp/7133093.02/webrev >>>>> >>>>> Tested with the full nightly test set (jdk, nsk, hotspot jtreg tests) on linux x64. >>>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>> On 6/30/17 4:45 PM, coleen.phillimore at oracle.com wrote: >>>>>> >>>>>> Ioi, Thank you for looking at this change. >>>>>> >>>>>> On 6/30/17 2:39 PM, Ioi Lam wrote: >>>>>>> Hi Coleen, >>>>>>> >>>>>>> Maybe the bug should be renamed to "Use one Dictionary per class loader instance"? That way it's more obvious what it is when you look at the repo history. >>>>>> >>>>>> I can do that. I made it One Dictionary per ClassLoaderData >>>>>>> >>>>>>> 1. >>>>>>> >>>>>>> Why is assert_locked_or_safepoint(SystemDictionary_lock) necessary in SystemDictionary::find_class (line 1826), but not necessary SystemDictionary::find (line 951)? Since you removed NoSafepointVerifier nosafepoint in the latter, maybe this means it's safe to remove the assert_locked_or_safepoint in the former? >>>>>> >>>>>> The call to SystemDictionary::find() is the (I believe) usual lock free lookup and the SystemDictionary::find_class() is used to verify that a class we're about to add or want to add hasn't been added by another thread. Or certain cases where we already have a lock to do something else, like add loader constraints. I took out the NoSafepointVerifier because it assumes that system dictionary entries would be moved by GC, which they won't. The old hash function could safepoint when getting the hash for the class_loader object. >>>>>> >>>>>>> >>>>>>> 2. >>>>>>> >>>>>>> 455 static ClassLoaderData* _current_loader_data = NULL; >>>>>>> 456 static Klass* _current_class_entry = NULL; >>>>>>> 457 >>>>>>> 458 InstanceKlass* ClassLoaderDataGraph::try_get_next_class() { >>>>>>> >>>>>>> How about moving the static fields into an iterator object. That way you don't need to keep track of the globals >>>>>>> >>>>>>> ClassLoaderDataGraphIterator { >>>>>>> ClassLoaderData* _current_loader_data >>>>>>> Klass* _current_class_entry; >>>>>>> >>>>>>> InstanceKlass* try_get_next_class() { ....} >>>>>>> }; >>>>>> >>>>>> Ok, there's a different iterator that iterates over all of the classes for GC. I will adapt that for this use. That would be better. >>>>>> >>>>>>> >>>>>>> 3. Double check locking in ClassLoaderData::dictionary() -- someone else should look at this :-) >>>>>> >>>>>> I copied code that David Holmes added for modules. >>>>>> >>>>>>> >>>>>>> 4. We may need a better strategy for deciding the size of each dictionary. >>>>>>> >>>>>>> 565 const int _primelist[10] = {1, 107, 1009}; >>>>>>> 571 Dictionary* ClassLoaderData::dictionary() { >>>>>>> 579 if ((dictionary = _dictionary) == NULL) { >>>>>>> 580 int size; >>>>>>> 581 if (this == the_null_class_loader_data() || is_system_class_loader_data()) { >>>>>>> 582 size = _primelist[2]; >>>>>>> 583 } else if (class_loader()->is_a(SystemDictionary::reflect_DelegatingClassLoader_klass())) { >>>>>>> 584 size = _primelist[0]; >>>>>>> 585 } else { >>>>>>> 586 size = _primelist[1]; >>>>>>> 587 } >>>>>>> 588 dictionary = new Dictionary(this, size); >>>>>>> >>>>>>> I'll do some investigation on this issue and get back to you. >>>>>>> >>>>>> >>>>>> How about if someone uses PredictedLoadedClassCount, then we use that to size all but the reflection and boot class loader? Then if there's an application that has a class loader with a huge amount of classes loaded in it, that would help this? It might cost some footprint but an oversized table would simply be a bigger array of pointers, so it might not be that bad to oversize. >>>>>> >>>>>> I think the long term solution is being able to resize these entries and make the application provide arguments. Please let me know what you find in your investigation and if that would work. >>>>>>> The rest of the changes look good to me. >>>>>> >>>>>> Thank you! >>>>>> Coleen >>>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>>> >>>>>>> >>>>>>> On 6/23/17 4:42 PM, coleen.phillimore at oracle.com wrote: >>>>>>>> Summary: Implement one dictionary per ClassLoaderData for faster lookup and removal during class unloading >>>>>>>> >>>>>>>> See RFE for more details. >>>>>>>> >>>>>>>> open webrev at http://cr.openjdk.java.net/~coleenp/7133093.01/webrev >>>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-7133093 >>>>>>>> >>>>>>>> Tested with full "nightly" run in rbt, plus locally class loading and unloading tests: >>>>>>>> >>>>>>>> jtreg hotspot/test/runtime/ClassUnload >>>>>>>> >>>>>>>> jtreg hotspot/test/runtime/modules >>>>>>>> >>>>>>>> jtreg hotspot/test/gc/class_unloading >>>>>>>> >>>>>>>> make test-hotspot-closed-tonga FILTER=quick TEST_JOBS=4 TEST=vm.parallel_class_loading >>>>>>>> >>>>>>>> csh ~/testing/run_jck9 (vm/lang/java_lang) >>>>>>>> >>>>>>>> runThese -jck - uses class loader isolation to run each jck test and unloads tests when done (at -gc:5 intervals) >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Coleen >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From vasili.svirski at gmail.com Fri Jul 28 09:22:27 2017 From: vasili.svirski at gmail.com (Vasili Svirski) Date: Fri, 28 Jul 2017 12:22:27 +0300 Subject: [PATCH] c1_Canonicalizer.hpp method const Message-ID: Hi, It is my first patch in the attached file. In my patch, I just make one class method const In c1_Canonicalizer.hpp - Compilation *compilation() { return _compilation; } + Compilation *compilation() const { return _compilation; } My steps are: - read StyleGuide - make the simple change - build all - run native tests by jtreg Could somebody review the patch and confirm that I right in my steps? Best regards, Vasili From coleen.phillimore at oracle.com Fri Jul 28 11:57:51 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 28 Jul 2017 07:57:51 -0400 Subject: RFR (L) 7133093: Improve system dictionary performance In-Reply-To: <4ABB23CC-844C-437E-9416-957DCA525533@oracle.com> References: <89f0b98c-3cbd-7b87-d76b-5e89a5a676fb@oracle.com> <9243da7c-a8e5-4024-c12e-cba9d4f424c8@oracle.com> <7814C032-0D4E-4E76-B8AD-62A072E8A625@oracle.com> <233c03e0-8958-7676-8e33-5d8f0ec4edf3@oracle.com> <7adf48f1-5908-aeaf-c409-d930895e1715@oracle.com> <4ABB23CC-844C-437E-9416-957DCA525533@oracle.com> Message-ID: <6429f624-8eb0-6402-3bb1-d46bcbbf4ba1@oracle.com> Hi Jiangli, On 7/27/17 9:06 PM, Jiangli Zhou wrote: > Hi Coleen, > > The changes look really good to me. I have one minor question/comment. > > - src/share/vm/classfile/classLoaderData.cpp > > Line 468, in try_get_next_class(), if the ?_current_loader_data? reaches the end, it restarts from the ClassLoaderDataGraph::_head (line 490). So a caller could see the same klass being returned from try_get_next_class(). Is that an intended behavior? Also, if none of the loader data contains a ?loaded' class, it could loop forever. In reality, it probably would never happen. Yes, the original code would loop through the global system dictionary to decay the counters, so for subsequent safepoints, the method counters get decayed again. This is what they want. I did talk to Igor about this particular code snippet. The compiler code looks like this, so limits the number of calls: int nclasses = InstanceKlass::number_of_instance_classes(); double classes_per_tick = nclasses * (CounterDecayMinIntervalLength * 1e-3 / CounterHalfLifeTime); for (int i = 0; i < classes_per_tick; i++) { InstanceKlass* k = ClassLoaderDataGraph::try_get_next_class(); But, I don't like the while(true) in try_get_next_class() so I'll change it to, this and retest: int max_classes = InstanceKlass::number_of_instance_classes(); for (int i = 0; i < max_classes; i++) { ... } // should never be reached: an InstanceKlass should be returned above ShouldNotReachHere(); // Object_klass not even loaded? return NULL; There's still a return when we find a class, but it can't loop forever. Thank you for looking at the code and for this in particular. This function was one that I had to stomp out some bugs, so I hoped someone would check my logic. Thank you for reviewing! Coleen > > Thanks, > Jiangli > >> On Jul 21, 2017, at 11:28 AM, coleen.phillimore at oracle.com wrote: >> >> >> From internal discussions, the only sizing of the dictionary for the java.system.class.loader will follow PredictedLoadedClassCount until dictionary resizing is done. The latest full webrev is here: >> >> open webrev at http://cr.openjdk.java.net/~coleenp/7133093.04/webrev >> >> The only change from .03 is >> >> http://cr.openjdk.java.net/~coleenp/7133093.04/webrev/src/share/vm/classfile/classLoaderData.cpp.frames.html >> >> lines 595-630. Please review! >> >> Thanks, >> Coleen >> >> >> On 7/19/17 10:52 PM, coleen.phillimore at oracle.com wrote: >>> Hi, I've made the changes in the replies below and retested with both our internal NSK tests, jtreg tests and jck tests. Also I've rerun the performance tests, including SPECjbb2015. >>> >>> An incremental webrev can be found at: >>> http://cr.openjdk.java.net/~coleenp/7133093.03.incremental/webrev/ >>> >>> Latest version at: >>> >>> http://cr.openjdk.java.net/~coleenp/7133093.03/webrev/ >>> >>> Thanks, >>> Coleen >>> >>> >>> On 7/18/17 10:56 AM, coleen.phillimore at oracle.com wrote: >>>> Karen, Thank you for reviewing the code and for the discussions and consulting along the way. >>>> >>>> On 7/17/17 4:10 PM, Karen Kinnear wrote: >>>>> Feedback on the initial web rev: >>>>> Thank you for using web rev - that way I can do searches of the overall patch! >>>>> >>>>> First - many thanks for doing this exercise so carefully, for following the specification in continuing >>>>> to record the initiating loader and for splitting this effort into multiple phases to allow further exploration >>>>> of the current expectations/behavior relative to class unloading. >>>>> >>>>> And thank you for all the testing. >>>>> >>>>> Just wanted to ensure that I understand some of the assumptions: >>>>> >>>>> 1) lock free reads of dictionaries - yes >>>> Yes. >>>>> 2) lock free adds of entries to dictionaries - yes >>>> No, adds still take out the SystemDictionary_lock. >>>>> 3) removal of entries or entire dictionaries - at safe point only >>>> Yes. >>>>> 4) rehashing - none today and none planned >>>> Yes. >>>>> 5) resizing of dictionaries - none today, some planned >>>> Yes. >>>>> 6) use of SystemDictionary_lock: essentially unchanged >>>> Yes. >>>>> 7) atomicity guarantees relative to checking loader constraints and updates to the dictionary: unchanged >>>> Yes. >>>>> I did not review the SA/vmStructs - sounds like you had Jini George do that - great. >>>>> >>>>> Minor questions/comments: >>>>> >>>>> SystemDictionary::verify: "Verify of system dictionary failed" -> >>>>> looks like "Verify of placeholders failed" >>>> Fixed. Good find. It's an odd assert: >>>> >>>> guarantee(placeholders()->number_of_entries() >= 0, >>>> "Verify of placeholders failed"); >>>> >>>>> thank you for asserts dictionary != NULL >>>>> Dictionary::print >>>>> is it worth an assert that loader_data != NULL? >>>> Added. It's already asserted in Dictionary::verify() >>>> >>>>> Can you please sanity check if there are other cases in which we need >>>>> to use dictionary_or_null() rather than dictionary()? >>>>> - are there cases in which we could be using a dictionary from an anonymous class? >>>>> e.g. there are a lot of places in systemDictionary.cpp where we >>>>> call class_loader_data(class_loader); loader_data->dictionary(); >>>>> - was the lazily allocated dictionary allocated yet (see next question) >>>> There's an assert in dictionary() that the CLD isn't anonymous. In these calls we get class_loader_data from the class_loader which is always the host class loader for anonymous classes, so this doesn't break with the current design. >>>> >>>>> CLD.cpp >>>>> Would it be less complex to eagerly create the dictionary for a CLD >>>>> since it is extremely rare to not need one for non-anonymous CLDs? >>>>> You could create before publishing the CLD >>>> Yes, I agree it would be better this way. I was looking at some data and it does seem rare to have a CLD with no classes. >>>>> Terminology: >>>>> How are you still using System Dictionary term? >>>>> It still shows up in some comments and in some printed strings >>>> Bad habit, we've been calling it that for years. :) I'll update the comments where it makes sense. >>>> I tried to refer to the ClassLoaderDataGraph dictionaries collectively as the "System Dictionary", as well as the loader constraints and placeholder tables, and the other things that SystemDictionary contains. >>>> >>>> I guess it gets confusing, but it's less awkward than "class loader data dictionaries", and more meaningful than simply "dictionaries". So that's why I didn't change a lot of the comments. >>>> >>>>> biasedLocking.cpp, line 59 >>>>> jvmtiGetLoadedClasses: line 298 >>>>> dictionary.hpp lines 113,114, 120 >>>>> dictionary.cpp 262-263, 478 >>>>> dictionary.cpp 491 >>>>> fix comments e.g. "Iterate the system dictionary" ... >>>>> - probably worth checking for other comments in the sources too >>>>> unless these all appear to match new usage model >>>> Fixed. I've found a couple more and some other comments that are incorrect. >>>>> hashTable.hpp >>>>> line 268: what does this comment mean? >>>>> "Don't create and use freelist"? >>>>> I assume this means that this is a direct malloc, not using any block >>>>> allocation, e.g. metaspace chunks, and this requires explicit freeing? >>>>> >>>> // Don't create and use freelist of HashtableEntry. >>>> >>>> The default Hashtable<> allocates entries from a block of entries and uses a freelist for freed entries. >>>> >>>>> dictionary.hpp line 161 >>>>> could you possibly clean up comment while you are in here? >>>>> I presume "a protection" is "a protection domain"? >>>> Fixed. >>>> >>>>> dictionary.cpp >>>>> why change from Hashtable::free_entry to explicit >>>>> unlink_entry/FREE_C_HEAP_ARRAY? >>>> I used the same code as the ModuleTable. The default hashtable doesn't delete the whole thing and left some memory around, if I remember correctly. I'll recheck it. >>>>> Future optimization: >>>>> I think that none of the built-in loaders is ever recorded as an initiating >>>>> loader for any class in a loader which permits unloading. >>>>> Note: we need to ensure that built-in includes boot, platform and >>>>> application class loader (and clean up any confusion with system >>>>> class loader - which can be set by the user to a custom class loader) >>>>> >>>> I think AppClassLoader can initiate loading and is a builtin loader. >>>> >>>> Dictionary for class loader class loader 0x00007f6abc6e3210 a 'jdk/internal/loader/ClassLoaders$AppClassLoader'{0x0000000101c8df00} >>>> Java system dictionary (table_size=107, classes=1) >>>> ^ indicates that initiating loader is different from defining loader >>>> 106: ^java.util.ArrayList, loader NULL class_loader >>>> >>>>> Any chance of changing DictionaryEntry* probe->klass() to be >>>>> probe->instanceKlass()? >>>> Okay. Changed to instance_klass() as per coding standard. >>>>> Thank you for assertion _is_at_safepoint for try_get_next_class >>>>> >>>>> CLD.cpp >>>>> If the names of the classes_do vs. all_entries_do were modified to be >>>>> defining_loader_classes_do vs. all_entries_classes_do or some way >>>>> to make the distinction in the name that would be helpful. >>>>> Otherwise, if you could please add comments to the CLDG::dictionary_classes_do >>>>> etc. to distinguish these. >>>> I added comments to distinguish these. >>>>> CLD.cpp >>>>> lines 1200-1201 >>>>> Can you please fix the comment? >>>>> I think you need separate comments for the ModuleEntry reads/PackageEntry's >>>>> exports which are looking for dead modules. >>>>> IIUC, the do_unloading is actually looking for classes for which this >>>>> is the initating loader, but their defining loader is dead. >>>> // Remove entries in the dictionary of live class loader that have >>>> // initiated loading classes in a dead class loader. >>>> data->dictionary()->do_unloading(); >>>> >>>> Moved the module comment to before the module code and added this for dictionary->do_unloading() call. >>>>> systemDictionary.cpp: 445-447 >>>>> THANK you for the comments about the java_mirror handle keeping the class >>>>> and class loader alive >>>>> Could you clarify that the class_loader handle is not for the class >>>>> we are checking, but rather for the initiating, i.e. requesting class loader >>>>> (is that accurate?) So we are keeping both the requestor and the >>>>> requested class alive >>>> Yes, that is true. I added >>>> >>>> // The class_loader handle passed in is the initiating loader. >>>> >>>>> You removed the comment that compute_hash could lead to a safepoint/GC >>>>> Is that not possible? >>>>> comment in ObjectSynchronizer::FastHashCode says that with UseBiasedLocking >>>>> we can revoke biases and must be able to handle a safepoint. >>>> I made it not possible with this change. The hashcode is now computed by the Symbol* of the Klass name, which is random (os::random plus some bits from the address and first two characters, which we did a study of for bug https://bugs.openjdk.java.net/browse/JDK-8181450). >>>> >>>> Also, I changed ClassLoaderData::identity_hash() to use the address >> 3 because the addresses of class loader datas are random enough for the one use of this function. Previously, the loader_data identity hash used to go to the class_loader oop and get to ObjectSynchronizer::FastHashCode (which was also os::random but now changed). >>>> >>>> The dictionary hashing doesn't need to add in the hash for the loader_data now. >>>>> systemDictionary.cpp: >>>>> NoSafepointVerifier >>>>> If we add resizing logic later, and that happens at a safepoint, >>>>> then the number of buckets change and the d_index is no longer valid >>>>> so we would need to add back the NSpV? >>>>> - this is a more general problem if compute_hash can lead to a safepoint/GC >>>> Yes this is true, except that compute_hash doesn't safepoint. Computing the hash and index should be done together with the dictionary operations. The function resolve_instance_class_or_null computes the hash and index once at the beginning and reuses it for several lookups, which may or may not affect performance. So we need to measure this in isolation before changing this. >>>>> systemDictionary.cpp: parse_stream >>>>> why did you remove the record_dependency? >>>> I realized it's trivially unnecessary. The class_loader is reached by the ClassLoaderData::_class_loader field, so we don't have to record a dependency from host_klass->class_loader in the anonymous class loader data. (I just checked this again because it's confusing). >>>> >>>>> Who calls Dictionary::~Dictionary? >>>> The ClassLoaderData::~ClassLoaderData() calls delete _dictionary, which calls the destructor. >>>>> Performance: >>>>> Do you have performance runs of our existing benchmarks? >>>> Yes, I ran our internal benchmarks and added links to the internal wiki page. I'll rerun them again before checkin. >>>> >>>>> Initial key requirement is that this not slow down throughput through >>>>> slowing down lookups. >>>>> I know you are very aware of longer-term goals for reducing safepoint pause >>>>> times and not increasing footprint significantly. >>>>> >>>>> hashtable sizes: future rfe? >>>>> given that we now have a lot more dictionaries, and they perhaps need >>>>> individual sizing >>>>> - would it be helpful to add statistics information about >>>>> - e.g. lookup_count, lookup_length, _lookup_warning >>>>> - i.e. number of entries, max lookup depth, etc. to use later for tuning? >>>> All the previous statistics didn't make very much sense to us, so they were removed. There is now logging for all the hashtables: >>>> >>>> busaa027.us.oracle.com% java -Xlog:hashtables -version >>>> java version "10-internal" >>>> Java(TM) SE Runtime Environment (fastdebug build 10-internal+0-2017-07-17-1456035.cphillim.10system-dictionary2) >>>> Java HotSpot(TM) 64-Bit Server VM (fastdebug build 10-internal+0-2017-07-17-1456035.cphillim.10system-dictionary2, mixed mode) >>>> [0.284s][info][hashtables] System Dictionary for jdk/internal/loader/ClassLoaders$AppClassLoader max bucket size 1 bucket 91 element count 1 table size 107 >>>> [0.285s][info][hashtables] System Dictionary for max bucket size 4 bucket 263 element count 468 table size 1009 >>>> [0.285s][info][hashtables] Placeholder Table max bucket size 0 bucket 0 element count 0 table size 1009 >>>> [0.285s][info][hashtables] Protection Domain Table max bucket size 0 bucket 0 element count 0 table size 1009 >>>> >>>>> - might make sense to make this JFR data? >>>> I'll file an RFE to add tracing data with our tools. I'll need some help with this part but the data is there. >>>>> Might think about a follow-up with more tuning parameters than >>>>> a single global PredictedLoadedClassCount. >>>>> >>>> I filed RFE to work on resizing, as this work was almost complete, but didn't show performance improvements at the time. It would be nice to have customers not need tuning parameters, but I guess these things are sometimes necessary. >>>> >>>> Thank you for the thorough review and all of your help with this! I've made these changes and will rerun testing and send out a new webrev and an incremental webrev. >>>> >>>> Thanks! >>>> Coleen >>>> >>>>> thanks, >>>>> Karen >>>>> >>>>> >>>>>> On Jul 17, 2017, at 2:37 PM, coleen.phillimore at oracle.com wrote: >>>>>> >>>>>> >>>>>> Hi, I have made changes for Ioi's comments below. The largest change since .01 is that PredictedLoadedClassCount is used to size dictionaries for class loader data, and should be used to specify the average number of classes loaded by each class loader. Since this is an option available via UnlockExperimentalVMOptions, I'm not sure what the process is for changing the meaning of this option. We will work with our customers who have used this option for specifying the total number of classes loaded. We are also working on the change to allow automatic resizing of the dictionaries during safepoints. >>>>>> >>>>>> See: >>>>>> open webrev at http://cr.openjdk.java.net/~coleenp/7133093.02/webrev >>>>>> >>>>>> Tested with the full nightly test set (jdk, nsk, hotspot jtreg tests) on linux x64. >>>>>> >>>>>> Thanks, >>>>>> Coleen >>>>>> >>>>>> On 6/30/17 4:45 PM, coleen.phillimore at oracle.com wrote: >>>>>>> Ioi, Thank you for looking at this change. >>>>>>> >>>>>>> On 6/30/17 2:39 PM, Ioi Lam wrote: >>>>>>>> Hi Coleen, >>>>>>>> >>>>>>>> Maybe the bug should be renamed to "Use one Dictionary per class loader instance"? That way it's more obvious what it is when you look at the repo history. >>>>>>> I can do that. I made it One Dictionary per ClassLoaderData >>>>>>>> 1. >>>>>>>> >>>>>>>> Why is assert_locked_or_safepoint(SystemDictionary_lock) necessary in SystemDictionary::find_class (line 1826), but not necessary SystemDictionary::find (line 951)? Since you removed NoSafepointVerifier nosafepoint in the latter, maybe this means it's safe to remove the assert_locked_or_safepoint in the former? >>>>>>> The call to SystemDictionary::find() is the (I believe) usual lock free lookup and the SystemDictionary::find_class() is used to verify that a class we're about to add or want to add hasn't been added by another thread. Or certain cases where we already have a lock to do something else, like add loader constraints. I took out the NoSafepointVerifier because it assumes that system dictionary entries would be moved by GC, which they won't. The old hash function could safepoint when getting the hash for the class_loader object. >>>>>>> >>>>>>>> 2. >>>>>>>> >>>>>>>> 455 static ClassLoaderData* _current_loader_data = NULL; >>>>>>>> 456 static Klass* _current_class_entry = NULL; >>>>>>>> 457 >>>>>>>> 458 InstanceKlass* ClassLoaderDataGraph::try_get_next_class() { >>>>>>>> >>>>>>>> How about moving the static fields into an iterator object. That way you don't need to keep track of the globals >>>>>>>> >>>>>>>> ClassLoaderDataGraphIterator { >>>>>>>> ClassLoaderData* _current_loader_data >>>>>>>> Klass* _current_class_entry; >>>>>>>> >>>>>>>> InstanceKlass* try_get_next_class() { ....} >>>>>>>> }; >>>>>>> Ok, there's a different iterator that iterates over all of the classes for GC. I will adapt that for this use. That would be better. >>>>>>> >>>>>>>> 3. Double check locking in ClassLoaderData::dictionary() -- someone else should look at this :-) >>>>>>> I copied code that David Holmes added for modules. >>>>>>> >>>>>>>> 4. We may need a better strategy for deciding the size of each dictionary. >>>>>>>> >>>>>>>> 565 const int _primelist[10] = {1, 107, 1009}; >>>>>>>> 571 Dictionary* ClassLoaderData::dictionary() { >>>>>>>> 579 if ((dictionary = _dictionary) == NULL) { >>>>>>>> 580 int size; >>>>>>>> 581 if (this == the_null_class_loader_data() || is_system_class_loader_data()) { >>>>>>>> 582 size = _primelist[2]; >>>>>>>> 583 } else if (class_loader()->is_a(SystemDictionary::reflect_DelegatingClassLoader_klass())) { >>>>>>>> 584 size = _primelist[0]; >>>>>>>> 585 } else { >>>>>>>> 586 size = _primelist[1]; >>>>>>>> 587 } >>>>>>>> 588 dictionary = new Dictionary(this, size); >>>>>>>> >>>>>>>> I'll do some investigation on this issue and get back to you. >>>>>>>> >>>>>>> How about if someone uses PredictedLoadedClassCount, then we use that to size all but the reflection and boot class loader? Then if there's an application that has a class loader with a huge amount of classes loaded in it, that would help this? It might cost some footprint but an oversized table would simply be a bigger array of pointers, so it might not be that bad to oversize. >>>>>>> >>>>>>> I think the long term solution is being able to resize these entries and make the application provide arguments. Please let me know what you find in your investigation and if that would work. >>>>>>>> The rest of the changes look good to me. >>>>>>> Thank you! >>>>>>> Coleen >>>>>>>> Thanks >>>>>>>> - Ioi >>>>>>>> >>>>>>>> >>>>>>>> On 6/23/17 4:42 PM, coleen.phillimore at oracle.com wrote: >>>>>>>>> Summary: Implement one dictionary per ClassLoaderData for faster lookup and removal during class unloading >>>>>>>>> >>>>>>>>> See RFE for more details. >>>>>>>>> >>>>>>>>> open webrev at http://cr.openjdk.java.net/~coleenp/7133093.01/webrev >>>>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-7133093 >>>>>>>>> >>>>>>>>> Tested with full "nightly" run in rbt, plus locally class loading and unloading tests: >>>>>>>>> >>>>>>>>> jtreg hotspot/test/runtime/ClassUnload >>>>>>>>> >>>>>>>>> jtreg hotspot/test/runtime/modules >>>>>>>>> >>>>>>>>> jtreg hotspot/test/gc/class_unloading >>>>>>>>> >>>>>>>>> make test-hotspot-closed-tonga FILTER=quick TEST_JOBS=4 TEST=vm.parallel_class_loading >>>>>>>>> >>>>>>>>> csh ~/testing/run_jck9 (vm/lang/java_lang) >>>>>>>>> >>>>>>>>> runThese -jck - uses class loader isolation to run each jck test and unloads tests when done (at -gc:5 intervals) >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Coleen >>>>>>>>> >>>>>>>>> From aph at redhat.com Fri Jul 28 14:15:01 2017 From: aph at redhat.com (Andrew Haley) Date: Fri, 28 Jul 2017 15:15:01 +0100 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: <3C8BDDFA-1044-47F6-B15C-6DE2085ACF7C@oracle.com> References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <596CD126.6080100@oracle.com> <596DD713.6080806@oracle.com> <77d5a9e7-b2b0-79e4-effd-903dfe226512@redhat.com> <596E1142.6040708@oracle.com> <3432d11c-5904-a446-1cb5-419627f1794f@redhat.com> <596E288B.90706@oracle.com> <773a0b64-8273-1a58-a46a-22ae18f927b7@redhat.com> <5970B3FD.607@oracle.com> <8e1d9a32-7e01-b917-df1d-595c4000e4b2@redhat.com> <597226E2.6090803@oracle.com> <2c637e28-8d95-cbaf-c29e-d01867ed9157@redhat.com> <5979FEB6.10506@oracle.com> <3C8BDDFA-1044-47F6-B15C-6DE2085ACF7C@oracle.com> Message-ID: Hi, On 27/07/17 18:27, Erik Osterlund wrote: > Hi Andrew, > >> On 27 Jul 2017, at 17:35, Andrew Haley wrote: >> >> Hi, >> >>> On 27/07/17 15:54, Erik ?sterlund wrote: >> >> Well, I think that was you, not me: it's simpler for everyone if we >> insist that all of the arguments to these functions are the same type. >> Some of the callers will have to change, sure. > > I did indeed introduce the ability to match any types, but with > applied sanity checks protecting users from being able to compile if > incompatible types are passed in. > > I did try earlier on to see what would happen if we forced the types > to be identical as you propose. That turned out to be too strict. It > made it impossible to pass in perfectly fine literals like NULL or 0 > where it should be fine (among other things). This would force too > many call sites to have to be changed to explicitly cast passed in > values to types that they could perfectly safely convert to > implicitly. Therefore I decided against it. OK. It looked to me like some much-needed discipline for HotSpot programmers, but I guess that's a matter of taste. :-) >>> Similarly, I believe similar problems may apply when different >>> integer types of with different sizes are passed in that would not >>> be implicitly convertible, but sure are explicitly convertible (with >>> possibly unintentional truncation). You mentioned the case of >>> extending a 32 bit integer to a 64 bit integer, which indeed is >>> fine. Unfortunately, the reverse is not fine yet possible in this >>> API. For example, if the destination is of type volatile jint* and >>> the passed in expected value is a jlong, then that is clearly >>> wrong. Yet the API will accept these types and explicitly convert >>> the jlong to jint, truncating it in the process without >>> warning. This is the kind of thing that the casting mechanism me and >>> Kim worked on prevents you from doing accidentally. >> >> OK, I see. But it's a heck of a lot of code just to do that. > > Perhaps. But IntegerTypes is a general reusable tool that might find > further use later on. Hmm. This too is perhaps a matter of taste: I'm a great believer in not speculatively coding. But that's a discussion for another day. >> Instead, we could simply insist that the types must be the same, and >> fix the callers. This makes everything clearer, including the point >> where the Atomic API is called. Mismatched types at this level are >> either unintentional oversights or plain bugs. I have made this >> change, and it took a few minutes to fix all eight of the callers >> in HotSpot which have mismatched types. > > I think I already covered why this was not done. Also, that is also > a "heck of a lot of code" invested in being able to reduce the > anount of code in Atomic. I think you will come to the same > conclusion once you apply that pattern to all atomic operations as > well as OrderAccess (coming up after we are done here as it builds > on Atomic). I believe not. >>> How would you feel about a mechanism where we allow each platform to >>> specialize on the top level with all type information available, but >>> by default send everything in to the Atomic implementation I have >>> proposed that canonicalizes and sanity checks all values. Then we >>> have solved a bunch of existing problems, not introduced any new >>> problems, yet are future proof for allowing arbitrary specialization >>> on each platform to allow further improving the solution on >>> platforms that have cooperative compilers that allow such >>> improvements, on a platform by platform basis. >>> >>> What do you think? >> >> It's a better solution, and it is much more well suited to the future, >> when we can move over to compiler intrinsics. That's really >> important, IMO: it gives us an escape hatch from the past. > > Okay. I am currently on my way home, but here is an untested prototype: > > http://cr.openjdk.java.net/~eosterlund/8184334/webrev.02/ Alright, let's see where this takes us. It's be good to see if we can get the desirable properties of only allowing safe type conversions but still passing all of the types down to the native layer where possible. >> However, this would certainly fail to compile if anyone passed a >> double. To handle floating-point operands I would do one of two >> things: >> >> Create explicit specializations for doubles and floats: these do >> bitwise copies into integer values when calling the assembly-language >> sinippets, or >> >> Make all of the snippets copy their arguments bitwise into integer >> values and back. I believe that the best-supported way of doing that >> is the "union trick", which is widely supported by C++ compilers, even >> when using strict aliasing. > > That makes sense. Unfortunately though, that would still violate > section 3.10 and remain undefined behaviour. It certainly wouldn't with GCC, because GCC defines that as an extension, and it is thus implementation-defined. I suspect that is true of other compilers used to build HotSpot. I don't much care about theoretical niceties, but I do care about incorrect code. The union trick is so widely used that its UB may not matter, but I don't have much experience of proprietary C++ compilers so I might be wrong about that. But never mind, we're using -fno-strict-aliasing anyway. > Hope we are converging! I think so. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From rkennke at redhat.com Fri Jul 28 14:55:42 2017 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 28 Jul 2017 16:55:42 +0200 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <596CD126.6080100@oracle.com> <596DD713.6080806@oracle.com> <77d5a9e7-b2b0-79e4-effd-903dfe226512@redhat.com> <596E1142.6040708@oracle.com> <3432d11c-5904-a446-1cb5-419627f1794f@redhat.com> <596E288B.90706@oracle.com> <773a0b64-8273-1a58-a46a-22ae18f927b7@redhat.com> <5970B3FD.607@oracle.com> <8e1d9a32-7e01-b917-df1d-595c4000e4b2@redhat.com> <597226E2.6090803@oracle.com> <2c637e28-8d95-cbaf-c29e-d01867ed9157@redhat.com> <5979FEB6.10506@oracle.com> <3C8BDDFA-1044-47F6-B15C-6DE2085ACF7C@oracle.com> Message-ID: +1 for enforcing same type for cmpxchg parameters +1 for keeping it as simple as possible Roman Just Am 28.07.2017 um 16:15 schrieb Andrew Haley: > Hi, > > On 27/07/17 18:27, Erik Osterlund wrote: >> Hi Andrew, >> >>> On 27 Jul 2017, at 17:35, Andrew Haley wrote: >>> >>> Hi, >>> >>>> On 27/07/17 15:54, Erik ?sterlund wrote: >>> Well, I think that was you, not me: it's simpler for everyone if we >>> insist that all of the arguments to these functions are the same type. >>> Some of the callers will have to change, sure. >> I did indeed introduce the ability to match any types, but with >> applied sanity checks protecting users from being able to compile if >> incompatible types are passed in. >> >> I did try earlier on to see what would happen if we forced the types >> to be identical as you propose. That turned out to be too strict. It >> made it impossible to pass in perfectly fine literals like NULL or 0 >> where it should be fine (among other things). This would force too >> many call sites to have to be changed to explicitly cast passed in >> values to types that they could perfectly safely convert to >> implicitly. Therefore I decided against it. > OK. It looked to me like some much-needed discipline for HotSpot > programmers, but I guess that's a matter of taste. :-) > >>>> Similarly, I believe similar problems may apply when different >>>> integer types of with different sizes are passed in that would not >>>> be implicitly convertible, but sure are explicitly convertible (with >>>> possibly unintentional truncation). You mentioned the case of >>>> extending a 32 bit integer to a 64 bit integer, which indeed is >>>> fine. Unfortunately, the reverse is not fine yet possible in this >>>> API. For example, if the destination is of type volatile jint* and >>>> the passed in expected value is a jlong, then that is clearly >>>> wrong. Yet the API will accept these types and explicitly convert >>>> the jlong to jint, truncating it in the process without >>>> warning. This is the kind of thing that the casting mechanism me and >>>> Kim worked on prevents you from doing accidentally. >>> OK, I see. But it's a heck of a lot of code just to do that. >> Perhaps. But IntegerTypes is a general reusable tool that might find >> further use later on. > Hmm. This too is perhaps a matter of taste: I'm a great believer in > not speculatively coding. But that's a discussion for another day. > >>> Instead, we could simply insist that the types must be the same, and >>> fix the callers. This makes everything clearer, including the point >>> where the Atomic API is called. Mismatched types at this level are >>> either unintentional oversights or plain bugs. I have made this >>> change, and it took a few minutes to fix all eight of the callers >>> in HotSpot which have mismatched types. >> I think I already covered why this was not done. Also, that is also >> a "heck of a lot of code" invested in being able to reduce the >> anount of code in Atomic. I think you will come to the same >> conclusion once you apply that pattern to all atomic operations as >> well as OrderAccess (coming up after we are done here as it builds >> on Atomic). > I believe not. > >>>> How would you feel about a mechanism where we allow each platform to >>>> specialize on the top level with all type information available, but >>>> by default send everything in to the Atomic implementation I have >>>> proposed that canonicalizes and sanity checks all values. Then we >>>> have solved a bunch of existing problems, not introduced any new >>>> problems, yet are future proof for allowing arbitrary specialization >>>> on each platform to allow further improving the solution on >>>> platforms that have cooperative compilers that allow such >>>> improvements, on a platform by platform basis. >>>> >>>> What do you think? >>> It's a better solution, and it is much more well suited to the future, >>> when we can move over to compiler intrinsics. That's really >>> important, IMO: it gives us an escape hatch from the past. >> Okay. I am currently on my way home, but here is an untested prototype: >> >> http://cr.openjdk.java.net/~eosterlund/8184334/webrev.02/ > Alright, let's see where this takes us. It's be good to see if we can > get the desirable properties of only allowing safe type conversions > but still passing all of the types down to the native layer where > possible. > >>> However, this would certainly fail to compile if anyone passed a >>> double. To handle floating-point operands I would do one of two >>> things: >>> >>> Create explicit specializations for doubles and floats: these do >>> bitwise copies into integer values when calling the assembly-language >>> sinippets, or >>> >>> Make all of the snippets copy their arguments bitwise into integer >>> values and back. I believe that the best-supported way of doing that >>> is the "union trick", which is widely supported by C++ compilers, even >>> when using strict aliasing. >> That makes sense. Unfortunately though, that would still violate >> section 3.10 and remain undefined behaviour. > It certainly wouldn't with GCC, because GCC defines that as an > extension, and it is thus implementation-defined. I suspect that is > true of other compilers used to build HotSpot. I don't much care > about theoretical niceties, but I do care about incorrect code. The > union trick is so widely used that its UB may not matter, but I don't > have much experience of proprietary C++ compilers so I might be wrong > about that. > > But never mind, we're using -fno-strict-aliasing anyway. > >> Hope we are converging! > I think so. > From karen.kinnear at oracle.com Fri Jul 28 15:51:14 2017 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Fri, 28 Jul 2017 11:51:14 -0400 Subject: RFR (L) 7133093: Improve system dictionary performance In-Reply-To: <233c03e0-8958-7676-8e33-5d8f0ec4edf3@oracle.com> References: <89f0b98c-3cbd-7b87-d76b-5e89a5a676fb@oracle.com> <9243da7c-a8e5-4024-c12e-cba9d4f424c8@oracle.com> <7814C032-0D4E-4E76-B8AD-62A072E8A625@oracle.com> <233c03e0-8958-7676-8e33-5d8f0ec4edf3@oracle.com> Message-ID: <87A0DCD8-80DC-4182-9F24-2EC92BFE00E6@oracle.com> Coleen, Ship it! webrev.03 looks good (I know you had other reviewers for the SA parts - I did not look at those). Sorry to slow you down. Many thanks for the incremental diff - and thank you for the incremental changes. Thank you for the eagerly created dictionaries - much simpler. Thank you for all the experimentation and testing. Thank you for the performance runs. One potential future rfe: >> >> Future optimization: >> I think that none of the built-in loaders is ever recorded as an initiating >> loader for any class in a loader which permits unloading. >> Note: we need to ensure that built-in includes boot, platform and >> application class loader (and clean up any confusion with system >> class loader - which can be set by the user to a custom class loader) >> > I think AppClassLoader can initiate loading and is a builtin loader. > > Dictionary for class loader class loader 0x00007f6abc6e3210 a 'jdk/internal/loader/ClassLoaders$AppClassLoader'{0x0000000101c8df00} > Java system dictionary (table_size=107, classes=1) > ^ indicates that initiating loader is different from defining loader > 106: ^java.util.ArrayList, loader NULL class_loader Let me clarify. It is not that they can not initiate loading, it is that they are ever recorded as an initiating loader for any other loaders other than our 3 built-in loaders which never unload. So the potential future improvement would be no need to walk our 3 built-in loaders to check if we have initiating loaders referencing dead defining loaders. thanks, Karen > > Thanks! > Coleen > >> >> thanks, >> Karen >> >> >>> On Jul 17, 2017, at 2:37 PM, coleen.phillimore at oracle.com wrote: >>> >>> >>> Hi, I have made changes for Ioi's comments below. The largest change since .01 is that PredictedLoadedClassCount is used to size dictionaries for class loader data, and should be used to specify the average number of classes loaded by each class loader. Since this is an option available via UnlockExperimentalVMOptions, I'm not sure what the process is for changing the meaning of this option. We will work with our customers who have used this option for specifying the total number of classes loaded. We are also working on the change to allow automatic resizing of the dictionaries during safepoints. >>> >>> See: >>> open webrev at http://cr.openjdk.java.net/~coleenp/7133093.02/webrev >>> >>> Tested with the full nightly test set (jdk, nsk, hotspot jtreg tests) on linux x64. >>> >>> Thanks, >>> Coleen >>> >>> On 6/30/17 4:45 PM, coleen.phillimore at oracle.com wrote: >>>> >>>> Ioi, Thank you for looking at this change. >>>> >>>> On 6/30/17 2:39 PM, Ioi Lam wrote: >>>>> Hi Coleen, >>>>> >>>>> Maybe the bug should be renamed to "Use one Dictionary per class loader instance"? That way it's more obvious what it is when you look at the repo history. >>>> >>>> I can do that. I made it One Dictionary per ClassLoaderData >>>>> >>>>> 1. >>>>> >>>>> Why is assert_locked_or_safepoint(SystemDictionary_lock) necessary in SystemDictionary::find_class (line 1826), but not necessary SystemDictionary::find (line 951)? Since you removed NoSafepointVerifier nosafepoint in the latter, maybe this means it's safe to remove the assert_locked_or_safepoint in the former? >>>> >>>> The call to SystemDictionary::find() is the (I believe) usual lock free lookup and the SystemDictionary::find_class() is used to verify that a class we're about to add or want to add hasn't been added by another thread. Or certain cases where we already have a lock to do something else, like add loader constraints. I took out the NoSafepointVerifier because it assumes that system dictionary entries would be moved by GC, which they won't. The old hash function could safepoint when getting the hash for the class_loader object. >>>> >>>>> >>>>> 2. >>>>> >>>>> 455 static ClassLoaderData* _current_loader_data = NULL; >>>>> 456 static Klass* _current_class_entry = NULL; >>>>> 457 >>>>> 458 InstanceKlass* ClassLoaderDataGraph::try_get_next_class() { >>>>> >>>>> How about moving the static fields into an iterator object. That way you don't need to keep track of the globals >>>>> >>>>> ClassLoaderDataGraphIterator { >>>>> ClassLoaderData* _current_loader_data >>>>> Klass* _current_class_entry; >>>>> >>>>> InstanceKlass* try_get_next_class() { ....} >>>>> }; >>>> >>>> Ok, there's a different iterator that iterates over all of the classes for GC. I will adapt that for this use. That would be better. >>>> >>>>> >>>>> 3. Double check locking in ClassLoaderData::dictionary() -- someone else should look at this :-) >>>> >>>> I copied code that David Holmes added for modules. >>>> >>>>> >>>>> 4. We may need a better strategy for deciding the size of each dictionary. >>>>> >>>>> 565 const int _primelist[10] = {1, 107, 1009}; >>>>> 571 Dictionary* ClassLoaderData::dictionary() { >>>>> 579 if ((dictionary = _dictionary) == NULL) { >>>>> 580 int size; >>>>> 581 if (this == the_null_class_loader_data() || is_system_class_loader_data()) { >>>>> 582 size = _primelist[2]; >>>>> 583 } else if (class_loader()->is_a(SystemDictionary::reflect_DelegatingClassLoader_klass())) { >>>>> 584 size = _primelist[0]; >>>>> 585 } else { >>>>> 586 size = _primelist[1]; >>>>> 587 } >>>>> 588 dictionary = new Dictionary(this, size); >>>>> >>>>> I'll do some investigation on this issue and get back to you. >>>>> >>>> >>>> How about if someone uses PredictedLoadedClassCount, then we use that to size all but the reflection and boot class loader? Then if there's an application that has a class loader with a huge amount of classes loaded in it, that would help this? It might cost some footprint but an oversized table would simply be a bigger array of pointers, so it might not be that bad to oversize. >>>> >>>> I think the long term solution is being able to resize these entries and make the application provide arguments. Please let me know what you find in your investigation and if that would work. >>>>> The rest of the changes look good to me. >>>> >>>> Thank you! >>>> Coleen >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> >>>>> On 6/23/17 4:42 PM, coleen.phillimore at oracle.com wrote: >>>>>> Summary: Implement one dictionary per ClassLoaderData for faster lookup and removal during class unloading >>>>>> >>>>>> See RFE for more details. >>>>>> >>>>>> open webrev at http://cr.openjdk.java.net/~coleenp/7133093.01/webrev >>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-7133093 >>>>>> >>>>>> Tested with full "nightly" run in rbt, plus locally class loading and unloading tests: >>>>>> >>>>>> jtreg hotspot/test/runtime/ClassUnload >>>>>> >>>>>> jtreg hotspot/test/runtime/modules >>>>>> >>>>>> jtreg hotspot/test/gc/class_unloading >>>>>> >>>>>> make test-hotspot-closed-tonga FILTER=quick TEST_JOBS=4 TEST=vm.parallel_class_loading >>>>>> >>>>>> csh ~/testing/run_jck9 (vm/lang/java_lang) >>>>>> >>>>>> runThese -jck - uses class loader isolation to run each jck test and unloads tests when done (at -gc:5 intervals) >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Coleen >>>>>> >>>>>> >>>>> >>>> >>> >> > From hzzengyijie at corp.netease.com Fri Jul 28 09:37:26 2017 From: hzzengyijie at corp.netease.com (zengyijie) Date: Fri, 28 Jul 2017 17:37:26 +0800 Subject: Where in OpenJDK repository can I find Hotspot VM source codes? In-Reply-To: References: <65511B52-BA7A-4F1B-AB3B-4217E25499FB@corp.netease.com> Message-ID: <18D26DA3-12A7-49AB-B247-40B2C10346C9@corp.netease.com> Hi Kris, Thank you very much for detailed explain! We are running that specific Oracle JVM in production, and encountered a strange native memory leak, so we plan to build a debug JVM from source codes. Per your explain, there is no way to get source codes of Oracle HotSpot VM after JDK6u32? How to get a Oracle Hotspot VM with debug symbol? Thanks a lot, Yijie > ? 2017?7?28????2:44?Krystal Mok ??? > > Hi Yijie, > > I don't think there is one that exactly matches what you want. > > I wrote a doc [1] long ago about the correspondence of Sun/Oracle JDK 6 versus OpenJDK 6 in terms of the JVM used. Unfortunately, because of the history behind OpenJDK 6 (it's a backport of OpenJDK 7 to be a JDK6 release, rather than the open sourced version of Sun/Oracle JDK 6), there was no exact correspondence to begin with. > > Back then, the HotSpot Expression release model was still in place, so the Oracle JDK 6 and the then still-in-development JDK 7 would mostly share the same JVM, that's how the OpenJDK 6 HotSpot VM still very closely match the Oracle JDK 6 version -- less some proprietary features like UseCompressedStrings, and a bit of delay in releasing the new tiered compilation features, etc. > > After Oracle JDK6u32 or something, I don't think the OpenJDK 6 HotSpot VM reflected changes to the Oracle JDK 6 any more, as JDK6 was going towards EOL'd and less changes from Oracle JDK 6 were sync'd to OpenJDK 6. > Red Hat later took over the maintenance of the OpenJDK 6 release, and it further diverged from the Oracle internal one. > > What's in Oracle JDK6u45 specifically that you wanted to check the sources for, though? > > - Kris > > [1]: https://gist.github.com/rednaxelafx/925323 > > On Wed, Jul 5, 2017 at 2:18 AM, zengyijie > wrote: > Hi, we are running an outdated Oracle Java JDK6 with Hotspot VM, I have spent quite a while browsing openjdk repository, > but I can?t find the source codes for the specific Hotspot VM version. The repository tip of http://hg.openjdk.java.net/jdk6/jdk6/hotspot/ > is only jdk6-b43, there?s no 1.6.0_45. > Could someone help tell where to find it? > The Oracle JDK version is as follows: > > java version "1.6.0_45" > Java(TM) SE Runtime Environment (build 1.6.0_45-b06) > Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01, mixed mode) > From erik.osterlund at oracle.com Fri Jul 28 16:01:10 2017 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 28 Jul 2017 18:01:10 +0200 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <596CD126.6080100@oracle.com> <596DD713.6080806@oracle.com> <77d5a9e7-b2b0-79e4-effd-903dfe226512@redhat.com> <596E1142.6040708@oracle.com> <3432d11c-5904-a446-1cb5-419627f1794f@redhat.com> <596E288B.90706@oracle.com> <773a0b64-8273-1a58-a46a-22ae18f927b7@redhat.com> <5970B3FD.607@oracle.com> <8e1d9a32-7e01-b917-df1d-595c4000e4b2@redhat.com> <597226E2.6090803@oracle.com> <2c637e28-8d95-cbaf-c29e-d01867ed9157@redhat.com> <5979FEB6.10506@oracle.com> <3C8BDDFA-1044-47F6-B15C-6DE2085ACF7C@oracle.com> Message-ID: <597B5FC6.5020702@oracle.com> Hi Andrew, I am glad we are converging. It seems we agree that we should go with the current new frontend that in a safe way connects existing users of the Atomic frontend to the existing backends of Atomic we have today (that uses truncated types), while sanity checking that new uses make sense and are safe. At the same time, it allows for the mechanism to be specialized with all type information available on a per platform basis in future backends as separate new RFEs where applicable, without requiring any changes to the shared frontend, by declaring a PlatformAtomic in an atomic platform header that arbitrarily overrides any Atomic operation completely with all type information preserved. I do hope this will make everyone happy involved. Here is a refined full webrev with that concept (polished) from yesterday: http://cr.openjdk.java.net/~eosterlund/8184334/webrev.03/ Incremental webrev from the initial webrev I sent out: http://cr.openjdk.java.net/~eosterlund/8184334/webrev.01_03/ I hope you like this. I think you already indicated that you did. Please be aware also that I am going on vacation now and will be back in 3 weeks time. Thanks, /Erik On 2017-07-28 16:15, Andrew Haley wrote: > Hi, > > On 27/07/17 18:27, Erik Osterlund wrote: >> Hi Andrew, >> >>> On 27 Jul 2017, at 17:35, Andrew Haley wrote: >>> >>> Hi, >>> >>>> On 27/07/17 15:54, Erik ?sterlund wrote: >>> Well, I think that was you, not me: it's simpler for everyone if we >>> insist that all of the arguments to these functions are the same type. >>> Some of the callers will have to change, sure. >> I did indeed introduce the ability to match any types, but with >> applied sanity checks protecting users from being able to compile if >> incompatible types are passed in. >> >> I did try earlier on to see what would happen if we forced the types >> to be identical as you propose. That turned out to be too strict. It >> made it impossible to pass in perfectly fine literals like NULL or 0 >> where it should be fine (among other things). This would force too >> many call sites to have to be changed to explicitly cast passed in >> values to types that they could perfectly safely convert to >> implicitly. Therefore I decided against it. > OK. It looked to me like some much-needed discipline for HotSpot > programmers, but I guess that's a matter of taste. :-) > >>>> Similarly, I believe similar problems may apply when different >>>> integer types of with different sizes are passed in that would not >>>> be implicitly convertible, but sure are explicitly convertible (with >>>> possibly unintentional truncation). You mentioned the case of >>>> extending a 32 bit integer to a 64 bit integer, which indeed is >>>> fine. Unfortunately, the reverse is not fine yet possible in this >>>> API. For example, if the destination is of type volatile jint* and >>>> the passed in expected value is a jlong, then that is clearly >>>> wrong. Yet the API will accept these types and explicitly convert >>>> the jlong to jint, truncating it in the process without >>>> warning. This is the kind of thing that the casting mechanism me and >>>> Kim worked on prevents you from doing accidentally. >>> OK, I see. But it's a heck of a lot of code just to do that. >> Perhaps. But IntegerTypes is a general reusable tool that might find >> further use later on. > Hmm. This too is perhaps a matter of taste: I'm a great believer in > not speculatively coding. But that's a discussion for another day. > >>> Instead, we could simply insist that the types must be the same, and >>> fix the callers. This makes everything clearer, including the point >>> where the Atomic API is called. Mismatched types at this level are >>> either unintentional oversights or plain bugs. I have made this >>> change, and it took a few minutes to fix all eight of the callers >>> in HotSpot which have mismatched types. >> I think I already covered why this was not done. Also, that is also >> a "heck of a lot of code" invested in being able to reduce the >> anount of code in Atomic. I think you will come to the same >> conclusion once you apply that pattern to all atomic operations as >> well as OrderAccess (coming up after we are done here as it builds >> on Atomic). > I believe not. > >>>> How would you feel about a mechanism where we allow each platform to >>>> specialize on the top level with all type information available, but >>>> by default send everything in to the Atomic implementation I have >>>> proposed that canonicalizes and sanity checks all values. Then we >>>> have solved a bunch of existing problems, not introduced any new >>>> problems, yet are future proof for allowing arbitrary specialization >>>> on each platform to allow further improving the solution on >>>> platforms that have cooperative compilers that allow such >>>> improvements, on a platform by platform basis. >>>> >>>> What do you think? >>> It's a better solution, and it is much more well suited to the future, >>> when we can move over to compiler intrinsics. That's really >>> important, IMO: it gives us an escape hatch from the past. >> Okay. I am currently on my way home, but here is an untested prototype: >> >> http://cr.openjdk.java.net/~eosterlund/8184334/webrev.02/ > Alright, let's see where this takes us. It's be good to see if we can > get the desirable properties of only allowing safe type conversions > but still passing all of the types down to the native layer where > possible. > >>> However, this would certainly fail to compile if anyone passed a >>> double. To handle floating-point operands I would do one of two >>> things: >>> >>> Create explicit specializations for doubles and floats: these do >>> bitwise copies into integer values when calling the assembly-language >>> sinippets, or >>> >>> Make all of the snippets copy their arguments bitwise into integer >>> values and back. I believe that the best-supported way of doing that >>> is the "union trick", which is widely supported by C++ compilers, even >>> when using strict aliasing. >> That makes sense. Unfortunately though, that would still violate >> section 3.10 and remain undefined behaviour. > It certainly wouldn't with GCC, because GCC defines that as an > extension, and it is thus implementation-defined. I suspect that is > true of other compilers used to build HotSpot. I don't much care > about theoretical niceties, but I do care about incorrect code. The > union trick is so widely used that its UB may not matter, but I don't > have much experience of proprietary C++ compilers so I might be wrong > about that. > > But never mind, we're using -fno-strict-aliasing anyway. > >> Hope we are converging! > I think so. > From aph at redhat.com Fri Jul 28 16:08:10 2017 From: aph at redhat.com (Andrew Haley) Date: Fri, 28 Jul 2017 17:08:10 +0100 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: <597B5FC6.5020702@oracle.com> References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <596CD126.6080100@oracle.com> <596DD713.6080806@oracle.com> <77d5a9e7-b2b0-79e4-effd-903dfe226512@redhat.com> <596E1142.6040708@oracle.com> <3432d11c-5904-a446-1cb5-419627f1794f@redhat.com> <596E288B.90706@oracle.com> <773a0b64-8273-1a58-a46a-22ae18f927b7@redhat.com> <5970B3FD.607@oracle.com> <8e1d9a32-7e01-b917-df1d-595c4000e4b2@redhat.com> <597226E2.6090803@oracle.com> <2c637e28-8d95-cbaf-c29e-d01867ed9157@redhat.com> <5979FEB6.10506@oracle.com> <3C8BDDFA-1044-47F6-B15C-6DE2085ACF7C@oracle.com> <597B5FC6.5020702@oracle.com> Message-ID: <4e10660a-4dee-78bb-333e-fe99a9c2295d@redhat.com> On 28/07/17 17:01, Erik ?sterlund wrote: > I hope you like this. I think you already indicated that you did. Not yet, no. I need a little more time. > Please be aware also that I am going on vacation now and will be > back in 3 weeks time. OK. I hope that this delay makes for a better solution for everyone. I think it will. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From erik.osterlund at oracle.com Fri Jul 28 16:25:54 2017 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Fri, 28 Jul 2017 18:25:54 +0200 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: <4e10660a-4dee-78bb-333e-fe99a9c2295d@redhat.com> References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <596CD126.6080100@oracle.com> <596DD713.6080806@oracle.com> <77d5a9e7-b2b0-79e4-effd-903dfe226512@redhat.com> <596E1142.6040708@oracle.com> <3432d11c-5904-a446-1cb5-419627f1794f@redhat.com> <596E288B.90706@oracle.com> <773a0b64-8273-1a58-a46a-22ae18f927b7@redhat.com> <5970B3FD.607@oracle.com> <8e1d9a32-7e01-b917-df1d-595c4000e4b2@redhat.com> <597226E2.6090803@oracle.com> <2c637e28-8d95-cbaf-c29e-d01867ed9157@redhat.com> <5979FEB6.10506@oracle.com> <3C8BDDFA-1044-47F6-B15C-6DE2085ACF7C@oracle.com> <597B5FC6.5020702@oracle.com> <4e10660a-4dee-78bb-333e-fe99a9c2295d@redhat.com> Message-ID: Hi Andrew, In that case, feel free to propose a revised solution while I am gone. Thanks, /Erik > On 28 Jul 2017, at 18:08, Andrew Haley wrote: > >> On 28/07/17 17:01, Erik ?sterlund wrote: >> I hope you like this. I think you already indicated that you did. > > Not yet, no. I need a little more time. > >> Please be aware also that I am going on vacation now and will be >> back in 3 weeks time. > > OK. I hope that this delay makes for a better solution for everyone. > I think it will. > > -- > Andrew Haley > Java Platform Lead Engineer > Red Hat UK Ltd. > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From coleen.phillimore at oracle.com Fri Jul 28 17:35:54 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 28 Jul 2017 13:35:54 -0400 Subject: RFR (L) 7133093: Improve system dictionary performance In-Reply-To: <87A0DCD8-80DC-4182-9F24-2EC92BFE00E6@oracle.com> References: <89f0b98c-3cbd-7b87-d76b-5e89a5a676fb@oracle.com> <9243da7c-a8e5-4024-c12e-cba9d4f424c8@oracle.com> <7814C032-0D4E-4E76-B8AD-62A072E8A625@oracle.com> <233c03e0-8958-7676-8e33-5d8f0ec4edf3@oracle.com> <87A0DCD8-80DC-4182-9F24-2EC92BFE00E6@oracle.com> Message-ID: <4a85b5ea-b714-2801-6d3d-c4d9b8041da9@oracle.com> Thank you for reviewing and your help and discussions. On 7/28/17 11:51 AM, Karen Kinnear wrote: > Coleen, > > Ship it! webrev.03 looks good (I know you had other reviewers for the > SA parts - I did not look at those). > > Sorry to slow you down. Many thanks for the incremental diff - and > thank you for the incremental changes. > Thank you for the eagerly created dictionaries - much simpler. > > Thank you for all the experimentation and testing. > Thank you for the performance runs. > > One potential future rfe: >>> >>> Future optimization: >>> I think that none of the built-in loaders is ever recorded as an >>> initiating >>> loader for any class in a loader which permits unloading. >>> Note: we need to ensure that built-in includes boot, platform and >>> application class loader (and clean up any confusion with system >>> class loader - which can be set by the user to a custom class loader) >>> >> I think AppClassLoader can initiate loading and is a builtin loader. >> >> Dictionary for class loader class loader 0x00007f6abc6e3210 a >> 'jdk/internal/loader/ClassLoaders$AppClassLoader'{0x0000000101c8df00} >> Java system dictionary (table_size=107, classes=1) >> ^ indicates that initiating loader is different from defining loader >> 106: ^java.util.ArrayList, loader NULL class_loader > Let me clarify. > It is not that they can not initiate loading, it is that they are > ever recorded as an > initiating loader for any other loaders other than our 3 built-in > loaders which never unload. > So the potential future improvement would be no need to walk our 3 > built-in loaders to check if we > have initiating loaders referencing dead defining loaders. Oh, I see. Yes. In Dictionary::do_unloading() we could verify this. Thanks, Coleen > > thanks, > Karen >> >> Thanks! >> Coleen >> >>> >>> thanks, >>> Karen >>> >>> >>>> On Jul 17, 2017, at 2:37 PM, coleen.phillimore at oracle.com >>>> wrote: >>>> >>>> >>>> Hi, I have made changes for Ioi's comments below. The largest >>>> change since .01 is that PredictedLoadedClassCount is used to size >>>> dictionaries for class loader data, and should be used to specify >>>> the average number of classes loaded by each class loader. Since >>>> this is an option available via UnlockExperimentalVMOptions, I'm >>>> not sure what the process is for changing the meaning of this >>>> option. We will work with our customers who have used this option >>>> for specifying the total number of classes loaded. We are also >>>> working on the change to allow automatic resizing of the >>>> dictionaries during safepoints. >>>> >>>> See: >>>> open webrev at >>>> http://cr.openjdk.java.net/~coleenp/7133093.02/webrev >>>> >>>> >>>> Tested with the full nightly test set (jdk, nsk, hotspot jtreg >>>> tests) on linux x64. >>>> >>>> Thanks, >>>> Coleen >>>> >>>> On 6/30/17 4:45 PM, coleen.phillimore at oracle.com >>>> wrote: >>>>> >>>>> Ioi, Thank you for looking at this change. >>>>> >>>>> On 6/30/17 2:39 PM, Ioi Lam wrote: >>>>>> Hi Coleen, >>>>>> >>>>>> Maybe the bug should be renamed to "Use one Dictionary per class >>>>>> loader instance"? That way it's more obvious what it is when you >>>>>> look at the repo history. >>>>> >>>>> I can do that. I made it One Dictionary per ClassLoaderData >>>>>> >>>>>> 1. >>>>>> >>>>>> Why is assert_locked_or_safepoint(SystemDictionary_lock) >>>>>> necessary in SystemDictionary::find_class (line 1826), but not >>>>>> necessary SystemDictionary::find (line 951)? Since you removed >>>>>> NoSafepointVerifier nosafepoint in the latter, maybe this means >>>>>> it's safe to remove the assert_locked_or_safepoint in the former? >>>>> >>>>> The call to SystemDictionary::find() is the (I believe) usual lock >>>>> free lookup and the SystemDictionary::find_class() is used to >>>>> verify that a class we're about to add or want to add hasn't been >>>>> added by another thread. Or certain cases where we already have >>>>> a lock to do something else, like add loader constraints. I took >>>>> out the NoSafepointVerifier because it assumes that system >>>>> dictionary entries would be moved by GC, which they won't. The >>>>> old hash function could safepoint when getting the hash for the >>>>> class_loader object. >>>>> >>>>>> >>>>>> 2. >>>>>> >>>>>> 455 static ClassLoaderData* _current_loader_data = NULL; >>>>>> 456 static Klass* _current_class_entry = NULL; >>>>>> 457 >>>>>> 458 InstanceKlass* ClassLoaderDataGraph::try_get_next_class() { >>>>>> >>>>>> How about moving the static fields into an iterator object. That >>>>>> way you don't need to keep track of the globals >>>>>> >>>>>> ClassLoaderDataGraphIterator { >>>>>> ClassLoaderData* _current_loader_data >>>>>> Klass* _current_class_entry; >>>>>> >>>>>> InstanceKlass* try_get_next_class() { ....} >>>>>> }; >>>>> >>>>> Ok, there's a different iterator that iterates over all of the >>>>> classes for GC. I will adapt that for this use. That would be >>>>> better. >>>>> >>>>>> >>>>>> 3. Double check locking in ClassLoaderData::dictionary() -- >>>>>> someone else should look at this :-) >>>>> >>>>> I copied code that David Holmes added for modules. >>>>> >>>>>> >>>>>> 4. We may need a better strategy for deciding the size of each >>>>>> dictionary. >>>>>> >>>>>> 565 const int _primelist[10] = {1, 107, 1009}; >>>>>> 571 Dictionary* ClassLoaderData::dictionary() { >>>>>> 579 if ((dictionary = _dictionary) == NULL) { >>>>>> 580 int size; >>>>>> 581 if (this == the_null_class_loader_data() || >>>>>> is_system_class_loader_data()) { >>>>>> 582 size = _primelist[2]; >>>>>> 583 } else if >>>>>> (class_loader()->is_a(SystemDictionary::reflect_DelegatingClassLoader_klass())) >>>>>> { >>>>>> 584 size = _primelist[0]; >>>>>> 585 } else { >>>>>> 586 size = _primelist[1]; >>>>>> 587 } >>>>>> 588 dictionary = new Dictionary(this, size); >>>>>> >>>>>> I'll do some investigation on this issue and get back to you. >>>>>> >>>>> >>>>> How about if someone uses PredictedLoadedClassCount, then we use >>>>> that to size all but the reflection and boot class loader? Then >>>>> if there's an application that has a class loader with a huge >>>>> amount of classes loaded in it, that would help this? It might >>>>> cost some footprint but an oversized table would simply be a >>>>> bigger array of pointers, so it might not be that bad to oversize. >>>>> >>>>> I think the long term solution is being able to resize these >>>>> entries and make the application provide arguments. Please let >>>>> me know what you find in your investigation and if that would work. >>>>>> The rest of the changes look good to me. >>>>> >>>>> Thank you! >>>>> Coleen >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>> >>>>>> On 6/23/17 4:42 PM, coleen.phillimore at oracle.com >>>>>> wrote: >>>>>>> Summary: Implement one dictionary per ClassLoaderData for faster >>>>>>> lookup and removal during class unloading >>>>>>> >>>>>>> See RFE for more details. >>>>>>> >>>>>>> open webrev at >>>>>>> http://cr.openjdk.java.net/~coleenp/7133093.01/webrev >>>>>>> >>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-7133093 >>>>>>> >>>>>>> Tested with full "nightly" run in rbt, plus locally class >>>>>>> loading and unloading tests: >>>>>>> >>>>>>> jtreg hotspot/test/runtime/ClassUnload >>>>>>> >>>>>>> jtreg hotspot/test/runtime/modules >>>>>>> >>>>>>> jtreg hotspot/test/gc/class_unloading >>>>>>> >>>>>>> make test-hotspot-closed-tonga FILTER=quick TEST_JOBS=4 >>>>>>> TEST=vm.parallel_class_loading >>>>>>> >>>>>>> csh ~/testing/run_jck9 (vm/lang/java_lang) >>>>>>> >>>>>>> runThese -jck - uses class loader isolation to run each jck test >>>>>>> and unloads tests when done (at -gc:5 intervals) >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Coleen >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From rednaxelafx at gmail.com Sat Jul 29 02:13:46 2017 From: rednaxelafx at gmail.com (Krystal Mok) Date: Fri, 28 Jul 2017 19:13:46 -0700 Subject: Where in OpenJDK repository can I find Hotspot VM source codes? In-Reply-To: <18D26DA3-12A7-49AB-B247-40B2C10346C9@corp.netease.com> References: <65511B52-BA7A-4F1B-AB3B-4217E25499FB@corp.netease.com> <18D26DA3-12A7-49AB-B247-40B2C10346C9@corp.netease.com> Message-ID: Hi Yijie, > Per your explain, there is no way to get source codes of Oracle HotSpot VM after JDK6u32? Unfortunately, no. What's opened sourced has always been only OpenJDK and not Oracle/Sun JDK. They do share most of the code but the license and restrictions and some certain little details are different. The debugging symbols of Oracle JDK product releases were never publicly available. You can always obtain a commercial license from Oracle for the JDK source code, under SCSL [1], but I doubt you're going to want that just for a one-off debugging scenario. There used to be a JRL [2] release of Sun/Oracle's JDK source code for non-commercial use, but that has been EOL'd and is no longer available. That's the closest you can get to a full Oracle JDK 6 source code package, under the JRL license. Note that it's not a open source license, and it's limited to research purposes only. Now, if the memory leak you're concerned about can be reproduced in earlier versions of Oracle JDK 6, you might want to try to reproduce that on a OpenJDK 6 build of HotSpot VM. If that does reproduce, you can easily make a debug / fastdebug build of HotSpot and debug with it; otherwise making a product build and debugging is also fine if it only reproduces there. If the memory leak does not reproduce on a latest OpenJDK 6 build / Zulu 6 binary release [3] / IcedTea 6, then if I were you I wouldn't worry about debugging the memory leak on Oracle JDK6u45 myself and just use what's there. - Kris [1]: https://en.wikipedia.org/wiki/Sun_Community_Source_License [2]: https://en.wikipedia.org/wiki/Java_Research_License [3]: http://zulu.org/download/ for Zulu 6u93 On Fri, Jul 28, 2017 at 2:37 AM, zengyijie wrote: > Hi Kris, > Thank you very much for detailed explain! > We are running that specific Oracle JVM in production, and encountered a > strange native memory leak, so we plan to build a debug JVM from source > codes. > Per your explain, there is no way to get source codes of Oracle HotSpot VM > after JDK6u32? How to get a Oracle Hotspot VM with debug symbol? > Thanks a lot, > > Yijie > > ? 2017?7?28????2:44?Krystal Mok ??? > > Hi Yijie, > > I don't think there is one that exactly matches what you want. > > I wrote a doc [1] long ago about the correspondence of Sun/Oracle JDK 6 > versus OpenJDK 6 in terms of the JVM used. Unfortunately, because of the > history behind OpenJDK 6 (it's a backport of OpenJDK 7 to be a JDK6 > release, rather than the open sourced version of Sun/Oracle JDK 6), there > was no exact correspondence to begin with. > > Back then, the HotSpot Expression release model was still in place, so the > Oracle JDK 6 and the then still-in-development JDK 7 would mostly share the > same JVM, that's how the OpenJDK 6 HotSpot VM still very closely match the > Oracle JDK 6 version -- less some proprietary features like > UseCompressedStrings, and a bit of delay in releasing the new tiered > compilation features, etc. > > After Oracle JDK6u32 or something, I don't think the OpenJDK 6 HotSpot VM > reflected changes to the Oracle JDK 6 any more, as JDK6 was going towards > EOL'd and less changes from Oracle JDK 6 were sync'd to OpenJDK 6. > Red Hat later took over the maintenance of the OpenJDK 6 release, and it > further diverged from the Oracle internal one. > > What's in Oracle JDK6u45 specifically that you wanted to check the sources > for, though? > > - Kris > > [1]: https://gist.github.com/rednaxelafx/925323 > > On Wed, Jul 5, 2017 at 2:18 AM, zengyijie > wrote: > >> Hi, we are running an outdated Oracle Java JDK6 with Hotspot VM, I have >> spent quite a while browsing openjdk repository, >> but I can?t find the source codes for the specific Hotspot VM version. >> The repository tip of http://hg.openjdk.java.net/jdk6/jdk6/hotspot/ < >> http://hg.openjdk.java.net/jdk6/jdk6/hotspot/> is only jdk6-b43, there?s >> no 1.6.0_45. >> Could someone help tell where to find it? >> The Oracle JDK version is as follows: >> >> java version "1.6.0_45" >> Java(TM) SE Runtime Environment (build 1.6.0_45-b06) >> Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01, mixed mode) > > > > From rohitarulraj at gmail.com Mon Jul 31 04:33:39 2017 From: rohitarulraj at gmail.com (Rohit Arul Raj) Date: Mon, 31 Jul 2017 10:03:39 +0530 Subject: Help with generating library call Message-ID: Hi All, I am newbie to openJDK and recently started working on openJDK8 hotspot. I need some help regarding one of the issues I am looking in to. My requirement is that for any object/array initialization, I want to replace the existing loop code with 'memset' and do a performance study. I have been looking through the code and haven't found any notes or documentation on how to generate a function call to native library function. Can any one point me in the right direction? Any reference to similar implementation of any other C or Math library function will be helpful? I ended up doing the following, but not sure if this is the right way to go about it. push (c_rarg2); push (c_rarg1); push (c_rarg0); call_VM_leaf(CAST_FROM_FN_PTR(address, memset),3); Thanks in advance, Rohit From muthusamy.chinnathambi at oracle.com Mon Jul 31 07:20:47 2017 From: muthusamy.chinnathambi at oracle.com (Muthusamy Chinnathambi) Date: Mon, 31 Jul 2017 00:20:47 -0700 (PDT) Subject: [8u] RFR for backport of JDK-8158012: Use SW prefetch instructions instead of BIS for allocation prefetches on SPARC Core C4 In-Reply-To: <3f0b16ad-2eeb-4326-bfd2-f94e17422527@default> References: <4768675b-cf31-4351-902a-51ab0a4bd5c7@default> <2d0ca8d5-139e-3b8b-a96d-6daa04d94c7b@oracle.com> <3f0b16ad-2eeb-4326-bfd2-f94e17422527@default> Message-ID: <53350135-c467-42da-84cc-a628435ab3e8@default> Ping... May I please get a second review Thanks, Muthusamy C -----Original Message----- From: Muthusamy Chinnathambi Sent: Tuesday, July 25, 2017 2:44 PM To: Vladimir Kozlov ; hotspot-dev at openjdk.java.net Subject: RE: [8u] RFR for backport of JDK-8158012: Use SW prefetch instructions instead of BIS for allocation prefetches on SPARC Core C4 May I please get a second review for this change. Thanks, Muthusamy C -----Original Message----- From: Vladimir Kozlov Sent: Monday, July 24, 2017 9:52 PM To: Muthusamy Chinnathambi ; hotspot-dev at openjdk.java.net Subject: Re: [8u] RFR for backport of JDK-8158012: Use SW prefetch instructions instead of BIS for allocation prefetches on SPARC Core C4 This looks good. Thanks, Vladimir On 7/23/17 11:56 PM, Muthusamy Chinnathambi wrote: > Hi Vladimir, > > Thanks for the review. > Please find the updated webrev at http://cr.openjdk.java.net/~shshahma/Muthu/8158012/webrev.01/ > > Regards, > Muthusamy C > > -----Original Message----- > From: Vladimir Kozlov > Sent: Friday, July 21, 2017 4:38 AM > To: Muthusamy Chinnathambi ; hotspot-dev at openjdk.java.net > Subject: Re: [8u] RFR for backport of JDK-8158012: Use SW prefetch instructions instead of BIS for allocation prefetches on SPARC Core C4 > > The indent for next comment line in vm_version_sparc.cpp is off: > > // Use BIS instr > > And AllocatePrefetchStyle set to 3 should be outside new check (should > be set for both cases). In JDK 9 there is following code which set it > for AllocatePrefetchInstr == 1 which JDK8 does not have: > > http://hg.openjdk.java.net/jdk9/jdk9/hotspot/file/eebdc7acebd4/src/cpu/sparc/vm/vm_version_sparc.cpp#l126 > > thanks, > Vladimir > > On 7/20/17 12:24 AM, Muthusamy Chinnathambi wrote: >> Hi, >> >> Please review the backport of bug: "JDK-8158012: Use SW prefetch instructions instead of BIS for allocation prefetches on SPARC Core C4" to jdk8u-dev >> >> Please note that this is not a clean backport due to conflict in comments and slight re-order of code. >> >> >> Webrev: http://cr.openjdk.java.net/~shshahma/Muthu/8158012/webrev.00/ >> jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-8158012 >> Original patch pushed to jdk9: http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/eebdc7acebd4 >> >> Test: Ran jprt and jtreg. >> >> Regards, >> Muthusamy C >> From aph at redhat.com Mon Jul 31 08:18:02 2017 From: aph at redhat.com (Andrew Haley) Date: Mon, 31 Jul 2017 09:18:02 +0100 Subject: Help with generating library call In-Reply-To: References: Message-ID: <436f03b4-f352-e2df-4afb-a70905d3927e@redhat.com> On 31/07/17 05:33, Rohit Arul Raj wrote: > Can any one point me in the right direction? Any reference to similar > implementation of any other C or Math library function will be > helpful? > > I ended up doing the following, but not sure if this is the right way > to go about it. > push (c_rarg2); > push (c_rarg1); > push (c_rarg0); > call_VM_leaf(CAST_FROM_FN_PTR(address, memset),3); This doesn't look totally wrong. There are many examples in the source. For example, look at any calls to MacroAssembler::call_VM and call_VM_leaf. Be aware, however, that if you're calling this from compiled code most registers are live so you're going to have to be extremely careful to save and restore them. One of the advantages of using HotSpot's builtins for zeroing is that we can know exactly which registers are used. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From kim.barrett at oracle.com Mon Jul 31 18:18:51 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 31 Jul 2017 14:18:51 -0400 Subject: RFR (M): 8184334: Generalizing Atomic with templates In-Reply-To: References: <72F3D5DE-E517-4E3A-81F3-556F5A2222DA@oracle.com> <596CD126.6080100@oracle.com> <596DD713.6080806@oracle.com> <77d5a9e7-b2b0-79e4-effd-903dfe226512@redhat.com> <596E1142.6040708@oracle.com> <3432d11c-5904-a446-1cb5-419627f1794f@redhat.com> <596E288B.90706@oracle.com> <773a0b64-8273-1a58-a46a-22ae18f927b7@redhat.com> <5970B3FD.607@oracle.com> <8e1d9a32-7e01-b917-df1d-595c4000e4b2@redhat.com> <597226E2.6090803@oracle.com> <2c637e28-8d95-cbaf-c29e-d01867ed9157@redhat.com> <5979FEB6.10506@oracle.com> <3C8BDDFA-1044-47F6-B15C-6DE2085ACF7C@oracle.com> <597B5FC6.5020702@oracle.com> <4e10660a-4dee-78bb-333e-fe99a9c2295d@redhat.com> Message-ID: > On Jul 28, 2017, at 12:25 PM, Erik Osterlund wrote: > > Hi Andrew, > > In that case, feel free to propose a revised solution while I am gone. Erik has asked me to try to make progress on this while he's on vacation, rather than possibly letting it sit until he gets back. (This is just one of a large stack of patches Erik has prepared toward the GC access interface that he's been talking about for a while. Providing a solid base for that work is the underlying goal for changing atomics.) I'm getting caught up on the discussion, and looking at Erik's most recent proposal. I should have something more concrete soon, possibly just agreeing with Erik's recent ideas, possibly some alternative. I'm also open to suggestions?