From dean.long at oracle.com Mon May 1 19:14:21 2017 From: dean.long at oracle.com (dean.long at oracle.com) Date: Mon, 1 May 2017 12:14:21 -0700 Subject: RFR[s] 8179305 - Avoid repeated calls to JavaThread::last_frame in InterpreterRuntime In-Reply-To: <59032224.8020005@oracle.com> References: <59032224.8020005@oracle.com> Message-ID: <139deaf0-ff94-ac4d-e5d4-5dfbe13202f9@oracle.com> Looks OK to me, but it seems arbitrary now which calls can use LastFrameAccessor methods and which need to go through the get_frame() back-door. dl On 4/28/17 4:06 AM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8179305 > http://cr.openjdk.java.net/~iklam/jdk10/8179305-avoid-last-frame.v01/ > > Summary: > > JavaThread::last_frame() is an expensive call. We use the helper class > LastFrameAccessor holds the value of last_frame: > > OLD: > methodHandle m (thread, method(thread));// calls > JavaThread::last_frame() internally > Bytecode_loadconstant ldc(m, bci(thread));// calls > JavaThread::last_frame() internally > > NEW: > LastFrameAccessor last_frame(thread); > methodHandle m (thread, last_frame.method()); > Bytecode_loadconstant ldc(m, last_frame.bci()); > > Testing: > > Preliminary benchmarking shows significant VM start-up improvement: > > java -version: > no CDS = 80.35 ms -> 79.14ms (-1.5%) > w/ CDS = 53.89ms -> 52.62ms (-2.36%) > > clojure sample app: > no CDS = 1442.46ms -> 1436.11ms (-0.44%) > w/ CDS = 695.78ms -> 679.52ms (-2.34%) > > Thanks > - Ioi From harold.seigel at oracle.com Mon May 1 19:36:15 2017 From: harold.seigel at oracle.com (harold seigel) Date: Mon, 1 May 2017 15:36:15 -0400 Subject: RFR 8178604: JVM does not allow defining boot loader modules in exploded build after module system initialization Message-ID: <85e36988-a145-6805-8659-56c738013f58@oracle.com> Hi, Please review this JDK-10 bug fix to allow defining of modules to the boot loader in exploded builds after module system initialization. The fix uses the module lock to synchronize access to the _exploded_entries data structure (which is only used by exploded builds). Note that the above capability already exists for regular builds. Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8178604/webrev/index.html JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8178604 The fix was tested with JCK tests, the JTreg hotspot, java/io, java/lang, java/util and other tests, the RBT tier2 -tier5 tests, the co-located NSK tests, and with JPRT. The JTReg and JCK tests were run with an exploded build. Also, the example program in the JBS bug was run by hand to verify the fix. Thanks, Harold From ioi.lam at oracle.com Mon May 1 19:54:25 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 01 May 2017 12:54:25 -0700 Subject: RFR[s] 8179305 - Avoid repeated calls to JavaThread::last_frame in InterpreterRuntime In-Reply-To: <139deaf0-ff94-ac4d-e5d4-5dfbe13202f9@oracle.com> References: <59032224.8020005@oracle.com> <139deaf0-ff94-ac4d-e5d4-5dfbe13202f9@oracle.com> Message-ID: <59079271.1070700@oracle.com> Hi Dean, Thanks for the review. I left a few thread->last_frame() that were inside ifdef ASSERT, but you're right, I should change those as well to be consistent: diff -r dbee2fa1d3df src/share/vm/interpreter/interpreterRuntime.cpp --- a/src/share/vm/interpreter/interpreterRuntime.cpp Mon May 01 11:16:01 2017 -0700 +++ b/src/share/vm/interpreter/interpreterRuntime.cpp Mon May 01 12:30:01 2017 -0700 @@ -642,7 +642,7 @@ IRT_ENTRY_NO_ASYNC(void, InterpreterRuntime::monitorenter(JavaThread* thread, BasicObjectLock* elem)) #ifdef ASSERT - thread->last_frame().interpreter_frame_verify_monitor(elem); + LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); #endif @@ -659,7 +659,7 @@ #ifdef ASSERT - thread->last_frame().interpreter_frame_verify_monitor(elem); + LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); #endif IRT_END @@ -667,7 +667,7 @@ IRT_ENTRY_NO_ASYNC(void, InterpreterRuntime::monitorexit(JavaThread* thread, BasicObjectLock* elem)) #ifdef ASSERT - thread->last_frame().interpreter_frame_verify_monitor(elem); + LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); #endif @@ -680,7 +680,7 @@ #ifdef ASSERT - thread->last_frame().interpreter_frame_verify_monitor(elem); + LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); #endif IRT_END What do you think? Thanks - Ioi On 5/1/17 12:14 PM, dean.long at oracle.com wrote: > Looks OK to me, but it seems arbitrary now which calls can use > LastFrameAccessor methods and which need to go through the get_frame() > back-door. > > dl > > > On 4/28/17 4:06 AM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8179305 >> http://cr.openjdk.java.net/~iklam/jdk10/8179305-avoid-last-frame.v01/ >> >> Summary: >> >> JavaThread::last_frame() is an expensive call. We use the helper >> class LastFrameAccessor holds the value of last_frame: >> >> OLD: >> methodHandle m (thread, method(thread));// calls >> JavaThread::last_frame() internally >> Bytecode_loadconstant ldc(m, bci(thread));// calls >> JavaThread::last_frame() internally >> >> NEW: >> LastFrameAccessor last_frame(thread); >> methodHandle m (thread, last_frame.method()); >> Bytecode_loadconstant ldc(m, last_frame.bci()); >> >> Testing: >> >> Preliminary benchmarking shows significant VM start-up improvement: >> >> java -version: >> no CDS = 80.35 ms -> 79.14ms (-1.5%) >> w/ CDS = 53.89ms -> 52.62ms (-2.36%) >> >> clojure sample app: >> no CDS = 1442.46ms -> 1436.11ms (-0.44%) >> w/ CDS = 695.78ms -> 679.52ms (-2.34%) >> >> Thanks >> - Ioi > From lois.foltan at oracle.com Mon May 1 20:14:34 2017 From: lois.foltan at oracle.com (Lois Foltan) Date: Mon, 1 May 2017 16:14:34 -0400 Subject: RFR 8178604: JVM does not allow defining boot loader modules in exploded build after module system initialization In-Reply-To: <85e36988-a145-6805-8659-56c738013f58@oracle.com> References: <85e36988-a145-6805-8659-56c738013f58@oracle.com> Message-ID: Hi Harold, Looks good. A couple of comments: src/share/vm/classfile/classLoader.cpp line #828 - please take out the Module_lock when the _exploded_entries is created line #1402 - I like the new factored out method find_first_module_cpe(), however, instead of adding a new "need_lock" parameter, I would rather see code in find_first_module_cpe() that takes the Module_lock if the module_list parameter equals ClassLoader::_exploded_entries Thanks, Lois On 5/1/2017 3:36 PM, harold seigel wrote: > Hi, > > Please review this JDK-10 bug fix to allow defining of modules to the > boot loader in exploded builds after module system initialization. > The fix uses the module lock to synchronize access to the > _exploded_entries data structure (which is only used by exploded builds). > > Note that the above capability already exists for regular builds. > > Open Webrev: > http://cr.openjdk.java.net/~hseigel/bug_8178604/webrev/index.html > > JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8178604 > > The fix was tested with JCK tests, the JTreg hotspot, java/io, > java/lang, java/util and other tests, the RBT tier2 -tier5 tests, the > co-located NSK tests, and with JPRT. The JTReg and JCK tests were run > with an exploded build. Also, the example program in the JBS bug was > run by hand to verify the fix. > > Thanks, Harold > From Alan.Bateman at oracle.com Mon May 1 20:28:42 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 1 May 2017 21:28:42 +0100 Subject: 8178380: Module system implementation refresh (5/2017 update) Message-ID: As I mentioned in another thread, we need to get the changes accumulated in jake to jdk9/dev. JDK-8178380 [1] has the summary of the changes that have accumulated since the last refresh. One note on the hotspot repo is that the changes to dynamically augment the platform modules run into JDK-8178604 when testing with an exploded build. So there is a partial fix for this in jake. Harold has the more complete fix in review on hotspot-runtime-dev for JDK 10. The webrevs with the changes is here: http://cr.openjdk.java.net/~alanb/8178380/1/ -Alan [1] https://bugs.openjdk.java.net/browse/JDK-8178380 From lois.foltan at oracle.com Mon May 1 20:34:08 2017 From: lois.foltan at oracle.com (Lois Foltan) Date: Mon, 1 May 2017 16:34:08 -0400 Subject: 8178380: Module system implementation refresh (5/2017 update) In-Reply-To: References: Message-ID: Hi Alan, Hotspot changes look good. Thanks, Lois On 5/1/2017 4:28 PM, Alan Bateman wrote: > As I mentioned in another thread, we need to get the changes > accumulated in jake to jdk9/dev. > > JDK-8178380 [1] has the summary of the changes that have accumulated > since the last refresh. > > One note on the hotspot repo is that the changes to dynamically > augment the platform modules run into JDK-8178604 when testing with an > exploded build. So there is a partial fix for this in jake. Harold has > the more complete fix in review on hotspot-runtime-dev for JDK 10. > > The webrevs with the changes is here: > http://cr.openjdk.java.net/~alanb/8178380/1/ > > -Alan > > [1] https://bugs.openjdk.java.net/browse/JDK-8178380 > From dean.long at oracle.com Mon May 1 21:03:52 2017 From: dean.long at oracle.com (dean.long at oracle.com) Date: Mon, 1 May 2017 14:03:52 -0700 Subject: RFR[s] 8179305 - Avoid repeated calls to JavaThread::last_frame in InterpreterRuntime In-Reply-To: <59079271.1070700@oracle.com> References: <59032224.8020005@oracle.com> <139deaf0-ff94-ac4d-e5d4-5dfbe13202f9@oracle.com> <59079271.1070700@oracle.com> Message-ID: <0f308a93-f06a-003e-8ae4-eac402a78604@oracle.com> I wasn't complaining about those, because you don't really need a LastFrameAccessor there. I was really talking about where you have a LastFrameAccessor but need to escape back to frame using get_frame() because you didn't add a new accessor method: 950 for( BasicObjectLock *kptr = last_frame.get_frame().interpreter_frame_monitor_end(); 951 kptr < last_frame.get_frame().interpreter_frame_monitor_begin(); 952 kptr = last_frame.get_frame().next_monitor_in_interpreter_frame(kptr) ) { versus something like last_frame.monitor_end()/monitor_begin()/next_monitor(). dl On 5/1/17 12:54 PM, Ioi Lam wrote: > Hi Dean, > > Thanks for the review. I left a few thread->last_frame() that were > inside ifdef ASSERT, but you're right, I should change those as well > to be consistent: > > diff -r dbee2fa1d3df src/share/vm/interpreter/interpreterRuntime.cpp > --- a/src/share/vm/interpreter/interpreterRuntime.cpp Mon May 01 > 11:16:01 2017 -0700 > +++ b/src/share/vm/interpreter/interpreterRuntime.cpp Mon May 01 > 12:30:01 2017 -0700 > @@ -642,7 +642,7 @@ > IRT_ENTRY_NO_ASYNC(void, InterpreterRuntime::monitorenter(JavaThread* > thread, BasicObjectLock* elem)) > #ifdef ASSERT > - thread->last_frame().interpreter_frame_verify_monitor(elem); > + > LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); > #endif > > @@ -659,7 +659,7 @@ > #ifdef ASSERT > - thread->last_frame().interpreter_frame_verify_monitor(elem); > + > LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); > #endif > IRT_END > > @@ -667,7 +667,7 @@ > IRT_ENTRY_NO_ASYNC(void, InterpreterRuntime::monitorexit(JavaThread* > thread, BasicObjectLock* elem)) > #ifdef ASSERT > - thread->last_frame().interpreter_frame_verify_monitor(elem); > + > LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); > #endif > > @@ -680,7 +680,7 @@ > #ifdef ASSERT > - thread->last_frame().interpreter_frame_verify_monitor(elem); > + > LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); > #endif > IRT_END > > What do you think? > > Thanks > - Ioi > > > > On 5/1/17 12:14 PM, dean.long at oracle.com wrote: >> Looks OK to me, but it seems arbitrary now which calls can use >> LastFrameAccessor methods and which need to go through the >> get_frame() back-door. >> >> dl >> >> >> On 4/28/17 4:06 AM, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8179305 >>> http://cr.openjdk.java.net/~iklam/jdk10/8179305-avoid-last-frame.v01/ >>> >>> Summary: >>> >>> JavaThread::last_frame() is an expensive call. We use the helper >>> class LastFrameAccessor holds the value of last_frame: >>> >>> OLD: >>> methodHandle m (thread, method(thread));// calls >>> JavaThread::last_frame() internally >>> Bytecode_loadconstant ldc(m, bci(thread));// calls >>> JavaThread::last_frame() internally >>> >>> NEW: >>> LastFrameAccessor last_frame(thread); >>> methodHandle m (thread, last_frame.method()); >>> Bytecode_loadconstant ldc(m, last_frame.bci()); >>> >>> Testing: >>> >>> Preliminary benchmarking shows significant VM start-up improvement: >>> >>> java -version: >>> no CDS = 80.35 ms -> 79.14ms (-1.5%) >>> w/ CDS = 53.89ms -> 52.62ms (-2.36%) >>> >>> clojure sample app: >>> no CDS = 1442.46ms -> 1436.11ms (-0.44%) >>> w/ CDS = 695.78ms -> 679.52ms (-2.34%) >>> >>> Thanks >>> - Ioi >> > From ioi.lam at oracle.com Mon May 1 21:18:18 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 01 May 2017 14:18:18 -0700 Subject: RFR[s] 8179305 - Avoid repeated calls to JavaThread::last_frame in InterpreterRuntime In-Reply-To: <0f308a93-f06a-003e-8ae4-eac402a78604@oracle.com> References: <59032224.8020005@oracle.com> <139deaf0-ff94-ac4d-e5d4-5dfbe13202f9@oracle.com> <59079271.1070700@oracle.com> <0f308a93-f06a-003e-8ae4-eac402a78604@oracle.com> Message-ID: <5907A61A.6030404@oracle.com> On 5/1/17 2:03 PM, dean.long at oracle.com wrote: > > I wasn't complaining about those, because you don't really need a > LastFrameAccessor there. I was really talking about where you have a > LastFrameAccessor but need to escape back to frame using get_frame() > because you didn't add a new accessor method: > > > 950 for( BasicObjectLock *kptr = last_frame.get_frame().interpreter_frame_monitor_end(); > 951 kptr < last_frame.get_frame().interpreter_frame_monitor_begin(); > 952 kptr = last_frame.get_frame().next_monitor_in_interpreter_frame(kptr) ) { > > versus something like > last_frame.monitor_end()/monitor_begin()/next_monitor(). > I could add those accessor methods, but frame has lots of methods, and I don't want to wrap each of them in case someone uses a new function in the future. An alternative (which I really don't like) is to override the "->" or "()" operators like the Handle class: last_frame->interpreter_frame_monitor_begin(); or last_frame().interpreter_frame_monitor_begin(); Thanks - Ioi > > dl > > > On 5/1/17 12:54 PM, Ioi Lam wrote: >> Hi Dean, >> >> Thanks for the review. I left a few thread->last_frame() that were >> inside ifdef ASSERT, but you're right, I should change those as well >> to be consistent: >> >> diff -r dbee2fa1d3df src/share/vm/interpreter/interpreterRuntime.cpp >> --- a/src/share/vm/interpreter/interpreterRuntime.cpp Mon May 01 >> 11:16:01 2017 -0700 >> +++ b/src/share/vm/interpreter/interpreterRuntime.cpp Mon May 01 >> 12:30:01 2017 -0700 >> @@ -642,7 +642,7 @@ >> IRT_ENTRY_NO_ASYNC(void, >> InterpreterRuntime::monitorenter(JavaThread* thread, BasicObjectLock* >> elem)) >> #ifdef ASSERT >> - thread->last_frame().interpreter_frame_verify_monitor(elem); >> + >> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >> #endif >> >> @@ -659,7 +659,7 @@ >> #ifdef ASSERT >> - thread->last_frame().interpreter_frame_verify_monitor(elem); >> + >> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >> #endif >> IRT_END >> >> @@ -667,7 +667,7 @@ >> IRT_ENTRY_NO_ASYNC(void, InterpreterRuntime::monitorexit(JavaThread* >> thread, BasicObjectLock* elem)) >> #ifdef ASSERT >> - thread->last_frame().interpreter_frame_verify_monitor(elem); >> + >> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >> #endif >> >> @@ -680,7 +680,7 @@ >> #ifdef ASSERT >> - thread->last_frame().interpreter_frame_verify_monitor(elem); >> + >> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >> #endif >> IRT_END >> >> What do you think? >> >> Thanks >> - Ioi >> >> >> >> On 5/1/17 12:14 PM, dean.long at oracle.com wrote: >>> Looks OK to me, but it seems arbitrary now which calls can use >>> LastFrameAccessor methods and which need to go through the >>> get_frame() back-door. >>> >>> dl >>> >>> >>> On 4/28/17 4:06 AM, Ioi Lam wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8179305 >>>> http://cr.openjdk.java.net/~iklam/jdk10/8179305-avoid-last-frame.v01/ >>>> >>>> Summary: >>>> >>>> JavaThread::last_frame() is an expensive call. We use the helper >>>> class LastFrameAccessor holds the value of last_frame: >>>> >>>> OLD: >>>> methodHandle m (thread, method(thread));// calls >>>> JavaThread::last_frame() internally >>>> Bytecode_loadconstant ldc(m, bci(thread));// calls >>>> JavaThread::last_frame() internally >>>> >>>> NEW: >>>> LastFrameAccessor last_frame(thread); >>>> methodHandle m (thread, last_frame.method()); >>>> Bytecode_loadconstant ldc(m, last_frame.bci()); >>>> >>>> Testing: >>>> >>>> Preliminary benchmarking shows significant VM start-up improvement: >>>> >>>> java -version: >>>> no CDS = 80.35 ms -> 79.14ms (-1.5%) >>>> w/ CDS = 53.89ms -> 52.62ms (-2.36%) >>>> >>>> clojure sample app: >>>> no CDS = 1442.46ms -> 1436.11ms (-0.44%) >>>> w/ CDS = 695.78ms -> 679.52ms (-2.34%) >>>> >>>> Thanks >>>> - Ioi >>> >> > From dean.long at oracle.com Mon May 1 22:58:11 2017 From: dean.long at oracle.com (dean.long at oracle.com) Date: Mon, 1 May 2017 15:58:11 -0700 Subject: RFR[s] 8179305 - Avoid repeated calls to JavaThread::last_frame in InterpreterRuntime In-Reply-To: <5907A61A.6030404@oracle.com> References: <59032224.8020005@oracle.com> <139deaf0-ff94-ac4d-e5d4-5dfbe13202f9@oracle.com> <59079271.1070700@oracle.com> <0f308a93-f06a-003e-8ae4-eac402a78604@oracle.com> <5907A61A.6030404@oracle.com> Message-ID: <7756ca89-e9e9-e012-1043-f3d0f3f6037f@oracle.com> I was expecting you to wrap just the methods that InterpreterRuntime uses. dl On 5/1/17 2:18 PM, Ioi Lam wrote: > > > On 5/1/17 2:03 PM, dean.long at oracle.com wrote: >> >> I wasn't complaining about those, because you don't really need a >> LastFrameAccessor there. I was really talking about where you have a >> LastFrameAccessor but need to escape back to frame using get_frame() >> because you didn't add a new accessor method: >> >> >> 950 for( BasicObjectLock *kptr = >> last_frame.get_frame().interpreter_frame_monitor_end(); >> 951 kptr < last_frame.get_frame().interpreter_frame_monitor_begin(); >> 952 kptr = >> last_frame.get_frame().next_monitor_in_interpreter_frame(kptr) ) { >> >> versus something like >> last_frame.monitor_end()/monitor_begin()/next_monitor(). >> > I could add those accessor methods, but frame has lots of methods, and > I don't want to wrap each of them in case someone uses a new function > in the future. > > An alternative (which I really don't like) is to override the "->" or > "()" operators like the Handle class: > > last_frame->interpreter_frame_monitor_begin(); > or > last_frame().interpreter_frame_monitor_begin(); > > Thanks > - Ioi >> >> dl >> >> >> On 5/1/17 12:54 PM, Ioi Lam wrote: >>> Hi Dean, >>> >>> Thanks for the review. I left a few thread->last_frame() that were >>> inside ifdef ASSERT, but you're right, I should change those as well >>> to be consistent: >>> >>> diff -r dbee2fa1d3df src/share/vm/interpreter/interpreterRuntime.cpp >>> --- a/src/share/vm/interpreter/interpreterRuntime.cpp Mon May 01 >>> 11:16:01 2017 -0700 >>> +++ b/src/share/vm/interpreter/interpreterRuntime.cpp Mon May 01 >>> 12:30:01 2017 -0700 >>> @@ -642,7 +642,7 @@ >>> IRT_ENTRY_NO_ASYNC(void, >>> InterpreterRuntime::monitorenter(JavaThread* thread, >>> BasicObjectLock* elem)) >>> #ifdef ASSERT >>> - thread->last_frame().interpreter_frame_verify_monitor(elem); >>> + >>> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >>> #endif >>> >>> @@ -659,7 +659,7 @@ >>> #ifdef ASSERT >>> - thread->last_frame().interpreter_frame_verify_monitor(elem); >>> + >>> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >>> #endif >>> IRT_END >>> >>> @@ -667,7 +667,7 @@ >>> IRT_ENTRY_NO_ASYNC(void, >>> InterpreterRuntime::monitorexit(JavaThread* thread, BasicObjectLock* >>> elem)) >>> #ifdef ASSERT >>> - thread->last_frame().interpreter_frame_verify_monitor(elem); >>> + >>> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >>> #endif >>> >>> @@ -680,7 +680,7 @@ >>> #ifdef ASSERT >>> - thread->last_frame().interpreter_frame_verify_monitor(elem); >>> + >>> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >>> #endif >>> IRT_END >>> >>> What do you think? >>> >>> Thanks >>> - Ioi >>> >>> >>> >>> On 5/1/17 12:14 PM, dean.long at oracle.com wrote: >>>> Looks OK to me, but it seems arbitrary now which calls can use >>>> LastFrameAccessor methods and which need to go through the >>>> get_frame() back-door. >>>> >>>> dl >>>> >>>> >>>> On 4/28/17 4:06 AM, Ioi Lam wrote: >>>>> https://bugs.openjdk.java.net/browse/JDK-8179305 >>>>> http://cr.openjdk.java.net/~iklam/jdk10/8179305-avoid-last-frame.v01/ >>>>> >>>>> Summary: >>>>> >>>>> JavaThread::last_frame() is an expensive call. We use the helper >>>>> class LastFrameAccessor holds the value of last_frame: >>>>> >>>>> OLD: >>>>> methodHandle m (thread, method(thread));// calls >>>>> JavaThread::last_frame() internally >>>>> Bytecode_loadconstant ldc(m, bci(thread));// calls >>>>> JavaThread::last_frame() internally >>>>> >>>>> NEW: >>>>> LastFrameAccessor last_frame(thread); >>>>> methodHandle m (thread, last_frame.method()); >>>>> Bytecode_loadconstant ldc(m, last_frame.bci()); >>>>> >>>>> Testing: >>>>> >>>>> Preliminary benchmarking shows significant VM start-up improvement: >>>>> >>>>> java -version: >>>>> no CDS = 80.35 ms -> 79.14ms (-1.5%) >>>>> w/ CDS = 53.89ms -> 52.62ms (-2.36%) >>>>> >>>>> clojure sample app: >>>>> no CDS = 1442.46ms -> 1436.11ms (-0.44%) >>>>> w/ CDS = 695.78ms -> 679.52ms (-2.34%) >>>>> >>>>> Thanks >>>>> - Ioi >>>> >>> >> > From ioi.lam at oracle.com Tue May 2 00:09:41 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 01 May 2017 17:09:41 -0700 Subject: RFR[s] 8179305 - Avoid repeated calls to JavaThread::last_frame in InterpreterRuntime In-Reply-To: <7756ca89-e9e9-e012-1043-f3d0f3f6037f@oracle.com> References: <59032224.8020005@oracle.com> <139deaf0-ff94-ac4d-e5d4-5dfbe13202f9@oracle.com> <59079271.1070700@oracle.com> <0f308a93-f06a-003e-8ae4-eac402a78604@oracle.com> <5907A61A.6030404@oracle.com> <7756ca89-e9e9-e012-1043-f3d0f3f6037f@oracle.com> Message-ID: <5907CE45.3080003@oracle.com> Hi Dean, I've wrapped those 4 methods. I also added 'const' to the wrapper methods where possible. Here's the delta from the last posted webrev: http://cr.openjdk.java.net/~iklam/jdk10/8179305-avoid-last-frame.v02.delta/ Thanks - Ioi On 5/1/17 3:58 PM, dean.long at oracle.com wrote: > > I was expecting you to wrap just the methods that InterpreterRuntime uses. > > dl > > > On 5/1/17 2:18 PM, Ioi Lam wrote: >> >> >> On 5/1/17 2:03 PM, dean.long at oracle.com wrote: >>> >>> I wasn't complaining about those, because you don't really need a >>> LastFrameAccessor there. I was really talking about where you have >>> a LastFrameAccessor but need to escape back to frame using >>> get_frame() because you didn't add a new accessor method: >>> >>> >>> 950 for( BasicObjectLock *kptr = last_frame.get_frame().interpreter_frame_monitor_end(); >>> 951 kptr < last_frame.get_frame().interpreter_frame_monitor_begin(); >>> 952 kptr = last_frame.get_frame().next_monitor_in_interpreter_frame(kptr) ) { >>> >>> versus something like >>> last_frame.monitor_end()/monitor_begin()/next_monitor(). >>> >> I could add those accessor methods, but frame has lots of methods, >> and I don't want to wrap each of them in case someone uses a new >> function in the future. >> >> An alternative (which I really don't like) is to override the "->" or >> "()" operators like the Handle class: >> >> last_frame->interpreter_frame_monitor_begin(); >> or >> last_frame().interpreter_frame_monitor_begin(); >> >> Thanks >> - Ioi >>> >>> dl >>> >>> >>> On 5/1/17 12:54 PM, Ioi Lam wrote: >>>> Hi Dean, >>>> >>>> Thanks for the review. I left a few thread->last_frame() that were >>>> inside ifdef ASSERT, but you're right, I should change those as >>>> well to be consistent: >>>> >>>> diff -r dbee2fa1d3df src/share/vm/interpreter/interpreterRuntime.cpp >>>> --- a/src/share/vm/interpreter/interpreterRuntime.cpp Mon May 01 >>>> 11:16:01 2017 -0700 >>>> +++ b/src/share/vm/interpreter/interpreterRuntime.cpp Mon May 01 >>>> 12:30:01 2017 -0700 >>>> @@ -642,7 +642,7 @@ >>>> IRT_ENTRY_NO_ASYNC(void, >>>> InterpreterRuntime::monitorenter(JavaThread* thread, >>>> BasicObjectLock* elem)) >>>> #ifdef ASSERT >>>> - thread->last_frame().interpreter_frame_verify_monitor(elem); >>>> + >>>> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >>>> #endif >>>> >>>> @@ -659,7 +659,7 @@ >>>> #ifdef ASSERT >>>> - thread->last_frame().interpreter_frame_verify_monitor(elem); >>>> + >>>> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >>>> #endif >>>> IRT_END >>>> >>>> @@ -667,7 +667,7 @@ >>>> IRT_ENTRY_NO_ASYNC(void, >>>> InterpreterRuntime::monitorexit(JavaThread* thread, >>>> BasicObjectLock* elem)) >>>> #ifdef ASSERT >>>> - thread->last_frame().interpreter_frame_verify_monitor(elem); >>>> + >>>> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >>>> #endif >>>> >>>> @@ -680,7 +680,7 @@ >>>> #ifdef ASSERT >>>> - thread->last_frame().interpreter_frame_verify_monitor(elem); >>>> + >>>> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >>>> #endif >>>> IRT_END >>>> >>>> What do you think? >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>> >>>> On 5/1/17 12:14 PM, dean.long at oracle.com wrote: >>>>> Looks OK to me, but it seems arbitrary now which calls can use >>>>> LastFrameAccessor methods and which need to go through the >>>>> get_frame() back-door. >>>>> >>>>> dl >>>>> >>>>> >>>>> On 4/28/17 4:06 AM, Ioi Lam wrote: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8179305 >>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8179305-avoid-last-frame.v01/ >>>>>> >>>>>> >>>>>> Summary: >>>>>> >>>>>> JavaThread::last_frame() is an expensive call. We use the helper >>>>>> class LastFrameAccessor holds the value of last_frame: >>>>>> >>>>>> OLD: >>>>>> methodHandle m (thread, method(thread));// calls >>>>>> JavaThread::last_frame() internally >>>>>> Bytecode_loadconstant ldc(m, bci(thread));// calls >>>>>> JavaThread::last_frame() internally >>>>>> >>>>>> NEW: >>>>>> LastFrameAccessor last_frame(thread); >>>>>> methodHandle m (thread, last_frame.method()); >>>>>> Bytecode_loadconstant ldc(m, last_frame.bci()); >>>>>> >>>>>> Testing: >>>>>> >>>>>> Preliminary benchmarking shows significant VM start-up improvement: >>>>>> >>>>>> java -version: >>>>>> no CDS = 80.35 ms -> 79.14ms (-1.5%) >>>>>> w/ CDS = 53.89ms -> 52.62ms (-2.36%) >>>>>> >>>>>> clojure sample app: >>>>>> no CDS = 1442.46ms -> 1436.11ms (-0.44%) >>>>>> w/ CDS = 695.78ms -> 679.52ms (-2.34%) >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>>> >>>> >>> >> > From dean.long at oracle.com Tue May 2 01:18:35 2017 From: dean.long at oracle.com (dean.long at oracle.com) Date: Mon, 1 May 2017 18:18:35 -0700 Subject: RFR[s] 8179305 - Avoid repeated calls to JavaThread::last_frame in InterpreterRuntime In-Reply-To: <5907CE45.3080003@oracle.com> References: <59032224.8020005@oracle.com> <139deaf0-ff94-ac4d-e5d4-5dfbe13202f9@oracle.com> <59079271.1070700@oracle.com> <0f308a93-f06a-003e-8ae4-eac402a78604@oracle.com> <5907A61A.6030404@oracle.com> <7756ca89-e9e9-e012-1043-f3d0f3f6037f@oracle.com> <5907CE45.3080003@oracle.com> Message-ID: This looks fine, but why not shorten those names like we do for bcp(), bci(), etc? dl On 5/1/17 5:09 PM, Ioi Lam wrote: > Hi Dean, > > I've wrapped those 4 methods. I also added 'const' to the wrapper > methods where possible. Here's the delta from the last posted webrev: > > http://cr.openjdk.java.net/~iklam/jdk10/8179305-avoid-last-frame.v02.delta/ > > Thanks > - Ioi > > On 5/1/17 3:58 PM, dean.long at oracle.com wrote: >> >> I was expecting you to wrap just the methods that InterpreterRuntime >> uses. >> >> dl >> >> >> On 5/1/17 2:18 PM, Ioi Lam wrote: >>> >>> >>> On 5/1/17 2:03 PM, dean.long at oracle.com wrote: >>>> >>>> I wasn't complaining about those, because you don't really need a >>>> LastFrameAccessor there. I was really talking about where you have >>>> a LastFrameAccessor but need to escape back to frame using >>>> get_frame() because you didn't add a new accessor method: >>>> >>>> >>>> 950 for( BasicObjectLock *kptr = >>>> last_frame.get_frame().interpreter_frame_monitor_end(); >>>> 951 kptr < last_frame.get_frame().interpreter_frame_monitor_begin(); >>>> 952 kptr = >>>> last_frame.get_frame().next_monitor_in_interpreter_frame(kptr) ) { >>>> >>>> versus something like >>>> last_frame.monitor_end()/monitor_begin()/next_monitor(). >>>> >>> I could add those accessor methods, but frame has lots of methods, >>> and I don't want to wrap each of them in case someone uses a new >>> function in the future. >>> >>> An alternative (which I really don't like) is to override the "->" >>> or "()" operators like the Handle class: >>> >>> last_frame->interpreter_frame_monitor_begin(); >>> or >>> last_frame().interpreter_frame_monitor_begin(); >>> >>> Thanks >>> - Ioi >>>> >>>> dl >>>> >>>> >>>> On 5/1/17 12:54 PM, Ioi Lam wrote: >>>>> Hi Dean, >>>>> >>>>> Thanks for the review. I left a few thread->last_frame() that were >>>>> inside ifdef ASSERT, but you're right, I should change those as >>>>> well to be consistent: >>>>> >>>>> diff -r dbee2fa1d3df src/share/vm/interpreter/interpreterRuntime.cpp >>>>> --- a/src/share/vm/interpreter/interpreterRuntime.cpp Mon May 01 >>>>> 11:16:01 2017 -0700 >>>>> +++ b/src/share/vm/interpreter/interpreterRuntime.cpp Mon May 01 >>>>> 12:30:01 2017 -0700 >>>>> @@ -642,7 +642,7 @@ >>>>> IRT_ENTRY_NO_ASYNC(void, >>>>> InterpreterRuntime::monitorenter(JavaThread* thread, >>>>> BasicObjectLock* elem)) >>>>> #ifdef ASSERT >>>>> - thread->last_frame().interpreter_frame_verify_monitor(elem); >>>>> + >>>>> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >>>>> #endif >>>>> >>>>> @@ -659,7 +659,7 @@ >>>>> #ifdef ASSERT >>>>> - thread->last_frame().interpreter_frame_verify_monitor(elem); >>>>> + >>>>> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >>>>> #endif >>>>> IRT_END >>>>> >>>>> @@ -667,7 +667,7 @@ >>>>> IRT_ENTRY_NO_ASYNC(void, >>>>> InterpreterRuntime::monitorexit(JavaThread* thread, >>>>> BasicObjectLock* elem)) >>>>> #ifdef ASSERT >>>>> - thread->last_frame().interpreter_frame_verify_monitor(elem); >>>>> + >>>>> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >>>>> #endif >>>>> >>>>> @@ -680,7 +680,7 @@ >>>>> #ifdef ASSERT >>>>> - thread->last_frame().interpreter_frame_verify_monitor(elem); >>>>> + >>>>> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >>>>> #endif >>>>> IRT_END >>>>> >>>>> What do you think? >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> >>>>> >>>>> On 5/1/17 12:14 PM, dean.long at oracle.com wrote: >>>>>> Looks OK to me, but it seems arbitrary now which calls can use >>>>>> LastFrameAccessor methods and which need to go through the >>>>>> get_frame() back-door. >>>>>> >>>>>> dl >>>>>> >>>>>> >>>>>> On 4/28/17 4:06 AM, Ioi Lam wrote: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8179305 >>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8179305-avoid-last-frame.v01/ >>>>>>> >>>>>>> >>>>>>> Summary: >>>>>>> >>>>>>> JavaThread::last_frame() is an expensive call. We use the helper >>>>>>> class LastFrameAccessor holds the value of last_frame: >>>>>>> >>>>>>> OLD: >>>>>>> methodHandle m (thread, method(thread));// calls >>>>>>> JavaThread::last_frame() internally >>>>>>> Bytecode_loadconstant ldc(m, bci(thread));// calls >>>>>>> JavaThread::last_frame() internally >>>>>>> >>>>>>> NEW: >>>>>>> LastFrameAccessor last_frame(thread); >>>>>>> methodHandle m (thread, last_frame.method()); >>>>>>> Bytecode_loadconstant ldc(m, last_frame.bci()); >>>>>>> >>>>>>> Testing: >>>>>>> >>>>>>> Preliminary benchmarking shows significant VM start-up improvement: >>>>>>> >>>>>>> java -version: >>>>>>> no CDS = 80.35 ms -> 79.14ms (-1.5%) >>>>>>> w/ CDS = 53.89ms -> 52.62ms (-2.36%) >>>>>>> >>>>>>> clojure sample app: >>>>>>> no CDS = 1442.46ms -> 1436.11ms (-0.44%) >>>>>>> w/ CDS = 695.78ms -> 679.52ms (-2.34%) >>>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>> >>>>> >>>> >>> >> > From ioi.lam at oracle.com Tue May 2 01:26:23 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 01 May 2017 18:26:23 -0700 Subject: RFR[s] 8179305 - Avoid repeated calls to JavaThread::last_frame in InterpreterRuntime In-Reply-To: References: <59032224.8020005@oracle.com> <139deaf0-ff94-ac4d-e5d4-5dfbe13202f9@oracle.com> <59079271.1070700@oracle.com> <0f308a93-f06a-003e-8ae4-eac402a78604@oracle.com> <5907A61A.6030404@oracle.com> <7756ca89-e9e9-e012-1043-f3d0f3f6037f@oracle.com> <5907CE45.3080003@oracle.com> Message-ID: <5907E03F.3050603@oracle.com> OK, I'll shorten them to: oop*callee_receiver*(Symbol* signature) { return _last_frame.interpreter_callee_receiver(signature); } BasicObjectLock**monitor_begin*() const { return _last_frame.interpreter_frame_monitor_end(); } BasicObjectLock**monitor_end*() const { return _last_frame.interpreter_frame_monitor_begin(); } BasicObjectLock**next_monitor*(BasicObjectLock* current) const { return _last_frame.next_monitor_in_interpreter_frame(current); } What do you think? - Ioi On 5/1/17 6:18 PM, dean.long at oracle.com wrote: > > This looks fine, but why not shorten those names like we do for bcp(), > bci(), etc? > > dl > > On 5/1/17 5:09 PM, Ioi Lam wrote: >> Hi Dean, >> >> I've wrapped those 4 methods. I also added 'const' to the wrapper >> methods where possible. Here's the delta from the last posted webrev: >> >> http://cr.openjdk.java.net/~iklam/jdk10/8179305-avoid-last-frame.v02.delta/ >> >> Thanks >> - Ioi >> >> On 5/1/17 3:58 PM, dean.long at oracle.com wrote: >>> >>> I was expecting you to wrap just the methods that InterpreterRuntime >>> uses. >>> >>> dl >>> >>> >>> On 5/1/17 2:18 PM, Ioi Lam wrote: >>>> >>>> >>>> On 5/1/17 2:03 PM, dean.long at oracle.com wrote: >>>>> >>>>> I wasn't complaining about those, because you don't really need a >>>>> LastFrameAccessor there. I was really talking about where you >>>>> have a LastFrameAccessor but need to escape back to frame using >>>>> get_frame() because you didn't add a new accessor method: >>>>> >>>>> >>>>> 950 for( BasicObjectLock *kptr = last_frame.get_frame().interpreter_frame_monitor_end(); >>>>> 951 kptr < last_frame.get_frame().interpreter_frame_monitor_begin(); >>>>> 952 kptr = last_frame.get_frame().next_monitor_in_interpreter_frame(kptr) ) { >>>>> >>>>> versus something like >>>>> last_frame.monitor_end()/monitor_begin()/next_monitor(). >>>>> >>>> I could add those accessor methods, but frame has lots of methods, >>>> and I don't want to wrap each of them in case someone uses a new >>>> function in the future. >>>> >>>> An alternative (which I really don't like) is to override the "->" >>>> or "()" operators like the Handle class: >>>> >>>> last_frame->interpreter_frame_monitor_begin(); >>>> or >>>> last_frame().interpreter_frame_monitor_begin(); >>>> >>>> Thanks >>>> - Ioi >>>>> >>>>> dl >>>>> >>>>> >>>>> On 5/1/17 12:54 PM, Ioi Lam wrote: >>>>>> Hi Dean, >>>>>> >>>>>> Thanks for the review. I left a few thread->last_frame() that >>>>>> were inside ifdef ASSERT, but you're right, I should change those >>>>>> as well to be consistent: >>>>>> >>>>>> diff -r dbee2fa1d3df src/share/vm/interpreter/interpreterRuntime.cpp >>>>>> --- a/src/share/vm/interpreter/interpreterRuntime.cpp Mon May 01 >>>>>> 11:16:01 2017 -0700 >>>>>> +++ b/src/share/vm/interpreter/interpreterRuntime.cpp Mon May 01 >>>>>> 12:30:01 2017 -0700 >>>>>> @@ -642,7 +642,7 @@ >>>>>> IRT_ENTRY_NO_ASYNC(void, >>>>>> InterpreterRuntime::monitorenter(JavaThread* thread, >>>>>> BasicObjectLock* elem)) >>>>>> #ifdef ASSERT >>>>>> - thread->last_frame().interpreter_frame_verify_monitor(elem); >>>>>> + >>>>>> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >>>>>> #endif >>>>>> >>>>>> @@ -659,7 +659,7 @@ >>>>>> #ifdef ASSERT >>>>>> - thread->last_frame().interpreter_frame_verify_monitor(elem); >>>>>> + >>>>>> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >>>>>> #endif >>>>>> IRT_END >>>>>> >>>>>> @@ -667,7 +667,7 @@ >>>>>> IRT_ENTRY_NO_ASYNC(void, >>>>>> InterpreterRuntime::monitorexit(JavaThread* thread, >>>>>> BasicObjectLock* elem)) >>>>>> #ifdef ASSERT >>>>>> - thread->last_frame().interpreter_frame_verify_monitor(elem); >>>>>> + >>>>>> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >>>>>> #endif >>>>>> >>>>>> @@ -680,7 +680,7 @@ >>>>>> #ifdef ASSERT >>>>>> - thread->last_frame().interpreter_frame_verify_monitor(elem); >>>>>> + >>>>>> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >>>>>> #endif >>>>>> IRT_END >>>>>> >>>>>> What do you think? >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>> >>>>>> >>>>>> On 5/1/17 12:14 PM, dean.long at oracle.com wrote: >>>>>>> Looks OK to me, but it seems arbitrary now which calls can use >>>>>>> LastFrameAccessor methods and which need to go through the >>>>>>> get_frame() back-door. >>>>>>> >>>>>>> dl >>>>>>> >>>>>>> >>>>>>> On 4/28/17 4:06 AM, Ioi Lam wrote: >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8179305 >>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8179305-avoid-last-frame.v01/ >>>>>>>> >>>>>>>> >>>>>>>> Summary: >>>>>>>> >>>>>>>> JavaThread::last_frame() is an expensive call. We use the >>>>>>>> helper class LastFrameAccessor holds the value of last_frame: >>>>>>>> >>>>>>>> OLD: >>>>>>>> methodHandle m (thread, method(thread));// calls >>>>>>>> JavaThread::last_frame() internally >>>>>>>> Bytecode_loadconstant ldc(m, bci(thread));// calls >>>>>>>> JavaThread::last_frame() internally >>>>>>>> >>>>>>>> NEW: >>>>>>>> LastFrameAccessor last_frame(thread); >>>>>>>> methodHandle m (thread, last_frame.method()); >>>>>>>> Bytecode_loadconstant ldc(m, last_frame.bci()); >>>>>>>> >>>>>>>> Testing: >>>>>>>> >>>>>>>> Preliminary benchmarking shows significant VM start-up >>>>>>>> improvement: >>>>>>>> >>>>>>>> java -version: >>>>>>>> no CDS = 80.35 ms -> 79.14ms (-1.5%) >>>>>>>> w/ CDS = 53.89ms -> 52.62ms (-2.36%) >>>>>>>> >>>>>>>> clojure sample app: >>>>>>>> no CDS = 1442.46ms -> 1436.11ms (-0.44%) >>>>>>>> w/ CDS = 695.78ms -> 679.52ms (-2.34%) >>>>>>>> >>>>>>>> Thanks >>>>>>>> - Ioi >>>>>>> >>>>>> >>>>> >>>> >>> >> > From dean.long at oracle.com Tue May 2 01:56:30 2017 From: dean.long at oracle.com (dean.long at oracle.com) Date: Mon, 1 May 2017 18:56:30 -0700 Subject: RFR[s] 8179305 - Avoid repeated calls to JavaThread::last_frame in InterpreterRuntime In-Reply-To: <5907E03F.3050603@oracle.com> References: <59032224.8020005@oracle.com> <139deaf0-ff94-ac4d-e5d4-5dfbe13202f9@oracle.com> <59079271.1070700@oracle.com> <0f308a93-f06a-003e-8ae4-eac402a78604@oracle.com> <5907A61A.6030404@oracle.com> <7756ca89-e9e9-e012-1043-f3d0f3f6037f@oracle.com> <5907CE45.3080003@oracle.com> <5907E03F.3050603@oracle.com> Message-ID: <11b2c1fa-06ff-7316-a6d1-190581d5af93@oracle.com> Perfectly reasonable. Thanks. dl On 5/1/17 6:26 PM, Ioi Lam wrote: > OK, I'll shorten them to: > oop*callee_receiver*(Symbol* signature) { > return _last_frame.interpreter_callee_receiver(signature); > } > BasicObjectLock**monitor_begin*() const { > return _last_frame.interpreter_frame_monitor_end(); > } > BasicObjectLock**monitor_end*() const { > return _last_frame.interpreter_frame_monitor_begin(); > } > BasicObjectLock**next_monitor*(BasicObjectLock* current) const { > return _last_frame.next_monitor_in_interpreter_frame(current); > } > What do you think? > - Ioi > > On 5/1/17 6:18 PM, dean.long at oracle.com wrote: >> >> This looks fine, but why not shorten those names like we do for >> bcp(), bci(), etc? >> >> dl >> >> On 5/1/17 5:09 PM, Ioi Lam wrote: >>> Hi Dean, >>> >>> I've wrapped those 4 methods. I also added 'const' to the wrapper >>> methods where possible. Here's the delta from the last posted webrev: >>> >>> http://cr.openjdk.java.net/~iklam/jdk10/8179305-avoid-last-frame.v02.delta/ >>> >>> Thanks >>> - Ioi >>> >>> On 5/1/17 3:58 PM, dean.long at oracle.com wrote: >>>> >>>> I was expecting you to wrap just the methods that >>>> InterpreterRuntime uses. >>>> >>>> dl >>>> >>>> >>>> On 5/1/17 2:18 PM, Ioi Lam wrote: >>>>> >>>>> >>>>> On 5/1/17 2:03 PM, dean.long at oracle.com wrote: >>>>>> >>>>>> I wasn't complaining about those, because you don't really need a >>>>>> LastFrameAccessor there. I was really talking about where you >>>>>> have a LastFrameAccessor but need to escape back to frame using >>>>>> get_frame() because you didn't add a new accessor method: >>>>>> >>>>>> >>>>>> 950 for( BasicObjectLock *kptr = >>>>>> last_frame.get_frame().interpreter_frame_monitor_end(); >>>>>> 951 kptr < last_frame.get_frame().interpreter_frame_monitor_begin(); >>>>>> 952 kptr = >>>>>> last_frame.get_frame().next_monitor_in_interpreter_frame(kptr) ) { >>>>>> >>>>>> versus something like >>>>>> last_frame.monitor_end()/monitor_begin()/next_monitor(). >>>>>> >>>>> I could add those accessor methods, but frame has lots of methods, >>>>> and I don't want to wrap each of them in case someone uses a new >>>>> function in the future. >>>>> >>>>> An alternative (which I really don't like) is to override the "->" >>>>> or "()" operators like the Handle class: >>>>> >>>>> last_frame->interpreter_frame_monitor_begin(); >>>>> or >>>>> last_frame().interpreter_frame_monitor_begin(); >>>>> >>>>> Thanks >>>>> - Ioi >>>>>> >>>>>> dl >>>>>> >>>>>> >>>>>> On 5/1/17 12:54 PM, Ioi Lam wrote: >>>>>>> Hi Dean, >>>>>>> >>>>>>> Thanks for the review. I left a few thread->last_frame() that >>>>>>> were inside ifdef ASSERT, but you're right, I should change >>>>>>> those as well to be consistent: >>>>>>> >>>>>>> diff -r dbee2fa1d3df >>>>>>> src/share/vm/interpreter/interpreterRuntime.cpp >>>>>>> --- a/src/share/vm/interpreter/interpreterRuntime.cpp Mon May 01 >>>>>>> 11:16:01 2017 -0700 >>>>>>> +++ b/src/share/vm/interpreter/interpreterRuntime.cpp Mon May 01 >>>>>>> 12:30:01 2017 -0700 >>>>>>> @@ -642,7 +642,7 @@ >>>>>>> IRT_ENTRY_NO_ASYNC(void, >>>>>>> InterpreterRuntime::monitorenter(JavaThread* thread, >>>>>>> BasicObjectLock* elem)) >>>>>>> #ifdef ASSERT >>>>>>> - thread->last_frame().interpreter_frame_verify_monitor(elem); >>>>>>> + >>>>>>> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >>>>>>> #endif >>>>>>> >>>>>>> @@ -659,7 +659,7 @@ >>>>>>> #ifdef ASSERT >>>>>>> - thread->last_frame().interpreter_frame_verify_monitor(elem); >>>>>>> + >>>>>>> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >>>>>>> #endif >>>>>>> IRT_END >>>>>>> >>>>>>> @@ -667,7 +667,7 @@ >>>>>>> IRT_ENTRY_NO_ASYNC(void, >>>>>>> InterpreterRuntime::monitorexit(JavaThread* thread, >>>>>>> BasicObjectLock* elem)) >>>>>>> #ifdef ASSERT >>>>>>> - thread->last_frame().interpreter_frame_verify_monitor(elem); >>>>>>> + >>>>>>> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >>>>>>> #endif >>>>>>> >>>>>>> @@ -680,7 +680,7 @@ >>>>>>> #ifdef ASSERT >>>>>>> - thread->last_frame().interpreter_frame_verify_monitor(elem); >>>>>>> + >>>>>>> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >>>>>>> #endif >>>>>>> IRT_END >>>>>>> >>>>>>> What do you think? >>>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 5/1/17 12:14 PM, dean.long at oracle.com wrote: >>>>>>>> Looks OK to me, but it seems arbitrary now which calls can use >>>>>>>> LastFrameAccessor methods and which need to go through the >>>>>>>> get_frame() back-door. >>>>>>>> >>>>>>>> dl >>>>>>>> >>>>>>>> >>>>>>>> On 4/28/17 4:06 AM, Ioi Lam wrote: >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8179305 >>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8179305-avoid-last-frame.v01/ >>>>>>>>> >>>>>>>>> >>>>>>>>> Summary: >>>>>>>>> >>>>>>>>> JavaThread::last_frame() is an expensive call. We use the >>>>>>>>> helper class LastFrameAccessor holds the value of last_frame: >>>>>>>>> >>>>>>>>> OLD: >>>>>>>>> methodHandle m (thread, method(thread));// calls >>>>>>>>> JavaThread::last_frame() internally >>>>>>>>> Bytecode_loadconstant ldc(m, bci(thread));// calls >>>>>>>>> JavaThread::last_frame() internally >>>>>>>>> >>>>>>>>> NEW: >>>>>>>>> LastFrameAccessor last_frame(thread); >>>>>>>>> methodHandle m (thread, last_frame.method()); >>>>>>>>> Bytecode_loadconstant ldc(m, last_frame.bci()); >>>>>>>>> >>>>>>>>> Testing: >>>>>>>>> >>>>>>>>> Preliminary benchmarking shows significant VM start-up >>>>>>>>> improvement: >>>>>>>>> >>>>>>>>> java -version: >>>>>>>>> no CDS = 80.35 ms -> 79.14ms (-1.5%) >>>>>>>>> w/ CDS = 53.89ms -> 52.62ms (-2.36%) >>>>>>>>> >>>>>>>>> clojure sample app: >>>>>>>>> no CDS = 1442.46ms -> 1436.11ms (-0.44%) >>>>>>>>> w/ CDS = 695.78ms -> 679.52ms (-2.34%) >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> - Ioi >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From mandy.chung at oracle.com Tue May 2 02:47:04 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 1 May 2017 19:47:04 -0700 Subject: 8178380: Module system implementation refresh (5/2017 update) In-Reply-To: References: Message-ID: <072A3264-1459-4E76-B7B7-11FD4C838602@oracle.com> > On May 1, 2017, at 1:28 PM, Alan Bateman wrote: > > The webrevs with the changes is here: > http://cr.openjdk.java.net/~alanb/8178380/1/ I have reviewed all repos. Mostly looks good. This is the first round of comments. I will continue the review and plan to play a little more with the new launcher options. src/share/vm/services/attachListener.cpp 355 // Dynamic loading agents is not default by default typo: s/not default/not enabled? src/share/vm/services/diagnosticCommand.cpp 771 loadAgentModule(CHECK); 772 Handle loader = Handle(THREAD, SystemDictionary::java_system_loader()); 773 Klass* k = SystemDictionary::resolve_or_fail(vmSymbols::jdk_internal_agent_Agent(), loader, Handle(), true, CHECK); 774 instanceKlassHandle ik (THREAD, k); It?s better to refactor this to loadAgentClass that returns Klass or handle? src/java.base/share/classes/java/lang/ClassLoader.java 134 * platform class loader may delegate to application class loader. ?the? application class loader 135 * In other words, classes in modules defined to the application class 136 * loader may be visible to the platform class loader. It would help if this statement makes it clear that the classes in modules defined to the application class loader (only that are required by the upgradeable modules) are visible to the platform class loader. It would also be good to mention that ClassLoader::getPackages will not return the packages defined to the application class loader since application class loader is not its ancestor. 377 * @apiNote If the parent is specified as {@code null} (for the 378 * bootstrap class loader) then there is no guarantee that all platform 379 * classes are visible. What about: If the parent is specified as {@code null} (for the bootstrap class loader) then there is no guarantee that all platform classes are visible. Platform classes defined by the platform class loader and its ancestors except bootstrap class loader are not visible to this class loader. src/java.base/share/classes/jdk/internal/loader/BuiltinClassLoader.java 174 /** 175 * Register a module this this class loader. This has the effect of making 176 * the types in the module visible. 177 */ 178 public void loadModule(ModuleReference mref) { Typo in line 175 ?this this class loader?. src/java.base/share/classes/jdk/internal/module/ModuleBootstrap.java 147 // special mode to boot with only java.base, ignores other options 148 if (System.getProperty("jdk.module.minimumBoot") != null) { 149 return createMinimalBootLayer(); 150 } src/java.base/share/classes/jdk/internal/module/ModulePatcher.java 123 try (JarFile jf = new JarFile(file.toString())) { - what is the bug being fixed by this line? src/java.base/share/classes/jdk/internal/module/ModulePath.java 536 if (packages.contains(pn)) builder.mainClass(mainClass); Nit: it would be consistent if this is broken into two lines. src/java.base/share/classes/sun/launcher/LauncherHelper.java 960 bootLayer.modules().stream() 961 .map(m -> cf.findModule(m.getName())) 962 .flatMap(Optional::stream) This can be replaced with cf.modules().stream() 988 .map(e -> Stream.concat(Stream.of(e.source()), 989 toStringStream(e.modifiers())) Formatting nit: line 989 should be indented more to the right 1064 * image ot be less than modules than not in the run-time image. typo: ?ot? Should these new tracing methods drop the printToStderr argument? This argument exists for compatibility to print help message to stderr. These new tracing options will print to stdout. For validate modules, you suggested to print the errors to stderr which I agree. src/java.base/share/classes/sun/launcher/resources/launcher.properties 60 \ --limit-modules [,...]\n\ 61 \ limit the universe of observable modules\n\ -?limit-modules is intended for testing purpose. I wonder if this should be moved to ?-extra-help. 72 \ The --validate-modules option may be is useful for finding\n\ typo: ?may be is useful? src/jdk.jartool/share/classes/sun/tools/jar/Main.java 619 if (dflag) { 620 // "--describe-module/-d" does not require file argument(s), 621 // but does accept --release 622 usageError(getMsg("error.bad.dflag")); 623 return false; 624 } Mandy From chris.hegarty at oracle.com Tue May 2 10:11:31 2017 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 2 May 2017 11:11:31 +0100 Subject: 8178380: Module system implementation refresh (5/2017 update) In-Reply-To: <072A3264-1459-4E76-B7B7-11FD4C838602@oracle.com> References: <072A3264-1459-4E76-B7B7-11FD4C838602@oracle.com> Message-ID: <44af0927-5fa5-7106-1a35-17a93c80e91d@oracle.com> Mandy, On 02/05/17 03:47, Mandy Chung wrote: > >> ... > src/jdk.jartool/share/classes/sun/tools/jar/Main.java > 619 if (dflag) { > 620 // "--describe-module/-d" does not require file argument(s), > 621 // but does accept --release > 622 usageError(getMsg("error.bad.dflag")); > 623 return false; > 624 } Sorry, I'm not sure of the issue here. I added this check when updating the describe option to accept an optional `--release` ( the effective runtime version ) when describing a multi-release modular JAR. This check is required as passing `-C`, in this context, is an error. There are exiting tests that verify this. -Chris. From Alan.Bateman at oracle.com Tue May 2 11:28:33 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 2 May 2017 12:28:33 +0100 Subject: 8178380: Module system implementation refresh (5/2017 update) In-Reply-To: <072A3264-1459-4E76-B7B7-11FD4C838602@oracle.com> References: <072A3264-1459-4E76-B7B7-11FD4C838602@oracle.com> Message-ID: <49754fcf-0016-424e-e5d7-5ae59ab3b7a8@oracle.com> On 02/05/2017 03:47, Mandy Chung wrote: > : > src/share/vm/services/attachListener.cpp > > 355 // Dynamic loading agents is not default by default > > typo: s/not default/not enabled? I agree, the comment is confused and better to remove it. > > src/share/vm/services/diagnosticCommand.cpp > 771 loadAgentModule(CHECK); > 772 Handle loader = Handle(THREAD, SystemDictionary::java_system_loader()); > 773 Klass* k = SystemDictionary::resolve_or_fail(vmSymbols::jdk_internal_agent_Agent(), loader, Handle(), true, CHECK); > 774 instanceKlassHandle ik (THREAD, k); > > It?s better to refactor this to loadAgentClass that returns Klass or handle? I agree although the duplicate code seems to have been there for some time, I think better if we create an issue to have it cleaned up in JDK 10. > > src/java.base/share/classes/java/lang/ClassLoader.java > > 134 * platform class loader may delegate to application class loader. > > ?the? application class loader > > 135 * In other words, classes in modules defined to the application class > 136 * loader may be visible to the platform class loader. > > It would help if this statement makes it clear that the classes in modules defined to the application class loader (only that are required by the upgradeable modules) are visible to the platform class loader. There is a typo in the javadoc, it should be "in named modules" and I think that will make it much clearer. > > It would also be good to mention that ClassLoader::getPackages will not > return the packages defined to the application class loader since > application class loader is not its ancestor. > > 377 * @apiNote If the parent is specified as {@code null} (for the > 378 * bootstrap class loader) then there is no guarantee that all platform > 379 * classes are visible. > > What about: > > If the parent is specified as {@code null} (for the bootstrap class loader) > then there is no guarantee that all platform classes are visible. > Platform classes defined by the platform class loader and its ancestors > except bootstrap class loader are not visible to this class loader. I think this would clutter the class description and it might be better if we added a note to the getPackages method. > > src/java.base/share/classes/jdk/internal/loader/BuiltinClassLoader.java > > 174 /** > 175 * Register a module this this class loader. This has the effect of making > 176 * the types in the module visible. > 177 */ > 178 public void loadModule(ModuleReference mref) { > > Typo in line 175 ?this this class loader?. Got it - thanks. > > src/java.base/share/classes/jdk/internal/module/ModuleBootstrap.java > > 147 // special mode to boot with only java.base, ignores other options > 148 if (System.getProperty("jdk.module.minimumBoot") != null) { > 149 return createMinimalBootLayer(); > 150 } Is there an issue here? > > src/java.base/share/classes/jdk/internal/module/ModulePatcher.java > 123 try (JarFile jf = new JarFile(file.toString())) { > - what is the bug being fixed by this line? This is part of the issue that Remi brought up last week related to overriding the default file system provider. I've replaced test/java/nio/file/spi/SetDefaultProvider.java with a more comprehensive test to test overriding with exploded and modular JAR that are patched / not-patched. > > src/java.base/share/classes/jdk/internal/module/ModulePath.java > 536 if (packages.contains(pn)) builder.mainClass(mainClass); > Nit: it would be consistent if this is broken into two lines. Okay. > > src/java.base/share/classes/sun/launcher/LauncherHelper.java > 960 bootLayer.modules().stream() > 961 .map(m -> cf.findModule(m.getName())) > 962 .flatMap(Optional::stream) > > This can be replaced with cf.modules().stream() Okay. > > 988 .map(e -> Stream.concat(Stream.of(e.source()), > 989 toStringStream(e.modifiers())) > Formatting nit: line 989 should be indented more to the right I think the IDE moved it :-) > > 1064 * image ot be less than modules than not in the run-time image. > > typo: ?ot? Okay. > > Should these new tracing methods drop the printToStderr argument? > This argument exists for compatibility to print help message to stderr. > These new tracing options will print to stdout. I was trying to keep it consistent but I think you are right, we should just init the output to System.out in these supporting methods. > For validate modules, > you suggested to print the errors to stderr which I agree. I think but it's not in jake yet. > > src/java.base/share/classes/sun/launcher/resources/launcher.properties > 60 \ --limit-modules [,...]\n\ > 61 \ limit the universe of observable modules\n\ > > -?limit-modules is intended for testing purpose. I wonder if > this should be moved to ?-extra-help. I've also been thinking it should move. > > 72 \ The --validate-modules option may be is useful for finding\n\ > > typo: ?may be is useful? Okay. Thanks for going through all the changes. I will publish a new set of webrevs tomorrow once we have all the changes. -Alan From thomas.stuefe at gmail.com Tue May 2 12:12:40 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 2 May 2017 14:12:40 +0200 Subject: RFR(xxs, jdk10): 8171508: os::jvm_path -XXaltjvm processing error after 8066474 In-Reply-To: References: <5fa63987-f7cf-4527-c420-58657c398da4@oracle.com> Message-ID: Hi David, On Sat, Apr 29, 2017 at 12:33 AM, David Holmes wrote: > On 29/04/2017 1:18 AM, coleen.phillimore at oracle.com wrote: > >> On 4/27/17 1:07 PM, Vladimir Kozlov wrote: >> >>> I think we just need to remove -XXaltjvm. >>> >> >> +1 >> >> I used to use this all the time. Now, never. >> > > I'll spell this out again just to be very sure there is no confusion - > including possibly mine :) > > The code under discussion does not come into play when passing -XXaltjvm > to the java launcher. So that use of -XXaltjvm is completely independent > and out of scope for this fix, so if it needs removing that is a separate > issue. > > The code under discussion previously was only activated when using the > gamma-launcher, and that was later changed to activate if the (badly named) > -Dsun.java.launcher.is_altjvm=true is set, which is then used by the > gtestlauncher. > > However the gtestlauncher always sets JAVA_HOME so the code looking for > jre/lib/... can fail and not cause any problems. And that code was > not updated when we got rid of the subdirectory, nor when we got rid > of the subdirectory; and on OSX never worked in the first place as it > never had an directory! > > So I think we can simply delete all the code code that tries to look for > jre/lib/... and only use the JAVA_HOME logic. > > Thanks, > David > > Thanks for the historical digging work! So, this has nothing to do with -XXaltjvm at all. I renamed the bug report accordingly and updated the bug description. Ok, so I will remove the JRE-detection; the new version will basically just if (is_altjvm) jdk_path = JAVA_HOME/... I'll repost the fix under the new bug description when it is done. Kind Regards, Thomas > > Coleen >> >> >>> Originally we use it for testing locally built JVM (when we had >>> ability to build Hotspot JVM separate). And for binary search of >>> changes which may cause a failure by selecting different JVM for the >>> same JDK. OR even different JDK releases (Hotspot Express). >>> >>> But since JDK 8 it become almost impossible since JVM code is tightly >>> connected to JDK code. >>> >>> Regards, >>> Vladimir >>> >>> On 4/27/17 5:21 AM, David Holmes wrote: >>> >>>> Hi Thomas, >>>> >>>> Thanks for the detailed investigation. I am very confused by all this. >>>> I'm wondering whether we should just kill off -XXaltjvm altogether? >>>> Otherwise I'm not at all sure what is the "right" thing to do as I'm not >>>> clear what it is intended to do. >>>> >>>> David >>>> >>>> On 27/04/2017 7:25 PM, Thomas St?fe wrote: >>>> >>>>> Hi David, >>>>> >>>>> You are right, this is more complicated. >>>>> >>>>> -- >>>>> >>>>> Here is what I think was originally intended: >>>>> >>>>> Scenario a): I start java from jdk A and give it, via -XXaltjvm, a path >>>>> pointing to a libjvm.so in jdk B. In this case, os::jvm_path() should >>>>> detect that jdk B is a full JDK, and return the path to jdk B. So, all >>>>> JDK libraries should be loaded from jdk B. >>>>> >>>>> Scenario b) I start java from jdk A and give it, via -XXaltjvm, a path >>>>> pointing to a standalone libjvm.so. In this case, os::jvm_path() should >>>>> see that there is no full valid JDK around the alternate libjvm.so, and >>>>> uses a JDK from JAVA_HOME. Failing that (no JAVA_HOME set), it should >>>>> use the path from the loaded alternative libjvm.so. >>>>> >>>>> -- >>>>> >>>>> However, there are errors: >>>>> >>>>> Error 1: -XXaltjvm is currently not handled by os::jvm_path(): >>>>> Arguments >>>>> >>>> >::sun_java_launcher_is_altjvm >>>>> >>>>> >>>>> >>>> m&project=openjdk-jdk10-hs>() >>>>> >>>>> >>>>> returns always false and we skip altjvm handling altogether. This is >>>>> because -Dsun.java.launcher.is_altjvm=true is missing. The Launcher >>>>> should set this property if XXaltjvm is set. >>>>> >>>>> Side note: the gtestLauncher sets >>>>> -Dsun.java.launcher.is_altjvm=true. So >>>>> starting gtests is currently the only way to hit the altjvm path in >>>>> os::jvm_path(). >>>>> >>>>> Side note 2: Your CDS scenario just works because the path you add to >>>>> -XXaltjvm is inside a full JDK (actually the primary JDK). This >>>>> needs no >>>>> altjvm handling at all in os::jvm_path() - it just returns the path to >>>>> the primary jdk. (I wonder why you even need to specify XXaltjvm? Is >>>>> this to distinguish between server and client VM?) >>>>> >>>>> -- >>>>> >>>>> So I do some trials with -XXaltjvm= together with explicitely >>>>> setting -Dsun.java.launcher.is_altjvm=true. >>>>> >>>>> Scenario a): jdk A = ./images, jdk B = images-2 >>>>> >>>>> without JAVA_HOME set: ./images/jdk/bin/java >>>>> -Dsun.java.launcher.is_altjvm=true -XXaltjvm=./images-2/jdk/lib/server >>>>> >>>>> We load libjvm.so and libjava.so from images-2. Looks correct? Yes, but >>>>> I stepped thru with a debugger, what actually happens is that the >>>>> full-JDK-recognition at os_linux.cpp:2348 fails. Then it checks for >>>>> JAVA_HOME, which it is not set. Then it just gives up and falls back to >>>>> the path to the alternate libjvm.so at os_linux.cpp:2364.Which happens >>>>> to be the correct path anyway for jdk B, >>>>> >>>>> Now lets set JAVA_HOME to some completely unrelated JDK. os::jvm_path() >>>>> should ignore the value and still return path to jdk b, yes? But no: >>>>> >>>>> - .../output $ export >>>>> JAVA_HOME=/sapmnt/depot/tools/gen/linuxx86_64/licenseware/jse/1.5.0 >>>>> - .../output $ ./images/jdk/bin/java -Dsun.java.launcher.is_altjvm= >>>>> true >>>>> -XXaltjvm=./images-2/images/jdk/lib/server >>>>> Error occurred during initialization of VM >>>>> Unable to load native library: >>>>> /net/sapmnt.depot/tools/gen/linuxx86_64/licenseware/jse/1.5. >>>>> 0/jre/lib/libjava.so: >>>>> >>>>> >>>>> cannot open shared object file: No such file or directory >>>>> >>>>> This is Error 2: os::jvm_path() does not recognize "images-2" to be a >>>>> valid full JDK, so it falls back to JAVA_HOME, which in this case >>>>> points >>>>> to some unrelated older JDK. Here, in this case, it fails to load the >>>>> libjava.so. But it should never have attempted to load it. >>>>> >>>>> This is the bug my patch attempts to fix, admittedly you are right, >>>>> there may be more to it (the "jre" for example). >>>>> >>>>> Scenario b) just works, if JAVA_HOME is set correctly. This is what the >>>>> gtestLauncher does: It sets -Dsun.java.launcher.is_altjvm=true and >>>>> sets >>>>> JAVA_HOME to the value of the -jdk: argument (see gtestMain.cpp). >>>>> >>>>> -- >>>>> >>>>> So the first error is IMHO that when -XXaltjvm is passed it should set >>>>> -Dsun.java.launcher.is_altjvm=true but it does not. >>>>> >>>>> The second error is that the half-baked full-jdk-detection in >>>>> os::jvm_path() does not work. You are right, there are more errors than >>>>> the 5-slashes-path-traversal. This makes Scenario a) misbehave if >>>>> JAVA_HOME happens to be set to another JDK. You will either not >>>>> load, as >>>>> in my case, or load JDK libraries from another JDK. >>>>> >>>>> I honestly do not like this coding at all and would love to see this >>>>> made simpler. This is very difficult to understand. >>>>> >>>>> Kind Regards, Thomas >>>>> >>>>> >>>>> On Thu, Apr 27, 2017 at 1:31 AM, David Holmes >>>> > wrote: >>>>> >>>>> Hi Thomas, >>>>> >>>>> Backing up a few steps ... >>>>> >>>>> On 26/04/2017 5:10 PM, Thomas St?fe wrote: >>>>> >>>>> Hi all, >>>>> >>>>> may I please have a review for this tiny fix. 8066474 removed >>>>> the >>>>> directory from the images and since then -XXaltjvm was slightly >>>>> broken. >>>>> When handling XXaltjvm, os::jvm_path() examines the path of the >>>>> libjvm.so >>>>> to check if it is part of what it considers a standard JDK by >>>>> traversing a >>>>> number of slashes up the path and looking for "/jre/lib". That >>>>> number of >>>>> slashes was off since 8066474. >>>>> >>>>> >>>>> Hold on a moment! We don't have a jre subdirectory any more either! >>>>> The image for a jdk directly contains lib/server/libjvm.so (for >>>>> example). >>>>> >>>>> Also jvm_path is used for dumping the shared archive using the >>>>> default location. If I do: >>>>> >>>>> > ./images/jdk/bin/java -XXaltjvm=images/jdk/lib/server/ >>>>> -Xshare:dump >>>>> >>>>> it works perfectly correctly: >>>>> >>>>> > find . -name classes.jsa >>>>> ./images/jdk/lib/server/classes.jsa >>>>> >>>>> So where's the bug ?? >>>>> >>>>> David >>>>> ----- >>>>> >>>>> >>>>> >>>>> webrev: >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8171508-os_jvm_pa >>>>> th_xxaltjvm_processing_error_after_8066474/jdk10-webrev.00/ >>>>> webrev/index.html >>>>> >>>>> >>>>> >>>>> >>>> ath_xxaltjvm_processing_error_after_8066474/jdk10-webrev.00/ >>>>> webrev/index.html> >>>>> >>>>> >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8171508 >>>>> >>>>> >>>>> Note that this only affects cases where the alternate libjvm.so >>>>> is part of >>>>> a full jdk, so it does not affect the gtestLauncher. >>>>> >>>>> Thanks & Regards, Thomas >>>>> >>>>> >>>>> >> From harold.seigel at oracle.com Tue May 2 13:03:07 2017 From: harold.seigel at oracle.com (harold seigel) Date: Tue, 2 May 2017 09:03:07 -0400 Subject: RFR 8178604: JVM does not allow defining boot loader modules in exploded build after module system initialization In-Reply-To: References: <85e36988-a145-6805-8659-56c738013f58@oracle.com> Message-ID: Hi Lois, Thanks for your comments. Please see my in-line responses. Harold On 5/1/2017 4:14 PM, Lois Foltan wrote: > Hi Harold, > > Looks good. A couple of comments: > > src/share/vm/classfile/classLoader.cpp > line #828 - please take out the Module_lock when the > _exploded_entries is created This fix does not change the possible need for synchronization when _exploded_entries gets created during module system initialization. Are you saying that the existing code is wrong and should have taken out the Module_lock before creating _exploding_entries? > line #1402 - I like the new factored out method > find_first_module_cpe(), however, instead of adding a new "need_lock" > parameter, I would rather see code > in find_first_module_cpe() that takes the > Module_lock if the module_list parameter equals > ClassLoader::_exploded_entries I think your suggestion makes the code less flexible and does not require potential future callers of search_module_entries() to think about synchronization. So, I'd prefer to leave that change as is. > > Thanks, > Lois > > On 5/1/2017 3:36 PM, harold seigel wrote: >> Hi, >> >> Please review this JDK-10 bug fix to allow defining of modules to the >> boot loader in exploded builds after module system initialization. >> The fix uses the module lock to synchronize access to the >> _exploded_entries data structure (which is only used by exploded >> builds). >> >> Note that the above capability already exists for regular builds. >> >> Open Webrev: >> http://cr.openjdk.java.net/~hseigel/bug_8178604/webrev/index.html >> >> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8178604 >> >> The fix was tested with JCK tests, the JTreg hotspot, java/io, >> java/lang, java/util and other tests, the RBT tier2 -tier5 tests, the >> co-located NSK tests, and with JPRT. The JTReg and JCK tests were >> run with an exploded build. Also, the example program in the JBS bug >> was run by hand to verify the fix. >> >> Thanks, Harold >> > From mandy.chung at oracle.com Tue May 2 15:37:53 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 2 May 2017 08:37:53 -0700 Subject: 8178380: Module system implementation refresh (5/2017 update) In-Reply-To: <49754fcf-0016-424e-e5d7-5ae59ab3b7a8@oracle.com> References: <072A3264-1459-4E76-B7B7-11FD4C838602@oracle.com> <49754fcf-0016-424e-e5d7-5ae59ab3b7a8@oracle.com> Message-ID: > On May 2, 2017, at 4:28 AM, Alan Bateman wrote: > > I think this would clutter the class description and it might be better if we added a note to the getPackages method. > That?s an alternative. >> src/java.base/share/classes/jdk/internal/module/ModuleBootstrap.java >> >> 147 // special mode to boot with only java.base, ignores other options >> 148 if (System.getProperty("jdk.module.minimumBoot") != null) { >> 149 return createMinimalBootLayer(); >> 150 } > Is there an issue here? > Not an issue. My mistake forgetting to go back to take this out after went through the webrevs. > >> >> src/java.base/share/classes/jdk/internal/module/ModulePatcher.java >> 123 try (JarFile jf = new JarFile(file.toString())) { >> - what is the bug being fixed by this line? > This is part of the issue that Remi brought up last week related to overriding the default file system provider. I've replaced test/java/nio/file/spi/SetDefaultProvider.java with a more comprehensive test to test overriding with exploded and modular JAR that are patched / not-patched. > Thanks. I now remember it. Mandy From mandy.chung at oracle.com Tue May 2 15:40:09 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 2 May 2017 08:40:09 -0700 Subject: 8178380: Module system implementation refresh (5/2017 update) In-Reply-To: <44af0927-5fa5-7106-1a35-17a93c80e91d@oracle.com> References: <072A3264-1459-4E76-B7B7-11FD4C838602@oracle.com> <44af0927-5fa5-7106-1a35-17a93c80e91d@oracle.com> Message-ID: <42973DF2-E3CB-486C-AC15-F16B43F7C594@oracle.com> > On May 2, 2017, at 3:11 AM, Chris Hegarty wrote: > > Mandy, > > On 02/05/17 03:47, Mandy Chung wrote: >> >>> ... >> src/jdk.jartool/share/classes/sun/tools/jar/Main.java >> 619 if (dflag) { >> 620 // "--describe-module/-d" does not require file argument(s), >> 621 // but does accept --release >> 622 usageError(getMsg("error.bad.dflag")); >> 623 return false; >> 624 } > > Sorry, I'm not sure of the issue here. I added this check when > updating the describe option to accept an optional `--release` > ( the effective runtime version ) when describing a multi-release > modular JAR. This check is required as passing `-C`, in this > context, is an error. There are exiting tests that verify this. Not an issue. I was a little confused of this change at the beginning mostly due to the jar tool implementation. This is fine. Mandy From rachel.protacio at oracle.com Tue May 2 17:32:23 2017 From: rachel.protacio at oracle.com (Rachel Protacio) Date: Tue, 2 May 2017 13:32:23 -0400 Subject: RFR: 8154791: Xlog classload too redundant msgs info/debug Message-ID: <6ff9f069-2da7-b725-d29f-e3d850650aca@oracle.com> Hello! Please review this enhancement correcting redundancies and neatening up the -Xlog:class+load code. The redundancy in the code before I believe stemmed from a misunderstanding about logging levels (the comment saying they are not mutually exclusive was misleading). Tested with JPRT and RBT. Bug: https://bugs.openjdk.java.net/browse/JDK-8154791 Webrev: http://cr.openjdk.java.net/~rprotacio/8154791.00 Thanks, Rachel From rachel.protacio at oracle.com Tue May 2 18:15:01 2017 From: rachel.protacio at oracle.com (Rachel Protacio) Date: Tue, 2 May 2017 14:15:01 -0400 Subject: RFR (XS): 8067728: Flag::unlock_diagnostic() should be called Flag::clear_diagnostic() Message-ID: Hi, Please review this tiny change, renaming Flag::unlock_diagnostic() as Flag::clear_diagnostic. Bug: https://bugs.openjdk.java.net/browse/JDK-8067728 Open webrev: http://cr.openjdk.java.net/~rprotacio/8067728.00/ Thanks! Rachel From ioi.lam at oracle.com Tue May 2 18:19:13 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 02 May 2017 11:19:13 -0700 Subject: RFR (XS): 8067728: Flag::unlock_diagnostic() should be called Flag::clear_diagnostic() In-Reply-To: References: Message-ID: <5908CDA1.3000804@oracle.com> Hi Rachel, Looks good, but while you're doing this, maybe add: 417 void Flag::clear_diagnostic() { 418 assert(is_diagnostic(), "sanity"); 419 _flags = Flags(_flags & ~KIND_DIAGNOSTIC); assert(!is_diagnostic(), "sanity"); 420 } Thanks - Ioi On 5/2/17 11:15 AM, Rachel Protacio wrote: > Hi, > > Please review this tiny change, renaming Flag::unlock_diagnostic() as > Flag::clear_diagnostic. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8067728 > Open webrev: http://cr.openjdk.java.net/~rprotacio/8067728.00/ > > Thanks! > Rachel From harold.seigel at oracle.com Tue May 2 18:29:20 2017 From: harold.seigel at oracle.com (harold seigel) Date: Tue, 2 May 2017 14:29:20 -0400 Subject: RFR 8056900: Enhance NoClassDefFound exception messaging Message-ID: <9329c480-6e86-d8f6-57f2-23949ee21fcb@oracle.com> Hi, Please review this small JDK-10 change to enhance NoClassDefFoundError exception messages. Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8056900/webrev/index.html JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8056900 The change was tested with JCK tests, the JTreg hotspot, java/io, java/lang, java/util and other tests, the RBT tier2 -tier5 tests, the co-located NSK tests, JPRT, and with the new test provided with the change. Thanks, Harold From ioi.lam at oracle.com Tue May 2 18:48:33 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 02 May 2017 11:48:33 -0700 Subject: RFR: 8154791: Xlog classload too redundant msgs info/debug In-Reply-To: <6ff9f069-2da7-b725-d29f-e3d850650aca@oracle.com> References: <6ff9f069-2da7-b725-d29f-e3d850650aca@oracle.com> Message-ID: <5908D481.2070104@oracle.com> Hi Rachel, There are a few reasons why the current output has duplications. The "logging levels are not mutually exclusive" comment means this: java -Xlog:class+load=debug:file=debug.log -Xlog:class+load=info:file=info.log So info.log contains only the info level logs, but debug.log contains both info and debug level logs. Ideally I want info.log to contain this: [0.162s][info ][class,load] java.lang.Comparable source: jrt:/java.base .. and debug.log to contain this: [0.162s][debug ][class,load] java.lang.Comparable source: jrt:/java.base [NULL class_loader] 0x00000007c00013a8 super: 0x00000007c0000fb0 bytes: 235 checksum: a75dadb6 Your current patch splits out the debug log into several lines. This makes it difficult to analyze the log when classes are loaded concurrently on different threads: [0.162s][info ][class,load] java.lang.Comparable source: jrt:/java.base [0.162s][info ][class,load] java.lang.Comparable loader: [NULL class_loader] [0.162s][debug][class,load] java.lang.Comparable klass: 0x00000007c00013a8 [0.163s][info ][class,load] java.lang.String source: jrt:/java.base <<< oops! from a different thread [0.164s][debug][class,load] java.lang.Comparable super: 0x00000007c0000fb0 [0.164s][info ][class,load] java.lang.Comparable bytes: 235 [0.164s][info ][class,load] java.lang.Comparable checksum: a75dadb6 Because the leveling in UL is by lines, in JDK-8079408, where this was first implemented, I couldn't find a way to print "java.lang.Comparable source: jrt:/java.base" in "info" mode, and print the rest in "debug" mode, while keeping the whole thing on the same line (in both the info.log and debug.log files). That's why we have the 2 lines with duplicated info. I think it's really important to keep everything on the same line (the output needs to be processed automatically by scripts). With the current UL restrictions, I think the only way to do it is make the "info" logging more verbose when "debug" level is selected. E.g., outputStream* info_log = Log(class, load)::info_stream(); info_log->print("%s", ext_name); info_log->print_cr(" source: jrt:/%s", module_name); .... if (log_is_enabled(Debug, class, load)) { // always print to info log, even if debug is selected info_log->print(" klass: " INTPTR_FORMAT, p2i(this)); .... } So now info.log and debug.log will have the same logs, but oh well ..... Thanks - Ioi On 5/2/17 10:32 AM, Rachel Protacio wrote: > Hello! > > Please review this enhancement correcting redundancies and neatening > up the -Xlog:class+load code. The redundancy in the code before I > believe stemmed from a misunderstanding about logging levels (the > comment saying they are not mutually exclusive was misleading). > > Tested with JPRT and RBT. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8154791 > Webrev: http://cr.openjdk.java.net/~rprotacio/8154791.00 > > Thanks, > Rachel From rachel.protacio at oracle.com Tue May 2 18:52:17 2017 From: rachel.protacio at oracle.com (Rachel Protacio) Date: Tue, 2 May 2017 14:52:17 -0400 Subject: RFR (XS): 8067728: Flag::unlock_diagnostic() should be called Flag::clear_diagnostic() In-Reply-To: <5908CDA1.3000804@oracle.com> References: <5908CDA1.3000804@oracle.com> Message-ID: <6ac527c2-c5cc-6fbe-4cbd-32606f9f0234@oracle.com> Sure thing. http://cr.openjdk.java.net/~rprotacio/8067728.01 Rachel On 5/2/2017 2:19 PM, Ioi Lam wrote: > Hi Rachel, > > Looks good, but while you're doing this, maybe add: > > 417 void Flag::clear_diagnostic() { > 418 assert(is_diagnostic(), "sanity"); > 419 _flags = Flags(_flags & ~KIND_DIAGNOSTIC); > assert(!is_diagnostic(), "sanity"); > 420 } > > Thanks > - Ioi > > On 5/2/17 11:15 AM, Rachel Protacio wrote: >> Hi, >> >> Please review this tiny change, renaming Flag::unlock_diagnostic() as >> Flag::clear_diagnostic. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8067728 >> Open webrev: http://cr.openjdk.java.net/~rprotacio/8067728.00/ >> >> Thanks! >> Rachel > From ioi.lam at oracle.com Tue May 2 18:54:03 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 2 May 2017 11:54:03 -0700 Subject: RFR (XS): 8067728: Flag::unlock_diagnostic() should be called Flag::clear_diagnostic() In-Reply-To: <6ac527c2-c5cc-6fbe-4cbd-32606f9f0234@oracle.com> References: <5908CDA1.3000804@oracle.com> <6ac527c2-c5cc-6fbe-4cbd-32606f9f0234@oracle.com> Message-ID: <83DA01C4-68A0-45C9-B7CA-B522ADBE579D@oracle.com> Looks good. Thanks Ioi > On May 2, 2017, at 11:52 AM, Rachel Protacio wrote: > > Sure thing. http://cr.openjdk.java.net/~rprotacio/8067728.01 > > Rachel > >> On 5/2/2017 2:19 PM, Ioi Lam wrote: >> Hi Rachel, >> >> Looks good, but while you're doing this, maybe add: >> >> 417 void Flag::clear_diagnostic() { >> 418 assert(is_diagnostic(), "sanity"); >> 419 _flags = Flags(_flags & ~KIND_DIAGNOSTIC); >> assert(!is_diagnostic(), "sanity"); >> 420 } >> >> Thanks >> - Ioi >> >>> On 5/2/17 11:15 AM, Rachel Protacio wrote: >>> Hi, >>> >>> Please review this tiny change, renaming Flag::unlock_diagnostic() as Flag::clear_diagnostic. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8067728 >>> Open webrev: http://cr.openjdk.java.net/~rprotacio/8067728.00/ >>> >>> Thanks! >>> Rachel >> > From harold.seigel at oracle.com Tue May 2 18:56:54 2017 From: harold.seigel at oracle.com (harold seigel) Date: Tue, 2 May 2017 14:56:54 -0400 Subject: RFR (XS): 8067728: Flag::unlock_diagnostic() should be called Flag::clear_diagnostic() In-Reply-To: <83DA01C4-68A0-45C9-B7CA-B522ADBE579D@oracle.com> References: <5908CDA1.3000804@oracle.com> <6ac527c2-c5cc-6fbe-4cbd-32606f9f0234@oracle.com> <83DA01C4-68A0-45C9-B7CA-B522ADBE579D@oracle.com> Message-ID: <58a0f679-8fb1-7d62-d769-87f33ddf59c9@oracle.com> +1 Harold On 5/2/2017 2:54 PM, Ioi Lam wrote: > Looks good. Thanks > Ioi > >> On May 2, 2017, at 11:52 AM, Rachel Protacio wrote: >> >> Sure thing. http://cr.openjdk.java.net/~rprotacio/8067728.01 >> >> Rachel >> >>> On 5/2/2017 2:19 PM, Ioi Lam wrote: >>> Hi Rachel, >>> >>> Looks good, but while you're doing this, maybe add: >>> >>> 417 void Flag::clear_diagnostic() { >>> 418 assert(is_diagnostic(), "sanity"); >>> 419 _flags = Flags(_flags & ~KIND_DIAGNOSTIC); >>> assert(!is_diagnostic(), "sanity"); >>> 420 } >>> >>> Thanks >>> - Ioi >>> >>>> On 5/2/17 11:15 AM, Rachel Protacio wrote: >>>> Hi, >>>> >>>> Please review this tiny change, renaming Flag::unlock_diagnostic() as Flag::clear_diagnostic. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8067728 >>>> Open webrev: http://cr.openjdk.java.net/~rprotacio/8067728.00/ >>>> >>>> Thanks! >>>> Rachel From stanislav.lukyanov at oracle.com Tue May 2 19:21:25 2017 From: stanislav.lukyanov at oracle.com (stanislav lukyanov) Date: Tue, 2 May 2017 22:21:25 +0300 Subject: 8178380: Module system implementation refresh (5/2017 update) In-Reply-To: References: Message-ID: Hi Alan, In the Launcher-Agent-Class, would it make sense to use 'premain' instead of 'agentmain' to align it with Premain-Class rather than with Agent-Class? (maybe even rename Launcher-Agent-Class to Launcher-Premain-Class?) AFAIU Launcher-Agent-Class is meant to replace a particular use case of self-attach in applications - hence the 'agentmain'. But on the other hand, it is essentially a "shortcut" for '-jar app.jar -javaagent:app.jar', so 'premain' might as well be expected to be used (especially by a new user). Also Launcher-Agent-Class is invoked before the application, like Premain-Class and unlike Agent-Class, and some agents out there might use the entry method (premain or agentmain declared by the same class) to determine how they were invoked. It will not be possible if 'agentmain' can be invoked both before and after 'main' (using different classes for Premain-Class, Agent-Class and Launcher-Agent-Class will still work though). Thanks, Stas On 01.05.2017 23:28, Alan Bateman wrote: > As I mentioned in another thread, we need to get the changes > accumulated in jake to jdk9/dev. > > JDK-8178380 [1] has the summary of the changes that have accumulated > since the last refresh. > > One note on the hotspot repo is that the changes to dynamically > augment the platform modules run into JDK-8178604 when testing with an > exploded build. So there is a partial fix for this in jake. Harold has > the more complete fix in review on hotspot-runtime-dev for JDK 10. > > The webrevs with the changes is here: > http://cr.openjdk.java.net/~alanb/8178380/1/ > > -Alan > > [1] https://bugs.openjdk.java.net/browse/JDK-8178380 > From rachel.protacio at oracle.com Tue May 2 20:31:28 2017 From: rachel.protacio at oracle.com (Rachel Protacio) Date: Tue, 2 May 2017 16:31:28 -0400 Subject: RFR: 8154791: Xlog classload too redundant msgs info/debug In-Reply-To: <5908D481.2070104@oracle.com> References: <6ff9f069-2da7-b725-d29f-e3d850650aca@oracle.com> <5908D481.2070104@oracle.com> Message-ID: <6a8ffd20-ce58-150b-a94d-c51c5956d191@oracle.com> Hi Ioi, Thanks for your reply, comments inline. On 5/2/2017 2:48 PM, Ioi Lam wrote: > Hi Rachel, > > There are a few reasons why the current output has duplications. The > "logging levels are not mutually exclusive" comment means this: > > java -Xlog:class+load=debug:file=debug.log > -Xlog:class+load=info:file=info.log > > So info.log contains only the info level logs, but debug.log contains > both info and debug level logs. > > Ideally I want info.log to contain this: > > [0.162s][info ][class,load] java.lang.Comparable source: jrt:/java.base > > .. and debug.log to contain this: > > [0.162s][debug ][class,load] java.lang.Comparable source: > jrt:/java.base [NULL class_loader] 0x00000007c00013a8 super: > 0x00000007c0000fb0 bytes: 235 > checksum: a75dadb6 I see what you want, but I think the problem is that logging doesn't work that way. If debug is specified, both debug and info levels are printed. So the command java -Xlog:class+load=debug:file=debug.log will print both info level logging and debug level logging to this "debug.log" file. And then tacking on -Xlog:class+load=info:file=info.log in the command would print just the info level logging to the "info.log" file. But since it's all the same java command, it will always know that both debug and info levels are specified. I believe what you want me to do is say "if debug is specified, print everything to debug only" but then the info.log would be empty. You would have to run a separate java command with just the info level specified to get the smaller amount of information. But I think that would violate user expectations - if they get certain information with info logging, adding the extra level of debug logging should not change what they see from the info logging. > > Your current patch splits out the debug log into several lines. This > makes it difficult to analyze the log when classes are loaded > concurrently on different threads: > > [0.162s][info ][class,load] java.lang.Comparable source: jrt:/java.base > [0.162s][info ][class,load] java.lang.Comparable loader: [NULL > class_loader] > [0.162s][debug][class,load] java.lang.Comparable klass: > 0x00000007c00013a8 > [0.163s][info ][class,load] java.lang.String source: jrt:/java.base > <<< oops! from a different thread > [0.164s][debug][class,load] java.lang.Comparable super: > 0x00000007c0000fb0 > [0.164s][info ][class,load] java.lang.Comparable bytes: 235 > [0.164s][info ][class,load] java.lang.Comparable checksum: a75dadb6 That's fair. I was trying to make it more readable since it's hard to visually parse one giant line. Maybe I could put some delimiter in between each piece of info? But if no one else cares, I can just put it back as one line, no delimiters. > > Because the leveling in UL is by lines, in JDK-8079408, where this was > first implemented, I couldn't find a way to print > "java.lang.Comparable source: jrt:/java.base" in "info" mode, and > print the rest in "debug" mode, while keeping the whole thing on the > same line (in both the info.log and debug.log files). That's why we > have the 2 lines with duplicated info. > > I think it's really important to keep everything on the same line (the > output needs to be processed automatically by scripts). With the > current UL restrictions, I think the only way to do it is make the > "info" logging more verbose when "debug" level is selected. E.g., > > outputStream* info_log = Log(class, load)::info_stream(); > info_log->print("%s", ext_name); > info_log->print_cr(" source: jrt:/%s", module_name); .... > > if (log_is_enabled(Debug, class, load)) { // always print to info > log, even if debug is selected > info_log->print(" klass: " INTPTR_FORMAT, p2i(this)); > .... > } > > So now info.log and debug.log will have the same logs, but oh well ..... This sounds a little different. It sounds like When only info is specified, info log will contain: external name, source, loader, bytes, checksum When debug is specified, info log will contain the same as above, plus: klass, super, interfaces When debug is specified, debug log will be an exact duplicate of info log so it can print to an external file Did I understand that right? Please correct me. If this is what you meant, I would again argue that this violates the expectation that "info is info is info" regardless of what else is specified (not a written rule, but how I think it should work). And it is redundant, which is the original issue in the name of this RFE. I think the best bet would be Info log: external name, source, loader, bytes, checksum Debug log: external name, klass, super, interfaces Or we get rid of the level distinction completely and put all the information together in one line so it never gets broken up. Probably under debug. Let me know what you think. Thanks, Rachel > > Thanks > - Ioi > > On 5/2/17 10:32 AM, Rachel Protacio wrote: >> Hello! >> >> Please review this enhancement correcting redundancies and neatening >> up the -Xlog:class+load code. The redundancy in the code before I >> believe stemmed from a misunderstanding about logging levels (the >> comment saying they are not mutually exclusive was misleading). >> >> Tested with JPRT and RBT. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8154791 >> Webrev: http://cr.openjdk.java.net/~rprotacio/8154791.00 >> >> Thanks, >> Rachel > From Alan.Bateman at oracle.com Tue May 2 20:45:14 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 2 May 2017 21:45:14 +0100 Subject: 8178380: Module system implementation refresh (5/2017 update) In-Reply-To: References: Message-ID: <373dbc2e-5993-e2a2-f3ac-fa96b195d53c@oracle.com> On 02/05/2017 20:21, stanislav lukyanov wrote: > Hi Alan, > > In the Launcher-Agent-Class, would it make sense to use 'premain' > instead of 'agentmain' > to align it with Premain-Class rather than with Agent-Class? > (maybe even rename Launcher-Agent-Class to Launcher-Premain-Class?) > > AFAIU Launcher-Agent-Class is meant to replace a particular use case > of self-attach in applications - hence the 'agentmain'. > But on the other hand, it is essentially a "shortcut" for '-jar > app.jar -javaagent:app.jar', so 'premain' might as well be expected to > be used (especially by a new user). > > Also Launcher-Agent-Class is invoked before the application, like > Premain-Class and unlike Agent-Class, > and some agents out there might use the entry method (premain or > agentmain declared by the same class) to determine how they were invoked. > It will not be possible if 'agentmain' can be invoked both before and > after 'main' > (using different classes for Premain-Class, Agent-Class and > Launcher-Agent-Class will still work though). Executable JARs is a launcher concept, specified in the JAR file spec, and not something that the VM knows anything about. You won't see anything about executable JARs in the JNI invocation interface for example. This means that the agent declared via Launcher-Agent-Class is started in a running VM and is more like Agent-Class/agentmain than Premain-Class/premain. In particular, it may not be possible to get all the capabilities that an agent specified via `-javaagent` can get. So I think the proposed names are okay. As regards using an executable JAR with `-javaagent`, or a tool loading the executable JAR as an agent library into a running VM, then I think there is enough flexibility with using different class names. -Alan. From ioi.lam at oracle.com Tue May 2 20:56:40 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 02 May 2017 13:56:40 -0700 Subject: RFR: 8154791: Xlog classload too redundant msgs info/debug In-Reply-To: <6a8ffd20-ce58-150b-a94d-c51c5956d191@oracle.com> References: <6ff9f069-2da7-b725-d29f-e3d850650aca@oracle.com> <5908D481.2070104@oracle.com> <6a8ffd20-ce58-150b-a94d-c51c5956d191@oracle.com> Message-ID: <5908F288.9020205@oracle.com> On 5/2/17 1:31 PM, Rachel Protacio wrote: > Hi Ioi, > > Thanks for your reply, comments inline. > > > On 5/2/2017 2:48 PM, Ioi Lam wrote: >> Hi Rachel, >> >> There are a few reasons why the current output has duplications. The >> "logging levels are not mutually exclusive" comment means this: >> >> java -Xlog:class+load=debug:file=debug.log >> -Xlog:class+load=info:file=info.log >> >> So info.log contains only the info level logs, but debug.log contains >> both info and debug level logs. >> >> Ideally I want info.log to contain this: >> >> [0.162s][info ][class,load] java.lang.Comparable source: jrt:/java.base >> >> .. and debug.log to contain this: >> >> [0.162s][debug ][class,load] java.lang.Comparable source: >> jrt:/java.base [NULL class_loader] 0x00000007c00013a8 super: >> 0x00000007c0000fb0 bytes: 235 >> checksum: a75dadb6 > I see what you want, but I think the problem is that logging doesn't > work that way. If debug is specified, both debug and info levels are > printed. So the command > > java -Xlog:class+load=debug:file=debug.log > > will print both info level logging and debug level logging to this > "debug.log" file. And then tacking on > > -Xlog:class+load=info:file=info.log > > in the command would print just the info level logging to the > "info.log" file. But since it's all the same java command, it will > always know that both debug and info levels are specified. I believe > what you want me to do is say "if debug is specified, print everything > to debug only" but then the info.log would be empty. You would have to > run a separate java command with just the info level specified to get > the smaller amount of information. No, that's not what I want. I am perfectly fine with how UL works today: info.log contains only the info level, and debug.info contains both debug and info level. All I want is everything related to the same class to be printed on the same line. For example, I don't want debug.log to contain this: [0.162s][info ][class,load] java.lang.Comparable source: jrt:/java.base [0.162s][debug ][class,load] [NULL class_loader] 0x00000007c00013a8 super: 0x00000007c0000fb0 bytes: 235 checksum: a75dadb6 That's because it's impossible to associated the second with the first line when you have parallel threads loading classes concurrently. > But I think that would violate user expectations - if they get certain > information with info logging, adding the extra level of debug logging > should not change what they see from the info logging. >> >> Your current patch splits out the debug log into several lines. This >> makes it difficult to analyze the log when classes are loaded >> concurrently on different threads: >> >> [0.162s][info ][class,load] java.lang.Comparable source: jrt:/java.base >> [0.162s][info ][class,load] java.lang.Comparable loader: [NULL >> class_loader] >> [0.162s][debug][class,load] java.lang.Comparable klass: >> 0x00000007c00013a8 >> [0.163s][info ][class,load] java.lang.String source: jrt:/java.base >> <<< oops! from a different thread >> [0.164s][debug][class,load] java.lang.Comparable super: >> 0x00000007c0000fb0 >> [0.164s][info ][class,load] java.lang.Comparable bytes: 235 >> [0.164s][info ][class,load] java.lang.Comparable checksum: a75dadb6 > That's fair. I was trying to make it more readable since it's hard to > visually parse one giant line. Maybe I could put some delimiter in > between each piece of info? But if no one else cares, I can just put > it back as one line, no delimiters. >> >> Because the leveling in UL is by lines, in JDK-8079408, where this >> was first implemented, I couldn't find a way to print >> "java.lang.Comparable source: jrt:/java.base" in "info" mode, and >> print the rest in "debug" mode, while keeping the whole thing on the >> same line (in both the info.log and debug.log files). That's why we >> have the 2 lines with duplicated info. >> >> I think it's really important to keep everything on the same line >> (the output needs to be processed automatically by scripts). With the >> current UL restrictions, I think the only way to do it is make the >> "info" logging more verbose when "debug" level is selected. E.g., >> >> outputStream* info_log = Log(class, load)::info_stream(); >> info_log->print("%s", ext_name); >> info_log->print_cr(" source: jrt:/%s", module_name); .... >> >> if (log_is_enabled(Debug, class, load)) { // always print to info >> log, even if debug is selected >> info_log->print(" klass: " INTPTR_FORMAT, p2i(this)); >> .... >> } >> >> So now info.log and debug.log will have the same logs, but oh well ..... > This sounds a little different. It sounds like > > When only info is specified, info log will contain: external name, > source, loader, bytes, checksum > > When debug is specified, info log will contain the same as above, > plus: klass, super, interfaces > When debug is specified, debug log will be an exact duplicate of > info log so it can print to an external file > > Did I understand that right? Please correct me. If this is what you > meant, I would again argue that this violates the expectation that > "info is info is info" regardless of what else is specified (not a > written rule, but how I think it should work). And it is redundant, > which is the original issue in the name of this RFE. > > I think the best bet would be > > Info log: external name, source, loader, bytes, checksum > Debug log: external name, klass, super, interfaces > > Or we get rid of the level distinction completely and put all the > information together in one line so it never gets broken up. Probably > under debug. > We need to keep the current debug output intact, on a single line, so that the information can be processed by scripts. So, it seems like our 3 choices are: (1) make the info output the same as debug, if -Xlog:class+load=debug is specified (2) remove the info output altogether (3) do nothing You don't like (1). I don't think (2) achieves what this RFE wants -- namely, make the output less redundant, because you will see all the details all the time, and you don't even get a choice for more terse output. It also makes the output of -verbose different (and much less useful) than in JDK 9. So I would vote for (3), keep the output as is. Yeah, with debug level you have a lot of output, but that's pretty easy to filter. I think we had a discussion about this when 8079408 was implemented, and that was the conclusion we had. Thanks - Ioi > Let me know what you think. > > Thanks, > Rachel > >> >> Thanks >> - Ioi >> >> On 5/2/17 10:32 AM, Rachel Protacio wrote: >>> Hello! >>> >>> Please review this enhancement correcting redundancies and neatening >>> up the -Xlog:class+load code. The redundancy in the code before I >>> believe stemmed from a misunderstanding about logging levels (the >>> comment saying they are not mutually exclusive was misleading). >>> >>> Tested with JPRT and RBT. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8154791 >>> Webrev: http://cr.openjdk.java.net/~rprotacio/8154791.00 >>> >>> Thanks, >>> Rachel >> > From ioi.lam at oracle.com Tue May 2 21:10:44 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 02 May 2017 14:10:44 -0700 Subject: RFR: 8154791: Xlog classload too redundant msgs info/debug In-Reply-To: <5908F288.9020205@oracle.com> References: <6ff9f069-2da7-b725-d29f-e3d850650aca@oracle.com> <5908D481.2070104@oracle.com> <6a8ffd20-ce58-150b-a94d-c51c5956d191@oracle.com> <5908F288.9020205@oracle.com> Message-ID: <5908F5D4.6040505@oracle.com> By the way, I think we should really fix this in the UL level -- have multi-line buffering, so you can prevent other threads from interleaving your input. Make it something like: Log(class, load).start_buffering(); log_info(class, load)(class name and source); log_debug(class, load)(other details); Log(class, load).flush(); That way, the output will look like this: [0.162s][debug ][class,load] java.lang.Comparable source: jrt:/java.base [0.162s][debug ][class,load] [NULL class_loader] 0x00000007c00013a8 super: 0x00000007c0000fb0 bytes: 235 checksum: a75dadb6 So the output will be terse, and still parsable. I think this will be useful for other types of logs as well, not just the class loading logs. Thanks - Ioi On 5/2/17 1:56 PM, Ioi Lam wrote: > > > On 5/2/17 1:31 PM, Rachel Protacio wrote: >> Hi Ioi, >> >> Thanks for your reply, comments inline. >> >> >> On 5/2/2017 2:48 PM, Ioi Lam wrote: >>> Hi Rachel, >>> >>> There are a few reasons why the current output has duplications. The >>> "logging levels are not mutually exclusive" comment means this: >>> >>> java -Xlog:class+load=debug:file=debug.log >>> -Xlog:class+load=info:file=info.log >>> >>> So info.log contains only the info level logs, but debug.log >>> contains both info and debug level logs. >>> >>> Ideally I want info.log to contain this: >>> >>> [0.162s][info ][class,load] java.lang.Comparable source: jrt:/java.base >>> >>> .. and debug.log to contain this: >>> >>> [0.162s][debug ][class,load] java.lang.Comparable source: >>> jrt:/java.base [NULL class_loader] 0x00000007c00013a8 super: >>> 0x00000007c0000fb0 bytes: 235 >>> checksum: a75dadb6 >> I see what you want, but I think the problem is that logging doesn't >> work that way. If debug is specified, both debug and info levels are >> printed. So the command >> >> java -Xlog:class+load=debug:file=debug.log >> >> will print both info level logging and debug level logging to this >> "debug.log" file. And then tacking on >> >> -Xlog:class+load=info:file=info.log >> >> in the command would print just the info level logging to the >> "info.log" file. But since it's all the same java command, it will >> always know that both debug and info levels are specified. I believe >> what you want me to do is say "if debug is specified, print >> everything to debug only" but then the info.log would be empty. You >> would have to run a separate java command with just the info level >> specified to get the smaller amount of information. > > No, that's not what I want. I am perfectly fine with how UL works > today: info.log contains only the info level, and debug.info contains > both debug and info level. > > All I want is everything related to the same class to be printed on > the same line. For example, I don't want debug.log to contain this: > > [0.162s][info ][class,load] java.lang.Comparable source: jrt:/java.base > [0.162s][debug ][class,load] [NULL class_loader] 0x00000007c00013a8 > super: 0x00000007c0000fb0 bytes: 235 checksum: a75dadb6 > > That's because it's impossible to associated the second with the first > line when you have parallel threads loading classes concurrently. > >> But I think that would violate user expectations - if they get >> certain information with info logging, adding the extra level of >> debug logging should not change what they see from the info logging. >>> >>> Your current patch splits out the debug log into several lines. This >>> makes it difficult to analyze the log when classes are loaded >>> concurrently on different threads: >>> >>> [0.162s][info ][class,load] java.lang.Comparable source: jrt:/java.base >>> [0.162s][info ][class,load] java.lang.Comparable loader: [NULL >>> class_loader] >>> [0.162s][debug][class,load] java.lang.Comparable klass: >>> 0x00000007c00013a8 >>> [0.163s][info ][class,load] java.lang.String source: jrt:/java.base >>> <<< oops! from a different thread >>> [0.164s][debug][class,load] java.lang.Comparable super: >>> 0x00000007c0000fb0 >>> [0.164s][info ][class,load] java.lang.Comparable bytes: 235 >>> [0.164s][info ][class,load] java.lang.Comparable checksum: a75dadb6 >> That's fair. I was trying to make it more readable since it's hard to >> visually parse one giant line. Maybe I could put some delimiter in >> between each piece of info? But if no one else cares, I can just put >> it back as one line, no delimiters. >>> >>> Because the leveling in UL is by lines, in JDK-8079408, where this >>> was first implemented, I couldn't find a way to print >>> "java.lang.Comparable source: jrt:/java.base" in "info" mode, and >>> print the rest in "debug" mode, while keeping the whole thing on the >>> same line (in both the info.log and debug.log files). That's why we >>> have the 2 lines with duplicated info. >>> >>> I think it's really important to keep everything on the same line >>> (the output needs to be processed automatically by scripts). With >>> the current UL restrictions, I think the only way to do it is make >>> the "info" logging more verbose when "debug" level is selected. E.g., >>> >>> outputStream* info_log = Log(class, load)::info_stream(); >>> info_log->print("%s", ext_name); >>> info_log->print_cr(" source: jrt:/%s", module_name); .... >>> >>> if (log_is_enabled(Debug, class, load)) { // always print to info >>> log, even if debug is selected >>> info_log->print(" klass: " INTPTR_FORMAT, p2i(this)); >>> .... >>> } >>> >>> So now info.log and debug.log will have the same logs, but oh well >>> ..... >> This sounds a little different. It sounds like >> >> When only info is specified, info log will contain: external name, >> source, loader, bytes, checksum >> >> When debug is specified, info log will contain the same as above, >> plus: klass, super, interfaces >> When debug is specified, debug log will be an exact duplicate of >> info log so it can print to an external file >> >> Did I understand that right? Please correct me. If this is what you >> meant, I would again argue that this violates the expectation that >> "info is info is info" regardless of what else is specified (not a >> written rule, but how I think it should work). And it is redundant, >> which is the original issue in the name of this RFE. >> >> I think the best bet would be >> >> Info log: external name, source, loader, bytes, checksum >> Debug log: external name, klass, super, interfaces >> >> Or we get rid of the level distinction completely and put all the >> information together in one line so it never gets broken up. Probably >> under debug. >> > > We need to keep the current debug output intact, on a single line, so > that the information can be processed by scripts. > > So, it seems like our 3 choices are: > > (1) make the info output the same as debug, if -Xlog:class+load=debug > is specified > (2) remove the info output altogether > (3) do nothing > > You don't like (1). > > I don't think (2) achieves what this RFE wants -- namely, make the > output less redundant, because you will see all the details all the > time, and you don't even get a choice for more terse output. It also > makes the output of -verbose different (and much less useful) than in > JDK 9. > > So I would vote for (3), keep the output as is. Yeah, with debug level > you have a lot of output, but that's pretty easy to filter. I think we > had a discussion about this when 8079408 was implemented, and that was > the conclusion we had. > > Thanks > - Ioi > >> Let me know what you think. >> >> Thanks, >> Rachel >> >>> >>> Thanks >>> - Ioi >>> >>> On 5/2/17 10:32 AM, Rachel Protacio wrote: >>>> Hello! >>>> >>>> Please review this enhancement correcting redundancies and >>>> neatening up the -Xlog:class+load code. The redundancy in the code >>>> before I believe stemmed from a misunderstanding about logging >>>> levels (the comment saying they are not mutually exclusive was >>>> misleading). >>>> >>>> Tested with JPRT and RBT. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8154791 >>>> Webrev: http://cr.openjdk.java.net/~rprotacio/8154791.00 >>>> >>>> Thanks, >>>> Rachel >>> >> > From lois.foltan at oracle.com Tue May 2 21:11:10 2017 From: lois.foltan at oracle.com (Lois Foltan) Date: Tue, 2 May 2017 17:11:10 -0400 Subject: RFR 8178604: JVM does not allow defining boot loader modules in exploded build after module system initialization In-Reply-To: References: <85e36988-a145-6805-8659-56c738013f58@oracle.com> Message-ID: <46a812ca-228d-01e7-e889-fd22f2af0537@oracle.com> On 5/2/2017 9:03 AM, harold seigel wrote: > Hi Lois, > > Thanks for your comments. Please see my in-line responses. > > Harold > > > On 5/1/2017 4:14 PM, Lois Foltan wrote: >> Hi Harold, >> >> Looks good. A couple of comments: >> >> src/share/vm/classfile/classLoader.cpp >> line #828 - please take out the Module_lock when the >> _exploded_entries is created > This fix does not change the possible need for synchronization when > _exploded_entries gets created during module system initialization. > Are you saying that the existing code is wrong and should have taken > out the Module_lock before creating _exploding_entries? The method ClassLoader::add_to_exploded_build_list can be called at anytime (during module system initialization and post module system initialization). Thus it seems prudent that no matter what the current JVM phase, that any access to ClassLoader::_exploding_entries be protected via a lock. That includes creation as well as insertion into the list. > >> line #1402 - I like the new factored out method >> find_first_module_cpe(), however, instead of adding a new "need_lock" >> parameter, I would rather see code >> in find_first_module_cpe() that takes the >> Module_lock if the module_list parameter equals >> ClassLoader::_exploded_entries > I think your suggestion makes the code less flexible and does not > require potential future callers of search_module_entries() to think > about synchronization. So, I'd prefer to leave that change as is. I see your point. My point was based on reduced code readability, the further away from the decision to establish the lock, the less apparent it is within the new method ClassLoader::find_first_module_cpe() as to what situations warranted the lock in the first place. Thanks, Lois >> >> Thanks, >> Lois >> >> On 5/1/2017 3:36 PM, harold seigel wrote: >>> Hi, >>> >>> Please review this JDK-10 bug fix to allow defining of modules to >>> the boot loader in exploded builds after module system >>> initialization. The fix uses the module lock to synchronize access >>> to the _exploded_entries data structure (which is only used by >>> exploded builds). >>> >>> Note that the above capability already exists for regular builds. >>> >>> Open Webrev: >>> http://cr.openjdk.java.net/~hseigel/bug_8178604/webrev/index.html >>> >>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8178604 >>> >>> The fix was tested with JCK tests, the JTreg hotspot, java/io, >>> java/lang, java/util and other tests, the RBT tier2 -tier5 tests, >>> the co-located NSK tests, and with JPRT. The JTReg and JCK tests >>> were run with an exploded build. Also, the example program in the >>> JBS bug was run by hand to verify the fix. >>> >>> Thanks, Harold >>> >> > From coleen.phillimore at oracle.com Tue May 2 23:35:27 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 2 May 2017 19:35:27 -0400 Subject: RFR[s] 8179305 - Avoid repeated calls to JavaThread::last_frame in InterpreterRuntime In-Reply-To: <5907E03F.3050603@oracle.com> References: <59032224.8020005@oracle.com> <139deaf0-ff94-ac4d-e5d4-5dfbe13202f9@oracle.com> <59079271.1070700@oracle.com> <0f308a93-f06a-003e-8ae4-eac402a78604@oracle.com> <5907A61A.6030404@oracle.com> <7756ca89-e9e9-e012-1043-f3d0f3f6037f@oracle.com> <5907CE45.3080003@oracle.com> <5907E03F.3050603@oracle.com> Message-ID: <6351a8fd-3a1d-7853-6c06-0c4fd584af22@oracle.com> This change looks really good! Coleen On 5/1/17 9:26 PM, Ioi Lam wrote: > OK, I'll shorten them to: > > oop*callee_receiver*(Symbol* signature) { > return _last_frame.interpreter_callee_receiver(signature); > } > BasicObjectLock**monitor_begin*() const { > return _last_frame.interpreter_frame_monitor_end(); > } > BasicObjectLock**monitor_end*() const { > return _last_frame.interpreter_frame_monitor_begin(); > } > BasicObjectLock**next_monitor*(BasicObjectLock* current) const { > return _last_frame.next_monitor_in_interpreter_frame(current); > } > > What do you think? > - Ioi > > On 5/1/17 6:18 PM, dean.long at oracle.com wrote: >> >> This looks fine, but why not shorten those names like we do for >> bcp(), bci(), etc? >> >> dl >> >> On 5/1/17 5:09 PM, Ioi Lam wrote: >>> Hi Dean, >>> >>> I've wrapped those 4 methods. I also added 'const' to the wrapper >>> methods where possible. Here's the delta from the last posted webrev: >>> >>> http://cr.openjdk.java.net/~iklam/jdk10/8179305-avoid-last-frame.v02.delta/ >>> >>> >>> Thanks >>> - Ioi >>> >>> On 5/1/17 3:58 PM, dean.long at oracle.com wrote: >>>> >>>> I was expecting you to wrap just the methods that >>>> InterpreterRuntime uses. >>>> >>>> dl >>>> >>>> >>>> On 5/1/17 2:18 PM, Ioi Lam wrote: >>>>> >>>>> >>>>> On 5/1/17 2:03 PM, dean.long at oracle.com wrote: >>>>>> >>>>>> I wasn't complaining about those, because you don't really need a >>>>>> LastFrameAccessor there. I was really talking about where you >>>>>> have a LastFrameAccessor but need to escape back to frame using >>>>>> get_frame() because you didn't add a new accessor method: >>>>>> >>>>>> >>>>>> 950 for( BasicObjectLock *kptr = >>>>>> last_frame.get_frame().interpreter_frame_monitor_end(); >>>>>> 951 kptr < >>>>>> last_frame.get_frame().interpreter_frame_monitor_begin(); >>>>>> 952 kptr = >>>>>> last_frame.get_frame().next_monitor_in_interpreter_frame(kptr) ) { >>>>>> >>>>>> versus something like >>>>>> last_frame.monitor_end()/monitor_begin()/next_monitor(). >>>>>> >>>>> I could add those accessor methods, but frame has lots of methods, >>>>> and I don't want to wrap each of them in case someone uses a new >>>>> function in the future. >>>>> >>>>> An alternative (which I really don't like) is to override the "->" >>>>> or "()" operators like the Handle class: >>>>> >>>>> last_frame->interpreter_frame_monitor_begin(); >>>>> or >>>>> last_frame().interpreter_frame_monitor_begin(); >>>>> >>>>> Thanks >>>>> - Ioi >>>>>> >>>>>> dl >>>>>> >>>>>> >>>>>> On 5/1/17 12:54 PM, Ioi Lam wrote: >>>>>>> Hi Dean, >>>>>>> >>>>>>> Thanks for the review. I left a few thread->last_frame() that >>>>>>> were inside ifdef ASSERT, but you're right, I should change >>>>>>> those as well to be consistent: >>>>>>> >>>>>>> diff -r dbee2fa1d3df >>>>>>> src/share/vm/interpreter/interpreterRuntime.cpp >>>>>>> --- a/src/share/vm/interpreter/interpreterRuntime.cpp Mon May 01 >>>>>>> 11:16:01 2017 -0700 >>>>>>> +++ b/src/share/vm/interpreter/interpreterRuntime.cpp Mon May 01 >>>>>>> 12:30:01 2017 -0700 >>>>>>> @@ -642,7 +642,7 @@ >>>>>>> IRT_ENTRY_NO_ASYNC(void, >>>>>>> InterpreterRuntime::monitorenter(JavaThread* thread, >>>>>>> BasicObjectLock* elem)) >>>>>>> #ifdef ASSERT >>>>>>> - thread->last_frame().interpreter_frame_verify_monitor(elem); >>>>>>> + >>>>>>> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >>>>>>> #endif >>>>>>> >>>>>>> @@ -659,7 +659,7 @@ >>>>>>> #ifdef ASSERT >>>>>>> - thread->last_frame().interpreter_frame_verify_monitor(elem); >>>>>>> + >>>>>>> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >>>>>>> #endif >>>>>>> IRT_END >>>>>>> >>>>>>> @@ -667,7 +667,7 @@ >>>>>>> IRT_ENTRY_NO_ASYNC(void, >>>>>>> InterpreterRuntime::monitorexit(JavaThread* thread, >>>>>>> BasicObjectLock* elem)) >>>>>>> #ifdef ASSERT >>>>>>> - thread->last_frame().interpreter_frame_verify_monitor(elem); >>>>>>> + >>>>>>> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >>>>>>> #endif >>>>>>> >>>>>>> @@ -680,7 +680,7 @@ >>>>>>> #ifdef ASSERT >>>>>>> - thread->last_frame().interpreter_frame_verify_monitor(elem); >>>>>>> + >>>>>>> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >>>>>>> #endif >>>>>>> IRT_END >>>>>>> >>>>>>> What do you think? >>>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 5/1/17 12:14 PM, dean.long at oracle.com wrote: >>>>>>>> Looks OK to me, but it seems arbitrary now which calls can use >>>>>>>> LastFrameAccessor methods and which need to go through the >>>>>>>> get_frame() back-door. >>>>>>>> >>>>>>>> dl >>>>>>>> >>>>>>>> >>>>>>>> On 4/28/17 4:06 AM, Ioi Lam wrote: >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8179305 >>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8179305-avoid-last-frame.v01/ >>>>>>>>> >>>>>>>>> >>>>>>>>> Summary: >>>>>>>>> >>>>>>>>> JavaThread::last_frame() is an expensive call. We use the >>>>>>>>> helper class LastFrameAccessor holds the value of last_frame: >>>>>>>>> >>>>>>>>> OLD: >>>>>>>>> methodHandle m (thread, method(thread));// calls >>>>>>>>> JavaThread::last_frame() internally >>>>>>>>> Bytecode_loadconstant ldc(m, bci(thread));// calls >>>>>>>>> JavaThread::last_frame() internally >>>>>>>>> >>>>>>>>> NEW: >>>>>>>>> LastFrameAccessor last_frame(thread); >>>>>>>>> methodHandle m (thread, last_frame.method()); >>>>>>>>> Bytecode_loadconstant ldc(m, last_frame.bci()); >>>>>>>>> >>>>>>>>> Testing: >>>>>>>>> >>>>>>>>> Preliminary benchmarking shows significant VM start-up >>>>>>>>> improvement: >>>>>>>>> >>>>>>>>> java -version: >>>>>>>>> no CDS = 80.35 ms -> 79.14ms (-1.5%) >>>>>>>>> w/ CDS = 53.89ms -> 52.62ms (-2.36%) >>>>>>>>> >>>>>>>>> clojure sample app: >>>>>>>>> no CDS = 1442.46ms -> 1436.11ms (-0.44%) >>>>>>>>> w/ CDS = 695.78ms -> 679.52ms (-2.34%) >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> - Ioi >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From ioi.lam at oracle.com Wed May 3 04:03:59 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 02 May 2017 21:03:59 -0700 Subject: RFR[s] 8179305 - Avoid repeated calls to JavaThread::last_frame in InterpreterRuntime In-Reply-To: <6351a8fd-3a1d-7853-6c06-0c4fd584af22@oracle.com> References: <59032224.8020005@oracle.com> <139deaf0-ff94-ac4d-e5d4-5dfbe13202f9@oracle.com> <59079271.1070700@oracle.com> <0f308a93-f06a-003e-8ae4-eac402a78604@oracle.com> <5907A61A.6030404@oracle.com> <7756ca89-e9e9-e012-1043-f3d0f3f6037f@oracle.com> <5907CE45.3080003@oracle.com> <5907E03F.3050603@oracle.com> <6351a8fd-3a1d-7853-6c06-0c4fd584af22@oracle.com> Message-ID: <590956AF.30805@oracle.com> Hi Coleen, Dean & David Thanks you so much for the review. I have pushed the changes. - Ioi On 5/2/17 4:35 PM, coleen.phillimore at oracle.com wrote: > > This change looks really good! > Coleen > > > On 5/1/17 9:26 PM, Ioi Lam wrote: >> OK, I'll shorten them to: >> >> oop*callee_receiver*(Symbol* signature) { >> return _last_frame.interpreter_callee_receiver(signature); >> } >> BasicObjectLock**monitor_begin*() const { >> return _last_frame.interpreter_frame_monitor_end(); >> } >> BasicObjectLock**monitor_end*() const { >> return _last_frame.interpreter_frame_monitor_begin(); >> } >> BasicObjectLock**next_monitor*(BasicObjectLock* current) const { >> return _last_frame.next_monitor_in_interpreter_frame(current); >> } >> >> What do you think? >> - Ioi >> >> On 5/1/17 6:18 PM, dean.long at oracle.com wrote: >>> >>> This looks fine, but why not shorten those names like we do for >>> bcp(), bci(), etc? >>> >>> dl >>> >>> On 5/1/17 5:09 PM, Ioi Lam wrote: >>>> Hi Dean, >>>> >>>> I've wrapped those 4 methods. I also added 'const' to the wrapper >>>> methods where possible. Here's the delta from the last posted webrev: >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk10/8179305-avoid-last-frame.v02.delta/ >>>> >>>> >>>> Thanks >>>> - Ioi >>>> >>>> On 5/1/17 3:58 PM, dean.long at oracle.com wrote: >>>>> >>>>> I was expecting you to wrap just the methods that >>>>> InterpreterRuntime uses. >>>>> >>>>> dl >>>>> >>>>> >>>>> On 5/1/17 2:18 PM, Ioi Lam wrote: >>>>>> >>>>>> >>>>>> On 5/1/17 2:03 PM, dean.long at oracle.com wrote: >>>>>>> >>>>>>> I wasn't complaining about those, because you don't really need >>>>>>> a LastFrameAccessor there. I was really talking about where you >>>>>>> have a LastFrameAccessor but need to escape back to frame using >>>>>>> get_frame() because you didn't add a new accessor method: >>>>>>> >>>>>>> >>>>>>> 950 for( BasicObjectLock *kptr = >>>>>>> last_frame.get_frame().interpreter_frame_monitor_end(); >>>>>>> 951 kptr < >>>>>>> last_frame.get_frame().interpreter_frame_monitor_begin(); >>>>>>> 952 kptr = >>>>>>> last_frame.get_frame().next_monitor_in_interpreter_frame(kptr) ) { >>>>>>> >>>>>>> versus something like >>>>>>> last_frame.monitor_end()/monitor_begin()/next_monitor(). >>>>>>> >>>>>> I could add those accessor methods, but frame has lots of >>>>>> methods, and I don't want to wrap each of them in case someone >>>>>> uses a new function in the future. >>>>>> >>>>>> An alternative (which I really don't like) is to override the >>>>>> "->" or "()" operators like the Handle class: >>>>>> >>>>>> last_frame->interpreter_frame_monitor_begin(); >>>>>> or >>>>>> last_frame().interpreter_frame_monitor_begin(); >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>>> >>>>>>> dl >>>>>>> >>>>>>> >>>>>>> On 5/1/17 12:54 PM, Ioi Lam wrote: >>>>>>>> Hi Dean, >>>>>>>> >>>>>>>> Thanks for the review. I left a few thread->last_frame() that >>>>>>>> were inside ifdef ASSERT, but you're right, I should change >>>>>>>> those as well to be consistent: >>>>>>>> >>>>>>>> diff -r dbee2fa1d3df >>>>>>>> src/share/vm/interpreter/interpreterRuntime.cpp >>>>>>>> --- a/src/share/vm/interpreter/interpreterRuntime.cpp Mon May >>>>>>>> 01 11:16:01 2017 -0700 >>>>>>>> +++ b/src/share/vm/interpreter/interpreterRuntime.cpp Mon May >>>>>>>> 01 12:30:01 2017 -0700 >>>>>>>> @@ -642,7 +642,7 @@ >>>>>>>> IRT_ENTRY_NO_ASYNC(void, >>>>>>>> InterpreterRuntime::monitorenter(JavaThread* thread, >>>>>>>> BasicObjectLock* elem)) >>>>>>>> #ifdef ASSERT >>>>>>>> - thread->last_frame().interpreter_frame_verify_monitor(elem); >>>>>>>> + >>>>>>>> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >>>>>>>> #endif >>>>>>>> >>>>>>>> @@ -659,7 +659,7 @@ >>>>>>>> #ifdef ASSERT >>>>>>>> - thread->last_frame().interpreter_frame_verify_monitor(elem); >>>>>>>> + >>>>>>>> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >>>>>>>> #endif >>>>>>>> IRT_END >>>>>>>> >>>>>>>> @@ -667,7 +667,7 @@ >>>>>>>> IRT_ENTRY_NO_ASYNC(void, >>>>>>>> InterpreterRuntime::monitorexit(JavaThread* thread, >>>>>>>> BasicObjectLock* elem)) >>>>>>>> #ifdef ASSERT >>>>>>>> - thread->last_frame().interpreter_frame_verify_monitor(elem); >>>>>>>> + >>>>>>>> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >>>>>>>> #endif >>>>>>>> >>>>>>>> @@ -680,7 +680,7 @@ >>>>>>>> #ifdef ASSERT >>>>>>>> - thread->last_frame().interpreter_frame_verify_monitor(elem); >>>>>>>> + >>>>>>>> LastFrameAccessor(thread).get_frame().interpreter_frame_verify_monitor(elem); >>>>>>>> #endif >>>>>>>> IRT_END >>>>>>>> >>>>>>>> What do you think? >>>>>>>> >>>>>>>> Thanks >>>>>>>> - Ioi >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 5/1/17 12:14 PM, dean.long at oracle.com wrote: >>>>>>>>> Looks OK to me, but it seems arbitrary now which calls can use >>>>>>>>> LastFrameAccessor methods and which need to go through the >>>>>>>>> get_frame() back-door. >>>>>>>>> >>>>>>>>> dl >>>>>>>>> >>>>>>>>> >>>>>>>>> On 4/28/17 4:06 AM, Ioi Lam wrote: >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8179305 >>>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8179305-avoid-last-frame.v01/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Summary: >>>>>>>>>> >>>>>>>>>> JavaThread::last_frame() is an expensive call. We use the >>>>>>>>>> helper class LastFrameAccessor holds the value of last_frame: >>>>>>>>>> >>>>>>>>>> OLD: >>>>>>>>>> methodHandle m (thread, method(thread));// calls >>>>>>>>>> JavaThread::last_frame() internally >>>>>>>>>> Bytecode_loadconstant ldc(m, bci(thread));// calls >>>>>>>>>> JavaThread::last_frame() internally >>>>>>>>>> >>>>>>>>>> NEW: >>>>>>>>>> LastFrameAccessor last_frame(thread); >>>>>>>>>> methodHandle m (thread, last_frame.method()); >>>>>>>>>> Bytecode_loadconstant ldc(m, last_frame.bci()); >>>>>>>>>> >>>>>>>>>> Testing: >>>>>>>>>> >>>>>>>>>> Preliminary benchmarking shows significant VM start-up >>>>>>>>>> improvement: >>>>>>>>>> >>>>>>>>>> java -version: >>>>>>>>>> no CDS = 80.35 ms -> 79.14ms (-1.5%) >>>>>>>>>> w/ CDS = 53.89ms -> 52.62ms (-2.36%) >>>>>>>>>> >>>>>>>>>> clojure sample app: >>>>>>>>>> no CDS = 1442.46ms -> 1436.11ms (-0.44%) >>>>>>>>>> w/ CDS = 695.78ms -> 679.52ms (-2.34%) >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> - Ioi >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From david.holmes at oracle.com Wed May 3 04:11:17 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 3 May 2017 14:11:17 +1000 Subject: RFR (XS): 8067728: Flag::unlock_diagnostic() should be called Flag::clear_diagnostic() In-Reply-To: References: Message-ID: Hi Rachel, Looks good. Do we have any tests that clear the diagnostic bit? That would be nicer for checking the logic instead of adding the assert Ioi suggested. Thanks, David On 3/05/2017 4:15 AM, Rachel Protacio wrote: > Hi, > > Please review this tiny change, renaming Flag::unlock_diagnostic() as > Flag::clear_diagnostic. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8067728 > Open webrev: http://cr.openjdk.java.net/~rprotacio/8067728.00/ > > Thanks! > Rachel From david.holmes at oracle.com Wed May 3 04:32:14 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 3 May 2017 14:32:14 +1000 Subject: RFR 8056900: Enhance NoClassDefFound exception messaging In-Reply-To: <9329c480-6e86-d8f6-57f2-23949ee21fcb@oracle.com> References: <9329c480-6e86-d8f6-57f2-23949ee21fcb@oracle.com> Message-ID: <45b7dc8c-07e7-3215-c40a-cc8851bd9bcf@oracle.com> Hi Harold, On 3/05/2017 4:29 AM, harold seigel wrote: > Hi, > > Please review this small JDK-10 change to enhance NoClassDefFoundError > exception messages. > > Open Webrev: > http://cr.openjdk.java.net/~hseigel/bug_8056900/webrev/index.html A few comments: In src/share/vm/oops/instanceKlass.cpp 517 // This is for CDS dumping phase only -- we use the in_error_state to indicate that 518 // the class has failed verification. That comment is somewhat misleading. The is_in_error_state() indicates that class initialization failed, not verification. However CDS dumping also uses it to indicate if a supertype was is_in_error_state(). I think using: 527 "Verification failed for class %s", would be confusing to an end user trying to figure out how a perfectly valid class could fail verification! I suggest the more direct: "Class %s, or one of its supertypes, failed class initialization" --- src/share/vm/prims/jni.cpp Suggestion: "Class name exceeds maximum length of %d: %s", ... in both cases. 386 THROW_MSG_0(vmSymbols::java_lang_NoClassDefFoundError(), "Class name is null"); Suggestion: "No class name given" --- Ditto for src/share/vm/prims/jvm.cpp changes. --- Thanks, David ----- > JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8056900 > > The change was tested with JCK tests, the JTreg hotspot, java/io, > java/lang, java/util and other tests, the RBT tier2 -tier5 tests, the > co-located NSK tests, JPRT, and with the new test provided with the change. > > Thanks, Harold > From david.holmes at oracle.com Wed May 3 04:59:13 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 3 May 2017 14:59:13 +1000 Subject: RFR 8178604: JVM does not allow defining boot loader modules in exploded build after module system initialization In-Reply-To: <46a812ca-228d-01e7-e889-fd22f2af0537@oracle.com> References: <85e36988-a145-6805-8659-56c738013f58@oracle.com> <46a812ca-228d-01e7-e889-fd22f2af0537@oracle.com> Message-ID: <11709d4a-0a41-1d38-fcd1-67d42c33eaeb@oracle.com> On 3/05/2017 7:11 AM, Lois Foltan wrote: > On 5/2/2017 9:03 AM, harold seigel wrote: >> Hi Lois, >> >> Thanks for your comments. Please see my in-line responses. >> >> Harold >> >> >> On 5/1/2017 4:14 PM, Lois Foltan wrote: >>> Hi Harold, >>> >>> Looks good. A couple of comments: >>> >>> src/share/vm/classfile/classLoader.cpp >>> line #828 - please take out the Module_lock when the >>> _exploded_entries is created >> This fix does not change the possible need for synchronization when >> _exploded_entries gets created during module system initialization. >> Are you saying that the existing code is wrong and should have taken >> out the Module_lock before creating _exploding_entries? > > The method ClassLoader::add_to_exploded_build_list can be called at > anytime (during module system initialization and post module system > initialization). Thus it seems prudent that no matter what the current > JVM phase, that any access to ClassLoader::_exploding_entries be > protected via a lock. That includes creation as well as insertion into > the list. If creation is not guaranteed to occur before other threads that will call the same method and use the _exploded_entries list, then synchronization of some form is essential. If it is guaranteed then we don't need create time sync just to be prudent - but the guarantee must be firmly established and clearly documented. >> >>> line #1402 - I like the new factored out method >>> find_first_module_cpe(), however, instead of adding a new "need_lock" >>> parameter, I would rather see code >>> in find_first_module_cpe() that takes the >>> Module_lock if the module_list parameter equals >>> ClassLoader::_exploded_entries >> I think your suggestion makes the code less flexible and does not >> require potential future callers of search_module_entries() to think >> about synchronization. So, I'd prefer to leave that change as is. > > I see your point. My point was based on reduced code readability, the > further away from the decision to establish the lock, the less apparent > it is within the new method ClassLoader::find_first_module_cpe() as to > what situations warranted the lock in the first place. Further, this: 1495 stream = search_module_entries(_exploded_entries, class_name, file_name, true, CHECK_NULL); is potentially unsafe if we can race with creation because we first access _exploded_entries without holding any lock! And the fact the method needs to do different things depending on which "list" was passed in suggests to me it should be two different initial entry points. Thanks, David ----- > Thanks, > Lois > >>> >>> Thanks, >>> Lois >>> >>> On 5/1/2017 3:36 PM, harold seigel wrote: >>>> Hi, >>>> >>>> Please review this JDK-10 bug fix to allow defining of modules to >>>> the boot loader in exploded builds after module system >>>> initialization. The fix uses the module lock to synchronize access >>>> to the _exploded_entries data structure (which is only used by >>>> exploded builds). >>>> >>>> Note that the above capability already exists for regular builds. >>>> >>>> Open Webrev: >>>> http://cr.openjdk.java.net/~hseigel/bug_8178604/webrev/index.html >>>> >>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8178604 >>>> >>>> The fix was tested with JCK tests, the JTreg hotspot, java/io, >>>> java/lang, java/util and other tests, the RBT tier2 -tier5 tests, >>>> the co-located NSK tests, and with JPRT. The JTReg and JCK tests >>>> were run with an exploded build. Also, the example program in the >>>> JBS bug was run by hand to verify the fix. >>>> >>>> Thanks, Harold >>>> >>> >> > From david.holmes at oracle.com Wed May 3 05:33:19 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 3 May 2017 15:33:19 +1000 Subject: RFR: 8154791: Xlog classload too redundant msgs info/debug In-Reply-To: <5908D481.2070104@oracle.com> References: <6ff9f069-2da7-b725-d29f-e3d850650aca@oracle.com> <5908D481.2070104@oracle.com> Message-ID: <1536f123-7a3a-4b25-682b-f1114573e8dc@oracle.com> Hi Ioi, I'm a bit confused ... On 3/05/2017 4:48 AM, Ioi Lam wrote: > Hi Rachel, > > There are a few reasons why the current output has duplications. The > "logging levels are not mutually exclusive" comment means this: > > java -Xlog:class+load=debug:file=debug.log > -Xlog:class+load=info:file=info.log > > So info.log contains only the info level logs, but debug.log contains > both info and debug level logs. Right - that is how it works. > Ideally I want info.log to contain this: > > [0.162s][info ][class,load] java.lang.Comparable source: jrt:/java.base > > .. and debug.log to contain this: > > [0.162s][debug ][class,load] java.lang.Comparable source: jrt:/java.base > [NULL class_loader] 0x00000007c00013a8 super: 0x00000007c0000fb0 bytes: 235 > checksum: a75dadb6 Okay, but you are only going to get that if you disable the info logging when debug is active - otherwise as you just noted the debug log contains debug+info. I'm unclear how you expect to produce the above output if you have enabled the two log files as above?? I don't think it can be done. Cheers, David > Your current patch splits out the debug log into several lines. This > makes it difficult to analyze the log when classes are loaded > concurrently on different threads: > > [0.162s][info ][class,load] java.lang.Comparable source: jrt:/java.base > [0.162s][info ][class,load] java.lang.Comparable loader: [NULL > class_loader] > [0.162s][debug][class,load] java.lang.Comparable klass: 0x00000007c00013a8 > [0.163s][info ][class,load] java.lang.String source: jrt:/java.base <<< > oops! from a different thread > [0.164s][debug][class,load] java.lang.Comparable super: 0x00000007c0000fb0 > [0.164s][info ][class,load] java.lang.Comparable bytes: 235 > [0.164s][info ][class,load] java.lang.Comparable checksum: a75dadb6 > > Because the leveling in UL is by lines, in JDK-8079408, where this was > first implemented, I couldn't find a way to print "java.lang.Comparable > source: jrt:/java.base" in "info" mode, and print the rest in "debug" > mode, while keeping the whole thing on the same line (in both the > info.log and debug.log files). That's why we have the 2 lines with > duplicated info. > > I think it's really important to keep everything on the same line (the > output needs to be processed automatically by scripts). With the current > UL restrictions, I think the only way to do it is make the "info" > logging more verbose when "debug" level is selected. E.g., > > outputStream* info_log = Log(class, load)::info_stream(); > info_log->print("%s", ext_name); > info_log->print_cr(" source: jrt:/%s", module_name); .... > > if (log_is_enabled(Debug, class, load)) { // always print to info > log, even if debug is selected > info_log->print(" klass: " INTPTR_FORMAT, p2i(this)); > .... > } > > So now info.log and debug.log will have the same logs, but oh well ..... > > Thanks > - Ioi > > On 5/2/17 10:32 AM, Rachel Protacio wrote: >> Hello! >> >> Please review this enhancement correcting redundancies and neatening >> up the -Xlog:class+load code. The redundancy in the code before I >> believe stemmed from a misunderstanding about logging levels (the >> comment saying they are not mutually exclusive was misleading). >> >> Tested with JPRT and RBT. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8154791 >> Webrev: http://cr.openjdk.java.net/~rprotacio/8154791.00 >> >> Thanks, >> Rachel > From ioi.lam at oracle.com Wed May 3 06:05:07 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 02 May 2017 23:05:07 -0700 Subject: RFR: 8154791: Xlog classload too redundant msgs info/debug In-Reply-To: <1536f123-7a3a-4b25-682b-f1114573e8dc@oracle.com> References: <6ff9f069-2da7-b725-d29f-e3d850650aca@oracle.com> <5908D481.2070104@oracle.com> <1536f123-7a3a-4b25-682b-f1114573e8dc@oracle.com> Message-ID: <59097313.8060604@oracle.com> On 5/2/17 10:33 PM, David Holmes wrote: > Hi Ioi, > > I'm a bit confused ... > > On 3/05/2017 4:48 AM, Ioi Lam wrote: >> Hi Rachel, >> >> There are a few reasons why the current output has duplications. The >> "logging levels are not mutually exclusive" comment means this: >> >> java -Xlog:class+load=debug:file=debug.log >> -Xlog:class+load=info:file=info.log >> >> So info.log contains only the info level logs, but debug.log contains >> both info and debug level logs. > > Right - that is how it works. > >> Ideally I want info.log to contain this: >> >> [0.162s][info ][class,load] java.lang.Comparable source: jrt:/java.base >> >> .. and debug.log to contain this: >> >> [0.162s][debug ][class,load] java.lang.Comparable source: jrt:/java.base >> [NULL class_loader] 0x00000007c00013a8 super: 0x00000007c0000fb0 >> bytes: 235 >> checksum: a75dadb6 > > Okay, but you are only going to get that if you disable the info > logging when debug is active - otherwise as you just noted the debug > log contains debug+info. I'm unclear how you expect to produce the > above output if you have enabled the two log files as above?? I don't > think it can be done. > Sorry I wasn't very clear. What I meant was this: For parsability, we need to atomically group together outputs that have different verbosity. Let's say you have this group of outputs: [info] A [debug] B Today the only atomic unit is a single line, so the only way to atomically group A and B together is to repeat A in the [debug] level: [info] A [debug] A B if we want to avoid the redundancy, we could potentially change the output format, so that in debug.log you have something like this: [info+debug] A B But anyway, this is probably too messy to implement. That's why I proposed in another e-mail that UL should provide an atomic group of multiple lines. Thanks - Ioi > Cheers, > David > >> Your current patch splits out the debug log into several lines. This >> makes it difficult to analyze the log when classes are loaded >> concurrently on different threads: >> >> [0.162s][info ][class,load] java.lang.Comparable source: jrt:/java.base >> [0.162s][info ][class,load] java.lang.Comparable loader: [NULL >> class_loader] >> [0.162s][debug][class,load] java.lang.Comparable klass: >> 0x00000007c00013a8 >> [0.163s][info ][class,load] java.lang.String source: jrt:/java.base <<< >> oops! from a different thread >> [0.164s][debug][class,load] java.lang.Comparable super: >> 0x00000007c0000fb0 >> [0.164s][info ][class,load] java.lang.Comparable bytes: 235 >> [0.164s][info ][class,load] java.lang.Comparable checksum: a75dadb6 >> >> Because the leveling in UL is by lines, in JDK-8079408, where this was >> first implemented, I couldn't find a way to print "java.lang.Comparable >> source: jrt:/java.base" in "info" mode, and print the rest in "debug" >> mode, while keeping the whole thing on the same line (in both the >> info.log and debug.log files). That's why we have the 2 lines with >> duplicated info. >> >> I think it's really important to keep everything on the same line (the >> output needs to be processed automatically by scripts). With the current >> UL restrictions, I think the only way to do it is make the "info" >> logging more verbose when "debug" level is selected. E.g., >> >> outputStream* info_log = Log(class, load)::info_stream(); >> info_log->print("%s", ext_name); >> info_log->print_cr(" source: jrt:/%s", module_name); .... >> >> if (log_is_enabled(Debug, class, load)) { // always print to info >> log, even if debug is selected >> info_log->print(" klass: " INTPTR_FORMAT, p2i(this)); >> .... >> } >> >> So now info.log and debug.log will have the same logs, but oh well ..... >> >> Thanks >> - Ioi >> >> On 5/2/17 10:32 AM, Rachel Protacio wrote: >>> Hello! >>> >>> Please review this enhancement correcting redundancies and neatening >>> up the -Xlog:class+load code. The redundancy in the code before I >>> believe stemmed from a misunderstanding about logging levels (the >>> comment saying they are not mutually exclusive was misleading). >>> >>> Tested with JPRT and RBT. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8154791 >>> Webrev: http://cr.openjdk.java.net/~rprotacio/8154791.00 >>> >>> Thanks, >>> Rachel >> From thomas.stuefe at gmail.com Wed May 3 06:23:09 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 3 May 2017 08:23:09 +0200 Subject: RFR: 8154791: Xlog classload too redundant msgs info/debug In-Reply-To: <5908F5D4.6040505@oracle.com> References: <6ff9f069-2da7-b725-d29f-e3d850650aca@oracle.com> <5908D481.2070104@oracle.com> <6a8ffd20-ce58-150b-a94d-c51c5956d191@oracle.com> <5908F288.9020205@oracle.com> <5908F5D4.6040505@oracle.com> Message-ID: Hi Ioi, On Tue, May 2, 2017 at 11:10 PM, Ioi Lam wrote: > By the way, I think we should really fix this in the UL level -- have > multi-line buffering, so you can prevent other threads from interleaving > your input. Make it something like: > > Log(class, load).start_buffering(); > log_info(class, load)(class name and source); > log_debug(class, load)(other details); > Log(class, load).flush(); > > That way, the output will look like this: > > [0.162s][debug ][class,load] java.lang.Comparable source: jrt:/java.base > [0.162s][debug ][class,load] [NULL class_loader] 0x00000007c00013a8 super: > 0x00000007c0000fb0 bytes: 235 checksum: a75dadb6 > > So the output will be terse, and still parsable. > > I think this will be useful for other types of logs as well, not just the > class loading logs. > > I thought that already exists: https://bugs.openjdk.java.net/browse/JDK-8145934 see logMessage.hpp. Kind Regards, Thomas > Thanks > - Ioi > > > On 5/2/17 1:56 PM, Ioi Lam wrote: > >> >> >> On 5/2/17 1:31 PM, Rachel Protacio wrote: >> >>> Hi Ioi, >>> >>> Thanks for your reply, comments inline. >>> >>> >>> On 5/2/2017 2:48 PM, Ioi Lam wrote: >>> >>>> Hi Rachel, >>>> >>>> There are a few reasons why the current output has duplications. The >>>> "logging levels are not mutually exclusive" comment means this: >>>> >>>> java -Xlog:class+load=debug:file=debug.log >>>> -Xlog:class+load=info:file=info.log >>>> >>>> So info.log contains only the info level logs, but debug.log contains >>>> both info and debug level logs. >>>> >>>> Ideally I want info.log to contain this: >>>> >>>> [0.162s][info ][class,load] java.lang.Comparable source: jrt:/java.base >>>> >>>> .. and debug.log to contain this: >>>> >>>> [0.162s][debug ][class,load] java.lang.Comparable source: >>>> jrt:/java.base [NULL class_loader] 0x00000007c00013a8 super: >>>> 0x00000007c0000fb0 bytes: 235 >>>> checksum: a75dadb6 >>>> >>> I see what you want, but I think the problem is that logging doesn't >>> work that way. If debug is specified, both debug and info levels are >>> printed. So the command >>> >>> java -Xlog:class+load=debug:file=debug.log >>> >>> will print both info level logging and debug level logging to this >>> "debug.log" file. And then tacking on >>> >>> -Xlog:class+load=info:file=info.log >>> >>> in the command would print just the info level logging to the "info.log" >>> file. But since it's all the same java command, it will always know that >>> both debug and info levels are specified. I believe what you want me to do >>> is say "if debug is specified, print everything to debug only" but then the >>> info.log would be empty. You would have to run a separate java command with >>> just the info level specified to get the smaller amount of information. >>> >> >> No, that's not what I want. I am perfectly fine with how UL works today: >> info.log contains only the info level, and debug.info contains both >> debug and info level. >> >> All I want is everything related to the same class to be printed on the >> same line. For example, I don't want debug.log to contain this: >> >> [0.162s][info ][class,load] java.lang.Comparable source: jrt:/java.base >> [0.162s][debug ][class,load] [NULL class_loader] 0x00000007c00013a8 >> super: 0x00000007c0000fb0 bytes: 235 checksum: a75dadb6 >> >> That's because it's impossible to associated the second with the first >> line when you have parallel threads loading classes concurrently. >> >> But I think that would violate user expectations - if they get certain >>> information with info logging, adding the extra level of debug logging >>> should not change what they see from the info logging. >>> >>>> >>>> Your current patch splits out the debug log into several lines. This >>>> makes it difficult to analyze the log when classes are loaded concurrently >>>> on different threads: >>>> >>>> [0.162s][info ][class,load] java.lang.Comparable source: jrt:/java.base >>>> [0.162s][info ][class,load] java.lang.Comparable loader: [NULL >>>> class_loader] >>>> [0.162s][debug][class,load] java.lang.Comparable klass: >>>> 0x00000007c00013a8 >>>> [0.163s][info ][class,load] java.lang.String source: jrt:/java.base <<< >>>> oops! from a different thread >>>> [0.164s][debug][class,load] java.lang.Comparable super: >>>> 0x00000007c0000fb0 >>>> [0.164s][info ][class,load] java.lang.Comparable bytes: 235 >>>> [0.164s][info ][class,load] java.lang.Comparable checksum: a75dadb6 >>>> >>> That's fair. I was trying to make it more readable since it's hard to >>> visually parse one giant line. Maybe I could put some delimiter in between >>> each piece of info? But if no one else cares, I can just put it back as one >>> line, no delimiters. >>> >>>> >>>> Because the leveling in UL is by lines, in JDK-8079408, where this was >>>> first implemented, I couldn't find a way to print "java.lang.Comparable >>>> source: jrt:/java.base" in "info" mode, and print the rest in "debug" mode, >>>> while keeping the whole thing on the same line (in both the info.log and >>>> debug.log files). That's why we have the 2 lines with duplicated info. >>>> >>>> I think it's really important to keep everything on the same line (the >>>> output needs to be processed automatically by scripts). With the current UL >>>> restrictions, I think the only way to do it is make the "info" logging more >>>> verbose when "debug" level is selected. E.g., >>>> >>>> outputStream* info_log = Log(class, load)::info_stream(); >>>> info_log->print("%s", ext_name); >>>> info_log->print_cr(" source: jrt:/%s", module_name); .... >>>> >>>> if (log_is_enabled(Debug, class, load)) { // always print to info >>>> log, even if debug is selected >>>> info_log->print(" klass: " INTPTR_FORMAT, p2i(this)); >>>> .... >>>> } >>>> >>>> So now info.log and debug.log will have the same logs, but oh well ..... >>>> >>> This sounds a little different. It sounds like >>> >>> When only info is specified, info log will contain: external name, >>> source, loader, bytes, checksum >>> >>> When debug is specified, info log will contain the same as above, >>> plus: klass, super, interfaces >>> When debug is specified, debug log will be an exact duplicate of >>> info log so it can print to an external file >>> >>> Did I understand that right? Please correct me. If this is what you >>> meant, I would again argue that this violates the expectation that "info is >>> info is info" regardless of what else is specified (not a written rule, but >>> how I think it should work). And it is redundant, which is the original >>> issue in the name of this RFE. >>> >>> I think the best bet would be >>> >>> Info log: external name, source, loader, bytes, checksum >>> Debug log: external name, klass, super, interfaces >>> >>> Or we get rid of the level distinction completely and put all the >>> information together in one line so it never gets broken up. Probably under >>> debug. >>> >>> >> We need to keep the current debug output intact, on a single line, so >> that the information can be processed by scripts. >> >> So, it seems like our 3 choices are: >> >> (1) make the info output the same as debug, if -Xlog:class+load=debug is >> specified >> (2) remove the info output altogether >> (3) do nothing >> >> You don't like (1). >> >> I don't think (2) achieves what this RFE wants -- namely, make the output >> less redundant, because you will see all the details all the time, and you >> don't even get a choice for more terse output. It also makes the output of >> -verbose different (and much less useful) than in JDK 9. >> >> So I would vote for (3), keep the output as is. Yeah, with debug level >> you have a lot of output, but that's pretty easy to filter. I think we had >> a discussion about this when 8079408 was implemented, and that was the >> conclusion we had. >> >> Thanks >> - Ioi >> >> Let me know what you think. >>> >>> Thanks, >>> Rachel >>> >>> >>>> Thanks >>>> - Ioi >>>> >>>> On 5/2/17 10:32 AM, Rachel Protacio wrote: >>>> >>>>> Hello! >>>>> >>>>> Please review this enhancement correcting redundancies and neatening >>>>> up the -Xlog:class+load code. The redundancy in the code before I believe >>>>> stemmed from a misunderstanding about logging levels (the comment saying >>>>> they are not mutually exclusive was misleading). >>>>> >>>>> Tested with JPRT and RBT. >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8154791 >>>>> Webrev: http://cr.openjdk.java.net/~rprotacio/8154791.00 >>>>> >>>>> Thanks, >>>>> Rachel >>>>> >>>> >>>> >>> >> > From marcus.larsson at oracle.com Wed May 3 07:03:21 2017 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Wed, 3 May 2017 09:03:21 +0200 Subject: RFR: 8154791: Xlog classload too redundant msgs info/debug In-Reply-To: References: <6ff9f069-2da7-b725-d29f-e3d850650aca@oracle.com> <5908D481.2070104@oracle.com> <6a8ffd20-ce58-150b-a94d-c51c5956d191@oracle.com> <5908F288.9020205@oracle.com> <5908F5D4.6040505@oracle.com> Message-ID: <00bc98e3-13d1-12bc-5d78-01a05fb1c45a@oracle.com> Hi, On 2017-05-03 08:23, Thomas St?fe wrote: > Hi Ioi, > > On Tue, May 2, 2017 at 11:10 PM, Ioi Lam wrote: > >> By the way, I think we should really fix this in the UL level -- have >> multi-line buffering, so you can prevent other threads from interleaving >> your input. Make it something like: >> >> Log(class, load).start_buffering(); >> log_info(class, load)(class name and source); >> log_debug(class, load)(other details); >> Log(class, load).flush(); >> >> That way, the output will look like this: >> >> [0.162s][debug ][class,load] java.lang.Comparable source: jrt:/java.base >> [0.162s][debug ][class,load] [NULL class_loader] 0x00000007c00013a8 super: >> 0x00000007c0000fb0 bytes: 235 checksum: a75dadb6 >> >> So the output will be terse, and still parsable. >> >> I think this will be useful for other types of logs as well, not just the >> class loading logs. >> >> > I thought that already exists: > > https://bugs.openjdk.java.net/browse/JDK-8145934 > > see logMessage.hpp. > > Kind Regards, Thomas Indeed it does. Thanks for pointing that out, Thomas. Marcus From harold.seigel at oracle.com Wed May 3 12:01:10 2017 From: harold.seigel at oracle.com (harold seigel) Date: Wed, 3 May 2017 08:01:10 -0400 Subject: RFR 8056900: Enhance NoClassDefFound exception messaging In-Reply-To: <45b7dc8c-07e7-3215-c40a-cc8851bd9bcf@oracle.com> References: <9329c480-6e86-d8f6-57f2-23949ee21fcb@oracle.com> <45b7dc8c-07e7-3215-c40a-cc8851bd9bcf@oracle.com> Message-ID: Hi David, Thanks for your comments. I'll make the changes and send out a new rfr. Harold On 5/3/2017 12:32 AM, David Holmes wrote: > Hi Harold, > > On 3/05/2017 4:29 AM, harold seigel wrote: >> Hi, >> >> Please review this small JDK-10 change to enhance NoClassDefFoundError >> exception messages. >> >> Open Webrev: >> http://cr.openjdk.java.net/~hseigel/bug_8056900/webrev/index.html > > A few comments: > > In src/share/vm/oops/instanceKlass.cpp > > 517 // This is for CDS dumping phase only -- we use the > in_error_state to indicate that > 518 // the class has failed verification. > > That comment is somewhat misleading. The is_in_error_state() indicates > that class initialization failed, not verification. However CDS > dumping also uses it to indicate if a supertype was > is_in_error_state(). I think using: > > 527 "Verification failed for class %s", > > would be confusing to an end user trying to figure out how a perfectly > valid class could fail verification! I suggest the more direct: > > "Class %s, or one of its supertypes, failed class initialization" > > --- > > src/share/vm/prims/jni.cpp > > Suggestion: > > "Class name exceeds maximum length of %d: %s", ... > > in both cases. > > 386 THROW_MSG_0(vmSymbols::java_lang_NoClassDefFoundError(), > "Class name is null"); > > Suggestion: "No class name given" > > --- > > Ditto for src/share/vm/prims/jvm.cpp changes. > > --- > > Thanks, > David > ----- > > >> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8056900 >> >> The change was tested with JCK tests, the JTreg hotspot, java/io, >> java/lang, java/util and other tests, the RBT tier2 -tier5 tests, the >> co-located NSK tests, JPRT, and with the new test provided with the >> change. >> >> Thanks, Harold >> From thomas.stuefe at gmail.com Wed May 3 13:22:39 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 3 May 2017 15:22:39 +0200 Subject: RFR: 8154791: Xlog classload too redundant msgs info/debug In-Reply-To: <00bc98e3-13d1-12bc-5d78-01a05fb1c45a@oracle.com> References: <6ff9f069-2da7-b725-d29f-e3d850650aca@oracle.com> <5908D481.2070104@oracle.com> <6a8ffd20-ce58-150b-a94d-c51c5956d191@oracle.com> <5908F288.9020205@oracle.com> <5908F5D4.6040505@oracle.com> <00bc98e3-13d1-12bc-5d78-01a05fb1c45a@oracle.com> Message-ID: On Wed, May 3, 2017 at 9:03 AM, Marcus Larsson wrote: > Hi, > > > On 2017-05-03 08:23, Thomas St?fe wrote: > >> Hi Ioi, >> >> On Tue, May 2, 2017 at 11:10 PM, Ioi Lam wrote: >> >> By the way, I think we should really fix this in the UL level -- have >>> multi-line buffering, so you can prevent other threads from interleaving >>> your input. Make it something like: >>> >>> Log(class, load).start_buffering(); >>> log_info(class, load)(class name and source); >>> log_debug(class, load)(other details); >>> Log(class, load).flush(); >>> >>> That way, the output will look like this: >>> >>> [0.162s][debug ][class,load] java.lang.Comparable source: jrt:/java.base >>> [0.162s][debug ][class,load] [NULL class_loader] 0x00000007c00013a8 >>> super: >>> 0x00000007c0000fb0 bytes: 235 checksum: a75dadb6 >>> >>> So the output will be terse, and still parsable. >>> >>> I think this will be useful for other types of logs as well, not just the >>> class loading logs. >>> >>> >>> I thought that already exists: >> >> https://bugs.openjdk.java.net/browse/JDK-8145934 >> >> see logMessage.hpp. >> >> Kind Regards, Thomas >> > > Indeed it does. Thanks for pointing that out, Thomas. > > Thanks for implementing it :) ..Thomas > Marcus > From harold.seigel at oracle.com Wed May 3 14:23:30 2017 From: harold.seigel at oracle.com (harold seigel) Date: Wed, 3 May 2017 10:23:30 -0400 Subject: RFR 8056900: Enhance NoClassDefFound exception messaging In-Reply-To: <45b7dc8c-07e7-3215-c40a-cc8851bd9bcf@oracle.com> References: <9329c480-6e86-d8f6-57f2-23949ee21fcb@oracle.com> <45b7dc8c-07e7-3215-c40a-cc8851bd9bcf@oracle.com> Message-ID: <8bae6d8a-7bd3-f576-0f9c-f4b17314a597@oracle.com> Hi, Please review this updated webrev: http://cr.openjdk.java.net/~hseigel/bug_8056900.2/webrev/index.html It includes the below changes suggested by David. Thanks, Harold On 5/3/2017 12:32 AM, David Holmes wrote: > Hi Harold, > > On 3/05/2017 4:29 AM, harold seigel wrote: >> Hi, >> >> Please review this small JDK-10 change to enhance NoClassDefFoundError >> exception messages. >> >> Open Webrev: >> http://cr.openjdk.java.net/~hseigel/bug_8056900/webrev/index.html > > A few comments: > > In src/share/vm/oops/instanceKlass.cpp > > 517 // This is for CDS dumping phase only -- we use the > in_error_state to indicate that > 518 // the class has failed verification. > > That comment is somewhat misleading. The is_in_error_state() indicates > that class initialization failed, not verification. However CDS > dumping also uses it to indicate if a supertype was > is_in_error_state(). I think using: > > 527 "Verification failed for class %s", > > would be confusing to an end user trying to figure out how a perfectly > valid class could fail verification! I suggest the more direct: > > "Class %s, or one of its supertypes, failed class initialization" > > --- > > src/share/vm/prims/jni.cpp > > Suggestion: > > "Class name exceeds maximum length of %d: %s", ... > > in both cases. > > 386 THROW_MSG_0(vmSymbols::java_lang_NoClassDefFoundError(), > "Class name is null"); > > Suggestion: "No class name given" > > --- > > Ditto for src/share/vm/prims/jvm.cpp changes. > > --- > > Thanks, > David > ----- > > >> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8056900 >> >> The change was tested with JCK tests, the JTreg hotspot, java/io, >> java/lang, java/util and other tests, the RBT tier2 -tier5 tests, the >> co-located NSK tests, JPRT, and with the new test provided with the >> change. >> >> Thanks, Harold >> From ioi.lam at oracle.com Wed May 3 16:20:30 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 03 May 2017 09:20:30 -0700 Subject: RFR [XS] JDK-8176863 Remove Hashtable::reverse() Message-ID: <590A034E.8000809@oracle.com> Hi, Please review this small change to remove an obsolete optimization: https://bugs.openjdk.java.net/browse/JDK-8176863 http://cr.openjdk.java.net/~iklam/jdk10/8176863_remove_hashtable_reverse.v01/ This is legacy code from JDK 1.5 that doesn't do what it intends to do and is no longer relevant. Thanks - Ioi From coleen.phillimore at oracle.com Wed May 3 18:17:30 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 3 May 2017 14:17:30 -0400 Subject: RFR [XS] JDK-8176863 Remove Hashtable::reverse() In-Reply-To: <590A034E.8000809@oracle.com> References: <590A034E.8000809@oracle.com> Message-ID: <1440a470-b3a6-ffd7-ad7a-08ad0828df05@oracle.com> Thank you!! Looks great to me. Coleen On 5/3/17 12:20 PM, Ioi Lam wrote: > Hi, > > Please review this small change to remove an obsolete optimization: > > https://bugs.openjdk.java.net/browse/JDK-8176863 > http://cr.openjdk.java.net/~iklam/jdk10/8176863_remove_hashtable_reverse.v01/ > > > This is legacy code from JDK 1.5 that doesn't do what it intends to do > and is no longer relevant. > > Thanks > - Ioi From harold.seigel at oracle.com Wed May 3 18:22:05 2017 From: harold.seigel at oracle.com (harold seigel) Date: Wed, 3 May 2017 14:22:05 -0400 Subject: RFR [XS] JDK-8176863 Remove Hashtable::reverse() In-Reply-To: <1440a470-b3a6-ffd7-ad7a-08ad0828df05@oracle.com> References: <590A034E.8000809@oracle.com> <1440a470-b3a6-ffd7-ad7a-08ad0828df05@oracle.com> Message-ID: Looks good to me, also. Thanks, Harold On 5/3/2017 2:17 PM, coleen.phillimore at oracle.com wrote: > Thank you!! Looks great to me. > Coleen > > On 5/3/17 12:20 PM, Ioi Lam wrote: >> Hi, >> >> Please review this small change to remove an obsolete optimization: >> >> https://bugs.openjdk.java.net/browse/JDK-8176863 >> http://cr.openjdk.java.net/~iklam/jdk10/8176863_remove_hashtable_reverse.v01/ >> >> >> This is legacy code from JDK 1.5 that doesn't do what it intends to >> do and is no longer relevant. >> >> Thanks >> - Ioi > From serguei.spitsyn at oracle.com Wed May 3 19:18:48 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 3 May 2017 12:18:48 -0700 Subject: RFR [XS] JDK-8176863 Remove Hashtable::reverse() In-Reply-To: <590A034E.8000809@oracle.com> References: <590A034E.8000809@oracle.com> Message-ID: Hi Ioi, Looks good. Thanks, Serguei On 5/3/17 09:20, Ioi Lam wrote: > Hi, > > Please review this small change to remove an obsolete optimization: > > https://bugs.openjdk.java.net/browse/JDK-8176863 > http://cr.openjdk.java.net/~iklam/jdk10/8176863_remove_hashtable_reverse.v01/ > > > This is legacy code from JDK 1.5 that doesn't do what it intends to do > and is no longer relevant. > > Thanks > - Ioi From serguei.spitsyn at oracle.com Wed May 3 19:39:55 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 3 May 2017 12:39:55 -0700 Subject: RFR 8056900: Enhance NoClassDefFound exception messaging In-Reply-To: <8bae6d8a-7bd3-f576-0f9c-f4b17314a597@oracle.com> References: <9329c480-6e86-d8f6-57f2-23949ee21fcb@oracle.com> <45b7dc8c-07e7-3215-c40a-cc8851bd9bcf@oracle.com> <8bae6d8a-7bd3-f576-0f9c-f4b17314a597@oracle.com> Message-ID: <29f9e6e8-5dd4-26ab-b5bc-3c1c83181c24@oracle.com> Hi Harold, The fix looks good to me. The test does not cover all the product changes. Is it possible to extend it to check the JNI FindClass as well? Thanks, Serguei On 5/3/17 07:23, harold seigel wrote: > Hi, > > Please review this updated webrev: > http://cr.openjdk.java.net/~hseigel/bug_8056900.2/webrev/index.html > > It includes the below changes suggested by David. > > Thanks, Harold > > > On 5/3/2017 12:32 AM, David Holmes wrote: >> Hi Harold, >> >> On 3/05/2017 4:29 AM, harold seigel wrote: >>> Hi, >>> >>> Please review this small JDK-10 change to enhance NoClassDefFoundError >>> exception messages. >>> >>> Open Webrev: >>> http://cr.openjdk.java.net/~hseigel/bug_8056900/webrev/index.html >> >> A few comments: >> >> In src/share/vm/oops/instanceKlass.cpp >> >> 517 // This is for CDS dumping phase only -- we use the >> in_error_state to indicate that >> 518 // the class has failed verification. >> >> That comment is somewhat misleading. The is_in_error_state() >> indicates that class initialization failed, not verification. However >> CDS dumping also uses it to indicate if a supertype was >> is_in_error_state(). I think using: >> >> 527 "Verification failed for class %s", >> >> would be confusing to an end user trying to figure out how a >> perfectly valid class could fail verification! I suggest the more >> direct: >> >> "Class %s, or one of its supertypes, failed class initialization" >> >> --- >> >> src/share/vm/prims/jni.cpp >> >> Suggestion: >> >> "Class name exceeds maximum length of %d: %s", ... >> >> in both cases. >> >> 386 THROW_MSG_0(vmSymbols::java_lang_NoClassDefFoundError(), >> "Class name is null"); >> >> Suggestion: "No class name given" >> >> --- >> >> Ditto for src/share/vm/prims/jvm.cpp changes. >> >> --- >> >> Thanks, >> David >> ----- >> >> >>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8056900 >>> >>> The change was tested with JCK tests, the JTreg hotspot, java/io, >>> java/lang, java/util and other tests, the RBT tier2 -tier5 tests, the >>> co-located NSK tests, JPRT, and with the new test provided with the >>> change. >>> >>> Thanks, Harold >>> > From harold.seigel at oracle.com Wed May 3 19:43:06 2017 From: harold.seigel at oracle.com (harold seigel) Date: Wed, 3 May 2017 15:43:06 -0400 Subject: RFR 8056900: Enhance NoClassDefFound exception messaging In-Reply-To: <29f9e6e8-5dd4-26ab-b5bc-3c1c83181c24@oracle.com> References: <9329c480-6e86-d8f6-57f2-23949ee21fcb@oracle.com> <45b7dc8c-07e7-3215-c40a-cc8851bd9bcf@oracle.com> <8bae6d8a-7bd3-f576-0f9c-f4b17314a597@oracle.com> <29f9e6e8-5dd4-26ab-b5bc-3c1c83181c24@oracle.com> Message-ID: <538302fc-646e-24fd-ef39-a3df7a380b63@oracle.com> Hi Serguei, Thanks for your comments. I'll look into adding a JNI component to the test. Harold On 5/3/2017 3:39 PM, serguei.spitsyn at oracle.com wrote: > Hi Harold, > > The fix looks good to me. > The test does not cover all the product changes. > Is it possible to extend it to check the JNI FindClass as well? > > Thanks, > Serguei > > > > On 5/3/17 07:23, harold seigel wrote: >> Hi, >> >> Please review this updated webrev: >> http://cr.openjdk.java.net/~hseigel/bug_8056900.2/webrev/index.html >> >> It includes the below changes suggested by David. >> >> Thanks, Harold >> >> >> On 5/3/2017 12:32 AM, David Holmes wrote: >>> Hi Harold, >>> >>> On 3/05/2017 4:29 AM, harold seigel wrote: >>>> Hi, >>>> >>>> Please review this small JDK-10 change to enhance NoClassDefFoundError >>>> exception messages. >>>> >>>> Open Webrev: >>>> http://cr.openjdk.java.net/~hseigel/bug_8056900/webrev/index.html >>> >>> A few comments: >>> >>> In src/share/vm/oops/instanceKlass.cpp >>> >>> 517 // This is for CDS dumping phase only -- we use the >>> in_error_state to indicate that >>> 518 // the class has failed verification. >>> >>> That comment is somewhat misleading. The is_in_error_state() >>> indicates that class initialization failed, not verification. >>> However CDS dumping also uses it to indicate if a supertype was >>> is_in_error_state(). I think using: >>> >>> 527 "Verification failed for class %s", >>> >>> would be confusing to an end user trying to figure out how a >>> perfectly valid class could fail verification! I suggest the more >>> direct: >>> >>> "Class %s, or one of its supertypes, failed class initialization" >>> >>> --- >>> >>> src/share/vm/prims/jni.cpp >>> >>> Suggestion: >>> >>> "Class name exceeds maximum length of %d: %s", ... >>> >>> in both cases. >>> >>> 386 THROW_MSG_0(vmSymbols::java_lang_NoClassDefFoundError(), "Class >>> name is null"); >>> >>> Suggestion: "No class name given" >>> >>> --- >>> >>> Ditto for src/share/vm/prims/jvm.cpp changes. >>> >>> --- >>> >>> Thanks, >>> David >>> ----- >>> >>> >>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8056900 >>>> >>>> The change was tested with JCK tests, the JTreg hotspot, java/io, >>>> java/lang, java/util and other tests, the RBT tier2 -tier5 tests, the >>>> co-located NSK tests, JPRT, and with the new test provided with the >>>> change. >>>> >>>> Thanks, Harold >>>> >> > From harold.seigel at oracle.com Wed May 3 21:06:23 2017 From: harold.seigel at oracle.com (harold seigel) Date: Wed, 3 May 2017 17:06:23 -0400 Subject: RFR 8056900: Enhance NoClassDefFound exception messaging In-Reply-To: <29f9e6e8-5dd4-26ab-b5bc-3c1c83181c24@oracle.com> References: <9329c480-6e86-d8f6-57f2-23949ee21fcb@oracle.com> <45b7dc8c-07e7-3215-c40a-cc8851bd9bcf@oracle.com> <8bae6d8a-7bd3-f576-0f9c-f4b17314a597@oracle.com> <29f9e6e8-5dd4-26ab-b5bc-3c1c83181c24@oracle.com> Message-ID: <94320113-27aa-48e1-f5ab-ac01634e229f@oracle.com> Hi, Please review this updated RFR that includes test cases for the JNI modifications. RFR: http://cr.openjdk.java.net/~hseigel/bug_8056900.3/webrev/index.html Thanks, Harold On 5/3/2017 3:39 PM, serguei.spitsyn at oracle.com wrote: > Hi Harold, > > The fix looks good to me. > The test does not cover all the product changes. > Is it possible to extend it to check the JNI FindClass as well? > > Thanks, > Serguei > > > > On 5/3/17 07:23, harold seigel wrote: >> Hi, >> >> Please review this updated webrev: >> http://cr.openjdk.java.net/~hseigel/bug_8056900.2/webrev/index.html >> >> It includes the below changes suggested by David. >> >> Thanks, Harold >> >> >> On 5/3/2017 12:32 AM, David Holmes wrote: >>> Hi Harold, >>> >>> On 3/05/2017 4:29 AM, harold seigel wrote: >>>> Hi, >>>> >>>> Please review this small JDK-10 change to enhance NoClassDefFoundError >>>> exception messages. >>>> >>>> Open Webrev: >>>> http://cr.openjdk.java.net/~hseigel/bug_8056900/webrev/index.html >>> >>> A few comments: >>> >>> In src/share/vm/oops/instanceKlass.cpp >>> >>> 517 // This is for CDS dumping phase only -- we use the >>> in_error_state to indicate that >>> 518 // the class has failed verification. >>> >>> That comment is somewhat misleading. The is_in_error_state() >>> indicates that class initialization failed, not verification. >>> However CDS dumping also uses it to indicate if a supertype was >>> is_in_error_state(). I think using: >>> >>> 527 "Verification failed for class %s", >>> >>> would be confusing to an end user trying to figure out how a >>> perfectly valid class could fail verification! I suggest the more >>> direct: >>> >>> "Class %s, or one of its supertypes, failed class initialization" >>> >>> --- >>> >>> src/share/vm/prims/jni.cpp >>> >>> Suggestion: >>> >>> "Class name exceeds maximum length of %d: %s", ... >>> >>> in both cases. >>> >>> 386 THROW_MSG_0(vmSymbols::java_lang_NoClassDefFoundError(), "Class >>> name is null"); >>> >>> Suggestion: "No class name given" >>> >>> --- >>> >>> Ditto for src/share/vm/prims/jvm.cpp changes. >>> >>> --- >>> >>> Thanks, >>> David >>> ----- >>> >>> >>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8056900 >>>> >>>> The change was tested with JCK tests, the JTreg hotspot, java/io, >>>> java/lang, java/util and other tests, the RBT tier2 -tier5 tests, the >>>> co-located NSK tests, JPRT, and with the new test provided with the >>>> change. >>>> >>>> Thanks, Harold >>>> >> > From serguei.spitsyn at oracle.com Wed May 3 21:24:07 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 3 May 2017 14:24:07 -0700 Subject: RFR 8056900: Enhance NoClassDefFound exception messaging In-Reply-To: <94320113-27aa-48e1-f5ab-ac01634e229f@oracle.com> References: <9329c480-6e86-d8f6-57f2-23949ee21fcb@oracle.com> <45b7dc8c-07e7-3215-c40a-cc8851bd9bcf@oracle.com> <8bae6d8a-7bd3-f576-0f9c-f4b17314a597@oracle.com> <29f9e6e8-5dd4-26ab-b5bc-3c1c83181c24@oracle.com> <94320113-27aa-48e1-f5ab-ac01634e229f@oracle.com> Message-ID: <106e5f11-7283-5ebf-f015-9e345548d2da@oracle.com> Hi Harold, Thank you for the update. It looks great! One tip is that you can pre-define the static String constants: "Class name exceeds maximum length of " "defineClass did not throw expected NoClassDefFoundError". But I leave it up to you. No webrev is needed if you decide to tweak it. Thanks, Serguei On 5/3/17 14:06, harold seigel wrote: > Hi, > > Please review this updated RFR that includes test cases for the JNI > modifications. > > RFR: http://cr.openjdk.java.net/~hseigel/bug_8056900.3/webrev/index.html > > Thanks, Harold > > > On 5/3/2017 3:39 PM, serguei.spitsyn at oracle.com wrote: >> Hi Harold, >> >> The fix looks good to me. >> The test does not cover all the product changes. >> Is it possible to extend it to check the JNI FindClass as well? >> >> Thanks, >> Serguei >> >> >> >> On 5/3/17 07:23, harold seigel wrote: >>> Hi, >>> >>> Please review this updated webrev: >>> http://cr.openjdk.java.net/~hseigel/bug_8056900.2/webrev/index.html >>> >>> It includes the below changes suggested by David. >>> >>> Thanks, Harold >>> >>> >>> On 5/3/2017 12:32 AM, David Holmes wrote: >>>> Hi Harold, >>>> >>>> On 3/05/2017 4:29 AM, harold seigel wrote: >>>>> Hi, >>>>> >>>>> Please review this small JDK-10 change to enhance >>>>> NoClassDefFoundError >>>>> exception messages. >>>>> >>>>> Open Webrev: >>>>> http://cr.openjdk.java.net/~hseigel/bug_8056900/webrev/index.html >>>> >>>> A few comments: >>>> >>>> In src/share/vm/oops/instanceKlass.cpp >>>> >>>> 517 // This is for CDS dumping phase only -- we use the >>>> in_error_state to indicate that >>>> 518 // the class has failed verification. >>>> >>>> That comment is somewhat misleading. The is_in_error_state() >>>> indicates that class initialization failed, not verification. >>>> However CDS dumping also uses it to indicate if a supertype was >>>> is_in_error_state(). I think using: >>>> >>>> 527 "Verification failed for class %s", >>>> >>>> would be confusing to an end user trying to figure out how a >>>> perfectly valid class could fail verification! I suggest the more >>>> direct: >>>> >>>> "Class %s, or one of its supertypes, failed class initialization" >>>> >>>> --- >>>> >>>> src/share/vm/prims/jni.cpp >>>> >>>> Suggestion: >>>> >>>> "Class name exceeds maximum length of %d: %s", ... >>>> >>>> in both cases. >>>> >>>> 386 THROW_MSG_0(vmSymbols::java_lang_NoClassDefFoundError(), "Class >>>> name is null"); >>>> >>>> Suggestion: "No class name given" >>>> >>>> --- >>>> >>>> Ditto for src/share/vm/prims/jvm.cpp changes. >>>> >>>> --- >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>> >>>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8056900 >>>>> >>>>> The change was tested with JCK tests, the JTreg hotspot, java/io, >>>>> java/lang, java/util and other tests, the RBT tier2 -tier5 tests, the >>>>> co-located NSK tests, JPRT, and with the new test provided with >>>>> the change. >>>>> >>>>> Thanks, Harold >>>>> >>> >> > From coleen.phillimore at oracle.com Wed May 3 22:08:51 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 3 May 2017 18:08:51 -0400 Subject: RFR 8056900: Enhance NoClassDefFound exception messaging In-Reply-To: <94320113-27aa-48e1-f5ab-ac01634e229f@oracle.com> References: <9329c480-6e86-d8f6-57f2-23949ee21fcb@oracle.com> <45b7dc8c-07e7-3215-c40a-cc8851bd9bcf@oracle.com> <8bae6d8a-7bd3-f576-0f9c-f4b17314a597@oracle.com> <29f9e6e8-5dd4-26ab-b5bc-3c1c83181c24@oracle.com> <94320113-27aa-48e1-f5ab-ac01634e229f@oracle.com> Message-ID: <9adcd093-6b93-ee72-304c-1e9717f745b3@oracle.com> Harold, This change looks good. Coleen On 5/3/17 5:06 PM, harold seigel wrote: > Hi, > > Please review this updated RFR that includes test cases for the JNI > modifications. > > RFR: http://cr.openjdk.java.net/~hseigel/bug_8056900.3/webrev/index.html > > Thanks, Harold > > > On 5/3/2017 3:39 PM, serguei.spitsyn at oracle.com wrote: >> Hi Harold, >> >> The fix looks good to me. >> The test does not cover all the product changes. >> Is it possible to extend it to check the JNI FindClass as well? >> >> Thanks, >> Serguei >> >> >> >> On 5/3/17 07:23, harold seigel wrote: >>> Hi, >>> >>> Please review this updated webrev: >>> http://cr.openjdk.java.net/~hseigel/bug_8056900.2/webrev/index.html >>> >>> It includes the below changes suggested by David. >>> >>> Thanks, Harold >>> >>> >>> On 5/3/2017 12:32 AM, David Holmes wrote: >>>> Hi Harold, >>>> >>>> On 3/05/2017 4:29 AM, harold seigel wrote: >>>>> Hi, >>>>> >>>>> Please review this small JDK-10 change to enhance >>>>> NoClassDefFoundError >>>>> exception messages. >>>>> >>>>> Open Webrev: >>>>> http://cr.openjdk.java.net/~hseigel/bug_8056900/webrev/index.html >>>> >>>> A few comments: >>>> >>>> In src/share/vm/oops/instanceKlass.cpp >>>> >>>> 517 // This is for CDS dumping phase only -- we use the >>>> in_error_state to indicate that >>>> 518 // the class has failed verification. >>>> >>>> That comment is somewhat misleading. The is_in_error_state() >>>> indicates that class initialization failed, not verification. >>>> However CDS dumping also uses it to indicate if a supertype was >>>> is_in_error_state(). I think using: >>>> >>>> 527 "Verification failed for class %s", >>>> >>>> would be confusing to an end user trying to figure out how a >>>> perfectly valid class could fail verification! I suggest the more >>>> direct: >>>> >>>> "Class %s, or one of its supertypes, failed class initialization" >>>> >>>> --- >>>> >>>> src/share/vm/prims/jni.cpp >>>> >>>> Suggestion: >>>> >>>> "Class name exceeds maximum length of %d: %s", ... >>>> >>>> in both cases. >>>> >>>> 386 THROW_MSG_0(vmSymbols::java_lang_NoClassDefFoundError(), "Class >>>> name is null"); >>>> >>>> Suggestion: "No class name given" >>>> >>>> --- >>>> >>>> Ditto for src/share/vm/prims/jvm.cpp changes. >>>> >>>> --- >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>> >>>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8056900 >>>>> >>>>> The change was tested with JCK tests, the JTreg hotspot, java/io, >>>>> java/lang, java/util and other tests, the RBT tier2 -tier5 tests, the >>>>> co-located NSK tests, JPRT, and with the new test provided with >>>>> the change. >>>>> >>>>> Thanks, Harold >>>>> >>> >> > From serguei.spitsyn at oracle.com Thu May 4 00:29:06 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 3 May 2017 17:29:06 -0700 Subject: 8178380: Module system implementation refresh (5/2017 update) In-Reply-To: References: Message-ID: Hi Alan, I reviewed the Hotspot changes and the Jdk changes related to java.instrument and jdk.attch (including a fragment in the LauncherHelper.java). The fixes look good to me. Thanks, Serguei On 5/1/17 13:28, Alan Bateman wrote: > As I mentioned in another thread, we need to get the changes > accumulated in jake to jdk9/dev. > > JDK-8178380 [1] has the summary of the changes that have accumulated > since the last refresh. > > One note on the hotspot repo is that the changes to dynamically > augment the platform modules run into JDK-8178604 when testing with an > exploded build. So there is a partial fix for this in jake. Harold has > the more complete fix in review on hotspot-runtime-dev for JDK 10. > > The webrevs with the changes is here: > http://cr.openjdk.java.net/~alanb/8178380/1/ > > -Alan > > [1] https://bugs.openjdk.java.net/browse/JDK-8178380 > From mandy.chung at oracle.com Thu May 4 02:08:21 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 3 May 2017 19:08:21 -0700 Subject: 8178380: Module system implementation refresh (5/2017 update) In-Reply-To: References: Message-ID: > On May 1, 2017, at 1:28 PM, Alan Bateman wrote: > > http://cr.openjdk.java.net/~alanb/8178380/2/ I reviewed all repos in the new version. src/java.base/share/classes/jdk/internal/module/ModuleBootstrap.java 148 if (System.getProperty("jdk.module.minimumBoot") != null) { This property can be removed after read the value, if present. 287 if (propValue != null && Boolean.getBoolean(propValue)) It should use Boolean.parseBoolean. I have fixed the above injake and also added a new test to verify -?show-module-resolution option. Otherwise, all looks good. Mandy From david.holmes at oracle.com Thu May 4 02:30:37 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 4 May 2017 12:30:37 +1000 Subject: RFR 8056900: Enhance NoClassDefFound exception messaging In-Reply-To: <94320113-27aa-48e1-f5ab-ac01634e229f@oracle.com> References: <9329c480-6e86-d8f6-57f2-23949ee21fcb@oracle.com> <45b7dc8c-07e7-3215-c40a-cc8851bd9bcf@oracle.com> <8bae6d8a-7bd3-f576-0f9c-f4b17314a597@oracle.com> <29f9e6e8-5dd4-26ab-b5bc-3c1c83181c24@oracle.com> <94320113-27aa-48e1-f5ab-ac01634e229f@oracle.com> Message-ID: <27cccc7b-d771-db6b-5b81-7f407df7b5f5@oracle.com> Hi Harold, Looks good! One nit in the java test: 53 for (int x=0; x<16; x++) { spaces needed around = and < Thanks. David On 4/05/2017 7:06 AM, harold seigel wrote: > Hi, > > Please review this updated RFR that includes test cases for the JNI > modifications. > > RFR: http://cr.openjdk.java.net/~hseigel/bug_8056900.3/webrev/index.html > > Thanks, Harold > > > On 5/3/2017 3:39 PM, serguei.spitsyn at oracle.com wrote: >> Hi Harold, >> >> The fix looks good to me. >> The test does not cover all the product changes. >> Is it possible to extend it to check the JNI FindClass as well? >> >> Thanks, >> Serguei >> >> >> >> On 5/3/17 07:23, harold seigel wrote: >>> Hi, >>> >>> Please review this updated webrev: >>> http://cr.openjdk.java.net/~hseigel/bug_8056900.2/webrev/index.html >>> >>> It includes the below changes suggested by David. >>> >>> Thanks, Harold >>> >>> >>> On 5/3/2017 12:32 AM, David Holmes wrote: >>>> Hi Harold, >>>> >>>> On 3/05/2017 4:29 AM, harold seigel wrote: >>>>> Hi, >>>>> >>>>> Please review this small JDK-10 change to enhance NoClassDefFoundError >>>>> exception messages. >>>>> >>>>> Open Webrev: >>>>> http://cr.openjdk.java.net/~hseigel/bug_8056900/webrev/index.html >>>> >>>> A few comments: >>>> >>>> In src/share/vm/oops/instanceKlass.cpp >>>> >>>> 517 // This is for CDS dumping phase only -- we use the >>>> in_error_state to indicate that >>>> 518 // the class has failed verification. >>>> >>>> That comment is somewhat misleading. The is_in_error_state() >>>> indicates that class initialization failed, not verification. >>>> However CDS dumping also uses it to indicate if a supertype was >>>> is_in_error_state(). I think using: >>>> >>>> 527 "Verification failed for class %s", >>>> >>>> would be confusing to an end user trying to figure out how a >>>> perfectly valid class could fail verification! I suggest the more >>>> direct: >>>> >>>> "Class %s, or one of its supertypes, failed class initialization" >>>> >>>> --- >>>> >>>> src/share/vm/prims/jni.cpp >>>> >>>> Suggestion: >>>> >>>> "Class name exceeds maximum length of %d: %s", ... >>>> >>>> in both cases. >>>> >>>> 386 THROW_MSG_0(vmSymbols::java_lang_NoClassDefFoundError(), "Class >>>> name is null"); >>>> >>>> Suggestion: "No class name given" >>>> >>>> --- >>>> >>>> Ditto for src/share/vm/prims/jvm.cpp changes. >>>> >>>> --- >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>> >>>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8056900 >>>>> >>>>> The change was tested with JCK tests, the JTreg hotspot, java/io, >>>>> java/lang, java/util and other tests, the RBT tier2 -tier5 tests, the >>>>> co-located NSK tests, JPRT, and with the new test provided with the >>>>> change. >>>>> >>>>> Thanks, Harold >>>>> >>> >> > From harold.seigel at oracle.com Thu May 4 13:31:21 2017 From: harold.seigel at oracle.com (harold seigel) Date: Thu, 4 May 2017 09:31:21 -0400 Subject: RFR 8056900: Enhance NoClassDefFound exception messaging In-Reply-To: <9adcd093-6b93-ee72-304c-1e9717f745b3@oracle.com> References: <9329c480-6e86-d8f6-57f2-23949ee21fcb@oracle.com> <45b7dc8c-07e7-3215-c40a-cc8851bd9bcf@oracle.com> <8bae6d8a-7bd3-f576-0f9c-f4b17314a597@oracle.com> <29f9e6e8-5dd4-26ab-b5bc-3c1c83181c24@oracle.com> <94320113-27aa-48e1-f5ab-ac01634e229f@oracle.com> <9adcd093-6b93-ee72-304c-1e9717f745b3@oracle.com> Message-ID: <11fd5096-6af8-5a90-ee0a-7dc9d316798d@oracle.com> Thanks Coleen! Harold On 5/3/2017 6:08 PM, coleen.phillimore at oracle.com wrote: > Harold, This change looks good. > Coleen > > > On 5/3/17 5:06 PM, harold seigel wrote: >> Hi, >> >> Please review this updated RFR that includes test cases for the JNI >> modifications. >> >> RFR: http://cr.openjdk.java.net/~hseigel/bug_8056900.3/webrev/index.html >> >> Thanks, Harold >> >> >> On 5/3/2017 3:39 PM, serguei.spitsyn at oracle.com wrote: >>> Hi Harold, >>> >>> The fix looks good to me. >>> The test does not cover all the product changes. >>> Is it possible to extend it to check the JNI FindClass as well? >>> >>> Thanks, >>> Serguei >>> >>> >>> >>> On 5/3/17 07:23, harold seigel wrote: >>>> Hi, >>>> >>>> Please review this updated webrev: >>>> http://cr.openjdk.java.net/~hseigel/bug_8056900.2/webrev/index.html >>>> >>>> It includes the below changes suggested by David. >>>> >>>> Thanks, Harold >>>> >>>> >>>> On 5/3/2017 12:32 AM, David Holmes wrote: >>>>> Hi Harold, >>>>> >>>>> On 3/05/2017 4:29 AM, harold seigel wrote: >>>>>> Hi, >>>>>> >>>>>> Please review this small JDK-10 change to enhance >>>>>> NoClassDefFoundError >>>>>> exception messages. >>>>>> >>>>>> Open Webrev: >>>>>> http://cr.openjdk.java.net/~hseigel/bug_8056900/webrev/index.html >>>>> >>>>> A few comments: >>>>> >>>>> In src/share/vm/oops/instanceKlass.cpp >>>>> >>>>> 517 // This is for CDS dumping phase only -- we use the >>>>> in_error_state to indicate that >>>>> 518 // the class has failed verification. >>>>> >>>>> That comment is somewhat misleading. The is_in_error_state() >>>>> indicates that class initialization failed, not verification. >>>>> However CDS dumping also uses it to indicate if a supertype was >>>>> is_in_error_state(). I think using: >>>>> >>>>> 527 "Verification failed for class %s", >>>>> >>>>> would be confusing to an end user trying to figure out how a >>>>> perfectly valid class could fail verification! I suggest the more >>>>> direct: >>>>> >>>>> "Class %s, or one of its supertypes, failed class initialization" >>>>> >>>>> --- >>>>> >>>>> src/share/vm/prims/jni.cpp >>>>> >>>>> Suggestion: >>>>> >>>>> "Class name exceeds maximum length of %d: %s", ... >>>>> >>>>> in both cases. >>>>> >>>>> 386 THROW_MSG_0(vmSymbols::java_lang_NoClassDefFoundError(), >>>>> "Class name is null"); >>>>> >>>>> Suggestion: "No class name given" >>>>> >>>>> --- >>>>> >>>>> Ditto for src/share/vm/prims/jvm.cpp changes. >>>>> >>>>> --- >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>> >>>>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8056900 >>>>>> >>>>>> The change was tested with JCK tests, the JTreg hotspot, java/io, >>>>>> java/lang, java/util and other tests, the RBT tier2 -tier5 tests, >>>>>> the >>>>>> co-located NSK tests, JPRT, and with the new test provided with >>>>>> the change. >>>>>> >>>>>> Thanks, Harold >>>>>> >>>> >>> >> > From harold.seigel at oracle.com Thu May 4 13:32:25 2017 From: harold.seigel at oracle.com (harold seigel) Date: Thu, 4 May 2017 09:32:25 -0400 Subject: RFR 8056900: Enhance NoClassDefFound exception messaging In-Reply-To: <27cccc7b-d771-db6b-5b81-7f407df7b5f5@oracle.com> References: <9329c480-6e86-d8f6-57f2-23949ee21fcb@oracle.com> <45b7dc8c-07e7-3215-c40a-cc8851bd9bcf@oracle.com> <8bae6d8a-7bd3-f576-0f9c-f4b17314a597@oracle.com> <29f9e6e8-5dd4-26ab-b5bc-3c1c83181c24@oracle.com> <94320113-27aa-48e1-f5ab-ac01634e229f@oracle.com> <27cccc7b-d771-db6b-5b81-7f407df7b5f5@oracle.com> Message-ID: Thanks David! I'll add the spaces before pushing the change. Harold On 5/3/2017 10:30 PM, David Holmes wrote: > Hi Harold, > > Looks good! > > One nit in the java test: > > 53 for (int x=0; x<16; x++) { > > spaces needed around = and < > > Thanks. > > David > > On 4/05/2017 7:06 AM, harold seigel wrote: >> Hi, >> >> Please review this updated RFR that includes test cases for the JNI >> modifications. >> >> RFR: http://cr.openjdk.java.net/~hseigel/bug_8056900.3/webrev/index.html >> >> Thanks, Harold >> >> >> On 5/3/2017 3:39 PM, serguei.spitsyn at oracle.com wrote: >>> Hi Harold, >>> >>> The fix looks good to me. >>> The test does not cover all the product changes. >>> Is it possible to extend it to check the JNI FindClass as well? >>> >>> Thanks, >>> Serguei >>> >>> >>> >>> On 5/3/17 07:23, harold seigel wrote: >>>> Hi, >>>> >>>> Please review this updated webrev: >>>> http://cr.openjdk.java.net/~hseigel/bug_8056900.2/webrev/index.html >>>> >>>> It includes the below changes suggested by David. >>>> >>>> Thanks, Harold >>>> >>>> >>>> On 5/3/2017 12:32 AM, David Holmes wrote: >>>>> Hi Harold, >>>>> >>>>> On 3/05/2017 4:29 AM, harold seigel wrote: >>>>>> Hi, >>>>>> >>>>>> Please review this small JDK-10 change to enhance >>>>>> NoClassDefFoundError >>>>>> exception messages. >>>>>> >>>>>> Open Webrev: >>>>>> http://cr.openjdk.java.net/~hseigel/bug_8056900/webrev/index.html >>>>> >>>>> A few comments: >>>>> >>>>> In src/share/vm/oops/instanceKlass.cpp >>>>> >>>>> 517 // This is for CDS dumping phase only -- we use the >>>>> in_error_state to indicate that >>>>> 518 // the class has failed verification. >>>>> >>>>> That comment is somewhat misleading. The is_in_error_state() >>>>> indicates that class initialization failed, not verification. >>>>> However CDS dumping also uses it to indicate if a supertype was >>>>> is_in_error_state(). I think using: >>>>> >>>>> 527 "Verification failed for class %s", >>>>> >>>>> would be confusing to an end user trying to figure out how a >>>>> perfectly valid class could fail verification! I suggest the more >>>>> direct: >>>>> >>>>> "Class %s, or one of its supertypes, failed class initialization" >>>>> >>>>> --- >>>>> >>>>> src/share/vm/prims/jni.cpp >>>>> >>>>> Suggestion: >>>>> >>>>> "Class name exceeds maximum length of %d: %s", ... >>>>> >>>>> in both cases. >>>>> >>>>> 386 THROW_MSG_0(vmSymbols::java_lang_NoClassDefFoundError(), "Class >>>>> name is null"); >>>>> >>>>> Suggestion: "No class name given" >>>>> >>>>> --- >>>>> >>>>> Ditto for src/share/vm/prims/jvm.cpp changes. >>>>> >>>>> --- >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>> >>>>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8056900 >>>>>> >>>>>> The change was tested with JCK tests, the JTreg hotspot, java/io, >>>>>> java/lang, java/util and other tests, the RBT tier2 -tier5 tests, >>>>>> the >>>>>> co-located NSK tests, JPRT, and with the new test provided with the >>>>>> change. >>>>>> >>>>>> Thanks, Harold >>>>>> >>>> >>> >> From harold.seigel at oracle.com Thu May 4 13:33:36 2017 From: harold.seigel at oracle.com (harold seigel) Date: Thu, 4 May 2017 09:33:36 -0400 Subject: RFR 8056900: Enhance NoClassDefFound exception messaging In-Reply-To: <106e5f11-7283-5ebf-f015-9e345548d2da@oracle.com> References: <9329c480-6e86-d8f6-57f2-23949ee21fcb@oracle.com> <45b7dc8c-07e7-3215-c40a-cc8851bd9bcf@oracle.com> <8bae6d8a-7bd3-f576-0f9c-f4b17314a597@oracle.com> <29f9e6e8-5dd4-26ab-b5bc-3c1c83181c24@oracle.com> <94320113-27aa-48e1-f5ab-ac01634e229f@oracle.com> <106e5f11-7283-5ebf-f015-9e345548d2da@oracle.com> Message-ID: <1d53aaba-a2c8-03fe-e9e2-38f604bd32d2@oracle.com> Thanks Serguei! Since it's just a test I'll leave it as is. Harold On 5/3/2017 5:24 PM, serguei.spitsyn at oracle.com wrote: > Hi Harold, > > Thank you for the update. > It looks great! > > One tip is that you can pre-define the static String constants: > "Class name exceeds maximum length of " > "defineClass did not throw expected NoClassDefFoundError". > > > But I leave it up to you. > No webrev is needed if you decide to tweak it. > > Thanks, > Serguei > > > On 5/3/17 14:06, harold seigel wrote: >> Hi, >> >> Please review this updated RFR that includes test cases for the JNI >> modifications. >> >> RFR: http://cr.openjdk.java.net/~hseigel/bug_8056900.3/webrev/index.html >> >> Thanks, Harold >> >> >> On 5/3/2017 3:39 PM, serguei.spitsyn at oracle.com wrote: >>> Hi Harold, >>> >>> The fix looks good to me. >>> The test does not cover all the product changes. >>> Is it possible to extend it to check the JNI FindClass as well? >>> >>> Thanks, >>> Serguei >>> >>> >>> >>> On 5/3/17 07:23, harold seigel wrote: >>>> Hi, >>>> >>>> Please review this updated webrev: >>>> http://cr.openjdk.java.net/~hseigel/bug_8056900.2/webrev/index.html >>>> >>>> It includes the below changes suggested by David. >>>> >>>> Thanks, Harold >>>> >>>> >>>> On 5/3/2017 12:32 AM, David Holmes wrote: >>>>> Hi Harold, >>>>> >>>>> On 3/05/2017 4:29 AM, harold seigel wrote: >>>>>> Hi, >>>>>> >>>>>> Please review this small JDK-10 change to enhance >>>>>> NoClassDefFoundError >>>>>> exception messages. >>>>>> >>>>>> Open Webrev: >>>>>> http://cr.openjdk.java.net/~hseigel/bug_8056900/webrev/index.html >>>>> >>>>> A few comments: >>>>> >>>>> In src/share/vm/oops/instanceKlass.cpp >>>>> >>>>> 517 // This is for CDS dumping phase only -- we use the >>>>> in_error_state to indicate that >>>>> 518 // the class has failed verification. >>>>> >>>>> That comment is somewhat misleading. The is_in_error_state() >>>>> indicates that class initialization failed, not verification. >>>>> However CDS dumping also uses it to indicate if a supertype was >>>>> is_in_error_state(). I think using: >>>>> >>>>> 527 "Verification failed for class %s", >>>>> >>>>> would be confusing to an end user trying to figure out how a >>>>> perfectly valid class could fail verification! I suggest the more >>>>> direct: >>>>> >>>>> "Class %s, or one of its supertypes, failed class initialization" >>>>> >>>>> --- >>>>> >>>>> src/share/vm/prims/jni.cpp >>>>> >>>>> Suggestion: >>>>> >>>>> "Class name exceeds maximum length of %d: %s", ... >>>>> >>>>> in both cases. >>>>> >>>>> 386 THROW_MSG_0(vmSymbols::java_lang_NoClassDefFoundError(), >>>>> "Class name is null"); >>>>> >>>>> Suggestion: "No class name given" >>>>> >>>>> --- >>>>> >>>>> Ditto for src/share/vm/prims/jvm.cpp changes. >>>>> >>>>> --- >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>> >>>>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8056900 >>>>>> >>>>>> The change was tested with JCK tests, the JTreg hotspot, java/io, >>>>>> java/lang, java/util and other tests, the RBT tier2 -tier5 tests, >>>>>> the >>>>>> co-located NSK tests, JPRT, and with the new test provided with >>>>>> the change. >>>>>> >>>>>> Thanks, Harold >>>>>> >>>> >>> >> > From ioi.lam at oracle.com Thu May 4 13:40:23 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 04 May 2017 06:40:23 -0700 Subject: RFR - JDK-8179625 [BACKOUT] of 8179305 Avoid repeated calls to JavaThread::last_frame in InterpreterRuntime Message-ID: <590B2F47.7080301@oracle.com> Hi, It turns out JDK- 8179305 causes intermittent failures in JPRT, so it needs to be backed out quickly: https://bugs.openjdk.java.net/browse/JDK-8179625 http://cr.openjdk.java.net/~iklam/jdk10/8179625-backout-8179305-last-frame.v01/ Thanks - Ioi From ioi.lam at oracle.com Thu May 4 13:46:24 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 04 May 2017 06:46:24 -0700 Subject: RFR [XS] JDK-8176863 Remove Hashtable::reverse() In-Reply-To: References: <590A034E.8000809@oracle.com> Message-ID: <590B30B0.4090704@oracle.com> Thanks Serguei, Coleen & Harold for the review! - Ioi On 5/3/17 12:18 PM, serguei.spitsyn at oracle.com wrote: > Hi Ioi, > > Looks good. > > Thanks, > Serguei > > > On 5/3/17 09:20, Ioi Lam wrote: >> Hi, >> >> Please review this small change to remove an obsolete optimization: >> >> https://bugs.openjdk.java.net/browse/JDK-8176863 >> http://cr.openjdk.java.net/~iklam/jdk10/8176863_remove_hashtable_reverse.v01/ >> >> >> This is legacy code from JDK 1.5 that doesn't do what it intends to >> do and is no longer relevant. >> >> Thanks >> - Ioi > From harold.seigel at oracle.com Thu May 4 13:50:55 2017 From: harold.seigel at oracle.com (harold seigel) Date: Thu, 4 May 2017 09:50:55 -0400 Subject: RFR - JDK-8179625 [BACKOUT] of 8179305 Avoid repeated calls to JavaThread::last_frame in InterpreterRuntime In-Reply-To: <590B2F47.7080301@oracle.com> References: <590B2F47.7080301@oracle.com> Message-ID: <5c2f65c8-382a-361d-3de3-9a2a7431e531@oracle.com> Hi Ioi, This looks good. Thanks, Harold On 5/4/2017 9:40 AM, Ioi Lam wrote: > Hi, > > It turns out JDK- 8179305 causes intermittent failures in JPRT, so it > needs to be backed out quickly: > > https://bugs.openjdk.java.net/browse/JDK-8179625 > http://cr.openjdk.java.net/~iklam/jdk10/8179625-backout-8179305-last-frame.v01/ > > > Thanks > - Ioi From ioi.lam at oracle.com Thu May 4 13:55:18 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 04 May 2017 06:55:18 -0700 Subject: RFR - JDK-8179625 [BACKOUT] of 8179305 Avoid repeated calls to JavaThread::last_frame in InterpreterRuntime In-Reply-To: <5c2f65c8-382a-361d-3de3-9a2a7431e531@oracle.com> References: <590B2F47.7080301@oracle.com> <5c2f65c8-382a-361d-3de3-9a2a7431e531@oracle.com> Message-ID: <590B32C6.2070401@oracle.com> Thanks Harold. I'll push now. - Ioi On 5/4/17 6:50 AM, harold seigel wrote: > Hi Ioi, > > This looks good. > > Thanks, Harold > > > On 5/4/2017 9:40 AM, Ioi Lam wrote: >> Hi, >> >> It turns out JDK- 8179305 causes intermittent failures in JPRT, so it >> needs to be backed out quickly: >> >> https://bugs.openjdk.java.net/browse/JDK-8179625 >> http://cr.openjdk.java.net/~iklam/jdk10/8179625-backout-8179305-last-frame.v01/ >> >> >> Thanks >> - Ioi > From rachel.protacio at oracle.com Thu May 4 15:52:34 2017 From: rachel.protacio at oracle.com (Rachel Protacio) Date: Thu, 4 May 2017 11:52:34 -0400 Subject: RFR: 8154791: Xlog classload too redundant msgs info/debug In-Reply-To: References: <6ff9f069-2da7-b725-d29f-e3d850650aca@oracle.com> <5908D481.2070104@oracle.com> <6a8ffd20-ce58-150b-a94d-c51c5956d191@oracle.com> <5908F288.9020205@oracle.com> <5908F5D4.6040505@oracle.com> Message-ID: <8ef74c07-1a6e-eede-d16c-4ba283a96199@oracle.com> If I'm using this incorrectly, please let me know, but I think LogBufferMessage also doesn't do exactly what we want. With this changeset: http://cr.openjdk.java.net/~rprotacio/8154791.01/ we get this output: [0.050s][info ][class,load] java.lang.CharSequence [0.050s][info ][class,load] source: jrt:/java.base [0.050s][info ][class,load] loader: [AllocatedObj(0x00000007c00015a0)] [0.050s][debug][class,load] klass: 0x00000007c00015a0 [0.050s][debug][class,load] super: 0x00000007c0000fb0 [0.050s][info ][class,load] bytes: 2045 checksum: 5b97c8c1 [0.051s][info ][class,load] java.lang.String [0.051s][info ][class,load] source: jrt:/java.base [0.051s][info ][class,load] loader: [AllocatedObj(0x00000007c0001798)] [0.051s][debug][class,load] klass: 0x00000007c0001798 [0.051s][debug][class,load] super: 0x00000007c0000fb0 [0.051s][debug][class,load] interfaces: [0.051s][debug][class,load] 0x00000007c00011b0 [0.051s][debug][class,load] 0x00000007c00013a8 [0.051s][debug][class,load] 0x00000007c00015a0 [0.051s][info ][class,load] bytes: 24872 checksum: 564a9fc2 It comes out "together", but each call is a separate line. But to put it all on one line would lead to messy code, something like adding each part to two temporary strings and putting them in the buffer at the end, I think. So the other option is to do this: http://cr.openjdk.java.net/~rprotacio/8154791.02 which is essentially the original code, but slightly cleaner and without redundancy of the info line inside the debug line. The only issue is it doesn't necessarily guarantee the two lines will come out next to each other, so we have to print the external name on each. [0.051s][info ][class,load] java.lang.CharSequence source: jrt:/java.basejava.lang.CharSequence loader: [NULL class_loader] bytes: 2045 checksum: 5b97c8c1 [0.051s][debug][class,load] java.lang.CharSequence klass: 0x00000007c00015a0 super: 0x00000007c0000fb0 [0.052s][info ][class,load] java.lang.String source: jrt:/java.basejava.lang.String loader: [NULL class_loader] bytes: 24872 checksum: 564a9fc2 [0.052s][debug][class,load] java.lang.String klass: 0x00000007c0001798 super: 0x00000007c0000fb0 interfaces: 0x00000007c00011b0 0x00000007c00013a8 0x00000007c00015a0 Or we can make the info output the same as debug, when debug is specified. Which I have reservations about but will certainly do if that's what makes the most sense. Thanks, Rachel On 5/3/2017 2:23 AM, Thomas St?fe wrote: > Hi Ioi, > > On Tue, May 2, 2017 at 11:10 PM, Ioi Lam wrote: > >> By the way, I think we should really fix this in the UL level -- have >> multi-line buffering, so you can prevent other threads from interleaving >> your input. Make it something like: >> >> Log(class, load).start_buffering(); >> log_info(class, load)(class name and source); >> log_debug(class, load)(other details); >> Log(class, load).flush(); >> >> That way, the output will look like this: >> >> [0.162s][debug ][class,load] java.lang.Comparable source: jrt:/java.base >> [0.162s][debug ][class,load] [NULL class_loader] 0x00000007c00013a8 super: >> 0x00000007c0000fb0 bytes: 235 checksum: a75dadb6 >> >> So the output will be terse, and still parsable. >> >> I think this will be useful for other types of logs as well, not just the >> class loading logs. >> >> > I thought that already exists: > > https://bugs.openjdk.java.net/browse/JDK-8145934 > > see logMessage.hpp. > > Kind Regards, Thomas > > >> Thanks >> - Ioi >> >> >> On 5/2/17 1:56 PM, Ioi Lam wrote: >> >>> >>> On 5/2/17 1:31 PM, Rachel Protacio wrote: >>> >>>> Hi Ioi, >>>> >>>> Thanks for your reply, comments inline. >>>> >>>> >>>> On 5/2/2017 2:48 PM, Ioi Lam wrote: >>>> >>>>> Hi Rachel, >>>>> >>>>> There are a few reasons why the current output has duplications. The >>>>> "logging levels are not mutually exclusive" comment means this: >>>>> >>>>> java -Xlog:class+load=debug:file=debug.log >>>>> -Xlog:class+load=info:file=info.log >>>>> >>>>> So info.log contains only the info level logs, but debug.log contains >>>>> both info and debug level logs. >>>>> >>>>> Ideally I want info.log to contain this: >>>>> >>>>> [0.162s][info ][class,load] java.lang.Comparable source: jrt:/java.base >>>>> >>>>> .. and debug.log to contain this: >>>>> >>>>> [0.162s][debug ][class,load] java.lang.Comparable source: >>>>> jrt:/java.base [NULL class_loader] 0x00000007c00013a8 super: >>>>> 0x00000007c0000fb0 bytes: 235 >>>>> checksum: a75dadb6 >>>>> >>>> I see what you want, but I think the problem is that logging doesn't >>>> work that way. If debug is specified, both debug and info levels are >>>> printed. So the command >>>> >>>> java -Xlog:class+load=debug:file=debug.log >>>> >>>> will print both info level logging and debug level logging to this >>>> "debug.log" file. And then tacking on >>>> >>>> -Xlog:class+load=info:file=info.log >>>> >>>> in the command would print just the info level logging to the "info.log" >>>> file. But since it's all the same java command, it will always know that >>>> both debug and info levels are specified. I believe what you want me to do >>>> is say "if debug is specified, print everything to debug only" but then the >>>> info.log would be empty. You would have to run a separate java command with >>>> just the info level specified to get the smaller amount of information. >>>> >>> No, that's not what I want. I am perfectly fine with how UL works today: >>> info.log contains only the info level, and debug.info contains both >>> debug and info level. >>> >>> All I want is everything related to the same class to be printed on the >>> same line. For example, I don't want debug.log to contain this: >>> >>> [0.162s][info ][class,load] java.lang.Comparable source: jrt:/java.base >>> [0.162s][debug ][class,load] [NULL class_loader] 0x00000007c00013a8 >>> super: 0x00000007c0000fb0 bytes: 235 checksum: a75dadb6 >>> >>> That's because it's impossible to associated the second with the first >>> line when you have parallel threads loading classes concurrently. >>> >>> But I think that would violate user expectations - if they get certain >>>> information with info logging, adding the extra level of debug logging >>>> should not change what they see from the info logging. >>>> >>>>> Your current patch splits out the debug log into several lines. This >>>>> makes it difficult to analyze the log when classes are loaded concurrently >>>>> on different threads: >>>>> >>>>> [0.162s][info ][class,load] java.lang.Comparable source: jrt:/java.base >>>>> [0.162s][info ][class,load] java.lang.Comparable loader: [NULL >>>>> class_loader] >>>>> [0.162s][debug][class,load] java.lang.Comparable klass: >>>>> 0x00000007c00013a8 >>>>> [0.163s][info ][class,load] java.lang.String source: jrt:/java.base <<< >>>>> oops! from a different thread >>>>> [0.164s][debug][class,load] java.lang.Comparable super: >>>>> 0x00000007c0000fb0 >>>>> [0.164s][info ][class,load] java.lang.Comparable bytes: 235 >>>>> [0.164s][info ][class,load] java.lang.Comparable checksum: a75dadb6 >>>>> >>>> That's fair. I was trying to make it more readable since it's hard to >>>> visually parse one giant line. Maybe I could put some delimiter in between >>>> each piece of info? But if no one else cares, I can just put it back as one >>>> line, no delimiters. >>>> >>>>> Because the leveling in UL is by lines, in JDK-8079408, where this was >>>>> first implemented, I couldn't find a way to print "java.lang.Comparable >>>>> source: jrt:/java.base" in "info" mode, and print the rest in "debug" mode, >>>>> while keeping the whole thing on the same line (in both the info.log and >>>>> debug.log files). That's why we have the 2 lines with duplicated info. >>>>> >>>>> I think it's really important to keep everything on the same line (the >>>>> output needs to be processed automatically by scripts). With the current UL >>>>> restrictions, I think the only way to do it is make the "info" logging more >>>>> verbose when "debug" level is selected. E.g., >>>>> >>>>> outputStream* info_log = Log(class, load)::info_stream(); >>>>> info_log->print("%s", ext_name); >>>>> info_log->print_cr(" source: jrt:/%s", module_name); .... >>>>> >>>>> if (log_is_enabled(Debug, class, load)) { // always print to info >>>>> log, even if debug is selected >>>>> info_log->print(" klass: " INTPTR_FORMAT, p2i(this)); >>>>> .... >>>>> } >>>>> >>>>> So now info.log and debug.log will have the same logs, but oh well ..... >>>>> >>>> This sounds a little different. It sounds like >>>> >>>> When only info is specified, info log will contain: external name, >>>> source, loader, bytes, checksum >>>> >>>> When debug is specified, info log will contain the same as above, >>>> plus: klass, super, interfaces >>>> When debug is specified, debug log will be an exact duplicate of >>>> info log so it can print to an external file >>>> >>>> Did I understand that right? Please correct me. If this is what you >>>> meant, I would again argue that this violates the expectation that "info is >>>> info is info" regardless of what else is specified (not a written rule, but >>>> how I think it should work). And it is redundant, which is the original >>>> issue in the name of this RFE. >>>> >>>> I think the best bet would be >>>> >>>> Info log: external name, source, loader, bytes, checksum >>>> Debug log: external name, klass, super, interfaces >>>> >>>> Or we get rid of the level distinction completely and put all the >>>> information together in one line so it never gets broken up. Probably under >>>> debug. >>>> >>>> >>> We need to keep the current debug output intact, on a single line, so >>> that the information can be processed by scripts. >>> >>> So, it seems like our 3 choices are: >>> >>> (1) make the info output the same as debug, if -Xlog:class+load=debug is >>> specified >>> (2) remove the info output altogether >>> (3) do nothing >>> >>> You don't like (1). >>> >>> I don't think (2) achieves what this RFE wants -- namely, make the output >>> less redundant, because you will see all the details all the time, and you >>> don't even get a choice for more terse output. It also makes the output of >>> -verbose different (and much less useful) than in JDK 9. >>> >>> So I would vote for (3), keep the output as is. Yeah, with debug level >>> you have a lot of output, but that's pretty easy to filter. I think we had >>> a discussion about this when 8079408 was implemented, and that was the >>> conclusion we had. >>> >>> Thanks >>> - Ioi >>> >>> Let me know what you think. >>>> Thanks, >>>> Rachel >>>> >>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> On 5/2/17 10:32 AM, Rachel Protacio wrote: >>>>> >>>>>> Hello! >>>>>> >>>>>> Please review this enhancement correcting redundancies and neatening >>>>>> up the -Xlog:class+load code. The redundancy in the code before I believe >>>>>> stemmed from a misunderstanding about logging levels (the comment saying >>>>>> they are not mutually exclusive was misleading). >>>>>> >>>>>> Tested with JPRT and RBT. >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8154791 >>>>>> Webrev: http://cr.openjdk.java.net/~rprotacio/8154791.00 >>>>>> >>>>>> Thanks, >>>>>> Rachel >>>>>> >>>>> From dean.long at oracle.com Thu May 4 16:53:17 2017 From: dean.long at oracle.com (dean.long at oracle.com) Date: Thu, 4 May 2017 09:53:17 -0700 Subject: RFR - JDK-8179625 [BACKOUT] of 8179305 Avoid repeated calls to JavaThread::last_frame in InterpreterRuntime In-Reply-To: <590B2F47.7080301@oracle.com> References: <590B2F47.7080301@oracle.com> Message-ID: <40f2b23b-731a-cb1d-42a0-3137a1fb828a@oracle.com> Sorry for not noticing before, but this looks like it could be the cause of the crash: - BasicObjectLock* monitor_begin() const { - return _last_frame.interpreter_frame_monitor_end(); - } - BasicObjectLock* monitor_end() const { - return _last_frame.interpreter_frame_monitor_begin(); - } _begin and _end are reversed. dl On 5/4/17 6:40 AM, Ioi Lam wrote: > Hi, > > It turns out JDK- 8179305 causes intermittent failures in JPRT, so it > needs to be backed out quickly: > > https://bugs.openjdk.java.net/browse/JDK-8179625 > http://cr.openjdk.java.net/~iklam/jdk10/8179625-backout-8179305-last-frame.v01/ > > > Thanks > - Ioi From harold.seigel at oracle.com Thu May 4 17:47:40 2017 From: harold.seigel at oracle.com (harold seigel) Date: Thu, 4 May 2017 13:47:40 -0400 Subject: RFR 8178604: JVM does not allow defining boot loader modules in exploded build after module system initialization In-Reply-To: <11709d4a-0a41-1d38-fcd1-67d42c33eaeb@oracle.com> References: <85e36988-a145-6805-8659-56c738013f58@oracle.com> <46a812ca-228d-01e7-e889-fd22f2af0537@oracle.com> <11709d4a-0a41-1d38-fcd1-67d42c33eaeb@oracle.com> Message-ID: <3ee613ff-f6da-9ef8-6781-efce4edba29e@oracle.com> Hi, Please review this updated webrev: http://cr.openjdk.java.net/~hseigel/bug_8178604.2/webrev/index.html The updated webrev takes out a lock when creating _exploded_entries. It also contains additional comments and an assert() to address Lois's concerns with reduced code readability involving what situations warrant taking out a lock in ClassLoader::search_module_entries(). I don't think there needs to be two different initial entry points as suggested by David below. The method behaves slightly differently depending on the value of its 'need_lock' parameter. But, there is enough common code for it to remain one method. Thanks, Harold On 5/3/2017 12:59 AM, David Holmes wrote: > On 3/05/2017 7:11 AM, Lois Foltan wrote: >> On 5/2/2017 9:03 AM, harold seigel wrote: >>> Hi Lois, >>> >>> Thanks for your comments. Please see my in-line responses. >>> >>> Harold >>> >>> >>> On 5/1/2017 4:14 PM, Lois Foltan wrote: >>>> Hi Harold, >>>> >>>> Looks good. A couple of comments: >>>> >>>> src/share/vm/classfile/classLoader.cpp >>>> line #828 - please take out the Module_lock when the >>>> _exploded_entries is created >>> This fix does not change the possible need for synchronization when >>> _exploded_entries gets created during module system initialization. >>> Are you saying that the existing code is wrong and should have taken >>> out the Module_lock before creating _exploding_entries? >> >> The method ClassLoader::add_to_exploded_build_list can be called at >> anytime (during module system initialization and post module system >> initialization). Thus it seems prudent that no matter what the current >> JVM phase, that any access to ClassLoader::_exploding_entries be >> protected via a lock. That includes creation as well as insertion into >> the list. > > If creation is not guaranteed to occur before other threads that will > call the same method and use the _exploded_entries list, then > synchronization of some form is essential. If it is guaranteed then we > don't need create time sync just to be prudent - but the guarantee > must be firmly established and clearly documented. > >>> >>>> line #1402 - I like the new factored out method >>>> find_first_module_cpe(), however, instead of adding a new "need_lock" >>>> parameter, I would rather see code >>>> in find_first_module_cpe() that takes the >>>> Module_lock if the module_list parameter equals >>>> ClassLoader::_exploded_entries >>> I think your suggestion makes the code less flexible and does not >>> require potential future callers of search_module_entries() to think >>> about synchronization. So, I'd prefer to leave that change as is. >> >> I see your point. My point was based on reduced code readability, the >> further away from the decision to establish the lock, the less apparent >> it is within the new method ClassLoader::find_first_module_cpe() as to >> what situations warranted the lock in the first place. > > Further, this: > > 1495 stream = search_module_entries(_exploded_entries, > class_name, file_name, true, CHECK_NULL); > > is potentially unsafe if we can race with creation because we first > access _exploded_entries without holding any lock! And the fact the > method needs to do different things depending on which "list" was > passed in suggests to me it should be two different initial entry points. > > Thanks, > David > ----- > >> Thanks, >> Lois >> >>>> >>>> Thanks, >>>> Lois >>>> >>>> On 5/1/2017 3:36 PM, harold seigel wrote: >>>>> Hi, >>>>> >>>>> Please review this JDK-10 bug fix to allow defining of modules to >>>>> the boot loader in exploded builds after module system >>>>> initialization. The fix uses the module lock to synchronize access >>>>> to the _exploded_entries data structure (which is only used by >>>>> exploded builds). >>>>> >>>>> Note that the above capability already exists for regular builds. >>>>> >>>>> Open Webrev: >>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604/webrev/index.html >>>>> >>>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8178604 >>>>> >>>>> The fix was tested with JCK tests, the JTreg hotspot, java/io, >>>>> java/lang, java/util and other tests, the RBT tier2 -tier5 tests, >>>>> the co-located NSK tests, and with JPRT. The JTReg and JCK tests >>>>> were run with an exploded build. Also, the example program in the >>>>> JBS bug was run by hand to verify the fix. >>>>> >>>>> Thanks, Harold >>>>> >>>> >>> >> From lois.foltan at oracle.com Thu May 4 18:05:58 2017 From: lois.foltan at oracle.com (Lois Foltan) Date: Thu, 4 May 2017 14:05:58 -0400 Subject: RFR 8178604: JVM does not allow defining boot loader modules in exploded build after module system initialization In-Reply-To: <3ee613ff-f6da-9ef8-6781-efce4edba29e@oracle.com> References: <85e36988-a145-6805-8659-56c738013f58@oracle.com> <46a812ca-228d-01e7-e889-fd22f2af0537@oracle.com> <11709d4a-0a41-1d38-fcd1-67d42c33eaeb@oracle.com> <3ee613ff-f6da-9ef8-6781-efce4edba29e@oracle.com> Message-ID: <6a8f5496-3eed-dbd6-3d94-a5ec2cc2c679@oracle.com> Thank you, looks good. Lois On 5/4/2017 1:47 PM, harold seigel wrote: > Hi, > > Please review this updated webrev: > > http://cr.openjdk.java.net/~hseigel/bug_8178604.2/webrev/index.html > > The updated webrev takes out a lock when creating _exploded_entries. > It also contains additional comments and an assert() to address Lois's > concerns with reduced code readability involving what situations > warrant taking out a lock in ClassLoader::search_module_entries(). > > I don't think there needs to be two different initial entry points as > suggested by David below. The method behaves slightly differently > depending on the value of its 'need_lock' parameter. But, there is > enough common code for it to remain one method. > > Thanks, Harold > > On 5/3/2017 12:59 AM, David Holmes wrote: >> On 3/05/2017 7:11 AM, Lois Foltan wrote: >>> On 5/2/2017 9:03 AM, harold seigel wrote: >>>> Hi Lois, >>>> >>>> Thanks for your comments. Please see my in-line responses. >>>> >>>> Harold >>>> >>>> >>>> On 5/1/2017 4:14 PM, Lois Foltan wrote: >>>>> Hi Harold, >>>>> >>>>> Looks good. A couple of comments: >>>>> >>>>> src/share/vm/classfile/classLoader.cpp >>>>> line #828 - please take out the Module_lock when the >>>>> _exploded_entries is created >>>> This fix does not change the possible need for synchronization when >>>> _exploded_entries gets created during module system initialization. >>>> Are you saying that the existing code is wrong and should have taken >>>> out the Module_lock before creating _exploding_entries? >>> >>> The method ClassLoader::add_to_exploded_build_list can be called at >>> anytime (during module system initialization and post module system >>> initialization). Thus it seems prudent that no matter what the current >>> JVM phase, that any access to ClassLoader::_exploding_entries be >>> protected via a lock. That includes creation as well as insertion into >>> the list. >> >> If creation is not guaranteed to occur before other threads that will >> call the same method and use the _exploded_entries list, then >> synchronization of some form is essential. If it is guaranteed then >> we don't need create time sync just to be prudent - but the guarantee >> must be firmly established and clearly documented. >> >>>> >>>>> line #1402 - I like the new factored out method >>>>> find_first_module_cpe(), however, instead of adding a new "need_lock" >>>>> parameter, I would rather see code >>>>> in find_first_module_cpe() that takes the >>>>> Module_lock if the module_list parameter equals >>>>> ClassLoader::_exploded_entries >>>> I think your suggestion makes the code less flexible and does not >>>> require potential future callers of search_module_entries() to think >>>> about synchronization. So, I'd prefer to leave that change as is. >>> >>> I see your point. My point was based on reduced code readability, the >>> further away from the decision to establish the lock, the less apparent >>> it is within the new method ClassLoader::find_first_module_cpe() as to >>> what situations warranted the lock in the first place. >> >> Further, this: >> >> 1495 stream = search_module_entries(_exploded_entries, >> class_name, file_name, true, CHECK_NULL); >> >> is potentially unsafe if we can race with creation because we first >> access _exploded_entries without holding any lock! And the fact the >> method needs to do different things depending on which "list" was >> passed in suggests to me it should be two different initial entry >> points. >> >> Thanks, >> David >> ----- >> >>> Thanks, >>> Lois >>> >>>>> >>>>> Thanks, >>>>> Lois >>>>> >>>>> On 5/1/2017 3:36 PM, harold seigel wrote: >>>>>> Hi, >>>>>> >>>>>> Please review this JDK-10 bug fix to allow defining of modules to >>>>>> the boot loader in exploded builds after module system >>>>>> initialization. The fix uses the module lock to synchronize access >>>>>> to the _exploded_entries data structure (which is only used by >>>>>> exploded builds). >>>>>> >>>>>> Note that the above capability already exists for regular builds. >>>>>> >>>>>> Open Webrev: >>>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604/webrev/index.html >>>>>> >>>>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8178604 >>>>>> >>>>>> The fix was tested with JCK tests, the JTreg hotspot, java/io, >>>>>> java/lang, java/util and other tests, the RBT tier2 -tier5 tests, >>>>>> the co-located NSK tests, and with JPRT. The JTReg and JCK tests >>>>>> were run with an exploded build. Also, the example program in the >>>>>> JBS bug was run by hand to verify the fix. >>>>>> >>>>>> Thanks, Harold >>>>>> >>>>> >>>> >>> > From harold.seigel at oracle.com Thu May 4 18:14:32 2017 From: harold.seigel at oracle.com (harold seigel) Date: Thu, 4 May 2017 14:14:32 -0400 Subject: RFR 8178604: JVM does not allow defining boot loader modules in exploded build after module system initialization In-Reply-To: <6a8f5496-3eed-dbd6-3d94-a5ec2cc2c679@oracle.com> References: <85e36988-a145-6805-8659-56c738013f58@oracle.com> <46a812ca-228d-01e7-e889-fd22f2af0537@oracle.com> <11709d4a-0a41-1d38-fcd1-67d42c33eaeb@oracle.com> <3ee613ff-f6da-9ef8-6781-efce4edba29e@oracle.com> <6a8f5496-3eed-dbd6-3d94-a5ec2cc2c679@oracle.com> Message-ID: <03343edb-dd76-bf8b-24d7-a79b96f347d3@oracle.com> Thanks Lois! Harold On 5/4/2017 2:05 PM, Lois Foltan wrote: > Thank you, looks good. > Lois > > On 5/4/2017 1:47 PM, harold seigel wrote: > >> Hi, >> >> Please review this updated webrev: >> >> http://cr.openjdk.java.net/~hseigel/bug_8178604.2/webrev/index.html >> >> The updated webrev takes out a lock when creating _exploded_entries. >> It also contains additional comments and an assert() to address >> Lois's concerns with reduced code readability involving what >> situations warrant taking out a lock in >> ClassLoader::search_module_entries(). >> >> I don't think there needs to be two different initial entry points as >> suggested by David below. The method behaves slightly differently >> depending on the value of its 'need_lock' parameter. But, there is >> enough common code for it to remain one method. >> >> Thanks, Harold >> >> On 5/3/2017 12:59 AM, David Holmes wrote: >>> On 3/05/2017 7:11 AM, Lois Foltan wrote: >>>> On 5/2/2017 9:03 AM, harold seigel wrote: >>>>> Hi Lois, >>>>> >>>>> Thanks for your comments. Please see my in-line responses. >>>>> >>>>> Harold >>>>> >>>>> >>>>> On 5/1/2017 4:14 PM, Lois Foltan wrote: >>>>>> Hi Harold, >>>>>> >>>>>> Looks good. A couple of comments: >>>>>> >>>>>> src/share/vm/classfile/classLoader.cpp >>>>>> line #828 - please take out the Module_lock when the >>>>>> _exploded_entries is created >>>>> This fix does not change the possible need for synchronization when >>>>> _exploded_entries gets created during module system initialization. >>>>> Are you saying that the existing code is wrong and should have taken >>>>> out the Module_lock before creating _exploding_entries? >>>> >>>> The method ClassLoader::add_to_exploded_build_list can be called at >>>> anytime (during module system initialization and post module system >>>> initialization). Thus it seems prudent that no matter what the >>>> current >>>> JVM phase, that any access to ClassLoader::_exploding_entries be >>>> protected via a lock. That includes creation as well as insertion >>>> into >>>> the list. >>> >>> If creation is not guaranteed to occur before other threads that >>> will call the same method and use the _exploded_entries list, then >>> synchronization of some form is essential. If it is guaranteed then >>> we don't need create time sync just to be prudent - but the >>> guarantee must be firmly established and clearly documented. >>> >>>>> >>>>>> line #1402 - I like the new factored out method >>>>>> find_first_module_cpe(), however, instead of adding a new >>>>>> "need_lock" >>>>>> parameter, I would rather see code >>>>>> in find_first_module_cpe() that takes the >>>>>> Module_lock if the module_list parameter equals >>>>>> ClassLoader::_exploded_entries >>>>> I think your suggestion makes the code less flexible and does not >>>>> require potential future callers of search_module_entries() to think >>>>> about synchronization. So, I'd prefer to leave that change as is. >>>> >>>> I see your point. My point was based on reduced code readability, the >>>> further away from the decision to establish the lock, the less >>>> apparent >>>> it is within the new method ClassLoader::find_first_module_cpe() as to >>>> what situations warranted the lock in the first place. >>> >>> Further, this: >>> >>> 1495 stream = search_module_entries(_exploded_entries, >>> class_name, file_name, true, CHECK_NULL); >>> >>> is potentially unsafe if we can race with creation because we first >>> access _exploded_entries without holding any lock! And the fact the >>> method needs to do different things depending on which "list" was >>> passed in suggests to me it should be two different initial entry >>> points. >>> >>> Thanks, >>> David >>> ----- >>> >>>> Thanks, >>>> Lois >>>> >>>>>> >>>>>> Thanks, >>>>>> Lois >>>>>> >>>>>> On 5/1/2017 3:36 PM, harold seigel wrote: >>>>>>> Hi, >>>>>>> >>>>>>> Please review this JDK-10 bug fix to allow defining of modules to >>>>>>> the boot loader in exploded builds after module system >>>>>>> initialization. The fix uses the module lock to synchronize access >>>>>>> to the _exploded_entries data structure (which is only used by >>>>>>> exploded builds). >>>>>>> >>>>>>> Note that the above capability already exists for regular builds. >>>>>>> >>>>>>> Open Webrev: >>>>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604/webrev/index.html >>>>>>> >>>>>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8178604 >>>>>>> >>>>>>> The fix was tested with JCK tests, the JTreg hotspot, java/io, >>>>>>> java/lang, java/util and other tests, the RBT tier2 -tier5 tests, >>>>>>> the co-located NSK tests, and with JPRT. The JTReg and JCK tests >>>>>>> were run with an exploded build. Also, the example program in the >>>>>>> JBS bug was run by hand to verify the fix. >>>>>>> >>>>>>> Thanks, Harold >>>>>>> >>>>>> >>>>> >>>> >> > From rachel.protacio at oracle.com Thu May 4 18:35:03 2017 From: rachel.protacio at oracle.com (Rachel Protacio) Date: Thu, 4 May 2017 14:35:03 -0400 Subject: RFR (XS): 8067728: Flag::unlock_diagnostic() should be called Flag::clear_diagnostic() In-Reply-To: References: Message-ID: Thanks for the reviews, Ioi, Harold, and David. Will check in. And yes, there are a number of tests that use this code. But I'll leave the assert as well. Rachel On 5/3/2017 12:11 AM, David Holmes wrote: > Hi Rachel, > > Looks good. > > Do we have any tests that clear the diagnostic bit? That would be > nicer for checking the logic instead of adding the assert Ioi suggested. > > Thanks, > David > > On 3/05/2017 4:15 AM, Rachel Protacio wrote: >> Hi, >> >> Please review this tiny change, renaming Flag::unlock_diagnostic() as >> Flag::clear_diagnostic. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8067728 >> Open webrev: http://cr.openjdk.java.net/~rprotacio/8067728.00/ >> >> Thanks! >> Rachel From david.holmes at oracle.com Thu May 4 21:32:32 2017 From: david.holmes at oracle.com (David Holmes) Date: Fri, 5 May 2017 07:32:32 +1000 Subject: RFR 8178604: JVM does not allow defining boot loader modules in exploded build after module system initialization In-Reply-To: <3ee613ff-f6da-9ef8-6781-efce4edba29e@oracle.com> References: <85e36988-a145-6805-8659-56c738013f58@oracle.com> <46a812ca-228d-01e7-e889-fd22f2af0537@oracle.com> <11709d4a-0a41-1d38-fcd1-67d42c33eaeb@oracle.com> <3ee613ff-f6da-9ef8-6781-efce4edba29e@oracle.com> Message-ID: <21b3fdc5-d8a7-0535-3ac6-f1a097a7431d@oracle.com> Hi Harold, On 5/05/2017 3:47 AM, harold seigel wrote: > Hi, > > Please review this updated webrev: > > http://cr.openjdk.java.net/~hseigel/bug_8178604.2/webrev/index.html > > The updated webrev takes out a lock when creating _exploded_entries. It > also contains additional comments and an assert() to address Lois's > concerns with reduced code readability involving what situations warrant > taking out a lock in ClassLoader::search_module_entries(). > > I don't think there needs to be two different initial entry points as > suggested by David below. The method behaves slightly differently > depending on the value of its 'need_lock' parameter. But, there is > enough common code for it to remain one method. I think this highlights uncertainty about the concurrent behaviour in relation to this code. On the one hand you are worried about an initialization race and so use the lock, but when you do: 1500 // The exploded build entries can be added to at any time so pass 'true' for 1501 // need_lock so that a lock is taken out when searching them. 1502 assert(_exploded_entries != NULL, "No exploded build entries present"); 1503 stream = search_module_entries(_exploded_entries, class_name, file_name, true, CHECK_NULL); you load _exploded_entries with no lock held and so must be certain that you can not possibly read a null value here. The needs_lock parameter seems superfluous - you know the condition for locking is that the list is the _exploded_entries, and you assert that. So no need to pass in needs_lock, just grab the lock when dealing with _exploded_entries. Cheers, David > Thanks, Harold > > On 5/3/2017 12:59 AM, David Holmes wrote: >> On 3/05/2017 7:11 AM, Lois Foltan wrote: >>> On 5/2/2017 9:03 AM, harold seigel wrote: >>>> Hi Lois, >>>> >>>> Thanks for your comments. Please see my in-line responses. >>>> >>>> Harold >>>> >>>> >>>> On 5/1/2017 4:14 PM, Lois Foltan wrote: >>>>> Hi Harold, >>>>> >>>>> Looks good. A couple of comments: >>>>> >>>>> src/share/vm/classfile/classLoader.cpp >>>>> line #828 - please take out the Module_lock when the >>>>> _exploded_entries is created >>>> This fix does not change the possible need for synchronization when >>>> _exploded_entries gets created during module system initialization. >>>> Are you saying that the existing code is wrong and should have taken >>>> out the Module_lock before creating _exploding_entries? >>> >>> The method ClassLoader::add_to_exploded_build_list can be called at >>> anytime (during module system initialization and post module system >>> initialization). Thus it seems prudent that no matter what the current >>> JVM phase, that any access to ClassLoader::_exploding_entries be >>> protected via a lock. That includes creation as well as insertion into >>> the list. >> >> If creation is not guaranteed to occur before other threads that will >> call the same method and use the _exploded_entries list, then >> synchronization of some form is essential. If it is guaranteed then we >> don't need create time sync just to be prudent - but the guarantee >> must be firmly established and clearly documented. >> >>>> >>>>> line #1402 - I like the new factored out method >>>>> find_first_module_cpe(), however, instead of adding a new "need_lock" >>>>> parameter, I would rather see code >>>>> in find_first_module_cpe() that takes the >>>>> Module_lock if the module_list parameter equals >>>>> ClassLoader::_exploded_entries >>>> I think your suggestion makes the code less flexible and does not >>>> require potential future callers of search_module_entries() to think >>>> about synchronization. So, I'd prefer to leave that change as is. >>> >>> I see your point. My point was based on reduced code readability, the >>> further away from the decision to establish the lock, the less apparent >>> it is within the new method ClassLoader::find_first_module_cpe() as to >>> what situations warranted the lock in the first place. >> >> Further, this: >> >> 1495 stream = search_module_entries(_exploded_entries, >> class_name, file_name, true, CHECK_NULL); >> >> is potentially unsafe if we can race with creation because we first >> access _exploded_entries without holding any lock! And the fact the >> method needs to do different things depending on which "list" was >> passed in suggests to me it should be two different initial entry points. >> >> Thanks, >> David >> ----- >> >>> Thanks, >>> Lois >>> >>>>> >>>>> Thanks, >>>>> Lois >>>>> >>>>> On 5/1/2017 3:36 PM, harold seigel wrote: >>>>>> Hi, >>>>>> >>>>>> Please review this JDK-10 bug fix to allow defining of modules to >>>>>> the boot loader in exploded builds after module system >>>>>> initialization. The fix uses the module lock to synchronize access >>>>>> to the _exploded_entries data structure (which is only used by >>>>>> exploded builds). >>>>>> >>>>>> Note that the above capability already exists for regular builds. >>>>>> >>>>>> Open Webrev: >>>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604/webrev/index.html >>>>>> >>>>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8178604 >>>>>> >>>>>> The fix was tested with JCK tests, the JTreg hotspot, java/io, >>>>>> java/lang, java/util and other tests, the RBT tier2 -tier5 tests, >>>>>> the co-located NSK tests, and with JPRT. The JTReg and JCK tests >>>>>> were run with an exploded build. Also, the example program in the >>>>>> JBS bug was run by hand to verify the fix. >>>>>> >>>>>> Thanks, Harold >>>>>> >>>>> >>>> >>> > From ioi.lam at oracle.com Thu May 4 22:46:24 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 04 May 2017 15:46:24 -0700 Subject: RFR - JDK-8179625 [BACKOUT] of 8179305 Avoid repeated calls to JavaThread::last_frame in InterpreterRuntime In-Reply-To: <40f2b23b-731a-cb1d-42a0-3137a1fb828a@oracle.com> References: <590B2F47.7080301@oracle.com> <40f2b23b-731a-cb1d-42a0-3137a1fb828a@oracle.com> Message-ID: <590BAF40.4020101@oracle.com> Hi Dean, That was it! I fixed this and reran the reproducer. No crash in 8000 iterations, so I think we're good :-) I'll do more stress testing before submitting the REDO review. Thanks - Ioi On 5/4/17 9:53 AM, dean.long at oracle.com wrote: > Sorry for not noticing before, but this looks like it could be the > cause of the crash: > > - BasicObjectLock* monitor_begin() const { > - return _last_frame.interpreter_frame_monitor_end(); > - } > - BasicObjectLock* monitor_end() const { > - return _last_frame.interpreter_frame_monitor_begin(); > - } > > _begin and _end are reversed. > > dl > > > On 5/4/17 6:40 AM, Ioi Lam wrote: >> Hi, >> >> It turns out JDK- 8179305 causes intermittent failures in JPRT, so it >> needs to be backed out quickly: >> >> https://bugs.openjdk.java.net/browse/JDK-8179625 >> http://cr.openjdk.java.net/~iklam/jdk10/8179625-backout-8179305-last-frame.v01/ >> >> >> Thanks >> - Ioi > From ioi.lam at oracle.com Thu May 4 23:02:00 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 04 May 2017 16:02:00 -0700 Subject: RFR: 8154791: Xlog classload too redundant msgs info/debug In-Reply-To: <8ef74c07-1a6e-eede-d16c-4ba283a96199@oracle.com> References: <6ff9f069-2da7-b725-d29f-e3d850650aca@oracle.com> <5908D481.2070104@oracle.com> <6a8ffd20-ce58-150b-a94d-c51c5956d191@oracle.com> <5908F288.9020205@oracle.com> <5908F5D4.6040505@oracle.com> <8ef74c07-1a6e-eede-d16c-4ba283a96199@oracle.com> Message-ID: <590BB2E8.1040308@oracle.com> Hi Rachel, On 5/4/17 8:52 AM, Rachel Protacio wrote: > If I'm using this incorrectly, please let me know, but I think > LogBufferMessage also doesn't do exactly what we want. With this > changeset: http://cr.openjdk.java.net/~rprotacio/8154791.01/ we get > this output: > > [0.050s][info ][class,load] java.lang.CharSequence > [0.050s][info ][class,load] source: jrt:/java.base > [0.050s][info ][class,load] loader: > [AllocatedObj(0x00000007c00015a0)] > [0.050s][debug][class,load] klass: 0x00000007c00015a0 > [0.050s][debug][class,load] super: 0x00000007c0000fb0 > [0.050s][info ][class,load] bytes: 2045 checksum: 5b97c8c1 > [0.051s][info ][class,load] java.lang.String > [0.051s][info ][class,load] source: jrt:/java.base > [0.051s][info ][class,load] loader: > [AllocatedObj(0x00000007c0001798)] > [0.051s][debug][class,load] klass: 0x00000007c0001798 > [0.051s][debug][class,load] super: 0x00000007c0000fb0 > [0.051s][debug][class,load] interfaces: > [0.051s][debug][class,load] 0x00000007c00011b0 > [0.051s][debug][class,load] 0x00000007c00013a8 > [0.051s][debug][class,load] 0x00000007c00015a0 > [0.051s][info ][class,load] bytes: 24872 checksum: 564a9fc2 > > It comes out "together", but each call is a separate line. But to put > it all on one line would lead to messy code, something like adding > each part to two temporary strings and putting them in the buffer at > the end, I think. > I think we can use stringStream to collect the output together, and then write the whole line at once. Here's what I tried on top of your patch above: http://cr.openjdk.java.net/~iklam/jdk10/8154791.01_log_class_load.delta/ I also moved the class loader info back into "debug" . JDK-8154791 suggested moving that into "info", but I think that should be considered in a separate issue. Personally I that think would make the "info" output too verbose. > So the other option is to do this: > http://cr.openjdk.java.net/~rprotacio/8154791.02 which is essentially > the original code, but slightly cleaner and without redundancy of the > info line inside the debug line. The only issue is it doesn't > necessarily guarantee the two lines will come out next to each other, > so we have to print the external name on each. > > [0.051s][info ][class,load] java.lang.CharSequence source: > jrt:/java.basejava.lang.CharSequence loader: [NULL class_loader] > bytes: 2045 checksum: 5b97c8c1 > [0.051s][debug][class,load] java.lang.CharSequence klass: > 0x00000007c00015a0 super: 0x00000007c0000fb0 > [0.052s][info ][class,load] java.lang.String source: > jrt:/java.basejava.lang.String loader: [NULL class_loader] bytes: > 24872 checksum: 564a9fc2 > [0.052s][debug][class,load] java.lang.String klass: > 0x00000007c0001798 super: 0x00000007c0000fb0 interfaces: > 0x00000007c00011b0 0x00000007c00013a8 0x00000007c00015a0 > > When two classes of the same name are loaded at the same time, and the output is interleaved, you can't find out which is which: HelloWorld source: foo.jar HelloWorld source: bar.jar HelloWorld klass: 0x1234 HelloWorld klass: 0x5678 E.g., is the klass loaded from foo.jar 0x1234 or 0x5678? Thanks - Ioi > Or we can make the info output the same as debug, when debug is > specified. Which I have reservations about but will certainly do if > that's what makes the most sense. > > Thanks, > Rachel > > On 5/3/2017 2:23 AM, Thomas St?fe wrote: >> Hi Ioi, >> >> On Tue, May 2, 2017 at 11:10 PM, Ioi Lam wrote: >> >>> By the way, I think we should really fix this in the UL level -- have >>> multi-line buffering, so you can prevent other threads from >>> interleaving >>> your input. Make it something like: >>> >>> Log(class, load).start_buffering(); >>> log_info(class, load)(class name and source); >>> log_debug(class, load)(other details); >>> Log(class, load).flush(); >>> >>> That way, the output will look like this: >>> >>> [0.162s][debug ][class,load] java.lang.Comparable source: >>> jrt:/java.base >>> [0.162s][debug ][class,load] [NULL class_loader] 0x00000007c00013a8 >>> super: >>> 0x00000007c0000fb0 bytes: 235 checksum: a75dadb6 >>> >>> So the output will be terse, and still parsable. >>> >>> I think this will be useful for other types of logs as well, not >>> just the >>> class loading logs. >>> >>> >> I thought that already exists: >> >> https://bugs.openjdk.java.net/browse/JDK-8145934 >> >> see logMessage.hpp. >> >> Kind Regards, Thomas >> >> >>> Thanks >>> - Ioi >>> >>> >>> On 5/2/17 1:56 PM, Ioi Lam wrote: >>> >>>> >>>> On 5/2/17 1:31 PM, Rachel Protacio wrote: >>>> >>>>> Hi Ioi, >>>>> >>>>> Thanks for your reply, comments inline. >>>>> >>>>> >>>>> On 5/2/2017 2:48 PM, Ioi Lam wrote: >>>>> >>>>>> Hi Rachel, >>>>>> >>>>>> There are a few reasons why the current output has duplications. The >>>>>> "logging levels are not mutually exclusive" comment means this: >>>>>> >>>>>> java -Xlog:class+load=debug:file=debug.log >>>>>> -Xlog:class+load=info:file=info.log >>>>>> >>>>>> So info.log contains only the info level logs, but debug.log >>>>>> contains >>>>>> both info and debug level logs. >>>>>> >>>>>> Ideally I want info.log to contain this: >>>>>> >>>>>> [0.162s][info ][class,load] java.lang.Comparable source: >>>>>> jrt:/java.base >>>>>> >>>>>> .. and debug.log to contain this: >>>>>> >>>>>> [0.162s][debug ][class,load] java.lang.Comparable source: >>>>>> jrt:/java.base [NULL class_loader] 0x00000007c00013a8 super: >>>>>> 0x00000007c0000fb0 bytes: 235 >>>>>> checksum: a75dadb6 >>>>>> >>>>> I see what you want, but I think the problem is that logging doesn't >>>>> work that way. If debug is specified, both debug and info levels are >>>>> printed. So the command >>>>> >>>>> java -Xlog:class+load=debug:file=debug.log >>>>> >>>>> will print both info level logging and debug level logging to this >>>>> "debug.log" file. And then tacking on >>>>> >>>>> -Xlog:class+load=info:file=info.log >>>>> >>>>> in the command would print just the info level logging to the >>>>> "info.log" >>>>> file. But since it's all the same java command, it will always >>>>> know that >>>>> both debug and info levels are specified. I believe what you want >>>>> me to do >>>>> is say "if debug is specified, print everything to debug only" but >>>>> then the >>>>> info.log would be empty. You would have to run a separate java >>>>> command with >>>>> just the info level specified to get the smaller amount of >>>>> information. >>>>> >>>> No, that's not what I want. I am perfectly fine with how UL works >>>> today: >>>> info.log contains only the info level, and debug.info contains both >>>> debug and info level. >>>> >>>> All I want is everything related to the same class to be printed on >>>> the >>>> same line. For example, I don't want debug.log to contain this: >>>> >>>> [0.162s][info ][class,load] java.lang.Comparable source: >>>> jrt:/java.base >>>> [0.162s][debug ][class,load] [NULL class_loader] 0x00000007c00013a8 >>>> super: 0x00000007c0000fb0 bytes: 235 checksum: a75dadb6 >>>> >>>> That's because it's impossible to associated the second with the first >>>> line when you have parallel threads loading classes concurrently. >>>> >>>> But I think that would violate user expectations - if they get certain >>>>> information with info logging, adding the extra level of debug >>>>> logging >>>>> should not change what they see from the info logging. >>>>> >>>>>> Your current patch splits out the debug log into several lines. This >>>>>> makes it difficult to analyze the log when classes are loaded >>>>>> concurrently >>>>>> on different threads: >>>>>> >>>>>> [0.162s][info ][class,load] java.lang.Comparable source: >>>>>> jrt:/java.base >>>>>> [0.162s][info ][class,load] java.lang.Comparable loader: [NULL >>>>>> class_loader] >>>>>> [0.162s][debug][class,load] java.lang.Comparable klass: >>>>>> 0x00000007c00013a8 >>>>>> [0.163s][info ][class,load] java.lang.String source: >>>>>> jrt:/java.base <<< >>>>>> oops! from a different thread >>>>>> [0.164s][debug][class,load] java.lang.Comparable super: >>>>>> 0x00000007c0000fb0 >>>>>> [0.164s][info ][class,load] java.lang.Comparable bytes: 235 >>>>>> [0.164s][info ][class,load] java.lang.Comparable checksum: a75dadb6 >>>>>> >>>>> That's fair. I was trying to make it more readable since it's hard to >>>>> visually parse one giant line. Maybe I could put some delimiter in >>>>> between >>>>> each piece of info? But if no one else cares, I can just put it >>>>> back as one >>>>> line, no delimiters. >>>>> >>>>>> Because the leveling in UL is by lines, in JDK-8079408, where >>>>>> this was >>>>>> first implemented, I couldn't find a way to print >>>>>> "java.lang.Comparable >>>>>> source: jrt:/java.base" in "info" mode, and print the rest in >>>>>> "debug" mode, >>>>>> while keeping the whole thing on the same line (in both the >>>>>> info.log and >>>>>> debug.log files). That's why we have the 2 lines with duplicated >>>>>> info. >>>>>> >>>>>> I think it's really important to keep everything on the same line >>>>>> (the >>>>>> output needs to be processed automatically by scripts). With the >>>>>> current UL >>>>>> restrictions, I think the only way to do it is make the "info" >>>>>> logging more >>>>>> verbose when "debug" level is selected. E.g., >>>>>> >>>>>> outputStream* info_log = Log(class, load)::info_stream(); >>>>>> info_log->print("%s", ext_name); >>>>>> info_log->print_cr(" source: jrt:/%s", module_name); .... >>>>>> >>>>>> if (log_is_enabled(Debug, class, load)) { // always print to >>>>>> info >>>>>> log, even if debug is selected >>>>>> info_log->print(" klass: " INTPTR_FORMAT, p2i(this)); >>>>>> .... >>>>>> } >>>>>> >>>>>> So now info.log and debug.log will have the same logs, but oh >>>>>> well ..... >>>>>> >>>>> This sounds a little different. It sounds like >>>>> >>>>> When only info is specified, info log will contain: external >>>>> name, >>>>> source, loader, bytes, checksum >>>>> >>>>> When debug is specified, info log will contain the same as above, >>>>> plus: klass, super, interfaces >>>>> When debug is specified, debug log will be an exact duplicate of >>>>> info log so it can print to an external file >>>>> >>>>> Did I understand that right? Please correct me. If this is what you >>>>> meant, I would again argue that this violates the expectation that >>>>> "info is >>>>> info is info" regardless of what else is specified (not a written >>>>> rule, but >>>>> how I think it should work). And it is redundant, which is the >>>>> original >>>>> issue in the name of this RFE. >>>>> >>>>> I think the best bet would be >>>>> >>>>> Info log: external name, source, loader, bytes, checksum >>>>> Debug log: external name, klass, super, interfaces >>>>> >>>>> Or we get rid of the level distinction completely and put all the >>>>> information together in one line so it never gets broken up. >>>>> Probably under >>>>> debug. >>>>> >>>>> >>>> We need to keep the current debug output intact, on a single line, so >>>> that the information can be processed by scripts. >>>> >>>> So, it seems like our 3 choices are: >>>> >>>> (1) make the info output the same as debug, if >>>> -Xlog:class+load=debug is >>>> specified >>>> (2) remove the info output altogether >>>> (3) do nothing >>>> >>>> You don't like (1). >>>> >>>> I don't think (2) achieves what this RFE wants -- namely, make the >>>> output >>>> less redundant, because you will see all the details all the time, >>>> and you >>>> don't even get a choice for more terse output. It also makes the >>>> output of >>>> -verbose different (and much less useful) than in JDK 9. >>>> >>>> So I would vote for (3), keep the output as is. Yeah, with debug level >>>> you have a lot of output, but that's pretty easy to filter. I think >>>> we had >>>> a discussion about this when 8079408 was implemented, and that was the >>>> conclusion we had. >>>> >>>> Thanks >>>> - Ioi >>>> >>>> Let me know what you think. >>>>> Thanks, >>>>> Rachel >>>>> >>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>> On 5/2/17 10:32 AM, Rachel Protacio wrote: >>>>>> >>>>>>> Hello! >>>>>>> >>>>>>> Please review this enhancement correcting redundancies and >>>>>>> neatening >>>>>>> up the -Xlog:class+load code. The redundancy in the code before >>>>>>> I believe >>>>>>> stemmed from a misunderstanding about logging levels (the >>>>>>> comment saying >>>>>>> they are not mutually exclusive was misleading). >>>>>>> >>>>>>> Tested with JPRT and RBT. >>>>>>> >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8154791 >>>>>>> Webrev: http://cr.openjdk.java.net/~rprotacio/8154791.00 >>>>>>> >>>>>>> Thanks, >>>>>>> Rachel >>>>>>> >>>>>> > From marcus.larsson at oracle.com Fri May 5 08:28:56 2017 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Fri, 5 May 2017 10:28:56 +0200 Subject: RFR: 8154791: Xlog classload too redundant msgs info/debug In-Reply-To: <590BB2E8.1040308@oracle.com> References: <6ff9f069-2da7-b725-d29f-e3d850650aca@oracle.com> <5908D481.2070104@oracle.com> <6a8ffd20-ce58-150b-a94d-c51c5956d191@oracle.com> <5908F288.9020205@oracle.com> <5908F5D4.6040505@oracle.com> <8ef74c07-1a6e-eede-d16c-4ba283a96199@oracle.com> <590BB2E8.1040308@oracle.com> Message-ID: <73701218-6b1e-0220-e6f0-8ba8114b4d26@oracle.com> Hi, On 2017-05-05 01:02, Ioi Lam wrote: > Hi Rachel, > > On 5/4/17 8:52 AM, Rachel Protacio wrote: >> If I'm using this incorrectly, please let me know, but I think >> LogBufferMessage also doesn't do exactly what we want. With this >> changeset: http://cr.openjdk.java.net/~rprotacio/8154791.01/ we get >> this output: >> >> [0.050s][info ][class,load] java.lang.CharSequence >> [0.050s][info ][class,load] source: jrt:/java.base >> [0.050s][info ][class,load] loader: >> [AllocatedObj(0x00000007c00015a0)] >> [0.050s][debug][class,load] klass: 0x00000007c00015a0 >> [0.050s][debug][class,load] super: 0x00000007c0000fb0 >> [0.050s][info ][class,load] bytes: 2045 checksum: 5b97c8c1 >> [0.051s][info ][class,load] java.lang.String >> [0.051s][info ][class,load] source: jrt:/java.base >> [0.051s][info ][class,load] loader: >> [AllocatedObj(0x00000007c0001798)] >> [0.051s][debug][class,load] klass: 0x00000007c0001798 >> [0.051s][debug][class,load] super: 0x00000007c0000fb0 >> [0.051s][debug][class,load] interfaces: >> [0.051s][debug][class,load] 0x00000007c00011b0 >> [0.051s][debug][class,load] 0x00000007c00013a8 >> [0.051s][debug][class,load] 0x00000007c00015a0 >> [0.051s][info ][class,load] bytes: 24872 checksum: 564a9fc2 >> >> It comes out "together", but each call is a separate line. But to put >> it all on one line would lead to messy code, something like adding >> each part to two temporary strings and putting them in the buffer at >> the end, I think. >> > > I think we can use stringStream to collect the output together, and > then write the whole line at once. Here's what I tried on top of your > patch above: > > http://cr.openjdk.java.net/~iklam/jdk10/8154791.01_log_class_load.delta/ Usage looks good to me, but it can be simplified by using LogMessage instead of directly using the LogMessageBuffer. With LogMessage you would change 3065 LogMessageBuffer msg; into 3065 LogMessage(class, load) msg; and remove the following lines: 3133 Log(class, load) _log; 3134 _log.write(msg); (The message gets written when the LogMessage object goes out of scope.) Thanks, Marcus > > I also moved the class loader info back into "debug" . JDK-8154791 > suggested moving that into "info", but I think that should be > considered in a separate issue. Personally I that think would make the > "info" output too verbose. > >> So the other option is to do this: >> http://cr.openjdk.java.net/~rprotacio/8154791.02 which is essentially >> the original code, but slightly cleaner and without redundancy of the >> info line inside the debug line. The only issue is it doesn't >> necessarily guarantee the two lines will come out next to each other, >> so we have to print the external name on each. >> >> [0.051s][info ][class,load] java.lang.CharSequence source: >> jrt:/java.basejava.lang.CharSequence loader: [NULL class_loader] >> bytes: 2045 checksum: 5b97c8c1 >> [0.051s][debug][class,load] java.lang.CharSequence klass: >> 0x00000007c00015a0 super: 0x00000007c0000fb0 >> [0.052s][info ][class,load] java.lang.String source: >> jrt:/java.basejava.lang.String loader: [NULL class_loader] bytes: >> 24872 checksum: 564a9fc2 >> [0.052s][debug][class,load] java.lang.String klass: >> 0x00000007c0001798 super: 0x00000007c0000fb0 interfaces: >> 0x00000007c00011b0 0x00000007c00013a8 0x00000007c00015a0 >> >> > > When two classes of the same name are loaded at the same time, and the > output is interleaved, you can't find out which is which: > > HelloWorld source: foo.jar > HelloWorld source: bar.jar > HelloWorld klass: 0x1234 > HelloWorld klass: 0x5678 > > E.g., is the klass loaded from foo.jar 0x1234 or 0x5678? > > Thanks > - Ioi > >> Or we can make the info output the same as debug, when debug is >> specified. Which I have reservations about but will certainly do if >> that's what makes the most sense. >> >> Thanks, >> Rachel >> >> On 5/3/2017 2:23 AM, Thomas St?fe wrote: >>> Hi Ioi, >>> >>> On Tue, May 2, 2017 at 11:10 PM, Ioi Lam wrote: >>> >>>> By the way, I think we should really fix this in the UL level -- have >>>> multi-line buffering, so you can prevent other threads from >>>> interleaving >>>> your input. Make it something like: >>>> >>>> Log(class, load).start_buffering(); >>>> log_info(class, load)(class name and source); >>>> log_debug(class, load)(other details); >>>> Log(class, load).flush(); >>>> >>>> That way, the output will look like this: >>>> >>>> [0.162s][debug ][class,load] java.lang.Comparable source: >>>> jrt:/java.base >>>> [0.162s][debug ][class,load] [NULL class_loader] 0x00000007c00013a8 >>>> super: >>>> 0x00000007c0000fb0 bytes: 235 checksum: a75dadb6 >>>> >>>> So the output will be terse, and still parsable. >>>> >>>> I think this will be useful for other types of logs as well, not >>>> just the >>>> class loading logs. >>>> >>>> >>> I thought that already exists: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8145934 >>> >>> see logMessage.hpp. >>> >>> Kind Regards, Thomas >>> >>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>> On 5/2/17 1:56 PM, Ioi Lam wrote: >>>> >>>>> >>>>> On 5/2/17 1:31 PM, Rachel Protacio wrote: >>>>> >>>>>> Hi Ioi, >>>>>> >>>>>> Thanks for your reply, comments inline. >>>>>> >>>>>> >>>>>> On 5/2/2017 2:48 PM, Ioi Lam wrote: >>>>>> >>>>>>> Hi Rachel, >>>>>>> >>>>>>> There are a few reasons why the current output has duplications. >>>>>>> The >>>>>>> "logging levels are not mutually exclusive" comment means this: >>>>>>> >>>>>>> java -Xlog:class+load=debug:file=debug.log >>>>>>> -Xlog:class+load=info:file=info.log >>>>>>> >>>>>>> So info.log contains only the info level logs, but debug.log >>>>>>> contains >>>>>>> both info and debug level logs. >>>>>>> >>>>>>> Ideally I want info.log to contain this: >>>>>>> >>>>>>> [0.162s][info ][class,load] java.lang.Comparable source: >>>>>>> jrt:/java.base >>>>>>> >>>>>>> .. and debug.log to contain this: >>>>>>> >>>>>>> [0.162s][debug ][class,load] java.lang.Comparable source: >>>>>>> jrt:/java.base [NULL class_loader] 0x00000007c00013a8 super: >>>>>>> 0x00000007c0000fb0 bytes: 235 >>>>>>> checksum: a75dadb6 >>>>>>> >>>>>> I see what you want, but I think the problem is that logging doesn't >>>>>> work that way. If debug is specified, both debug and info levels are >>>>>> printed. So the command >>>>>> >>>>>> java -Xlog:class+load=debug:file=debug.log >>>>>> >>>>>> will print both info level logging and debug level logging to this >>>>>> "debug.log" file. And then tacking on >>>>>> >>>>>> -Xlog:class+load=info:file=info.log >>>>>> >>>>>> in the command would print just the info level logging to the >>>>>> "info.log" >>>>>> file. But since it's all the same java command, it will always >>>>>> know that >>>>>> both debug and info levels are specified. I believe what you want >>>>>> me to do >>>>>> is say "if debug is specified, print everything to debug only" >>>>>> but then the >>>>>> info.log would be empty. You would have to run a separate java >>>>>> command with >>>>>> just the info level specified to get the smaller amount of >>>>>> information. >>>>>> >>>>> No, that's not what I want. I am perfectly fine with how UL works >>>>> today: >>>>> info.log contains only the info level, and debug.info contains both >>>>> debug and info level. >>>>> >>>>> All I want is everything related to the same class to be printed >>>>> on the >>>>> same line. For example, I don't want debug.log to contain this: >>>>> >>>>> [0.162s][info ][class,load] java.lang.Comparable source: >>>>> jrt:/java.base >>>>> [0.162s][debug ][class,load] [NULL class_loader] 0x00000007c00013a8 >>>>> super: 0x00000007c0000fb0 bytes: 235 checksum: a75dadb6 >>>>> >>>>> That's because it's impossible to associated the second with the >>>>> first >>>>> line when you have parallel threads loading classes concurrently. >>>>> >>>>> But I think that would violate user expectations - if they get >>>>> certain >>>>>> information with info logging, adding the extra level of debug >>>>>> logging >>>>>> should not change what they see from the info logging. >>>>>> >>>>>>> Your current patch splits out the debug log into several lines. >>>>>>> This >>>>>>> makes it difficult to analyze the log when classes are loaded >>>>>>> concurrently >>>>>>> on different threads: >>>>>>> >>>>>>> [0.162s][info ][class,load] java.lang.Comparable source: >>>>>>> jrt:/java.base >>>>>>> [0.162s][info ][class,load] java.lang.Comparable loader: [NULL >>>>>>> class_loader] >>>>>>> [0.162s][debug][class,load] java.lang.Comparable klass: >>>>>>> 0x00000007c00013a8 >>>>>>> [0.163s][info ][class,load] java.lang.String source: >>>>>>> jrt:/java.base <<< >>>>>>> oops! from a different thread >>>>>>> [0.164s][debug][class,load] java.lang.Comparable super: >>>>>>> 0x00000007c0000fb0 >>>>>>> [0.164s][info ][class,load] java.lang.Comparable bytes: 235 >>>>>>> [0.164s][info ][class,load] java.lang.Comparable checksum: a75dadb6 >>>>>>> >>>>>> That's fair. I was trying to make it more readable since it's >>>>>> hard to >>>>>> visually parse one giant line. Maybe I could put some delimiter >>>>>> in between >>>>>> each piece of info? But if no one else cares, I can just put it >>>>>> back as one >>>>>> line, no delimiters. >>>>>> >>>>>>> Because the leveling in UL is by lines, in JDK-8079408, where >>>>>>> this was >>>>>>> first implemented, I couldn't find a way to print >>>>>>> "java.lang.Comparable >>>>>>> source: jrt:/java.base" in "info" mode, and print the rest in >>>>>>> "debug" mode, >>>>>>> while keeping the whole thing on the same line (in both the >>>>>>> info.log and >>>>>>> debug.log files). That's why we have the 2 lines with duplicated >>>>>>> info. >>>>>>> >>>>>>> I think it's really important to keep everything on the same >>>>>>> line (the >>>>>>> output needs to be processed automatically by scripts). With the >>>>>>> current UL >>>>>>> restrictions, I think the only way to do it is make the "info" >>>>>>> logging more >>>>>>> verbose when "debug" level is selected. E.g., >>>>>>> >>>>>>> outputStream* info_log = Log(class, load)::info_stream(); >>>>>>> info_log->print("%s", ext_name); >>>>>>> info_log->print_cr(" source: jrt:/%s", module_name); .... >>>>>>> >>>>>>> if (log_is_enabled(Debug, class, load)) { // always print to >>>>>>> info >>>>>>> log, even if debug is selected >>>>>>> info_log->print(" klass: " INTPTR_FORMAT, p2i(this)); >>>>>>> .... >>>>>>> } >>>>>>> >>>>>>> So now info.log and debug.log will have the same logs, but oh >>>>>>> well ..... >>>>>>> >>>>>> This sounds a little different. It sounds like >>>>>> >>>>>> When only info is specified, info log will contain: external >>>>>> name, >>>>>> source, loader, bytes, checksum >>>>>> >>>>>> When debug is specified, info log will contain the same as >>>>>> above, >>>>>> plus: klass, super, interfaces >>>>>> When debug is specified, debug log will be an exact duplicate of >>>>>> info log so it can print to an external file >>>>>> >>>>>> Did I understand that right? Please correct me. If this is what you >>>>>> meant, I would again argue that this violates the expectation >>>>>> that "info is >>>>>> info is info" regardless of what else is specified (not a written >>>>>> rule, but >>>>>> how I think it should work). And it is redundant, which is the >>>>>> original >>>>>> issue in the name of this RFE. >>>>>> >>>>>> I think the best bet would be >>>>>> >>>>>> Info log: external name, source, loader, bytes, checksum >>>>>> Debug log: external name, klass, super, interfaces >>>>>> >>>>>> Or we get rid of the level distinction completely and put all the >>>>>> information together in one line so it never gets broken up. >>>>>> Probably under >>>>>> debug. >>>>>> >>>>>> >>>>> We need to keep the current debug output intact, on a single line, so >>>>> that the information can be processed by scripts. >>>>> >>>>> So, it seems like our 3 choices are: >>>>> >>>>> (1) make the info output the same as debug, if >>>>> -Xlog:class+load=debug is >>>>> specified >>>>> (2) remove the info output altogether >>>>> (3) do nothing >>>>> >>>>> You don't like (1). >>>>> >>>>> I don't think (2) achieves what this RFE wants -- namely, make the >>>>> output >>>>> less redundant, because you will see all the details all the time, >>>>> and you >>>>> don't even get a choice for more terse output. It also makes the >>>>> output of >>>>> -verbose different (and much less useful) than in JDK 9. >>>>> >>>>> So I would vote for (3), keep the output as is. Yeah, with debug >>>>> level >>>>> you have a lot of output, but that's pretty easy to filter. I >>>>> think we had >>>>> a discussion about this when 8079408 was implemented, and that was >>>>> the >>>>> conclusion we had. >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> Let me know what you think. >>>>>> Thanks, >>>>>> Rachel >>>>>> >>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>>> >>>>>>> On 5/2/17 10:32 AM, Rachel Protacio wrote: >>>>>>> >>>>>>>> Hello! >>>>>>>> >>>>>>>> Please review this enhancement correcting redundancies and >>>>>>>> neatening >>>>>>>> up the -Xlog:class+load code. The redundancy in the code before >>>>>>>> I believe >>>>>>>> stemmed from a misunderstanding about logging levels (the >>>>>>>> comment saying >>>>>>>> they are not mutually exclusive was misleading). >>>>>>>> >>>>>>>> Tested with JPRT and RBT. >>>>>>>> >>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8154791 >>>>>>>> Webrev: http://cr.openjdk.java.net/~rprotacio/8154791.00 >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Rachel >>>>>>>> >>>>>>> >> > From rachel.protacio at oracle.com Fri May 5 18:34:49 2017 From: rachel.protacio at oracle.com (Rachel Protacio) Date: Fri, 5 May 2017 14:34:49 -0400 Subject: RFR: 8154791: Xlog classload too redundant msgs info/debug In-Reply-To: <73701218-6b1e-0220-e6f0-8ba8114b4d26@oracle.com> References: <6ff9f069-2da7-b725-d29f-e3d850650aca@oracle.com> <5908D481.2070104@oracle.com> <6a8ffd20-ce58-150b-a94d-c51c5956d191@oracle.com> <5908F288.9020205@oracle.com> <5908F5D4.6040505@oracle.com> <8ef74c07-1a6e-eede-d16c-4ba283a96199@oracle.com> <590BB2E8.1040308@oracle.com> <73701218-6b1e-0220-e6f0-8ba8114b4d26@oracle.com> Message-ID: <8c880811-fa82-d5b1-a9e7-42ef4fbb9eac@oracle.com> Thank you both for your help! Here's my latest webrev: http://cr.openjdk.java.net/~rprotacio/8154791.03 Rachel On 5/5/2017 4:28 AM, Marcus Larsson wrote: > Hi, > > > On 2017-05-05 01:02, Ioi Lam wrote: >> Hi Rachel, >> >> On 5/4/17 8:52 AM, Rachel Protacio wrote: >>> If I'm using this incorrectly, please let me know, but I think >>> LogBufferMessage also doesn't do exactly what we want. With this >>> changeset: http://cr.openjdk.java.net/~rprotacio/8154791.01/ we get >>> this output: >>> >>> [0.050s][info ][class,load] java.lang.CharSequence >>> [0.050s][info ][class,load] source: jrt:/java.base >>> [0.050s][info ][class,load] loader: >>> [AllocatedObj(0x00000007c00015a0)] >>> [0.050s][debug][class,load] klass: 0x00000007c00015a0 >>> [0.050s][debug][class,load] super: 0x00000007c0000fb0 >>> [0.050s][info ][class,load] bytes: 2045 checksum: 5b97c8c1 >>> [0.051s][info ][class,load] java.lang.String >>> [0.051s][info ][class,load] source: jrt:/java.base >>> [0.051s][info ][class,load] loader: >>> [AllocatedObj(0x00000007c0001798)] >>> [0.051s][debug][class,load] klass: 0x00000007c0001798 >>> [0.051s][debug][class,load] super: 0x00000007c0000fb0 >>> [0.051s][debug][class,load] interfaces: >>> [0.051s][debug][class,load] 0x00000007c00011b0 >>> [0.051s][debug][class,load] 0x00000007c00013a8 >>> [0.051s][debug][class,load] 0x00000007c00015a0 >>> [0.051s][info ][class,load] bytes: 24872 checksum: 564a9fc2 >>> >>> It comes out "together", but each call is a separate line. But to >>> put it all on one line would lead to messy code, something like >>> adding each part to two temporary strings and putting them in the >>> buffer at the end, I think. >>> >> >> I think we can use stringStream to collect the output together, and >> then write the whole line at once. Here's what I tried on top of your >> patch above: >> >> http://cr.openjdk.java.net/~iklam/jdk10/8154791.01_log_class_load.delta/ > > Usage looks good to me, but it can be simplified by using LogMessage > instead of directly using the LogMessageBuffer. > > With LogMessage you would change > > 3065 LogMessageBuffer msg; > > into > > 3065 LogMessage(class, load) msg; > > and remove the following lines: > > 3133 Log(class, load) _log; > 3134 _log.write(msg); > > (The message gets written when the LogMessage object goes out of scope.) > > Thanks, > Marcus > >> >> I also moved the class loader info back into "debug" . JDK-8154791 >> suggested moving that into "info", but I think that should be >> considered in a separate issue. Personally I that think would make >> the "info" output too verbose. >> >>> So the other option is to do this: >>> http://cr.openjdk.java.net/~rprotacio/8154791.02 which is >>> essentially the original code, but slightly cleaner and without >>> redundancy of the info line inside the debug line. The only issue is >>> it doesn't necessarily guarantee the two lines will come out next to >>> each other, so we have to print the external name on each. >>> >>> [0.051s][info ][class,load] java.lang.CharSequence source: >>> jrt:/java.basejava.lang.CharSequence loader: [NULL class_loader] >>> bytes: 2045 checksum: 5b97c8c1 >>> [0.051s][debug][class,load] java.lang.CharSequence klass: >>> 0x00000007c00015a0 super: 0x00000007c0000fb0 >>> [0.052s][info ][class,load] java.lang.String source: >>> jrt:/java.basejava.lang.String loader: [NULL class_loader] bytes: >>> 24872 checksum: 564a9fc2 >>> [0.052s][debug][class,load] java.lang.String klass: >>> 0x00000007c0001798 super: 0x00000007c0000fb0 interfaces: >>> 0x00000007c00011b0 0x00000007c00013a8 0x00000007c00015a0 >>> >>> >> >> When two classes of the same name are loaded at the same time, and >> the output is interleaved, you can't find out which is which: >> >> HelloWorld source: foo.jar >> HelloWorld source: bar.jar >> HelloWorld klass: 0x1234 >> HelloWorld klass: 0x5678 >> >> E.g., is the klass loaded from foo.jar 0x1234 or 0x5678? >> >> Thanks >> - Ioi >> >>> Or we can make the info output the same as debug, when debug is >>> specified. Which I have reservations about but will certainly do if >>> that's what makes the most sense. >>> >>> Thanks, >>> Rachel >>> >>> On 5/3/2017 2:23 AM, Thomas St?fe wrote: >>>> Hi Ioi, >>>> >>>> On Tue, May 2, 2017 at 11:10 PM, Ioi Lam wrote: >>>> >>>>> By the way, I think we should really fix this in the UL level -- have >>>>> multi-line buffering, so you can prevent other threads from >>>>> interleaving >>>>> your input. Make it something like: >>>>> >>>>> Log(class, load).start_buffering(); >>>>> log_info(class, load)(class name and source); >>>>> log_debug(class, load)(other details); >>>>> Log(class, load).flush(); >>>>> >>>>> That way, the output will look like this: >>>>> >>>>> [0.162s][debug ][class,load] java.lang.Comparable source: >>>>> jrt:/java.base >>>>> [0.162s][debug ][class,load] [NULL class_loader] >>>>> 0x00000007c00013a8 super: >>>>> 0x00000007c0000fb0 bytes: 235 checksum: a75dadb6 >>>>> >>>>> So the output will be terse, and still parsable. >>>>> >>>>> I think this will be useful for other types of logs as well, not >>>>> just the >>>>> class loading logs. >>>>> >>>>> >>>> I thought that already exists: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8145934 >>>> >>>> see logMessage.hpp. >>>> >>>> Kind Regards, Thomas >>>> >>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> >>>>> On 5/2/17 1:56 PM, Ioi Lam wrote: >>>>> >>>>>> >>>>>> On 5/2/17 1:31 PM, Rachel Protacio wrote: >>>>>> >>>>>>> Hi Ioi, >>>>>>> >>>>>>> Thanks for your reply, comments inline. >>>>>>> >>>>>>> >>>>>>> On 5/2/2017 2:48 PM, Ioi Lam wrote: >>>>>>> >>>>>>>> Hi Rachel, >>>>>>>> >>>>>>>> There are a few reasons why the current output has >>>>>>>> duplications. The >>>>>>>> "logging levels are not mutually exclusive" comment means this: >>>>>>>> >>>>>>>> java -Xlog:class+load=debug:file=debug.log >>>>>>>> -Xlog:class+load=info:file=info.log >>>>>>>> >>>>>>>> So info.log contains only the info level logs, but debug.log >>>>>>>> contains >>>>>>>> both info and debug level logs. >>>>>>>> >>>>>>>> Ideally I want info.log to contain this: >>>>>>>> >>>>>>>> [0.162s][info ][class,load] java.lang.Comparable source: >>>>>>>> jrt:/java.base >>>>>>>> >>>>>>>> .. and debug.log to contain this: >>>>>>>> >>>>>>>> [0.162s][debug ][class,load] java.lang.Comparable source: >>>>>>>> jrt:/java.base [NULL class_loader] 0x00000007c00013a8 super: >>>>>>>> 0x00000007c0000fb0 bytes: 235 >>>>>>>> checksum: a75dadb6 >>>>>>>> >>>>>>> I see what you want, but I think the problem is that logging >>>>>>> doesn't >>>>>>> work that way. If debug is specified, both debug and info levels >>>>>>> are >>>>>>> printed. So the command >>>>>>> >>>>>>> java -Xlog:class+load=debug:file=debug.log >>>>>>> >>>>>>> will print both info level logging and debug level logging to this >>>>>>> "debug.log" file. And then tacking on >>>>>>> >>>>>>> -Xlog:class+load=info:file=info.log >>>>>>> >>>>>>> in the command would print just the info level logging to the >>>>>>> "info.log" >>>>>>> file. But since it's all the same java command, it will always >>>>>>> know that >>>>>>> both debug and info levels are specified. I believe what you >>>>>>> want me to do >>>>>>> is say "if debug is specified, print everything to debug only" >>>>>>> but then the >>>>>>> info.log would be empty. You would have to run a separate java >>>>>>> command with >>>>>>> just the info level specified to get the smaller amount of >>>>>>> information. >>>>>>> >>>>>> No, that's not what I want. I am perfectly fine with how UL works >>>>>> today: >>>>>> info.log contains only the info level, and debug.info contains both >>>>>> debug and info level. >>>>>> >>>>>> All I want is everything related to the same class to be printed >>>>>> on the >>>>>> same line. For example, I don't want debug.log to contain this: >>>>>> >>>>>> [0.162s][info ][class,load] java.lang.Comparable source: >>>>>> jrt:/java.base >>>>>> [0.162s][debug ][class,load] [NULL class_loader] 0x00000007c00013a8 >>>>>> super: 0x00000007c0000fb0 bytes: 235 checksum: a75dadb6 >>>>>> >>>>>> That's because it's impossible to associated the second with the >>>>>> first >>>>>> line when you have parallel threads loading classes concurrently. >>>>>> >>>>>> But I think that would violate user expectations - if they get >>>>>> certain >>>>>>> information with info logging, adding the extra level of debug >>>>>>> logging >>>>>>> should not change what they see from the info logging. >>>>>>> >>>>>>>> Your current patch splits out the debug log into several lines. >>>>>>>> This >>>>>>>> makes it difficult to analyze the log when classes are loaded >>>>>>>> concurrently >>>>>>>> on different threads: >>>>>>>> >>>>>>>> [0.162s][info ][class,load] java.lang.Comparable source: >>>>>>>> jrt:/java.base >>>>>>>> [0.162s][info ][class,load] java.lang.Comparable loader: [NULL >>>>>>>> class_loader] >>>>>>>> [0.162s][debug][class,load] java.lang.Comparable klass: >>>>>>>> 0x00000007c00013a8 >>>>>>>> [0.163s][info ][class,load] java.lang.String source: >>>>>>>> jrt:/java.base <<< >>>>>>>> oops! from a different thread >>>>>>>> [0.164s][debug][class,load] java.lang.Comparable super: >>>>>>>> 0x00000007c0000fb0 >>>>>>>> [0.164s][info ][class,load] java.lang.Comparable bytes: 235 >>>>>>>> [0.164s][info ][class,load] java.lang.Comparable checksum: >>>>>>>> a75dadb6 >>>>>>>> >>>>>>> That's fair. I was trying to make it more readable since it's >>>>>>> hard to >>>>>>> visually parse one giant line. Maybe I could put some delimiter >>>>>>> in between >>>>>>> each piece of info? But if no one else cares, I can just put it >>>>>>> back as one >>>>>>> line, no delimiters. >>>>>>> >>>>>>>> Because the leveling in UL is by lines, in JDK-8079408, where >>>>>>>> this was >>>>>>>> first implemented, I couldn't find a way to print >>>>>>>> "java.lang.Comparable >>>>>>>> source: jrt:/java.base" in "info" mode, and print the rest in >>>>>>>> "debug" mode, >>>>>>>> while keeping the whole thing on the same line (in both the >>>>>>>> info.log and >>>>>>>> debug.log files). That's why we have the 2 lines with >>>>>>>> duplicated info. >>>>>>>> >>>>>>>> I think it's really important to keep everything on the same >>>>>>>> line (the >>>>>>>> output needs to be processed automatically by scripts). With >>>>>>>> the current UL >>>>>>>> restrictions, I think the only way to do it is make the "info" >>>>>>>> logging more >>>>>>>> verbose when "debug" level is selected. E.g., >>>>>>>> >>>>>>>> outputStream* info_log = Log(class, load)::info_stream(); >>>>>>>> info_log->print("%s", ext_name); >>>>>>>> info_log->print_cr(" source: jrt:/%s", module_name); .... >>>>>>>> >>>>>>>> if (log_is_enabled(Debug, class, load)) { // always print >>>>>>>> to info >>>>>>>> log, even if debug is selected >>>>>>>> info_log->print(" klass: " INTPTR_FORMAT, p2i(this)); >>>>>>>> .... >>>>>>>> } >>>>>>>> >>>>>>>> So now info.log and debug.log will have the same logs, but oh >>>>>>>> well ..... >>>>>>>> >>>>>>> This sounds a little different. It sounds like >>>>>>> >>>>>>> When only info is specified, info log will contain: external >>>>>>> name, >>>>>>> source, loader, bytes, checksum >>>>>>> >>>>>>> When debug is specified, info log will contain the same as >>>>>>> above, >>>>>>> plus: klass, super, interfaces >>>>>>> When debug is specified, debug log will be an exact >>>>>>> duplicate of >>>>>>> info log so it can print to an external file >>>>>>> >>>>>>> Did I understand that right? Please correct me. If this is what you >>>>>>> meant, I would again argue that this violates the expectation >>>>>>> that "info is >>>>>>> info is info" regardless of what else is specified (not a >>>>>>> written rule, but >>>>>>> how I think it should work). And it is redundant, which is the >>>>>>> original >>>>>>> issue in the name of this RFE. >>>>>>> >>>>>>> I think the best bet would be >>>>>>> >>>>>>> Info log: external name, source, loader, bytes, checksum >>>>>>> Debug log: external name, klass, super, interfaces >>>>>>> >>>>>>> Or we get rid of the level distinction completely and put all the >>>>>>> information together in one line so it never gets broken up. >>>>>>> Probably under >>>>>>> debug. >>>>>>> >>>>>>> >>>>>> We need to keep the current debug output intact, on a single >>>>>> line, so >>>>>> that the information can be processed by scripts. >>>>>> >>>>>> So, it seems like our 3 choices are: >>>>>> >>>>>> (1) make the info output the same as debug, if >>>>>> -Xlog:class+load=debug is >>>>>> specified >>>>>> (2) remove the info output altogether >>>>>> (3) do nothing >>>>>> >>>>>> You don't like (1). >>>>>> >>>>>> I don't think (2) achieves what this RFE wants -- namely, make >>>>>> the output >>>>>> less redundant, because you will see all the details all the >>>>>> time, and you >>>>>> don't even get a choice for more terse output. It also makes the >>>>>> output of >>>>>> -verbose different (and much less useful) than in JDK 9. >>>>>> >>>>>> So I would vote for (3), keep the output as is. Yeah, with debug >>>>>> level >>>>>> you have a lot of output, but that's pretty easy to filter. I >>>>>> think we had >>>>>> a discussion about this when 8079408 was implemented, and that >>>>>> was the >>>>>> conclusion we had. >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>> Let me know what you think. >>>>>>> Thanks, >>>>>>> Rachel >>>>>>> >>>>>>> >>>>>>>> Thanks >>>>>>>> - Ioi >>>>>>>> >>>>>>>> On 5/2/17 10:32 AM, Rachel Protacio wrote: >>>>>>>> >>>>>>>>> Hello! >>>>>>>>> >>>>>>>>> Please review this enhancement correcting redundancies and >>>>>>>>> neatening >>>>>>>>> up the -Xlog:class+load code. The redundancy in the code >>>>>>>>> before I believe >>>>>>>>> stemmed from a misunderstanding about logging levels (the >>>>>>>>> comment saying >>>>>>>>> they are not mutually exclusive was misleading). >>>>>>>>> >>>>>>>>> Tested with JPRT and RBT. >>>>>>>>> >>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8154791 >>>>>>>>> Webrev: http://cr.openjdk.java.net/~rprotacio/8154791.00 >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Rachel >>>>>>>>> >>>>>>>> >>> >> > From ioi.lam at oracle.com Fri May 5 19:57:30 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 05 May 2017 12:57:30 -0700 Subject: RFR: 8154791: Xlog classload too redundant msgs info/debug In-Reply-To: <8c880811-fa82-d5b1-a9e7-42ef4fbb9eac@oracle.com> References: <6ff9f069-2da7-b725-d29f-e3d850650aca@oracle.com> <5908D481.2070104@oracle.com> <6a8ffd20-ce58-150b-a94d-c51c5956d191@oracle.com> <5908F288.9020205@oracle.com> <5908F5D4.6040505@oracle.com> <8ef74c07-1a6e-eede-d16c-4ba283a96199@oracle.com> <590BB2E8.1040308@oracle.com> <73701218-6b1e-0220-e6f0-8ba8114b4d26@oracle.com> <8c880811-fa82-d5b1-a9e7-42ef4fbb9eac@oracle.com> Message-ID: <590CD92A.1080602@oracle.com> Looks good! Thanks - Ioi On 5/5/17 11:34 AM, Rachel Protacio wrote: > Thank you both for your help! Here's my latest webrev: > http://cr.openjdk.java.net/~rprotacio/8154791.03 > > Rachel > > > On 5/5/2017 4:28 AM, Marcus Larsson wrote: >> Hi, >> >> >> On 2017-05-05 01:02, Ioi Lam wrote: >>> Hi Rachel, >>> >>> On 5/4/17 8:52 AM, Rachel Protacio wrote: >>>> If I'm using this incorrectly, please let me know, but I think >>>> LogBufferMessage also doesn't do exactly what we want. With this >>>> changeset: http://cr.openjdk.java.net/~rprotacio/8154791.01/ we get >>>> this output: >>>> >>>> [0.050s][info ][class,load] java.lang.CharSequence >>>> [0.050s][info ][class,load] source: jrt:/java.base >>>> [0.050s][info ][class,load] loader: >>>> [AllocatedObj(0x00000007c00015a0)] >>>> [0.050s][debug][class,load] klass: 0x00000007c00015a0 >>>> [0.050s][debug][class,load] super: 0x00000007c0000fb0 >>>> [0.050s][info ][class,load] bytes: 2045 checksum: 5b97c8c1 >>>> [0.051s][info ][class,load] java.lang.String >>>> [0.051s][info ][class,load] source: jrt:/java.base >>>> [0.051s][info ][class,load] loader: >>>> [AllocatedObj(0x00000007c0001798)] >>>> [0.051s][debug][class,load] klass: 0x00000007c0001798 >>>> [0.051s][debug][class,load] super: 0x00000007c0000fb0 >>>> [0.051s][debug][class,load] interfaces: >>>> [0.051s][debug][class,load] 0x00000007c00011b0 >>>> [0.051s][debug][class,load] 0x00000007c00013a8 >>>> [0.051s][debug][class,load] 0x00000007c00015a0 >>>> [0.051s][info ][class,load] bytes: 24872 checksum: 564a9fc2 >>>> >>>> It comes out "together", but each call is a separate line. But to >>>> put it all on one line would lead to messy code, something like >>>> adding each part to two temporary strings and putting them in the >>>> buffer at the end, I think. >>>> >>> >>> I think we can use stringStream to collect the output together, and >>> then write the whole line at once. Here's what I tried on top of >>> your patch above: >>> >>> http://cr.openjdk.java.net/~iklam/jdk10/8154791.01_log_class_load.delta/ >>> >> >> Usage looks good to me, but it can be simplified by using LogMessage >> instead of directly using the LogMessageBuffer. >> >> With LogMessage you would change >> >> 3065 LogMessageBuffer msg; >> >> into >> >> 3065 LogMessage(class, load) msg; >> >> and remove the following lines: >> >> 3133 Log(class, load) _log; >> 3134 _log.write(msg); >> >> (The message gets written when the LogMessage object goes out of scope.) >> >> Thanks, >> Marcus >> >>> >>> I also moved the class loader info back into "debug" . JDK-8154791 >>> suggested moving that into "info", but I think that should be >>> considered in a separate issue. Personally I that think would make >>> the "info" output too verbose. >>> >>>> So the other option is to do this: >>>> http://cr.openjdk.java.net/~rprotacio/8154791.02 which is >>>> essentially the original code, but slightly cleaner and without >>>> redundancy of the info line inside the debug line. The only issue >>>> is it doesn't necessarily guarantee the two lines will come out >>>> next to each other, so we have to print the external name on each. >>>> >>>> [0.051s][info ][class,load] java.lang.CharSequence source: >>>> jrt:/java.basejava.lang.CharSequence loader: [NULL class_loader] >>>> bytes: 2045 checksum: 5b97c8c1 >>>> [0.051s][debug][class,load] java.lang.CharSequence klass: >>>> 0x00000007c00015a0 super: 0x00000007c0000fb0 >>>> [0.052s][info ][class,load] java.lang.String source: >>>> jrt:/java.basejava.lang.String loader: [NULL class_loader] bytes: >>>> 24872 checksum: 564a9fc2 >>>> [0.052s][debug][class,load] java.lang.String klass: >>>> 0x00000007c0001798 super: 0x00000007c0000fb0 interfaces: >>>> 0x00000007c00011b0 0x00000007c00013a8 0x00000007c00015a0 >>>> >>>> >>> >>> When two classes of the same name are loaded at the same time, and >>> the output is interleaved, you can't find out which is which: >>> >>> HelloWorld source: foo.jar >>> HelloWorld source: bar.jar >>> HelloWorld klass: 0x1234 >>> HelloWorld klass: 0x5678 >>> >>> E.g., is the klass loaded from foo.jar 0x1234 or 0x5678? >>> >>> Thanks >>> - Ioi >>> >>>> Or we can make the info output the same as debug, when debug is >>>> specified. Which I have reservations about but will certainly do if >>>> that's what makes the most sense. >>>> >>>> Thanks, >>>> Rachel >>>> >>>> On 5/3/2017 2:23 AM, Thomas St?fe wrote: >>>>> Hi Ioi, >>>>> >>>>> On Tue, May 2, 2017 at 11:10 PM, Ioi Lam wrote: >>>>> >>>>>> By the way, I think we should really fix this in the UL level -- >>>>>> have >>>>>> multi-line buffering, so you can prevent other threads from >>>>>> interleaving >>>>>> your input. Make it something like: >>>>>> >>>>>> Log(class, load).start_buffering(); >>>>>> log_info(class, load)(class name and source); >>>>>> log_debug(class, load)(other details); >>>>>> Log(class, load).flush(); >>>>>> >>>>>> That way, the output will look like this: >>>>>> >>>>>> [0.162s][debug ][class,load] java.lang.Comparable source: >>>>>> jrt:/java.base >>>>>> [0.162s][debug ][class,load] [NULL class_loader] >>>>>> 0x00000007c00013a8 super: >>>>>> 0x00000007c0000fb0 bytes: 235 checksum: a75dadb6 >>>>>> >>>>>> So the output will be terse, and still parsable. >>>>>> >>>>>> I think this will be useful for other types of logs as well, not >>>>>> just the >>>>>> class loading logs. >>>>>> >>>>>> >>>>> I thought that already exists: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8145934 >>>>> >>>>> see logMessage.hpp. >>>>> >>>>> Kind Regards, Thomas >>>>> >>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>> >>>>>> On 5/2/17 1:56 PM, Ioi Lam wrote: >>>>>> >>>>>>> >>>>>>> On 5/2/17 1:31 PM, Rachel Protacio wrote: >>>>>>> >>>>>>>> Hi Ioi, >>>>>>>> >>>>>>>> Thanks for your reply, comments inline. >>>>>>>> >>>>>>>> >>>>>>>> On 5/2/2017 2:48 PM, Ioi Lam wrote: >>>>>>>> >>>>>>>>> Hi Rachel, >>>>>>>>> >>>>>>>>> There are a few reasons why the current output has >>>>>>>>> duplications. The >>>>>>>>> "logging levels are not mutually exclusive" comment means this: >>>>>>>>> >>>>>>>>> java -Xlog:class+load=debug:file=debug.log >>>>>>>>> -Xlog:class+load=info:file=info.log >>>>>>>>> >>>>>>>>> So info.log contains only the info level logs, but debug.log >>>>>>>>> contains >>>>>>>>> both info and debug level logs. >>>>>>>>> >>>>>>>>> Ideally I want info.log to contain this: >>>>>>>>> >>>>>>>>> [0.162s][info ][class,load] java.lang.Comparable source: >>>>>>>>> jrt:/java.base >>>>>>>>> >>>>>>>>> .. and debug.log to contain this: >>>>>>>>> >>>>>>>>> [0.162s][debug ][class,load] java.lang.Comparable source: >>>>>>>>> jrt:/java.base [NULL class_loader] 0x00000007c00013a8 super: >>>>>>>>> 0x00000007c0000fb0 bytes: 235 >>>>>>>>> checksum: a75dadb6 >>>>>>>>> >>>>>>>> I see what you want, but I think the problem is that logging >>>>>>>> doesn't >>>>>>>> work that way. If debug is specified, both debug and info >>>>>>>> levels are >>>>>>>> printed. So the command >>>>>>>> >>>>>>>> java -Xlog:class+load=debug:file=debug.log >>>>>>>> >>>>>>>> will print both info level logging and debug level logging to this >>>>>>>> "debug.log" file. And then tacking on >>>>>>>> >>>>>>>> -Xlog:class+load=info:file=info.log >>>>>>>> >>>>>>>> in the command would print just the info level logging to the >>>>>>>> "info.log" >>>>>>>> file. But since it's all the same java command, it will always >>>>>>>> know that >>>>>>>> both debug and info levels are specified. I believe what you >>>>>>>> want me to do >>>>>>>> is say "if debug is specified, print everything to debug only" >>>>>>>> but then the >>>>>>>> info.log would be empty. You would have to run a separate java >>>>>>>> command with >>>>>>>> just the info level specified to get the smaller amount of >>>>>>>> information. >>>>>>>> >>>>>>> No, that's not what I want. I am perfectly fine with how UL >>>>>>> works today: >>>>>>> info.log contains only the info level, and debug.info contains both >>>>>>> debug and info level. >>>>>>> >>>>>>> All I want is everything related to the same class to be printed >>>>>>> on the >>>>>>> same line. For example, I don't want debug.log to contain this: >>>>>>> >>>>>>> [0.162s][info ][class,load] java.lang.Comparable source: >>>>>>> jrt:/java.base >>>>>>> [0.162s][debug ][class,load] [NULL class_loader] 0x00000007c00013a8 >>>>>>> super: 0x00000007c0000fb0 bytes: 235 checksum: a75dadb6 >>>>>>> >>>>>>> That's because it's impossible to associated the second with the >>>>>>> first >>>>>>> line when you have parallel threads loading classes concurrently. >>>>>>> >>>>>>> But I think that would violate user expectations - if they get >>>>>>> certain >>>>>>>> information with info logging, adding the extra level of debug >>>>>>>> logging >>>>>>>> should not change what they see from the info logging. >>>>>>>> >>>>>>>>> Your current patch splits out the debug log into several >>>>>>>>> lines. This >>>>>>>>> makes it difficult to analyze the log when classes are loaded >>>>>>>>> concurrently >>>>>>>>> on different threads: >>>>>>>>> >>>>>>>>> [0.162s][info ][class,load] java.lang.Comparable source: >>>>>>>>> jrt:/java.base >>>>>>>>> [0.162s][info ][class,load] java.lang.Comparable loader: [NULL >>>>>>>>> class_loader] >>>>>>>>> [0.162s][debug][class,load] java.lang.Comparable klass: >>>>>>>>> 0x00000007c00013a8 >>>>>>>>> [0.163s][info ][class,load] java.lang.String source: >>>>>>>>> jrt:/java.base <<< >>>>>>>>> oops! from a different thread >>>>>>>>> [0.164s][debug][class,load] java.lang.Comparable super: >>>>>>>>> 0x00000007c0000fb0 >>>>>>>>> [0.164s][info ][class,load] java.lang.Comparable bytes: 235 >>>>>>>>> [0.164s][info ][class,load] java.lang.Comparable checksum: >>>>>>>>> a75dadb6 >>>>>>>>> >>>>>>>> That's fair. I was trying to make it more readable since it's >>>>>>>> hard to >>>>>>>> visually parse one giant line. Maybe I could put some delimiter >>>>>>>> in between >>>>>>>> each piece of info? But if no one else cares, I can just put it >>>>>>>> back as one >>>>>>>> line, no delimiters. >>>>>>>> >>>>>>>>> Because the leveling in UL is by lines, in JDK-8079408, where >>>>>>>>> this was >>>>>>>>> first implemented, I couldn't find a way to print >>>>>>>>> "java.lang.Comparable >>>>>>>>> source: jrt:/java.base" in "info" mode, and print the rest in >>>>>>>>> "debug" mode, >>>>>>>>> while keeping the whole thing on the same line (in both the >>>>>>>>> info.log and >>>>>>>>> debug.log files). That's why we have the 2 lines with >>>>>>>>> duplicated info. >>>>>>>>> >>>>>>>>> I think it's really important to keep everything on the same >>>>>>>>> line (the >>>>>>>>> output needs to be processed automatically by scripts). With >>>>>>>>> the current UL >>>>>>>>> restrictions, I think the only way to do it is make the "info" >>>>>>>>> logging more >>>>>>>>> verbose when "debug" level is selected. E.g., >>>>>>>>> >>>>>>>>> outputStream* info_log = Log(class, load)::info_stream(); >>>>>>>>> info_log->print("%s", ext_name); >>>>>>>>> info_log->print_cr(" source: jrt:/%s", module_name); .... >>>>>>>>> >>>>>>>>> if (log_is_enabled(Debug, class, load)) { // always print >>>>>>>>> to info >>>>>>>>> log, even if debug is selected >>>>>>>>> info_log->print(" klass: " INTPTR_FORMAT, p2i(this)); >>>>>>>>> .... >>>>>>>>> } >>>>>>>>> >>>>>>>>> So now info.log and debug.log will have the same logs, but oh >>>>>>>>> well ..... >>>>>>>>> >>>>>>>> This sounds a little different. It sounds like >>>>>>>> >>>>>>>> When only info is specified, info log will contain: >>>>>>>> external name, >>>>>>>> source, loader, bytes, checksum >>>>>>>> >>>>>>>> When debug is specified, info log will contain the same as >>>>>>>> above, >>>>>>>> plus: klass, super, interfaces >>>>>>>> When debug is specified, debug log will be an exact >>>>>>>> duplicate of >>>>>>>> info log so it can print to an external file >>>>>>>> >>>>>>>> Did I understand that right? Please correct me. If this is what >>>>>>>> you >>>>>>>> meant, I would again argue that this violates the expectation >>>>>>>> that "info is >>>>>>>> info is info" regardless of what else is specified (not a >>>>>>>> written rule, but >>>>>>>> how I think it should work). And it is redundant, which is the >>>>>>>> original >>>>>>>> issue in the name of this RFE. >>>>>>>> >>>>>>>> I think the best bet would be >>>>>>>> >>>>>>>> Info log: external name, source, loader, bytes, checksum >>>>>>>> Debug log: external name, klass, super, interfaces >>>>>>>> >>>>>>>> Or we get rid of the level distinction completely and put all the >>>>>>>> information together in one line so it never gets broken up. >>>>>>>> Probably under >>>>>>>> debug. >>>>>>>> >>>>>>>> >>>>>>> We need to keep the current debug output intact, on a single >>>>>>> line, so >>>>>>> that the information can be processed by scripts. >>>>>>> >>>>>>> So, it seems like our 3 choices are: >>>>>>> >>>>>>> (1) make the info output the same as debug, if >>>>>>> -Xlog:class+load=debug is >>>>>>> specified >>>>>>> (2) remove the info output altogether >>>>>>> (3) do nothing >>>>>>> >>>>>>> You don't like (1). >>>>>>> >>>>>>> I don't think (2) achieves what this RFE wants -- namely, make >>>>>>> the output >>>>>>> less redundant, because you will see all the details all the >>>>>>> time, and you >>>>>>> don't even get a choice for more terse output. It also makes the >>>>>>> output of >>>>>>> -verbose different (and much less useful) than in JDK 9. >>>>>>> >>>>>>> So I would vote for (3), keep the output as is. Yeah, with debug >>>>>>> level >>>>>>> you have a lot of output, but that's pretty easy to filter. I >>>>>>> think we had >>>>>>> a discussion about this when 8079408 was implemented, and that >>>>>>> was the >>>>>>> conclusion we had. >>>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>>> >>>>>>> Let me know what you think. >>>>>>>> Thanks, >>>>>>>> Rachel >>>>>>>> >>>>>>>> >>>>>>>>> Thanks >>>>>>>>> - Ioi >>>>>>>>> >>>>>>>>> On 5/2/17 10:32 AM, Rachel Protacio wrote: >>>>>>>>> >>>>>>>>>> Hello! >>>>>>>>>> >>>>>>>>>> Please review this enhancement correcting redundancies and >>>>>>>>>> neatening >>>>>>>>>> up the -Xlog:class+load code. The redundancy in the code >>>>>>>>>> before I believe >>>>>>>>>> stemmed from a misunderstanding about logging levels (the >>>>>>>>>> comment saying >>>>>>>>>> they are not mutually exclusive was misleading). >>>>>>>>>> >>>>>>>>>> Tested with JPRT and RBT. >>>>>>>>>> >>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8154791 >>>>>>>>>> Webrev: http://cr.openjdk.java.net/~rprotacio/8154791.00 >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Rachel >>>>>>>>>> >>>>>>>>> >>>> >>> >> > From rachel.protacio at oracle.com Fri May 5 20:31:31 2017 From: rachel.protacio at oracle.com (Rachel Protacio) Date: Fri, 5 May 2017 16:31:31 -0400 Subject: RFR: 8154791: Xlog classload too redundant msgs info/debug In-Reply-To: <590CD92A.1080602@oracle.com> References: <6ff9f069-2da7-b725-d29f-e3d850650aca@oracle.com> <5908D481.2070104@oracle.com> <6a8ffd20-ce58-150b-a94d-c51c5956d191@oracle.com> <5908F288.9020205@oracle.com> <5908F5D4.6040505@oracle.com> <8ef74c07-1a6e-eede-d16c-4ba283a96199@oracle.com> <590BB2E8.1040308@oracle.com> <73701218-6b1e-0220-e6f0-8ba8114b4d26@oracle.com> <8c880811-fa82-d5b1-a9e7-42ef4fbb9eac@oracle.com> <590CD92A.1080602@oracle.com> Message-ID: <8bae3621-6e08-736c-cb36-ec96be4687e2@oracle.com> Thank you, Ioi! Rachel On 5/5/2017 3:57 PM, Ioi Lam wrote: > Looks good! > > Thanks > - Ioi > > On 5/5/17 11:34 AM, Rachel Protacio wrote: >> Thank you both for your help! Here's my latest webrev: >> http://cr.openjdk.java.net/~rprotacio/8154791.03 >> >> Rachel >> >> >> On 5/5/2017 4:28 AM, Marcus Larsson wrote: >>> Hi, >>> >>> >>> On 2017-05-05 01:02, Ioi Lam wrote: >>>> Hi Rachel, >>>> >>>> On 5/4/17 8:52 AM, Rachel Protacio wrote: >>>>> If I'm using this incorrectly, please let me know, but I think >>>>> LogBufferMessage also doesn't do exactly what we want. With this >>>>> changeset: http://cr.openjdk.java.net/~rprotacio/8154791.01/ we >>>>> get this output: >>>>> >>>>> [0.050s][info ][class,load] java.lang.CharSequence >>>>> [0.050s][info ][class,load] source: jrt:/java.base >>>>> [0.050s][info ][class,load] loader: >>>>> [AllocatedObj(0x00000007c00015a0)] >>>>> [0.050s][debug][class,load] klass: 0x00000007c00015a0 >>>>> [0.050s][debug][class,load] super: 0x00000007c0000fb0 >>>>> [0.050s][info ][class,load] bytes: 2045 checksum: 5b97c8c1 >>>>> [0.051s][info ][class,load] java.lang.String >>>>> [0.051s][info ][class,load] source: jrt:/java.base >>>>> [0.051s][info ][class,load] loader: >>>>> [AllocatedObj(0x00000007c0001798)] >>>>> [0.051s][debug][class,load] klass: 0x00000007c0001798 >>>>> [0.051s][debug][class,load] super: 0x00000007c0000fb0 >>>>> [0.051s][debug][class,load] interfaces: >>>>> [0.051s][debug][class,load] 0x00000007c00011b0 >>>>> [0.051s][debug][class,load] 0x00000007c00013a8 >>>>> [0.051s][debug][class,load] 0x00000007c00015a0 >>>>> [0.051s][info ][class,load] bytes: 24872 checksum: 564a9fc2 >>>>> >>>>> It comes out "together", but each call is a separate line. But to >>>>> put it all on one line would lead to messy code, something like >>>>> adding each part to two temporary strings and putting them in the >>>>> buffer at the end, I think. >>>>> >>>> >>>> I think we can use stringStream to collect the output together, and >>>> then write the whole line at once. Here's what I tried on top of >>>> your patch above: >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk10/8154791.01_log_class_load.delta/ >>>> >>> >>> Usage looks good to me, but it can be simplified by using LogMessage >>> instead of directly using the LogMessageBuffer. >>> >>> With LogMessage you would change >>> >>> 3065 LogMessageBuffer msg; >>> >>> into >>> >>> 3065 LogMessage(class, load) msg; >>> >>> and remove the following lines: >>> >>> 3133 Log(class, load) _log; >>> 3134 _log.write(msg); >>> >>> (The message gets written when the LogMessage object goes out of >>> scope.) >>> >>> Thanks, >>> Marcus >>> >>>> >>>> I also moved the class loader info back into "debug" . JDK-8154791 >>>> suggested moving that into "info", but I think that should be >>>> considered in a separate issue. Personally I that think would make >>>> the "info" output too verbose. >>>> >>>>> So the other option is to do this: >>>>> http://cr.openjdk.java.net/~rprotacio/8154791.02 which is >>>>> essentially the original code, but slightly cleaner and without >>>>> redundancy of the info line inside the debug line. The only issue >>>>> is it doesn't necessarily guarantee the two lines will come out >>>>> next to each other, so we have to print the external name on each. >>>>> >>>>> [0.051s][info ][class,load] java.lang.CharSequence source: >>>>> jrt:/java.basejava.lang.CharSequence loader: [NULL class_loader] >>>>> bytes: 2045 checksum: 5b97c8c1 >>>>> [0.051s][debug][class,load] java.lang.CharSequence klass: >>>>> 0x00000007c00015a0 super: 0x00000007c0000fb0 >>>>> [0.052s][info ][class,load] java.lang.String source: >>>>> jrt:/java.basejava.lang.String loader: [NULL class_loader] bytes: >>>>> 24872 checksum: 564a9fc2 >>>>> [0.052s][debug][class,load] java.lang.String klass: >>>>> 0x00000007c0001798 super: 0x00000007c0000fb0 interfaces: >>>>> 0x00000007c00011b0 0x00000007c00013a8 0x00000007c00015a0 >>>>> >>>>> >>>> >>>> When two classes of the same name are loaded at the same time, and >>>> the output is interleaved, you can't find out which is which: >>>> >>>> HelloWorld source: foo.jar >>>> HelloWorld source: bar.jar >>>> HelloWorld klass: 0x1234 >>>> HelloWorld klass: 0x5678 >>>> >>>> E.g., is the klass loaded from foo.jar 0x1234 or 0x5678? >>>> >>>> Thanks >>>> - Ioi >>>> >>>>> Or we can make the info output the same as debug, when debug is >>>>> specified. Which I have reservations about but will certainly do >>>>> if that's what makes the most sense. >>>>> >>>>> Thanks, >>>>> Rachel >>>>> >>>>> On 5/3/2017 2:23 AM, Thomas St?fe wrote: >>>>>> Hi Ioi, >>>>>> >>>>>> On Tue, May 2, 2017 at 11:10 PM, Ioi Lam wrote: >>>>>> >>>>>>> By the way, I think we should really fix this in the UL level -- >>>>>>> have >>>>>>> multi-line buffering, so you can prevent other threads from >>>>>>> interleaving >>>>>>> your input. Make it something like: >>>>>>> >>>>>>> Log(class, load).start_buffering(); >>>>>>> log_info(class, load)(class name and source); >>>>>>> log_debug(class, load)(other details); >>>>>>> Log(class, load).flush(); >>>>>>> >>>>>>> That way, the output will look like this: >>>>>>> >>>>>>> [0.162s][debug ][class,load] java.lang.Comparable source: >>>>>>> jrt:/java.base >>>>>>> [0.162s][debug ][class,load] [NULL class_loader] >>>>>>> 0x00000007c00013a8 super: >>>>>>> 0x00000007c0000fb0 bytes: 235 checksum: a75dadb6 >>>>>>> >>>>>>> So the output will be terse, and still parsable. >>>>>>> >>>>>>> I think this will be useful for other types of logs as well, not >>>>>>> just the >>>>>>> class loading logs. >>>>>>> >>>>>>> >>>>>> I thought that already exists: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8145934 >>>>>> >>>>>> see logMessage.hpp. >>>>>> >>>>>> Kind Regards, Thomas >>>>>> >>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>>> >>>>>>> >>>>>>> On 5/2/17 1:56 PM, Ioi Lam wrote: >>>>>>> >>>>>>>> >>>>>>>> On 5/2/17 1:31 PM, Rachel Protacio wrote: >>>>>>>> >>>>>>>>> Hi Ioi, >>>>>>>>> >>>>>>>>> Thanks for your reply, comments inline. >>>>>>>>> >>>>>>>>> >>>>>>>>> On 5/2/2017 2:48 PM, Ioi Lam wrote: >>>>>>>>> >>>>>>>>>> Hi Rachel, >>>>>>>>>> >>>>>>>>>> There are a few reasons why the current output has >>>>>>>>>> duplications. The >>>>>>>>>> "logging levels are not mutually exclusive" comment means this: >>>>>>>>>> >>>>>>>>>> java -Xlog:class+load=debug:file=debug.log >>>>>>>>>> -Xlog:class+load=info:file=info.log >>>>>>>>>> >>>>>>>>>> So info.log contains only the info level logs, but debug.log >>>>>>>>>> contains >>>>>>>>>> both info and debug level logs. >>>>>>>>>> >>>>>>>>>> Ideally I want info.log to contain this: >>>>>>>>>> >>>>>>>>>> [0.162s][info ][class,load] java.lang.Comparable source: >>>>>>>>>> jrt:/java.base >>>>>>>>>> >>>>>>>>>> .. and debug.log to contain this: >>>>>>>>>> >>>>>>>>>> [0.162s][debug ][class,load] java.lang.Comparable source: >>>>>>>>>> jrt:/java.base [NULL class_loader] 0x00000007c00013a8 super: >>>>>>>>>> 0x00000007c0000fb0 bytes: 235 >>>>>>>>>> checksum: a75dadb6 >>>>>>>>>> >>>>>>>>> I see what you want, but I think the problem is that logging >>>>>>>>> doesn't >>>>>>>>> work that way. If debug is specified, both debug and info >>>>>>>>> levels are >>>>>>>>> printed. So the command >>>>>>>>> >>>>>>>>> java -Xlog:class+load=debug:file=debug.log >>>>>>>>> >>>>>>>>> will print both info level logging and debug level logging to >>>>>>>>> this >>>>>>>>> "debug.log" file. And then tacking on >>>>>>>>> >>>>>>>>> -Xlog:class+load=info:file=info.log >>>>>>>>> >>>>>>>>> in the command would print just the info level logging to the >>>>>>>>> "info.log" >>>>>>>>> file. But since it's all the same java command, it will always >>>>>>>>> know that >>>>>>>>> both debug and info levels are specified. I believe what you >>>>>>>>> want me to do >>>>>>>>> is say "if debug is specified, print everything to debug only" >>>>>>>>> but then the >>>>>>>>> info.log would be empty. You would have to run a separate java >>>>>>>>> command with >>>>>>>>> just the info level specified to get the smaller amount of >>>>>>>>> information. >>>>>>>>> >>>>>>>> No, that's not what I want. I am perfectly fine with how UL >>>>>>>> works today: >>>>>>>> info.log contains only the info level, and debug.info contains >>>>>>>> both >>>>>>>> debug and info level. >>>>>>>> >>>>>>>> All I want is everything related to the same class to be >>>>>>>> printed on the >>>>>>>> same line. For example, I don't want debug.log to contain this: >>>>>>>> >>>>>>>> [0.162s][info ][class,load] java.lang.Comparable source: >>>>>>>> jrt:/java.base >>>>>>>> [0.162s][debug ][class,load] [NULL class_loader] >>>>>>>> 0x00000007c00013a8 >>>>>>>> super: 0x00000007c0000fb0 bytes: 235 checksum: a75dadb6 >>>>>>>> >>>>>>>> That's because it's impossible to associated the second with >>>>>>>> the first >>>>>>>> line when you have parallel threads loading classes concurrently. >>>>>>>> >>>>>>>> But I think that would violate user expectations - if they get >>>>>>>> certain >>>>>>>>> information with info logging, adding the extra level of debug >>>>>>>>> logging >>>>>>>>> should not change what they see from the info logging. >>>>>>>>> >>>>>>>>>> Your current patch splits out the debug log into several >>>>>>>>>> lines. This >>>>>>>>>> makes it difficult to analyze the log when classes are loaded >>>>>>>>>> concurrently >>>>>>>>>> on different threads: >>>>>>>>>> >>>>>>>>>> [0.162s][info ][class,load] java.lang.Comparable source: >>>>>>>>>> jrt:/java.base >>>>>>>>>> [0.162s][info ][class,load] java.lang.Comparable loader: [NULL >>>>>>>>>> class_loader] >>>>>>>>>> [0.162s][debug][class,load] java.lang.Comparable klass: >>>>>>>>>> 0x00000007c00013a8 >>>>>>>>>> [0.163s][info ][class,load] java.lang.String source: >>>>>>>>>> jrt:/java.base <<< >>>>>>>>>> oops! from a different thread >>>>>>>>>> [0.164s][debug][class,load] java.lang.Comparable super: >>>>>>>>>> 0x00000007c0000fb0 >>>>>>>>>> [0.164s][info ][class,load] java.lang.Comparable bytes: 235 >>>>>>>>>> [0.164s][info ][class,load] java.lang.Comparable checksum: >>>>>>>>>> a75dadb6 >>>>>>>>>> >>>>>>>>> That's fair. I was trying to make it more readable since it's >>>>>>>>> hard to >>>>>>>>> visually parse one giant line. Maybe I could put some >>>>>>>>> delimiter in between >>>>>>>>> each piece of info? But if no one else cares, I can just put >>>>>>>>> it back as one >>>>>>>>> line, no delimiters. >>>>>>>>> >>>>>>>>>> Because the leveling in UL is by lines, in JDK-8079408, where >>>>>>>>>> this was >>>>>>>>>> first implemented, I couldn't find a way to print >>>>>>>>>> "java.lang.Comparable >>>>>>>>>> source: jrt:/java.base" in "info" mode, and print the rest in >>>>>>>>>> "debug" mode, >>>>>>>>>> while keeping the whole thing on the same line (in both the >>>>>>>>>> info.log and >>>>>>>>>> debug.log files). That's why we have the 2 lines with >>>>>>>>>> duplicated info. >>>>>>>>>> >>>>>>>>>> I think it's really important to keep everything on the same >>>>>>>>>> line (the >>>>>>>>>> output needs to be processed automatically by scripts). With >>>>>>>>>> the current UL >>>>>>>>>> restrictions, I think the only way to do it is make the >>>>>>>>>> "info" logging more >>>>>>>>>> verbose when "debug" level is selected. E.g., >>>>>>>>>> >>>>>>>>>> outputStream* info_log = Log(class, load)::info_stream(); >>>>>>>>>> info_log->print("%s", ext_name); >>>>>>>>>> info_log->print_cr(" source: jrt:/%s", module_name); .... >>>>>>>>>> >>>>>>>>>> if (log_is_enabled(Debug, class, load)) { // always print >>>>>>>>>> to info >>>>>>>>>> log, even if debug is selected >>>>>>>>>> info_log->print(" klass: " INTPTR_FORMAT, p2i(this)); >>>>>>>>>> .... >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> So now info.log and debug.log will have the same logs, but oh >>>>>>>>>> well ..... >>>>>>>>>> >>>>>>>>> This sounds a little different. It sounds like >>>>>>>>> >>>>>>>>> When only info is specified, info log will contain: >>>>>>>>> external name, >>>>>>>>> source, loader, bytes, checksum >>>>>>>>> >>>>>>>>> When debug is specified, info log will contain the same as >>>>>>>>> above, >>>>>>>>> plus: klass, super, interfaces >>>>>>>>> When debug is specified, debug log will be an exact >>>>>>>>> duplicate of >>>>>>>>> info log so it can print to an external file >>>>>>>>> >>>>>>>>> Did I understand that right? Please correct me. If this is >>>>>>>>> what you >>>>>>>>> meant, I would again argue that this violates the expectation >>>>>>>>> that "info is >>>>>>>>> info is info" regardless of what else is specified (not a >>>>>>>>> written rule, but >>>>>>>>> how I think it should work). And it is redundant, which is the >>>>>>>>> original >>>>>>>>> issue in the name of this RFE. >>>>>>>>> >>>>>>>>> I think the best bet would be >>>>>>>>> >>>>>>>>> Info log: external name, source, loader, bytes, checksum >>>>>>>>> Debug log: external name, klass, super, interfaces >>>>>>>>> >>>>>>>>> Or we get rid of the level distinction completely and put all the >>>>>>>>> information together in one line so it never gets broken up. >>>>>>>>> Probably under >>>>>>>>> debug. >>>>>>>>> >>>>>>>>> >>>>>>>> We need to keep the current debug output intact, on a single >>>>>>>> line, so >>>>>>>> that the information can be processed by scripts. >>>>>>>> >>>>>>>> So, it seems like our 3 choices are: >>>>>>>> >>>>>>>> (1) make the info output the same as debug, if >>>>>>>> -Xlog:class+load=debug is >>>>>>>> specified >>>>>>>> (2) remove the info output altogether >>>>>>>> (3) do nothing >>>>>>>> >>>>>>>> You don't like (1). >>>>>>>> >>>>>>>> I don't think (2) achieves what this RFE wants -- namely, make >>>>>>>> the output >>>>>>>> less redundant, because you will see all the details all the >>>>>>>> time, and you >>>>>>>> don't even get a choice for more terse output. It also makes >>>>>>>> the output of >>>>>>>> -verbose different (and much less useful) than in JDK 9. >>>>>>>> >>>>>>>> So I would vote for (3), keep the output as is. Yeah, with >>>>>>>> debug level >>>>>>>> you have a lot of output, but that's pretty easy to filter. I >>>>>>>> think we had >>>>>>>> a discussion about this when 8079408 was implemented, and that >>>>>>>> was the >>>>>>>> conclusion we had. >>>>>>>> >>>>>>>> Thanks >>>>>>>> - Ioi >>>>>>>> >>>>>>>> Let me know what you think. >>>>>>>>> Thanks, >>>>>>>>> Rachel >>>>>>>>> >>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> - Ioi >>>>>>>>>> >>>>>>>>>> On 5/2/17 10:32 AM, Rachel Protacio wrote: >>>>>>>>>> >>>>>>>>>>> Hello! >>>>>>>>>>> >>>>>>>>>>> Please review this enhancement correcting redundancies and >>>>>>>>>>> neatening >>>>>>>>>>> up the -Xlog:class+load code. The redundancy in the code >>>>>>>>>>> before I believe >>>>>>>>>>> stemmed from a misunderstanding about logging levels (the >>>>>>>>>>> comment saying >>>>>>>>>>> they are not mutually exclusive was misleading). >>>>>>>>>>> >>>>>>>>>>> Tested with JPRT and RBT. >>>>>>>>>>> >>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8154791 >>>>>>>>>>> Webrev: http://cr.openjdk.java.net/~rprotacio/8154791.00 >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Rachel >>>>>>>>>>> >>>>>>>>>> >>>>> >>>> >>> >> > From david.holmes at oracle.com Mon May 8 03:47:02 2017 From: david.holmes at oracle.com (David Holmes) Date: Mon, 8 May 2017 13:47:02 +1000 Subject: RFR: 8154791: Xlog classload too redundant msgs info/debug In-Reply-To: <8c880811-fa82-d5b1-a9e7-42ef4fbb9eac@oracle.com> References: <6ff9f069-2da7-b725-d29f-e3d850650aca@oracle.com> <5908D481.2070104@oracle.com> <6a8ffd20-ce58-150b-a94d-c51c5956d191@oracle.com> <5908F288.9020205@oracle.com> <5908F5D4.6040505@oracle.com> <8ef74c07-1a6e-eede-d16c-4ba283a96199@oracle.com> <590BB2E8.1040308@oracle.com> <73701218-6b1e-0220-e6f0-8ba8114b4d26@oracle.com> <8c880811-fa82-d5b1-a9e7-42ef4fbb9eac@oracle.com> Message-ID: Hi Rachel, This looks okay to me too! Thanks, David On 6/05/2017 4:34 AM, Rachel Protacio wrote: > Thank you both for your help! Here's my latest webrev: > http://cr.openjdk.java.net/~rprotacio/8154791.03 > > Rachel > > > On 5/5/2017 4:28 AM, Marcus Larsson wrote: >> Hi, >> >> >> On 2017-05-05 01:02, Ioi Lam wrote: >>> Hi Rachel, >>> >>> On 5/4/17 8:52 AM, Rachel Protacio wrote: >>>> If I'm using this incorrectly, please let me know, but I think >>>> LogBufferMessage also doesn't do exactly what we want. With this >>>> changeset: http://cr.openjdk.java.net/~rprotacio/8154791.01/ we get >>>> this output: >>>> >>>> [0.050s][info ][class,load] java.lang.CharSequence >>>> [0.050s][info ][class,load] source: jrt:/java.base >>>> [0.050s][info ][class,load] loader: >>>> [AllocatedObj(0x00000007c00015a0)] >>>> [0.050s][debug][class,load] klass: 0x00000007c00015a0 >>>> [0.050s][debug][class,load] super: 0x00000007c0000fb0 >>>> [0.050s][info ][class,load] bytes: 2045 checksum: 5b97c8c1 >>>> [0.051s][info ][class,load] java.lang.String >>>> [0.051s][info ][class,load] source: jrt:/java.base >>>> [0.051s][info ][class,load] loader: >>>> [AllocatedObj(0x00000007c0001798)] >>>> [0.051s][debug][class,load] klass: 0x00000007c0001798 >>>> [0.051s][debug][class,load] super: 0x00000007c0000fb0 >>>> [0.051s][debug][class,load] interfaces: >>>> [0.051s][debug][class,load] 0x00000007c00011b0 >>>> [0.051s][debug][class,load] 0x00000007c00013a8 >>>> [0.051s][debug][class,load] 0x00000007c00015a0 >>>> [0.051s][info ][class,load] bytes: 24872 checksum: 564a9fc2 >>>> >>>> It comes out "together", but each call is a separate line. But to >>>> put it all on one line would lead to messy code, something like >>>> adding each part to two temporary strings and putting them in the >>>> buffer at the end, I think. >>>> >>> >>> I think we can use stringStream to collect the output together, and >>> then write the whole line at once. Here's what I tried on top of your >>> patch above: >>> >>> http://cr.openjdk.java.net/~iklam/jdk10/8154791.01_log_class_load.delta/ >> >> Usage looks good to me, but it can be simplified by using LogMessage >> instead of directly using the LogMessageBuffer. >> >> With LogMessage you would change >> >> 3065 LogMessageBuffer msg; >> >> into >> >> 3065 LogMessage(class, load) msg; >> >> and remove the following lines: >> >> 3133 Log(class, load) _log; >> 3134 _log.write(msg); >> >> (The message gets written when the LogMessage object goes out of scope.) >> >> Thanks, >> Marcus >> >>> >>> I also moved the class loader info back into "debug" . JDK-8154791 >>> suggested moving that into "info", but I think that should be >>> considered in a separate issue. Personally I that think would make >>> the "info" output too verbose. >>> >>>> So the other option is to do this: >>>> http://cr.openjdk.java.net/~rprotacio/8154791.02 which is >>>> essentially the original code, but slightly cleaner and without >>>> redundancy of the info line inside the debug line. The only issue is >>>> it doesn't necessarily guarantee the two lines will come out next to >>>> each other, so we have to print the external name on each. >>>> >>>> [0.051s][info ][class,load] java.lang.CharSequence source: >>>> jrt:/java.basejava.lang.CharSequence loader: [NULL class_loader] >>>> bytes: 2045 checksum: 5b97c8c1 >>>> [0.051s][debug][class,load] java.lang.CharSequence klass: >>>> 0x00000007c00015a0 super: 0x00000007c0000fb0 >>>> [0.052s][info ][class,load] java.lang.String source: >>>> jrt:/java.basejava.lang.String loader: [NULL class_loader] bytes: >>>> 24872 checksum: 564a9fc2 >>>> [0.052s][debug][class,load] java.lang.String klass: >>>> 0x00000007c0001798 super: 0x00000007c0000fb0 interfaces: >>>> 0x00000007c00011b0 0x00000007c00013a8 0x00000007c00015a0 >>>> >>>> >>> >>> When two classes of the same name are loaded at the same time, and >>> the output is interleaved, you can't find out which is which: >>> >>> HelloWorld source: foo.jar >>> HelloWorld source: bar.jar >>> HelloWorld klass: 0x1234 >>> HelloWorld klass: 0x5678 >>> >>> E.g., is the klass loaded from foo.jar 0x1234 or 0x5678? >>> >>> Thanks >>> - Ioi >>> >>>> Or we can make the info output the same as debug, when debug is >>>> specified. Which I have reservations about but will certainly do if >>>> that's what makes the most sense. >>>> >>>> Thanks, >>>> Rachel >>>> >>>> On 5/3/2017 2:23 AM, Thomas St?fe wrote: >>>>> Hi Ioi, >>>>> >>>>> On Tue, May 2, 2017 at 11:10 PM, Ioi Lam wrote: >>>>> >>>>>> By the way, I think we should really fix this in the UL level -- have >>>>>> multi-line buffering, so you can prevent other threads from >>>>>> interleaving >>>>>> your input. Make it something like: >>>>>> >>>>>> Log(class, load).start_buffering(); >>>>>> log_info(class, load)(class name and source); >>>>>> log_debug(class, load)(other details); >>>>>> Log(class, load).flush(); >>>>>> >>>>>> That way, the output will look like this: >>>>>> >>>>>> [0.162s][debug ][class,load] java.lang.Comparable source: >>>>>> jrt:/java.base >>>>>> [0.162s][debug ][class,load] [NULL class_loader] >>>>>> 0x00000007c00013a8 super: >>>>>> 0x00000007c0000fb0 bytes: 235 checksum: a75dadb6 >>>>>> >>>>>> So the output will be terse, and still parsable. >>>>>> >>>>>> I think this will be useful for other types of logs as well, not >>>>>> just the >>>>>> class loading logs. >>>>>> >>>>>> >>>>> I thought that already exists: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8145934 >>>>> >>>>> see logMessage.hpp. >>>>> >>>>> Kind Regards, Thomas >>>>> >>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>> >>>>>> On 5/2/17 1:56 PM, Ioi Lam wrote: >>>>>> >>>>>>> >>>>>>> On 5/2/17 1:31 PM, Rachel Protacio wrote: >>>>>>> >>>>>>>> Hi Ioi, >>>>>>>> >>>>>>>> Thanks for your reply, comments inline. >>>>>>>> >>>>>>>> >>>>>>>> On 5/2/2017 2:48 PM, Ioi Lam wrote: >>>>>>>> >>>>>>>>> Hi Rachel, >>>>>>>>> >>>>>>>>> There are a few reasons why the current output has >>>>>>>>> duplications. The >>>>>>>>> "logging levels are not mutually exclusive" comment means this: >>>>>>>>> >>>>>>>>> java -Xlog:class+load=debug:file=debug.log >>>>>>>>> -Xlog:class+load=info:file=info.log >>>>>>>>> >>>>>>>>> So info.log contains only the info level logs, but debug.log >>>>>>>>> contains >>>>>>>>> both info and debug level logs. >>>>>>>>> >>>>>>>>> Ideally I want info.log to contain this: >>>>>>>>> >>>>>>>>> [0.162s][info ][class,load] java.lang.Comparable source: >>>>>>>>> jrt:/java.base >>>>>>>>> >>>>>>>>> .. and debug.log to contain this: >>>>>>>>> >>>>>>>>> [0.162s][debug ][class,load] java.lang.Comparable source: >>>>>>>>> jrt:/java.base [NULL class_loader] 0x00000007c00013a8 super: >>>>>>>>> 0x00000007c0000fb0 bytes: 235 >>>>>>>>> checksum: a75dadb6 >>>>>>>>> >>>>>>>> I see what you want, but I think the problem is that logging >>>>>>>> doesn't >>>>>>>> work that way. If debug is specified, both debug and info levels >>>>>>>> are >>>>>>>> printed. So the command >>>>>>>> >>>>>>>> java -Xlog:class+load=debug:file=debug.log >>>>>>>> >>>>>>>> will print both info level logging and debug level logging to this >>>>>>>> "debug.log" file. And then tacking on >>>>>>>> >>>>>>>> -Xlog:class+load=info:file=info.log >>>>>>>> >>>>>>>> in the command would print just the info level logging to the >>>>>>>> "info.log" >>>>>>>> file. But since it's all the same java command, it will always >>>>>>>> know that >>>>>>>> both debug and info levels are specified. I believe what you >>>>>>>> want me to do >>>>>>>> is say "if debug is specified, print everything to debug only" >>>>>>>> but then the >>>>>>>> info.log would be empty. You would have to run a separate java >>>>>>>> command with >>>>>>>> just the info level specified to get the smaller amount of >>>>>>>> information. >>>>>>>> >>>>>>> No, that's not what I want. I am perfectly fine with how UL works >>>>>>> today: >>>>>>> info.log contains only the info level, and debug.info contains both >>>>>>> debug and info level. >>>>>>> >>>>>>> All I want is everything related to the same class to be printed >>>>>>> on the >>>>>>> same line. For example, I don't want debug.log to contain this: >>>>>>> >>>>>>> [0.162s][info ][class,load] java.lang.Comparable source: >>>>>>> jrt:/java.base >>>>>>> [0.162s][debug ][class,load] [NULL class_loader] 0x00000007c00013a8 >>>>>>> super: 0x00000007c0000fb0 bytes: 235 checksum: a75dadb6 >>>>>>> >>>>>>> That's because it's impossible to associated the second with the >>>>>>> first >>>>>>> line when you have parallel threads loading classes concurrently. >>>>>>> >>>>>>> But I think that would violate user expectations - if they get >>>>>>> certain >>>>>>>> information with info logging, adding the extra level of debug >>>>>>>> logging >>>>>>>> should not change what they see from the info logging. >>>>>>>> >>>>>>>>> Your current patch splits out the debug log into several lines. >>>>>>>>> This >>>>>>>>> makes it difficult to analyze the log when classes are loaded >>>>>>>>> concurrently >>>>>>>>> on different threads: >>>>>>>>> >>>>>>>>> [0.162s][info ][class,load] java.lang.Comparable source: >>>>>>>>> jrt:/java.base >>>>>>>>> [0.162s][info ][class,load] java.lang.Comparable loader: [NULL >>>>>>>>> class_loader] >>>>>>>>> [0.162s][debug][class,load] java.lang.Comparable klass: >>>>>>>>> 0x00000007c00013a8 >>>>>>>>> [0.163s][info ][class,load] java.lang.String source: >>>>>>>>> jrt:/java.base <<< >>>>>>>>> oops! from a different thread >>>>>>>>> [0.164s][debug][class,load] java.lang.Comparable super: >>>>>>>>> 0x00000007c0000fb0 >>>>>>>>> [0.164s][info ][class,load] java.lang.Comparable bytes: 235 >>>>>>>>> [0.164s][info ][class,load] java.lang.Comparable checksum: >>>>>>>>> a75dadb6 >>>>>>>>> >>>>>>>> That's fair. I was trying to make it more readable since it's >>>>>>>> hard to >>>>>>>> visually parse one giant line. Maybe I could put some delimiter >>>>>>>> in between >>>>>>>> each piece of info? But if no one else cares, I can just put it >>>>>>>> back as one >>>>>>>> line, no delimiters. >>>>>>>> >>>>>>>>> Because the leveling in UL is by lines, in JDK-8079408, where >>>>>>>>> this was >>>>>>>>> first implemented, I couldn't find a way to print >>>>>>>>> "java.lang.Comparable >>>>>>>>> source: jrt:/java.base" in "info" mode, and print the rest in >>>>>>>>> "debug" mode, >>>>>>>>> while keeping the whole thing on the same line (in both the >>>>>>>>> info.log and >>>>>>>>> debug.log files). That's why we have the 2 lines with >>>>>>>>> duplicated info. >>>>>>>>> >>>>>>>>> I think it's really important to keep everything on the same >>>>>>>>> line (the >>>>>>>>> output needs to be processed automatically by scripts). With >>>>>>>>> the current UL >>>>>>>>> restrictions, I think the only way to do it is make the "info" >>>>>>>>> logging more >>>>>>>>> verbose when "debug" level is selected. E.g., >>>>>>>>> >>>>>>>>> outputStream* info_log = Log(class, load)::info_stream(); >>>>>>>>> info_log->print("%s", ext_name); >>>>>>>>> info_log->print_cr(" source: jrt:/%s", module_name); .... >>>>>>>>> >>>>>>>>> if (log_is_enabled(Debug, class, load)) { // always print >>>>>>>>> to info >>>>>>>>> log, even if debug is selected >>>>>>>>> info_log->print(" klass: " INTPTR_FORMAT, p2i(this)); >>>>>>>>> .... >>>>>>>>> } >>>>>>>>> >>>>>>>>> So now info.log and debug.log will have the same logs, but oh >>>>>>>>> well ..... >>>>>>>>> >>>>>>>> This sounds a little different. It sounds like >>>>>>>> >>>>>>>> When only info is specified, info log will contain: external >>>>>>>> name, >>>>>>>> source, loader, bytes, checksum >>>>>>>> >>>>>>>> When debug is specified, info log will contain the same as >>>>>>>> above, >>>>>>>> plus: klass, super, interfaces >>>>>>>> When debug is specified, debug log will be an exact >>>>>>>> duplicate of >>>>>>>> info log so it can print to an external file >>>>>>>> >>>>>>>> Did I understand that right? Please correct me. If this is what you >>>>>>>> meant, I would again argue that this violates the expectation >>>>>>>> that "info is >>>>>>>> info is info" regardless of what else is specified (not a >>>>>>>> written rule, but >>>>>>>> how I think it should work). And it is redundant, which is the >>>>>>>> original >>>>>>>> issue in the name of this RFE. >>>>>>>> >>>>>>>> I think the best bet would be >>>>>>>> >>>>>>>> Info log: external name, source, loader, bytes, checksum >>>>>>>> Debug log: external name, klass, super, interfaces >>>>>>>> >>>>>>>> Or we get rid of the level distinction completely and put all the >>>>>>>> information together in one line so it never gets broken up. >>>>>>>> Probably under >>>>>>>> debug. >>>>>>>> >>>>>>>> >>>>>>> We need to keep the current debug output intact, on a single >>>>>>> line, so >>>>>>> that the information can be processed by scripts. >>>>>>> >>>>>>> So, it seems like our 3 choices are: >>>>>>> >>>>>>> (1) make the info output the same as debug, if >>>>>>> -Xlog:class+load=debug is >>>>>>> specified >>>>>>> (2) remove the info output altogether >>>>>>> (3) do nothing >>>>>>> >>>>>>> You don't like (1). >>>>>>> >>>>>>> I don't think (2) achieves what this RFE wants -- namely, make >>>>>>> the output >>>>>>> less redundant, because you will see all the details all the >>>>>>> time, and you >>>>>>> don't even get a choice for more terse output. It also makes the >>>>>>> output of >>>>>>> -verbose different (and much less useful) than in JDK 9. >>>>>>> >>>>>>> So I would vote for (3), keep the output as is. Yeah, with debug >>>>>>> level >>>>>>> you have a lot of output, but that's pretty easy to filter. I >>>>>>> think we had >>>>>>> a discussion about this when 8079408 was implemented, and that >>>>>>> was the >>>>>>> conclusion we had. >>>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>>> >>>>>>> Let me know what you think. >>>>>>>> Thanks, >>>>>>>> Rachel >>>>>>>> >>>>>>>> >>>>>>>>> Thanks >>>>>>>>> - Ioi >>>>>>>>> >>>>>>>>> On 5/2/17 10:32 AM, Rachel Protacio wrote: >>>>>>>>> >>>>>>>>>> Hello! >>>>>>>>>> >>>>>>>>>> Please review this enhancement correcting redundancies and >>>>>>>>>> neatening >>>>>>>>>> up the -Xlog:class+load code. The redundancy in the code >>>>>>>>>> before I believe >>>>>>>>>> stemmed from a misunderstanding about logging levels (the >>>>>>>>>> comment saying >>>>>>>>>> they are not mutually exclusive was misleading). >>>>>>>>>> >>>>>>>>>> Tested with JPRT and RBT. >>>>>>>>>> >>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8154791 >>>>>>>>>> Webrev: http://cr.openjdk.java.net/~rprotacio/8154791.00 >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Rachel >>>>>>>>>> >>>>>>>>> >>>> >>> >> > From rachel.protacio at oracle.com Mon May 8 13:34:02 2017 From: rachel.protacio at oracle.com (Rachel Protacio) Date: Mon, 8 May 2017 09:34:02 -0400 Subject: RFR: 8154791: Xlog classload too redundant msgs info/debug In-Reply-To: References: <6ff9f069-2da7-b725-d29f-e3d850650aca@oracle.com> <5908D481.2070104@oracle.com> <6a8ffd20-ce58-150b-a94d-c51c5956d191@oracle.com> <5908F288.9020205@oracle.com> <5908F5D4.6040505@oracle.com> <8ef74c07-1a6e-eede-d16c-4ba283a96199@oracle.com> <590BB2E8.1040308@oracle.com> <73701218-6b1e-0220-e6f0-8ba8114b4d26@oracle.com> <8c880811-fa82-d5b1-a9e7-42ef4fbb9eac@oracle.com> Message-ID: Thank you, David! I'll check it in. Rachel On 5/7/2017 11:47 PM, David Holmes wrote: > Hi Rachel, > > This looks okay to me too! > > Thanks, > David > > On 6/05/2017 4:34 AM, Rachel Protacio wrote: >> Thank you both for your help! Here's my latest webrev: >> http://cr.openjdk.java.net/~rprotacio/8154791.03 >> >> Rachel >> >> >> On 5/5/2017 4:28 AM, Marcus Larsson wrote: >>> Hi, >>> >>> >>> On 2017-05-05 01:02, Ioi Lam wrote: >>>> Hi Rachel, >>>> >>>> On 5/4/17 8:52 AM, Rachel Protacio wrote: >>>>> If I'm using this incorrectly, please let me know, but I think >>>>> LogBufferMessage also doesn't do exactly what we want. With this >>>>> changeset: http://cr.openjdk.java.net/~rprotacio/8154791.01/ we get >>>>> this output: >>>>> >>>>> [0.050s][info ][class,load] java.lang.CharSequence >>>>> [0.050s][info ][class,load] source: jrt:/java.base >>>>> [0.050s][info ][class,load] loader: >>>>> [AllocatedObj(0x00000007c00015a0)] >>>>> [0.050s][debug][class,load] klass: 0x00000007c00015a0 >>>>> [0.050s][debug][class,load] super: 0x00000007c0000fb0 >>>>> [0.050s][info ][class,load] bytes: 2045 checksum: 5b97c8c1 >>>>> [0.051s][info ][class,load] java.lang.String >>>>> [0.051s][info ][class,load] source: jrt:/java.base >>>>> [0.051s][info ][class,load] loader: >>>>> [AllocatedObj(0x00000007c0001798)] >>>>> [0.051s][debug][class,load] klass: 0x00000007c0001798 >>>>> [0.051s][debug][class,load] super: 0x00000007c0000fb0 >>>>> [0.051s][debug][class,load] interfaces: >>>>> [0.051s][debug][class,load] 0x00000007c00011b0 >>>>> [0.051s][debug][class,load] 0x00000007c00013a8 >>>>> [0.051s][debug][class,load] 0x00000007c00015a0 >>>>> [0.051s][info ][class,load] bytes: 24872 checksum: 564a9fc2 >>>>> >>>>> It comes out "together", but each call is a separate line. But to >>>>> put it all on one line would lead to messy code, something like >>>>> adding each part to two temporary strings and putting them in the >>>>> buffer at the end, I think. >>>>> >>>> >>>> I think we can use stringStream to collect the output together, and >>>> then write the whole line at once. Here's what I tried on top of your >>>> patch above: >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk10/8154791.01_log_class_load.delta/ >>>> >>> >>> Usage looks good to me, but it can be simplified by using LogMessage >>> instead of directly using the LogMessageBuffer. >>> >>> With LogMessage you would change >>> >>> 3065 LogMessageBuffer msg; >>> >>> into >>> >>> 3065 LogMessage(class, load) msg; >>> >>> and remove the following lines: >>> >>> 3133 Log(class, load) _log; >>> 3134 _log.write(msg); >>> >>> (The message gets written when the LogMessage object goes out of >>> scope.) >>> >>> Thanks, >>> Marcus >>> >>>> >>>> I also moved the class loader info back into "debug" . JDK-8154791 >>>> suggested moving that into "info", but I think that should be >>>> considered in a separate issue. Personally I that think would make >>>> the "info" output too verbose. >>>> >>>>> So the other option is to do this: >>>>> http://cr.openjdk.java.net/~rprotacio/8154791.02 which is >>>>> essentially the original code, but slightly cleaner and without >>>>> redundancy of the info line inside the debug line. The only issue is >>>>> it doesn't necessarily guarantee the two lines will come out next to >>>>> each other, so we have to print the external name on each. >>>>> >>>>> [0.051s][info ][class,load] java.lang.CharSequence source: >>>>> jrt:/java.basejava.lang.CharSequence loader: [NULL class_loader] >>>>> bytes: 2045 checksum: 5b97c8c1 >>>>> [0.051s][debug][class,load] java.lang.CharSequence klass: >>>>> 0x00000007c00015a0 super: 0x00000007c0000fb0 >>>>> [0.052s][info ][class,load] java.lang.String source: >>>>> jrt:/java.basejava.lang.String loader: [NULL class_loader] bytes: >>>>> 24872 checksum: 564a9fc2 >>>>> [0.052s][debug][class,load] java.lang.String klass: >>>>> 0x00000007c0001798 super: 0x00000007c0000fb0 interfaces: >>>>> 0x00000007c00011b0 0x00000007c00013a8 0x00000007c00015a0 >>>>> >>>>> >>>> >>>> When two classes of the same name are loaded at the same time, and >>>> the output is interleaved, you can't find out which is which: >>>> >>>> HelloWorld source: foo.jar >>>> HelloWorld source: bar.jar >>>> HelloWorld klass: 0x1234 >>>> HelloWorld klass: 0x5678 >>>> >>>> E.g., is the klass loaded from foo.jar 0x1234 or 0x5678? >>>> >>>> Thanks >>>> - Ioi >>>> >>>>> Or we can make the info output the same as debug, when debug is >>>>> specified. Which I have reservations about but will certainly do if >>>>> that's what makes the most sense. >>>>> >>>>> Thanks, >>>>> Rachel >>>>> >>>>> On 5/3/2017 2:23 AM, Thomas St?fe wrote: >>>>>> Hi Ioi, >>>>>> >>>>>> On Tue, May 2, 2017 at 11:10 PM, Ioi Lam wrote: >>>>>> >>>>>>> By the way, I think we should really fix this in the UL level -- >>>>>>> have >>>>>>> multi-line buffering, so you can prevent other threads from >>>>>>> interleaving >>>>>>> your input. Make it something like: >>>>>>> >>>>>>> Log(class, load).start_buffering(); >>>>>>> log_info(class, load)(class name and source); >>>>>>> log_debug(class, load)(other details); >>>>>>> Log(class, load).flush(); >>>>>>> >>>>>>> That way, the output will look like this: >>>>>>> >>>>>>> [0.162s][debug ][class,load] java.lang.Comparable source: >>>>>>> jrt:/java.base >>>>>>> [0.162s][debug ][class,load] [NULL class_loader] >>>>>>> 0x00000007c00013a8 super: >>>>>>> 0x00000007c0000fb0 bytes: 235 checksum: a75dadb6 >>>>>>> >>>>>>> So the output will be terse, and still parsable. >>>>>>> >>>>>>> I think this will be useful for other types of logs as well, not >>>>>>> just the >>>>>>> class loading logs. >>>>>>> >>>>>>> >>>>>> I thought that already exists: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8145934 >>>>>> >>>>>> see logMessage.hpp. >>>>>> >>>>>> Kind Regards, Thomas >>>>>> >>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>>> >>>>>>> >>>>>>> On 5/2/17 1:56 PM, Ioi Lam wrote: >>>>>>> >>>>>>>> >>>>>>>> On 5/2/17 1:31 PM, Rachel Protacio wrote: >>>>>>>> >>>>>>>>> Hi Ioi, >>>>>>>>> >>>>>>>>> Thanks for your reply, comments inline. >>>>>>>>> >>>>>>>>> >>>>>>>>> On 5/2/2017 2:48 PM, Ioi Lam wrote: >>>>>>>>> >>>>>>>>>> Hi Rachel, >>>>>>>>>> >>>>>>>>>> There are a few reasons why the current output has >>>>>>>>>> duplications. The >>>>>>>>>> "logging levels are not mutually exclusive" comment means this: >>>>>>>>>> >>>>>>>>>> java -Xlog:class+load=debug:file=debug.log >>>>>>>>>> -Xlog:class+load=info:file=info.log >>>>>>>>>> >>>>>>>>>> So info.log contains only the info level logs, but debug.log >>>>>>>>>> contains >>>>>>>>>> both info and debug level logs. >>>>>>>>>> >>>>>>>>>> Ideally I want info.log to contain this: >>>>>>>>>> >>>>>>>>>> [0.162s][info ][class,load] java.lang.Comparable source: >>>>>>>>>> jrt:/java.base >>>>>>>>>> >>>>>>>>>> .. and debug.log to contain this: >>>>>>>>>> >>>>>>>>>> [0.162s][debug ][class,load] java.lang.Comparable source: >>>>>>>>>> jrt:/java.base [NULL class_loader] 0x00000007c00013a8 super: >>>>>>>>>> 0x00000007c0000fb0 bytes: 235 >>>>>>>>>> checksum: a75dadb6 >>>>>>>>>> >>>>>>>>> I see what you want, but I think the problem is that logging >>>>>>>>> doesn't >>>>>>>>> work that way. If debug is specified, both debug and info levels >>>>>>>>> are >>>>>>>>> printed. So the command >>>>>>>>> >>>>>>>>> java -Xlog:class+load=debug:file=debug.log >>>>>>>>> >>>>>>>>> will print both info level logging and debug level logging to >>>>>>>>> this >>>>>>>>> "debug.log" file. And then tacking on >>>>>>>>> >>>>>>>>> -Xlog:class+load=info:file=info.log >>>>>>>>> >>>>>>>>> in the command would print just the info level logging to the >>>>>>>>> "info.log" >>>>>>>>> file. But since it's all the same java command, it will always >>>>>>>>> know that >>>>>>>>> both debug and info levels are specified. I believe what you >>>>>>>>> want me to do >>>>>>>>> is say "if debug is specified, print everything to debug only" >>>>>>>>> but then the >>>>>>>>> info.log would be empty. You would have to run a separate java >>>>>>>>> command with >>>>>>>>> just the info level specified to get the smaller amount of >>>>>>>>> information. >>>>>>>>> >>>>>>>> No, that's not what I want. I am perfectly fine with how UL works >>>>>>>> today: >>>>>>>> info.log contains only the info level, and debug.info contains >>>>>>>> both >>>>>>>> debug and info level. >>>>>>>> >>>>>>>> All I want is everything related to the same class to be printed >>>>>>>> on the >>>>>>>> same line. For example, I don't want debug.log to contain this: >>>>>>>> >>>>>>>> [0.162s][info ][class,load] java.lang.Comparable source: >>>>>>>> jrt:/java.base >>>>>>>> [0.162s][debug ][class,load] [NULL class_loader] >>>>>>>> 0x00000007c00013a8 >>>>>>>> super: 0x00000007c0000fb0 bytes: 235 checksum: a75dadb6 >>>>>>>> >>>>>>>> That's because it's impossible to associated the second with the >>>>>>>> first >>>>>>>> line when you have parallel threads loading classes concurrently. >>>>>>>> >>>>>>>> But I think that would violate user expectations - if they get >>>>>>>> certain >>>>>>>>> information with info logging, adding the extra level of debug >>>>>>>>> logging >>>>>>>>> should not change what they see from the info logging. >>>>>>>>> >>>>>>>>>> Your current patch splits out the debug log into several lines. >>>>>>>>>> This >>>>>>>>>> makes it difficult to analyze the log when classes are loaded >>>>>>>>>> concurrently >>>>>>>>>> on different threads: >>>>>>>>>> >>>>>>>>>> [0.162s][info ][class,load] java.lang.Comparable source: >>>>>>>>>> jrt:/java.base >>>>>>>>>> [0.162s][info ][class,load] java.lang.Comparable loader: [NULL >>>>>>>>>> class_loader] >>>>>>>>>> [0.162s][debug][class,load] java.lang.Comparable klass: >>>>>>>>>> 0x00000007c00013a8 >>>>>>>>>> [0.163s][info ][class,load] java.lang.String source: >>>>>>>>>> jrt:/java.base <<< >>>>>>>>>> oops! from a different thread >>>>>>>>>> [0.164s][debug][class,load] java.lang.Comparable super: >>>>>>>>>> 0x00000007c0000fb0 >>>>>>>>>> [0.164s][info ][class,load] java.lang.Comparable bytes: 235 >>>>>>>>>> [0.164s][info ][class,load] java.lang.Comparable checksum: >>>>>>>>>> a75dadb6 >>>>>>>>>> >>>>>>>>> That's fair. I was trying to make it more readable since it's >>>>>>>>> hard to >>>>>>>>> visually parse one giant line. Maybe I could put some delimiter >>>>>>>>> in between >>>>>>>>> each piece of info? But if no one else cares, I can just put it >>>>>>>>> back as one >>>>>>>>> line, no delimiters. >>>>>>>>> >>>>>>>>>> Because the leveling in UL is by lines, in JDK-8079408, where >>>>>>>>>> this was >>>>>>>>>> first implemented, I couldn't find a way to print >>>>>>>>>> "java.lang.Comparable >>>>>>>>>> source: jrt:/java.base" in "info" mode, and print the rest in >>>>>>>>>> "debug" mode, >>>>>>>>>> while keeping the whole thing on the same line (in both the >>>>>>>>>> info.log and >>>>>>>>>> debug.log files). That's why we have the 2 lines with >>>>>>>>>> duplicated info. >>>>>>>>>> >>>>>>>>>> I think it's really important to keep everything on the same >>>>>>>>>> line (the >>>>>>>>>> output needs to be processed automatically by scripts). With >>>>>>>>>> the current UL >>>>>>>>>> restrictions, I think the only way to do it is make the "info" >>>>>>>>>> logging more >>>>>>>>>> verbose when "debug" level is selected. E.g., >>>>>>>>>> >>>>>>>>>> outputStream* info_log = Log(class, load)::info_stream(); >>>>>>>>>> info_log->print("%s", ext_name); >>>>>>>>>> info_log->print_cr(" source: jrt:/%s", module_name); .... >>>>>>>>>> >>>>>>>>>> if (log_is_enabled(Debug, class, load)) { // always print >>>>>>>>>> to info >>>>>>>>>> log, even if debug is selected >>>>>>>>>> info_log->print(" klass: " INTPTR_FORMAT, p2i(this)); >>>>>>>>>> .... >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> So now info.log and debug.log will have the same logs, but oh >>>>>>>>>> well ..... >>>>>>>>>> >>>>>>>>> This sounds a little different. It sounds like >>>>>>>>> >>>>>>>>> When only info is specified, info log will contain: external >>>>>>>>> name, >>>>>>>>> source, loader, bytes, checksum >>>>>>>>> >>>>>>>>> When debug is specified, info log will contain the same as >>>>>>>>> above, >>>>>>>>> plus: klass, super, interfaces >>>>>>>>> When debug is specified, debug log will be an exact >>>>>>>>> duplicate of >>>>>>>>> info log so it can print to an external file >>>>>>>>> >>>>>>>>> Did I understand that right? Please correct me. If this is >>>>>>>>> what you >>>>>>>>> meant, I would again argue that this violates the expectation >>>>>>>>> that "info is >>>>>>>>> info is info" regardless of what else is specified (not a >>>>>>>>> written rule, but >>>>>>>>> how I think it should work). And it is redundant, which is the >>>>>>>>> original >>>>>>>>> issue in the name of this RFE. >>>>>>>>> >>>>>>>>> I think the best bet would be >>>>>>>>> >>>>>>>>> Info log: external name, source, loader, bytes, checksum >>>>>>>>> Debug log: external name, klass, super, interfaces >>>>>>>>> >>>>>>>>> Or we get rid of the level distinction completely and put all the >>>>>>>>> information together in one line so it never gets broken up. >>>>>>>>> Probably under >>>>>>>>> debug. >>>>>>>>> >>>>>>>>> >>>>>>>> We need to keep the current debug output intact, on a single >>>>>>>> line, so >>>>>>>> that the information can be processed by scripts. >>>>>>>> >>>>>>>> So, it seems like our 3 choices are: >>>>>>>> >>>>>>>> (1) make the info output the same as debug, if >>>>>>>> -Xlog:class+load=debug is >>>>>>>> specified >>>>>>>> (2) remove the info output altogether >>>>>>>> (3) do nothing >>>>>>>> >>>>>>>> You don't like (1). >>>>>>>> >>>>>>>> I don't think (2) achieves what this RFE wants -- namely, make >>>>>>>> the output >>>>>>>> less redundant, because you will see all the details all the >>>>>>>> time, and you >>>>>>>> don't even get a choice for more terse output. It also makes the >>>>>>>> output of >>>>>>>> -verbose different (and much less useful) than in JDK 9. >>>>>>>> >>>>>>>> So I would vote for (3), keep the output as is. Yeah, with debug >>>>>>>> level >>>>>>>> you have a lot of output, but that's pretty easy to filter. I >>>>>>>> think we had >>>>>>>> a discussion about this when 8079408 was implemented, and that >>>>>>>> was the >>>>>>>> conclusion we had. >>>>>>>> >>>>>>>> Thanks >>>>>>>> - Ioi >>>>>>>> >>>>>>>> Let me know what you think. >>>>>>>>> Thanks, >>>>>>>>> Rachel >>>>>>>>> >>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> - Ioi >>>>>>>>>> >>>>>>>>>> On 5/2/17 10:32 AM, Rachel Protacio wrote: >>>>>>>>>> >>>>>>>>>>> Hello! >>>>>>>>>>> >>>>>>>>>>> Please review this enhancement correcting redundancies and >>>>>>>>>>> neatening >>>>>>>>>>> up the -Xlog:class+load code. The redundancy in the code >>>>>>>>>>> before I believe >>>>>>>>>>> stemmed from a misunderstanding about logging levels (the >>>>>>>>>>> comment saying >>>>>>>>>>> they are not mutually exclusive was misleading). >>>>>>>>>>> >>>>>>>>>>> Tested with JPRT and RBT. >>>>>>>>>>> >>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8154791 >>>>>>>>>>> Webrev: http://cr.openjdk.java.net/~rprotacio/8154791.00 >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Rachel >>>>>>>>>>> >>>>>>>>>> >>>>> >>>> >>> >> From harold.seigel at oracle.com Mon May 8 18:00:09 2017 From: harold.seigel at oracle.com (harold seigel) Date: Mon, 8 May 2017 14:00:09 -0400 Subject: RFR 8178604: JVM does not allow defining boot loader modules in exploded build after module system initialization In-Reply-To: <21b3fdc5-d8a7-0535-3ac6-f1a097a7431d@oracle.com> References: <85e36988-a145-6805-8659-56c738013f58@oracle.com> <46a812ca-228d-01e7-e889-fd22f2af0537@oracle.com> <11709d4a-0a41-1d38-fcd1-67d42c33eaeb@oracle.com> <3ee613ff-f6da-9ef8-6781-efce4edba29e@oracle.com> <21b3fdc5-d8a7-0535-3ac6-f1a097a7431d@oracle.com> Message-ID: <388b80ed-e436-f888-ffed-5b92e0117ce2@oracle.com> Hi David, Please see this updated webrev: http://cr.openjdk.java.net/~hseigel/bug_8178604.3/webrev/index.html It contains the changes you requested below. Note that the _exploded_entries table gets created very early during SystemDictionary initialization, before initializing the pre-loaded classes. So, the code in load_class() at line ~1503 can assume that _exploded_entries is not null. Thanks, Harold On 5/4/2017 5:32 PM, David Holmes wrote: > Hi Harold, > > On 5/05/2017 3:47 AM, harold seigel wrote: >> Hi, >> >> Please review this updated webrev: >> >> http://cr.openjdk.java.net/~hseigel/bug_8178604.2/webrev/index.html >> >> The updated webrev takes out a lock when creating _exploded_entries. It >> also contains additional comments and an assert() to address Lois's >> concerns with reduced code readability involving what situations warrant >> taking out a lock in ClassLoader::search_module_entries(). >> >> I don't think there needs to be two different initial entry points as >> suggested by David below. The method behaves slightly differently >> depending on the value of its 'need_lock' parameter. But, there is >> enough common code for it to remain one method. > > I think this highlights uncertainty about the concurrent behaviour in > relation to this code. On the one hand you are worried about an > initialization race and so use the lock, but when you do: > > 1500 // The exploded build entries can be added to at any time > so pass 'true' for > 1501 // need_lock so that a lock is taken out when searching them. > 1502 assert(_exploded_entries != NULL, "No exploded build > entries present"); > 1503 stream = search_module_entries(_exploded_entries, > class_name, file_name, true, CHECK_NULL); > > you load _exploded_entries with no lock held and so must be certain > that you can not possibly read a null value here. > > The needs_lock parameter seems superfluous - you know the condition > for locking is that the list is the _exploded_entries, and you assert > that. So no need to pass in needs_lock, just grab the lock when > dealing with _exploded_entries. > > Cheers, > David > >> Thanks, Harold >> >> On 5/3/2017 12:59 AM, David Holmes wrote: >>> On 3/05/2017 7:11 AM, Lois Foltan wrote: >>>> On 5/2/2017 9:03 AM, harold seigel wrote: >>>>> Hi Lois, >>>>> >>>>> Thanks for your comments. Please see my in-line responses. >>>>> >>>>> Harold >>>>> >>>>> >>>>> On 5/1/2017 4:14 PM, Lois Foltan wrote: >>>>>> Hi Harold, >>>>>> >>>>>> Looks good. A couple of comments: >>>>>> >>>>>> src/share/vm/classfile/classLoader.cpp >>>>>> line #828 - please take out the Module_lock when the >>>>>> _exploded_entries is created >>>>> This fix does not change the possible need for synchronization when >>>>> _exploded_entries gets created during module system initialization. >>>>> Are you saying that the existing code is wrong and should have taken >>>>> out the Module_lock before creating _exploding_entries? >>>> >>>> The method ClassLoader::add_to_exploded_build_list can be called at >>>> anytime (during module system initialization and post module system >>>> initialization). Thus it seems prudent that no matter what the >>>> current >>>> JVM phase, that any access to ClassLoader::_exploding_entries be >>>> protected via a lock. That includes creation as well as insertion >>>> into >>>> the list. >>> >>> If creation is not guaranteed to occur before other threads that will >>> call the same method and use the _exploded_entries list, then >>> synchronization of some form is essential. If it is guaranteed then we >>> don't need create time sync just to be prudent - but the guarantee >>> must be firmly established and clearly documented. >>> >>>>> >>>>>> line #1402 - I like the new factored out method >>>>>> find_first_module_cpe(), however, instead of adding a new >>>>>> "need_lock" >>>>>> parameter, I would rather see code >>>>>> in find_first_module_cpe() that takes the >>>>>> Module_lock if the module_list parameter equals >>>>>> ClassLoader::_exploded_entries >>>>> I think your suggestion makes the code less flexible and does not >>>>> require potential future callers of search_module_entries() to think >>>>> about synchronization. So, I'd prefer to leave that change as is. >>>> >>>> I see your point. My point was based on reduced code readability, the >>>> further away from the decision to establish the lock, the less >>>> apparent >>>> it is within the new method ClassLoader::find_first_module_cpe() as to >>>> what situations warranted the lock in the first place. >>> >>> Further, this: >>> >>> 1495 stream = search_module_entries(_exploded_entries, >>> class_name, file_name, true, CHECK_NULL); >>> >>> is potentially unsafe if we can race with creation because we first >>> access _exploded_entries without holding any lock! And the fact the >>> method needs to do different things depending on which "list" was >>> passed in suggests to me it should be two different initial entry >>> points. >>> >>> Thanks, >>> David >>> ----- >>> >>>> Thanks, >>>> Lois >>>> >>>>>> >>>>>> Thanks, >>>>>> Lois >>>>>> >>>>>> On 5/1/2017 3:36 PM, harold seigel wrote: >>>>>>> Hi, >>>>>>> >>>>>>> Please review this JDK-10 bug fix to allow defining of modules to >>>>>>> the boot loader in exploded builds after module system >>>>>>> initialization. The fix uses the module lock to synchronize access >>>>>>> to the _exploded_entries data structure (which is only used by >>>>>>> exploded builds). >>>>>>> >>>>>>> Note that the above capability already exists for regular builds. >>>>>>> >>>>>>> Open Webrev: >>>>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604/webrev/index.html >>>>>>> >>>>>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8178604 >>>>>>> >>>>>>> The fix was tested with JCK tests, the JTreg hotspot, java/io, >>>>>>> java/lang, java/util and other tests, the RBT tier2 -tier5 tests, >>>>>>> the co-located NSK tests, and with JPRT. The JTReg and JCK tests >>>>>>> were run with an exploded build. Also, the example program in the >>>>>>> JBS bug was run by hand to verify the fix. >>>>>>> >>>>>>> Thanks, Harold >>>>>>> >>>>>> >>>>> >>>> >> From david.holmes at oracle.com Tue May 9 00:52:01 2017 From: david.holmes at oracle.com (David Holmes) Date: Tue, 9 May 2017 10:52:01 +1000 Subject: RFR 8178604: JVM does not allow defining boot loader modules in exploded build after module system initialization In-Reply-To: <388b80ed-e436-f888-ffed-5b92e0117ce2@oracle.com> References: <85e36988-a145-6805-8659-56c738013f58@oracle.com> <46a812ca-228d-01e7-e889-fd22f2af0537@oracle.com> <11709d4a-0a41-1d38-fcd1-67d42c33eaeb@oracle.com> <3ee613ff-f6da-9ef8-6781-efce4edba29e@oracle.com> <21b3fdc5-d8a7-0535-3ac6-f1a097a7431d@oracle.com> <388b80ed-e436-f888-ffed-5b92e0117ce2@oracle.com> Message-ID: Hi Harold, On 9/05/2017 4:00 AM, harold seigel wrote: > Hi David, > > Please see this updated webrev: > http://cr.openjdk.java.net/~hseigel/bug_8178604.3/webrev/index.html > > It contains the changes you requested below. Thank you. I will point out that Lois originally suggested the same changes. > Note that the _exploded_entries table gets created very early during > SystemDictionary initialization, before initializing the pre-loaded > classes. So, the code in load_class() at line ~1503 can assume that > _exploded_entries is not null. Which thread executes ClassLoader::add_to_exploded_build_list, and which thread executes load_class() first? If they are guaranteed to be the same thread then you don't need locking on the construction. If they can be different threads then there must be some synchronization between them that ensures _exploded_entries is properly visible after construction. Thanks, David ----- > Thanks, Harold > > On 5/4/2017 5:32 PM, David Holmes wrote: >> Hi Harold, >> >> On 5/05/2017 3:47 AM, harold seigel wrote: >>> Hi, >>> >>> Please review this updated webrev: >>> >>> http://cr.openjdk.java.net/~hseigel/bug_8178604.2/webrev/index.html >>> >>> The updated webrev takes out a lock when creating _exploded_entries. It >>> also contains additional comments and an assert() to address Lois's >>> concerns with reduced code readability involving what situations warrant >>> taking out a lock in ClassLoader::search_module_entries(). >>> >>> I don't think there needs to be two different initial entry points as >>> suggested by David below. The method behaves slightly differently >>> depending on the value of its 'need_lock' parameter. But, there is >>> enough common code for it to remain one method. >> >> I think this highlights uncertainty about the concurrent behaviour in >> relation to this code. On the one hand you are worried about an >> initialization race and so use the lock, but when you do: >> >> 1500 // The exploded build entries can be added to at any time >> so pass 'true' for >> 1501 // need_lock so that a lock is taken out when searching them. >> 1502 assert(_exploded_entries != NULL, "No exploded build >> entries present"); >> 1503 stream = search_module_entries(_exploded_entries, >> class_name, file_name, true, CHECK_NULL); >> >> you load _exploded_entries with no lock held and so must be certain >> that you can not possibly read a null value here. >> >> The needs_lock parameter seems superfluous - you know the condition >> for locking is that the list is the _exploded_entries, and you assert >> that. So no need to pass in needs_lock, just grab the lock when >> dealing with _exploded_entries. >> >> Cheers, >> David >> >>> Thanks, Harold >>> >>> On 5/3/2017 12:59 AM, David Holmes wrote: >>>> On 3/05/2017 7:11 AM, Lois Foltan wrote: >>>>> On 5/2/2017 9:03 AM, harold seigel wrote: >>>>>> Hi Lois, >>>>>> >>>>>> Thanks for your comments. Please see my in-line responses. >>>>>> >>>>>> Harold >>>>>> >>>>>> >>>>>> On 5/1/2017 4:14 PM, Lois Foltan wrote: >>>>>>> Hi Harold, >>>>>>> >>>>>>> Looks good. A couple of comments: >>>>>>> >>>>>>> src/share/vm/classfile/classLoader.cpp >>>>>>> line #828 - please take out the Module_lock when the >>>>>>> _exploded_entries is created >>>>>> This fix does not change the possible need for synchronization when >>>>>> _exploded_entries gets created during module system initialization. >>>>>> Are you saying that the existing code is wrong and should have taken >>>>>> out the Module_lock before creating _exploding_entries? >>>>> >>>>> The method ClassLoader::add_to_exploded_build_list can be called at >>>>> anytime (during module system initialization and post module system >>>>> initialization). Thus it seems prudent that no matter what the >>>>> current >>>>> JVM phase, that any access to ClassLoader::_exploding_entries be >>>>> protected via a lock. That includes creation as well as insertion >>>>> into >>>>> the list. >>>> >>>> If creation is not guaranteed to occur before other threads that will >>>> call the same method and use the _exploded_entries list, then >>>> synchronization of some form is essential. If it is guaranteed then we >>>> don't need create time sync just to be prudent - but the guarantee >>>> must be firmly established and clearly documented. >>>> >>>>>> >>>>>>> line #1402 - I like the new factored out method >>>>>>> find_first_module_cpe(), however, instead of adding a new >>>>>>> "need_lock" >>>>>>> parameter, I would rather see code >>>>>>> in find_first_module_cpe() that takes the >>>>>>> Module_lock if the module_list parameter equals >>>>>>> ClassLoader::_exploded_entries >>>>>> I think your suggestion makes the code less flexible and does not >>>>>> require potential future callers of search_module_entries() to think >>>>>> about synchronization. So, I'd prefer to leave that change as is. >>>>> >>>>> I see your point. My point was based on reduced code readability, the >>>>> further away from the decision to establish the lock, the less >>>>> apparent >>>>> it is within the new method ClassLoader::find_first_module_cpe() as to >>>>> what situations warranted the lock in the first place. >>>> >>>> Further, this: >>>> >>>> 1495 stream = search_module_entries(_exploded_entries, >>>> class_name, file_name, true, CHECK_NULL); >>>> >>>> is potentially unsafe if we can race with creation because we first >>>> access _exploded_entries without holding any lock! And the fact the >>>> method needs to do different things depending on which "list" was >>>> passed in suggests to me it should be two different initial entry >>>> points. >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> Thanks, >>>>> Lois >>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Lois >>>>>>> >>>>>>> On 5/1/2017 3:36 PM, harold seigel wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> Please review this JDK-10 bug fix to allow defining of modules to >>>>>>>> the boot loader in exploded builds after module system >>>>>>>> initialization. The fix uses the module lock to synchronize access >>>>>>>> to the _exploded_entries data structure (which is only used by >>>>>>>> exploded builds). >>>>>>>> >>>>>>>> Note that the above capability already exists for regular builds. >>>>>>>> >>>>>>>> Open Webrev: >>>>>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604/webrev/index.html >>>>>>>> >>>>>>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8178604 >>>>>>>> >>>>>>>> The fix was tested with JCK tests, the JTreg hotspot, java/io, >>>>>>>> java/lang, java/util and other tests, the RBT tier2 -tier5 tests, >>>>>>>> the co-located NSK tests, and with JPRT. The JTReg and JCK tests >>>>>>>> were run with an exploded build. Also, the example program in the >>>>>>>> JBS bug was run by hand to verify the fix. >>>>>>>> >>>>>>>> Thanks, Harold >>>>>>>> >>>>>>> >>>>>> >>>>> >>> > From harold.seigel at oracle.com Tue May 9 13:14:21 2017 From: harold.seigel at oracle.com (harold seigel) Date: Tue, 9 May 2017 09:14:21 -0400 Subject: RFR 8178604: JVM does not allow defining boot loader modules in exploded build after module system initialization In-Reply-To: References: <85e36988-a145-6805-8659-56c738013f58@oracle.com> <46a812ca-228d-01e7-e889-fd22f2af0537@oracle.com> <11709d4a-0a41-1d38-fcd1-67d42c33eaeb@oracle.com> <3ee613ff-f6da-9ef8-6781-efce4edba29e@oracle.com> <21b3fdc5-d8a7-0535-3ac6-f1a097a7431d@oracle.com> <388b80ed-e436-f888-ffed-5b92e0117ce2@oracle.com> Message-ID: <2e1d1c53-8d6d-e606-f047-a223f76b63a9@oracle.com> Hi David, See comments embedded below. Thanks, Harold On 5/8/2017 8:52 PM, David Holmes wrote: > Hi Harold, > > On 9/05/2017 4:00 AM, harold seigel wrote: >> Hi David, >> >> Please see this updated webrev: >> http://cr.openjdk.java.net/~hseigel/bug_8178604.3/webrev/index.html >> >> It contains the changes you requested below. > > Thank you. I will point out that Lois originally suggested the same > changes. > >> Note that the _exploded_entries table gets created very early during >> SystemDictionary initialization, before initializing the pre-loaded >> classes. So, the code in load_class() at line ~1503 can assume that >> _exploded_entries is not null. > > Which thread executes ClassLoader::add_to_exploded_build_list, and > which thread executes load_class() first? If they are guaranteed to be > the same thread then you don't need locking on the construction. If > they can be different threads then there must be some synchronization > between them that ensures _exploded_entries is properly visible after > construction. They are the same thread. At startup, the thread calls SystemDictionary::initialize_preloaded_classes(), which calls ClassLoader::classLoader_init2(), which calls add_to_exploded_build_list(). Control then returns back to SystemDictionary::initialize_preloaded_classes() which eventually calls SystemDictionary::load_instance_class(), which calls ClassLoader::load_class(). Here's the call stack showing these calls: V [libjvm.so+0x974c0c] ClassLoader::load_class(Symbol*, bool, Thread*)+0x41c V [libjvm.so+0x1648339] SystemDictionary::load_instance_class(Symbol*, Handle, Thread*)+0x819 V [libjvm.so+0x1648e25] SystemDictionary::resolve_instance_class_or_null(Symbol*, Handle, Handle, Thread*)+0x985 V [libjvm.so+0x16494ba] SystemDictionary::resolve_or_null(Symbol*, Handle, Handle, Thread*)+0x5a V [libjvm.so+0x16498fe] SystemDictionary::resolve_or_fail(Symbol*, Handle, Handle, bool, Thread*)+0x1e V [libjvm.so+0x1649aa9] SystemDictionary::initialize_wk_klass(SystemDictionary::WKID, int, Thread*)+0x149 V [libjvm.so+0x1649bfe] SystemDictionary::initialize_wk_klasses_until(SystemDictionary::WKID, SystemDictionary::WKID&, Thread*)+0x5e V [libjvm.so+0x1649dba] SystemDictionary::*initialize_preloaded_classes*(Thread*)+0xea V [libjvm.so+0x164a28a] SystemDictionary::initialize(Thread*)+0x26a V [libjvm.so+0x16cd7e0] Universe::genesis(Thread*)+0x7c0 V [libjvm.so+0x16ce4cc] universe2_init()+0x2c V [libjvm.so+0xe0f268] init_globals()+0xa8 V [libjvm.so+0x1696e6f] Threads::create_vm(JavaVMInitArgs*, bool*)+0x2df ... So, is the existing synchronization in this change sufficient? Thanks, Harold > > Thanks, > David > ----- > >> Thanks, Harold >> >> On 5/4/2017 5:32 PM, David Holmes wrote: >>> Hi Harold, >>> >>> On 5/05/2017 3:47 AM, harold seigel wrote: >>>> Hi, >>>> >>>> Please review this updated webrev: >>>> >>>> http://cr.openjdk.java.net/~hseigel/bug_8178604.2/webrev/index.html >>>> >>>> The updated webrev takes out a lock when creating >>>> _exploded_entries. It >>>> also contains additional comments and an assert() to address Lois's >>>> concerns with reduced code readability involving what situations >>>> warrant >>>> taking out a lock in ClassLoader::search_module_entries(). >>>> >>>> I don't think there needs to be two different initial entry points as >>>> suggested by David below. The method behaves slightly differently >>>> depending on the value of its 'need_lock' parameter. But, there is >>>> enough common code for it to remain one method. >>> >>> I think this highlights uncertainty about the concurrent behaviour in >>> relation to this code. On the one hand you are worried about an >>> initialization race and so use the lock, but when you do: >>> >>> 1500 // The exploded build entries can be added to at any time >>> so pass 'true' for >>> 1501 // need_lock so that a lock is taken out when searching >>> them. >>> 1502 assert(_exploded_entries != NULL, "No exploded build >>> entries present"); >>> 1503 stream = search_module_entries(_exploded_entries, >>> class_name, file_name, true, CHECK_NULL); >>> >>> you load _exploded_entries with no lock held and so must be certain >>> that you can not possibly read a null value here. >>> >>> The needs_lock parameter seems superfluous - you know the condition >>> for locking is that the list is the _exploded_entries, and you assert >>> that. So no need to pass in needs_lock, just grab the lock when >>> dealing with _exploded_entries. >>> >>> Cheers, >>> David >>> >>>> Thanks, Harold >>>> >>>> On 5/3/2017 12:59 AM, David Holmes wrote: >>>>> On 3/05/2017 7:11 AM, Lois Foltan wrote: >>>>>> On 5/2/2017 9:03 AM, harold seigel wrote: >>>>>>> Hi Lois, >>>>>>> >>>>>>> Thanks for your comments. Please see my in-line responses. >>>>>>> >>>>>>> Harold >>>>>>> >>>>>>> >>>>>>> On 5/1/2017 4:14 PM, Lois Foltan wrote: >>>>>>>> Hi Harold, >>>>>>>> >>>>>>>> Looks good. A couple of comments: >>>>>>>> >>>>>>>> src/share/vm/classfile/classLoader.cpp >>>>>>>> line #828 - please take out the Module_lock when the >>>>>>>> _exploded_entries is created >>>>>>> This fix does not change the possible need for synchronization when >>>>>>> _exploded_entries gets created during module system initialization. >>>>>>> Are you saying that the existing code is wrong and should have >>>>>>> taken >>>>>>> out the Module_lock before creating _exploding_entries? >>>>>> >>>>>> The method ClassLoader::add_to_exploded_build_list can be called at >>>>>> anytime (during module system initialization and post module system >>>>>> initialization). Thus it seems prudent that no matter what the >>>>>> current >>>>>> JVM phase, that any access to ClassLoader::_exploding_entries be >>>>>> protected via a lock. That includes creation as well as insertion >>>>>> into >>>>>> the list. >>>>> >>>>> If creation is not guaranteed to occur before other threads that will >>>>> call the same method and use the _exploded_entries list, then >>>>> synchronization of some form is essential. If it is guaranteed >>>>> then we >>>>> don't need create time sync just to be prudent - but the guarantee >>>>> must be firmly established and clearly documented. >>>>> >>>>>>> >>>>>>>> line #1402 - I like the new factored out method >>>>>>>> find_first_module_cpe(), however, instead of adding a new >>>>>>>> "need_lock" >>>>>>>> parameter, I would rather see code >>>>>>>> in find_first_module_cpe() that takes the >>>>>>>> Module_lock if the module_list parameter equals >>>>>>>> ClassLoader::_exploded_entries >>>>>>> I think your suggestion makes the code less flexible and does not >>>>>>> require potential future callers of search_module_entries() to >>>>>>> think >>>>>>> about synchronization. So, I'd prefer to leave that change as is. >>>>>> >>>>>> I see your point. My point was based on reduced code >>>>>> readability, the >>>>>> further away from the decision to establish the lock, the less >>>>>> apparent >>>>>> it is within the new method ClassLoader::find_first_module_cpe() >>>>>> as to >>>>>> what situations warranted the lock in the first place. >>>>> >>>>> Further, this: >>>>> >>>>> 1495 stream = search_module_entries(_exploded_entries, >>>>> class_name, file_name, true, CHECK_NULL); >>>>> >>>>> is potentially unsafe if we can race with creation because we first >>>>> access _exploded_entries without holding any lock! And the fact the >>>>> method needs to do different things depending on which "list" was >>>>> passed in suggests to me it should be two different initial entry >>>>> points. >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>>> Thanks, >>>>>> Lois >>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Lois >>>>>>>> >>>>>>>> On 5/1/2017 3:36 PM, harold seigel wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Please review this JDK-10 bug fix to allow defining of modules to >>>>>>>>> the boot loader in exploded builds after module system >>>>>>>>> initialization. The fix uses the module lock to synchronize >>>>>>>>> access >>>>>>>>> to the _exploded_entries data structure (which is only used by >>>>>>>>> exploded builds). >>>>>>>>> >>>>>>>>> Note that the above capability already exists for regular builds. >>>>>>>>> >>>>>>>>> Open Webrev: >>>>>>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604/webrev/index.html >>>>>>>>> >>>>>>>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8178604 >>>>>>>>> >>>>>>>>> The fix was tested with JCK tests, the JTreg hotspot, java/io, >>>>>>>>> java/lang, java/util and other tests, the RBT tier2 -tier5 tests, >>>>>>>>> the co-located NSK tests, and with JPRT. The JTReg and JCK tests >>>>>>>>> were run with an exploded build. Also, the example program in >>>>>>>>> the >>>>>>>>> JBS bug was run by hand to verify the fix. >>>>>>>>> >>>>>>>>> Thanks, Harold >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>> >> From goetz.lindenmaier at sap.com Tue May 9 14:56:42 2017 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 9 May 2017 14:56:42 +0000 Subject: RFR(S): 8179953: [ppc] TLABWasteIncrement not loaded correctly Message-ID: Hi, Please review this change. As I fix the test, I please need a sponsor. http://cr.openjdk.java.net/~goetz/wr17/8179953-ppc_flag/webrev.01/ On ppc, the mentioned flag is loaded as a 16 bit immediate, while it can have bigger values. I also adapt TestOptionWithRanges, which did not show the bug because the flag has no effect with G1. Best regards, Goetz. From claes.redestad at oracle.com Tue May 9 18:06:19 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 9 May 2017 20:06:19 +0200 Subject: [10] RFR: 8179040: Avoid Ticks::now calls when EventClassLoad is not enabled In-Reply-To: References: Message-ID: <91450a5c-b5c6-f831-d10c-b2485b425bfc@oracle.com> Hi David, On 2017-04-25 22:58, David Holmes wrote: > Hi Claes, > > On 25/04/2017 11:21 PM, Claes Redestad wrote: >> Hi, >> >> this patch removes calling Ticks::now when EventClassLoad isn't enabled, >> which has an effect on class loading performance: >> >> http://cr.openjdk.java.net/~redestad/8179040/hotspot.01/ >> >> When tracing isn't enabled trace/tracing.hpp has dummy >> implementations which are easily optimized away by a compiler, which >> I've verified happens on linux OpenJDK builds with tracing disabled. > > What you have done achieves your goal and could be pushed as-is. Thanks! > > That said, I would consider revisiting the conditional versus > unconditional use of trace-related code in general. So that things like: > > 649 EventClassLoad class_load_start_event; > 895 post_class_load_event(&class_load_start_event, k, loader_data); > > were done using TRACE_ONLY, so that all trace-related code was > completely excluded when not using INCLUDE_TRACE. Is your concern here that some compilers won't be smart enough to completely[1] remove these allocations and calls from the generated code? /Claes [1] Evidently there might always be some minute trace somewhere, e.g., gcc seems to compile and leave unused methods around in the compiled binary, so if there's a footprint concern when building minimal VMs then it seems methods should be completely enveloped in INCLUDE_TRACE and all inline code be wrapped in TRACE_ONLY, aesthetic concerns notwithstanding. > > Thanks, > David > >> On builds with tracing enabled then the changes means the call to >> get the time only happen if the event is enabled, which achieves the >> sought after startup optimization. > >> Thanks! >> >> /Claes From david.holmes at oracle.com Tue May 9 21:18:46 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 10 May 2017 07:18:46 +1000 Subject: RFR 8178604: JVM does not allow defining boot loader modules in exploded build after module system initialization In-Reply-To: <2e1d1c53-8d6d-e606-f047-a223f76b63a9@oracle.com> References: <85e36988-a145-6805-8659-56c738013f58@oracle.com> <46a812ca-228d-01e7-e889-fd22f2af0537@oracle.com> <11709d4a-0a41-1d38-fcd1-67d42c33eaeb@oracle.com> <3ee613ff-f6da-9ef8-6781-efce4edba29e@oracle.com> <21b3fdc5-d8a7-0535-3ac6-f1a097a7431d@oracle.com> <388b80ed-e436-f888-ffed-5b92e0117ce2@oracle.com> <2e1d1c53-8d6d-e606-f047-a223f76b63a9@oracle.com> Message-ID: On 9/05/2017 11:14 PM, harold seigel wrote: > Hi David, > > See comments embedded below. > > Thanks, Harold > > > On 5/8/2017 8:52 PM, David Holmes wrote: >> Hi Harold, >> >> On 9/05/2017 4:00 AM, harold seigel wrote: >>> Hi David, >>> >>> Please see this updated webrev: >>> http://cr.openjdk.java.net/~hseigel/bug_8178604.3/webrev/index.html >>> >>> It contains the changes you requested below. >> >> Thank you. I will point out that Lois originally suggested the same >> changes. >> >>> Note that the _exploded_entries table gets created very early during >>> SystemDictionary initialization, before initializing the pre-loaded >>> classes. So, the code in load_class() at line ~1503 can assume that >>> _exploded_entries is not null. >> >> Which thread executes ClassLoader::add_to_exploded_build_list, and >> which thread executes load_class() first? If they are guaranteed to be >> the same thread then you don't need locking on the construction. If >> they can be different threads then there must be some synchronization >> between them that ensures _exploded_entries is properly visible after >> construction. > They are the same thread. At startup, the thread calls Then as I said, why do we need any locking? I note Lois is the one that requested this and you questioned it. I also question it. Thanks, David ----- > SystemDictionary::initialize_preloaded_classes(), which calls > ClassLoader::classLoader_init2(), which calls add_to_exploded_build_list(). > > Control then returns back to > SystemDictionary::initialize_preloaded_classes() which eventually calls > SystemDictionary::load_instance_class(), which calls > ClassLoader::load_class(). Here's the call stack showing these calls: > > V [libjvm.so+0x974c0c] ClassLoader::load_class(Symbol*, bool, > Thread*)+0x41c > V [libjvm.so+0x1648339] > SystemDictionary::load_instance_class(Symbol*, Handle, Thread*)+0x819 > V [libjvm.so+0x1648e25] > SystemDictionary::resolve_instance_class_or_null(Symbol*, Handle, > Handle, Thread*)+0x985 > V [libjvm.so+0x16494ba] SystemDictionary::resolve_or_null(Symbol*, > Handle, Handle, Thread*)+0x5a > V [libjvm.so+0x16498fe] SystemDictionary::resolve_or_fail(Symbol*, > Handle, Handle, bool, Thread*)+0x1e > V [libjvm.so+0x1649aa9] > SystemDictionary::initialize_wk_klass(SystemDictionary::WKID, int, > Thread*)+0x149 > V [libjvm.so+0x1649bfe] > SystemDictionary::initialize_wk_klasses_until(SystemDictionary::WKID, SystemDictionary::WKID&, > Thread*)+0x5e > V [libjvm.so+0x1649dba] > SystemDictionary::*initialize_preloaded_classes*(Thread*)+0xea > V [libjvm.so+0x164a28a] SystemDictionary::initialize(Thread*)+0x26a > V [libjvm.so+0x16cd7e0] Universe::genesis(Thread*)+0x7c0 > V [libjvm.so+0x16ce4cc] universe2_init()+0x2c > V [libjvm.so+0xe0f268] init_globals()+0xa8 > V [libjvm.so+0x1696e6f] Threads::create_vm(JavaVMInitArgs*, > bool*)+0x2df > ... > > So, is the existing synchronization in this change sufficient? > > Thanks, Harold >> >> Thanks, >> David >> ----- >> >>> Thanks, Harold >>> >>> On 5/4/2017 5:32 PM, David Holmes wrote: >>>> Hi Harold, >>>> >>>> On 5/05/2017 3:47 AM, harold seigel wrote: >>>>> Hi, >>>>> >>>>> Please review this updated webrev: >>>>> >>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604.2/webrev/index.html >>>>> >>>>> The updated webrev takes out a lock when creating >>>>> _exploded_entries. It >>>>> also contains additional comments and an assert() to address Lois's >>>>> concerns with reduced code readability involving what situations >>>>> warrant >>>>> taking out a lock in ClassLoader::search_module_entries(). >>>>> >>>>> I don't think there needs to be two different initial entry points as >>>>> suggested by David below. The method behaves slightly differently >>>>> depending on the value of its 'need_lock' parameter. But, there is >>>>> enough common code for it to remain one method. >>>> >>>> I think this highlights uncertainty about the concurrent behaviour in >>>> relation to this code. On the one hand you are worried about an >>>> initialization race and so use the lock, but when you do: >>>> >>>> 1500 // The exploded build entries can be added to at any time >>>> so pass 'true' for >>>> 1501 // need_lock so that a lock is taken out when searching >>>> them. >>>> 1502 assert(_exploded_entries != NULL, "No exploded build >>>> entries present"); >>>> 1503 stream = search_module_entries(_exploded_entries, >>>> class_name, file_name, true, CHECK_NULL); >>>> >>>> you load _exploded_entries with no lock held and so must be certain >>>> that you can not possibly read a null value here. >>>> >>>> The needs_lock parameter seems superfluous - you know the condition >>>> for locking is that the list is the _exploded_entries, and you assert >>>> that. So no need to pass in needs_lock, just grab the lock when >>>> dealing with _exploded_entries. >>>> >>>> Cheers, >>>> David >>>> >>>>> Thanks, Harold >>>>> >>>>> On 5/3/2017 12:59 AM, David Holmes wrote: >>>>>> On 3/05/2017 7:11 AM, Lois Foltan wrote: >>>>>>> On 5/2/2017 9:03 AM, harold seigel wrote: >>>>>>>> Hi Lois, >>>>>>>> >>>>>>>> Thanks for your comments. Please see my in-line responses. >>>>>>>> >>>>>>>> Harold >>>>>>>> >>>>>>>> >>>>>>>> On 5/1/2017 4:14 PM, Lois Foltan wrote: >>>>>>>>> Hi Harold, >>>>>>>>> >>>>>>>>> Looks good. A couple of comments: >>>>>>>>> >>>>>>>>> src/share/vm/classfile/classLoader.cpp >>>>>>>>> line #828 - please take out the Module_lock when the >>>>>>>>> _exploded_entries is created >>>>>>>> This fix does not change the possible need for synchronization when >>>>>>>> _exploded_entries gets created during module system initialization. >>>>>>>> Are you saying that the existing code is wrong and should have >>>>>>>> taken >>>>>>>> out the Module_lock before creating _exploding_entries? >>>>>>> >>>>>>> The method ClassLoader::add_to_exploded_build_list can be called at >>>>>>> anytime (during module system initialization and post module system >>>>>>> initialization). Thus it seems prudent that no matter what the >>>>>>> current >>>>>>> JVM phase, that any access to ClassLoader::_exploding_entries be >>>>>>> protected via a lock. That includes creation as well as insertion >>>>>>> into >>>>>>> the list. >>>>>> >>>>>> If creation is not guaranteed to occur before other threads that will >>>>>> call the same method and use the _exploded_entries list, then >>>>>> synchronization of some form is essential. If it is guaranteed >>>>>> then we >>>>>> don't need create time sync just to be prudent - but the guarantee >>>>>> must be firmly established and clearly documented. >>>>>> >>>>>>>> >>>>>>>>> line #1402 - I like the new factored out method >>>>>>>>> find_first_module_cpe(), however, instead of adding a new >>>>>>>>> "need_lock" >>>>>>>>> parameter, I would rather see code >>>>>>>>> in find_first_module_cpe() that takes the >>>>>>>>> Module_lock if the module_list parameter equals >>>>>>>>> ClassLoader::_exploded_entries >>>>>>>> I think your suggestion makes the code less flexible and does not >>>>>>>> require potential future callers of search_module_entries() to >>>>>>>> think >>>>>>>> about synchronization. So, I'd prefer to leave that change as is. >>>>>>> >>>>>>> I see your point. My point was based on reduced code >>>>>>> readability, the >>>>>>> further away from the decision to establish the lock, the less >>>>>>> apparent >>>>>>> it is within the new method ClassLoader::find_first_module_cpe() >>>>>>> as to >>>>>>> what situations warranted the lock in the first place. >>>>>> >>>>>> Further, this: >>>>>> >>>>>> 1495 stream = search_module_entries(_exploded_entries, >>>>>> class_name, file_name, true, CHECK_NULL); >>>>>> >>>>>> is potentially unsafe if we can race with creation because we first >>>>>> access _exploded_entries without holding any lock! And the fact the >>>>>> method needs to do different things depending on which "list" was >>>>>> passed in suggests to me it should be two different initial entry >>>>>> points. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>>> Thanks, >>>>>>> Lois >>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Lois >>>>>>>>> >>>>>>>>> On 5/1/2017 3:36 PM, harold seigel wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> Please review this JDK-10 bug fix to allow defining of modules to >>>>>>>>>> the boot loader in exploded builds after module system >>>>>>>>>> initialization. The fix uses the module lock to synchronize >>>>>>>>>> access >>>>>>>>>> to the _exploded_entries data structure (which is only used by >>>>>>>>>> exploded builds). >>>>>>>>>> >>>>>>>>>> Note that the above capability already exists for regular builds. >>>>>>>>>> >>>>>>>>>> Open Webrev: >>>>>>>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604/webrev/index.html >>>>>>>>>> >>>>>>>>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8178604 >>>>>>>>>> >>>>>>>>>> The fix was tested with JCK tests, the JTreg hotspot, java/io, >>>>>>>>>> java/lang, java/util and other tests, the RBT tier2 -tier5 tests, >>>>>>>>>> the co-located NSK tests, and with JPRT. The JTReg and JCK tests >>>>>>>>>> were run with an exploded build. Also, the example program in >>>>>>>>>> the >>>>>>>>>> JBS bug was run by hand to verify the fix. >>>>>>>>>> >>>>>>>>>> Thanks, Harold >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>> > From david.holmes at oracle.com Tue May 9 21:25:38 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 10 May 2017 07:25:38 +1000 Subject: [10] RFR: 8179040: Avoid Ticks::now calls when EventClassLoad is not enabled In-Reply-To: <91450a5c-b5c6-f831-d10c-b2485b425bfc@oracle.com> References: <91450a5c-b5c6-f831-d10c-b2485b425bfc@oracle.com> Message-ID: <0c70d5e7-f9a5-5265-0b75-89ba1f558f21@oracle.com> Hi Claes, On 10/05/2017 4:06 AM, Claes Redestad wrote: > Hi David, > > On 2017-04-25 22:58, David Holmes wrote: >> Hi Claes, >> >> On 25/04/2017 11:21 PM, Claes Redestad wrote: >>> Hi, >>> >>> this patch removes calling Ticks::now when EventClassLoad isn't enabled, >>> which has an effect on class loading performance: >>> >>> http://cr.openjdk.java.net/~redestad/8179040/hotspot.01/ >>> >>> When tracing isn't enabled trace/tracing.hpp has dummy >>> implementations which are easily optimized away by a compiler, which >>> I've verified happens on linux OpenJDK builds with tracing disabled. >> >> What you have done achieves your goal and could be pushed as-is. > > Thanks! > >> >> That said, I would consider revisiting the conditional versus >> unconditional use of trace-related code in general. So that things like: >> >> 649 EventClassLoad class_load_start_event; >> 895 post_class_load_event(&class_load_start_event, k, loader_data); >> >> were done using TRACE_ONLY, so that all trace-related code was >> completely excluded when not using INCLUDE_TRACE. > > Is your concern here that some compilers won't be smart enough to > completely[1] remove these allocations and calls > from the generated code? Just a code cleanliness issue. But at the time I wrote the above I hadn't realized that TRACE_ONLY macros only exist in the closed sources. Markus is/was looking more generally at this issue so any cleanup can be left for that work. Thanks, David > /Claes > > [1] Evidently there might always be some minute trace somewhere, e.g., > gcc seems to compile and leave unused methods around > in the compiled binary, so if there's a footprint concern when building > minimal VMs then it seems methods should be completely > enveloped in INCLUDE_TRACE and all inline code be wrapped in TRACE_ONLY, > aesthetic concerns notwithstanding. > >> >> Thanks, >> David >> >>> On builds with tracing enabled then the changes means the call to >>> get the time only happen if the event is enabled, which achieves the >>> sought after startup optimization. > >>> Thanks! >>> >>> /Claes > From mikael.vidstedt at oracle.com Tue May 9 22:59:42 2017 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Tue, 9 May 2017 15:59:42 -0700 Subject: RFR(XS): 8180036: Guard include of fpu_control.h Message-ID: <7D4A9A57-5855-4664-A78A-15F96250ADC9@oracle.com> Please review this small change which makes the inclusion of fpu_control.h in os_linux_x86.c conditional: Bug: https://bugs.openjdk.java.net/browse/JDK-8180036 Webrev: http://cr.openjdk.java.net/~mikael/webrevs/8180036/webrev.00/hotspot/webrev/ The reason why the file is included in the first place is that on 32-bit x86 there is some logic in os_linux_x86.c to set/get/change the FPU control word using functions implemented by fpu_control.h. This is only done on 32-bit x86, and the code in question is guarded by #ifndef AMD64. However, the inclusion of the fpu_control.h is unconditional. This change makes that consistent by adding the same #ifndef AMD64 guard around the #include. Cheers, Mikael From david.holmes at oracle.com Tue May 9 23:21:38 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 10 May 2017 09:21:38 +1000 Subject: RFR(XS): 8180036: Guard include of fpu_control.h In-Reply-To: <7D4A9A57-5855-4664-A78A-15F96250ADC9@oracle.com> References: <7D4A9A57-5855-4664-A78A-15F96250ADC9@oracle.com> Message-ID: <477967b1-8e2c-f1ea-b9c0-8595c2297f24@oracle.com> Looks trivially fine. Thanks, David On 10/05/2017 8:59 AM, Mikael Vidstedt wrote: > > Please review this small change which makes the inclusion of fpu_control.h in os_linux_x86.c conditional: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8180036 > Webrev: http://cr.openjdk.java.net/~mikael/webrevs/8180036/webrev.00/hotspot/webrev/ > > The reason why the file is included in the first place is that on 32-bit x86 there is some logic in os_linux_x86.c to set/get/change the FPU control word using functions implemented by fpu_control.h. This is only done on 32-bit x86, and the code in question is guarded by #ifndef AMD64. However, the inclusion of the fpu_control.h is unconditional. This change makes that consistent by adding the same #ifndef AMD64 guard around the #include. > > Cheers, > Mikael > From mikael.vidstedt at oracle.com Tue May 9 23:38:20 2017 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Tue, 9 May 2017 16:38:20 -0700 Subject: RFR(S): 8180039: Use more portable print format/arguments for rlim_t Message-ID: <78FE21D0-F4EA-47B5-AC90-7D69990B45F0@oracle.com> Please review this small change which updates the code in the print_rlimit_info method in os_posix.cpp which prints out rlim_t/rlim_cur values (returned from getrlimit) to use more portable format specifiers and arguments/argument types. Bug: https://bugs.openjdk.java.net/browse/JDK-8180039 Webrev: http://cr.openjdk.java.net/~mikael/webrevs/8180039/webrev.00/hotspot/webrev/ The spec says that rlim_t is an "unsigned integer type?, so casting it to a uint64_t will either zero extend it (if the rlim_t type is 32-bit), or end up with the exact same width/value (if rlim_t is 64-bit). Printing it out as a 64-bit unsigned type (using UINT64_FORMAT) will handle both the 64-bit case and the zero extended 32-bit case. Cheers, Mikael From david.holmes at oracle.com Wed May 10 00:27:58 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 10 May 2017 10:27:58 +1000 Subject: RFR(S): 8180039: Use more portable print format/arguments for rlim_t In-Reply-To: <78FE21D0-F4EA-47B5-AC90-7D69990B45F0@oracle.com> References: <78FE21D0-F4EA-47B5-AC90-7D69990B45F0@oracle.com> Message-ID: <5eed7e52-529f-ba28-11a9-71ba6d45080f@oracle.com> Hi Mikael, On 10/05/2017 9:38 AM, Mikael Vidstedt wrote: > > Please review this small change which updates the code in the print_rlimit_info method in os_posix.cpp which prints out rlim_t/rlim_cur values (returned from getrlimit) to use more portable format specifiers and arguments/argument types. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8180039 > Webrev: http://cr.openjdk.java.net/~mikael/webrevs/8180039/webrev.00/hotspot/webrev/ > > The spec says that rlim_t is an "unsigned integer type?, so casting it to a uint64_t will either zero extend it (if the rlim_t type is 32-bit), or end up with the exact same width/value (if rlim_t is 64-bit). Printing it out as a 64-bit unsigned type (using UINT64_FORMAT) will handle both the 64-bit case and the zero extended 32-bit case. Not unreasonable. :) Alternatively we could define RLIM_FORMAT depending on sizeof rlim_t, but that may be over the top. Makes me wonder why for every xxx_t they define they don't also define a XXX_T_FORMAT value! Thanks, David > Cheers, > Mikael > From david.holmes at oracle.com Wed May 10 03:19:10 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 10 May 2017 13:19:10 +1000 Subject: RFR(S): 8179953: [ppc] TLABWasteIncrement not loaded correctly In-Reply-To: References: Message-ID: Hi Goetz, On 10/05/2017 12:56 AM, Lindenmaier, Goetz wrote: > Hi, > > Please review this change. As I fix the test, I please need a sponsor. > http://cr.openjdk.java.net/~goetz/wr17/8179953-ppc_flag/webrev.01/ I'm unclear on the test changes. It isn't obvious to me that TLABWasteIncrement is a ParallelGC only flag. By adding that as an additional flag you then had to exclude adding the GCType (if any GC selector is given) to avoid conflicting options - is that right? Thanks, David > On ppc, the mentioned flag is loaded as a 16 bit immediate, while it can have > bigger values. > I also adapt TestOptionWithRanges, which did not show the bug because > the flag has no effect with G1. > > Best regards, > Goetz. > > From mikael.gerdin at oracle.com Wed May 10 06:21:08 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Wed, 10 May 2017 08:21:08 +0200 Subject: RFR(S): 8180039: Use more portable print format/arguments for rlim_t In-Reply-To: <78FE21D0-F4EA-47B5-AC90-7D69990B45F0@oracle.com> References: <78FE21D0-F4EA-47B5-AC90-7D69990B45F0@oracle.com> Message-ID: <58985e9c-ca42-85b3-87af-52ab71c17dc3@oracle.com> Hi Mikael On 2017-05-10 01:38, Mikael Vidstedt wrote: > > Please review this small change which updates the code in the > print_rlimit_info method in os_posix.cpp which prints out > rlim_t/rlim_cur values (returned from getrlimit) to use more portable > format specifiers and arguments/argument types. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8180039 > Webrev: http://cr.openjdk.java.net/~mikael/webrevs/8180039/webrev.00/hotspot/webrev/ Looks good. /Mikael > > The spec says that rlim_t is an "unsigned integer type?, so casting it > to a uint64_t will either zero extend it (if the rlim_t type is 32-bit), > or end up with the exact same width/value (if rlim_t is 64-bit). > Printing it out as a 64-bit unsigned type (using UINT64_FORMAT) will > handle both the 64-bit case and the zero extended 32-bit case. > > Cheers, > Mikael > From thomas.stuefe at gmail.com Wed May 10 10:51:28 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 10 May 2017 12:51:28 +0200 Subject: RFR(S): 8180039: Use more portable print format/arguments for rlim_t In-Reply-To: <78FE21D0-F4EA-47B5-AC90-7D69990B45F0@oracle.com> References: <78FE21D0-F4EA-47B5-AC90-7D69990B45F0@oracle.com> Message-ID: Hi Mikael, this looks fine. While you are on it, would it be possible to add printouts for DATA and FSIZE? Another small nit, I would prefer /1024 instead of >>10. Kind Regards, Thomas On Wed, May 10, 2017 at 1:38 AM, Mikael Vidstedt wrote: > > Please review this small change which updates the code in the > print_rlimit_info method in os_posix.cpp which prints out rlim_t/rlim_cur > values (returned from getrlimit) to use more portable format specifiers and > arguments/argument types. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8180039 < > https://bugs.openjdk.java.net/browse/JDK-8180039> > Webrev: http://cr.openjdk.java.net/~mikael/webrevs/8180039/webrev. > 00/hotspot/webrev/ mikael/webrevs/8180039/webrev.00/hotspot/webrev/> > > The spec says that rlim_t is an "unsigned integer type?, so casting it to > a uint64_t will either zero extend it (if the rlim_t type is 32-bit), or > end up with the exact same width/value (if rlim_t is 64-bit). Printing it > out as a 64-bit unsigned type (using UINT64_FORMAT) will handle both the > 64-bit case and the zero extended 32-bit case. > > Cheers, > Mikael > > From lois.foltan at oracle.com Wed May 10 17:19:30 2017 From: lois.foltan at oracle.com (Lois Foltan) Date: Wed, 10 May 2017 13:19:30 -0400 Subject: RFR 8178604: JVM does not allow defining boot loader modules in exploded build after module system initialization In-Reply-To: References: <85e36988-a145-6805-8659-56c738013f58@oracle.com> <46a812ca-228d-01e7-e889-fd22f2af0537@oracle.com> <11709d4a-0a41-1d38-fcd1-67d42c33eaeb@oracle.com> <3ee613ff-f6da-9ef8-6781-efce4edba29e@oracle.com> <21b3fdc5-d8a7-0535-3ac6-f1a097a7431d@oracle.com> <388b80ed-e436-f888-ffed-5b92e0117ce2@oracle.com> <2e1d1c53-8d6d-e606-f047-a223f76b63a9@oracle.com> Message-ID: On 5/9/2017 5:18 PM, David Holmes wrote: > On 9/05/2017 11:14 PM, harold seigel wrote: >> Hi David, >> >> See comments embedded below. >> >> Thanks, Harold >> >> >> On 5/8/2017 8:52 PM, David Holmes wrote: >>> Hi Harold, >>> >>> On 9/05/2017 4:00 AM, harold seigel wrote: >>>> Hi David, >>>> >>>> Please see this updated webrev: >>>> http://cr.openjdk.java.net/~hseigel/bug_8178604.3/webrev/index.html >>>> >>>> It contains the changes you requested below. >>> >>> Thank you. I will point out that Lois originally suggested the same >>> changes. >>> >>>> Note that the _exploded_entries table gets created very early during >>>> SystemDictionary initialization, before initializing the pre-loaded >>>> classes. So, the code in load_class() at line ~1503 can assume that >>>> _exploded_entries is not null. >>> >>> Which thread executes ClassLoader::add_to_exploded_build_list, and >>> which thread executes load_class() first? If they are guaranteed to be >>> the same thread then you don't need locking on the construction. If >>> they can be different threads then there must be some synchronization >>> between them that ensures _exploded_entries is properly visible after >>> construction. >> They are the same thread. At startup, the thread calls > > Then as I said, why do we need any locking? I note Lois is the one > that requested this and you questioned it. I also question it. Thank you Harold for investigating this. I stand corrected, it seems that we do not need the lock after all when allocating ClassLoader::_exploded_entries. Lois > > Thanks, > David > ----- > >> SystemDictionary::initialize_preloaded_classes(), which calls >> ClassLoader::classLoader_init2(), which calls >> add_to_exploded_build_list(). >> >> Control then returns back to >> SystemDictionary::initialize_preloaded_classes() which eventually calls >> SystemDictionary::load_instance_class(), which calls >> ClassLoader::load_class(). Here's the call stack showing these calls: >> >> V [libjvm.so+0x974c0c] ClassLoader::load_class(Symbol*, bool, >> Thread*)+0x41c >> V [libjvm.so+0x1648339] >> SystemDictionary::load_instance_class(Symbol*, Handle, >> Thread*)+0x819 >> V [libjvm.so+0x1648e25] >> SystemDictionary::resolve_instance_class_or_null(Symbol*, Handle, >> Handle, Thread*)+0x985 >> V [libjvm.so+0x16494ba] SystemDictionary::resolve_or_null(Symbol*, >> Handle, Handle, Thread*)+0x5a >> V [libjvm.so+0x16498fe] SystemDictionary::resolve_or_fail(Symbol*, >> Handle, Handle, bool, Thread*)+0x1e >> V [libjvm.so+0x1649aa9] >> SystemDictionary::initialize_wk_klass(SystemDictionary::WKID, int, >> Thread*)+0x149 >> V [libjvm.so+0x1649bfe] >> SystemDictionary::initialize_wk_klasses_until(SystemDictionary::WKID, >> SystemDictionary::WKID&, >> Thread*)+0x5e >> V [libjvm.so+0x1649dba] >> SystemDictionary::*initialize_preloaded_classes*(Thread*)+0xea >> V [libjvm.so+0x164a28a] SystemDictionary::initialize(Thread*)+0x26a >> V [libjvm.so+0x16cd7e0] Universe::genesis(Thread*)+0x7c0 >> V [libjvm.so+0x16ce4cc] universe2_init()+0x2c >> V [libjvm.so+0xe0f268] init_globals()+0xa8 >> V [libjvm.so+0x1696e6f] Threads::create_vm(JavaVMInitArgs*, >> bool*)+0x2df >> ... >> >> So, is the existing synchronization in this change sufficient? >> >> Thanks, Harold >>> >>> Thanks, >>> David >>> ----- >>> >>>> Thanks, Harold >>>> >>>> On 5/4/2017 5:32 PM, David Holmes wrote: >>>>> Hi Harold, >>>>> >>>>> On 5/05/2017 3:47 AM, harold seigel wrote: >>>>>> Hi, >>>>>> >>>>>> Please review this updated webrev: >>>>>> >>>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604.2/webrev/index.html >>>>>> >>>>>> The updated webrev takes out a lock when creating >>>>>> _exploded_entries. It >>>>>> also contains additional comments and an assert() to address Lois's >>>>>> concerns with reduced code readability involving what situations >>>>>> warrant >>>>>> taking out a lock in ClassLoader::search_module_entries(). >>>>>> >>>>>> I don't think there needs to be two different initial entry >>>>>> points as >>>>>> suggested by David below. The method behaves slightly differently >>>>>> depending on the value of its 'need_lock' parameter. But, there is >>>>>> enough common code for it to remain one method. >>>>> >>>>> I think this highlights uncertainty about the concurrent behaviour in >>>>> relation to this code. On the one hand you are worried about an >>>>> initialization race and so use the lock, but when you do: >>>>> >>>>> 1500 // The exploded build entries can be added to at any time >>>>> so pass 'true' for >>>>> 1501 // need_lock so that a lock is taken out when searching >>>>> them. >>>>> 1502 assert(_exploded_entries != NULL, "No exploded build >>>>> entries present"); >>>>> 1503 stream = search_module_entries(_exploded_entries, >>>>> class_name, file_name, true, CHECK_NULL); >>>>> >>>>> you load _exploded_entries with no lock held and so must be certain >>>>> that you can not possibly read a null value here. >>>>> >>>>> The needs_lock parameter seems superfluous - you know the condition >>>>> for locking is that the list is the _exploded_entries, and you assert >>>>> that. So no need to pass in needs_lock, just grab the lock when >>>>> dealing with _exploded_entries. >>>>> >>>>> Cheers, >>>>> David >>>>> >>>>>> Thanks, Harold >>>>>> >>>>>> On 5/3/2017 12:59 AM, David Holmes wrote: >>>>>>> On 3/05/2017 7:11 AM, Lois Foltan wrote: >>>>>>>> On 5/2/2017 9:03 AM, harold seigel wrote: >>>>>>>>> Hi Lois, >>>>>>>>> >>>>>>>>> Thanks for your comments. Please see my in-line responses. >>>>>>>>> >>>>>>>>> Harold >>>>>>>>> >>>>>>>>> >>>>>>>>> On 5/1/2017 4:14 PM, Lois Foltan wrote: >>>>>>>>>> Hi Harold, >>>>>>>>>> >>>>>>>>>> Looks good. A couple of comments: >>>>>>>>>> >>>>>>>>>> src/share/vm/classfile/classLoader.cpp >>>>>>>>>> line #828 - please take out the Module_lock when the >>>>>>>>>> _exploded_entries is created >>>>>>>>> This fix does not change the possible need for synchronization >>>>>>>>> when >>>>>>>>> _exploded_entries gets created during module system >>>>>>>>> initialization. >>>>>>>>> Are you saying that the existing code is wrong and should have >>>>>>>>> taken >>>>>>>>> out the Module_lock before creating _exploding_entries? >>>>>>>> >>>>>>>> The method ClassLoader::add_to_exploded_build_list can be >>>>>>>> called at >>>>>>>> anytime (during module system initialization and post module >>>>>>>> system >>>>>>>> initialization). Thus it seems prudent that no matter what the >>>>>>>> current >>>>>>>> JVM phase, that any access to ClassLoader::_exploding_entries be >>>>>>>> protected via a lock. That includes creation as well as insertion >>>>>>>> into >>>>>>>> the list. >>>>>>> >>>>>>> If creation is not guaranteed to occur before other threads that >>>>>>> will >>>>>>> call the same method and use the _exploded_entries list, then >>>>>>> synchronization of some form is essential. If it is guaranteed >>>>>>> then we >>>>>>> don't need create time sync just to be prudent - but the guarantee >>>>>>> must be firmly established and clearly documented. >>>>>>> >>>>>>>>> >>>>>>>>>> line #1402 - I like the new factored out method >>>>>>>>>> find_first_module_cpe(), however, instead of adding a new >>>>>>>>>> "need_lock" >>>>>>>>>> parameter, I would rather see code >>>>>>>>>> in find_first_module_cpe() that >>>>>>>>>> takes the >>>>>>>>>> Module_lock if the module_list parameter equals >>>>>>>>>> ClassLoader::_exploded_entries >>>>>>>>> I think your suggestion makes the code less flexible and does not >>>>>>>>> require potential future callers of search_module_entries() to >>>>>>>>> think >>>>>>>>> about synchronization. So, I'd prefer to leave that change as >>>>>>>>> is. >>>>>>>> >>>>>>>> I see your point. My point was based on reduced code >>>>>>>> readability, the >>>>>>>> further away from the decision to establish the lock, the less >>>>>>>> apparent >>>>>>>> it is within the new method ClassLoader::find_first_module_cpe() >>>>>>>> as to >>>>>>>> what situations warranted the lock in the first place. >>>>>>> >>>>>>> Further, this: >>>>>>> >>>>>>> 1495 stream = search_module_entries(_exploded_entries, >>>>>>> class_name, file_name, true, CHECK_NULL); >>>>>>> >>>>>>> is potentially unsafe if we can race with creation because we first >>>>>>> access _exploded_entries without holding any lock! And the fact the >>>>>>> method needs to do different things depending on which "list" was >>>>>>> passed in suggests to me it should be two different initial entry >>>>>>> points. >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>>> Thanks, >>>>>>>> Lois >>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Lois >>>>>>>>>> >>>>>>>>>> On 5/1/2017 3:36 PM, harold seigel wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> Please review this JDK-10 bug fix to allow defining of >>>>>>>>>>> modules to >>>>>>>>>>> the boot loader in exploded builds after module system >>>>>>>>>>> initialization. The fix uses the module lock to synchronize >>>>>>>>>>> access >>>>>>>>>>> to the _exploded_entries data structure (which is only used by >>>>>>>>>>> exploded builds). >>>>>>>>>>> >>>>>>>>>>> Note that the above capability already exists for regular >>>>>>>>>>> builds. >>>>>>>>>>> >>>>>>>>>>> Open Webrev: >>>>>>>>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604/webrev/index.html >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8178604 >>>>>>>>>>> >>>>>>>>>>> The fix was tested with JCK tests, the JTreg hotspot, java/io, >>>>>>>>>>> java/lang, java/util and other tests, the RBT tier2 -tier5 >>>>>>>>>>> tests, >>>>>>>>>>> the co-located NSK tests, and with JPRT. The JTReg and JCK >>>>>>>>>>> tests >>>>>>>>>>> were run with an exploded build. Also, the example program in >>>>>>>>>>> the >>>>>>>>>>> JBS bug was run by hand to verify the fix. >>>>>>>>>>> >>>>>>>>>>> Thanks, Harold >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>> >>>> >> From rkennke at redhat.com Wed May 10 20:41:33 2017 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 10 May 2017 22:41:33 +0200 Subject: ObjectSynchronizer iterate only in-use monitors? Message-ID: Hello, I have a question related to ObjectSynchronizer. We (the Shenandoah GC devs) found that for some programs, scanning ObjectSynchronizer roots takes quite long. ObjectSynchronizer::oops_do() scans all the blocks in gBlockList. As far as I understand, this contains all the monitor blocks of all threads, both currently in-use and free blocks. If I understand it correctly, it would be sufficient to scan only in-use monitors. And since each thread has its own in-use list (at least with MonitorInUseLists), it should be ok to scan that during each thread's scan, plus one additional scan of the gOmInUseList. I am writing here because I would like to get confirmation that what I'm doing is sane, or if there are any pitfalls that I'm not aware of. The webrev in question (against shenandoah/jdk9) is this: http://cr.openjdk.java.net/~rkennke/fastsyncroots/webrev.00/ I tested it by running with SPECjvm2008 and jcstress and found no ill-effects. Performance-wise it makes a very significant difference (running gc-bench's roots.Sync test, which exaggerates synchronizer usage): baseline: [14,393s][info][gc,stats] S: Thread Roots = 0,34 s (a = 37748 us) (n = 9) (lvls, us = 36523, 36523, 36914, 37305, 42215) [14,393s][info][gc,stats] S: Synchronizer Roots = 0,14 s (a = 15115 us) (n = 9) (lvls, us = 9746, 10938, 14258, 14648, 25847) [14,393s][info][gc,stats] UR: Thread Roots = 0,22 s (a = 24967 us) (n = 9) (lvls, us = 12305, 24219, 25977, 27148, 27758) [14,393s][info][gc,stats] UR: Synchronizer Roots = 0,11 s (a = 11906 us) (n = 9) (lvls, us = 8340, 9082, 12109, 12695, 13787) patched: [14,293s][info][gc,stats] S: Thread Roots = 0,36 s (a = 40365 us) (n = 9) (lvls, us = 32031, 32031, 34570, 37109, 67224) [14,293s][info][gc,stats] S: Synchronizer Roots = 0,00 s (a = 0 us) (n = 9) (lvls, us = 0, 0, 0, 0, 0) [14,294s][info][gc,stats] UR: Thread Roots = 0,22 s (a = 24459 us) (n = 9) (lvls, us = 15820, 20508, 22070, 26172, 32573) [14,294s][info][gc,stats] UR: Synchronizer Roots = 0,00 s (a = 0 us) (n = 9) (lvls, us = 0, 0, 0, 0, 0) Notice how thread roots scanning goes a little bit up, but by far not as much as sync root scanning goes down. If you think what I'm doing is sane, this might even be useful for other GCs (although they're probably not as much bound by roots scanning as Shenandoah is). Thanks, Roman From harold.seigel at oracle.com Wed May 10 21:18:22 2017 From: harold.seigel at oracle.com (harold seigel) Date: Wed, 10 May 2017 17:18:22 -0400 Subject: RFR 8178604: JVM does not allow defining boot loader modules in exploded build after module system initialization In-Reply-To: References: <85e36988-a145-6805-8659-56c738013f58@oracle.com> <46a812ca-228d-01e7-e889-fd22f2af0537@oracle.com> <11709d4a-0a41-1d38-fcd1-67d42c33eaeb@oracle.com> <3ee613ff-f6da-9ef8-6781-efce4edba29e@oracle.com> <21b3fdc5-d8a7-0535-3ac6-f1a097a7431d@oracle.com> <388b80ed-e436-f888-ffed-5b92e0117ce2@oracle.com> <2e1d1c53-8d6d-e606-f047-a223f76b63a9@oracle.com> Message-ID: Hi Lois, David, Please review this latest webrev that initializes _exploded_entries without locking: http://cr.openjdk.java.net/~hseigel/bug_8178604.4/webrev/index.html Thanks, Harold On 5/10/2017 1:19 PM, Lois Foltan wrote: > On 5/9/2017 5:18 PM, David Holmes wrote: >> On 9/05/2017 11:14 PM, harold seigel wrote: >>> Hi David, >>> >>> See comments embedded below. >>> >>> Thanks, Harold >>> >>> >>> On 5/8/2017 8:52 PM, David Holmes wrote: >>>> Hi Harold, >>>> >>>> On 9/05/2017 4:00 AM, harold seigel wrote: >>>>> Hi David, >>>>> >>>>> Please see this updated webrev: >>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604.3/webrev/index.html >>>>> >>>>> It contains the changes you requested below. >>>> >>>> Thank you. I will point out that Lois originally suggested the same >>>> changes. >>>> >>>>> Note that the _exploded_entries table gets created very early during >>>>> SystemDictionary initialization, before initializing the pre-loaded >>>>> classes. So, the code in load_class() at line ~1503 can assume that >>>>> _exploded_entries is not null. >>>> >>>> Which thread executes ClassLoader::add_to_exploded_build_list, and >>>> which thread executes load_class() first? If they are guaranteed to be >>>> the same thread then you don't need locking on the construction. If >>>> they can be different threads then there must be some synchronization >>>> between them that ensures _exploded_entries is properly visible after >>>> construction. >>> They are the same thread. At startup, the thread calls >> >> Then as I said, why do we need any locking? I note Lois is the one >> that requested this and you questioned it. I also question it. > > Thank you Harold for investigating this. I stand corrected, it seems > that we do not need the lock after all when allocating > ClassLoader::_exploded_entries. > Lois > >> >> Thanks, >> David >> ----- >> >>> SystemDictionary::initialize_preloaded_classes(), which calls >>> ClassLoader::classLoader_init2(), which calls >>> add_to_exploded_build_list(). >>> >>> Control then returns back to >>> SystemDictionary::initialize_preloaded_classes() which eventually calls >>> SystemDictionary::load_instance_class(), which calls >>> ClassLoader::load_class(). Here's the call stack showing these calls: >>> >>> V [libjvm.so+0x974c0c] ClassLoader::load_class(Symbol*, bool, >>> Thread*)+0x41c >>> V [libjvm.so+0x1648339] >>> SystemDictionary::load_instance_class(Symbol*, Handle, >>> Thread*)+0x819 >>> V [libjvm.so+0x1648e25] >>> SystemDictionary::resolve_instance_class_or_null(Symbol*, Handle, >>> Handle, Thread*)+0x985 >>> V [libjvm.so+0x16494ba] SystemDictionary::resolve_or_null(Symbol*, >>> Handle, Handle, Thread*)+0x5a >>> V [libjvm.so+0x16498fe] SystemDictionary::resolve_or_fail(Symbol*, >>> Handle, Handle, bool, Thread*)+0x1e >>> V [libjvm.so+0x1649aa9] >>> SystemDictionary::initialize_wk_klass(SystemDictionary::WKID, int, >>> Thread*)+0x149 >>> V [libjvm.so+0x1649bfe] >>> SystemDictionary::initialize_wk_klasses_until(SystemDictionary::WKID, >>> SystemDictionary::WKID&, >>> Thread*)+0x5e >>> V [libjvm.so+0x1649dba] >>> SystemDictionary::*initialize_preloaded_classes*(Thread*)+0xea >>> V [libjvm.so+0x164a28a] >>> SystemDictionary::initialize(Thread*)+0x26a >>> V [libjvm.so+0x16cd7e0] Universe::genesis(Thread*)+0x7c0 >>> V [libjvm.so+0x16ce4cc] universe2_init()+0x2c >>> V [libjvm.so+0xe0f268] init_globals()+0xa8 >>> V [libjvm.so+0x1696e6f] Threads::create_vm(JavaVMInitArgs*, >>> bool*)+0x2df >>> ... >>> >>> So, is the existing synchronization in this change sufficient? >>> >>> Thanks, Harold >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> Thanks, Harold >>>>> >>>>> On 5/4/2017 5:32 PM, David Holmes wrote: >>>>>> Hi Harold, >>>>>> >>>>>> On 5/05/2017 3:47 AM, harold seigel wrote: >>>>>>> Hi, >>>>>>> >>>>>>> Please review this updated webrev: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604.2/webrev/index.html >>>>>>> >>>>>>> The updated webrev takes out a lock when creating >>>>>>> _exploded_entries. It >>>>>>> also contains additional comments and an assert() to address Lois's >>>>>>> concerns with reduced code readability involving what situations >>>>>>> warrant >>>>>>> taking out a lock in ClassLoader::search_module_entries(). >>>>>>> >>>>>>> I don't think there needs to be two different initial entry >>>>>>> points as >>>>>>> suggested by David below. The method behaves slightly differently >>>>>>> depending on the value of its 'need_lock' parameter. But, there is >>>>>>> enough common code for it to remain one method. >>>>>> >>>>>> I think this highlights uncertainty about the concurrent >>>>>> behaviour in >>>>>> relation to this code. On the one hand you are worried about an >>>>>> initialization race and so use the lock, but when you do: >>>>>> >>>>>> 1500 // The exploded build entries can be added to at any time >>>>>> so pass 'true' for >>>>>> 1501 // need_lock so that a lock is taken out when searching >>>>>> them. >>>>>> 1502 assert(_exploded_entries != NULL, "No exploded build >>>>>> entries present"); >>>>>> 1503 stream = search_module_entries(_exploded_entries, >>>>>> class_name, file_name, true, CHECK_NULL); >>>>>> >>>>>> you load _exploded_entries with no lock held and so must be certain >>>>>> that you can not possibly read a null value here. >>>>>> >>>>>> The needs_lock parameter seems superfluous - you know the condition >>>>>> for locking is that the list is the _exploded_entries, and you >>>>>> assert >>>>>> that. So no need to pass in needs_lock, just grab the lock when >>>>>> dealing with _exploded_entries. >>>>>> >>>>>> Cheers, >>>>>> David >>>>>> >>>>>>> Thanks, Harold >>>>>>> >>>>>>> On 5/3/2017 12:59 AM, David Holmes wrote: >>>>>>>> On 3/05/2017 7:11 AM, Lois Foltan wrote: >>>>>>>>> On 5/2/2017 9:03 AM, harold seigel wrote: >>>>>>>>>> Hi Lois, >>>>>>>>>> >>>>>>>>>> Thanks for your comments. Please see my in-line responses. >>>>>>>>>> >>>>>>>>>> Harold >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 5/1/2017 4:14 PM, Lois Foltan wrote: >>>>>>>>>>> Hi Harold, >>>>>>>>>>> >>>>>>>>>>> Looks good. A couple of comments: >>>>>>>>>>> >>>>>>>>>>> src/share/vm/classfile/classLoader.cpp >>>>>>>>>>> line #828 - please take out the Module_lock when the >>>>>>>>>>> _exploded_entries is created >>>>>>>>>> This fix does not change the possible need for >>>>>>>>>> synchronization when >>>>>>>>>> _exploded_entries gets created during module system >>>>>>>>>> initialization. >>>>>>>>>> Are you saying that the existing code is wrong and should have >>>>>>>>>> taken >>>>>>>>>> out the Module_lock before creating _exploding_entries? >>>>>>>>> >>>>>>>>> The method ClassLoader::add_to_exploded_build_list can be >>>>>>>>> called at >>>>>>>>> anytime (during module system initialization and post module >>>>>>>>> system >>>>>>>>> initialization). Thus it seems prudent that no matter what the >>>>>>>>> current >>>>>>>>> JVM phase, that any access to ClassLoader::_exploding_entries be >>>>>>>>> protected via a lock. That includes creation as well as >>>>>>>>> insertion >>>>>>>>> into >>>>>>>>> the list. >>>>>>>> >>>>>>>> If creation is not guaranteed to occur before other threads >>>>>>>> that will >>>>>>>> call the same method and use the _exploded_entries list, then >>>>>>>> synchronization of some form is essential. If it is guaranteed >>>>>>>> then we >>>>>>>> don't need create time sync just to be prudent - but the guarantee >>>>>>>> must be firmly established and clearly documented. >>>>>>>> >>>>>>>>>> >>>>>>>>>>> line #1402 - I like the new factored out method >>>>>>>>>>> find_first_module_cpe(), however, instead of adding a new >>>>>>>>>>> "need_lock" >>>>>>>>>>> parameter, I would rather see code >>>>>>>>>>> in find_first_module_cpe() that >>>>>>>>>>> takes the >>>>>>>>>>> Module_lock if the module_list parameter equals >>>>>>>>>>> ClassLoader::_exploded_entries >>>>>>>>>> I think your suggestion makes the code less flexible and does >>>>>>>>>> not >>>>>>>>>> require potential future callers of search_module_entries() to >>>>>>>>>> think >>>>>>>>>> about synchronization. So, I'd prefer to leave that change >>>>>>>>>> as is. >>>>>>>>> >>>>>>>>> I see your point. My point was based on reduced code >>>>>>>>> readability, the >>>>>>>>> further away from the decision to establish the lock, the less >>>>>>>>> apparent >>>>>>>>> it is within the new method ClassLoader::find_first_module_cpe() >>>>>>>>> as to >>>>>>>>> what situations warranted the lock in the first place. >>>>>>>> >>>>>>>> Further, this: >>>>>>>> >>>>>>>> 1495 stream = search_module_entries(_exploded_entries, >>>>>>>> class_name, file_name, true, CHECK_NULL); >>>>>>>> >>>>>>>> is potentially unsafe if we can race with creation because we >>>>>>>> first >>>>>>>> access _exploded_entries without holding any lock! And the fact >>>>>>>> the >>>>>>>> method needs to do different things depending on which "list" was >>>>>>>> passed in suggests to me it should be two different initial entry >>>>>>>> points. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> ----- >>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Lois >>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Lois >>>>>>>>>>> >>>>>>>>>>> On 5/1/2017 3:36 PM, harold seigel wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> Please review this JDK-10 bug fix to allow defining of >>>>>>>>>>>> modules to >>>>>>>>>>>> the boot loader in exploded builds after module system >>>>>>>>>>>> initialization. The fix uses the module lock to synchronize >>>>>>>>>>>> access >>>>>>>>>>>> to the _exploded_entries data structure (which is only used by >>>>>>>>>>>> exploded builds). >>>>>>>>>>>> >>>>>>>>>>>> Note that the above capability already exists for regular >>>>>>>>>>>> builds. >>>>>>>>>>>> >>>>>>>>>>>> Open Webrev: >>>>>>>>>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604/webrev/index.html >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8178604 >>>>>>>>>>>> >>>>>>>>>>>> The fix was tested with JCK tests, the JTreg hotspot, java/io, >>>>>>>>>>>> java/lang, java/util and other tests, the RBT tier2 -tier5 >>>>>>>>>>>> tests, >>>>>>>>>>>> the co-located NSK tests, and with JPRT. The JTReg and JCK >>>>>>>>>>>> tests >>>>>>>>>>>> were run with an exploded build. Also, the example program in >>>>>>>>>>>> the >>>>>>>>>>>> JBS bug was run by hand to verify the fix. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, Harold >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >>> > From david.holmes at oracle.com Thu May 11 01:53:45 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 11 May 2017 11:53:45 +1000 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: References: Message-ID: Hi Roman, On 11/05/2017 6:41 AM, Roman Kennke wrote: > Hello, > > I have a question related to ObjectSynchronizer. We (the Shenandoah GC > devs) found that for some programs, scanning ObjectSynchronizer roots > takes quite long. ObjectSynchronizer::oops_do() scans all the blocks in > gBlockList. As far as I understand, this contains all the monitor blocks > of all threads, both currently in-use and free blocks. > > If I understand it correctly, it would be sufficient to scan only in-use > monitors. And since each thread has its own in-use list (at least with > MonitorInUseLists), it should be ok to scan that during each thread's > scan, plus one additional scan of the gOmInUseList. That seems reasonable to me. It sounds like the scanning code was not updated to reflect the introduction of MonitorInUseLists. > I am writing here because I would like to get confirmation that what I'm > doing is sane, or if there are any pitfalls that I'm not aware of. The > webrev in question (against shenandoah/jdk9) is this: > > http://cr.openjdk.java.net/~rkennke/fastsyncroots/webrev.00/ > > > I tested it by running with SPECjvm2008 and jcstress and found no > ill-effects. > > Performance-wise it makes a very significant difference (running > gc-bench's roots.Sync test, which exaggerates synchronizer usage): > > baseline: > [14,393s][info][gc,stats] S: Thread Roots = 0,34 s (a > = 37748 us) (n = 9) (lvls, us = 36523, 36523, 36914, > 37305, 42215) > [14,393s][info][gc,stats] S: Synchronizer Roots = 0,14 s (a > = 15115 us) (n = 9) (lvls, us = 9746, 10938, 14258, > 14648, 25847) > [14,393s][info][gc,stats] UR: Thread Roots = 0,22 s (a > = 24967 us) (n = 9) (lvls, us = 12305, 24219, 25977, > 27148, 27758) > [14,393s][info][gc,stats] UR: Synchronizer Roots = 0,11 s (a > = 11906 us) (n = 9) (lvls, us = 8340, 9082, 12109, > 12695, 13787) > > patched: > [14,293s][info][gc,stats] S: Thread Roots = 0,36 s (a > = 40365 us) (n = 9) (lvls, us = 32031, 32031, 34570, > 37109, 67224) > [14,293s][info][gc,stats] S: Synchronizer Roots = 0,00 s (a > = 0 us) (n = 9) (lvls, us = 0, 0, > 0, 0, 0) > [14,294s][info][gc,stats] UR: Thread Roots = 0,22 s (a > = 24459 us) (n = 9) (lvls, us = 15820, 20508, 22070, > 26172, 32573) > [14,294s][info][gc,stats] UR: Synchronizer Roots = 0,00 s (a > = 0 us) (n = 9) (lvls, us = 0, 0, > 0, 0, 0) > > Notice how thread roots scanning goes a little bit up, but by far not as > much as sync root scanning goes down. > > If you think what I'm doing is sane, this might even be useful for other > GCs (although they're probably not as much bound by roots scanning as > Shenandoah is). I would not want to see such a Shenandoah specific patch if we can avoid it. I think this may be of general benefit but am not a GC person so best to ask on the hotspot-gc-dev list as well (cc'd). Thanks, David ----- > Thanks, Roman > From david.holmes at oracle.com Thu May 11 02:11:07 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 11 May 2017 12:11:07 +1000 Subject: RFR 8178604: JVM does not allow defining boot loader modules in exploded build after module system initialization In-Reply-To: References: <85e36988-a145-6805-8659-56c738013f58@oracle.com> <46a812ca-228d-01e7-e889-fd22f2af0537@oracle.com> <11709d4a-0a41-1d38-fcd1-67d42c33eaeb@oracle.com> <3ee613ff-f6da-9ef8-6781-efce4edba29e@oracle.com> <21b3fdc5-d8a7-0535-3ac6-f1a097a7431d@oracle.com> <388b80ed-e436-f888-ffed-5b92e0117ce2@oracle.com> <2e1d1c53-8d6d-e606-f047-a223f76b63a9@oracle.com> Message-ID: <1b618a5c-671d-9063-326e-16991efab441@oracle.com> Hi Harold, On 11/05/2017 7:18 AM, harold seigel wrote: > Hi Lois, David, > > Please review this latest webrev that initializes _exploded_entries > without locking: > > http://cr.openjdk.java.net/~hseigel/bug_8178604.4/webrev/index.html Ok. It would be a lot clearer, in my opinion, that the initialization is indeed thread-safe, if it were done eagerly instead of lazily. The execution contexts of the first and second calls (in separate threads) to that chunk of code are not readily discernible by any reasonable means. But the extensive comment is appreciated. Thanks, David > Thanks, Harold > > > On 5/10/2017 1:19 PM, Lois Foltan wrote: >> On 5/9/2017 5:18 PM, David Holmes wrote: >>> On 9/05/2017 11:14 PM, harold seigel wrote: >>>> Hi David, >>>> >>>> See comments embedded below. >>>> >>>> Thanks, Harold >>>> >>>> >>>> On 5/8/2017 8:52 PM, David Holmes wrote: >>>>> Hi Harold, >>>>> >>>>> On 9/05/2017 4:00 AM, harold seigel wrote: >>>>>> Hi David, >>>>>> >>>>>> Please see this updated webrev: >>>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604.3/webrev/index.html >>>>>> >>>>>> It contains the changes you requested below. >>>>> >>>>> Thank you. I will point out that Lois originally suggested the same >>>>> changes. >>>>> >>>>>> Note that the _exploded_entries table gets created very early during >>>>>> SystemDictionary initialization, before initializing the pre-loaded >>>>>> classes. So, the code in load_class() at line ~1503 can assume that >>>>>> _exploded_entries is not null. >>>>> >>>>> Which thread executes ClassLoader::add_to_exploded_build_list, and >>>>> which thread executes load_class() first? If they are guaranteed to be >>>>> the same thread then you don't need locking on the construction. If >>>>> they can be different threads then there must be some synchronization >>>>> between them that ensures _exploded_entries is properly visible after >>>>> construction. >>>> They are the same thread. At startup, the thread calls >>> >>> Then as I said, why do we need any locking? I note Lois is the one >>> that requested this and you questioned it. I also question it. >> >> Thank you Harold for investigating this. I stand corrected, it seems >> that we do not need the lock after all when allocating >> ClassLoader::_exploded_entries. >> Lois >> >>> >>> Thanks, >>> David >>> ----- >>> >>>> SystemDictionary::initialize_preloaded_classes(), which calls >>>> ClassLoader::classLoader_init2(), which calls >>>> add_to_exploded_build_list(). >>>> >>>> Control then returns back to >>>> SystemDictionary::initialize_preloaded_classes() which eventually calls >>>> SystemDictionary::load_instance_class(), which calls >>>> ClassLoader::load_class(). Here's the call stack showing these calls: >>>> >>>> V [libjvm.so+0x974c0c] ClassLoader::load_class(Symbol*, bool, >>>> Thread*)+0x41c >>>> V [libjvm.so+0x1648339] >>>> SystemDictionary::load_instance_class(Symbol*, Handle, >>>> Thread*)+0x819 >>>> V [libjvm.so+0x1648e25] >>>> SystemDictionary::resolve_instance_class_or_null(Symbol*, Handle, >>>> Handle, Thread*)+0x985 >>>> V [libjvm.so+0x16494ba] SystemDictionary::resolve_or_null(Symbol*, >>>> Handle, Handle, Thread*)+0x5a >>>> V [libjvm.so+0x16498fe] SystemDictionary::resolve_or_fail(Symbol*, >>>> Handle, Handle, bool, Thread*)+0x1e >>>> V [libjvm.so+0x1649aa9] >>>> SystemDictionary::initialize_wk_klass(SystemDictionary::WKID, int, >>>> Thread*)+0x149 >>>> V [libjvm.so+0x1649bfe] >>>> SystemDictionary::initialize_wk_klasses_until(SystemDictionary::WKID, SystemDictionary::WKID&, >>>> >>>> Thread*)+0x5e >>>> V [libjvm.so+0x1649dba] >>>> SystemDictionary::*initialize_preloaded_classes*(Thread*)+0xea >>>> V [libjvm.so+0x164a28a] >>>> SystemDictionary::initialize(Thread*)+0x26a >>>> V [libjvm.so+0x16cd7e0] Universe::genesis(Thread*)+0x7c0 >>>> V [libjvm.so+0x16ce4cc] universe2_init()+0x2c >>>> V [libjvm.so+0xe0f268] init_globals()+0xa8 >>>> V [libjvm.so+0x1696e6f] Threads::create_vm(JavaVMInitArgs*, >>>> bool*)+0x2df >>>> ... >>>> >>>> So, is the existing synchronization in this change sufficient? >>>> >>>> Thanks, Harold >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>>> Thanks, Harold >>>>>> >>>>>> On 5/4/2017 5:32 PM, David Holmes wrote: >>>>>>> Hi Harold, >>>>>>> >>>>>>> On 5/05/2017 3:47 AM, harold seigel wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> Please review this updated webrev: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604.2/webrev/index.html >>>>>>>> >>>>>>>> The updated webrev takes out a lock when creating >>>>>>>> _exploded_entries. It >>>>>>>> also contains additional comments and an assert() to address Lois's >>>>>>>> concerns with reduced code readability involving what situations >>>>>>>> warrant >>>>>>>> taking out a lock in ClassLoader::search_module_entries(). >>>>>>>> >>>>>>>> I don't think there needs to be two different initial entry >>>>>>>> points as >>>>>>>> suggested by David below. The method behaves slightly differently >>>>>>>> depending on the value of its 'need_lock' parameter. But, there is >>>>>>>> enough common code for it to remain one method. >>>>>>> >>>>>>> I think this highlights uncertainty about the concurrent >>>>>>> behaviour in >>>>>>> relation to this code. On the one hand you are worried about an >>>>>>> initialization race and so use the lock, but when you do: >>>>>>> >>>>>>> 1500 // The exploded build entries can be added to at any time >>>>>>> so pass 'true' for >>>>>>> 1501 // need_lock so that a lock is taken out when searching >>>>>>> them. >>>>>>> 1502 assert(_exploded_entries != NULL, "No exploded build >>>>>>> entries present"); >>>>>>> 1503 stream = search_module_entries(_exploded_entries, >>>>>>> class_name, file_name, true, CHECK_NULL); >>>>>>> >>>>>>> you load _exploded_entries with no lock held and so must be certain >>>>>>> that you can not possibly read a null value here. >>>>>>> >>>>>>> The needs_lock parameter seems superfluous - you know the condition >>>>>>> for locking is that the list is the _exploded_entries, and you >>>>>>> assert >>>>>>> that. So no need to pass in needs_lock, just grab the lock when >>>>>>> dealing with _exploded_entries. >>>>>>> >>>>>>> Cheers, >>>>>>> David >>>>>>> >>>>>>>> Thanks, Harold >>>>>>>> >>>>>>>> On 5/3/2017 12:59 AM, David Holmes wrote: >>>>>>>>> On 3/05/2017 7:11 AM, Lois Foltan wrote: >>>>>>>>>> On 5/2/2017 9:03 AM, harold seigel wrote: >>>>>>>>>>> Hi Lois, >>>>>>>>>>> >>>>>>>>>>> Thanks for your comments. Please see my in-line responses. >>>>>>>>>>> >>>>>>>>>>> Harold >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 5/1/2017 4:14 PM, Lois Foltan wrote: >>>>>>>>>>>> Hi Harold, >>>>>>>>>>>> >>>>>>>>>>>> Looks good. A couple of comments: >>>>>>>>>>>> >>>>>>>>>>>> src/share/vm/classfile/classLoader.cpp >>>>>>>>>>>> line #828 - please take out the Module_lock when the >>>>>>>>>>>> _exploded_entries is created >>>>>>>>>>> This fix does not change the possible need for >>>>>>>>>>> synchronization when >>>>>>>>>>> _exploded_entries gets created during module system >>>>>>>>>>> initialization. >>>>>>>>>>> Are you saying that the existing code is wrong and should have >>>>>>>>>>> taken >>>>>>>>>>> out the Module_lock before creating _exploding_entries? >>>>>>>>>> >>>>>>>>>> The method ClassLoader::add_to_exploded_build_list can be >>>>>>>>>> called at >>>>>>>>>> anytime (during module system initialization and post module >>>>>>>>>> system >>>>>>>>>> initialization). Thus it seems prudent that no matter what the >>>>>>>>>> current >>>>>>>>>> JVM phase, that any access to ClassLoader::_exploding_entries be >>>>>>>>>> protected via a lock. That includes creation as well as >>>>>>>>>> insertion >>>>>>>>>> into >>>>>>>>>> the list. >>>>>>>>> >>>>>>>>> If creation is not guaranteed to occur before other threads >>>>>>>>> that will >>>>>>>>> call the same method and use the _exploded_entries list, then >>>>>>>>> synchronization of some form is essential. If it is guaranteed >>>>>>>>> then we >>>>>>>>> don't need create time sync just to be prudent - but the guarantee >>>>>>>>> must be firmly established and clearly documented. >>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> line #1402 - I like the new factored out method >>>>>>>>>>>> find_first_module_cpe(), however, instead of adding a new >>>>>>>>>>>> "need_lock" >>>>>>>>>>>> parameter, I would rather see code >>>>>>>>>>>> in find_first_module_cpe() that >>>>>>>>>>>> takes the >>>>>>>>>>>> Module_lock if the module_list parameter equals >>>>>>>>>>>> ClassLoader::_exploded_entries >>>>>>>>>>> I think your suggestion makes the code less flexible and does >>>>>>>>>>> not >>>>>>>>>>> require potential future callers of search_module_entries() to >>>>>>>>>>> think >>>>>>>>>>> about synchronization. So, I'd prefer to leave that change >>>>>>>>>>> as is. >>>>>>>>>> >>>>>>>>>> I see your point. My point was based on reduced code >>>>>>>>>> readability, the >>>>>>>>>> further away from the decision to establish the lock, the less >>>>>>>>>> apparent >>>>>>>>>> it is within the new method ClassLoader::find_first_module_cpe() >>>>>>>>>> as to >>>>>>>>>> what situations warranted the lock in the first place. >>>>>>>>> >>>>>>>>> Further, this: >>>>>>>>> >>>>>>>>> 1495 stream = search_module_entries(_exploded_entries, >>>>>>>>> class_name, file_name, true, CHECK_NULL); >>>>>>>>> >>>>>>>>> is potentially unsafe if we can race with creation because we >>>>>>>>> first >>>>>>>>> access _exploded_entries without holding any lock! And the fact >>>>>>>>> the >>>>>>>>> method needs to do different things depending on which "list" was >>>>>>>>> passed in suggests to me it should be two different initial entry >>>>>>>>> points. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> David >>>>>>>>> ----- >>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Lois >>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Lois >>>>>>>>>>>> >>>>>>>>>>>> On 5/1/2017 3:36 PM, harold seigel wrote: >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> Please review this JDK-10 bug fix to allow defining of >>>>>>>>>>>>> modules to >>>>>>>>>>>>> the boot loader in exploded builds after module system >>>>>>>>>>>>> initialization. The fix uses the module lock to synchronize >>>>>>>>>>>>> access >>>>>>>>>>>>> to the _exploded_entries data structure (which is only used by >>>>>>>>>>>>> exploded builds). >>>>>>>>>>>>> >>>>>>>>>>>>> Note that the above capability already exists for regular >>>>>>>>>>>>> builds. >>>>>>>>>>>>> >>>>>>>>>>>>> Open Webrev: >>>>>>>>>>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604/webrev/index.html >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8178604 >>>>>>>>>>>>> >>>>>>>>>>>>> The fix was tested with JCK tests, the JTreg hotspot, java/io, >>>>>>>>>>>>> java/lang, java/util and other tests, the RBT tier2 -tier5 >>>>>>>>>>>>> tests, >>>>>>>>>>>>> the co-located NSK tests, and with JPRT. The JTReg and JCK >>>>>>>>>>>>> tests >>>>>>>>>>>>> were run with an exploded build. Also, the example program in >>>>>>>>>>>>> the >>>>>>>>>>>>> JBS bug was run by hand to verify the fix. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, Harold >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>> >>>> >> > From robbin.ehn at oracle.com Thu May 11 07:44:57 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Thu, 11 May 2017 09:44:57 +0200 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: References: Message-ID: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> Hi, We have actually been discussing this last few days: https://bugs.openjdk.java.net/browse/JDK-8153224 On 05/10/2017 10:41 PM, Roman Kennke wrote: > > If you think what I'm doing is sane, this might even be useful for other > GCs (although they're probably not as much bound by roots scanning as > Shenandoah is). Sane to me at least. I see no reason why we have the "-XX:+MonitorInUseLists" option (assuming always on), is there any case when we do not want this? Let me know how you plan to continue this! /Robbin > > Thanks, Roman > From rkennke at redhat.com Thu May 11 07:51:10 2017 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 11 May 2017 09:51:10 +0200 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> Message-ID: <8f3a99fe-5ce5-262a-ef71-429aed936848@redhat.com> Am 11.05.2017 um 09:44 schrieb Robbin Ehn: > Hi, > > We have actually been discussing this last few days: > > https://bugs.openjdk.java.net/browse/JDK-8153224 The bug looks related, but focuses on deflation. > On 05/10/2017 10:41 PM, Roman Kennke wrote: >> >> If you think what I'm doing is sane, this might even be useful for other >> GCs (although they're probably not as much bound by roots scanning as >> Shenandoah is). > > Sane to me at least. I see no reason why we have the > "-XX:+MonitorInUseLists" option (assuming always on), is there any > case when we do not want this? > > Let me know how you plan to continue this! I can change the patch to hook up thread-local in-use-monitor scanning to Thread::oops_do() and have ObjectSynchronizer::oops_do() only scan the gOmInUseList. I'd make it conditional on -XX:+MonitorInUseLists, for consistency and just in case anybody's ever turning this off, or else we should remove that flag? This change should transparently enable this for all GCs. I'll also file a new enhancement bug for it. Does that sound good? Roman From shade at redhat.com Thu May 11 08:01:34 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 11 May 2017 10:01:34 +0200 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: <8f3a99fe-5ce5-262a-ef71-429aed936848@redhat.com> References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> <8f3a99fe-5ce5-262a-ef71-429aed936848@redhat.com> Message-ID: On 05/11/2017 09:51 AM, Roman Kennke wrote: > Am 11.05.2017 um 09:44 schrieb Robbin Ehn: >> Let me know how you plan to continue this! > > I can change the patch to hook up thread-local in-use-monitor scanning > to Thread::oops_do() and have ObjectSynchronizer::oops_do() only scan > the gOmInUseList. I'd make it conditional on -XX:+MonitorInUseLists, for > consistency and just in case anybody's ever turning this off, or else we > should remove that flag? This change should transparently enable this > for all GCs. I'll also file a new enhancement bug for it. Does that > sound good? That sounds like a sane tactics to me. -Aleksey From robbin.ehn at oracle.com Thu May 11 08:06:44 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Thu, 11 May 2017 10:06:44 +0200 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> <8f3a99fe-5ce5-262a-ef71-429aed936848@redhat.com> Message-ID: On 05/11/2017 10:01 AM, Aleksey Shipilev wrote: > On 05/11/2017 09:51 AM, Roman Kennke wrote: >> Am 11.05.2017 um 09:44 schrieb Robbin Ehn: >>> Let me know how you plan to continue this! >> >> I can change the patch to hook up thread-local in-use-monitor scanning >> to Thread::oops_do() and have ObjectSynchronizer::oops_do() only scan >> the gOmInUseList. I'd make it conditional on -XX:+MonitorInUseLists, for >> consistency and just in case anybody's ever turning this off, or else we >> should remove that flag? This change should transparently enable this >> for all GCs. I'll also file a new enhancement bug for it. Does that >> sound good? > > That sounds like a sane tactics to me. +1 I guess first step is to deprecate it ? So we need a CSR for that. (CSR is not yet available) (assuming no objections here) Yes, enable for all GCs sounds great to me, maybe run some numbers on them just in case. Thanks! /Robbin > > -Aleksey > > From robbin.ehn at oracle.com Thu May 11 08:12:35 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Thu, 11 May 2017 10:12:35 +0200 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> <8f3a99fe-5ce5-262a-ef71-429aed936848@redhat.com> Message-ID: Re-sending, since Davids reply got sorted into another folder, so I accidentally dropped GC list and missed David H replies, sorry about that. /Robbin On 05/11/2017 10:06 AM, Robbin Ehn wrote: > On 05/11/2017 10:01 AM, Aleksey Shipilev wrote: >> On 05/11/2017 09:51 AM, Roman Kennke wrote: >>> Am 11.05.2017 um 09:44 schrieb Robbin Ehn: >>>> Let me know how you plan to continue this! >>> >>> I can change the patch to hook up thread-local in-use-monitor scanning >>> to Thread::oops_do() and have ObjectSynchronizer::oops_do() only scan >>> the gOmInUseList. I'd make it conditional on -XX:+MonitorInUseLists, for >>> consistency and just in case anybody's ever turning this off, or else we >>> should remove that flag? This change should transparently enable this >>> for all GCs. I'll also file a new enhancement bug for it. Does that >>> sound good? >> >> That sounds like a sane tactics to me. > > +1 > > I guess first step is to deprecate it ? So we need a CSR for that. (CSR is not yet available) > (assuming no objections here) > > Yes, enable for all GCs sounds great to me, maybe run some numbers on them just in case. > > Thanks! > > /Robbin > > >> >> -Aleksey >> >> From claes.redestad at oracle.com Thu May 11 08:32:59 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 11 May 2017 10:32:59 +0200 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> <8f3a99fe-5ce5-262a-ef71-429aed936848@redhat.com> Message-ID: <1a031b52-8da1-a710-1125-34a2f23bd952@oracle.com> W.r.t. deprecating/removing the MonitorInUseLists flag: this flag has been around for a long time but was only recently made default in 9 (build 120) as a result of analysis related to JEP-143. The only downside I know of is a minuscule footprint increase, but there might be unknown bugs where an application could break with this flag enabled since it hasn't seen wide usage. Having seen the flag in internal production use for several years makes us confident it won't, though, but you never know and it definitely doesn't hurt to have a way to switch it back off, at least during the course of 9. So... Deprecate in 10, remove flag in 11, remove internal code in ... 11? (You can technically remove the code using the flag in the same release that you deprecate it, but I don't know -- is that frowned upon?) Thanks! /Claes On 2017-05-11 10:12, Robbin Ehn wrote: > Re-sending, since Davids reply got sorted into another folder, > so I accidentally dropped GC list and missed David H replies, sorry > about that. > > /Robbin > > On 05/11/2017 10:06 AM, Robbin Ehn wrote: >> On 05/11/2017 10:01 AM, Aleksey Shipilev wrote: >>> On 05/11/2017 09:51 AM, Roman Kennke wrote: >>>> Am 11.05.2017 um 09:44 schrieb Robbin Ehn: >>>>> Let me know how you plan to continue this! >>>> >>>> I can change the patch to hook up thread-local in-use-monitor scanning >>>> to Thread::oops_do() and have ObjectSynchronizer::oops_do() only scan >>>> the gOmInUseList. I'd make it conditional on >>>> -XX:+MonitorInUseLists, for >>>> consistency and just in case anybody's ever turning this off, or >>>> else we >>>> should remove that flag? This change should transparently enable this >>>> for all GCs. I'll also file a new enhancement bug for it. Does that >>>> sound good? >>> >>> That sounds like a sane tactics to me. >> >> +1 >> >> I guess first step is to deprecate it ? So we need a CSR for that. >> (CSR is not yet available) >> (assuming no objections here) >> >> Yes, enable for all GCs sounds great to me, maybe run some numbers on >> them just in case. >> >> Thanks! >> >> /Robbin >> >> >>> >>> -Aleksey >>> >>> From robbin.ehn at oracle.com Thu May 11 09:16:24 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Thu, 11 May 2017 11:16:24 +0200 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: <1a031b52-8da1-a710-1125-34a2f23bd952@oracle.com> References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> <8f3a99fe-5ce5-262a-ef71-429aed936848@redhat.com> <1a031b52-8da1-a710-1125-34a2f23bd952@oracle.com> Message-ID: <079532c0-1b30-97fa-a054-deda8fe4a39d@oracle.com> Hi, On 05/11/2017 10:32 AM, Claes Redestad wrote: > So... Deprecate in 10, remove flag in 11, remove internal code in ... 11? (You can technically > remove the code using the flag in the same release that you deprecate it, but I don't know -- > is that frowned upon?) Yes, effect of flag should still be in place on first deprecation, this is the runtime processes: http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2015-June/015277.html /Robbin > > Thanks! > > /Claes > > On 2017-05-11 10:12, Robbin Ehn wrote: >> Re-sending, since Davids reply got sorted into another folder, >> so I accidentally dropped GC list and missed David H replies, sorry about that. >> >> /Robbin >> >> On 05/11/2017 10:06 AM, Robbin Ehn wrote: >>> On 05/11/2017 10:01 AM, Aleksey Shipilev wrote: >>>> On 05/11/2017 09:51 AM, Roman Kennke wrote: >>>>> Am 11.05.2017 um 09:44 schrieb Robbin Ehn: >>>>>> Let me know how you plan to continue this! >>>>> >>>>> I can change the patch to hook up thread-local in-use-monitor scanning >>>>> to Thread::oops_do() and have ObjectSynchronizer::oops_do() only scan >>>>> the gOmInUseList. I'd make it conditional on -XX:+MonitorInUseLists, for >>>>> consistency and just in case anybody's ever turning this off, or else we >>>>> should remove that flag? This change should transparently enable this >>>>> for all GCs. I'll also file a new enhancement bug for it. Does that >>>>> sound good? >>>> >>>> That sounds like a sane tactics to me. >>> >>> +1 >>> >>> I guess first step is to deprecate it ? So we need a CSR for that. (CSR is not yet available) >>> (assuming no objections here) >>> >>> Yes, enable for all GCs sounds great to me, maybe run some numbers on them just in case. >>> >>> Thanks! >>> >>> /Robbin >>> >>> >>>> >>>> -Aleksey >>>> >>>> > From rkennke at redhat.com Thu May 11 10:08:20 2017 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 11 May 2017 12:08:20 +0200 Subject: RFR (jdk10): JDK-8180175: ObjectSynchronizer only needs to iterate in-use monitors Message-ID: <89399a72-0263-5ebd-5fda-82f404fbe2ca@redhat.com> Hi, ObjectSynchronizer::oops_do() currently iterates all monitor blocks, even free ones. With -XX:+MonitorInUseLists, each thread has its own in-use monitors list, and ObjectSynchronizer has a global gOmInUseList for moribund threads. Iterating over in-use monitors only significantly inmproves scanning time for monitors, because it avoids scanning free blocks, improves caching (no padding, and better chances to have thread-local oops in cache already). Performance-wise it makes a very significant difference (running gc-bench's roots.Sync test, which exaggerates synchronizer usage, with Shenandoah): baseline: S: Thread Roots 37748 us S: Synchronizer Roots 15115 us UR: Thread Roots 24967 us UR: Synchronizer Roots 11906 us patched: S: Thread Roots 40365 us S: Synchronizer Roots 0 us UR: Thread Roots 24459 us UR: Synchronizer Roots 0 us Testing: hotspot_gc, specjvm, jcstress -m quick http://cr.openjdk.java.net/~rkennke/8180175/webrev.00/ Roman From shade at redhat.com Thu May 11 10:13:18 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 11 May 2017 12:13:18 +0200 Subject: RFR (jdk10): JDK-8180175: ObjectSynchronizer only needs to iterate in-use monitors In-Reply-To: <89399a72-0263-5ebd-5fda-82f404fbe2ca@redhat.com> References: <89399a72-0263-5ebd-5fda-82f404fbe2ca@redhat.com> Message-ID: On 05/11/2017 12:08 PM, Roman Kennke wrote: > Performance-wise it makes a very significant difference (running > gc-bench's roots.Sync test, which exaggerates synchronizer usage, with > Shenandoah): > > baseline: > S: Thread Roots 37748 us > S: Synchronizer Roots 15115 us > UR: Thread Roots 24967 us > UR: Synchronizer Roots 11906 us > > patched: > S: Thread Roots 40365 us > S: Synchronizer Roots 0 us > UR: Thread Roots 24459 us > UR: Synchronizer Roots 0 us This also cuts Shenandoah's pauses at least 2x on SPECjbb2015 -- so it is not just some glitch that helps a targeted test. I fully expect the same 10 ms improvement to be visible in pause times for other OpenJDK GCs on workloads like these. Thanks, -Aleksey From david.holmes at oracle.com Thu May 11 10:53:35 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 11 May 2017 20:53:35 +1000 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: <1a031b52-8da1-a710-1125-34a2f23bd952@oracle.com> References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> <8f3a99fe-5ce5-262a-ef71-429aed936848@redhat.com> <1a031b52-8da1-a710-1125-34a2f23bd952@oracle.com> Message-ID: On 11/05/2017 6:32 PM, Claes Redestad wrote: > W.r.t. deprecating/removing the MonitorInUseLists flag: this flag has > been around for a long > time but was only recently made default in 9 (build 120) as a result of > analysis related to > JEP-143. > > The only downside I know of is a minuscule footprint increase, but there > might be unknown > bugs where an application could break with this flag enabled since it > hasn't seen wide usage. > > Having seen the flag in internal production use for several years makes > us confident it won't, > though, but you never know and it definitely doesn't hurt to have a way > to switch it back off, > at least during the course of 9. > > So... Deprecate in 10, remove flag in 11, remove internal code in ... > 11? (You can technically > remove the code using the flag in the same release that you deprecate > it, but I don't know --> is that frowned upon?) Deprecation of VM options means continuing to act on the option but issue a deprecation warning - so no you can't remove the code at the same time you deprecate it. Cheers, David > > Thanks! > > /Claes > > On 2017-05-11 10:12, Robbin Ehn wrote: >> Re-sending, since Davids reply got sorted into another folder, >> so I accidentally dropped GC list and missed David H replies, sorry >> about that. >> >> /Robbin >> >> On 05/11/2017 10:06 AM, Robbin Ehn wrote: >>> On 05/11/2017 10:01 AM, Aleksey Shipilev wrote: >>>> On 05/11/2017 09:51 AM, Roman Kennke wrote: >>>>> Am 11.05.2017 um 09:44 schrieb Robbin Ehn: >>>>>> Let me know how you plan to continue this! >>>>> >>>>> I can change the patch to hook up thread-local in-use-monitor scanning >>>>> to Thread::oops_do() and have ObjectSynchronizer::oops_do() only scan >>>>> the gOmInUseList. I'd make it conditional on >>>>> -XX:+MonitorInUseLists, for >>>>> consistency and just in case anybody's ever turning this off, or >>>>> else we >>>>> should remove that flag? This change should transparently enable this >>>>> for all GCs. I'll also file a new enhancement bug for it. Does that >>>>> sound good? >>>> >>>> That sounds like a sane tactics to me. >>> >>> +1 >>> >>> I guess first step is to deprecate it ? So we need a CSR for that. >>> (CSR is not yet available) >>> (assuming no objections here) >>> >>> Yes, enable for all GCs sounds great to me, maybe run some numbers on >>> them just in case. >>> >>> Thanks! >>> >>> /Robbin >>> >>> >>>> >>>> -Aleksey >>>> >>>> > From david.holmes at oracle.com Thu May 11 10:57:13 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 11 May 2017 20:57:13 +1000 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> <8f3a99fe-5ce5-262a-ef71-429aed936848@redhat.com> <1a031b52-8da1-a710-1125-34a2f23bd952@oracle.com> Message-ID: Sorry missed that Robbin already responded to this part. David On 11/05/2017 8:53 PM, David Holmes wrote: > On 11/05/2017 6:32 PM, Claes Redestad wrote: >> W.r.t. deprecating/removing the MonitorInUseLists flag: this flag has >> been around for a long >> time but was only recently made default in 9 (build 120) as a result of >> analysis related to >> JEP-143. >> >> The only downside I know of is a minuscule footprint increase, but there >> might be unknown >> bugs where an application could break with this flag enabled since it >> hasn't seen wide usage. >> >> Having seen the flag in internal production use for several years makes >> us confident it won't, >> though, but you never know and it definitely doesn't hurt to have a way >> to switch it back off, >> at least during the course of 9. >> >> So... Deprecate in 10, remove flag in 11, remove internal code in ... >> 11? (You can technically >> remove the code using the flag in the same release that you deprecate >> it, but I don't know --> is that frowned upon?) > > Deprecation of VM options means continuing to act on the option but > issue a deprecation warning - so no you can't remove the code at the > same time you deprecate it. > > Cheers, > David > > >> >> Thanks! >> >> /Claes >> >> On 2017-05-11 10:12, Robbin Ehn wrote: >>> Re-sending, since Davids reply got sorted into another folder, >>> so I accidentally dropped GC list and missed David H replies, sorry >>> about that. >>> >>> /Robbin >>> >>> On 05/11/2017 10:06 AM, Robbin Ehn wrote: >>>> On 05/11/2017 10:01 AM, Aleksey Shipilev wrote: >>>>> On 05/11/2017 09:51 AM, Roman Kennke wrote: >>>>>> Am 11.05.2017 um 09:44 schrieb Robbin Ehn: >>>>>>> Let me know how you plan to continue this! >>>>>> >>>>>> I can change the patch to hook up thread-local in-use-monitor >>>>>> scanning >>>>>> to Thread::oops_do() and have ObjectSynchronizer::oops_do() only scan >>>>>> the gOmInUseList. I'd make it conditional on >>>>>> -XX:+MonitorInUseLists, for >>>>>> consistency and just in case anybody's ever turning this off, or >>>>>> else we >>>>>> should remove that flag? This change should transparently enable this >>>>>> for all GCs. I'll also file a new enhancement bug for it. Does that >>>>>> sound good? >>>>> >>>>> That sounds like a sane tactics to me. >>>> >>>> +1 >>>> >>>> I guess first step is to deprecate it ? So we need a CSR for that. >>>> (CSR is not yet available) >>>> (assuming no objections here) >>>> >>>> Yes, enable for all GCs sounds great to me, maybe run some numbers on >>>> them just in case. >>>> >>>> Thanks! >>>> >>>> /Robbin >>>> >>>> >>>>> >>>>> -Aleksey >>>>> >>>>> >> From lois.foltan at oracle.com Thu May 11 11:13:31 2017 From: lois.foltan at oracle.com (Lois Foltan) Date: Thu, 11 May 2017 07:13:31 -0400 Subject: RFR 8178604: JVM does not allow defining boot loader modules in exploded build after module system initialization In-Reply-To: References: <85e36988-a145-6805-8659-56c738013f58@oracle.com> <46a812ca-228d-01e7-e889-fd22f2af0537@oracle.com> <11709d4a-0a41-1d38-fcd1-67d42c33eaeb@oracle.com> <3ee613ff-f6da-9ef8-6781-efce4edba29e@oracle.com> <21b3fdc5-d8a7-0535-3ac6-f1a097a7431d@oracle.com> <388b80ed-e436-f888-ffed-5b92e0117ce2@oracle.com> <2e1d1c53-8d6d-e606-f047-a223f76b63a9@oracle.com> Message-ID: This looks good. Lois On 5/10/2017 5:18 PM, harold seigel wrote: > > Hi Lois, David, > > Please review this latest webrev that initializes _exploded_entries > without locking: > > http://cr.openjdk.java.net/~hseigel/bug_8178604.4/webrev/index.html > > Thanks, Harold > > > On 5/10/2017 1:19 PM, Lois Foltan wrote: >> On 5/9/2017 5:18 PM, David Holmes wrote: >>> On 9/05/2017 11:14 PM, harold seigel wrote: >>>> Hi David, >>>> >>>> See comments embedded below. >>>> >>>> Thanks, Harold >>>> >>>> >>>> On 5/8/2017 8:52 PM, David Holmes wrote: >>>>> Hi Harold, >>>>> >>>>> On 9/05/2017 4:00 AM, harold seigel wrote: >>>>>> Hi David, >>>>>> >>>>>> Please see this updated webrev: >>>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604.3/webrev/index.html >>>>>> >>>>>> It contains the changes you requested below. >>>>> >>>>> Thank you. I will point out that Lois originally suggested the same >>>>> changes. >>>>> >>>>>> Note that the _exploded_entries table gets created very early during >>>>>> SystemDictionary initialization, before initializing the pre-loaded >>>>>> classes. So, the code in load_class() at line ~1503 can assume that >>>>>> _exploded_entries is not null. >>>>> >>>>> Which thread executes ClassLoader::add_to_exploded_build_list, and >>>>> which thread executes load_class() first? If they are guaranteed >>>>> to be >>>>> the same thread then you don't need locking on the construction. If >>>>> they can be different threads then there must be some synchronization >>>>> between them that ensures _exploded_entries is properly visible after >>>>> construction. >>>> They are the same thread. At startup, the thread calls >>> >>> Then as I said, why do we need any locking? I note Lois is the one >>> that requested this and you questioned it. I also question it. >> >> Thank you Harold for investigating this. I stand corrected, it seems >> that we do not need the lock after all when allocating >> ClassLoader::_exploded_entries. >> Lois >> >>> >>> Thanks, >>> David >>> ----- >>> >>>> SystemDictionary::initialize_preloaded_classes(), which calls >>>> ClassLoader::classLoader_init2(), which calls >>>> add_to_exploded_build_list(). >>>> >>>> Control then returns back to >>>> SystemDictionary::initialize_preloaded_classes() which eventually >>>> calls >>>> SystemDictionary::load_instance_class(), which calls >>>> ClassLoader::load_class(). Here's the call stack showing these >>>> calls: >>>> >>>> V [libjvm.so+0x974c0c] ClassLoader::load_class(Symbol*, bool, >>>> Thread*)+0x41c >>>> V [libjvm.so+0x1648339] >>>> SystemDictionary::load_instance_class(Symbol*, Handle, >>>> Thread*)+0x819 >>>> V [libjvm.so+0x1648e25] >>>> SystemDictionary::resolve_instance_class_or_null(Symbol*, Handle, >>>> Handle, Thread*)+0x985 >>>> V [libjvm.so+0x16494ba] >>>> SystemDictionary::resolve_or_null(Symbol*, >>>> Handle, Handle, Thread*)+0x5a >>>> V [libjvm.so+0x16498fe] >>>> SystemDictionary::resolve_or_fail(Symbol*, >>>> Handle, Handle, bool, Thread*)+0x1e >>>> V [libjvm.so+0x1649aa9] >>>> SystemDictionary::initialize_wk_klass(SystemDictionary::WKID, int, >>>> Thread*)+0x149 >>>> V [libjvm.so+0x1649bfe] >>>> SystemDictionary::initialize_wk_klasses_until(SystemDictionary::WKID, >>>> SystemDictionary::WKID&, >>>> Thread*)+0x5e >>>> V [libjvm.so+0x1649dba] >>>> SystemDictionary::*initialize_preloaded_classes*(Thread*)+0xea >>>> V [libjvm.so+0x164a28a] >>>> SystemDictionary::initialize(Thread*)+0x26a >>>> V [libjvm.so+0x16cd7e0] Universe::genesis(Thread*)+0x7c0 >>>> V [libjvm.so+0x16ce4cc] universe2_init()+0x2c >>>> V [libjvm.so+0xe0f268] init_globals()+0xa8 >>>> V [libjvm.so+0x1696e6f] Threads::create_vm(JavaVMInitArgs*, >>>> bool*)+0x2df >>>> ... >>>> >>>> So, is the existing synchronization in this change sufficient? >>>> >>>> Thanks, Harold >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>>> Thanks, Harold >>>>>> >>>>>> On 5/4/2017 5:32 PM, David Holmes wrote: >>>>>>> Hi Harold, >>>>>>> >>>>>>> On 5/05/2017 3:47 AM, harold seigel wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> Please review this updated webrev: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604.2/webrev/index.html >>>>>>>> >>>>>>>> >>>>>>>> The updated webrev takes out a lock when creating >>>>>>>> _exploded_entries. It >>>>>>>> also contains additional comments and an assert() to address >>>>>>>> Lois's >>>>>>>> concerns with reduced code readability involving what situations >>>>>>>> warrant >>>>>>>> taking out a lock in ClassLoader::search_module_entries(). >>>>>>>> >>>>>>>> I don't think there needs to be two different initial entry >>>>>>>> points as >>>>>>>> suggested by David below. The method behaves slightly differently >>>>>>>> depending on the value of its 'need_lock' parameter. But, there is >>>>>>>> enough common code for it to remain one method. >>>>>>> >>>>>>> I think this highlights uncertainty about the concurrent >>>>>>> behaviour in >>>>>>> relation to this code. On the one hand you are worried about an >>>>>>> initialization race and so use the lock, but when you do: >>>>>>> >>>>>>> 1500 // The exploded build entries can be added to at any >>>>>>> time >>>>>>> so pass 'true' for >>>>>>> 1501 // need_lock so that a lock is taken out when searching >>>>>>> them. >>>>>>> 1502 assert(_exploded_entries != NULL, "No exploded build >>>>>>> entries present"); >>>>>>> 1503 stream = search_module_entries(_exploded_entries, >>>>>>> class_name, file_name, true, CHECK_NULL); >>>>>>> >>>>>>> you load _exploded_entries with no lock held and so must be certain >>>>>>> that you can not possibly read a null value here. >>>>>>> >>>>>>> The needs_lock parameter seems superfluous - you know the condition >>>>>>> for locking is that the list is the _exploded_entries, and you >>>>>>> assert >>>>>>> that. So no need to pass in needs_lock, just grab the lock when >>>>>>> dealing with _exploded_entries. >>>>>>> >>>>>>> Cheers, >>>>>>> David >>>>>>> >>>>>>>> Thanks, Harold >>>>>>>> >>>>>>>> On 5/3/2017 12:59 AM, David Holmes wrote: >>>>>>>>> On 3/05/2017 7:11 AM, Lois Foltan wrote: >>>>>>>>>> On 5/2/2017 9:03 AM, harold seigel wrote: >>>>>>>>>>> Hi Lois, >>>>>>>>>>> >>>>>>>>>>> Thanks for your comments. Please see my in-line responses. >>>>>>>>>>> >>>>>>>>>>> Harold >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 5/1/2017 4:14 PM, Lois Foltan wrote: >>>>>>>>>>>> Hi Harold, >>>>>>>>>>>> >>>>>>>>>>>> Looks good. A couple of comments: >>>>>>>>>>>> >>>>>>>>>>>> src/share/vm/classfile/classLoader.cpp >>>>>>>>>>>> line #828 - please take out the Module_lock when the >>>>>>>>>>>> _exploded_entries is created >>>>>>>>>>> This fix does not change the possible need for >>>>>>>>>>> synchronization when >>>>>>>>>>> _exploded_entries gets created during module system >>>>>>>>>>> initialization. >>>>>>>>>>> Are you saying that the existing code is wrong and should have >>>>>>>>>>> taken >>>>>>>>>>> out the Module_lock before creating _exploding_entries? >>>>>>>>>> >>>>>>>>>> The method ClassLoader::add_to_exploded_build_list can be >>>>>>>>>> called at >>>>>>>>>> anytime (during module system initialization and post module >>>>>>>>>> system >>>>>>>>>> initialization). Thus it seems prudent that no matter what the >>>>>>>>>> current >>>>>>>>>> JVM phase, that any access to ClassLoader::_exploding_entries be >>>>>>>>>> protected via a lock. That includes creation as well as >>>>>>>>>> insertion >>>>>>>>>> into >>>>>>>>>> the list. >>>>>>>>> >>>>>>>>> If creation is not guaranteed to occur before other threads >>>>>>>>> that will >>>>>>>>> call the same method and use the _exploded_entries list, then >>>>>>>>> synchronization of some form is essential. If it is guaranteed >>>>>>>>> then we >>>>>>>>> don't need create time sync just to be prudent - but the >>>>>>>>> guarantee >>>>>>>>> must be firmly established and clearly documented. >>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> line #1402 - I like the new factored out method >>>>>>>>>>>> find_first_module_cpe(), however, instead of adding a new >>>>>>>>>>>> "need_lock" >>>>>>>>>>>> parameter, I would rather see code >>>>>>>>>>>> in find_first_module_cpe() that >>>>>>>>>>>> takes the >>>>>>>>>>>> Module_lock if the module_list parameter equals >>>>>>>>>>>> ClassLoader::_exploded_entries >>>>>>>>>>> I think your suggestion makes the code less flexible and >>>>>>>>>>> does not >>>>>>>>>>> require potential future callers of search_module_entries() to >>>>>>>>>>> think >>>>>>>>>>> about synchronization. So, I'd prefer to leave that change >>>>>>>>>>> as is. >>>>>>>>>> >>>>>>>>>> I see your point. My point was based on reduced code >>>>>>>>>> readability, the >>>>>>>>>> further away from the decision to establish the lock, the less >>>>>>>>>> apparent >>>>>>>>>> it is within the new method ClassLoader::find_first_module_cpe() >>>>>>>>>> as to >>>>>>>>>> what situations warranted the lock in the first place. >>>>>>>>> >>>>>>>>> Further, this: >>>>>>>>> >>>>>>>>> 1495 stream = search_module_entries(_exploded_entries, >>>>>>>>> class_name, file_name, true, CHECK_NULL); >>>>>>>>> >>>>>>>>> is potentially unsafe if we can race with creation because we >>>>>>>>> first >>>>>>>>> access _exploded_entries without holding any lock! And the >>>>>>>>> fact the >>>>>>>>> method needs to do different things depending on which "list" was >>>>>>>>> passed in suggests to me it should be two different initial entry >>>>>>>>> points. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> David >>>>>>>>> ----- >>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Lois >>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Lois >>>>>>>>>>>> >>>>>>>>>>>> On 5/1/2017 3:36 PM, harold seigel wrote: >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> Please review this JDK-10 bug fix to allow defining of >>>>>>>>>>>>> modules to >>>>>>>>>>>>> the boot loader in exploded builds after module system >>>>>>>>>>>>> initialization. The fix uses the module lock to synchronize >>>>>>>>>>>>> access >>>>>>>>>>>>> to the _exploded_entries data structure (which is only >>>>>>>>>>>>> used by >>>>>>>>>>>>> exploded builds). >>>>>>>>>>>>> >>>>>>>>>>>>> Note that the above capability already exists for regular >>>>>>>>>>>>> builds. >>>>>>>>>>>>> >>>>>>>>>>>>> Open Webrev: >>>>>>>>>>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604/webrev/index.html >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8178604 >>>>>>>>>>>>> >>>>>>>>>>>>> The fix was tested with JCK tests, the JTreg hotspot, >>>>>>>>>>>>> java/io, >>>>>>>>>>>>> java/lang, java/util and other tests, the RBT tier2 -tier5 >>>>>>>>>>>>> tests, >>>>>>>>>>>>> the co-located NSK tests, and with JPRT. The JTReg and JCK >>>>>>>>>>>>> tests >>>>>>>>>>>>> were run with an exploded build. Also, the example >>>>>>>>>>>>> program in >>>>>>>>>>>>> the >>>>>>>>>>>>> JBS bug was run by hand to verify the fix. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, Harold >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>> >>>> >> > From robbin.ehn at oracle.com Thu May 11 11:18:01 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Thu, 11 May 2017 13:18:01 +0200 Subject: RFR (jdk10): JDK-8180175: ObjectSynchronizer only needs to iterate in-use monitors In-Reply-To: <89399a72-0263-5ebd-5fda-82f404fbe2ca@redhat.com> References: <89399a72-0263-5ebd-5fda-82f404fbe2ca@redhat.com> Message-ID: <6b162e43-3b0f-9e23-e63f-c844b0bc21da@oracle.com> Adding gc list. Thanks! /Robbin On 05/11/2017 12:08 PM, Roman Kennke wrote: > Hi, > > ObjectSynchronizer::oops_do() currently iterates all monitor blocks, > even free ones. With -XX:+MonitorInUseLists, each thread has its own > in-use monitors list, and ObjectSynchronizer has a global gOmInUseList > for moribund threads. Iterating over in-use monitors only significantly > inmproves scanning time for monitors, because it avoids scanning free > blocks, improves caching (no padding, and better chances to have > thread-local oops in cache already). > > Performance-wise it makes a very significant difference (running > gc-bench's roots.Sync test, which exaggerates synchronizer usage, with > Shenandoah): > > baseline: > S: Thread Roots 37748 us > S: Synchronizer Roots 15115 us > UR: Thread Roots 24967 us > UR: Synchronizer Roots 11906 us > > patched: > S: Thread Roots 40365 us > S: Synchronizer Roots 0 us > UR: Thread Roots 24459 us > UR: Synchronizer Roots 0 us > > Testing: hotspot_gc, specjvm, jcstress -m quick > > http://cr.openjdk.java.net/~rkennke/8180175/webrev.00/ > > > Roman > > From mikael.vidstedt at oracle.com Thu May 11 15:51:33 2017 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Thu, 11 May 2017 08:51:33 -0700 Subject: RFR(S): 8180039: Use more portable print format/arguments for rlim_t In-Reply-To: References: <78FE21D0-F4EA-47B5-AC90-7D69990B45F0@oracle.com> Message-ID: <65F6B28B-970B-473C-9C28-F0D1117C011C@oracle.com> David/Mikael/Thomas, Thanks for the reviews! > On May 10, 2017, at 3:51 AM, Thomas St?fe wrote: > > Hi Mikael, > > this looks fine. > > While you are on it, would it be possible to add printouts for DATA and FSIZE? I filed a separate follow up enhancement for this: https://bugs.openjdk.java.net/browse/JDK-8180184 "Add DATA and FSIZE to os::Posix::print_rlimit_info? and will look at that separately. > Another small nit, I would prefer /1024 instead of >>10. I agree, and will (likely) make that change as part of the follow up work. If we want to over-engineer it we can also use the globalDefinitions.hpp helper functions (proper_unit_for_byte_size & byte_size_in_proper_unit) to print out the values using different suffixes depending on the size of the value. :) Cheers, Mikael > > Kind Regards, Thomas > > On Wed, May 10, 2017 at 1:38 AM, Mikael Vidstedt > wrote: > > Please review this small change which updates the code in the print_rlimit_info method in os_posix.cpp which prints out rlim_t/rlim_cur values (returned from getrlimit) to use more portable format specifiers and arguments/argument types. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8180039 > > Webrev: http://cr.openjdk.java.net/~mikael/webrevs/8180039/webrev.00/hotspot/webrev/ > > > The spec says that rlim_t is an "unsigned integer type?, so casting it to a uint64_t will either zero extend it (if the rlim_t type is 32-bit), or end up with the exact same width/value (if rlim_t is 64-bit). Printing it out as a 64-bit unsigned type (using UINT64_FORMAT) will handle both the 64-bit case and the zero extended 32-bit case. > > Cheers, > Mikael > > From thomas.schatzl at oracle.com Thu May 11 16:04:03 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 11 May 2017 18:04:03 +0200 Subject: [9, P1] RFR (S/M): 8180048: Interned string and symbol table leak memory during parallel unlinking Message-ID: <1494518643.3120.18.camel@oracle.com> Hi all, ? can I have reviews for this change that fixes a memory leak when unlinking (removing) elements from the string and symbol tables in parallel? The problem is the use of BasicHashtable::free_entry() that in a non- thread safe way adds elements to the hash table's bucket free list. If multiple threads do that, the unlinking process looses elements/memory forever. This can be quite serious, the reproducer attached to the CR leaks ~1GB of memory within like a minute or so. It only affects G1 as it is the only collector that does string/symbol table unlinking in parallel. There is no test for this issue at this time: the minimum duration of the test (that is similar to the reproducer) takes a significant amount of time (>15s in product build per GC on my machine) to leak enough memory so that a leak can be detected with reasonable accuracy. This makes it kind of unusable to run every time during jprt. I am going to add a CR to add a proper test, trying to improve what I have, and if I can't get it down to a reasonable runtime make it so that is run at the appropriate tier only. I am also going to try to get this change into 8u. Ideally I would also like to have a reviewer from the runtime team (cc'ed). CR: https://bugs.openjdk.java.net/browse/JDK-8180048 Webrev: http://cr.openjdk.java.net/~tschatzl/8180048/webrev/ Testing: jprt, manual testing Thanks, ? Thomas From shade at redhat.com Thu May 11 16:26:31 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 11 May 2017 18:26:31 +0200 Subject: [9, P1] RFR (S/M): 8180048: Interned string and symbol table leak memory during parallel unlinking In-Reply-To: <1494518643.3120.18.camel@oracle.com> References: <1494518643.3120.18.camel@oracle.com> Message-ID: On 05/11/2017 06:04 PM, Thomas Schatzl wrote: > It only affects G1 as it is the only collector that does string/symbol > table unlinking in parallel. Shenandoah too, it actually shares a part of G1 code for that. > CR: > https://bugs.openjdk.java.net/browse/JDK-8180048 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8180048/webrev/ So the idea is to collect pending ops to BucketUnlinkContext, and then purge them in BHT::bulk_free_entries. Sounds sane to me, probably also has performance benefits. Nits: *) Modifier order? 176 BasicHashtableEntry* volatile _free_list; Otherwise seems fine. Thanks, -Aleksey From mikael.vidstedt at oracle.com Thu May 11 17:01:00 2017 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Thu, 11 May 2017 10:01:00 -0700 Subject: RFR(XS): 8180036: Guard include of fpu_control.h In-Reply-To: <477967b1-8e2c-f1ea-b9c0-8595c2297f24@oracle.com> References: <7D4A9A57-5855-4664-A78A-15F96250ADC9@oracle.com> <477967b1-8e2c-f1ea-b9c0-8595c2297f24@oracle.com> Message-ID: Thanks! Cheers, Mikael > On May 9, 2017, at 4:21 PM, David Holmes wrote: > > Looks trivially fine. > > Thanks, > David > > On 10/05/2017 8:59 AM, Mikael Vidstedt wrote: >> >> Please review this small change which makes the inclusion of fpu_control.h in os_linux_x86.c conditional: >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8180036 >> Webrev: http://cr.openjdk.java.net/~mikael/webrevs/8180036/webrev.00/hotspot/webrev/ >> >> The reason why the file is included in the first place is that on 32-bit x86 there is some logic in os_linux_x86.c to set/get/change the FPU control word using functions implemented by fpu_control.h. This is only done on 32-bit x86, and the code in question is guarded by #ifndef AMD64. However, the inclusion of the fpu_control.h is unconditional. This change makes that consistent by adding the same #ifndef AMD64 guard around the #include. >> >> Cheers, >> Mikael >> From david.holmes at oracle.com Fri May 12 01:38:07 2017 From: david.holmes at oracle.com (David Holmes) Date: Fri, 12 May 2017 11:38:07 +1000 Subject: [9, P1] RFR (S/M): 8180048: Interned string and symbol table leak memory during parallel unlinking In-Reply-To: References: <1494518643.3120.18.camel@oracle.com> Message-ID: <1ff3056e-2f54-6f0f-f573-70412d1082dc@oracle.com> On 12/05/2017 2:26 AM, Aleksey Shipilev wrote: > On 05/11/2017 06:04 PM, Thomas Schatzl wrote: >> It only affects G1 as it is the only collector that does string/symbol >> table unlinking in parallel. > > Shenandoah too, it actually shares a part of G1 code for that. > >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8180048 >> Webrev: >> http://cr.openjdk.java.net/~tschatzl/8180048/webrev/ > > So the idea is to collect pending ops to BucketUnlinkContext, and then purge > them in BHT::bulk_free_entries. Sounds sane to me, probably also has performance > benefits. > > Nits: > > *) Modifier order? > 176 BasicHashtableEntry* volatile _free_list; _free_list is updated lock-free using Atomic::cmpxchg_ptr so it is appropriate to mark it as being volatile. David > Otherwise seems fine. > > Thanks, > -Aleksey > > > > From ioi.lam at oracle.com Fri May 12 05:21:24 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 11 May 2017 22:21:24 -0700 Subject: RFR[S] 8179769 serviceability/sa/TestCpoolForInvokeDynamic.java failing after changes for JDK-8171392 Message-ID: <59154654.9070005@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8179769 http://cr.openjdk.java.net/~iklam/jdk10/8179769-sa-cp-failures.v01/ Summary: In JDK-8171392 ( Move Klass pointers outside of ConstantPool entries so ConstantPool can be read-only), I changed how classes are stored in ConstantPool. However, I missed the changes that are required in Serviceability Agent (SA). In this patch, the SA java classes are changed to match the native code in constantPool.cpp. Testing: I tested with "rbt -j hs-tier2.js,hs-tier3.js" to run all the SA JTREG tests on all supported platforms. I also tested locally and verified in the JTR files that the bug has been fixed Thanks - Ioi From goetz.lindenmaier at sap.com Fri May 12 06:45:32 2017 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 12 May 2017 06:45:32 +0000 Subject: RFR(S): 8179953: [ppc] TLABWasteIncrement not loaded correctly In-Reply-To: References: Message-ID: Hi David, the broken code is protected by if (allow_shared_alloc) { which evaluates to true with ParallelGC but not with G1GC. It might be true with other GCs, too, but the design of JVMOptionsUtils:addNameDependency() is quite simple, and I just did it as for other flags. I could put in more logic into that case there, like if (GCType == G1GC || GCTyep == ... ) { option.addPrepend("-XX:+UseParallelGC"); } enumerating all GCs that don't support TLABWastIncrement. But I thought one test case for the flag is fine. The way it is we missed the error, so I would like to fix the test. Yes, in JVMOption I must exclude setting G1GC else I get an error claiming that two different GCs are set. Best regards, Goetz. > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Wednesday, May 10, 2017 5:19 AM > To: Lindenmaier, Goetz ; hotspot-runtime- > dev at openjdk.java.net > Subject: Re: RFR(S): 8179953: [ppc] TLABWasteIncrement not loaded > correctly > > Hi Goetz, > > On 10/05/2017 12:56 AM, Lindenmaier, Goetz wrote: > > Hi, > > > > Please review this change. As I fix the test, I please need a sponsor. > > http://cr.openjdk.java.net/~goetz/wr17/8179953-ppc_flag/webrev.01/ > > I'm unclear on the test changes. It isn't obvious to me that > TLABWasteIncrement is a ParallelGC only flag. By adding that as an > additional flag you then had to exclude adding the GCType (if any GC > selector is given) to avoid conflicting options - is that right? > > Thanks, > David > > > On ppc, the mentioned flag is loaded as a 16 bit immediate, while it can > have > > bigger values. > > I also adapt TestOptionWithRanges, which did not show the bug because > > the flag has no effect with G1. > > > > Best regards, > > Goetz. > > > > From shade at redhat.com Fri May 12 06:49:22 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 12 May 2017 08:49:22 +0200 Subject: [9, P1] RFR (S/M): 8180048: Interned string and symbol table leak memory during parallel unlinking In-Reply-To: <1ff3056e-2f54-6f0f-f573-70412d1082dc@oracle.com> References: <1494518643.3120.18.camel@oracle.com> <1ff3056e-2f54-6f0f-f573-70412d1082dc@oracle.com> Message-ID: <07a893f3-3fa1-a651-8b33-12b6961923e7@redhat.com> On 05/12/2017 03:38 AM, David Holmes wrote: > On 12/05/2017 2:26 AM, Aleksey Shipilev wrote: >> Nits: >> >> *) Modifier order? >> 176 BasicHashtableEntry* volatile _free_list; > > _free_list is updated lock-free using Atomic::cmpxchg_ptr so it is appropriate > to mark it as being volatile. Of course. My comment was about the modifier _order_, see in the same file: 176 BasicHashtableEntry* volatile _free_list; ... 180 volatile int _number_of_entries; I think Hotspot uses "volatile $type $id" everywhere I saw. Thanks, -Aleksey From mikael.gerdin at oracle.com Fri May 12 07:28:39 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Fri, 12 May 2017 09:28:39 +0200 Subject: [9, P1] RFR (S/M): 8180048: Interned string and symbol table leak memory during parallel unlinking In-Reply-To: <07a893f3-3fa1-a651-8b33-12b6961923e7@redhat.com> References: <1494518643.3120.18.camel@oracle.com> <1ff3056e-2f54-6f0f-f573-70412d1082dc@oracle.com> <07a893f3-3fa1-a651-8b33-12b6961923e7@redhat.com> Message-ID: Hi Aleksey, On 2017-05-12 08:49, Aleksey Shipilev wrote: > On 05/12/2017 03:38 AM, David Holmes wrote: >> On 12/05/2017 2:26 AM, Aleksey Shipilev wrote: >>> Nits: >>> >>> *) Modifier order? >>> 176 BasicHashtableEntry* volatile _free_list; >> >> _free_list is updated lock-free using Atomic::cmpxchg_ptr so it is appropriate >> to mark it as being volatile. > > Of course. My comment was about the modifier _order_, see in the same file: > > 176 BasicHashtableEntry* volatile _free_list; For pointer types there is actually a difference between volatile Foo* _ptr; and Foo* volatile _ptr; in the first case _ptr is a non-volatile pointer to a volatile Foo and in the second case _ptr is a volatile pointer to a non-volatile Foo. /Mikael > ... > 180 volatile int _number_of_entries; > > I think Hotspot uses "volatile $type $id" everywhere I saw. > > Thanks, > -Aleksey > > From david.holmes at oracle.com Fri May 12 07:33:55 2017 From: david.holmes at oracle.com (David Holmes) Date: Fri, 12 May 2017 17:33:55 +1000 Subject: [9, P1] RFR (S/M): 8180048: Interned string and symbol table leak memory during parallel unlinking In-Reply-To: <07a893f3-3fa1-a651-8b33-12b6961923e7@redhat.com> References: <1494518643.3120.18.camel@oracle.com> <1ff3056e-2f54-6f0f-f573-70412d1082dc@oracle.com> <07a893f3-3fa1-a651-8b33-12b6961923e7@redhat.com> Message-ID: <87095650-03dc-e565-0e9b-0b46a96973be@oracle.com> On 12/05/2017 4:49 PM, Aleksey Shipilev wrote: > On 05/12/2017 03:38 AM, David Holmes wrote: >> On 12/05/2017 2:26 AM, Aleksey Shipilev wrote: >>> Nits: >>> >>> *) Modifier order? >>> 176 BasicHashtableEntry* volatile _free_list; >> >> _free_list is updated lock-free using Atomic::cmpxchg_ptr so it is appropriate >> to mark it as being volatile. > > Of course. My comment was about the modifier _order_, see in the same file: The order is critical for correctness. > 176 BasicHashtableEntry* volatile _free_list; > ... > 180 volatile int _number_of_entries; > > I think Hotspot uses "volatile $type $id" everywhere I saw. It's different for pointers: BasicHashtableEntry* volatile _free_list; _free_list is a volatile pointer to a non-volatile entry volatile BasicHashtableEntry* _free_list; _free_list is a non-volatile pointer to a volatile entry volatile BasicHashtableEntry* volatile _free_list; _free_list is a volatile pointer to a volatile entry (I don't claim all volatile decls in hotspot necessarily have this right!) Cheers, David > Thanks, > -Aleksey > > From shade at redhat.com Fri May 12 07:35:55 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 12 May 2017 09:35:55 +0200 Subject: [9, P1] RFR (S/M): 8180048: Interned string and symbol table leak memory during parallel unlinking In-Reply-To: References: <1494518643.3120.18.camel@oracle.com> <1ff3056e-2f54-6f0f-f573-70412d1082dc@oracle.com> <07a893f3-3fa1-a651-8b33-12b6961923e7@redhat.com> Message-ID: On 05/12/2017 09:28 AM, Mikael Gerdin wrote: > Hi Aleksey, > > On 2017-05-12 08:49, Aleksey Shipilev wrote: >> On 05/12/2017 03:38 AM, David Holmes wrote: >>> On 12/05/2017 2:26 AM, Aleksey Shipilev wrote: >>>> Nits: >>>> >>>> *) Modifier order? >>>> 176 BasicHashtableEntry* volatile _free_list; >>> >>> _free_list is updated lock-free using Atomic::cmpxchg_ptr so it is appropriate >>> to mark it as being volatile. >> >> Of course. My comment was about the modifier _order_, see in the same file: >> >> 176 BasicHashtableEntry* volatile _free_list; > > For pointer types there is actually a difference between > > volatile Foo* _ptr; > > and > > Foo* volatile _ptr; > > in the first case _ptr is a non-volatile pointer to a volatile Foo and in the > second case _ptr is a volatile pointer to a non-volatile Foo. Argh. Gotta love this language. This is not the first time I forget about this difference. Thanks, -Aleksey From shade at redhat.com Fri May 12 07:39:11 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 12 May 2017 09:39:11 +0200 Subject: [9, P1] RFR (S/M): 8180048: Interned string and symbol table leak memory during parallel unlinking In-Reply-To: <87095650-03dc-e565-0e9b-0b46a96973be@oracle.com> References: <1494518643.3120.18.camel@oracle.com> <1ff3056e-2f54-6f0f-f573-70412d1082dc@oracle.com> <07a893f3-3fa1-a651-8b33-12b6961923e7@redhat.com> <87095650-03dc-e565-0e9b-0b46a96973be@oracle.com> Message-ID: <4bbf9f00-5c82-10ac-9e30-4770e9ed9244@redhat.com> On 05/12/2017 09:33 AM, David Holmes wrote: > On 12/05/2017 4:49 PM, Aleksey Shipilev wrote: >> On 05/12/2017 03:38 AM, David Holmes wrote: >>> On 12/05/2017 2:26 AM, Aleksey Shipilev wrote: >>>> Nits: >>>> >>>> *) Modifier order? >>>> 176 BasicHashtableEntry* volatile _free_list; >>> >>> _free_list is updated lock-free using Atomic::cmpxchg_ptr so it is appropriate >>> to mark it as being volatile. >> >> Of course. My comment was about the modifier _order_, see in the same file: > > The order is critical for correctness. Yes, got that. Sorry for distracting. Let me reiterate: the patch looks good, and I can't wait to pull it in to Shenandoah dev forest :) -Aleksey From thomas.stuefe at gmail.com Fri May 12 08:00:45 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 12 May 2017 10:00:45 +0200 Subject: RFR(S): 8180039: Use more portable print format/arguments for rlim_t In-Reply-To: <65F6B28B-970B-473C-9C28-F0D1117C011C@oracle.com> References: <78FE21D0-F4EA-47B5-AC90-7D69990B45F0@oracle.com> <65F6B28B-970B-473C-9C28-F0D1117C011C@oracle.com> Message-ID: Hi Mikael, On Thu, May 11, 2017 at 5:51 PM, Mikael Vidstedt wrote: > > David/Mikael/Thomas, > > Thanks for the reviews! > > On May 10, 2017, at 3:51 AM, Thomas St?fe wrote: > > Hi Mikael, > > this looks fine. > > While you are on it, would it be possible to add printouts for DATA > and FSIZE? > > > Thanks! DATA especially is interesting on AIX, and I assume Solaris as well. > I filed a separate follow up enhancement for this: > https://bugs.openjdk.java.net/browse/JDK-8180184 "Add DATA and FSIZE to > os::Posix::print_rlimit_info? and will look at that separately. > > Another small nit, I would prefer /1024 instead of >>10. > > > I agree, and will (likely) make that change as part of the follow up work. > > Thanks! > If we want to over-engineer it we can also use the globalDefinitions.hpp > helper functions (proper_unit_for_byte_size & byte_size_in_proper_unit) to > print out the values using different suffixes depending on the size of the > value. :) > No need :) ..Thomas > Cheers,Mikael > > > Kind Regards, Thomas > > On Wed, May 10, 2017 at 1:38 AM, Mikael Vidstedt < > mikael.vidstedt at oracle.com> wrote: > >> >> Please review this small change which updates the code in the >> print_rlimit_info method in os_posix.cpp which prints out rlim_t/rlim_cur >> values (returned from getrlimit) to use more portable format specifiers and >> arguments/argument types. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8180039 < >> https://bugs.openjdk.java.net/browse/JDK-8180039> >> Webrev: http://cr.openjdk.java.net/~mikael/webrevs/8180039/webrev.00 >> /hotspot/webrev/ > ikael/webrevs/8180039/webrev.00/hotspot/webrev/> >> >> The spec says that rlim_t is an "unsigned integer type?, so casting it to >> a uint64_t will either zero extend it (if the rlim_t type is 32-bit), or >> end up with the exact same width/value (if rlim_t is 64-bit). Printing it >> out as a 64-bit unsigned type (using UINT64_FORMAT) will handle both the >> 64-bit case and the zero extended 32-bit case. >> >> Cheers, >> Mikael >> >> > > From thomas.schatzl at oracle.com Fri May 12 08:38:22 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 12 May 2017 10:38:22 +0200 Subject: [9, P1] RFR (S/M): 8180048: Interned string and symbol table leak memory during parallel unlinking In-Reply-To: References: <1494518643.3120.18.camel@oracle.com> Message-ID: <1494578302.2459.8.camel@oracle.com> Hi Aleksey, ? thanks for your review. On Thu, 2017-05-11 at 18:26 +0200, Aleksey Shipilev wrote: > On 05/11/2017 06:04 PM, Thomas Schatzl wrote: > > > > It only affects G1 as it is the only collector that does > > string/symbol table unlinking in parallel. > Shenandoah too, it actually shares a part of G1 code for that. > > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8180048 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8180048/webrev/ > So the idea is to collect pending ops to BucketUnlinkContext, and > then purge them in BHT::bulk_free_entries. Sounds sane to me, > probably also has performance benefits. > Since the question came up and I had already sent the email when noticing this. I figured out there are three options to fix this: 1) make string/symbol table unlinking single threaded like other collectors do. Will cause massive performance regressions in some cases. 2) do the move of old entries to the free list using atomics by fixing BasicHashtable::free_entry(), entry by entry. This may cause performance regressions as this is somewhat equivalent to a global lock on the _free_list member. 3) during unlinking, collect entries in thread local free lists that are merged into the global free list at the end. Has the least (if any) performance impact, but most complicated of the three options (but has been done elsewhere a few times for the same reasons).? I opted to implement 3) here, i.e. every thread (caller of buckets_unlink_or_oops_do() for the string table) collects all unlinked BasicHashtableEntrys into a local linked list (stored in the BucketUnlinkContext to avoid lots of additional parameters to the method), and then at the very end every thread adds the locally collected list to the global free list once. Then there is only one atomic op for every thread at the end, minimizing contention. However that has not been the goal, rather minimum performance impact of the change. > Nits: > > ?*) Modifier order? > ? 176???BasicHashtableEntry* volatile _free_list; > > Otherwise seems fine. Thanks, ? Thomas From mikael.gerdin at oracle.com Fri May 12 08:59:03 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Fri, 12 May 2017 10:59:03 +0200 Subject: RFR (S) 8180181: Get rid of FAST_DISPATCH from interpreter Message-ID: <11715a26-8dc4-19d8-d516-b805bb260963@oracle.com> Hi, Please review this change to delete the FAST_DISPATCH support from the SPARC template interpreter. This code has been disabled for 15+ years: # 1.103.3.4 22-Aug-2000 rasbold undef FAST_DISPATCH; fix merge boo-boos I think it's safe to remove it now. The FAST_DISPATCH code has also leaked into the s390 port with references to SPARC registers and all so that should go away as well. I've ran it through JPRT on SPARC. Webrev: http://cr.openjdk.java.net/~mgerdin/8180181/webrev.0/ Bug: https://bugs.openjdk.java.net/browse/JDK-8180181 Thanks /Mikael From thomas.schatzl at oracle.com Fri May 12 10:46:42 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 12 May 2017 12:46:42 +0200 Subject: [9, P1] RFR (S/M): 8180048: Interned string and symbol table leak memory during parallel unlinking In-Reply-To: <1494518643.3120.18.camel@oracle.com> References: <1494518643.3120.18.camel@oracle.com> Message-ID: <1494586002.2459.13.camel@oracle.com> Hi again, On Thu, 2017-05-11 at 18:04 +0200, Thomas Schatzl wrote: > Hi all, > > ? can I have reviews for this change that fixes a memory leak when > unlinking (removing) elements from the string and symbol tables in > parallel? > > [...] > > I am also going to try to get this change into 8u. > > Ideally I would also like to have a reviewer from the runtime team > (cc'ed). > > CR: > https://bugs.openjdk.java.net/browse/JDK-8180048 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8180048/webrev/ > Testing: > jprt, manual testing ? I got some comments from Erik Helin, with the following webrev as result: http://cr.openjdk.java.net/~tschatzl/8180048/webrev.0_to_1/ (diff) http://cr.openjdk.java.net/~tschatzl/8180048/webrev.1/ (full) Testing: jprt Thanks, ? Thomas From erik.helin at oracle.com Fri May 12 11:08:11 2017 From: erik.helin at oracle.com (Erik Helin) Date: Fri, 12 May 2017 13:08:11 +0200 Subject: [9, P1] RFR (S/M): 8180048: Interned string and symbol table leak memory during parallel unlinking In-Reply-To: <1494586002.2459.13.camel@oracle.com> References: <1494518643.3120.18.camel@oracle.com> <1494586002.2459.13.camel@oracle.com> Message-ID: <9799ab18-1141-4503-930c-d68b28cfd0e4@oracle.com> On 05/12/2017 12:46 PM, Thomas Schatzl wrote: > Hi again, > > On Thu, 2017-05-11 at 18:04 +0200, Thomas Schatzl wrote: >> Hi all, >> >> can I have reviews for this change that fixes a memory leak when >> unlinking (removing) elements from the string and symbol tables in >> parallel? >> >> [...] >> >> I am also going to try to get this change into 8u. >> >> Ideally I would also like to have a reviewer from the runtime team >> (cc'ed). >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8180048 >> Webrev: >> http://cr.openjdk.java.net/~tschatzl/8180048/webrev/ >> Testing: >> jprt, manual testing > > I got some comments from Erik Helin, with the following webrev as > result: > > http://cr.openjdk.java.net/~tschatzl/8180048/webrev.0_to_1/ (diff) > http://cr.openjdk.java.net/~tschatzl/8180048/webrev.1/ (full) Looks good, Reviewed. Thanks for fixing this Thomas! Erik > Testing: > jprt > > Thanks, > Thomas > From coleen.phillimore at oracle.com Fri May 12 12:05:47 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 12 May 2017 08:05:47 -0400 Subject: RFR (S) 8180181: Get rid of FAST_DISPATCH from interpreter In-Reply-To: <11715a26-8dc4-19d8-d516-b805bb260963@oracle.com> References: <11715a26-8dc4-19d8-d516-b805bb260963@oracle.com> Message-ID: <16c824d9-dfa9-eb3e-dd92-476b35f8fa62@oracle.com> Looks great, thank you for removing it! We may have experimented with it as late as 2004 :) thanks!! Coleen On 5/12/17 4:59 AM, Mikael Gerdin wrote: > Hi, > > Please review this change to delete the FAST_DISPATCH support from the > SPARC template interpreter. This code has been disabled for 15+ years: > > # 1.103.3.4 22-Aug-2000 rasbold undef FAST_DISPATCH; fix merge boo-boos > > I think it's safe to remove it now. > The FAST_DISPATCH code has also leaked into the s390 port with > references to SPARC registers and all so that should go away as well. > > I've ran it through JPRT on SPARC. > > Webrev: http://cr.openjdk.java.net/~mgerdin/8180181/webrev.0/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8180181 > > Thanks > /Mikael From coleen.phillimore at oracle.com Fri May 12 12:07:51 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 12 May 2017 08:07:51 -0400 Subject: RFR[S] 8179769 serviceability/sa/TestCpoolForInvokeDynamic.java failing after changes for JDK-8171392 In-Reply-To: <59154654.9070005@oracle.com> References: <59154654.9070005@oracle.com> Message-ID: Looks good. Coleen On 5/12/17 1:21 AM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8179769 > http://cr.openjdk.java.net/~iklam/jdk10/8179769-sa-cp-failures.v01/ > > Summary: > > In JDK-8171392 ( Move Klass pointers outside of ConstantPool entries > so ConstantPool can be read-only), I changed how classes are stored in > ConstantPool. However, I missed the changes that are required in > Serviceability Agent (SA). > > In this patch, the SA java classes are changed to match the native > code in constantPool.cpp. > > Testing: > > I tested with "rbt -j hs-tier2.js,hs-tier3.js" to run all the SA JTREG > tests on all supported platforms. I also tested locally and verified > in the JTR files that the bug has been fixed > > Thanks > - Ioi From thomas.schatzl at oracle.com Fri May 12 14:32:36 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 12 May 2017 16:32:36 +0200 Subject: RFR (S) 8180181: Get rid of FAST_DISPATCH from interpreter In-Reply-To: <11715a26-8dc4-19d8-d516-b805bb260963@oracle.com> References: <11715a26-8dc4-19d8-d516-b805bb260963@oracle.com> Message-ID: <1494599556.2677.0.camel@oracle.com> Hi, On Fri, 2017-05-12 at 10:59 +0200, Mikael Gerdin wrote: > Hi, > > Please review this change to delete the FAST_DISPATCH support from > the? > SPARC template interpreter. This code has been disabled for 15+ > years: > > # 1.103.3.4 22-Aug-2000 rasbold undef FAST_DISPATCH; fix merge boo- > boos > > I think it's safe to remove it now. > The FAST_DISPATCH code has also leaked into the s390 port with? > references to SPARC registers and all so that should go away as well. > > I've ran it through JPRT on SPARC. > > Webrev: http://cr.openjdk.java.net/~mgerdin/8180181/webrev.0/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8180181 ? looks good to me. Thomas From george.triantafillou at oracle.com Fri May 12 14:34:39 2017 From: george.triantafillou at oracle.com (George Triantafillou) Date: Fri, 12 May 2017 10:34:39 -0400 Subject: RFR 8179903: Clean up SPARC 32-bit support Message-ID: Please review this fix to clean up SPARC 32-bit support. JBS: https://bugs.openjdk.java.net/browse/JDK-8179903 webrev: http://cr.openjdk.java.net/~gtriantafill/8179903-webrev/webrev/index.html This is a followup RFE to "JDK-8150388 Remove SPARC 32-bit support". The work includes addressing formatting, 32-bit comments, and other issues that Kim raised after JDK-8150388 was reviewed and checked in. Built and tested on solaris-sparcv9-debug, solaris-x64-debug with the nsk.jvmti, nsk.jdwp, and nsk.jdi testlists. Thanks. -George From ioi.lam at oracle.com Fri May 12 15:46:46 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 12 May 2017 08:46:46 -0700 Subject: RFR[S] 8179769 serviceability/sa/TestCpoolForInvokeDynamic.java failing after changes for JDK-8171392 In-Reply-To: References: <59154654.9070005@oracle.com> Message-ID: <5915D8E6.1090302@oracle.com> Thanks Coleen. I also started running tier4 and tier5 tests just to make sure I don't screw things up ... - Ioi On 5/12/17 5:07 AM, coleen.phillimore at oracle.com wrote: > Looks good. > Coleen > > On 5/12/17 1:21 AM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8179769 >> http://cr.openjdk.java.net/~iklam/jdk10/8179769-sa-cp-failures.v01/ >> >> Summary: >> >> In JDK-8171392 ( Move Klass pointers outside of ConstantPool entries >> so ConstantPool can be read-only), I changed how classes are stored >> in ConstantPool. However, I missed the changes that are required in >> Serviceability Agent (SA). >> >> In this patch, the SA java classes are changed to match the native >> code in constantPool.cpp. >> >> Testing: >> >> I tested with "rbt -j hs-tier2.js,hs-tier3.js" to run all the SA >> JTREG tests on all supported platforms. I also tested locally and >> verified in the JTR files that the bug has been fixed >> >> Thanks >> - Ioi > From frederic.parain at oracle.com Fri May 12 15:47:33 2017 From: frederic.parain at oracle.com (Frederic Parain) Date: Fri, 12 May 2017 11:47:33 -0400 Subject: RFR 8179903: Clean up SPARC 32-bit support In-Reply-To: References: Message-ID: <7F106736-E4B7-4683-B396-9703559C0790@oracle.com> Looks good to me. Thank you for doing this clean up. Fred > On May 12, 2017, at 10:34, George Triantafillou wrote: > > Please review this fix to clean up SPARC 32-bit support. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8179903 > webrev: http://cr.openjdk.java.net/~gtriantafill/8179903-webrev/webrev/index.html > > This is a followup RFE to "JDK-8150388 Remove SPARC 32-bit support". The work includes addressing formatting, 32-bit comments, and other issues that Kim raised after JDK-8150388 was reviewed and checked in. > > Built and tested on solaris-sparcv9-debug, solaris-x64-debug with the nsk.jvmti, nsk.jdwp, and nsk.jdi testlists. > > Thanks. > > -George > From gerald.thornbrugh at oracle.com Fri May 12 16:10:27 2017 From: gerald.thornbrugh at oracle.com (Gerald Thornbrugh) Date: Fri, 12 May 2017 10:10:27 -0600 Subject: RFR 8179903: Clean up SPARC 32-bit support In-Reply-To: References: Message-ID: <3A83A992-62FF-4575-9906-91DEB4154113@oracle.com> Hi George, Your changes look good to me. Jerry > On May 12, 2017, at 8:34 AM, George Triantafillou wrote: > > Please review this fix to clean up SPARC 32-bit support. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8179903 > webrev: http://cr.openjdk.java.net/~gtriantafill/8179903-webrev/webrev/index.html > > This is a followup RFE to "JDK-8150388 Remove SPARC 32-bit support". The work includes addressing formatting, 32-bit comments, and other issues that Kim raised after JDK-8150388 was reviewed and checked in. > > Built and tested on solaris-sparcv9-debug, solaris-x64-debug with the nsk.jvmti, nsk.jdwp, and nsk.jdi testlists. > > Thanks. > > -George > From george.triantafillou at oracle.com Fri May 12 16:29:57 2017 From: george.triantafillou at oracle.com (George Triantafillou) Date: Fri, 12 May 2017 12:29:57 -0400 Subject: RFR 8179903: Clean up SPARC 32-bit support In-Reply-To: <7F106736-E4B7-4683-B396-9703559C0790@oracle.com> References: <7F106736-E4B7-4683-B396-9703559C0790@oracle.com> Message-ID: <437c965d-176d-cde8-d175-a403d914b49b@oracle.com> Thanks Fred! -George On 5/12/2017 11:47 AM, Frederic Parain wrote: > Looks good to me. > Thank you for doing this clean up. > > Fred > > >> On May 12, 2017, at 10:34, George Triantafillou wrote: >> >> Please review this fix to clean up SPARC 32-bit support. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8179903 >> webrev: http://cr.openjdk.java.net/~gtriantafill/8179903-webrev/webrev/index.html >> >> This is a followup RFE to "JDK-8150388 Remove SPARC 32-bit support". The work includes addressing formatting, 32-bit comments, and other issues that Kim raised after JDK-8150388 was reviewed and checked in. >> >> Built and tested on solaris-sparcv9-debug, solaris-x64-debug with the nsk.jvmti, nsk.jdwp, and nsk.jdi testlists. >> >> Thanks. >> >> -George >> From george.triantafillou at oracle.com Fri May 12 16:30:18 2017 From: george.triantafillou at oracle.com (George Triantafillou) Date: Fri, 12 May 2017 12:30:18 -0400 Subject: RFR 8179903: Clean up SPARC 32-bit support In-Reply-To: <3A83A992-62FF-4575-9906-91DEB4154113@oracle.com> References: <3A83A992-62FF-4575-9906-91DEB4154113@oracle.com> Message-ID: <6f6b82c0-d159-c459-56d4-5fbfbeb84004@oracle.com> Thanks Jerry! -George On 5/12/2017 12:10 PM, Gerald Thornbrugh wrote: > Hi George, > > Your changes look good to me. > > Jerry >> On May 12, 2017, at 8:34 AM, George Triantafillou wrote: >> >> Please review this fix to clean up SPARC 32-bit support. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8179903 >> webrev: http://cr.openjdk.java.net/~gtriantafill/8179903-webrev/webrev/index.html >> >> This is a followup RFE to "JDK-8150388 Remove SPARC 32-bit support". The work includes addressing formatting, 32-bit comments, and other issues that Kim raised after JDK-8150388 was reviewed and checked in. >> >> Built and tested on solaris-sparcv9-debug, solaris-x64-debug with the nsk.jvmti, nsk.jdwp, and nsk.jdi testlists. >> >> Thanks. >> >> -George >> From chenyt at cs.sjtu.edu.cn Fri May 12 22:29:49 2017 From: chenyt at cs.sjtu.edu.cn (=?gb2312?B?s8LT6s2k?=) Date: Fri, 12 May 2017 15:29:49 -0700 Subject: Should JVM know an app is in a shutdown state when a timeout event occurs? Message-ID: <007501d2cb6f$45857bf0$d09073d0$@cs.sjtu.edu.cn> I recently observed that there is a minor difference between HotSpot and IBM's J9 when they process the timeout events. I downloaded the dacapo benchmark (http://dacapobench.org/) and ran "timeout 20s java Harness avrora" . J9 sometimes throws a (java.lang.IllegalStateException: Shutdown in progress) while HotSpot cannot. The IllegalStateException may be thrown by J9 when Runtime.getRuntime().removeShutdownHook(shutdownThread) (see line 103, avrora.actions.SimAction) is executed and at the same time the timeout event arrives. In order to reproduce the event, I slightly rewrote avrora.Main.main a little so that avrora.Main.runAction will be called for 30 times. I also ran the command "timeout 20s java Harness avrora" for 30+ times so that Runtime.getRuntime().removeShutdownHook(shutdownThread) (the code at line 103, avrora.actions.SimAction) is running when "timeout 20s java" completes. It seems that when the timeout event occurs, the app is in a shutdown state. J9 is able to detect it, while HotSpot may miss catching it. I wonder what can be the real cause of the difference, the JVMs, or the API implementations (i.e., Runtime. getRuntime().addShutdownHook), or the application itself. Which JVM is correct? Actually when the timeout event occurs, HotSpot will terminate without executing the following statements, while J9 throws the exception and enters the exception handling block. avrora.actions.SimAction 84 public void run(String[] args) throws Exception { 85 SimUtil.REPORT_SECONDS = REPORT_SECONDS.get(); 86 SimUtil.SECONDS_PRECISION = (int)SECONDS_PRECISION.get(); 87 88 simulation = Defaults.getSimulation(SIMULATION.get()); 89 simulation.process(options, args); 90 91 ShutdownThread shutdownThread = new ShutdownThread(); 92 Runtime.getRuntime().addShutdownHook(shutdownThread); 93 printSimHeader(); 94 try { 95 startms = System.currentTimeMillis(); 96 simulation.start(); 97 simulation.join(); 98 } catch (Throwable t) { 99 exitSimulation(t); 100 Runtime.getRuntime().removeShutdownHook(shutdownThread); 101 } finally { 102 exitSimulation(null); 103 Runtime.getRuntime().removeShutdownHook(shutdownThread); 104 } 105 } The exception message is given as the follows: java.lang.reflect.InvocationTargetException java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95 ) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl .java:55) at java.lang.reflect.Method.invoke(Method.java:508) at org.dacapo.harness.Avrora.iterate(Avrora.java:42) at org.dacapo.harness.Benchmark.run(Benchmark.java:166) at org.dacapo.harness.TestHarness.runBenchmark(TestHarness.java:218) at org.dacapo.harness.TestHarness.main(TestHarness.java:171) at Harness.main(Harness.java:17) Caused by: java.lang.IllegalStateException: Shutdown in progress at java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:93) at java.lang.Runtime.removeShutdownHook(Runtime.java:250) at avrora.actions.SimAction.run(SimAction.java:103) at avrora.Main.runAction(Unknown Source) at avrora.Main.main(Unknown Source) ... 9 more From coleen.phillimore at oracle.com Fri May 12 22:46:55 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 12 May 2017 18:46:55 -0400 Subject: RFR 8180325: Use ClassLoaderData::classes_do for CDS classes Message-ID: <8b59e7b3-2832-98fd-e0a3-8ece93e65023@oracle.com> Summary: Use closures and ClassLoaderData::classes_do instead of SystemDictionary::classes_do Migrate CDS code to ClassLoaderData::loaded_classes_do. The loaded_classes_do function should be called without a lock, like the rest of the ClassLoaderData::classes_do functions. Also, I didn't filter anonymous classes with the CDS functions because at the moment, we don't encounter them (there's an assert). If the code changes so that we do encounter them, we might want to link them or decide what to do with them at that time. open webrev at http://cr.openjdk.java.net/~coleenp/8180325.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8180325 Tested with nightly rbt run on linux x86, including and additionally all the CDS tests locally. Thanks, Coleen From coleen.phillimore at oracle.com Fri May 12 23:47:09 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 12 May 2017 19:47:09 -0400 Subject: [9, P1] RFR (S/M): 8180048: Interned string and symbol table leak memory during parallel unlinking In-Reply-To: <1494586002.2459.13.camel@oracle.com> References: <1494518643.3120.18.camel@oracle.com> <1494586002.2459.13.camel@oracle.com> Message-ID: <1a7196cf-087e-9e8e-902a-bcffaf5dc9bf@oracle.com> Thomas, This looks good. It's a bit hard to understand quickly. A couple of comments would help. http://cr.openjdk.java.net/~tschatzl/8180048/webrev.1/src/share/vm/utilities/hashtable.cpp.udiff.html Add a comment to bulk_free_entries before the cas like: // Add thread local list of removed entries in removed_head to the head of the table's _free_list thread safely. It would help that _removed_head and _removed_tail (and the other fields) have leading underscores as is the convention of nonstatic data members, so it's easy to see these are the fields of BucketUnlinkContext. I think accessor functions are unnecessary though. http://cr.openjdk.java.net/~tschatzl/8180048/webrev.1/src/share/vm/classfile/stringTable.cpp.udiff.html http://cr.openjdk.java.net/~tschatzl/8180048/webrev.1/src/share/vm/classfile/symbolTable.cpp.udiff.html Say for BucketUnlinkContext a short reminder why we need this here. // unlink_or_oops do is called by multiple threads so needs to hold deleted elements in a thread local context to bulk delete (or something like this) If you change this, and add a some comment, I don't require a new webrev. This looks like a good change. Thanks, Coleen On 5/12/17 6:46 AM, Thomas Schatzl wrote: > Hi again, > > On Thu, 2017-05-11 at 18:04 +0200, Thomas Schatzl wrote: >> Hi all, >> >> can I have reviews for this change that fixes a memory leak when >> unlinking (removing) elements from the string and symbol tables in >> parallel? >> >> [...] >> >> I am also going to try to get this change into 8u. >> >> Ideally I would also like to have a reviewer from the runtime team >> (cc'ed). >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8180048 >> Webrev: >> http://cr.openjdk.java.net/~tschatzl/8180048/webrev/ >> Testing: >> jprt, manual testing > I got some comments from Erik Helin, with the following webrev as > result: > > http://cr.openjdk.java.net/~tschatzl/8180048/webrev.0_to_1/ (diff) > http://cr.openjdk.java.net/~tschatzl/8180048/webrev.1/ (full) > > Testing: > jprt > > Thanks, > Thomas > From rkennke at redhat.com Mon May 15 14:52:46 2017 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 15 May 2017 16:52:46 +0200 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> Message-ID: Am 11.05.2017 um 09:44 schrieb Robbin Ehn: > Hi, > > We have actually been discussing this last few days: > > https://bugs.openjdk.java.net/browse/JDK-8153224 We are seeing the exact same issue as mentioned in the bug report too. Sometimes, time-to-safepoint is very high, in the area of ~200ms. The reason for this seems to be deflate_idle_monitors(). In the current implementation, the VM thread iterates over all threads, and deflates each thread's idle monitors one after another. I would rather have each Java thread deflate its own idle monitors before arriving at a safepoint. This should be much faster because higher chances a thread has its own monitors in cache, and deflation processing is done in parallel too. This requires some thought... e.g. what to do with non-runnable threads. If you think that's a good way to go, and don't already have a fix in the pipeline, I can take on the bug. I've got synchronizer still hot in memory, and experimented with self-processing Java threads before safepoint a while back too. Let me know! Roman From david.holmes at oracle.com Tue May 16 04:49:46 2017 From: david.holmes at oracle.com (David Holmes) Date: Tue, 16 May 2017 14:49:46 +1000 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> Message-ID: On 16/05/2017 12:52 AM, Roman Kennke wrote: > Am 11.05.2017 um 09:44 schrieb Robbin Ehn: >> Hi, >> >> We have actually been discussing this last few days: >> >> https://bugs.openjdk.java.net/browse/JDK-8153224 > > We are seeing the exact same issue as mentioned in the bug report too. > Sometimes, time-to-safepoint is very high, in the area of ~200ms. The > reason for this seems to be deflate_idle_monitors(). > > In the current implementation, the VM thread iterates over all threads, > and deflates each thread's idle monitors one after another. > > I would rather have each Java thread deflate its own idle monitors > before arriving at a safepoint. This should be much faster because > higher chances a thread has its own monitors in cache, and deflation > processing is done in parallel too. This requires some thought... e.g. > what to do with non-runnable threads. This is risky and will need very careful analysis. The type-stable-memory property of Monitors depends on very careful synchronization and the fact that deflation only happens at safepoints. David ----- > If you think that's a good way to go, and don't already have a fix in > the pipeline, I can take on the bug. I've got synchronizer still hot in > memory, and experimented with self-processing Java threads before > safepoint a while back too. > > Let me know! > > Roman > > From david.holmes at oracle.com Mon May 15 05:16:23 2017 From: david.holmes at oracle.com (David Holmes) Date: Mon, 15 May 2017 15:16:23 +1000 Subject: =?UTF-8?Q?Re:_=e7=ad=94=e5=a4=8d:_Should_JVM_know_an_app_is_in_a_sh?= =?UTF-8?Q?utdown_state_when_a_timeout_event_occurs=3f?= In-Reply-To: References: <007501d2cb6f$45857bf0$d09073d0$@cs.sjtu.edu.cn> <70ad5403-8094-6349-f636-57b04b3e18a7@oracle.com> <000e01d2cc13$e6a2a770$b3e7f650$@cs.sjtu.edu.cn> Message-ID: <726f9b11-6423-6f12-6aad-477cca8aebed@oracle.com> Re-sending to hotspot-runtime-dev at openjdk.java.net On 15/05/2017 7:31 AM, David Holmes wrote: > The specification for add/removeShutdownHook states: > > * @throws IllegalStateException > * If the virtual machine is already in the process of > shutting > * down > > So if the VM is shutting down and calls to those functions are made then > they will throw the IllegalStateException. > > SIGTERM is one of the shutdown signals that the hotspot VM will respond > to.Or more accurately it is communicated through to the signal handling > thread which will create a new Java thread to perform an orderly > shutdown of the JVM. > > The exact way in which signals are processed is not specified so may > vary across different JVMs. > > If I understand the stack trace you supplied it seems that decapo > installed a shutdownhook thread and then as part of its normal > termination process it will remove that hook. If you are terminating the > JVM externally using "timeout" then you have introduced a race > condition. If the shutdown has commenced when the hook is removed then > the exception will be thrown; otherwise it won't. > > I can't comment on any specifics of the J9 VM. > > HTH. > > David > > On 14/05/2017 4:08 AM, ??? wrote: >> I quickly went through the "Troubleshooting guide for HotSpot VM" ( >> http://docs.oracle.com/javase/7/docs/webnotes/tsg/TSG-VM/html/signals.html#g >> >> bzbl). >> >> In my case, the -Xrs option is not specified. >> >> The guide does not explain very clearly about my test case: when the JVM >> itself is in >> a shutting down progress, should the IllegalStateException be thrown out >> when >> it attempts to remove the shutdown hook? >> >> Regards, >> Yuting >> >> -----????----- >> ???: ??? [mailto:chenyt at cs.sjtu.edu.cn] >> ????: 2017?5?13? 10:31 >> ???: 'David Holmes' ; >> 'hotspot-runtime-dev at openjdk.java.net' >> >> ??: ??: Should JVM know an app is in a shutdown state when a timeout >> event occurs? >> >> Besides, the TERM signal kills any process that does not block or >> catch that >> signal (https://linux.die.net/man/1/timeout). Does the test case indicate >> that J9 can catch the TERM signal, while HotSpot cannot? :) >> >> Yuting >> >> -----????----- >> ???: ??? [mailto:chenyt at cs.sjtu.edu.cn] >> ????: 2017?5?13? 10:23 >> ???: 'David Holmes' ; >> 'hotspot-runtime-dev at openjdk.java.net' >> >> ??: ??: Should JVM know an app is in a shutdown state when a timeout >> event occurs? >> >> Hi?David? >> >> I tested and compared JVMs on Ubuntu 16.04 using >> timeout 20s java ... >> The command kills the jvm if it is still running after 20seconds (i.e., >> after 20 seconds, the TERM signal is sent and then the jvm will be in the >> process of shutting down). >> >> Runtime. getRuntime().removeShutdownHook() may throw >> IllegalStateException >> if the virtual machine is already in the process of shutting down. >> >> Indeed, IBM's J9 can throw the IllegalStateException (when it removes the >> hook and the JVM is shutting down), while HotSpot cannot. >> I'm curious about the difference (I rethought about this. Possibly >> J9 should not throw out the exception when it is shutting down.). >> >> Yuting >> >> >> -----????----- >> ???: David Holmes [mailto:david.holmes at oracle.com] >> ????: 2017?5?13? 1:02 >> ???: ??? ; >> hotspot-runtime-dev at openjdk.java.net >> ??: Re: Should JVM know an app is in a shutdown state when a timeout >> event >> occurs? >> >> Hi, >> >> What "timeout events" are you referring to? >> >> What you show below indicates an attempt to remove a shutdown hook >> after the >> shutdown sequence has started - that is all in the JDK library code, >> nothing >> to do with hotspot. >> >> Cheers, >> David >> >> On 13/05/2017 8:29 AM, ??? wrote: >>> I recently observed that there is a minor difference between HotSpot >>> and IBM's J9 when they process the timeout events. I downloaded the >>> dacapo benchmark >>> (http://dacapobench.org/) and ran >>> "timeout 20s java Harness avrora" >>> . J9 sometimes throws a (java.lang.IllegalStateException: Shutdown in >>> progress) while HotSpot cannot. >>> The IllegalStateException may be thrown by J9 when >>> Runtime.getRuntime().removeShutdownHook(shutdownThread) >>> (see line 103, avrora.actions.SimAction) is executed and at the same >>> time the timeout event arrives. >>> >>> In order to reproduce the event, I slightly rewrote avrora.Main.main a >>> little so that avrora.Main.runAction will be called for 30 times. I >>> also ran the command "timeout 20s java Harness avrora" for 30+ times >>> so that Runtime.getRuntime().removeShutdownHook(shutdownThread) (the >>> code at line 103, avrora.actions.SimAction) is running when "timeout >>> 20s java" completes. >>> >>> It seems that when the timeout event occurs, the app is in a shutdown >> state. >>> J9 is able to detect it, while >>> HotSpot may miss catching it. >>> >>> I wonder what can be the real cause of the difference, the JVMs, or >>> the API implementations (i.e., Runtime. >>> getRuntime().addShutdownHook), or the application itself. Which JVM is >>> correct? Actually when the timeout event occurs, HotSpot will >>> terminate without executing the following statements, while J9 throws >>> the exception and enters the exception handling block. >>> >>> avrora.actions.SimAction >>> 84 public void run(String[] args) throws Exception { >>> 85 SimUtil.REPORT_SECONDS = REPORT_SECONDS.get(); >>> 86 SimUtil.SECONDS_PRECISION = (int)SECONDS_PRECISION.get(); >>> 87 >>> 88 simulation = Defaults.getSimulation(SIMULATION.get()); >>> 89 simulation.process(options, args); >>> 90 >>> 91 ShutdownThread shutdownThread = new ShutdownThread(); >>> 92 Runtime.getRuntime().addShutdownHook(shutdownThread); >>> 93 printSimHeader(); >>> 94 try { >>> 95 startms = System.currentTimeMillis(); >>> 96 simulation.start(); >>> 97 simulation.join(); >>> 98 } catch (Throwable t) { >>> 99 exitSimulation(t); >>> 100 >>> Runtime.getRuntime().removeShutdownHook(shutdownThread); >>> 101 } finally { >>> 102 exitSimulation(null); >>> 103 >>> Runtime.getRuntime().removeShutdownHook(shutdownThread); >>> 104 } >>> 105 } >>> >>> The exception message is given as the follows: >>> >>> java.lang.reflect.InvocationTargetException >>> java.lang.reflect.InvocationTargetException >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j >>> ava:95 >>> ) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccess >>> orImpl >>> .java:55) >>> at java.lang.reflect.Method.invoke(Method.java:508) >>> at org.dacapo.harness.Avrora.iterate(Avrora.java:42) >>> at org.dacapo.harness.Benchmark.run(Benchmark.java:166) >>> at org.dacapo.harness.TestHarness.runBenchmark(TestHarness.java:218) >>> at org.dacapo.harness.TestHarness.main(TestHarness.java:171) >>> at Harness.main(Harness.java:17) >>> Caused by: java.lang.IllegalStateException: Shutdown in progress >>> at >>> >> java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:93) >> >>> at java.lang.Runtime.removeShutdownHook(Runtime.java:250) >>> at avrora.actions.SimAction.run(SimAction.java:103) >>> at avrora.Main.runAction(Unknown Source) >>> at avrora.Main.main(Unknown Source) >>> ... 9 more >>> >> From goetz.lindenmaier at sap.com Mon May 15 10:31:05 2017 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 15 May 2017 10:31:05 +0000 Subject: RFR(S): 8179953: [ppc] TLABWasteIncrement not loaded correctly In-Reply-To: References: Message-ID: <00a572990741428e9b1935f77c536f07@sap.com> Hi David, yes, what you are saying is correct, and I share your concerns about the design of JVMOptionsUtils. Actually I think a test is considered as passed if the VM issues the message that gc flags arecontradictory. Thus, if you run with GCType=ParallelGC those flags that set G1GC explicitly are not properly tested. This is what first happened with my fix as we test with default settings which is G1. And here I didn't consider that I might change test behavior for the other flags. But if this raises new errors, they should be addressed anyways. Best regards, Goetz. > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Montag, 15. Mai 2017 06:25 > To: Lindenmaier, Goetz ; hotspot-runtime- > dev at openjdk.java.net > Subject: Re: RFR(S): 8179953: [ppc] TLABWasteIncrement not loaded correctly > > Hi Goetz, > > On 12/05/2017 4:45 PM, Lindenmaier, Goetz wrote: > > Hi David, > > > > the broken code is protected by > > if (allow_shared_alloc) { > > which evaluates to true with ParallelGC but not with G1GC. > > It might be true with other GCs, too, but the design of > > JVMOptionsUtils:addNameDependency() is quite simple, > > and I just did it as for other flags. > > I could put in more logic into that case there, like > > if (GCType == G1GC || GCTyep == ... ) { > > option.addPrepend("-XX:+UseParallelGC"); > > } > > enumerating all GCs that don't support TLABWastIncrement. > > But I thought one test case for the flag is fine. The way it is > > we missed the error, so I would like to fix the test. > > So if I understand correctly you just want to test use of > TLABWasteIncrement, and you know ParallelGC uses it, so you added > -XX:+UseParallelGC to the test options. Okay I get that part. Yes. > However the more I look at JVMOptionsUtils.java the less I understand > how it expects things to work. I don't find addNameDependency simple, > but confused! It looks for things like G1* and so adds UseG1GC, but > completely ignores that GCType may already be set - but that works > because AFAICS when it finds a G1* option in the existing set of > options, it has to be that G1GC is already the current GC type - > otherwise the test would fail - no? > > Looking at the history of this code it never set the GC so would use the > default, unless addNameDependency explicitly set one. That worked fine. > Then 8144578 took issue with the fact the test only ran with the default > GC and so introduced GCType to run the test using the same GC as was > used to launch the main test. But AFAICS that completely overlooked the > fact that addNameDependency might set an explicit GC! Mea culpa as I was > one of the reviewers of that change and it was obviously incomplete! :( > It also seem there are mails missing from the archives as you find now > mail from me here: > > http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-January/021390.html > > BTW in that review it refers to TLABWasteIncrement as a "non-dedicated > gc flag". That suggests to me that testing of this flag was expected to > be GC agnostic. ?? > > Anyway the question is, what is the right/best way to reconcile the use > of GCType with explicit setting of GC selection flags? Your fix is one > way. Another option would be to clear GCType when requiring a specifc > GC. Not sure either way is significantly better - pros and cons to both. > > My concern is whether this change may have an unexpected side-effects > given that I don't see how it can have been working correctly in the > first place. > > Unfortunately the primary author of that code is no longer around. > > David > > > > Yes, in JVMOption I must exclude setting G1GC else I get > > an error claiming that two different GCs are set. > > > > Best regards, > > Goetz. > > > > > > > > > >> -----Original Message----- > >> From: David Holmes [mailto:david.holmes at oracle.com] > >> Sent: Wednesday, May 10, 2017 5:19 AM > >> To: Lindenmaier, Goetz ; hotspot-runtime- > >> dev at openjdk.java.net > >> Subject: Re: RFR(S): 8179953: [ppc] TLABWasteIncrement not loaded > >> correctly > >> > >> Hi Goetz, > >> > >> On 10/05/2017 12:56 AM, Lindenmaier, Goetz wrote: > >>> Hi, > >>> > >>> Please review this change. As I fix the test, I please need a sponsor. > >>> http://cr.openjdk.java.net/~goetz/wr17/8179953-ppc_flag/webrev.01/ > >> > >> I'm unclear on the test changes. It isn't obvious to me that > >> TLABWasteIncrement is a ParallelGC only flag. By adding that as an > >> additional flag you then had to exclude adding the GCType (if any GC > >> selector is given) to avoid conflicting options - is that right? > >> > >> Thanks, > >> David > >> > >>> On ppc, the mentioned flag is loaded as a 16 bit immediate, while it can > >> have > >>> bigger values. > >>> I also adapt TestOptionWithRanges, which did not show the bug because > >>> the flag has no effect with G1. > >>> > >>> Best regards, > >>> Goetz. > >>> > >>> From coleen.phillimore at oracle.com Mon May 15 12:28:46 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 15 May 2017 08:28:46 -0400 Subject: [9, P1] RFR (S/M): 8180048: Interned string and symbol table leak memory during parallel unlinking In-Reply-To: <1494843751.2707.7.camel@oracle.com> References: <1494518643.3120.18.camel@oracle.com> <1494586002.2459.13.camel@oracle.com> <1a7196cf-087e-9e8e-902a-bcffaf5dc9bf@oracle.com> <1494843751.2707.7.camel@oracle.com> Message-ID: <9d096824-17c9-7d86-82e0-288bd5f7c4f5@oracle.com> Yes, I like your additional comments. Thanks, Coleen On 5/15/17 6:22 AM, Thomas Schatzl wrote: > Hi Coleen, > > thanks a lot for your review. :) > > On Fri, 2017-05-12 at 19:47 -0400, coleen.phillimore at oracle.com wrote: >> Thomas, >> >> This looks good. It's a bit hard to understand quickly. A couple >> of comments would help. >> >> http://cr.openjdk.java.net/~tschatzl/8180048/webrev.1/src/share/vm/ut >> ilities/hashtable.cpp.udiff.html >> >> Add a comment to bulk_free_entries before the cas like: >> >> // Add thread local list of removed entries in removed_head to the >> head of the table's _free_list thread safely. > I tried to improve the comments in the changed areas to better indicate > why the change uses the BucketUnlinkContext. > >> It would help that _removed_head and _removed_tail (and the other >> fields) have leading underscores as is the convention of nonstatic >> data members, so it's easy to see these are the fields of >> BucketUnlinkContext. I think accessor functions are unnecessary >> though. > Fixed. > >> http://cr.openjdk.java.net/~tschatzl/8180048/webrev.1/src/share/vm/cl >> assfile/stringTable.cpp.udiff.html >> http://cr.openjdk.java.net/~tschatzl/8180048/webrev.1/src/share/vm/cl >> assfile/symbolTable.cpp.udiff.html >> >> Say for BucketUnlinkContext a short reminder why we need this here. >> >> // unlink_or_oops do is called by multiple threads so needs to hold >> deleted elements in a thread local context to bulk delete >> >> (or something like this) >> >> If you change this, and add a some comment, I don't require a new >> webrev. This looks like a good change. > For reference: > http://cr.openjdk.java.net/~tschatzl/8180048/webrev.1_to_2/ (diff) > http://cr.openjdk.java.net/~tschatzl/8180048/webrev.2/ (full) > > Thanks, > Thomas > From ioi.lam at oracle.com Mon May 15 15:51:52 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 15 May 2017 08:51:52 -0700 Subject: RFR 8180325: Use ClassLoaderData::classes_do for CDS classes In-Reply-To: <8b59e7b3-2832-98fd-e0a3-8ece93e65023@oracle.com> References: <8b59e7b3-2832-98fd-e0a3-8ece93e65023@oracle.com> Message-ID: <5919CE98.8010605@oracle.com> HI Coleen, The changes look good. FYI 1998 void InstanceKlass::remove_unshareable_info() { 1999 Klass::remove_unshareable_info(); 2000 2001 if (is_in_error_state()) { 2002 // Classes are attempted to link during dumping and may fail, 2003 // but these classes are still in the dictionary and class list in CLD. 2004 // Check in_error state first because in_error is > linked state, so 2005 // is_linked() is true. 2006 // If there's a linking error, there is nothing else to remove. 2007 return; 2008 } 2009 The above code will be changed to an assert with JDK-8072061 (Automatically determine optimal sizes for the CDS regions), because JDK-8072061 will eliminate all classes in error state. Thanks - Ioi On 5/12/17 3:46 PM, coleen.phillimore at oracle.com wrote: > Summary: Use closures and ClassLoaderData::classes_do instead of > SystemDictionary::classes_do > > Migrate CDS code to ClassLoaderData::loaded_classes_do. The > loaded_classes_do function should be called without a lock, like the > rest of the ClassLoaderData::classes_do functions. Also, I didn't > filter anonymous classes with the CDS functions because at the moment, > we don't encounter them (there's an assert). If the code changes so > that we do encounter them, we might want to link them or decide what > to do with them at that time. > > open webrev at http://cr.openjdk.java.net/~coleenp/8180325.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8180325 > > Tested with nightly rbt run on linux x86, including and additionally > all the CDS tests locally. > > Thanks, > Coleen > From coleen.phillimore at oracle.com Mon May 15 12:30:19 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 15 May 2017 08:30:19 -0400 Subject: RFR 8166848: Performance bug: SystemDictionary - optimization In-Reply-To: <249af7dc-a3a4-52b4-ca6d-ffdb7ec13764@oracle.com> References: <249af7dc-a3a4-52b4-ca6d-ffdb7ec13764@oracle.com> Message-ID: <1cc33d02-6fca-7f67-34ff-0fcc8c309cad@oracle.com> > Summary: Check instead that a bucket isn't 10x the average > > See bug and linked bugs for more details. > > Tested with RBT nightly tests (tier2-5). > > open webrev at http://cr.openjdk.java.net/~coleenp/8166848.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8166848 > > Thanks, > Coleen From david.holmes at oracle.com Mon May 15 05:12:53 2017 From: david.holmes at oracle.com (David Holmes) Date: Mon, 15 May 2017 15:12:53 +1000 Subject: =?UTF-8?Q?Re:_=e7=ad=94=e5=a4=8d:_Should_JVM_know_an_app_is_in_a_sh?= =?UTF-8?Q?utdown_state_when_a_timeout_event_occurs=3f?= In-Reply-To: References: <007501d2cb6f$45857bf0$d09073d0$@cs.sjtu.edu.cn> <70ad5403-8094-6349-f636-57b04b3e18a7@oracle.com> <000e01d2cc13$e6a2a770$b3e7f650$@cs.sjtu.edu.cn> Message-ID: <30633820-edbd-c6fc-9983-390f48cb946e@oracle.com> Re-sending to hotspot-runtime-dev at openjdk.java.net On 15/05/2017 7:31 AM, David Holmes wrote: > The specification for add/removeShutdownHook states: > > * @throws IllegalStateException > * If the virtual machine is already in the process of > shutting > * down > > So if the VM is shutting down and calls to those functions are made then > they will throw the IllegalStateException. > > SIGTERM is one of the shutdown signals that the hotspot VM will respond > to.Or more accurately it is communicated through to the signal handling > thread which will create a new Java thread to perform an orderly > shutdown of the JVM. > > The exact way in which signals are processed is not specified so may > vary across different JVMs. > > If I understand the stack trace you supplied it seems that decapo > installed a shutdownhook thread and then as part of its normal > termination process it will remove that hook. If you are terminating the > JVM externally using "timeout" then you have introduced a race > condition. If the shutdown has commenced when the hook is removed then > the exception will be thrown; otherwise it won't. > > I can't comment on any specifics of the J9 VM. > > HTH. > > David > > On 14/05/2017 4:08 AM, ??? wrote: >> I quickly went through the "Troubleshooting guide for HotSpot VM" ( >> http://docs.oracle.com/javase/7/docs/webnotes/tsg/TSG-VM/html/signals.html#g >> >> bzbl). >> >> In my case, the -Xrs option is not specified. >> >> The guide does not explain very clearly about my test case: when the JVM >> itself is in >> a shutting down progress, should the IllegalStateException be thrown out >> when >> it attempts to remove the shutdown hook? >> >> Regards, >> Yuting >> >> -----????----- >> ???: ??? [mailto:chenyt at cs.sjtu.edu.cn] >> ????: 2017?5?13? 10:31 >> ???: 'David Holmes' ; >> 'hotspot-runtime-dev at openjdk.java.net' >> >> ??: ??: Should JVM know an app is in a shutdown state when a timeout >> event occurs? >> >> Besides, the TERM signal kills any process that does not block or >> catch that >> signal (https://linux.die.net/man/1/timeout). Does the test case indicate >> that J9 can catch the TERM signal, while HotSpot cannot? :) >> >> Yuting >> >> -----????----- >> ???: ??? [mailto:chenyt at cs.sjtu.edu.cn] >> ????: 2017?5?13? 10:23 >> ???: 'David Holmes' ; >> 'hotspot-runtime-dev at openjdk.java.net' >> >> ??: ??: Should JVM know an app is in a shutdown state when a timeout >> event occurs? >> >> Hi?David? >> >> I tested and compared JVMs on Ubuntu 16.04 using >> timeout 20s java ... >> The command kills the jvm if it is still running after 20seconds (i.e., >> after 20 seconds, the TERM signal is sent and then the jvm will be in the >> process of shutting down). >> >> Runtime. getRuntime().removeShutdownHook() may throw >> IllegalStateException >> if the virtual machine is already in the process of shutting down. >> >> Indeed, IBM's J9 can throw the IllegalStateException (when it removes the >> hook and the JVM is shutting down), while HotSpot cannot. >> I'm curious about the difference (I rethought about this. Possibly >> J9 should not throw out the exception when it is shutting down.). >> >> Yuting >> >> >> -----????----- >> ???: David Holmes [mailto:david.holmes at oracle.com] >> ????: 2017?5?13? 1:02 >> ???: ??? ; >> hotspot-runtime-dev at openjdk.java.net >> ??: Re: Should JVM know an app is in a shutdown state when a timeout >> event >> occurs? >> >> Hi, >> >> What "timeout events" are you referring to? >> >> What you show below indicates an attempt to remove a shutdown hook >> after the >> shutdown sequence has started - that is all in the JDK library code, >> nothing >> to do with hotspot. >> >> Cheers, >> David >> >> On 13/05/2017 8:29 AM, ??? wrote: >>> I recently observed that there is a minor difference between HotSpot >>> and IBM's J9 when they process the timeout events. I downloaded the >>> dacapo benchmark >>> (http://dacapobench.org/) and ran >>> "timeout 20s java Harness avrora" >>> . J9 sometimes throws a (java.lang.IllegalStateException: Shutdown in >>> progress) while HotSpot cannot. >>> The IllegalStateException may be thrown by J9 when >>> Runtime.getRuntime().removeShutdownHook(shutdownThread) >>> (see line 103, avrora.actions.SimAction) is executed and at the same >>> time the timeout event arrives. >>> >>> In order to reproduce the event, I slightly rewrote avrora.Main.main a >>> little so that avrora.Main.runAction will be called for 30 times. I >>> also ran the command "timeout 20s java Harness avrora" for 30+ times >>> so that Runtime.getRuntime().removeShutdownHook(shutdownThread) (the >>> code at line 103, avrora.actions.SimAction) is running when "timeout >>> 20s java" completes. >>> >>> It seems that when the timeout event occurs, the app is in a shutdown >> state. >>> J9 is able to detect it, while >>> HotSpot may miss catching it. >>> >>> I wonder what can be the real cause of the difference, the JVMs, or >>> the API implementations (i.e., Runtime. >>> getRuntime().addShutdownHook), or the application itself. Which JVM is >>> correct? Actually when the timeout event occurs, HotSpot will >>> terminate without executing the following statements, while J9 throws >>> the exception and enters the exception handling block. >>> >>> avrora.actions.SimAction >>> 84 public void run(String[] args) throws Exception { >>> 85 SimUtil.REPORT_SECONDS = REPORT_SECONDS.get(); >>> 86 SimUtil.SECONDS_PRECISION = (int)SECONDS_PRECISION.get(); >>> 87 >>> 88 simulation = Defaults.getSimulation(SIMULATION.get()); >>> 89 simulation.process(options, args); >>> 90 >>> 91 ShutdownThread shutdownThread = new ShutdownThread(); >>> 92 Runtime.getRuntime().addShutdownHook(shutdownThread); >>> 93 printSimHeader(); >>> 94 try { >>> 95 startms = System.currentTimeMillis(); >>> 96 simulation.start(); >>> 97 simulation.join(); >>> 98 } catch (Throwable t) { >>> 99 exitSimulation(t); >>> 100 >>> Runtime.getRuntime().removeShutdownHook(shutdownThread); >>> 101 } finally { >>> 102 exitSimulation(null); >>> 103 >>> Runtime.getRuntime().removeShutdownHook(shutdownThread); >>> 104 } >>> 105 } >>> >>> The exception message is given as the follows: >>> >>> java.lang.reflect.InvocationTargetException >>> java.lang.reflect.InvocationTargetException >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j >>> ava:95 >>> ) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccess >>> orImpl >>> .java:55) >>> at java.lang.reflect.Method.invoke(Method.java:508) >>> at org.dacapo.harness.Avrora.iterate(Avrora.java:42) >>> at org.dacapo.harness.Benchmark.run(Benchmark.java:166) >>> at org.dacapo.harness.TestHarness.runBenchmark(TestHarness.java:218) >>> at org.dacapo.harness.TestHarness.main(TestHarness.java:171) >>> at Harness.main(Harness.java:17) >>> Caused by: java.lang.IllegalStateException: Shutdown in progress >>> at >>> >> java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:93) >> >>> at java.lang.Runtime.removeShutdownHook(Runtime.java:250) >>> at avrora.actions.SimAction.run(SimAction.java:103) >>> at avrora.Main.runAction(Unknown Source) >>> at avrora.Main.main(Unknown Source) >>> ... 9 more >>> >> From david.holmes at oracle.com Mon May 15 04:24:44 2017 From: david.holmes at oracle.com (David Holmes) Date: Mon, 15 May 2017 14:24:44 +1000 Subject: RFR(S): 8179953: [ppc] TLABWasteIncrement not loaded correctly In-Reply-To: References: Message-ID: Hi Goetz, On 12/05/2017 4:45 PM, Lindenmaier, Goetz wrote: > Hi David, > > the broken code is protected by > if (allow_shared_alloc) { > which evaluates to true with ParallelGC but not with G1GC. > It might be true with other GCs, too, but the design of > JVMOptionsUtils:addNameDependency() is quite simple, > and I just did it as for other flags. > I could put in more logic into that case there, like > if (GCType == G1GC || GCTyep == ... ) { > option.addPrepend("-XX:+UseParallelGC"); > } > enumerating all GCs that don't support TLABWastIncrement. > But I thought one test case for the flag is fine. The way it is > we missed the error, so I would like to fix the test. So if I understand correctly you just want to test use of TLABWasteIncrement, and you know ParallelGC uses it, so you added -XX:+UseParallelGC to the test options. Okay I get that part. However the more I look at JVMOptionsUtils.java the less I understand how it expects things to work. I don't find addNameDependency simple, but confused! It looks for things like G1* and so adds UseG1GC, but completely ignores that GCType may already be set - but that works because AFAICS when it finds a G1* option in the existing set of options, it has to be that G1GC is already the current GC type - otherwise the test would fail - no? Looking at the history of this code it never set the GC so would use the default, unless addNameDependency explicitly set one. That worked fine. Then 8144578 took issue with the fact the test only ran with the default GC and so introduced GCType to run the test using the same GC as was used to launch the main test. But AFAICS that completely overlooked the fact that addNameDependency might set an explicit GC! Mea culpa as I was one of the reviewers of that change and it was obviously incomplete! :( It also seem there are mails missing from the archives as you find now mail from me here: http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-January/021390.html BTW in that review it refers to TLABWasteIncrement as a "non-dedicated gc flag". That suggests to me that testing of this flag was expected to be GC agnostic. ?? Anyway the question is, what is the right/best way to reconcile the use of GCType with explicit setting of GC selection flags? Your fix is one way. Another option would be to clear GCType when requiring a specifc GC. Not sure either way is significantly better - pros and cons to both. My concern is whether this change may have an unexpected side-effects given that I don't see how it can have been working correctly in the first place. Unfortunately the primary author of that code is no longer around. David > Yes, in JVMOption I must exclude setting G1GC else I get > an error claiming that two different GCs are set. > > Best regards, > Goetz. > > > > >> -----Original Message----- >> From: David Holmes [mailto:david.holmes at oracle.com] >> Sent: Wednesday, May 10, 2017 5:19 AM >> To: Lindenmaier, Goetz ; hotspot-runtime- >> dev at openjdk.java.net >> Subject: Re: RFR(S): 8179953: [ppc] TLABWasteIncrement not loaded >> correctly >> >> Hi Goetz, >> >> On 10/05/2017 12:56 AM, Lindenmaier, Goetz wrote: >>> Hi, >>> >>> Please review this change. As I fix the test, I please need a sponsor. >>> http://cr.openjdk.java.net/~goetz/wr17/8179953-ppc_flag/webrev.01/ >> >> I'm unclear on the test changes. It isn't obvious to me that >> TLABWasteIncrement is a ParallelGC only flag. By adding that as an >> additional flag you then had to exclude adding the GCType (if any GC >> selector is given) to avoid conflicting options - is that right? >> >> Thanks, >> David >> >>> On ppc, the mentioned flag is loaded as a 16 bit immediate, while it can >> have >>> bigger values. >>> I also adapt TestOptionWithRanges, which did not show the bug because >>> the flag has no effect with G1. >>> >>> Best regards, >>> Goetz. >>> >>> From robbin.ehn at oracle.com Mon May 15 16:05:04 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Mon, 15 May 2017 18:05:04 +0200 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> Message-ID: Hi, On 05/15/2017 04:52 PM, Roman Kennke wrote: > Am 11.05.2017 um 09:44 schrieb Robbin Ehn: >> Hi, >> >> We have actually been discussing this last few days: >> >> https://bugs.openjdk.java.net/browse/JDK-8153224 > > We are seeing the exact same issue as mentioned in the bug report too. > Sometimes, time-to-safepoint is very high, in the area of ~200ms. The > reason for this seems to be deflate_idle_monitors(). > > In the current implementation, the VM thread iterates over all threads, > and deflates each thread's idle monitors one after another. > > I would rather have each Java thread deflate its own idle monitors > before arriving at a safepoint. This should be much faster because > higher chances a thread has its own monitors in cache, and deflation > processing is done in parallel too. This requires some thought... e.g. > what to do with non-runnable threads. Sounds like a reasonable approach. > > If you think that's a good way to go, and don't already have a fix in > the pipeline, I can take on the bug. I've got synchronizer still hot in > memory, and experimented with self-processing Java threads before > safepoint a while back too. Sure, give it go, thanks! /Robbin > > Let me know! > > Roman > > From thomas.schatzl at oracle.com Mon May 15 10:22:31 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 15 May 2017 12:22:31 +0200 Subject: [9, P1] RFR (S/M): 8180048: Interned string and symbol table leak memory during parallel unlinking In-Reply-To: <1a7196cf-087e-9e8e-902a-bcffaf5dc9bf@oracle.com> References: <1494518643.3120.18.camel@oracle.com> <1494586002.2459.13.camel@oracle.com> <1a7196cf-087e-9e8e-902a-bcffaf5dc9bf@oracle.com> Message-ID: <1494843751.2707.7.camel@oracle.com> Hi Coleen, ? thanks a lot for your review. :) On Fri, 2017-05-12 at 19:47 -0400, coleen.phillimore at oracle.com wrote: > Thomas, > > This looks good.??It's a bit hard to understand quickly.???A couple > of comments would help. > > http://cr.openjdk.java.net/~tschatzl/8180048/webrev.1/src/share/vm/ut > ilities/hashtable.cpp.udiff.html > > Add a comment to bulk_free_entries before the cas like: > > // Add thread local list of removed entries in removed_head to the > head of the table's _free_list thread safely. I tried to improve the comments in the changed areas to better indicate why the change uses the BucketUnlinkContext. > It would help that _removed_head and _removed_tail (and the other? > fields) have leading underscores as is the convention of nonstatic > data members, so it's easy to see these are the fields of? > BucketUnlinkContext. I think accessor functions are unnecessary > though. Fixed. > http://cr.openjdk.java.net/~tschatzl/8180048/webrev.1/src/share/vm/cl > assfile/stringTable.cpp.udiff.html > http://cr.openjdk.java.net/~tschatzl/8180048/webrev.1/src/share/vm/cl > assfile/symbolTable.cpp.udiff.html > > Say for BucketUnlinkContext a short reminder why we need this here. > > // unlink_or_oops do is called by multiple threads so needs to hold? > deleted elements in a thread local context to bulk delete > > (or something like this) > > If you change this, and add a some comment, I don't require a new? > webrev.??This looks like a good change. For reference: http://cr.openjdk.java.net/~tschatzl/8180048/webrev.1_to_2/?(diff) http://cr.openjdk.java.net/~tschatzl/8180048/webrev.2/?(full) Thanks, ? Thomas From rkennke at redhat.com Mon May 15 21:01:07 2017 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 15 May 2017 23:01:07 +0200 Subject: RFR (jdk10): JDK-8180175: ObjectSynchronizer only needs to iterate in-use monitors In-Reply-To: <6b162e43-3b0f-9e23-e63f-c844b0bc21da@oracle.com> References: <89399a72-0263-5ebd-5fda-82f404fbe2ca@redhat.com> <6b162e43-3b0f-9e23-e63f-c844b0bc21da@oracle.com> Message-ID: Ping? > Adding gc list. > > Thanks! > > /Robbin > > On 05/11/2017 12:08 PM, Roman Kennke wrote: >> Hi, >> >> ObjectSynchronizer::oops_do() currently iterates all monitor blocks, >> even free ones. With -XX:+MonitorInUseLists, each thread has its own >> in-use monitors list, and ObjectSynchronizer has a global gOmInUseList >> for moribund threads. Iterating over in-use monitors only significantly >> inmproves scanning time for monitors, because it avoids scanning free >> blocks, improves caching (no padding, and better chances to have >> thread-local oops in cache already). >> >> Performance-wise it makes a very significant difference (running >> gc-bench's roots.Sync test, which exaggerates synchronizer usage, with >> Shenandoah): >> >> baseline: >> S: Thread Roots 37748 us >> S: Synchronizer Roots 15115 us >> UR: Thread Roots 24967 us >> UR: Synchronizer Roots 11906 us >> >> patched: >> S: Thread Roots 40365 us >> S: Synchronizer Roots 0 us >> UR: Thread Roots 24459 us >> UR: Synchronizer Roots 0 us >> >> Testing: hotspot_gc, specjvm, jcstress -m quick >> >> http://cr.openjdk.java.net/~rkennke/8180175/webrev.00/ >> >> >> Roman >> >> From david.holmes at oracle.com Mon May 15 05:15:32 2017 From: david.holmes at oracle.com (David Holmes) Date: Mon, 15 May 2017 15:15:32 +1000 Subject: Should JVM know an app is in a shutdown state when a timeout event occurs? In-Reply-To: <70ad5403-8094-6349-f636-57b04b3e18a7@oracle.com> References: <007501d2cb6f$45857bf0$d09073d0$@cs.sjtu.edu.cn> <70ad5403-8094-6349-f636-57b04b3e18a7@oracle.com> Message-ID: Re-sending to hotspot-runtime-dev at openjdk.java.net On 13/05/2017 6:02 PM, David Holmes wrote: > Hi, > > What "timeout events" are you referring to? > > What you show below indicates an attempt to remove a shutdown hook after > the shutdown sequence has started - that is all in the JDK library code, > nothing to do with hotspot. > > Cheers, > David > > On 13/05/2017 8:29 AM, ??? wrote: >> I recently observed that there is a minor difference between HotSpot and >> IBM's J9 when they process >> the timeout events. I downloaded the dacapo benchmark >> (http://dacapobench.org/) and ran >> "timeout 20s java Harness avrora" >> . J9 sometimes throws a (java.lang.IllegalStateException: Shutdown in >> progress) while HotSpot cannot. >> The IllegalStateException may be thrown by J9 when >> Runtime.getRuntime().removeShutdownHook(shutdownThread) >> (see line 103, avrora.actions.SimAction) is executed and at the same time >> the timeout event >> arrives. >> >> In order to reproduce the event, I slightly rewrote avrora.Main.main a >> little so that avrora.Main.runAction >> will be called for 30 times. I also ran the command "timeout 20s java >> Harness avrora" for 30+ times >> so that Runtime.getRuntime().removeShutdownHook(shutdownThread) (the >> code at >> line 103, avrora.actions.SimAction) >> is running when "timeout 20s java" completes. >> >> It seems that when the timeout event occurs, the app is in a shutdown >> state. >> J9 is able to detect it, while >> HotSpot may miss catching it. >> >> I wonder what can be the real cause of the difference, the JVMs, or >> the API >> implementations (i.e., Runtime. >> getRuntime().addShutdownHook), or the application itself. Which JVM is >> correct? Actually when the >> timeout event occurs, HotSpot will terminate without executing the >> following >> statements, while J9 throws >> the exception and enters the exception handling block. >> >> avrora.actions.SimAction >> 84 public void run(String[] args) throws Exception { >> 85 SimUtil.REPORT_SECONDS = REPORT_SECONDS.get(); >> 86 SimUtil.SECONDS_PRECISION = (int)SECONDS_PRECISION.get(); >> 87 >> 88 simulation = Defaults.getSimulation(SIMULATION.get()); >> 89 simulation.process(options, args); >> 90 >> 91 ShutdownThread shutdownThread = new ShutdownThread(); >> 92 Runtime.getRuntime().addShutdownHook(shutdownThread); >> 93 printSimHeader(); >> 94 try { >> 95 startms = System.currentTimeMillis(); >> 96 simulation.start(); >> 97 simulation.join(); >> 98 } catch (Throwable t) { >> 99 exitSimulation(t); >> 100 >> Runtime.getRuntime().removeShutdownHook(shutdownThread); >> 101 } finally { >> 102 exitSimulation(null); >> 103 >> Runtime.getRuntime().removeShutdownHook(shutdownThread); >> 104 } >> 105 } >> >> The exception message is given as the follows: >> >> java.lang.reflect.InvocationTargetException >> java.lang.reflect.InvocationTargetException >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95 >> >> ) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl >> >> .java:55) >> at java.lang.reflect.Method.invoke(Method.java:508) >> at org.dacapo.harness.Avrora.iterate(Avrora.java:42) >> at org.dacapo.harness.Benchmark.run(Benchmark.java:166) >> at org.dacapo.harness.TestHarness.runBenchmark(TestHarness.java:218) >> at org.dacapo.harness.TestHarness.main(TestHarness.java:171) >> at Harness.main(Harness.java:17) >> Caused by: java.lang.IllegalStateException: Shutdown in progress >> at >> java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:93) >> >> at java.lang.Runtime.removeShutdownHook(Runtime.java:250) >> at avrora.actions.SimAction.run(SimAction.java:103) >> at avrora.Main.runAction(Unknown Source) >> at avrora.Main.main(Unknown Source) >> ... 9 more >> From david.holmes at oracle.com Tue May 16 06:52:47 2017 From: david.holmes at oracle.com (David Holmes) Date: Tue, 16 May 2017 16:52:47 +1000 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> Message-ID: <5db8f6b0-916c-ee46-d4c5-83136ad60c71@oracle.com> Correction ... On 16/05/2017 2:49 PM, David Holmes wrote: > On 16/05/2017 12:52 AM, Roman Kennke wrote: >> Am 11.05.2017 um 09:44 schrieb Robbin Ehn: >>> Hi, >>> >>> We have actually been discussing this last few days: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8153224 >> >> We are seeing the exact same issue as mentioned in the bug report too. >> Sometimes, time-to-safepoint is very high, in the area of ~200ms. The >> reason for this seems to be deflate_idle_monitors(). >> >> In the current implementation, the VM thread iterates over all threads, >> and deflates each thread's idle monitors one after another. >> >> I would rather have each Java thread deflate its own idle monitors >> before arriving at a safepoint. This should be much faster because >> higher chances a thread has its own monitors in cache, and deflation >> processing is done in parallel too. This requires some thought... e.g. >> what to do with non-runnable threads. > > This is risky and will need very careful analysis. The > type-stable-memory property of Monitors depends on very careful Monitors don't use TSM - ignore that part. :) > synchronization and the fact that deflation only happens at safepoints. This might not be as fragile as I was initially thinking, but a good design walkthrough of Monitor-lifecycle will be essential for understanding any subtleties of existing and proposed approaches. Thanks, David > David > ----- > >> If you think that's a good way to go, and don't already have a fix in >> the pipeline, I can take on the bug. I've got synchronizer still hot in >> memory, and experimented with self-processing Java threads before >> safepoint a while back too. >> >> Let me know! >> >> Roman >> >> From robbin.ehn at oracle.com Tue May 16 07:01:36 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 16 May 2017 09:01:36 +0200 Subject: RFR (jdk10): JDK-8180175: ObjectSynchronizer only needs to iterate in-use monitors In-Reply-To: References: <89399a72-0263-5ebd-5fda-82f404fbe2ca@redhat.com> <6b162e43-3b0f-9e23-e63f-c844b0bc21da@oracle.com> Message-ID: On 05/15/2017 11:01 PM, Roman Kennke wrote: > Ping? Looks ok, will look good when we can drop MonitorInUseLists branches. Are you handling the deprecation of MonitorInUseLists when CSR becomes available? /Robbin (not a 'R'eviewer) > > >> Adding gc list. >> >> Thanks! >> >> /Robbin >> >> On 05/11/2017 12:08 PM, Roman Kennke wrote: >>> Hi, >>> >>> ObjectSynchronizer::oops_do() currently iterates all monitor blocks, >>> even free ones. With -XX:+MonitorInUseLists, each thread has its own >>> in-use monitors list, and ObjectSynchronizer has a global gOmInUseList >>> for moribund threads. Iterating over in-use monitors only significantly >>> inmproves scanning time for monitors, because it avoids scanning free >>> blocks, improves caching (no padding, and better chances to have >>> thread-local oops in cache already). >>> >>> Performance-wise it makes a very significant difference (running >>> gc-bench's roots.Sync test, which exaggerates synchronizer usage, with >>> Shenandoah): >>> >>> baseline: >>> S: Thread Roots 37748 us >>> S: Synchronizer Roots 15115 us >>> UR: Thread Roots 24967 us >>> UR: Synchronizer Roots 11906 us >>> >>> patched: >>> S: Thread Roots 40365 us >>> S: Synchronizer Roots 0 us >>> UR: Thread Roots 24459 us >>> UR: Synchronizer Roots 0 us >>> >>> Testing: hotspot_gc, specjvm, jcstress -m quick >>> >>> http://cr.openjdk.java.net/~rkennke/8180175/webrev.00/ >>> >>> >>> Roman >>> >>> > > From rkennke at redhat.com Tue May 16 07:43:38 2017 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 16 May 2017 09:43:38 +0200 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: <5db8f6b0-916c-ee46-d4c5-83136ad60c71@oracle.com> References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> <5db8f6b0-916c-ee46-d4c5-83136ad60c71@oracle.com> Message-ID: <4329774c-aebf-0bcf-3022-060b8dc76028@redhat.com> Am 16.05.2017 um 08:52 schrieb David Holmes: > Correction ... > > On 16/05/2017 2:49 PM, David Holmes wrote: >> On 16/05/2017 12:52 AM, Roman Kennke wrote: >>> Am 11.05.2017 um 09:44 schrieb Robbin Ehn: >>>> Hi, >>>> >>>> We have actually been discussing this last few days: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8153224 >>> >>> We are seeing the exact same issue as mentioned in the bug report too. >>> Sometimes, time-to-safepoint is very high, in the area of ~200ms. The >>> reason for this seems to be deflate_idle_monitors(). >>> >>> In the current implementation, the VM thread iterates over all threads, >>> and deflates each thread's idle monitors one after another. >>> >>> I would rather have each Java thread deflate its own idle monitors >>> before arriving at a safepoint. This should be much faster because >>> higher chances a thread has its own monitors in cache, and deflation >>> processing is done in parallel too. This requires some thought... e.g. >>> what to do with non-runnable threads. >> >> This is risky and will need very careful analysis. The >> type-stable-memory property of Monitors depends on very careful > > Monitors don't use TSM - ignore that part. :) > >> synchronization and the fact that deflation only happens at safepoints. > > This might not be as fragile as I was initially thinking, but a good > design walkthrough of Monitor-lifecycle will be essential for > understanding any subtleties of existing and proposed approaches. I realized that monitor deflation depends strictly on all Java threads having arrived at safepoint. If a thread starts deflating its monitors while other threads are still in the process of getting to a safepoint, those other threads could mess with the monitors that the first threads is attempting to deflate. This is a no-go. However, I devised a scheme that works well: instead of letting the VM thread deflate all idle monitors after all threads arrived, and doing so single-threaded, I am now letting GC worker threads deflate idle monitors right before scanning/processing the synchronizer roots. E.g.: http://cr.openjdk.java.net/~rkennke/deflate-per-thread/webrev.01/ I can probably collapse monitor deflation and iteration into one pass too. This approach should be safe, right? I tested it with jcstress, specjvm and specjbb, with no ill effects so far. What do you think? Pause times also improve nicely in my experiments: Before: Initial Mark Pauses (G) = 12.76 s (a = 85047 us) Initial Mark Pauses (N) = 0.96 s (a = 6393 us) Patched: Initial Mark Pauses (G) = 4.49 s (a = 31814 us) Initial Mark Pauses (N) = 3.70 s (a = 26206 us) The (G) timing includes TTSP, the (N) timing is purely GC work. You can see that before it was very heavily dominated by TTSP, which was mostly deflating idle monitors. This work is now moved into the GC phase, and since it parallelizes better, the total pause is less than half compared to before. Roman From rkennke at redhat.com Tue May 16 07:45:05 2017 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 16 May 2017 09:45:05 +0200 Subject: RFR (jdk10): JDK-8180175: ObjectSynchronizer only needs to iterate in-use monitors In-Reply-To: References: <89399a72-0263-5ebd-5fda-82f404fbe2ca@redhat.com> <6b162e43-3b0f-9e23-e63f-c844b0bc21da@oracle.com> Message-ID: <32bfbca7-3eaa-1d83-3b75-702f0b36f918@redhat.com> Am 16.05.2017 um 09:01 schrieb Robbin Ehn: > On 05/15/2017 11:01 PM, Roman Kennke wrote: >> Ping? > > Looks ok, will look good when we can drop MonitorInUseLists branches. Ok. > Are you handling the deprecation of MonitorInUseLists when CSR becomes > available? Sure, can do this. Roman From thomas.schatzl at oracle.com Tue May 16 07:54:40 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 16 May 2017 09:54:40 +0200 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: <4329774c-aebf-0bcf-3022-060b8dc76028@redhat.com> References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> <5db8f6b0-916c-ee46-d4c5-83136ad60c71@oracle.com> <4329774c-aebf-0bcf-3022-060b8dc76028@redhat.com> Message-ID: <1494921280.2770.6.camel@oracle.com> Hi, On Tue, 2017-05-16 at 09:43 +0200, Roman Kennke wrote: > Am 16.05.2017 um 08:52 schrieb David Holmes: > > > > Correction ... > > > > On 16/05/2017 2:49 PM, David Holmes wrote: > > > > > > On 16/05/2017 12:52 AM, Roman Kennke wrote: > > > > > > > > Am 11.05.2017 um 09:44 schrieb Robbin Ehn: [...] > > > > > > synchronization and the fact that deflation only happens at > > > safepoints. > > This might not be as fragile as I was initially thinking, but a > > good design walkthrough of Monitor-lifecycle will be essential for > > understanding any subtleties of existing and proposed approaches. > I realized that monitor deflation depends strictly on all Java > threads having arrived at safepoint. If a thread starts deflating its > monitors while other threads are still in the process of getting to a > safepoint, those other threads could mess with the monitors that the > first threads is attempting to deflate. This is a no-go. > > However, I devised a scheme that works well: instead of letting the > VM thread deflate all idle monitors after all threads arrived, and > doing so single-threaded, I am now letting GC worker threads deflate > idle monitors right before scanning/processing the synchronizer > roots. E.g.: > > http://cr.openjdk.java.net/~rkennke/deflate-per-thread/webrev.01/ >? > I can probably collapse monitor deflation and iteration into one pass > too. ? is there a problem to just activate this by default and avoid adding an (experimental) option? Even experimental options are part of the public API (as they can be accessed in the product) and need a CSR request to add and need to go through the usual long-winded procedure to remove later (after we find out after 10 years that nobody ever actually uses that one). I mean, this change seems to be a win in all cases (or at most be as slow as before), so why allow the user to mess with it? Thanks, ? Thomas From thomas.schatzl at oracle.com Tue May 16 07:56:36 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 16 May 2017 09:56:36 +0200 Subject: [9, P1] RFR (S/M): 8180048: Interned string and symbol table leak memory during parallel unlinking In-Reply-To: <9d096824-17c9-7d86-82e0-288bd5f7c4f5@oracle.com> References: <1494518643.3120.18.camel@oracle.com> <1494586002.2459.13.camel@oracle.com> <1a7196cf-087e-9e8e-902a-bcffaf5dc9bf@oracle.com> <1494843751.2707.7.camel@oracle.com> <9d096824-17c9-7d86-82e0-288bd5f7c4f5@oracle.com> Message-ID: <1494921396.2770.7.camel@oracle.com> Hi Coleen, On Mon, 2017-05-15 at 08:28 -0400, coleen.phillimore at oracle.com wrote: > Yes, I like your additional comments. > Thanks, > Coleen > ? thanks for your review. Thomas From rkennke at redhat.com Tue May 16 07:58:57 2017 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 16 May 2017 09:58:57 +0200 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: <1494921280.2770.6.camel@oracle.com> References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> <5db8f6b0-916c-ee46-d4c5-83136ad60c71@oracle.com> <4329774c-aebf-0bcf-3022-060b8dc76028@redhat.com> <1494921280.2770.6.camel@oracle.com> Message-ID: Am 16.05.2017 um 09:54 schrieb Thomas Schatzl: > Hi, > > On Tue, 2017-05-16 at 09:43 +0200, Roman Kennke wrote: >> Am 16.05.2017 um 08:52 schrieb David Holmes: >>> Correction ... >>> >>> On 16/05/2017 2:49 PM, David Holmes wrote: >>>> On 16/05/2017 12:52 AM, Roman Kennke wrote: >>>>> Am 11.05.2017 um 09:44 schrieb Robbin Ehn: > [...] >>>> synchronization and the fact that deflation only happens at >>>> safepoints. >>> This might not be as fragile as I was initially thinking, but a >>> good design walkthrough of Monitor-lifecycle will be essential for >>> understanding any subtleties of existing and proposed approaches. >> I realized that monitor deflation depends strictly on all Java >> threads having arrived at safepoint. If a thread starts deflating its >> monitors while other threads are still in the process of getting to a >> safepoint, those other threads could mess with the monitors that the >> first threads is attempting to deflate. This is a no-go. >> >> However, I devised a scheme that works well: instead of letting the >> VM thread deflate all idle monitors after all threads arrived, and >> doing so single-threaded, I am now letting GC worker threads deflate >> idle monitors right before scanning/processing the synchronizer >> roots. E.g.: >> >> http://cr.openjdk.java.net/~rkennke/deflate-per-thread/webrev.01/ >> >> I can probably collapse monitor deflation and iteration into one pass >> too. > is there a problem to just activate this by default and avoid adding > an (experimental) option? Even experimental options are part of the > public API (as they can be accessed in the product) and need a CSR > request to add and need to go through the usual long-winded procedure > to remove later (after we find out after 10 years that nobody ever > actually uses that one). > > I mean, this change seems to be a win in all cases (or at most be as > slow as before), so why allow the user to mess with it? There's no problem. I made it this way mostly for me to be able to test and compare side-by-side easily :-) Will post RFR without the flag after I've done some more testing... Roman From thomas.schatzl at oracle.com Tue May 16 08:05:24 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 16 May 2017 10:05:24 +0200 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> <5db8f6b0-916c-ee46-d4c5-83136ad60c71@oracle.com> <4329774c-aebf-0bcf-3022-060b8dc76028@redhat.com> <1494921280.2770.6.camel@oracle.com> Message-ID: <1494921924.2770.10.camel@oracle.com> Hi, On Tue, 2017-05-16 at 09:58 +0200, Roman Kennke wrote: > Am 16.05.2017 um 09:54 schrieb Thomas Schatzl: > > On Tue, 2017-05-16 at 09:43 +0200, Roman Kennke wrote: > > > ?E.g.: > > > > > > http://cr.openjdk.java.net/~rkennke/deflate-per-thread/webrev.01/ > > > ? > > > I can probably collapse monitor deflation and iteration into one > > > pass too. > > > > ? is there a problem to just activate this by default and avoid > > adding an (experimental) option? Even experimental options are part > > of the public API (as they can be accessed in the product) and need > > a CSR request to add and need to go through the usual long-winded > > procedure to remove later (after we find out after 10 years that > > nobody ever actually uses that one). > > > > I mean, this change seems to be a win in all cases (or at most be > > as slow as before), so why allow the user to mess with it? > There's no problem. I made it this way mostly for me to be able to > test and compare side-by-side easily :-) Will post RFR without the > flag after I've done some more testing... ? Okay, thanks a lot :D Thanks, ? Thomas From rkennke at redhat.com Tue May 16 09:13:12 2017 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 16 May 2017 11:13:12 +0200 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: <1494921924.2770.10.camel@oracle.com> References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> <5db8f6b0-916c-ee46-d4c5-83136ad60c71@oracle.com> <4329774c-aebf-0bcf-3022-060b8dc76028@redhat.com> <1494921280.2770.6.camel@oracle.com> <1494921924.2770.10.camel@oracle.com> Message-ID: Am 16.05.2017 um 10:05 schrieb Thomas Schatzl: > Hi, > > On Tue, 2017-05-16 at 09:58 +0200, Roman Kennke wrote: >> Am 16.05.2017 um 09:54 schrieb Thomas Schatzl: >>> On Tue, 2017-05-16 at 09:43 +0200, Roman Kennke wrote: >>>> E.g.: >>>> >>>> http://cr.openjdk.java.net/~rkennke/deflate-per-thread/webrev.01/ >>>> >>>> I can probably collapse monitor deflation and iteration into one >>>> pass too. >>> is there a problem to just activate this by default and avoid >>> adding an (experimental) option? Even experimental options are part >>> of the public API (as they can be accessed in the product) and need >>> a CSR request to add and need to go through the usual long-winded >>> procedure to remove later (after we find out after 10 years that >>> nobody ever actually uses that one). >>> >>> I mean, this change seems to be a win in all cases (or at most be >>> as slow as before), so why allow the user to mess with it? >> There's no problem. I made it this way mostly for me to be able to >> test and compare side-by-side easily :-) Will post RFR without the >> flag after I've done some more testing... > Okay, thanks a lot :D In the meantime, need review for: http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2017-May/023348.html this change is depending on that. :-) Roman From robbin.ehn at oracle.com Tue May 16 09:24:31 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 16 May 2017 11:24:31 +0200 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> <5db8f6b0-916c-ee46-d4c5-83136ad60c71@oracle.com> <4329774c-aebf-0bcf-3022-060b8dc76028@redhat.com> <1494921280.2770.6.camel@oracle.com> <1494921924.2770.10.camel@oracle.com> Message-ID: <378b058f-e1bb-20e3-cace-282aafeeade4@oracle.com> Hi, On 05/16/2017 11:13 AM, Roman Kennke wrote: >>>>> http://cr.openjdk.java.net/~rkennke/deflate-per-thread/webrev.01/ I only had a quick look, one thing that is troubling is that: SafepointSynchronize::begin() -> SafepointSynchronize::do_cleanup_tasks() -> ObjectSynchronizer::deflate_idle_monitors() no longer actually deflates threads om list, so that ObjectSynchronizer::omAlloc(..) -> InduceScavenge() -> VMThread::execute(new VM_ForceAsyncSafepoint()) do not have intended effect? /Robbin > > Roman > From rkennke at redhat.com Tue May 16 09:37:56 2017 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 16 May 2017 11:37:56 +0200 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: <378b058f-e1bb-20e3-cace-282aafeeade4@oracle.com> References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> <5db8f6b0-916c-ee46-d4c5-83136ad60c71@oracle.com> <4329774c-aebf-0bcf-3022-060b8dc76028@redhat.com> <1494921280.2770.6.camel@oracle.com> <1494921924.2770.10.camel@oracle.com> <378b058f-e1bb-20e3-cace-282aafeeade4@oracle.com> Message-ID: <57243bb9-8ea1-7649-0157-e38b20a0e37f@redhat.com> Am 16.05.2017 um 11:24 schrieb Robbin Ehn: > Hi, > > On 05/16/2017 11:13 AM, Roman Kennke wrote: >>>>>> http://cr.openjdk.java.net/~rkennke/deflate-per-thread/webrev.01/ > > I only had a quick look, one thing that is troubling is that: > > SafepointSynchronize::begin() -> > SafepointSynchronize::do_cleanup_tasks() -> > ObjectSynchronizer::deflate_idle_monitors() > no longer actually deflates threads om list, so that > ObjectSynchronizer::omAlloc(..) -> InduceScavenge() -> > VMThread::execute(new VM_ForceAsyncSafepoint()) > do not have intended effect? Good point! It should be easy to solve though, by checking if (ForceMonitorScavenge) in deflate_idle_monitors() and scavenging all monitors in this case. Right? It comes with the disadvantage of potentially long-ish pause though (esp. when running with many Java threads) -- but not worse than it is now. Roman From robbin.ehn at oracle.com Tue May 16 09:45:16 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 16 May 2017 11:45:16 +0200 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: <57243bb9-8ea1-7649-0157-e38b20a0e37f@redhat.com> References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> <5db8f6b0-916c-ee46-d4c5-83136ad60c71@oracle.com> <4329774c-aebf-0bcf-3022-060b8dc76028@redhat.com> <1494921280.2770.6.camel@oracle.com> <1494921924.2770.10.camel@oracle.com> <378b058f-e1bb-20e3-cace-282aafeeade4@oracle.com> <57243bb9-8ea1-7649-0157-e38b20a0e37f@redhat.com> Message-ID: <86c15f52-f6c0-e5f6-ee45-48ac1ac744d2@oracle.com> On 05/16/2017 11:37 AM, Roman Kennke wrote: > Am 16.05.2017 um 11:24 schrieb Robbin Ehn: >> Hi, >> >> On 05/16/2017 11:13 AM, Roman Kennke wrote: >>>>>>> http://cr.openjdk.java.net/~rkennke/deflate-per-thread/webrev.01/ >> >> I only had a quick look, one thing that is troubling is that: >> >> SafepointSynchronize::begin() -> >> SafepointSynchronize::do_cleanup_tasks() -> >> ObjectSynchronizer::deflate_idle_monitors() >> no longer actually deflates threads om list, so that >> ObjectSynchronizer::omAlloc(..) -> InduceScavenge() -> >> VMThread::execute(new VM_ForceAsyncSafepoint()) >> do not have intended effect? > > Good point! > > It should be easy to solve though, by checking if (ForceMonitorScavenge) > in deflate_idle_monitors() and scavenging all monitors in this case. > Right? It comes with the disadvantage of potentially long-ish pause > though (esp. when running with many Java threads) -- but not worse than > it is now. Yes, but since we are at a safepoint you should be able to use gc worker in that case also? So I think a more elaborate fix here is to prefer? /Robbin > > Roman > From robbin.ehn at oracle.com Tue May 16 09:48:20 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 16 May 2017 11:48:20 +0200 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: <86c15f52-f6c0-e5f6-ee45-48ac1ac744d2@oracle.com> References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> <5db8f6b0-916c-ee46-d4c5-83136ad60c71@oracle.com> <4329774c-aebf-0bcf-3022-060b8dc76028@redhat.com> <1494921280.2770.6.camel@oracle.com> <1494921924.2770.10.camel@oracle.com> <378b058f-e1bb-20e3-cace-282aafeeade4@oracle.com> <57243bb9-8ea1-7649-0157-e38b20a0e37f@redhat.com> <86c15f52-f6c0-e5f6-ee45-48ac1ac744d2@oracle.com> Message-ID: <0db4a13e-dae1-4a81-0c46-c9187990662d@oracle.com> Correction, On 05/16/2017 11:45 AM, Robbin Ehn wrote: > Yes, but since we are at a safepoint you should be able to use gc worker in that case also? or the java thread. And scratch 'also' :) /Robbin From rkennke at redhat.com Tue May 16 10:32:22 2017 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 16 May 2017 12:32:22 +0200 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: <0db4a13e-dae1-4a81-0c46-c9187990662d@oracle.com> References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> <5db8f6b0-916c-ee46-d4c5-83136ad60c71@oracle.com> <4329774c-aebf-0bcf-3022-060b8dc76028@redhat.com> <1494921280.2770.6.camel@oracle.com> <1494921924.2770.10.camel@oracle.com> <378b058f-e1bb-20e3-cace-282aafeeade4@oracle.com> <57243bb9-8ea1-7649-0157-e38b20a0e37f@redhat.com> <86c15f52-f6c0-e5f6-ee45-48ac1ac744d2@oracle.com> <0db4a13e-dae1-4a81-0c46-c9187990662d@oracle.com> Message-ID: Am 16.05.2017 um 11:48 schrieb Robbin Ehn: > Correction, > > On 05/16/2017 11:45 AM, Robbin Ehn wrote: >> Yes, but since we are at a safepoint you should be able to use gc >> worker in that case also? > > or the java thread. And scratch 'also' :) This clearly requires co-operation from the GC. Also, I noticed that some GCs cannot do deflation during root scanning, because they stow away mark words during marking, and temporarily store forwarding pointers into the mark words, and only restore mark words after GC. This obviously conflicts with monitor deflation. Which means we need some GC cooperation there too... I will figure something out. Roman From varming at gmail.com Tue May 16 11:37:42 2017 From: varming at gmail.com (Carsten Varming) Date: Tue, 16 May 2017 07:37:42 -0400 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> <5db8f6b0-916c-ee46-d4c5-83136ad60c71@oracle.com> <4329774c-aebf-0bcf-3022-060b8dc76028@redhat.com> <1494921280.2770.6.camel@oracle.com> <1494921924.2770.10.camel@oracle.com> <378b058f-e1bb-20e3-cace-282aafeeade4@oracle.com> <57243bb9-8ea1-7649-0157-e38b20a0e37f@redhat.com> <86c15f52-f6c0-e5f6-ee45-48ac1ac744d2@oracle.com> <0db4a13e-dae1-4a81-0c46-c9187990662d@oracle.com> Message-ID: FYI. Last year I wrote a small patch[1] against JDK9 that enables monitor deflation to happen outside a safepoint. There was little interest then, so I never proposed it as a patch. The JDK9 code has moved on a little bit, so the patch probably doesn't apply cleanly, but the patch shows one way to complete the life of a monitor without going through a safepoint. Carsten [1] http://cr.openjdk.java.net/~cvarming/monitor_deflate_conc/0/ On Tue, May 16, 2017 at 6:32 AM, Roman Kennke wrote: > Am 16.05.2017 um 11:48 schrieb Robbin Ehn: > > Correction, > > > > On 05/16/2017 11:45 AM, Robbin Ehn wrote: > >> Yes, but since we are at a safepoint you should be able to use gc > >> worker in that case also? > > > > or the java thread. And scratch 'also' :) > > This clearly requires co-operation from the GC. > > Also, I noticed that some GCs cannot do deflation during root scanning, > because they stow away mark words during marking, and temporarily store > forwarding pointers into the mark words, and only restore mark words > after GC. This obviously conflicts with monitor deflation. Which means > we need some GC cooperation there too... I will figure something out. > > Roman > > From rkennke at redhat.com Tue May 16 12:05:38 2017 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 16 May 2017 14:05:38 +0200 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> <5db8f6b0-916c-ee46-d4c5-83136ad60c71@oracle.com> <4329774c-aebf-0bcf-3022-060b8dc76028@redhat.com> <1494921280.2770.6.camel@oracle.com> <1494921924.2770.10.camel@oracle.com> <378b058f-e1bb-20e3-cace-282aafeeade4@oracle.com> <57243bb9-8ea1-7649-0157-e38b20a0e37f@redhat.com> <86c15f52-f6c0-e5f6-ee45-48ac1ac744d2@oracle.com> <0db4a13e-dae1-4a81-0c46-c9187990662d@oracle.com> Message-ID: <0faa93a3-4075-9cae-cd72-fbed1d00ab07@redhat.com> Am 16.05.2017 um 13:37 schrieb Carsten Varming: > FYI. Last year I wrote a small patch[1] against JDK9 that enables > monitor deflation to happen outside a safepoint. There was little > interest then, so I never proposed it as a patch. The JDK9 code has > moved on a little bit, so the patch probably doesn't apply cleanly, > but the patch shows one way to complete the life of a monitor without > going through a safepoint. Wow, now that is interesting! Can you give me a quick summary-description how you avoid racing with other threads that might inflate the monitor? As far as I know, monitor deflation (currently) requires that all Java threads stand still at a safepoint. Can it also be made to work with MonitorInUseLists? Roman > > Carsten > > [1] http://cr.openjdk.java.net/~cvarming/monitor_deflate_conc/0/ > > > On Tue, May 16, 2017 at 6:32 AM, Roman Kennke > wrote: > > Am 16.05.2017 um 11:48 schrieb Robbin Ehn: > > Correction, > > > > On 05/16/2017 11:45 AM, Robbin Ehn wrote: > >> Yes, but since we are at a safepoint you should be able to use gc > >> worker in that case also? > > > > or the java thread. And scratch 'also' :) > > This clearly requires co-operation from the GC. > > Also, I noticed that some GCs cannot do deflation during root > scanning, > because they stow away mark words during marking, and temporarily > store > forwarding pointers into the mark words, and only restore mark words > after GC. This obviously conflicts with monitor deflation. Which means > we need some GC cooperation there too... I will figure something out. > > Roman > > From robbin.ehn at oracle.com Tue May 16 12:33:06 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 16 May 2017 14:33:06 +0200 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> <5db8f6b0-916c-ee46-d4c5-83136ad60c71@oracle.com> <4329774c-aebf-0bcf-3022-060b8dc76028@redhat.com> <1494921280.2770.6.camel@oracle.com> <1494921924.2770.10.camel@oracle.com> <378b058f-e1bb-20e3-cace-282aafeeade4@oracle.com> <57243bb9-8ea1-7649-0157-e38b20a0e37f@redhat.com> <86c15f52-f6c0-e5f6-ee45-48ac1ac744d2@oracle.com> <0db4a13e-dae1-4a81-0c46-c9187990662d@oracle.com> Message-ID: On 05/16/2017 01:37 PM, Carsten Varming wrote: > FYI. Last year I wrote a small patch[1] against JDK9 that enables monitor deflation to happen outside a safepoint. There was little interest then, so I never proposed it as > a patch. The JDK9 code has moved on a little bit, so the patch probably doesn't apply cleanly, but the patch shows one way to complete the life of a monitor without going > through a safepoint. > > Carsten > > [1] http://cr.openjdk.java.net/~cvarming/monitor_deflate_conc/0/ Interesting, and a bit scary, I'll need to take a deeper look. Thanks! /Robbin > > On Tue, May 16, 2017 at 6:32 AM, Roman Kennke > wrote: > > Am 16.05.2017 um 11:48 schrieb Robbin Ehn: > > Correction, > > > > On 05/16/2017 11:45 AM, Robbin Ehn wrote: > >> Yes, but since we are at a safepoint you should be able to use gc > >> worker in that case also? > > > > or the java thread. And scratch 'also' :) > > This clearly requires co-operation from the GC. > > Also, I noticed that some GCs cannot do deflation during root scanning, > because they stow away mark words during marking, and temporarily store > forwarding pointers into the mark words, and only restore mark words > after GC. This obviously conflicts with monitor deflation. Which means > we need some GC cooperation there too... I will figure something out. > > Roman > > From rkennke at redhat.com Tue May 16 12:53:30 2017 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 16 May 2017 14:53:30 +0200 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> <5db8f6b0-916c-ee46-d4c5-83136ad60c71@oracle.com> <4329774c-aebf-0bcf-3022-060b8dc76028@redhat.com> <1494921280.2770.6.camel@oracle.com> <1494921924.2770.10.camel@oracle.com> <378b058f-e1bb-20e3-cace-282aafeeade4@oracle.com> <57243bb9-8ea1-7649-0157-e38b20a0e37f@redhat.com> <86c15f52-f6c0-e5f6-ee45-48ac1ac744d2@oracle.com> <0db4a13e-dae1-4a81-0c46-c9187990662d@oracle.com> Message-ID: <373f1d3a-4c60-bc4c-5ed6-0cdc0f23a1db@redhat.com> I'd be interested in giving it a spin with Shenandoah. I might even hack up MonitorInUseLists support. Would you be ok with me taking the patch and doing some modifications to it, and maybe pushing it to the Shenandoah repo? Instead of hooking up to the Service thread, I'd hook it up to GC scheduler thread (G1 and Shenandoah only). This way we'd ensure that monitors have been deflated when we get to marking the roots, which considerably shortens the respective pauses. Roman Am 16.05.2017 um 13:37 schrieb Carsten Varming: > FYI. Last year I wrote a small patch[1] against JDK9 that enables > monitor deflation to happen outside a safepoint. There was little > interest then, so I never proposed it as a patch. The JDK9 code has > moved on a little bit, so the patch probably doesn't apply cleanly, > but the patch shows one way to complete the life of a monitor without > going through a safepoint. > > Carsten > > [1] http://cr.openjdk.java.net/~cvarming/monitor_deflate_conc/0/ > > > On Tue, May 16, 2017 at 6:32 AM, Roman Kennke > wrote: > > Am 16.05.2017 um 11:48 schrieb Robbin Ehn: > > Correction, > > > > On 05/16/2017 11:45 AM, Robbin Ehn wrote: > >> Yes, but since we are at a safepoint you should be able to use gc > >> worker in that case also? > > > > or the java thread. And scratch 'also' :) > > This clearly requires co-operation from the GC. > > Also, I noticed that some GCs cannot do deflation during root > scanning, > because they stow away mark words during marking, and temporarily > store > forwarding pointers into the mark words, and only restore mark words > after GC. This obviously conflicts with monitor deflation. Which means > we need some GC cooperation there too... I will figure something out. > > Roman > > From varming at gmail.com Tue May 16 12:54:27 2017 From: varming at gmail.com (Carsten Varming) Date: Tue, 16 May 2017 08:54:27 -0400 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: <373f1d3a-4c60-bc4c-5ed6-0cdc0f23a1db@redhat.com> References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> <5db8f6b0-916c-ee46-d4c5-83136ad60c71@oracle.com> <4329774c-aebf-0bcf-3022-060b8dc76028@redhat.com> <1494921280.2770.6.camel@oracle.com> <1494921924.2770.10.camel@oracle.com> <378b058f-e1bb-20e3-cace-282aafeeade4@oracle.com> <57243bb9-8ea1-7649-0157-e38b20a0e37f@redhat.com> <86c15f52-f6c0-e5f6-ee45-48ac1ac744d2@oracle.com> <0db4a13e-dae1-4a81-0c46-c9187990662d@oracle.com> <373f1d3a-4c60-bc4c-5ed6-0cdc0f23a1db@redhat.com> Message-ID: Dear Roman, Give it a go. Carsten On Tue, May 16, 2017 at 8:53 AM, Roman Kennke wrote: > I'd be interested in giving it a spin with Shenandoah. I might even hack > up MonitorInUseLists support. Would you be ok with me taking the patch and > doing some modifications to it, and maybe pushing it to the Shenandoah repo? > > Instead of hooking up to the Service thread, I'd hook it up to GC > scheduler thread (G1 and Shenandoah only). This way we'd ensure that > monitors have been deflated when we get to marking the roots, which > considerably shortens the respective pauses. > > Roman > > Am 16.05.2017 um 13:37 schrieb Carsten Varming: > > FYI. Last year I wrote a small patch[1] against JDK9 that enables monitor > deflation to happen outside a safepoint. There was little interest then, so > I never proposed it as a patch. The JDK9 code has moved on a little bit, so > the patch probably doesn't apply cleanly, but the patch shows one way to > complete the life of a monitor without going through a safepoint. > > Carsten > > [1] http://cr.openjdk.java.net/~cvarming/monitor_deflate_conc/0/ > > On Tue, May 16, 2017 at 6:32 AM, Roman Kennke wrote: > >> Am 16.05.2017 um 11:48 schrieb Robbin Ehn: >> > Correction, >> > >> > On 05/16/2017 11:45 AM, Robbin Ehn wrote: >> >> Yes, but since we are at a safepoint you should be able to use gc >> >> worker in that case also? >> > >> > or the java thread. And scratch 'also' :) >> >> This clearly requires co-operation from the GC. >> >> Also, I noticed that some GCs cannot do deflation during root scanning, >> because they stow away mark words during marking, and temporarily store >> forwarding pointers into the mark words, and only restore mark words >> after GC. This obviously conflicts with monitor deflation. Which means >> we need some GC cooperation there too... I will figure something out. >> >> Roman >> >> > > From harold.seigel at oracle.com Tue May 16 13:36:25 2017 From: harold.seigel at oracle.com (harold seigel) Date: Tue, 16 May 2017 09:36:25 -0400 Subject: RFR 8178604: JVM does not allow defining boot loader modules in exploded build after module system initialization In-Reply-To: <1b618a5c-671d-9063-326e-16991efab441@oracle.com> References: <85e36988-a145-6805-8659-56c738013f58@oracle.com> <46a812ca-228d-01e7-e889-fd22f2af0537@oracle.com> <11709d4a-0a41-1d38-fcd1-67d42c33eaeb@oracle.com> <3ee613ff-f6da-9ef8-6781-efce4edba29e@oracle.com> <21b3fdc5-d8a7-0535-3ac6-f1a097a7431d@oracle.com> <388b80ed-e436-f888-ffed-5b92e0117ce2@oracle.com> <2e1d1c53-8d6d-e606-f047-a223f76b63a9@oracle.com> <1b618a5c-671d-9063-326e-16991efab441@oracle.com> Message-ID: <4550b8c7-9bff-a44d-fefb-b5b589da371f@oracle.com> Hi David, Thanks for the review! I'll change the initialization of _exploded_entries so that it is done eagerly, before I push the change. Harold On 5/10/2017 10:11 PM, David Holmes wrote: > Hi Harold, > > On 11/05/2017 7:18 AM, harold seigel wrote: >> Hi Lois, David, >> >> Please review this latest webrev that initializes _exploded_entries >> without locking: >> >> http://cr.openjdk.java.net/~hseigel/bug_8178604.4/webrev/index.html > > Ok. > > It would be a lot clearer, in my opinion, that the initialization is > indeed thread-safe, if it were done eagerly instead of lazily. The > execution contexts of the first and second calls (in separate threads) > to that chunk of code are not readily discernible by any reasonable > means. But the extensive comment is appreciated. > > Thanks, > David > >> Thanks, Harold >> >> >> On 5/10/2017 1:19 PM, Lois Foltan wrote: >>> On 5/9/2017 5:18 PM, David Holmes wrote: >>>> On 9/05/2017 11:14 PM, harold seigel wrote: >>>>> Hi David, >>>>> >>>>> See comments embedded below. >>>>> >>>>> Thanks, Harold >>>>> >>>>> >>>>> On 5/8/2017 8:52 PM, David Holmes wrote: >>>>>> Hi Harold, >>>>>> >>>>>> On 9/05/2017 4:00 AM, harold seigel wrote: >>>>>>> Hi David, >>>>>>> >>>>>>> Please see this updated webrev: >>>>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604.3/webrev/index.html >>>>>>> >>>>>>> It contains the changes you requested below. >>>>>> >>>>>> Thank you. I will point out that Lois originally suggested the same >>>>>> changes. >>>>>> >>>>>>> Note that the _exploded_entries table gets created very early >>>>>>> during >>>>>>> SystemDictionary initialization, before initializing the pre-loaded >>>>>>> classes. So, the code in load_class() at line ~1503 can assume >>>>>>> that >>>>>>> _exploded_entries is not null. >>>>>> >>>>>> Which thread executes ClassLoader::add_to_exploded_build_list, and >>>>>> which thread executes load_class() first? If they are guaranteed >>>>>> to be >>>>>> the same thread then you don't need locking on the construction. If >>>>>> they can be different threads then there must be some >>>>>> synchronization >>>>>> between them that ensures _exploded_entries is properly visible >>>>>> after >>>>>> construction. >>>>> They are the same thread. At startup, the thread calls >>>> >>>> Then as I said, why do we need any locking? I note Lois is the one >>>> that requested this and you questioned it. I also question it. >>> >>> Thank you Harold for investigating this. I stand corrected, it seems >>> that we do not need the lock after all when allocating >>> ClassLoader::_exploded_entries. >>> Lois >>> >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> SystemDictionary::initialize_preloaded_classes(), which calls >>>>> ClassLoader::classLoader_init2(), which calls >>>>> add_to_exploded_build_list(). >>>>> >>>>> Control then returns back to >>>>> SystemDictionary::initialize_preloaded_classes() which eventually >>>>> calls >>>>> SystemDictionary::load_instance_class(), which calls >>>>> ClassLoader::load_class(). Here's the call stack showing these >>>>> calls: >>>>> >>>>> V [libjvm.so+0x974c0c] ClassLoader::load_class(Symbol*, bool, >>>>> Thread*)+0x41c >>>>> V [libjvm.so+0x1648339] >>>>> SystemDictionary::load_instance_class(Symbol*, Handle, >>>>> Thread*)+0x819 >>>>> V [libjvm.so+0x1648e25] >>>>> SystemDictionary::resolve_instance_class_or_null(Symbol*, Handle, >>>>> Handle, Thread*)+0x985 >>>>> V [libjvm.so+0x16494ba] >>>>> SystemDictionary::resolve_or_null(Symbol*, >>>>> Handle, Handle, Thread*)+0x5a >>>>> V [libjvm.so+0x16498fe] >>>>> SystemDictionary::resolve_or_fail(Symbol*, >>>>> Handle, Handle, bool, Thread*)+0x1e >>>>> V [libjvm.so+0x1649aa9] >>>>> SystemDictionary::initialize_wk_klass(SystemDictionary::WKID, int, >>>>> Thread*)+0x149 >>>>> V [libjvm.so+0x1649bfe] >>>>> SystemDictionary::initialize_wk_klasses_until(SystemDictionary::WKID, >>>>> SystemDictionary::WKID&, >>>>> >>>>> Thread*)+0x5e >>>>> V [libjvm.so+0x1649dba] >>>>> SystemDictionary::*initialize_preloaded_classes*(Thread*)+0xea >>>>> V [libjvm.so+0x164a28a] >>>>> SystemDictionary::initialize(Thread*)+0x26a >>>>> V [libjvm.so+0x16cd7e0] Universe::genesis(Thread*)+0x7c0 >>>>> V [libjvm.so+0x16ce4cc] universe2_init()+0x2c >>>>> V [libjvm.so+0xe0f268] init_globals()+0xa8 >>>>> V [libjvm.so+0x1696e6f] Threads::create_vm(JavaVMInitArgs*, >>>>> bool*)+0x2df >>>>> ... >>>>> >>>>> So, is the existing synchronization in this change sufficient? >>>>> >>>>> Thanks, Harold >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>>> Thanks, Harold >>>>>>> >>>>>>> On 5/4/2017 5:32 PM, David Holmes wrote: >>>>>>>> Hi Harold, >>>>>>>> >>>>>>>> On 5/05/2017 3:47 AM, harold seigel wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Please review this updated webrev: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604.2/webrev/index.html >>>>>>>>> >>>>>>>>> >>>>>>>>> The updated webrev takes out a lock when creating >>>>>>>>> _exploded_entries. It >>>>>>>>> also contains additional comments and an assert() to address >>>>>>>>> Lois's >>>>>>>>> concerns with reduced code readability involving what situations >>>>>>>>> warrant >>>>>>>>> taking out a lock in ClassLoader::search_module_entries(). >>>>>>>>> >>>>>>>>> I don't think there needs to be two different initial entry >>>>>>>>> points as >>>>>>>>> suggested by David below. The method behaves slightly >>>>>>>>> differently >>>>>>>>> depending on the value of its 'need_lock' parameter. But, >>>>>>>>> there is >>>>>>>>> enough common code for it to remain one method. >>>>>>>> >>>>>>>> I think this highlights uncertainty about the concurrent >>>>>>>> behaviour in >>>>>>>> relation to this code. On the one hand you are worried about an >>>>>>>> initialization race and so use the lock, but when you do: >>>>>>>> >>>>>>>> 1500 // The exploded build entries can be added to at any >>>>>>>> time >>>>>>>> so pass 'true' for >>>>>>>> 1501 // need_lock so that a lock is taken out when searching >>>>>>>> them. >>>>>>>> 1502 assert(_exploded_entries != NULL, "No exploded build >>>>>>>> entries present"); >>>>>>>> 1503 stream = search_module_entries(_exploded_entries, >>>>>>>> class_name, file_name, true, CHECK_NULL); >>>>>>>> >>>>>>>> you load _exploded_entries with no lock held and so must be >>>>>>>> certain >>>>>>>> that you can not possibly read a null value here. >>>>>>>> >>>>>>>> The needs_lock parameter seems superfluous - you know the >>>>>>>> condition >>>>>>>> for locking is that the list is the _exploded_entries, and you >>>>>>>> assert >>>>>>>> that. So no need to pass in needs_lock, just grab the lock when >>>>>>>> dealing with _exploded_entries. >>>>>>>> >>>>>>>> Cheers, >>>>>>>> David >>>>>>>> >>>>>>>>> Thanks, Harold >>>>>>>>> >>>>>>>>> On 5/3/2017 12:59 AM, David Holmes wrote: >>>>>>>>>> On 3/05/2017 7:11 AM, Lois Foltan wrote: >>>>>>>>>>> On 5/2/2017 9:03 AM, harold seigel wrote: >>>>>>>>>>>> Hi Lois, >>>>>>>>>>>> >>>>>>>>>>>> Thanks for your comments. Please see my in-line responses. >>>>>>>>>>>> >>>>>>>>>>>> Harold >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 5/1/2017 4:14 PM, Lois Foltan wrote: >>>>>>>>>>>>> Hi Harold, >>>>>>>>>>>>> >>>>>>>>>>>>> Looks good. A couple of comments: >>>>>>>>>>>>> >>>>>>>>>>>>> src/share/vm/classfile/classLoader.cpp >>>>>>>>>>>>> line #828 - please take out the Module_lock when the >>>>>>>>>>>>> _exploded_entries is created >>>>>>>>>>>> This fix does not change the possible need for >>>>>>>>>>>> synchronization when >>>>>>>>>>>> _exploded_entries gets created during module system >>>>>>>>>>>> initialization. >>>>>>>>>>>> Are you saying that the existing code is wrong and should have >>>>>>>>>>>> taken >>>>>>>>>>>> out the Module_lock before creating _exploding_entries? >>>>>>>>>>> >>>>>>>>>>> The method ClassLoader::add_to_exploded_build_list can be >>>>>>>>>>> called at >>>>>>>>>>> anytime (during module system initialization and post module >>>>>>>>>>> system >>>>>>>>>>> initialization). Thus it seems prudent that no matter what the >>>>>>>>>>> current >>>>>>>>>>> JVM phase, that any access to >>>>>>>>>>> ClassLoader::_exploding_entries be >>>>>>>>>>> protected via a lock. That includes creation as well as >>>>>>>>>>> insertion >>>>>>>>>>> into >>>>>>>>>>> the list. >>>>>>>>>> >>>>>>>>>> If creation is not guaranteed to occur before other threads >>>>>>>>>> that will >>>>>>>>>> call the same method and use the _exploded_entries list, then >>>>>>>>>> synchronization of some form is essential. If it is guaranteed >>>>>>>>>> then we >>>>>>>>>> don't need create time sync just to be prudent - but the >>>>>>>>>> guarantee >>>>>>>>>> must be firmly established and clearly documented. >>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> line #1402 - I like the new factored out method >>>>>>>>>>>>> find_first_module_cpe(), however, instead of adding a new >>>>>>>>>>>>> "need_lock" >>>>>>>>>>>>> parameter, I would rather see code >>>>>>>>>>>>> in find_first_module_cpe() that >>>>>>>>>>>>> takes the >>>>>>>>>>>>> Module_lock if the module_list parameter equals >>>>>>>>>>>>> ClassLoader::_exploded_entries >>>>>>>>>>>> I think your suggestion makes the code less flexible and does >>>>>>>>>>>> not >>>>>>>>>>>> require potential future callers of search_module_entries() to >>>>>>>>>>>> think >>>>>>>>>>>> about synchronization. So, I'd prefer to leave that change >>>>>>>>>>>> as is. >>>>>>>>>>> >>>>>>>>>>> I see your point. My point was based on reduced code >>>>>>>>>>> readability, the >>>>>>>>>>> further away from the decision to establish the lock, the less >>>>>>>>>>> apparent >>>>>>>>>>> it is within the new method >>>>>>>>>>> ClassLoader::find_first_module_cpe() >>>>>>>>>>> as to >>>>>>>>>>> what situations warranted the lock in the first place. >>>>>>>>>> >>>>>>>>>> Further, this: >>>>>>>>>> >>>>>>>>>> 1495 stream = search_module_entries(_exploded_entries, >>>>>>>>>> class_name, file_name, true, CHECK_NULL); >>>>>>>>>> >>>>>>>>>> is potentially unsafe if we can race with creation because we >>>>>>>>>> first >>>>>>>>>> access _exploded_entries without holding any lock! And the fact >>>>>>>>>> the >>>>>>>>>> method needs to do different things depending on which "list" >>>>>>>>>> was >>>>>>>>>> passed in suggests to me it should be two different initial >>>>>>>>>> entry >>>>>>>>>> points. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> David >>>>>>>>>> ----- >>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Lois >>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Lois >>>>>>>>>>>>> >>>>>>>>>>>>> On 5/1/2017 3:36 PM, harold seigel wrote: >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Please review this JDK-10 bug fix to allow defining of >>>>>>>>>>>>>> modules to >>>>>>>>>>>>>> the boot loader in exploded builds after module system >>>>>>>>>>>>>> initialization. The fix uses the module lock to synchronize >>>>>>>>>>>>>> access >>>>>>>>>>>>>> to the _exploded_entries data structure (which is only >>>>>>>>>>>>>> used by >>>>>>>>>>>>>> exploded builds). >>>>>>>>>>>>>> >>>>>>>>>>>>>> Note that the above capability already exists for regular >>>>>>>>>>>>>> builds. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Open Webrev: >>>>>>>>>>>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604/webrev/index.html >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8178604 >>>>>>>>>>>>>> >>>>>>>>>>>>>> The fix was tested with JCK tests, the JTreg hotspot, >>>>>>>>>>>>>> java/io, >>>>>>>>>>>>>> java/lang, java/util and other tests, the RBT tier2 -tier5 >>>>>>>>>>>>>> tests, >>>>>>>>>>>>>> the co-located NSK tests, and with JPRT. The JTReg and JCK >>>>>>>>>>>>>> tests >>>>>>>>>>>>>> were run with an exploded build. Also, the example >>>>>>>>>>>>>> program in >>>>>>>>>>>>>> the >>>>>>>>>>>>>> JBS bug was run by hand to verify the fix. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, Harold >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >>> >> From varming at gmail.com Tue May 16 13:53:24 2017 From: varming at gmail.com (Carsten Varming) Date: Tue, 16 May 2017 09:53:24 -0400 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: <0faa93a3-4075-9cae-cd72-fbed1d00ab07@redhat.com> References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> <5db8f6b0-916c-ee46-d4c5-83136ad60c71@oracle.com> <4329774c-aebf-0bcf-3022-060b8dc76028@redhat.com> <1494921280.2770.6.camel@oracle.com> <1494921924.2770.10.camel@oracle.com> <378b058f-e1bb-20e3-cace-282aafeeade4@oracle.com> <57243bb9-8ea1-7649-0157-e38b20a0e37f@redhat.com> <86c15f52-f6c0-e5f6-ee45-48ac1ac744d2@oracle.com> <0db4a13e-dae1-4a81-0c46-c9187990662d@oracle.com> <0faa93a3-4075-9cae-cd72-fbed1d00ab07@redhat.com> Message-ID: Dear Roman, I didn't do anything to extend it to work with MonitorInUseLists (it wasn't on by default at the time). If MonitorInUseLists is enabled, then my patch is a noop. MonitorInUseLists makes every java thread do more work when inflating locks. That work is thread local and trying to get it to work in the presence of a deflater thread might not be productive. Re races: 1. To avoid endless inflation / deflation cycles I only attempt to deflate a monitor the second time I see it: If the service thread (the only thread deflating monitors) see a monitor in state New, then mark it as Old and move on. So there is little interaction between a thread inflating a lock to a monitor and the deflating thread (the service thread), the inflating thread just have to make sure the monitor is marked New and this marker is published using appropriate barriers. The code already does the right thing as the displaced mark word has to be correctly published for the hashcode races to be resolved. 2. An interesting race is between a thread acquiring the monitor and the service thread trying to deflate the monitor. The the monitor is free (_owner == NULL), then the service thread installs -1 in _owner to let everyone else know that it will attempt to deflate the monitor. This causes other threads to use the slow path on this monitor, but otherwise general threads do not make a difference between _owner == -1 and _owner == NULL. The -1 marker is used to ensure that no other thread has acquired the monitor while we read _count and _waiters. If the deflater thread manage to install -1 in _owner, read _waiters == 0, and make _count very negative if _count == 0, and _owner is still -1, then _waiters must still be 0 as other threads has to acquire the monitor to increase _waiters, and _count is still negative (I displaced _count by a very large amount). That the is the signal to other threads deflation is in progress. The actual deflation (installing the displaced mark word in java object) is an idempotent operation and both deflater thread and general threads will attempt to complete the deflation. If any of the conditions mentioned above are not true, then the monitor is in use. In that case we restore _count if needed. If the deflating thread managed to install -1 in _owner, but fail to make _count negative, then the next thread writing to _owner (acquiring the lock), erases the -1 installed by the deflater thread. 3. Installing the displaced mark word in the java object might also race with a thread writing the hashcode. This race was pretty simple to sort out as it is very similar to two thread both trying to install a hashcode. Finally, do we really want monitors on the C-heap? We could store monitors in the java heap. It wouldn't be that hard to teach the GCs to check if the mark word is a pointer into the java heap, and potentially copy the monitor object and update the pointer in the mark word. This would avoid treating monitors as roots in the first place. When should monitors be deflated? In such a scheme I propose to deflate monitors when a java thread releases the lock using a simpler approach than outlined in my patch (since the thread owns the monitor is just have to make _count very negative to signal that this monitor is being deflated), the safepoint counter could be used to prevent more than one deflation per safepoint. WDYT? Carsten On Tue, May 16, 2017 at 8:05 AM, Roman Kennke wrote: > Am 16.05.2017 um 13:37 schrieb Carsten Varming: > > FYI. Last year I wrote a small patch[1] against JDK9 that enables monitor > deflation to happen outside a safepoint. There was little interest then, so > I never proposed it as a patch. The JDK9 code has moved on a little bit, so > the patch probably doesn't apply cleanly, but the patch shows one way to > complete the life of a monitor without going through a safepoint. > > > Wow, now that is interesting! > > Can you give me a quick summary-description how you avoid racing with > other threads that might inflate the monitor? As far as I know, monitor > deflation (currently) requires that all Java threads stand still at a > safepoint. > > Can it also be made to work with MonitorInUseLists? > > Roman > > > Carsten > > [1] http://cr.openjdk.java.net/~cvarming/monitor_deflate_conc/0/ > > On Tue, May 16, 2017 at 6:32 AM, Roman Kennke wrote: > >> Am 16.05.2017 um 11:48 schrieb Robbin Ehn: >> > Correction, >> > >> > On 05/16/2017 11:45 AM, Robbin Ehn wrote: >> >> Yes, but since we are at a safepoint you should be able to use gc >> >> worker in that case also? >> > >> > or the java thread. And scratch 'also' :) >> >> This clearly requires co-operation from the GC. >> >> Also, I noticed that some GCs cannot do deflation during root scanning, >> because they stow away mark words during marking, and temporarily store >> forwarding pointers into the mark words, and only restore mark words >> after GC. This obviously conflicts with monitor deflation. Which means >> we need some GC cooperation there too... I will figure something out. >> >> Roman >> >> > > From rkennke at redhat.com Tue May 16 14:38:21 2017 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 16 May 2017 16:38:21 +0200 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> <5db8f6b0-916c-ee46-d4c5-83136ad60c71@oracle.com> <4329774c-aebf-0bcf-3022-060b8dc76028@redhat.com> <1494921280.2770.6.camel@oracle.com> <1494921924.2770.10.camel@oracle.com> <378b058f-e1bb-20e3-cace-282aafeeade4@oracle.com> <57243bb9-8ea1-7649-0157-e38b20a0e37f@redhat.com> <86c15f52-f6c0-e5f6-ee45-48ac1ac744d2@oracle.com> <0db4a13e-dae1-4a81-0c46-c9187990662d@oracle.com> <0faa93a3-4075-9cae-cd72-fbed1d00ab07@redhat.com> Message-ID: <9ad80400-924a-6501-8178-ec22472aca16@redhat.com> Hi Carsten, thanks for the detailed explanations! Regarding MonitorInUseLists, we have seen very considerable improvements in GC processing time when using MonitorInUseLists, I wouldn't really like the idea of having to decide between SP-free-deflation and MonitorInUseLists. However, I also see that it doesn't make much sense to have a deflater thread mess with thread-local monitor allocs. My current idea is basically what I was starting with: to make each JavaThread deflate its own monitors when arriving at or leaving from a safepoint. Doing it on SP arrival has the disadvantage that it potentially stalls everybody else, but has the advantage the the GC would benefit from fewer monitors. Doing it on SP leave is opposite: threads don't have to wait for each other when deflating their stuff, but GC would potentially see more monitors than it strictly needs to. I need to give it some more thought. I also need to give your idea to place monitors on Java heap more thought :-) (You are a crazy man!) Cheers, Roman Am 16.05.2017 um 15:53 schrieb Carsten Varming: > Dear Roman, > > I didn't do anything to extend it to work with MonitorInUseLists (it > wasn't on by default at the time). If MonitorInUseLists is enabled, > then my patch is a noop. MonitorInUseLists makes every java thread do > more work when inflating locks. That work is thread local and trying > to get it to work in the presence of a deflater thread might not be > productive. > > Re races: > > 1. To avoid endless inflation / deflation cycles I only attempt to > deflate a monitor the second time I see it: If the service thread (the > only thread deflating monitors) see a monitor in state New, then mark > it as Old and move on. So there is little interaction between a thread > inflating a lock to a monitor and the deflating thread (the service > thread), the inflating thread just have to make sure the monitor is > marked New and this marker is published using appropriate barriers. > The code already does the right thing as the displaced mark word has > to be correctly published for the hashcode races to be resolved. > > 2. An interesting race is between a thread acquiring the monitor and > the service thread trying to deflate the monitor. The the monitor is > free (_owner == NULL), then the service thread installs -1 in _owner > to let everyone else know that it will attempt to deflate the monitor. > This causes other threads to use the slow path on this monitor, but > otherwise general threads do not make a difference between _owner == > -1 and _owner == NULL. The -1 marker is used to ensure that no other > thread has acquired the monitor while we read _count and _waiters. If > the deflater thread manage to install -1 in _owner, read _waiters == > 0, and make _count very negative if _count == 0, and _owner is still > -1, then _waiters must still be 0 as other threads has to acquire the > monitor to increase _waiters, and _count is still negative (I > displaced _count by a very large amount). That the is the signal to > other threads deflation is in progress. The actual deflation > (installing the displaced mark word in java object) is an idempotent > operation and both deflater thread and general threads will attempt to > complete the deflation. If any of the conditions mentioned above are > not true, then the monitor is in use. In that case we restore _count > if needed. If the deflating thread managed to install -1 in _owner, > but fail to make _count negative, then the next thread writing to > _owner (acquiring the lock), erases the -1 installed by the deflater > thread. > > 3. Installing the displaced mark word in the java object might also > race with a thread writing the hashcode. This race was pretty simple > to sort out as it is very similar to two thread both trying to install > a hashcode. > > Finally, do we really want monitors on the C-heap? We could store > monitors in the java heap. It wouldn't be that hard to teach the GCs > to check if the mark word is a pointer into the java heap, and > potentially copy the monitor object and update the pointer in the mark > word. This would avoid treating monitors as roots in the first place. > When should monitors be deflated? In such a scheme I propose to > deflate monitors when a java thread releases the lock using a simpler > approach than outlined in my patch (since the thread owns the monitor > is just have to make _count very negative to signal that this monitor > is being deflated), the safepoint counter could be used to prevent > more than one deflation per safepoint. WDYT? > > Carsten > > On Tue, May 16, 2017 at 8:05 AM, Roman Kennke > wrote: > > Am 16.05.2017 um 13:37 schrieb Carsten Varming: >> FYI. Last year I wrote a small patch[1] against JDK9 that enables >> monitor deflation to happen outside a safepoint. There was little >> interest then, so I never proposed it as a patch. The JDK9 code >> has moved on a little bit, so the patch probably doesn't apply >> cleanly, but the patch shows one way to complete the life of a >> monitor without going through a safepoint. > > Wow, now that is interesting! > > Can you give me a quick summary-description how you avoid racing > with other threads that might inflate the monitor? As far as I > know, monitor deflation (currently) requires that all Java threads > stand still at a safepoint. > > Can it also be made to work with MonitorInUseLists? > > Roman > >> >> Carsten >> >> [1] http://cr.openjdk.java.net/~cvarming/monitor_deflate_conc/0/ >> >> >> On Tue, May 16, 2017 at 6:32 AM, Roman Kennke > > wrote: >> >> Am 16.05.2017 um 11:48 schrieb Robbin Ehn: >> > Correction, >> > >> > On 05/16/2017 11:45 AM, Robbin Ehn wrote: >> >> Yes, but since we are at a safepoint you should be able to >> use gc >> >> worker in that case also? >> > >> > or the java thread. And scratch 'also' :) >> >> This clearly requires co-operation from the GC. >> >> Also, I noticed that some GCs cannot do deflation during root >> scanning, >> because they stow away mark words during marking, and >> temporarily store >> forwarding pointers into the mark words, and only restore >> mark words >> after GC. This obviously conflicts with monitor deflation. >> Which means >> we need some GC cooperation there too... I will figure >> something out. >> >> Roman >> >> > > From daniel.daugherty at oracle.com Tue May 16 14:46:32 2017 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 16 May 2017 08:46:32 -0600 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: <5db8f6b0-916c-ee46-d4c5-83136ad60c71@oracle.com> References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> <5db8f6b0-916c-ee46-d4c5-83136ad60c71@oracle.com> Message-ID: <4ae4f34c-862d-3f7c-159b-3f0e3965e7f5@oracle.com> On 5/16/17 12:52 AM, David Holmes wrote: > Correction ... > > On 16/05/2017 2:49 PM, David Holmes wrote: >> On 16/05/2017 12:52 AM, Roman Kennke wrote: >>> Am 11.05.2017 um 09:44 schrieb Robbin Ehn: >>>> Hi, >>>> >>>> We have actually been discussing this last few days: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8153224 >>> >>> We are seeing the exact same issue as mentioned in the bug report too. >>> Sometimes, time-to-safepoint is very high, in the area of ~200ms. The >>> reason for this seems to be deflate_idle_monitors(). >>> >>> In the current implementation, the VM thread iterates over all threads, >>> and deflates each thread's idle monitors one after another. >>> >>> I would rather have each Java thread deflate its own idle monitors >>> before arriving at a safepoint. This should be much faster because >>> higher chances a thread has its own monitors in cache, and deflation >>> processing is done in parallel too. This requires some thought... e.g. >>> what to do with non-runnable threads. >> >> This is risky and will need very careful analysis. The >> type-stable-memory property of Monitors depends on very careful > > Monitors don't use TSM - ignore that part. :) Unless something has changed ObjectMonitor is TSM. Once allocated as an ObjectMonitor that memory isn't reused as anything else. So while we may deflate an unused ObjectMonitor, it will get added to the free list. Dan > >> synchronization and the fact that deflation only happens at safepoints. > > This might not be as fragile as I was initially thinking, but a good > design walkthrough of Monitor-lifecycle will be essential for > understanding any subtleties of existing and proposed approaches. > > Thanks, > David > >> David >> ----- >> >>> If you think that's a good way to go, and don't already have a fix in >>> the pipeline, I can take on the bug. I've got synchronizer still hot in >>> memory, and experimented with self-processing Java threads before >>> safepoint a while back too. >>> >>> Let me know! >>> >>> Roman >>> >>> From coleen.phillimore at oracle.com Tue May 16 16:59:14 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 16 May 2017 12:59:14 -0400 Subject: RFR 8180325: Use ClassLoaderData::classes_do for CDS classes In-Reply-To: <5919CE98.8010605@oracle.com> References: <8b59e7b3-2832-98fd-e0a3-8ece93e65023@oracle.com> <5919CE98.8010605@oracle.com> Message-ID: <0a78648f-71bb-da1c-e634-1b6b2f5fb906@oracle.com> Thanks Ioi for the review. On 5/15/17 11:51 AM, Ioi Lam wrote: > HI Coleen, > > The changes look good. > > FYI > > 1998 void InstanceKlass::remove_unshareable_info() { > 1999 Klass::remove_unshareable_info(); > 2000 > 2001 if (is_in_error_state()) { > 2002 // Classes are attempted to link during dumping and may fail, > 2003 // but these classes are still in the dictionary and class list > in CLD. > 2004 // Check in_error state first because in_error is > linked state, so > 2005 // is_linked() is true. > 2006 // If there's a linking error, there is nothing else to remove. > 2007 return; > 2008 } > 2009 > > The above code will be changed to an assert with JDK-8072061 > (Automatically determine optimal sizes for the CDS regions), because > JDK-8072061 will eliminate all classes in error state. You might have to explicitly remove this entry from the CLD as well as the SystemDictionary if this is the case. thanks, Coleen > > Thanks > - Ioi > > On 5/12/17 3:46 PM, coleen.phillimore at oracle.com wrote: >> Summary: Use closures and ClassLoaderData::classes_do instead of >> SystemDictionary::classes_do >> >> Migrate CDS code to ClassLoaderData::loaded_classes_do. The >> loaded_classes_do function should be called without a lock, like the >> rest of the ClassLoaderData::classes_do functions. Also, I didn't >> filter anonymous classes with the CDS functions because at the >> moment, we don't encounter them (there's an assert). If the code >> changes so that we do encounter them, we might want to link them or >> decide what to do with them at that time. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8180325.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8180325 >> >> Tested with nightly rbt run on linux x86, including and additionally >> all the CDS tests locally. >> >> Thanks, >> Coleen >> > From harold.seigel at oracle.com Tue May 16 17:10:48 2017 From: harold.seigel at oracle.com (harold seigel) Date: Tue, 16 May 2017 13:10:48 -0400 Subject: RFR 8152295: Redundant CLCs for classes resolved in both loaders Message-ID: <898fefde-428e-5b4d-f7d1-09d1ff50f8a3@oracle.com> Hi, Please review this JDK-10 change to not record loader constraints if both class loader classes are the same. This change reduces the number of recorded loader constraints when running the Queens program from six to four. It reduces the number of recorded loader constraints when running javac to compile Queens.java from 110 to 55. Additionally, the logging code for LaderConstraintTable::add_entry() was restructured to simplify things and improve readability. Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8152295/webrev/ JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8152295 The fix was tested with JCK Lang and VM tests, the JTreg hotspot, java/io, java/lang, java/util and other tests, the co-located NSK tests, RBT tests, and with JPRT. Thanks, Harold From coleen.phillimore at oracle.com Tue May 16 17:29:35 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 16 May 2017 13:29:35 -0400 Subject: RFR 8152295: Redundant CLCs for classes resolved in both loaders In-Reply-To: <898fefde-428e-5b4d-f7d1-09d1ff50f8a3@oracle.com> References: <898fefde-428e-5b4d-f7d1-09d1ff50f8a3@oracle.com> Message-ID: <25861681-97c4-db06-a03e-aad875bcb816@oracle.com> Hi Harold, this is a nice improvement, expecially not having this failure_code logic. Thanks! Coleen On 5/16/17 1:10 PM, harold seigel wrote: > Hi, > > Please review this JDK-10 change to not record loader constraints if > both class loader classes are the same. This change reduces the > number of recorded loader constraints when running the Queens program > from six to four. It reduces the number of recorded loader > constraints when running javac to compile Queens.java from 110 to 55. > > Additionally, the logging code for LaderConstraintTable::add_entry() > was restructured to simplify things and improve readability. > > Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8152295/webrev/ > > JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8152295 > > The fix was tested with JCK Lang and VM tests, the JTreg hotspot, > java/io, java/lang, java/util and other tests, the co-located NSK > tests, RBT tests, and with JPRT. > > Thanks, Harold > From harold.seigel at oracle.com Tue May 16 17:33:11 2017 From: harold.seigel at oracle.com (harold seigel) Date: Tue, 16 May 2017 13:33:11 -0400 Subject: RFR 8152295: Redundant CLCs for classes resolved in both loaders In-Reply-To: <25861681-97c4-db06-a03e-aad875bcb816@oracle.com> References: <898fefde-428e-5b4d-f7d1-09d1ff50f8a3@oracle.com> <25861681-97c4-db06-a03e-aad875bcb816@oracle.com> Message-ID: Thanks Coleen! Harold On 5/16/2017 1:29 PM, coleen.phillimore at oracle.com wrote: > > Hi Harold, this is a nice improvement, expecially not having this > failure_code logic. > > Thanks! > Coleen > > On 5/16/17 1:10 PM, harold seigel wrote: >> Hi, >> >> Please review this JDK-10 change to not record loader constraints if >> both class loader classes are the same. This change reduces the >> number of recorded loader constraints when running the Queens program >> from six to four. It reduces the number of recorded loader >> constraints when running javac to compile Queens.java from 110 to 55. >> >> Additionally, the logging code for LaderConstraintTable::add_entry() >> was restructured to simplify things and improve readability. >> >> Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8152295/webrev/ >> >> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8152295 >> >> The fix was tested with JCK Lang and VM tests, the JTreg hotspot, >> java/io, java/lang, java/util and other tests, the co-located NSK >> tests, RBT tests, and with JPRT. >> >> Thanks, Harold >> > From george.triantafillou at oracle.com Tue May 16 17:40:59 2017 From: george.triantafillou at oracle.com (George Triantafillou) Date: Tue, 16 May 2017 13:40:59 -0400 Subject: RFR 8152295: Redundant CLCs for classes resolved in both loaders In-Reply-To: <25861681-97c4-db06-a03e-aad875bcb816@oracle.com> References: <898fefde-428e-5b4d-f7d1-09d1ff50f8a3@oracle.com> <25861681-97c4-db06-a03e-aad875bcb816@oracle.com> Message-ID: +1 -George On 5/16/2017 1:29 PM, coleen.phillimore at oracle.com wrote: > > Hi Harold, this is a nice improvement, expecially not having this > failure_code logic. > > Thanks! > Coleen > > On 5/16/17 1:10 PM, harold seigel wrote: >> Hi, >> >> Please review this JDK-10 change to not record loader constraints if >> both class loader classes are the same. This change reduces the >> number of recorded loader constraints when running the Queens program >> from six to four. It reduces the number of recorded loader >> constraints when running javac to compile Queens.java from 110 to 55. >> >> Additionally, the logging code for LaderConstraintTable::add_entry() >> was restructured to simplify things and improve readability. >> >> Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8152295/webrev/ >> >> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8152295 >> >> The fix was tested with JCK Lang and VM tests, the JTreg hotspot, >> java/io, java/lang, java/util and other tests, the co-located NSK >> tests, RBT tests, and with JPRT. >> >> Thanks, Harold >> > From vladimir.x.ivanov at oracle.com Tue May 16 17:46:34 2017 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 16 May 2017 20:46:34 +0300 Subject: RFR 8152295: Redundant CLCs for classes resolved in both loaders In-Reply-To: <898fefde-428e-5b4d-f7d1-09d1ff50f8a3@oracle.com> References: <898fefde-428e-5b4d-f7d1-09d1ff50f8a3@oracle.com> Message-ID: <2c445a81-a4e8-758a-ae6f-2e6a5e8a827f@oracle.com> Looks good. Best regards, Vladimir Ivanov On 5/16/17 8:10 PM, harold seigel wrote: > Hi, > > Please review this JDK-10 change to not record loader constraints if > both class loader classes are the same. This change reduces the number > of recorded loader constraints when running the Queens program from six > to four. It reduces the number of recorded loader constraints when > running javac to compile Queens.java from 110 to 55. > > Additionally, the logging code for LaderConstraintTable::add_entry() was > restructured to simplify things and improve readability. > > Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8152295/webrev/ > > JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8152295 > > The fix was tested with JCK Lang and VM tests, the JTreg hotspot, > java/io, java/lang, java/util and other tests, the co-located NSK tests, > RBT tests, and with JPRT. > > Thanks, Harold > From harold.seigel at oracle.com Tue May 16 17:57:23 2017 From: harold.seigel at oracle.com (harold seigel) Date: Tue, 16 May 2017 13:57:23 -0400 Subject: RFR 8152295: Redundant CLCs for classes resolved in both loaders In-Reply-To: References: <898fefde-428e-5b4d-f7d1-09d1ff50f8a3@oracle.com> <25861681-97c4-db06-a03e-aad875bcb816@oracle.com> Message-ID: <2234d3c2-2d95-5167-c6b4-1c3dd20b5a33@oracle.com> Thanks George! Harold On 5/16/2017 1:40 PM, George Triantafillou wrote: > +1 > > -George > > On 5/16/2017 1:29 PM, coleen.phillimore at oracle.com wrote: >> >> Hi Harold, this is a nice improvement, expecially not having this >> failure_code logic. >> >> Thanks! >> Coleen >> >> On 5/16/17 1:10 PM, harold seigel wrote: >>> Hi, >>> >>> Please review this JDK-10 change to not record loader constraints if >>> both class loader classes are the same. This change reduces the >>> number of recorded loader constraints when running the Queens >>> program from six to four. It reduces the number of recorded loader >>> constraints when running javac to compile Queens.java from 110 to 55. >>> >>> Additionally, the logging code for LaderConstraintTable::add_entry() >>> was restructured to simplify things and improve readability. >>> >>> Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8152295/webrev/ >>> >>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8152295 >>> >>> The fix was tested with JCK Lang and VM tests, the JTreg hotspot, >>> java/io, java/lang, java/util and other tests, the co-located NSK >>> tests, RBT tests, and with JPRT. >>> >>> Thanks, Harold >>> >> > From harold.seigel at oracle.com Tue May 16 17:57:41 2017 From: harold.seigel at oracle.com (harold seigel) Date: Tue, 16 May 2017 13:57:41 -0400 Subject: RFR 8152295: Redundant CLCs for classes resolved in both loaders In-Reply-To: <2c445a81-a4e8-758a-ae6f-2e6a5e8a827f@oracle.com> References: <898fefde-428e-5b4d-f7d1-09d1ff50f8a3@oracle.com> <2c445a81-a4e8-758a-ae6f-2e6a5e8a827f@oracle.com> Message-ID: Thanks Vladimir! Harold On 5/16/2017 1:46 PM, Vladimir Ivanov wrote: > Looks good. > > Best regards, > Vladimir Ivanov > > On 5/16/17 8:10 PM, harold seigel wrote: >> Hi, >> >> Please review this JDK-10 change to not record loader constraints if >> both class loader classes are the same. This change reduces the number >> of recorded loader constraints when running the Queens program from six >> to four. It reduces the number of recorded loader constraints when >> running javac to compile Queens.java from 110 to 55. >> >> Additionally, the logging code for LaderConstraintTable::add_entry() was >> restructured to simplify things and improve readability. >> >> Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8152295/webrev/ >> >> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8152295 >> >> The fix was tested with JCK Lang and VM tests, the JTreg hotspot, >> java/io, java/lang, java/util and other tests, the co-located NSK tests, >> RBT tests, and with JPRT. >> >> Thanks, Harold >> From lois.foltan at oracle.com Tue May 16 17:59:42 2017 From: lois.foltan at oracle.com (Lois Foltan) Date: Tue, 16 May 2017 13:59:42 -0400 Subject: RFR 8152295: Redundant CLCs for classes resolved in both loaders In-Reply-To: <898fefde-428e-5b4d-f7d1-09d1ff50f8a3@oracle.com> References: <898fefde-428e-5b4d-f7d1-09d1ff50f8a3@oracle.com> Message-ID: <363af85f-3d60-11e3-e4d1-4a498e2895e2@oracle.com> Looks good. Minor nit, the two if statements on line #192 could be collapsed into one, something like "if (klass1 == klass2 && klass1 != NULL)" Lois On 5/16/2017 1:10 PM, harold seigel wrote: > Hi, > > Please review this JDK-10 change to not record loader constraints if > both class loader classes are the same. This change reduces the > number of recorded loader constraints when running the Queens program > from six to four. It reduces the number of recorded loader > constraints when running javac to compile Queens.java from 110 to 55. > > Additionally, the logging code for LaderConstraintTable::add_entry() > was restructured to simplify things and improve readability. > > Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8152295/webrev/ > > JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8152295 > > The fix was tested with JCK Lang and VM tests, the JTreg hotspot, > java/io, java/lang, java/util and other tests, the co-located NSK > tests, RBT tests, and with JPRT. > > Thanks, Harold > From mikhailo.seledtsov at oracle.com Tue May 16 18:29:07 2017 From: mikhailo.seledtsov at oracle.com (mikhailo) Date: Tue, 16 May 2017 11:29:07 -0700 Subject: RFR(XS): JDK-8180393: [TESTBUG] CDSTestUtils property test.cds.copy.child.stdout should be true by default Message-ID: Please review this trivial change; see bug description for rational and details. JBS: https://bugs.openjdk.java.net/browse/JDK-8180393 Webrev: http://cr.openjdk.java.net/~mseledtsov/8180393.00/ Testing: 1. Locally: exercised all CDS tests locally on Linux-x64 Checked several jtr files to make sure that stdout of child process is copied into the jtr file PASS 2. Automated: CDS tests on multiple platforms via automated test system In progress Thank you, Misha From harold.seigel at oracle.com Tue May 16 18:50:00 2017 From: harold.seigel at oracle.com (harold seigel) Date: Tue, 16 May 2017 14:50:00 -0400 Subject: RFR(XS): JDK-8180393: [TESTBUG] CDSTestUtils property test.cds.copy.child.stdout should be true by default In-Reply-To: References: Message-ID: Looks good. Thanks, Harold On 5/16/2017 2:29 PM, mikhailo wrote: > Please review this trivial change; see bug description for rational > and details. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8180393 > Webrev: http://cr.openjdk.java.net/~mseledtsov/8180393.00/ > Testing: > 1. Locally: exercised all CDS tests locally on Linux-x64 > > Checked several jtr files to make sure that stdout of > child process is copied into the jtr file > > PASS > > 2. Automated: CDS tests on multiple platforms via automated > test system > > In progress > > > Thank you, > Misha > From ioi.lam at oracle.com Tue May 16 19:17:18 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 16 May 2017 12:17:18 -0700 Subject: RFR(XS): JDK-8180393: [TESTBUG] CDSTestUtils property test.cds.copy.child.stdout should be true by default In-Reply-To: References: Message-ID: <591B503E.1020503@oracle.com> Looks good to me, too. Thanks! - Ioi On 5/16/17 11:50 AM, harold seigel wrote: > Looks good. > > Thanks, Harold > > > On 5/16/2017 2:29 PM, mikhailo wrote: >> Please review this trivial change; see bug description for rational >> and details. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8180393 >> Webrev: http://cr.openjdk.java.net/~mseledtsov/8180393.00/ >> Testing: >> 1. Locally: exercised all CDS tests locally on Linux-x64 >> >> Checked several jtr files to make sure that stdout of >> child process is copied into the jtr file >> >> PASS >> >> 2. Automated: CDS tests on multiple platforms via automated >> test system >> >> In progress >> >> >> Thank you, >> Misha >> > From harold.seigel at oracle.com Tue May 16 19:25:52 2017 From: harold.seigel at oracle.com (harold seigel) Date: Tue, 16 May 2017 15:25:52 -0400 Subject: RFR 8152295: Redundant CLCs for classes resolved in both loaders In-Reply-To: <363af85f-3d60-11e3-e4d1-4a498e2895e2@oracle.com> References: <898fefde-428e-5b4d-f7d1-09d1ff50f8a3@oracle.com> <363af85f-3d60-11e3-e4d1-4a498e2895e2@oracle.com> Message-ID: <7bb767d3-b115-84a7-73a8-14b971a3d911@oracle.com> Thanks Lois for the review! Harold On 5/16/2017 1:59 PM, Lois Foltan wrote: > Looks good. Minor nit, the two if statements on line #192 could be > collapsed into one, something like "if (klass1 == klass2 && klass1 != > NULL)" > Lois > > On 5/16/2017 1:10 PM, harold seigel wrote: >> Hi, >> >> Please review this JDK-10 change to not record loader constraints if >> both class loader classes are the same. This change reduces the >> number of recorded loader constraints when running the Queens program >> from six to four. It reduces the number of recorded loader >> constraints when running javac to compile Queens.java from 110 to 55. >> >> Additionally, the logging code for LaderConstraintTable::add_entry() >> was restructured to simplify things and improve readability. >> >> Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8152295/webrev/ >> >> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8152295 >> >> The fix was tested with JCK Lang and VM tests, the JTreg hotspot, >> java/io, java/lang, java/util and other tests, the co-located NSK >> tests, RBT tests, and with JPRT. >> >> Thanks, Harold >> > From jiangli.zhou at oracle.com Tue May 16 20:27:42 2017 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Tue, 16 May 2017 13:27:42 -0700 Subject: RFR 8180325: Use ClassLoaderData::classes_do for CDS classes In-Reply-To: <8b59e7b3-2832-98fd-e0a3-8ece93e65023@oracle.com> References: <8b59e7b3-2832-98fd-e0a3-8ece93e65023@oracle.com> Message-ID: <5FAFF6B3-079F-487A-BCF0-97223ED15B7E@oracle.com> Looks good. Thanks, Jiangli > On May 12, 2017, at 3:46 PM, coleen.phillimore at oracle.com wrote: > > Summary: Use closures and ClassLoaderData::classes_do instead of SystemDictionary::classes_do > > Migrate CDS code to ClassLoaderData::loaded_classes_do. The loaded_classes_do function should be called without a lock, like the rest of the ClassLoaderData::classes_do functions. Also, I didn't filter anonymous classes with the CDS functions because at the moment, we don't encounter them (there's an assert). If the code changes so that we do encounter them, we might want to link them or decide what to do with them at that time. > > open webrev at http://cr.openjdk.java.net/~coleenp/8180325.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8180325 > > Tested with nightly rbt run on linux x86, including and additionally all the CDS tests locally. > > Thanks, > Coleen > From coleen.phillimore at oracle.com Tue May 16 20:47:02 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 16 May 2017 16:47:02 -0400 Subject: RFR 8180325: Use ClassLoaderData::classes_do for CDS classes In-Reply-To: <5FAFF6B3-079F-487A-BCF0-97223ED15B7E@oracle.com> References: <8b59e7b3-2832-98fd-e0a3-8ece93e65023@oracle.com> <5FAFF6B3-079F-487A-BCF0-97223ED15B7E@oracle.com> Message-ID: <9483ef63-48e3-48f3-8f45-fc264ff049f8@oracle.com> Thank you Jiangli, and thanks for the preliminary review also. Coleen On 5/16/17 4:27 PM, Jiangli Zhou wrote: > Looks good. > > Thanks, > Jiangli > >> On May 12, 2017, at 3:46 PM, coleen.phillimore at oracle.com wrote: >> >> Summary: Use closures and ClassLoaderData::classes_do instead of SystemDictionary::classes_do >> >> Migrate CDS code to ClassLoaderData::loaded_classes_do. The loaded_classes_do function should be called without a lock, like the rest of the ClassLoaderData::classes_do functions. Also, I didn't filter anonymous classes with the CDS functions because at the moment, we don't encounter them (there's an assert). If the code changes so that we do encounter them, we might want to link them or decide what to do with them at that time. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8180325.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8180325 >> >> Tested with nightly rbt run on linux x86, including and additionally all the CDS tests locally. >> >> Thanks, >> Coleen >> From david.holmes at oracle.com Wed May 17 02:53:08 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 17 May 2017 12:53:08 +1000 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: <9ad80400-924a-6501-8178-ec22472aca16@redhat.com> References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> <5db8f6b0-916c-ee46-d4c5-83136ad60c71@oracle.com> <4329774c-aebf-0bcf-3022-060b8dc76028@redhat.com> <1494921280.2770.6.camel@oracle.com> <1494921924.2770.10.camel@oracle.com> <378b058f-e1bb-20e3-cace-282aafeeade4@oracle.com> <57243bb9-8ea1-7649-0157-e38b20a0e37f@redhat.com> <86c15f52-f6c0-e5f6-ee45-48ac1ac744d2@oracle.com> <0db4a13e-dae1-4a81-0c46-c9187990662d@oracle.com> <0faa93a3-4075-9cae-cd72-fbed1d00ab07@redhat.com> <9ad80400-924a-6501-8178-ec22472aca16@redhat.com> Message-ID: This seems to be diverging and expanding in scope somewhat. Any changes to the monitor lifecycle requires very careful consideration. Moving monitors to the Java heap could have wide spreading implications and cause quite severe perturbations in heap usage patterns. You would need very extensive testing of numerous real world apps to ensure this change is actually safe and performant. To me we are now well into JEP territory, not just tweaking existing policies. Such experiments should be trialed in a sandbox repo before being proposed to the mainline. Cheers, David On 17/05/2017 12:38 AM, Roman Kennke wrote: > Hi Carsten, > > thanks for the detailed explanations! > > Regarding MonitorInUseLists, we have seen very considerable improvements > in GC processing time when using MonitorInUseLists, I wouldn't really > like the idea of having to decide between SP-free-deflation and > MonitorInUseLists. However, I also see that it doesn't make much sense > to have a deflater thread mess with thread-local monitor allocs. My > current idea is basically what I was starting with: to make each > JavaThread deflate its own monitors when arriving at or leaving from a > safepoint. Doing it on SP arrival has the disadvantage that it > potentially stalls everybody else, but has the advantage the the GC > would benefit from fewer monitors. Doing it on SP leave is opposite: > threads don't have to wait for each other when deflating their stuff, > but GC would potentially see more monitors than it strictly needs to. I > need to give it some more thought. > > I also need to give your idea to place monitors on Java heap more > thought :-) (You are a crazy man!) > > Cheers, > Roman > > Am 16.05.2017 um 15:53 schrieb Carsten Varming: >> Dear Roman, >> >> I didn't do anything to extend it to work with MonitorInUseLists (it >> wasn't on by default at the time). If MonitorInUseLists is enabled, >> then my patch is a noop. MonitorInUseLists makes every java thread do >> more work when inflating locks. That work is thread local and trying >> to get it to work in the presence of a deflater thread might not be >> productive. >> >> Re races: >> >> 1. To avoid endless inflation / deflation cycles I only attempt to >> deflate a monitor the second time I see it: If the service thread (the >> only thread deflating monitors) see a monitor in state New, then mark >> it as Old and move on. So there is little interaction between a thread >> inflating a lock to a monitor and the deflating thread (the service >> thread), the inflating thread just have to make sure the monitor is >> marked New and this marker is published using appropriate barriers. >> The code already does the right thing as the displaced mark word has >> to be correctly published for the hashcode races to be resolved. >> >> 2. An interesting race is between a thread acquiring the monitor and >> the service thread trying to deflate the monitor. The the monitor is >> free (_owner == NULL), then the service thread installs -1 in _owner >> to let everyone else know that it will attempt to deflate the monitor. >> This causes other threads to use the slow path on this monitor, but >> otherwise general threads do not make a difference between _owner == >> -1 and _owner == NULL. The -1 marker is used to ensure that no other >> thread has acquired the monitor while we read _count and _waiters. If >> the deflater thread manage to install -1 in _owner, read _waiters == >> 0, and make _count very negative if _count == 0, and _owner is still >> -1, then _waiters must still be 0 as other threads has to acquire the >> monitor to increase _waiters, and _count is still negative (I >> displaced _count by a very large amount). That the is the signal to >> other threads deflation is in progress. The actual deflation >> (installing the displaced mark word in java object) is an idempotent >> operation and both deflater thread and general threads will attempt to >> complete the deflation. If any of the conditions mentioned above are >> not true, then the monitor is in use. In that case we restore _count >> if needed. If the deflating thread managed to install -1 in _owner, >> but fail to make _count negative, then the next thread writing to >> _owner (acquiring the lock), erases the -1 installed by the deflater >> thread. >> >> 3. Installing the displaced mark word in the java object might also >> race with a thread writing the hashcode. This race was pretty simple >> to sort out as it is very similar to two thread both trying to install >> a hashcode. >> >> Finally, do we really want monitors on the C-heap? We could store >> monitors in the java heap. It wouldn't be that hard to teach the GCs >> to check if the mark word is a pointer into the java heap, and >> potentially copy the monitor object and update the pointer in the mark >> word. This would avoid treating monitors as roots in the first place. >> When should monitors be deflated? In such a scheme I propose to >> deflate monitors when a java thread releases the lock using a simpler >> approach than outlined in my patch (since the thread owns the monitor >> is just have to make _count very negative to signal that this monitor >> is being deflated), the safepoint counter could be used to prevent >> more than one deflation per safepoint. WDYT? >> >> Carsten >> >> On Tue, May 16, 2017 at 8:05 AM, Roman Kennke > > wrote: >> >> Am 16.05.2017 um 13:37 schrieb Carsten Varming: >>> FYI. Last year I wrote a small patch[1] against JDK9 that enables >>> monitor deflation to happen outside a safepoint. There was little >>> interest then, so I never proposed it as a patch. The JDK9 code >>> has moved on a little bit, so the patch probably doesn't apply >>> cleanly, but the patch shows one way to complete the life of a >>> monitor without going through a safepoint. >> >> Wow, now that is interesting! >> >> Can you give me a quick summary-description how you avoid racing >> with other threads that might inflate the monitor? As far as I >> know, monitor deflation (currently) requires that all Java threads >> stand still at a safepoint. >> >> Can it also be made to work with MonitorInUseLists? >> >> Roman >> >>> >>> Carsten >>> >>> [1] http://cr.openjdk.java.net/~cvarming/monitor_deflate_conc/0/ >>> >>> >>> On Tue, May 16, 2017 at 6:32 AM, Roman Kennke >> > wrote: >>> >>> Am 16.05.2017 um 11:48 schrieb Robbin Ehn: >>> > Correction, >>> > >>> > On 05/16/2017 11:45 AM, Robbin Ehn wrote: >>> >> Yes, but since we are at a safepoint you should be able to >>> use gc >>> >> worker in that case also? >>> > >>> > or the java thread. And scratch 'also' :) >>> >>> This clearly requires co-operation from the GC. >>> >>> Also, I noticed that some GCs cannot do deflation during root >>> scanning, >>> because they stow away mark words during marking, and >>> temporarily store >>> forwarding pointers into the mark words, and only restore >>> mark words >>> after GC. This obviously conflicts with monitor deflation. >>> Which means >>> we need some GC cooperation there too... I will figure >>> something out. >>> >>> Roman >>> >>> >> >> > From david.holmes at oracle.com Wed May 17 02:56:35 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 17 May 2017 12:56:35 +1000 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: <1494921280.2770.6.camel@oracle.com> References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> <5db8f6b0-916c-ee46-d4c5-83136ad60c71@oracle.com> <4329774c-aebf-0bcf-3022-060b8dc76028@redhat.com> <1494921280.2770.6.camel@oracle.com> Message-ID: Hi Thomas, > is there a problem to just activate this by default and avoid adding > an (experimental) option? Yes! Whether you add an experimental flag to turn this on, or make it the default and add a product flag to run it off, it will need to be controllable. You need a lot of bake time for these kinds of changes and if they break things in the wild there must be a way for customers to go back to the old behaviour. And of course it is the only reasonable way to do side-by-side performance comparisons. Cheers, David ----- On 16/05/2017 5:54 PM, Thomas Schatzl wrote: > Hi, > > On Tue, 2017-05-16 at 09:43 +0200, Roman Kennke wrote: >> Am 16.05.2017 um 08:52 schrieb David Holmes: >>> >>> Correction ... >>> >>> On 16/05/2017 2:49 PM, David Holmes wrote: >>>> >>>> On 16/05/2017 12:52 AM, Roman Kennke wrote: >>>>> >>>>> Am 11.05.2017 um 09:44 schrieb Robbin Ehn: > [...] >>>> >>>> synchronization and the fact that deflation only happens at >>>> safepoints. >>> This might not be as fragile as I was initially thinking, but a >>> good design walkthrough of Monitor-lifecycle will be essential for >>> understanding any subtleties of existing and proposed approaches. >> I realized that monitor deflation depends strictly on all Java >> threads having arrived at safepoint. If a thread starts deflating its >> monitors while other threads are still in the process of getting to a >> safepoint, those other threads could mess with the monitors that the >> first threads is attempting to deflate. This is a no-go. >> >> However, I devised a scheme that works well: instead of letting the >> VM thread deflate all idle monitors after all threads arrived, and >> doing so single-threaded, I am now letting GC worker threads deflate >> idle monitors right before scanning/processing the synchronizer >> roots. E.g.: >> >> http://cr.openjdk.java.net/~rkennke/deflate-per-thread/webrev.01/ >> >> I can probably collapse monitor deflation and iteration into one pass >> too. > > is there a problem to just activate this by default and avoid adding > an (experimental) option? Even experimental options are part of the > public API (as they can be accessed in the product) and need a CSR > request to add and need to go through the usual long-winded procedure > to remove later (after we find out after 10 years that nobody ever > actually uses that one). > > I mean, this change seems to be a win in all cases (or at most be as > slow as before), so why allow the user to mess with it? > > Thanks, > Thomas > From david.holmes at oracle.com Wed May 17 02:57:29 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 17 May 2017 12:57:29 +1000 Subject: ObjectSynchronizer iterate only in-use monitors? In-Reply-To: <4ae4f34c-862d-3f7c-159b-3f0e3965e7f5@oracle.com> References: <46cbebff-4c90-d2b6-7daf-3fc52bd865c9@oracle.com> <5db8f6b0-916c-ee46-d4c5-83136ad60c71@oracle.com> <4ae4f34c-862d-3f7c-159b-3f0e3965e7f5@oracle.com> Message-ID: <53cf06cc-a615-3dd3-5e82-e69086035196@oracle.com> Thanks Dan - my bad again. David On 17/05/2017 12:46 AM, Daniel D. Daugherty wrote: > On 5/16/17 12:52 AM, David Holmes wrote: >> Correction ... >> >> On 16/05/2017 2:49 PM, David Holmes wrote: >>> On 16/05/2017 12:52 AM, Roman Kennke wrote: >>>> Am 11.05.2017 um 09:44 schrieb Robbin Ehn: >>>>> Hi, >>>>> >>>>> We have actually been discussing this last few days: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8153224 >>>> >>>> We are seeing the exact same issue as mentioned in the bug report too. >>>> Sometimes, time-to-safepoint is very high, in the area of ~200ms. The >>>> reason for this seems to be deflate_idle_monitors(). >>>> >>>> In the current implementation, the VM thread iterates over all threads, >>>> and deflates each thread's idle monitors one after another. >>>> >>>> I would rather have each Java thread deflate its own idle monitors >>>> before arriving at a safepoint. This should be much faster because >>>> higher chances a thread has its own monitors in cache, and deflation >>>> processing is done in parallel too. This requires some thought... e.g. >>>> what to do with non-runnable threads. >>> >>> This is risky and will need very careful analysis. The >>> type-stable-memory property of Monitors depends on very careful >> >> Monitors don't use TSM - ignore that part. :) > > Unless something has changed ObjectMonitor is TSM. Once allocated as an > ObjectMonitor that memory isn't reused as anything else. So while we > may deflate an unused ObjectMonitor, it will get added to the free list. > > Dan > > >> >>> synchronization and the fact that deflation only happens at safepoints. >> >> This might not be as fragile as I was initially thinking, but a good >> design walkthrough of Monitor-lifecycle will be essential for >> understanding any subtleties of existing and proposed approaches. >> >> Thanks, >> David >> >>> David >>> ----- >>> >>>> If you think that's a good way to go, and don't already have a fix in >>>> the pipeline, I can take on the bug. I've got synchronizer still hot in >>>> memory, and experimented with self-processing Java threads before >>>> safepoint a while back too. >>>> >>>> Let me know! >>>> >>>> Roman >>>> >>>> > From david.holmes at oracle.com Wed May 17 03:48:27 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 17 May 2017 13:48:27 +1000 Subject: RFR 8178604: JVM does not allow defining boot loader modules in exploded build after module system initialization In-Reply-To: <4550b8c7-9bff-a44d-fefb-b5b589da371f@oracle.com> References: <85e36988-a145-6805-8659-56c738013f58@oracle.com> <46a812ca-228d-01e7-e889-fd22f2af0537@oracle.com> <11709d4a-0a41-1d38-fcd1-67d42c33eaeb@oracle.com> <3ee613ff-f6da-9ef8-6781-efce4edba29e@oracle.com> <21b3fdc5-d8a7-0535-3ac6-f1a097a7431d@oracle.com> <388b80ed-e436-f888-ffed-5b92e0117ce2@oracle.com> <2e1d1c53-8d6d-e606-f047-a223f76b63a9@oracle.com> <1b618a5c-671d-9063-326e-16991efab441@oracle.com> <4550b8c7-9bff-a44d-fefb-b5b589da371f@oracle.com> Message-ID: <3082901f-a299-c517-1b7e-2cc80aa52b3e@oracle.com> Hi Harold, On 16/05/2017 11:36 PM, harold seigel wrote: > Hi David, > > Thanks for the review! I'll change the initialization of > _exploded_entries so that it is done eagerly, before I push the change. ??? If you do that then it needs to be re-reviewed! David > Harold > > > On 5/10/2017 10:11 PM, David Holmes wrote: >> Hi Harold, >> >> On 11/05/2017 7:18 AM, harold seigel wrote: >>> Hi Lois, David, >>> >>> Please review this latest webrev that initializes _exploded_entries >>> without locking: >>> >>> http://cr.openjdk.java.net/~hseigel/bug_8178604.4/webrev/index.html >> >> Ok. >> >> It would be a lot clearer, in my opinion, that the initialization is >> indeed thread-safe, if it were done eagerly instead of lazily. The >> execution contexts of the first and second calls (in separate threads) >> to that chunk of code are not readily discernible by any reasonable >> means. But the extensive comment is appreciated. >> >> Thanks, >> David >> >>> Thanks, Harold >>> >>> >>> On 5/10/2017 1:19 PM, Lois Foltan wrote: >>>> On 5/9/2017 5:18 PM, David Holmes wrote: >>>>> On 9/05/2017 11:14 PM, harold seigel wrote: >>>>>> Hi David, >>>>>> >>>>>> See comments embedded below. >>>>>> >>>>>> Thanks, Harold >>>>>> >>>>>> >>>>>> On 5/8/2017 8:52 PM, David Holmes wrote: >>>>>>> Hi Harold, >>>>>>> >>>>>>> On 9/05/2017 4:00 AM, harold seigel wrote: >>>>>>>> Hi David, >>>>>>>> >>>>>>>> Please see this updated webrev: >>>>>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604.3/webrev/index.html >>>>>>>> >>>>>>>> It contains the changes you requested below. >>>>>>> >>>>>>> Thank you. I will point out that Lois originally suggested the same >>>>>>> changes. >>>>>>> >>>>>>>> Note that the _exploded_entries table gets created very early >>>>>>>> during >>>>>>>> SystemDictionary initialization, before initializing the pre-loaded >>>>>>>> classes. So, the code in load_class() at line ~1503 can assume >>>>>>>> that >>>>>>>> _exploded_entries is not null. >>>>>>> >>>>>>> Which thread executes ClassLoader::add_to_exploded_build_list, and >>>>>>> which thread executes load_class() first? If they are guaranteed >>>>>>> to be >>>>>>> the same thread then you don't need locking on the construction. If >>>>>>> they can be different threads then there must be some >>>>>>> synchronization >>>>>>> between them that ensures _exploded_entries is properly visible >>>>>>> after >>>>>>> construction. >>>>>> They are the same thread. At startup, the thread calls >>>>> >>>>> Then as I said, why do we need any locking? I note Lois is the one >>>>> that requested this and you questioned it. I also question it. >>>> >>>> Thank you Harold for investigating this. I stand corrected, it seems >>>> that we do not need the lock after all when allocating >>>> ClassLoader::_exploded_entries. >>>> Lois >>>> >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>>> SystemDictionary::initialize_preloaded_classes(), which calls >>>>>> ClassLoader::classLoader_init2(), which calls >>>>>> add_to_exploded_build_list(). >>>>>> >>>>>> Control then returns back to >>>>>> SystemDictionary::initialize_preloaded_classes() which eventually >>>>>> calls >>>>>> SystemDictionary::load_instance_class(), which calls >>>>>> ClassLoader::load_class(). Here's the call stack showing these >>>>>> calls: >>>>>> >>>>>> V [libjvm.so+0x974c0c] ClassLoader::load_class(Symbol*, bool, >>>>>> Thread*)+0x41c >>>>>> V [libjvm.so+0x1648339] >>>>>> SystemDictionary::load_instance_class(Symbol*, Handle, >>>>>> Thread*)+0x819 >>>>>> V [libjvm.so+0x1648e25] >>>>>> SystemDictionary::resolve_instance_class_or_null(Symbol*, Handle, >>>>>> Handle, Thread*)+0x985 >>>>>> V [libjvm.so+0x16494ba] >>>>>> SystemDictionary::resolve_or_null(Symbol*, >>>>>> Handle, Handle, Thread*)+0x5a >>>>>> V [libjvm.so+0x16498fe] >>>>>> SystemDictionary::resolve_or_fail(Symbol*, >>>>>> Handle, Handle, bool, Thread*)+0x1e >>>>>> V [libjvm.so+0x1649aa9] >>>>>> SystemDictionary::initialize_wk_klass(SystemDictionary::WKID, int, >>>>>> Thread*)+0x149 >>>>>> V [libjvm.so+0x1649bfe] >>>>>> SystemDictionary::initialize_wk_klasses_until(SystemDictionary::WKID, >>>>>> SystemDictionary::WKID&, >>>>>> >>>>>> Thread*)+0x5e >>>>>> V [libjvm.so+0x1649dba] >>>>>> SystemDictionary::*initialize_preloaded_classes*(Thread*)+0xea >>>>>> V [libjvm.so+0x164a28a] >>>>>> SystemDictionary::initialize(Thread*)+0x26a >>>>>> V [libjvm.so+0x16cd7e0] Universe::genesis(Thread*)+0x7c0 >>>>>> V [libjvm.so+0x16ce4cc] universe2_init()+0x2c >>>>>> V [libjvm.so+0xe0f268] init_globals()+0xa8 >>>>>> V [libjvm.so+0x1696e6f] Threads::create_vm(JavaVMInitArgs*, >>>>>> bool*)+0x2df >>>>>> ... >>>>>> >>>>>> So, is the existing synchronization in this change sufficient? >>>>>> >>>>>> Thanks, Harold >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>>> Thanks, Harold >>>>>>>> >>>>>>>> On 5/4/2017 5:32 PM, David Holmes wrote: >>>>>>>>> Hi Harold, >>>>>>>>> >>>>>>>>> On 5/05/2017 3:47 AM, harold seigel wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> Please review this updated webrev: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604.2/webrev/index.html >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> The updated webrev takes out a lock when creating >>>>>>>>>> _exploded_entries. It >>>>>>>>>> also contains additional comments and an assert() to address >>>>>>>>>> Lois's >>>>>>>>>> concerns with reduced code readability involving what situations >>>>>>>>>> warrant >>>>>>>>>> taking out a lock in ClassLoader::search_module_entries(). >>>>>>>>>> >>>>>>>>>> I don't think there needs to be two different initial entry >>>>>>>>>> points as >>>>>>>>>> suggested by David below. The method behaves slightly >>>>>>>>>> differently >>>>>>>>>> depending on the value of its 'need_lock' parameter. But, >>>>>>>>>> there is >>>>>>>>>> enough common code for it to remain one method. >>>>>>>>> >>>>>>>>> I think this highlights uncertainty about the concurrent >>>>>>>>> behaviour in >>>>>>>>> relation to this code. On the one hand you are worried about an >>>>>>>>> initialization race and so use the lock, but when you do: >>>>>>>>> >>>>>>>>> 1500 // The exploded build entries can be added to at any >>>>>>>>> time >>>>>>>>> so pass 'true' for >>>>>>>>> 1501 // need_lock so that a lock is taken out when searching >>>>>>>>> them. >>>>>>>>> 1502 assert(_exploded_entries != NULL, "No exploded build >>>>>>>>> entries present"); >>>>>>>>> 1503 stream = search_module_entries(_exploded_entries, >>>>>>>>> class_name, file_name, true, CHECK_NULL); >>>>>>>>> >>>>>>>>> you load _exploded_entries with no lock held and so must be >>>>>>>>> certain >>>>>>>>> that you can not possibly read a null value here. >>>>>>>>> >>>>>>>>> The needs_lock parameter seems superfluous - you know the >>>>>>>>> condition >>>>>>>>> for locking is that the list is the _exploded_entries, and you >>>>>>>>> assert >>>>>>>>> that. So no need to pass in needs_lock, just grab the lock when >>>>>>>>> dealing with _exploded_entries. >>>>>>>>> >>>>>>>>> Cheers, >>>>>>>>> David >>>>>>>>> >>>>>>>>>> Thanks, Harold >>>>>>>>>> >>>>>>>>>> On 5/3/2017 12:59 AM, David Holmes wrote: >>>>>>>>>>> On 3/05/2017 7:11 AM, Lois Foltan wrote: >>>>>>>>>>>> On 5/2/2017 9:03 AM, harold seigel wrote: >>>>>>>>>>>>> Hi Lois, >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks for your comments. Please see my in-line responses. >>>>>>>>>>>>> >>>>>>>>>>>>> Harold >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 5/1/2017 4:14 PM, Lois Foltan wrote: >>>>>>>>>>>>>> Hi Harold, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Looks good. A couple of comments: >>>>>>>>>>>>>> >>>>>>>>>>>>>> src/share/vm/classfile/classLoader.cpp >>>>>>>>>>>>>> line #828 - please take out the Module_lock when the >>>>>>>>>>>>>> _exploded_entries is created >>>>>>>>>>>>> This fix does not change the possible need for >>>>>>>>>>>>> synchronization when >>>>>>>>>>>>> _exploded_entries gets created during module system >>>>>>>>>>>>> initialization. >>>>>>>>>>>>> Are you saying that the existing code is wrong and should have >>>>>>>>>>>>> taken >>>>>>>>>>>>> out the Module_lock before creating _exploding_entries? >>>>>>>>>>>> >>>>>>>>>>>> The method ClassLoader::add_to_exploded_build_list can be >>>>>>>>>>>> called at >>>>>>>>>>>> anytime (during module system initialization and post module >>>>>>>>>>>> system >>>>>>>>>>>> initialization). Thus it seems prudent that no matter what the >>>>>>>>>>>> current >>>>>>>>>>>> JVM phase, that any access to >>>>>>>>>>>> ClassLoader::_exploding_entries be >>>>>>>>>>>> protected via a lock. That includes creation as well as >>>>>>>>>>>> insertion >>>>>>>>>>>> into >>>>>>>>>>>> the list. >>>>>>>>>>> >>>>>>>>>>> If creation is not guaranteed to occur before other threads >>>>>>>>>>> that will >>>>>>>>>>> call the same method and use the _exploded_entries list, then >>>>>>>>>>> synchronization of some form is essential. If it is guaranteed >>>>>>>>>>> then we >>>>>>>>>>> don't need create time sync just to be prudent - but the >>>>>>>>>>> guarantee >>>>>>>>>>> must be firmly established and clearly documented. >>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> line #1402 - I like the new factored out method >>>>>>>>>>>>>> find_first_module_cpe(), however, instead of adding a new >>>>>>>>>>>>>> "need_lock" >>>>>>>>>>>>>> parameter, I would rather see code >>>>>>>>>>>>>> in find_first_module_cpe() that >>>>>>>>>>>>>> takes the >>>>>>>>>>>>>> Module_lock if the module_list parameter equals >>>>>>>>>>>>>> ClassLoader::_exploded_entries >>>>>>>>>>>>> I think your suggestion makes the code less flexible and does >>>>>>>>>>>>> not >>>>>>>>>>>>> require potential future callers of search_module_entries() to >>>>>>>>>>>>> think >>>>>>>>>>>>> about synchronization. So, I'd prefer to leave that change >>>>>>>>>>>>> as is. >>>>>>>>>>>> >>>>>>>>>>>> I see your point. My point was based on reduced code >>>>>>>>>>>> readability, the >>>>>>>>>>>> further away from the decision to establish the lock, the less >>>>>>>>>>>> apparent >>>>>>>>>>>> it is within the new method >>>>>>>>>>>> ClassLoader::find_first_module_cpe() >>>>>>>>>>>> as to >>>>>>>>>>>> what situations warranted the lock in the first place. >>>>>>>>>>> >>>>>>>>>>> Further, this: >>>>>>>>>>> >>>>>>>>>>> 1495 stream = search_module_entries(_exploded_entries, >>>>>>>>>>> class_name, file_name, true, CHECK_NULL); >>>>>>>>>>> >>>>>>>>>>> is potentially unsafe if we can race with creation because we >>>>>>>>>>> first >>>>>>>>>>> access _exploded_entries without holding any lock! And the fact >>>>>>>>>>> the >>>>>>>>>>> method needs to do different things depending on which "list" >>>>>>>>>>> was >>>>>>>>>>> passed in suggests to me it should be two different initial >>>>>>>>>>> entry >>>>>>>>>>> points. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> David >>>>>>>>>>> ----- >>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Lois >>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Lois >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 5/1/2017 3:36 PM, harold seigel wrote: >>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Please review this JDK-10 bug fix to allow defining of >>>>>>>>>>>>>>> modules to >>>>>>>>>>>>>>> the boot loader in exploded builds after module system >>>>>>>>>>>>>>> initialization. The fix uses the module lock to synchronize >>>>>>>>>>>>>>> access >>>>>>>>>>>>>>> to the _exploded_entries data structure (which is only >>>>>>>>>>>>>>> used by >>>>>>>>>>>>>>> exploded builds). >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Note that the above capability already exists for regular >>>>>>>>>>>>>>> builds. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Open Webrev: >>>>>>>>>>>>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604/webrev/index.html >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8178604 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The fix was tested with JCK tests, the JTreg hotspot, >>>>>>>>>>>>>>> java/io, >>>>>>>>>>>>>>> java/lang, java/util and other tests, the RBT tier2 -tier5 >>>>>>>>>>>>>>> tests, >>>>>>>>>>>>>>> the co-located NSK tests, and with JPRT. The JTReg and JCK >>>>>>>>>>>>>>> tests >>>>>>>>>>>>>>> were run with an exploded build. Also, the example >>>>>>>>>>>>>>> program in >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>> JBS bug was run by hand to verify the fix. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, Harold >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>> >>>> >>> > From serguei.spitsyn at oracle.com Wed May 17 04:21:16 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 16 May 2017 21:21:16 -0700 Subject: RFR 8166848: Performance bug: SystemDictionary - optimization In-Reply-To: <1cc33d02-6fca-7f67-34ff-0fcc8c309cad@oracle.com> References: <249af7dc-a3a4-52b4-ca6d-ffdb7ec13764@oracle.com> <1cc33d02-6fca-7f67-34ff-0fcc8c309cad@oracle.com> Message-ID: Hi Coleen, It looks good to me. Nice simplification! One tip is to use the bucket_count for incrementing the element_count: element_count += bucket_count; out of the internal for-loop. It is to underline the relationship between the two counters. Thanks, Serguei On 5/15/17 05:30, coleen.phillimore at oracle.com wrote: > >> Summary: Check instead that a bucket isn't 10x the average >> >> See bug and linked bugs for more details. >> >> Tested with RBT nightly tests (tier2-5). >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8166848.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8166848 >> >> Thanks, >> Coleen > From serguei.spitsyn at oracle.com Wed May 17 05:12:59 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 16 May 2017 22:12:59 -0700 Subject: RFR(XS): JDK-8180393: [TESTBUG] CDSTestUtils property test.cds.copy.child.stdout should be true by default In-Reply-To: References: Message-ID: Hi Misha, Looks good to me. Thanks, Serguei On 5/16/17 11:29, mikhailo wrote: > Please review this trivial change; see bug description for rational > and details. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8180393 > Webrev: http://cr.openjdk.java.net/~mseledtsov/8180393.00/ > Testing: > 1. Locally: exercised all CDS tests locally on Linux-x64 > > Checked several jtr files to make sure that stdout of > child process is copied into the jtr file > > PASS > > 2. Automated: CDS tests on multiple platforms via automated > test system > > In progress > > > Thank you, > Misha > From david.holmes at oracle.com Wed May 17 07:01:37 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 17 May 2017 17:01:37 +1000 Subject: RFR(S): 8179953: [ppc] TLABWasteIncrement not loaded correctly In-Reply-To: <00a572990741428e9b1935f77c536f07@sap.com> References: <00a572990741428e9b1935f77c536f07@sap.com> Message-ID: Hi Goetz, On 15/05/2017 8:31 PM, Lindenmaier, Goetz wrote: > Hi David, > > yes, what you are saying is correct, and I share your concerns about the > design of JVMOptionsUtils. > Actually I think a test is considered as passed if the VM issues the message > that gc flags arecontradictory. Thus, if you run with GCType=ParallelGC those > flags that set G1GC explicitly are not properly tested. This is what first > happened with my fix as we test with default settings which is G1. > And here I didn't consider that I might change test behavior for the other > flags. But if this raises new errors, they should be addressed anyways. I'm trying to test this change with each explicit GC flag but the thing is so slow that it keeps timing out on me. Will take about 5 or 6 hours to test at this rate. David > Best regards, > Goetz. > > >> -----Original Message----- >> From: David Holmes [mailto:david.holmes at oracle.com] >> Sent: Montag, 15. Mai 2017 06:25 >> To: Lindenmaier, Goetz ; hotspot-runtime- >> dev at openjdk.java.net >> Subject: Re: RFR(S): 8179953: [ppc] TLABWasteIncrement not loaded correctly >> >> Hi Goetz, >> >> On 12/05/2017 4:45 PM, Lindenmaier, Goetz wrote: >>> Hi David, >>> >>> the broken code is protected by >>> if (allow_shared_alloc) { >>> which evaluates to true with ParallelGC but not with G1GC. >>> It might be true with other GCs, too, but the design of >>> JVMOptionsUtils:addNameDependency() is quite simple, >>> and I just did it as for other flags. >>> I could put in more logic into that case there, like >>> if (GCType == G1GC || GCTyep == ... ) { >>> option.addPrepend("-XX:+UseParallelGC"); >>> } >>> enumerating all GCs that don't support TLABWastIncrement. >>> But I thought one test case for the flag is fine. The way it is >>> we missed the error, so I would like to fix the test. >> >> So if I understand correctly you just want to test use of >> TLABWasteIncrement, and you know ParallelGC uses it, so you added >> -XX:+UseParallelGC to the test options. Okay I get that part. > Yes. > >> However the more I look at JVMOptionsUtils.java the less I understand >> how it expects things to work. I don't find addNameDependency simple, >> but confused! It looks for things like G1* and so adds UseG1GC, but >> completely ignores that GCType may already be set - but that works >> because AFAICS when it finds a G1* option in the existing set of >> options, it has to be that G1GC is already the current GC type - >> otherwise the test would fail - no? >> >> Looking at the history of this code it never set the GC so would use the >> default, unless addNameDependency explicitly set one. That worked fine. >> Then 8144578 took issue with the fact the test only ran with the default >> GC and so introduced GCType to run the test using the same GC as was >> used to launch the main test. But AFAICS that completely overlooked the >> fact that addNameDependency might set an explicit GC! Mea culpa as I was >> one of the reviewers of that change and it was obviously incomplete! :( >> It also seem there are mails missing from the archives as you find now >> mail from me here: >> >> http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-January/021390.html >> >> BTW in that review it refers to TLABWasteIncrement as a "non-dedicated >> gc flag". That suggests to me that testing of this flag was expected to >> be GC agnostic. ?? >> >> Anyway the question is, what is the right/best way to reconcile the use >> of GCType with explicit setting of GC selection flags? Your fix is one >> way. Another option would be to clear GCType when requiring a specifc >> GC. Not sure either way is significantly better - pros and cons to both. >> >> My concern is whether this change may have an unexpected side-effects >> given that I don't see how it can have been working correctly in the >> first place. >> >> Unfortunately the primary author of that code is no longer around. >> >> David >> >> >>> Yes, in JVMOption I must exclude setting G1GC else I get >>> an error claiming that two different GCs are set. >>> >>> Best regards, >>> Goetz. >>> >>> >>> >>> >>>> -----Original Message----- >>>> From: David Holmes [mailto:david.holmes at oracle.com] >>>> Sent: Wednesday, May 10, 2017 5:19 AM >>>> To: Lindenmaier, Goetz ; hotspot-runtime- >>>> dev at openjdk.java.net >>>> Subject: Re: RFR(S): 8179953: [ppc] TLABWasteIncrement not loaded >>>> correctly >>>> >>>> Hi Goetz, >>>> >>>> On 10/05/2017 12:56 AM, Lindenmaier, Goetz wrote: >>>>> Hi, >>>>> >>>>> Please review this change. As I fix the test, I please need a sponsor. >>>>> http://cr.openjdk.java.net/~goetz/wr17/8179953-ppc_flag/webrev.01/ >>>> >>>> I'm unclear on the test changes. It isn't obvious to me that >>>> TLABWasteIncrement is a ParallelGC only flag. By adding that as an >>>> additional flag you then had to exclude adding the GCType (if any GC >>>> selector is given) to avoid conflicting options - is that right? >>>> >>>> Thanks, >>>> David >>>> >>>>> On ppc, the mentioned flag is loaded as a 16 bit immediate, while it can >>>> have >>>>> bigger values. >>>>> I also adapt TestOptionWithRanges, which did not show the bug because >>>>> the flag has no effect with G1. >>>>> >>>>> Best regards, >>>>> Goetz. >>>>> >>>>> From david.holmes at oracle.com Wed May 17 09:49:44 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 17 May 2017 19:49:44 +1000 Subject: RFR(S): 8179953: [ppc] TLABWasteIncrement not loaded correctly In-Reply-To: References: <00a572990741428e9b1935f77c536f07@sap.com> Message-ID: <55e4d2ca-5496-c232-bcf4-a8ba5999b4f8@oracle.com> Okay testing passed. If you finalize the changeset and send me a link I will sponsor it for you. Thanks, David On 17/05/2017 5:01 PM, David Holmes wrote: > Hi Goetz, > > On 15/05/2017 8:31 PM, Lindenmaier, Goetz wrote: >> Hi David, >> >> yes, what you are saying is correct, and I share your concerns about the >> design of JVMOptionsUtils. >> Actually I think a test is considered as passed if the VM issues the >> message >> that gc flags arecontradictory. Thus, if you run with >> GCType=ParallelGC those >> flags that set G1GC explicitly are not properly tested. This is what >> first >> happened with my fix as we test with default settings which is G1. >> And here I didn't consider that I might change test behavior for the >> other >> flags. But if this raises new errors, they should be addressed anyways. > > I'm trying to test this change with each explicit GC flag but the thing > is so slow that it keeps timing out on me. Will take about 5 or 6 hours > to test at this rate. > > David > >> Best regards, >> Goetz. >> >> >>> -----Original Message----- >>> From: David Holmes [mailto:david.holmes at oracle.com] >>> Sent: Montag, 15. Mai 2017 06:25 >>> To: Lindenmaier, Goetz ; hotspot-runtime- >>> dev at openjdk.java.net >>> Subject: Re: RFR(S): 8179953: [ppc] TLABWasteIncrement not loaded >>> correctly >>> >>> Hi Goetz, >>> >>> On 12/05/2017 4:45 PM, Lindenmaier, Goetz wrote: >>>> Hi David, >>>> >>>> the broken code is protected by >>>> if (allow_shared_alloc) { >>>> which evaluates to true with ParallelGC but not with G1GC. >>>> It might be true with other GCs, too, but the design of >>>> JVMOptionsUtils:addNameDependency() is quite simple, >>>> and I just did it as for other flags. >>>> I could put in more logic into that case there, like >>>> if (GCType == G1GC || GCTyep == ... ) { >>>> option.addPrepend("-XX:+UseParallelGC"); >>>> } >>>> enumerating all GCs that don't support TLABWastIncrement. >>>> But I thought one test case for the flag is fine. The way it is >>>> we missed the error, so I would like to fix the test. >>> >>> So if I understand correctly you just want to test use of >>> TLABWasteIncrement, and you know ParallelGC uses it, so you added >>> -XX:+UseParallelGC to the test options. Okay I get that part. >> Yes. >> >>> However the more I look at JVMOptionsUtils.java the less I understand >>> how it expects things to work. I don't find addNameDependency simple, >>> but confused! It looks for things like G1* and so adds UseG1GC, but >>> completely ignores that GCType may already be set - but that works >>> because AFAICS when it finds a G1* option in the existing set of >>> options, it has to be that G1GC is already the current GC type - >>> otherwise the test would fail - no? >>> >>> Looking at the history of this code it never set the GC so would use the >>> default, unless addNameDependency explicitly set one. That worked fine. >>> Then 8144578 took issue with the fact the test only ran with the default >>> GC and so introduced GCType to run the test using the same GC as was >>> used to launch the main test. But AFAICS that completely overlooked the >>> fact that addNameDependency might set an explicit GC! Mea culpa as I was >>> one of the reviewers of that change and it was obviously incomplete! :( >>> It also seem there are mails missing from the archives as you find now >>> mail from me here: >>> >>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-January/021390.html >>> >>> >>> BTW in that review it refers to TLABWasteIncrement as a "non-dedicated >>> gc flag". That suggests to me that testing of this flag was expected to >>> be GC agnostic. ?? >>> >>> Anyway the question is, what is the right/best way to reconcile the use >>> of GCType with explicit setting of GC selection flags? Your fix is one >>> way. Another option would be to clear GCType when requiring a specifc >>> GC. Not sure either way is significantly better - pros and cons to both. >>> >>> My concern is whether this change may have an unexpected side-effects >>> given that I don't see how it can have been working correctly in the >>> first place. >>> >>> Unfortunately the primary author of that code is no longer around. >>> >>> David >>> >>> >>>> Yes, in JVMOption I must exclude setting G1GC else I get >>>> an error claiming that two different GCs are set. >>>> >>>> Best regards, >>>> Goetz. >>>> >>>> >>>> >>>> >>>>> -----Original Message----- >>>>> From: David Holmes [mailto:david.holmes at oracle.com] >>>>> Sent: Wednesday, May 10, 2017 5:19 AM >>>>> To: Lindenmaier, Goetz ; hotspot-runtime- >>>>> dev at openjdk.java.net >>>>> Subject: Re: RFR(S): 8179953: [ppc] TLABWasteIncrement not loaded >>>>> correctly >>>>> >>>>> Hi Goetz, >>>>> >>>>> On 10/05/2017 12:56 AM, Lindenmaier, Goetz wrote: >>>>>> Hi, >>>>>> >>>>>> Please review this change. As I fix the test, I please need a >>>>>> sponsor. >>>>>> http://cr.openjdk.java.net/~goetz/wr17/8179953-ppc_flag/webrev.01/ >>>>> >>>>> I'm unclear on the test changes. It isn't obvious to me that >>>>> TLABWasteIncrement is a ParallelGC only flag. By adding that as an >>>>> additional flag you then had to exclude adding the GCType (if any GC >>>>> selector is given) to avoid conflicting options - is that right? >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> On ppc, the mentioned flag is loaded as a 16 bit immediate, while >>>>>> it can >>>>> have >>>>>> bigger values. >>>>>> I also adapt TestOptionWithRanges, which did not show the bug because >>>>>> the flag has no effect with G1. >>>>>> >>>>>> Best regards, >>>>>> Goetz. >>>>>> >>>>>> From david.holmes at oracle.com Wed May 17 13:02:48 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 17 May 2017 23:02:48 +1000 Subject: RFR 8178604: JVM does not allow defining boot loader modules in exploded build after module system initialization In-Reply-To: <3082901f-a299-c517-1b7e-2cc80aa52b3e@oracle.com> References: <85e36988-a145-6805-8659-56c738013f58@oracle.com> <46a812ca-228d-01e7-e889-fd22f2af0537@oracle.com> <11709d4a-0a41-1d38-fcd1-67d42c33eaeb@oracle.com> <3ee613ff-f6da-9ef8-6781-efce4edba29e@oracle.com> <21b3fdc5-d8a7-0535-3ac6-f1a097a7431d@oracle.com> <388b80ed-e436-f888-ffed-5b92e0117ce2@oracle.com> <2e1d1c53-8d6d-e606-f047-a223f76b63a9@oracle.com> <1b618a5c-671d-9063-326e-16991efab441@oracle.com> <4550b8c7-9bff-a44d-fefb-b5b589da371f@oracle.com> <3082901f-a299-c517-1b7e-2cc80aa52b3e@oracle.com> Message-ID: For the record I post-re-reviewed what was pushed. Doing the allocation in ClassLoader::classLoader_init2 seems much cleaner given that immediately after the allocation we call add_to_exploded_build_list() - where the allocation previously was. Thanks, David On 17/05/2017 1:48 PM, David Holmes wrote: > Hi Harold, > > On 16/05/2017 11:36 PM, harold seigel wrote: >> Hi David, >> >> Thanks for the review! I'll change the initialization of >> _exploded_entries so that it is done eagerly, before I push the change. > > ??? If you do that then it needs to be re-reviewed! > > David > >> Harold >> >> >> On 5/10/2017 10:11 PM, David Holmes wrote: >>> Hi Harold, >>> >>> On 11/05/2017 7:18 AM, harold seigel wrote: >>>> Hi Lois, David, >>>> >>>> Please review this latest webrev that initializes _exploded_entries >>>> without locking: >>>> >>>> http://cr.openjdk.java.net/~hseigel/bug_8178604.4/webrev/index.html >>> >>> Ok. >>> >>> It would be a lot clearer, in my opinion, that the initialization is >>> indeed thread-safe, if it were done eagerly instead of lazily. The >>> execution contexts of the first and second calls (in separate threads) >>> to that chunk of code are not readily discernible by any reasonable >>> means. But the extensive comment is appreciated. >>> >>> Thanks, >>> David >>> >>>> Thanks, Harold >>>> >>>> >>>> On 5/10/2017 1:19 PM, Lois Foltan wrote: >>>>> On 5/9/2017 5:18 PM, David Holmes wrote: >>>>>> On 9/05/2017 11:14 PM, harold seigel wrote: >>>>>>> Hi David, >>>>>>> >>>>>>> See comments embedded below. >>>>>>> >>>>>>> Thanks, Harold >>>>>>> >>>>>>> >>>>>>> On 5/8/2017 8:52 PM, David Holmes wrote: >>>>>>>> Hi Harold, >>>>>>>> >>>>>>>> On 9/05/2017 4:00 AM, harold seigel wrote: >>>>>>>>> Hi David, >>>>>>>>> >>>>>>>>> Please see this updated webrev: >>>>>>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604.3/webrev/index.html >>>>>>>>> >>>>>>>>> >>>>>>>>> It contains the changes you requested below. >>>>>>>> >>>>>>>> Thank you. I will point out that Lois originally suggested the same >>>>>>>> changes. >>>>>>>> >>>>>>>>> Note that the _exploded_entries table gets created very early >>>>>>>>> during >>>>>>>>> SystemDictionary initialization, before initializing the >>>>>>>>> pre-loaded >>>>>>>>> classes. So, the code in load_class() at line ~1503 can assume >>>>>>>>> that >>>>>>>>> _exploded_entries is not null. >>>>>>>> >>>>>>>> Which thread executes ClassLoader::add_to_exploded_build_list, and >>>>>>>> which thread executes load_class() first? If they are guaranteed >>>>>>>> to be >>>>>>>> the same thread then you don't need locking on the construction. If >>>>>>>> they can be different threads then there must be some >>>>>>>> synchronization >>>>>>>> between them that ensures _exploded_entries is properly visible >>>>>>>> after >>>>>>>> construction. >>>>>>> They are the same thread. At startup, the thread calls >>>>>> >>>>>> Then as I said, why do we need any locking? I note Lois is the one >>>>>> that requested this and you questioned it. I also question it. >>>>> >>>>> Thank you Harold for investigating this. I stand corrected, it seems >>>>> that we do not need the lock after all when allocating >>>>> ClassLoader::_exploded_entries. >>>>> Lois >>>>> >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>>> SystemDictionary::initialize_preloaded_classes(), which calls >>>>>>> ClassLoader::classLoader_init2(), which calls >>>>>>> add_to_exploded_build_list(). >>>>>>> >>>>>>> Control then returns back to >>>>>>> SystemDictionary::initialize_preloaded_classes() which eventually >>>>>>> calls >>>>>>> SystemDictionary::load_instance_class(), which calls >>>>>>> ClassLoader::load_class(). Here's the call stack showing these >>>>>>> calls: >>>>>>> >>>>>>> V [libjvm.so+0x974c0c] ClassLoader::load_class(Symbol*, bool, >>>>>>> Thread*)+0x41c >>>>>>> V [libjvm.so+0x1648339] >>>>>>> SystemDictionary::load_instance_class(Symbol*, Handle, >>>>>>> Thread*)+0x819 >>>>>>> V [libjvm.so+0x1648e25] >>>>>>> SystemDictionary::resolve_instance_class_or_null(Symbol*, Handle, >>>>>>> Handle, Thread*)+0x985 >>>>>>> V [libjvm.so+0x16494ba] >>>>>>> SystemDictionary::resolve_or_null(Symbol*, >>>>>>> Handle, Handle, Thread*)+0x5a >>>>>>> V [libjvm.so+0x16498fe] >>>>>>> SystemDictionary::resolve_or_fail(Symbol*, >>>>>>> Handle, Handle, bool, Thread*)+0x1e >>>>>>> V [libjvm.so+0x1649aa9] >>>>>>> SystemDictionary::initialize_wk_klass(SystemDictionary::WKID, int, >>>>>>> Thread*)+0x149 >>>>>>> V [libjvm.so+0x1649bfe] >>>>>>> SystemDictionary::initialize_wk_klasses_until(SystemDictionary::WKID, >>>>>>> >>>>>>> SystemDictionary::WKID&, >>>>>>> >>>>>>> Thread*)+0x5e >>>>>>> V [libjvm.so+0x1649dba] >>>>>>> SystemDictionary::*initialize_preloaded_classes*(Thread*)+0xea >>>>>>> V [libjvm.so+0x164a28a] >>>>>>> SystemDictionary::initialize(Thread*)+0x26a >>>>>>> V [libjvm.so+0x16cd7e0] Universe::genesis(Thread*)+0x7c0 >>>>>>> V [libjvm.so+0x16ce4cc] universe2_init()+0x2c >>>>>>> V [libjvm.so+0xe0f268] init_globals()+0xa8 >>>>>>> V [libjvm.so+0x1696e6f] Threads::create_vm(JavaVMInitArgs*, >>>>>>> bool*)+0x2df >>>>>>> ... >>>>>>> >>>>>>> So, is the existing synchronization in this change sufficient? >>>>>>> >>>>>>> Thanks, Harold >>>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> ----- >>>>>>>> >>>>>>>>> Thanks, Harold >>>>>>>>> >>>>>>>>> On 5/4/2017 5:32 PM, David Holmes wrote: >>>>>>>>>> Hi Harold, >>>>>>>>>> >>>>>>>>>> On 5/05/2017 3:47 AM, harold seigel wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> Please review this updated webrev: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604.2/webrev/index.html >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> The updated webrev takes out a lock when creating >>>>>>>>>>> _exploded_entries. It >>>>>>>>>>> also contains additional comments and an assert() to address >>>>>>>>>>> Lois's >>>>>>>>>>> concerns with reduced code readability involving what situations >>>>>>>>>>> warrant >>>>>>>>>>> taking out a lock in ClassLoader::search_module_entries(). >>>>>>>>>>> >>>>>>>>>>> I don't think there needs to be two different initial entry >>>>>>>>>>> points as >>>>>>>>>>> suggested by David below. The method behaves slightly >>>>>>>>>>> differently >>>>>>>>>>> depending on the value of its 'need_lock' parameter. But, >>>>>>>>>>> there is >>>>>>>>>>> enough common code for it to remain one method. >>>>>>>>>> >>>>>>>>>> I think this highlights uncertainty about the concurrent >>>>>>>>>> behaviour in >>>>>>>>>> relation to this code. On the one hand you are worried about an >>>>>>>>>> initialization race and so use the lock, but when you do: >>>>>>>>>> >>>>>>>>>> 1500 // The exploded build entries can be added to at any >>>>>>>>>> time >>>>>>>>>> so pass 'true' for >>>>>>>>>> 1501 // need_lock so that a lock is taken out when >>>>>>>>>> searching >>>>>>>>>> them. >>>>>>>>>> 1502 assert(_exploded_entries != NULL, "No exploded build >>>>>>>>>> entries present"); >>>>>>>>>> 1503 stream = search_module_entries(_exploded_entries, >>>>>>>>>> class_name, file_name, true, CHECK_NULL); >>>>>>>>>> >>>>>>>>>> you load _exploded_entries with no lock held and so must be >>>>>>>>>> certain >>>>>>>>>> that you can not possibly read a null value here. >>>>>>>>>> >>>>>>>>>> The needs_lock parameter seems superfluous - you know the >>>>>>>>>> condition >>>>>>>>>> for locking is that the list is the _exploded_entries, and you >>>>>>>>>> assert >>>>>>>>>> that. So no need to pass in needs_lock, just grab the lock when >>>>>>>>>> dealing with _exploded_entries. >>>>>>>>>> >>>>>>>>>> Cheers, >>>>>>>>>> David >>>>>>>>>> >>>>>>>>>>> Thanks, Harold >>>>>>>>>>> >>>>>>>>>>> On 5/3/2017 12:59 AM, David Holmes wrote: >>>>>>>>>>>> On 3/05/2017 7:11 AM, Lois Foltan wrote: >>>>>>>>>>>>> On 5/2/2017 9:03 AM, harold seigel wrote: >>>>>>>>>>>>>> Hi Lois, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks for your comments. Please see my in-line responses. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Harold >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 5/1/2017 4:14 PM, Lois Foltan wrote: >>>>>>>>>>>>>>> Hi Harold, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Looks good. A couple of comments: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> src/share/vm/classfile/classLoader.cpp >>>>>>>>>>>>>>> line #828 - please take out the Module_lock when the >>>>>>>>>>>>>>> _exploded_entries is created >>>>>>>>>>>>>> This fix does not change the possible need for >>>>>>>>>>>>>> synchronization when >>>>>>>>>>>>>> _exploded_entries gets created during module system >>>>>>>>>>>>>> initialization. >>>>>>>>>>>>>> Are you saying that the existing code is wrong and should >>>>>>>>>>>>>> have >>>>>>>>>>>>>> taken >>>>>>>>>>>>>> out the Module_lock before creating _exploding_entries? >>>>>>>>>>>>> >>>>>>>>>>>>> The method ClassLoader::add_to_exploded_build_list can be >>>>>>>>>>>>> called at >>>>>>>>>>>>> anytime (during module system initialization and post module >>>>>>>>>>>>> system >>>>>>>>>>>>> initialization). Thus it seems prudent that no matter what >>>>>>>>>>>>> the >>>>>>>>>>>>> current >>>>>>>>>>>>> JVM phase, that any access to >>>>>>>>>>>>> ClassLoader::_exploding_entries be >>>>>>>>>>>>> protected via a lock. That includes creation as well as >>>>>>>>>>>>> insertion >>>>>>>>>>>>> into >>>>>>>>>>>>> the list. >>>>>>>>>>>> >>>>>>>>>>>> If creation is not guaranteed to occur before other threads >>>>>>>>>>>> that will >>>>>>>>>>>> call the same method and use the _exploded_entries list, then >>>>>>>>>>>> synchronization of some form is essential. If it is guaranteed >>>>>>>>>>>> then we >>>>>>>>>>>> don't need create time sync just to be prudent - but the >>>>>>>>>>>> guarantee >>>>>>>>>>>> must be firmly established and clearly documented. >>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> line #1402 - I like the new factored out method >>>>>>>>>>>>>>> find_first_module_cpe(), however, instead of adding a new >>>>>>>>>>>>>>> "need_lock" >>>>>>>>>>>>>>> parameter, I would rather see code >>>>>>>>>>>>>>> in find_first_module_cpe() that >>>>>>>>>>>>>>> takes the >>>>>>>>>>>>>>> Module_lock if the module_list parameter equals >>>>>>>>>>>>>>> ClassLoader::_exploded_entries >>>>>>>>>>>>>> I think your suggestion makes the code less flexible and does >>>>>>>>>>>>>> not >>>>>>>>>>>>>> require potential future callers of >>>>>>>>>>>>>> search_module_entries() to >>>>>>>>>>>>>> think >>>>>>>>>>>>>> about synchronization. So, I'd prefer to leave that change >>>>>>>>>>>>>> as is. >>>>>>>>>>>>> >>>>>>>>>>>>> I see your point. My point was based on reduced code >>>>>>>>>>>>> readability, the >>>>>>>>>>>>> further away from the decision to establish the lock, the less >>>>>>>>>>>>> apparent >>>>>>>>>>>>> it is within the new method >>>>>>>>>>>>> ClassLoader::find_first_module_cpe() >>>>>>>>>>>>> as to >>>>>>>>>>>>> what situations warranted the lock in the first place. >>>>>>>>>>>> >>>>>>>>>>>> Further, this: >>>>>>>>>>>> >>>>>>>>>>>> 1495 stream = search_module_entries(_exploded_entries, >>>>>>>>>>>> class_name, file_name, true, CHECK_NULL); >>>>>>>>>>>> >>>>>>>>>>>> is potentially unsafe if we can race with creation because we >>>>>>>>>>>> first >>>>>>>>>>>> access _exploded_entries without holding any lock! And the fact >>>>>>>>>>>> the >>>>>>>>>>>> method needs to do different things depending on which "list" >>>>>>>>>>>> was >>>>>>>>>>>> passed in suggests to me it should be two different initial >>>>>>>>>>>> entry >>>>>>>>>>>> points. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> David >>>>>>>>>>>> ----- >>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Lois >>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Lois >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 5/1/2017 3:36 PM, harold seigel wrote: >>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Please review this JDK-10 bug fix to allow defining of >>>>>>>>>>>>>>>> modules to >>>>>>>>>>>>>>>> the boot loader in exploded builds after module system >>>>>>>>>>>>>>>> initialization. The fix uses the module lock to >>>>>>>>>>>>>>>> synchronize >>>>>>>>>>>>>>>> access >>>>>>>>>>>>>>>> to the _exploded_entries data structure (which is only >>>>>>>>>>>>>>>> used by >>>>>>>>>>>>>>>> exploded builds). >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Note that the above capability already exists for regular >>>>>>>>>>>>>>>> builds. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Open Webrev: >>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604/webrev/index.html >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8178604 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> The fix was tested with JCK tests, the JTreg hotspot, >>>>>>>>>>>>>>>> java/io, >>>>>>>>>>>>>>>> java/lang, java/util and other tests, the RBT tier2 -tier5 >>>>>>>>>>>>>>>> tests, >>>>>>>>>>>>>>>> the co-located NSK tests, and with JPRT. The JTReg and JCK >>>>>>>>>>>>>>>> tests >>>>>>>>>>>>>>>> were run with an exploded build. Also, the example >>>>>>>>>>>>>>>> program in >>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>> JBS bug was run by hand to verify the fix. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, Harold >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >>>> >> From harold.seigel at oracle.com Wed May 17 13:04:57 2017 From: harold.seigel at oracle.com (harold seigel) Date: Wed, 17 May 2017 09:04:57 -0400 Subject: RFR 8178604: JVM does not allow defining boot loader modules in exploded build after module system initialization In-Reply-To: References: <85e36988-a145-6805-8659-56c738013f58@oracle.com> <46a812ca-228d-01e7-e889-fd22f2af0537@oracle.com> <11709d4a-0a41-1d38-fcd1-67d42c33eaeb@oracle.com> <3ee613ff-f6da-9ef8-6781-efce4edba29e@oracle.com> <21b3fdc5-d8a7-0535-3ac6-f1a097a7431d@oracle.com> <388b80ed-e436-f888-ffed-5b92e0117ce2@oracle.com> <2e1d1c53-8d6d-e606-f047-a223f76b63a9@oracle.com> <1b618a5c-671d-9063-326e-16991efab441@oracle.com> <4550b8c7-9bff-a44d-fefb-b5b589da371f@oracle.com> <3082901f-a299-c517-1b7e-2cc80aa52b3e@oracle.com> Message-ID: Thanks David! Harold On 5/17/2017 9:02 AM, David Holmes wrote: > For the record I post-re-reviewed what was pushed. > > Doing the allocation in ClassLoader::classLoader_init2 seems much > cleaner given that immediately after the allocation we call > add_to_exploded_build_list() - where the allocation previously was. > > Thanks, > David > > On 17/05/2017 1:48 PM, David Holmes wrote: >> Hi Harold, >> >> On 16/05/2017 11:36 PM, harold seigel wrote: >>> Hi David, >>> >>> Thanks for the review! I'll change the initialization of >>> _exploded_entries so that it is done eagerly, before I push the change. >> >> ??? If you do that then it needs to be re-reviewed! >> >> David >> >>> Harold >>> >>> >>> On 5/10/2017 10:11 PM, David Holmes wrote: >>>> Hi Harold, >>>> >>>> On 11/05/2017 7:18 AM, harold seigel wrote: >>>>> Hi Lois, David, >>>>> >>>>> Please review this latest webrev that initializes _exploded_entries >>>>> without locking: >>>>> >>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604.4/webrev/index.html >>>> >>>> Ok. >>>> >>>> It would be a lot clearer, in my opinion, that the initialization is >>>> indeed thread-safe, if it were done eagerly instead of lazily. The >>>> execution contexts of the first and second calls (in separate threads) >>>> to that chunk of code are not readily discernible by any reasonable >>>> means. But the extensive comment is appreciated. >>>> >>>> Thanks, >>>> David >>>> >>>>> Thanks, Harold >>>>> >>>>> >>>>> On 5/10/2017 1:19 PM, Lois Foltan wrote: >>>>>> On 5/9/2017 5:18 PM, David Holmes wrote: >>>>>>> On 9/05/2017 11:14 PM, harold seigel wrote: >>>>>>>> Hi David, >>>>>>>> >>>>>>>> See comments embedded below. >>>>>>>> >>>>>>>> Thanks, Harold >>>>>>>> >>>>>>>> >>>>>>>> On 5/8/2017 8:52 PM, David Holmes wrote: >>>>>>>>> Hi Harold, >>>>>>>>> >>>>>>>>> On 9/05/2017 4:00 AM, harold seigel wrote: >>>>>>>>>> Hi David, >>>>>>>>>> >>>>>>>>>> Please see this updated webrev: >>>>>>>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604.3/webrev/index.html >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> It contains the changes you requested below. >>>>>>>>> >>>>>>>>> Thank you. I will point out that Lois originally suggested the >>>>>>>>> same >>>>>>>>> changes. >>>>>>>>> >>>>>>>>>> Note that the _exploded_entries table gets created very early >>>>>>>>>> during >>>>>>>>>> SystemDictionary initialization, before initializing the >>>>>>>>>> pre-loaded >>>>>>>>>> classes. So, the code in load_class() at line ~1503 can assume >>>>>>>>>> that >>>>>>>>>> _exploded_entries is not null. >>>>>>>>> >>>>>>>>> Which thread executes ClassLoader::add_to_exploded_build_list, >>>>>>>>> and >>>>>>>>> which thread executes load_class() first? If they are guaranteed >>>>>>>>> to be >>>>>>>>> the same thread then you don't need locking on the >>>>>>>>> construction. If >>>>>>>>> they can be different threads then there must be some >>>>>>>>> synchronization >>>>>>>>> between them that ensures _exploded_entries is properly visible >>>>>>>>> after >>>>>>>>> construction. >>>>>>>> They are the same thread. At startup, the thread calls >>>>>>> >>>>>>> Then as I said, why do we need any locking? I note Lois is the one >>>>>>> that requested this and you questioned it. I also question it. >>>>>> >>>>>> Thank you Harold for investigating this. I stand corrected, it >>>>>> seems >>>>>> that we do not need the lock after all when allocating >>>>>> ClassLoader::_exploded_entries. >>>>>> Lois >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>>> SystemDictionary::initialize_preloaded_classes(), which calls >>>>>>>> ClassLoader::classLoader_init2(), which calls >>>>>>>> add_to_exploded_build_list(). >>>>>>>> >>>>>>>> Control then returns back to >>>>>>>> SystemDictionary::initialize_preloaded_classes() which eventually >>>>>>>> calls >>>>>>>> SystemDictionary::load_instance_class(), which calls >>>>>>>> ClassLoader::load_class(). Here's the call stack showing these >>>>>>>> calls: >>>>>>>> >>>>>>>> V [libjvm.so+0x974c0c] ClassLoader::load_class(Symbol*, bool, >>>>>>>> Thread*)+0x41c >>>>>>>> V [libjvm.so+0x1648339] >>>>>>>> SystemDictionary::load_instance_class(Symbol*, Handle, >>>>>>>> Thread*)+0x819 >>>>>>>> V [libjvm.so+0x1648e25] >>>>>>>> SystemDictionary::resolve_instance_class_or_null(Symbol*, Handle, >>>>>>>> Handle, Thread*)+0x985 >>>>>>>> V [libjvm.so+0x16494ba] >>>>>>>> SystemDictionary::resolve_or_null(Symbol*, >>>>>>>> Handle, Handle, Thread*)+0x5a >>>>>>>> V [libjvm.so+0x16498fe] >>>>>>>> SystemDictionary::resolve_or_fail(Symbol*, >>>>>>>> Handle, Handle, bool, Thread*)+0x1e >>>>>>>> V [libjvm.so+0x1649aa9] >>>>>>>> SystemDictionary::initialize_wk_klass(SystemDictionary::WKID, int, >>>>>>>> Thread*)+0x149 >>>>>>>> V [libjvm.so+0x1649bfe] >>>>>>>> SystemDictionary::initialize_wk_klasses_until(SystemDictionary::WKID, >>>>>>>> >>>>>>>> >>>>>>>> SystemDictionary::WKID&, >>>>>>>> >>>>>>>> Thread*)+0x5e >>>>>>>> V [libjvm.so+0x1649dba] >>>>>>>> SystemDictionary::*initialize_preloaded_classes*(Thread*)+0xea >>>>>>>> V [libjvm.so+0x164a28a] >>>>>>>> SystemDictionary::initialize(Thread*)+0x26a >>>>>>>> V [libjvm.so+0x16cd7e0] Universe::genesis(Thread*)+0x7c0 >>>>>>>> V [libjvm.so+0x16ce4cc] universe2_init()+0x2c >>>>>>>> V [libjvm.so+0xe0f268] init_globals()+0xa8 >>>>>>>> V [libjvm.so+0x1696e6f] Threads::create_vm(JavaVMInitArgs*, >>>>>>>> bool*)+0x2df >>>>>>>> ... >>>>>>>> >>>>>>>> So, is the existing synchronization in this change sufficient? >>>>>>>> >>>>>>>> Thanks, Harold >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> David >>>>>>>>> ----- >>>>>>>>> >>>>>>>>>> Thanks, Harold >>>>>>>>>> >>>>>>>>>> On 5/4/2017 5:32 PM, David Holmes wrote: >>>>>>>>>>> Hi Harold, >>>>>>>>>>> >>>>>>>>>>> On 5/05/2017 3:47 AM, harold seigel wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> Please review this updated webrev: >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604.2/webrev/index.html >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> The updated webrev takes out a lock when creating >>>>>>>>>>>> _exploded_entries. It >>>>>>>>>>>> also contains additional comments and an assert() to address >>>>>>>>>>>> Lois's >>>>>>>>>>>> concerns with reduced code readability involving what >>>>>>>>>>>> situations >>>>>>>>>>>> warrant >>>>>>>>>>>> taking out a lock in ClassLoader::search_module_entries(). >>>>>>>>>>>> >>>>>>>>>>>> I don't think there needs to be two different initial entry >>>>>>>>>>>> points as >>>>>>>>>>>> suggested by David below. The method behaves slightly >>>>>>>>>>>> differently >>>>>>>>>>>> depending on the value of its 'need_lock' parameter. But, >>>>>>>>>>>> there is >>>>>>>>>>>> enough common code for it to remain one method. >>>>>>>>>>> >>>>>>>>>>> I think this highlights uncertainty about the concurrent >>>>>>>>>>> behaviour in >>>>>>>>>>> relation to this code. On the one hand you are worried about an >>>>>>>>>>> initialization race and so use the lock, but when you do: >>>>>>>>>>> >>>>>>>>>>> 1500 // The exploded build entries can be added to at any >>>>>>>>>>> time >>>>>>>>>>> so pass 'true' for >>>>>>>>>>> 1501 // need_lock so that a lock is taken out when >>>>>>>>>>> searching >>>>>>>>>>> them. >>>>>>>>>>> 1502 assert(_exploded_entries != NULL, "No exploded build >>>>>>>>>>> entries present"); >>>>>>>>>>> 1503 stream = search_module_entries(_exploded_entries, >>>>>>>>>>> class_name, file_name, true, CHECK_NULL); >>>>>>>>>>> >>>>>>>>>>> you load _exploded_entries with no lock held and so must be >>>>>>>>>>> certain >>>>>>>>>>> that you can not possibly read a null value here. >>>>>>>>>>> >>>>>>>>>>> The needs_lock parameter seems superfluous - you know the >>>>>>>>>>> condition >>>>>>>>>>> for locking is that the list is the _exploded_entries, and you >>>>>>>>>>> assert >>>>>>>>>>> that. So no need to pass in needs_lock, just grab the lock when >>>>>>>>>>> dealing with _exploded_entries. >>>>>>>>>>> >>>>>>>>>>> Cheers, >>>>>>>>>>> David >>>>>>>>>>> >>>>>>>>>>>> Thanks, Harold >>>>>>>>>>>> >>>>>>>>>>>> On 5/3/2017 12:59 AM, David Holmes wrote: >>>>>>>>>>>>> On 3/05/2017 7:11 AM, Lois Foltan wrote: >>>>>>>>>>>>>> On 5/2/2017 9:03 AM, harold seigel wrote: >>>>>>>>>>>>>>> Hi Lois, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks for your comments. Please see my in-line responses. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Harold >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 5/1/2017 4:14 PM, Lois Foltan wrote: >>>>>>>>>>>>>>>> Hi Harold, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Looks good. A couple of comments: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> src/share/vm/classfile/classLoader.cpp >>>>>>>>>>>>>>>> line #828 - please take out the Module_lock when the >>>>>>>>>>>>>>>> _exploded_entries is created >>>>>>>>>>>>>>> This fix does not change the possible need for >>>>>>>>>>>>>>> synchronization when >>>>>>>>>>>>>>> _exploded_entries gets created during module system >>>>>>>>>>>>>>> initialization. >>>>>>>>>>>>>>> Are you saying that the existing code is wrong and should >>>>>>>>>>>>>>> have >>>>>>>>>>>>>>> taken >>>>>>>>>>>>>>> out the Module_lock before creating _exploding_entries? >>>>>>>>>>>>>> >>>>>>>>>>>>>> The method ClassLoader::add_to_exploded_build_list can be >>>>>>>>>>>>>> called at >>>>>>>>>>>>>> anytime (during module system initialization and post module >>>>>>>>>>>>>> system >>>>>>>>>>>>>> initialization). Thus it seems prudent that no matter what >>>>>>>>>>>>>> the >>>>>>>>>>>>>> current >>>>>>>>>>>>>> JVM phase, that any access to >>>>>>>>>>>>>> ClassLoader::_exploding_entries be >>>>>>>>>>>>>> protected via a lock. That includes creation as well as >>>>>>>>>>>>>> insertion >>>>>>>>>>>>>> into >>>>>>>>>>>>>> the list. >>>>>>>>>>>>> >>>>>>>>>>>>> If creation is not guaranteed to occur before other threads >>>>>>>>>>>>> that will >>>>>>>>>>>>> call the same method and use the _exploded_entries list, then >>>>>>>>>>>>> synchronization of some form is essential. If it is >>>>>>>>>>>>> guaranteed >>>>>>>>>>>>> then we >>>>>>>>>>>>> don't need create time sync just to be prudent - but the >>>>>>>>>>>>> guarantee >>>>>>>>>>>>> must be firmly established and clearly documented. >>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> line #1402 - I like the new factored out method >>>>>>>>>>>>>>>> find_first_module_cpe(), however, instead of adding a new >>>>>>>>>>>>>>>> "need_lock" >>>>>>>>>>>>>>>> parameter, I would rather see code >>>>>>>>>>>>>>>> in find_first_module_cpe() that >>>>>>>>>>>>>>>> takes the >>>>>>>>>>>>>>>> Module_lock if the module_list parameter equals >>>>>>>>>>>>>>>> ClassLoader::_exploded_entries >>>>>>>>>>>>>>> I think your suggestion makes the code less flexible and >>>>>>>>>>>>>>> does >>>>>>>>>>>>>>> not >>>>>>>>>>>>>>> require potential future callers of >>>>>>>>>>>>>>> search_module_entries() to >>>>>>>>>>>>>>> think >>>>>>>>>>>>>>> about synchronization. So, I'd prefer to leave that change >>>>>>>>>>>>>>> as is. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I see your point. My point was based on reduced code >>>>>>>>>>>>>> readability, the >>>>>>>>>>>>>> further away from the decision to establish the lock, the >>>>>>>>>>>>>> less >>>>>>>>>>>>>> apparent >>>>>>>>>>>>>> it is within the new method >>>>>>>>>>>>>> ClassLoader::find_first_module_cpe() >>>>>>>>>>>>>> as to >>>>>>>>>>>>>> what situations warranted the lock in the first place. >>>>>>>>>>>>> >>>>>>>>>>>>> Further, this: >>>>>>>>>>>>> >>>>>>>>>>>>> 1495 stream = search_module_entries(_exploded_entries, >>>>>>>>>>>>> class_name, file_name, true, CHECK_NULL); >>>>>>>>>>>>> >>>>>>>>>>>>> is potentially unsafe if we can race with creation because we >>>>>>>>>>>>> first >>>>>>>>>>>>> access _exploded_entries without holding any lock! And the >>>>>>>>>>>>> fact >>>>>>>>>>>>> the >>>>>>>>>>>>> method needs to do different things depending on which "list" >>>>>>>>>>>>> was >>>>>>>>>>>>> passed in suggests to me it should be two different initial >>>>>>>>>>>>> entry >>>>>>>>>>>>> points. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> David >>>>>>>>>>>>> ----- >>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Lois >>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> Lois >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 5/1/2017 3:36 PM, harold seigel wrote: >>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Please review this JDK-10 bug fix to allow defining of >>>>>>>>>>>>>>>>> modules to >>>>>>>>>>>>>>>>> the boot loader in exploded builds after module system >>>>>>>>>>>>>>>>> initialization. The fix uses the module lock to >>>>>>>>>>>>>>>>> synchronize >>>>>>>>>>>>>>>>> access >>>>>>>>>>>>>>>>> to the _exploded_entries data structure (which is only >>>>>>>>>>>>>>>>> used by >>>>>>>>>>>>>>>>> exploded builds). >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Note that the above capability already exists for regular >>>>>>>>>>>>>>>>> builds. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Open Webrev: >>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~hseigel/bug_8178604/webrev/index.html >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8178604 >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> The fix was tested with JCK tests, the JTreg hotspot, >>>>>>>>>>>>>>>>> java/io, >>>>>>>>>>>>>>>>> java/lang, java/util and other tests, the RBT tier2 >>>>>>>>>>>>>>>>> -tier5 >>>>>>>>>>>>>>>>> tests, >>>>>>>>>>>>>>>>> the co-located NSK tests, and with JPRT. The JTReg >>>>>>>>>>>>>>>>> and JCK >>>>>>>>>>>>>>>>> tests >>>>>>>>>>>>>>>>> were run with an exploded build. Also, the example >>>>>>>>>>>>>>>>> program in >>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>> JBS bug was run by hand to verify the fix. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, Harold >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>> >>>>> >>> From mikhailo.seledtsov at oracle.com Wed May 17 13:33:23 2017 From: mikhailo.seledtsov at oracle.com (Mikhailo Seledtsov) Date: Wed, 17 May 2017 06:33:23 -0700 Subject: RFR(XS): JDK-8180393: [TESTBUG] CDSTestUtils property test.cds.copy.child.stdout should be true by default In-Reply-To: References: Message-ID: <591C5123.9020003@oracle.com> Harold, Ioi, Serguei, Thank you for review. Misha On 5/16/17, 10:12 PM, serguei.spitsyn at oracle.com wrote: > Hi Misha, > > Looks good to me. > > Thanks, > Serguei > > > On 5/16/17 11:29, mikhailo wrote: >> Please review this trivial change; see bug description for rational >> and details. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8180393 >> Webrev: http://cr.openjdk.java.net/~mseledtsov/8180393.00/ >> Testing: >> 1. Locally: exercised all CDS tests locally on Linux-x64 >> >> Checked several jtr files to make sure that stdout of >> child process is copied into the jtr file >> >> PASS >> >> 2. Automated: CDS tests on multiple platforms via automated >> test system >> >> In progress >> >> >> Thank you, >> Misha >> > From gerard.ziemski at oracle.com Wed May 17 15:40:15 2017 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Wed, 17 May 2017 10:40:15 -0500 Subject: RFR 8166848: Performance bug: SystemDictionary - optimization In-Reply-To: References: <249af7dc-a3a4-52b4-ca6d-ffdb7ec13764@oracle.com> <1cc33d02-6fca-7f67-34ff-0fcc8c309cad@oracle.com> Message-ID: hi Coleen, Thank you for tackling this issue. Looks good to me and I like Serguei suggestion. cheers > On May 16, 2017, at 11:21 PM, serguei.spitsyn at oracle.com wrote: > > Hi Coleen, > > It looks good to me. > Nice simplification! > > One tip is to use the bucket_count for incrementing the element_count: > element_count += bucket_count; > > out of the internal for-loop. > It is to underline the relationship between the two counters. > > Thanks, > Serguei > > > > On 5/15/17 05:30, coleen.phillimore at oracle.com wrote: >> >>> Summary: Check instead that a bucket isn't 10x the average >>> >>> See bug and linked bugs for more details. >>> >>> Tested with RBT nightly tests (tier2-5). >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8166848.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8166848 >>> >>> Thanks, >>> Coleen >> > From coleen.phillimore at oracle.com Wed May 17 15:45:02 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 17 May 2017 11:45:02 -0400 Subject: RFR 8166848: Performance bug: SystemDictionary - optimization In-Reply-To: References: <249af7dc-a3a4-52b4-ca6d-ffdb7ec13764@oracle.com> <1cc33d02-6fca-7f67-34ff-0fcc8c309cad@oracle.com> Message-ID: <14bc424a-08e0-18c0-80c7-d3678130c8ae@oracle.com> Hi Serguei, thank you for reviewing. On 5/17/17 12:21 AM, serguei.spitsyn at oracle.com wrote: > Hi Coleen, > > It looks good to me. > Nice simplification! > > One tip is to use the bucket_count for incrementing the element_count: > element_count += bucket_count; > > out of the internal for-loop. > It is to underline the relationship between the two counters. This is a good suggestion. I made this change and retested: open webrev at http://cr.openjdk.java.net/~coleenp/8166848.02/webrev Thanks! Coleen > > Thanks, > Serguei > > > > On 5/15/17 05:30, coleen.phillimore at oracle.com wrote: >> >>> Summary: Check instead that a bucket isn't 10x the average >>> >>> See bug and linked bugs for more details. >>> >>> Tested with RBT nightly tests (tier2-5). >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8166848.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8166848 >>> >>> Thanks, >>> Coleen >> > From coleen.phillimore at oracle.com Wed May 17 15:49:31 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 17 May 2017 11:49:31 -0400 Subject: RFR 8166848: Performance bug: SystemDictionary - optimization In-Reply-To: References: <249af7dc-a3a4-52b4-ca6d-ffdb7ec13764@oracle.com> <1cc33d02-6fca-7f67-34ff-0fcc8c309cad@oracle.com> Message-ID: <9b5c8274-8b24-5054-7d5b-065ef07de3e5@oracle.com> Gerard, Thank you for the code review and help with this change. Coleen On 5/17/17 11:40 AM, Gerard Ziemski wrote: > hi Coleen, > > Thank you for tackling this issue. > > Looks good to me and I like Serguei suggestion. > > > cheers > >> On May 16, 2017, at 11:21 PM, serguei.spitsyn at oracle.com wrote: >> >> Hi Coleen, >> >> It looks good to me. >> Nice simplification! >> >> One tip is to use the bucket_count for incrementing the element_count: >> element_count += bucket_count; >> >> out of the internal for-loop. >> It is to underline the relationship between the two counters. >> >> Thanks, >> Serguei >> >> >> >> On 5/15/17 05:30, coleen.phillimore at oracle.com wrote: >>>> Summary: Check instead that a bucket isn't 10x the average >>>> >>>> See bug and linked bugs for more details. >>>> >>>> Tested with RBT nightly tests (tier2-5). >>>> >>>> open webrev at http://cr.openjdk.java.net/~coleenp/8166848.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8166848 >>>> >>>> Thanks, >>>> Coleen From rkennke at redhat.com Wed May 17 15:57:07 2017 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 17 May 2017 17:57:07 +0200 Subject: RFR (jdk10): JDK-8180175: ObjectSynchronizer only needs to iterate in-use monitors In-Reply-To: References: <89399a72-0263-5ebd-5fda-82f404fbe2ca@redhat.com> <6b162e43-3b0f-9e23-e63f-c844b0bc21da@oracle.com> Message-ID: <03378add-534e-bbd1-23cf-5e496786347c@redhat.com> Am 16.05.2017 um 09:01 schrieb Robbin Ehn: > On 05/15/2017 11:01 PM, Roman Kennke wrote: >> Ping? > > Looks ok, will look good when we can drop MonitorInUseLists branches. > > Are you handling the deprecation of MonitorInUseLists when CSR becomes > available? We don't have to stall this RFR until CSR becomes available though, right? Any reviewer wants to give it a review? Cheers, Roman From ioi.lam at oracle.com Wed May 17 16:43:22 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 17 May 2017 09:43:22 -0700 Subject: RFR 8166848: Performance bug: SystemDictionary - optimization In-Reply-To: <1cc33d02-6fca-7f67-34ff-0fcc8c309cad@oracle.com> References: <249af7dc-a3a4-52b4-ca6d-ffdb7ec13764@oracle.com> <1cc33d02-6fca-7f67-34ff-0fcc8c309cad@oracle.com> Message-ID: <591C7DAA.6070801@oracle.com> I know many people are not fans of macros or templates, but the hashtable iteration code is repeated in many places. For example, the verification code is the same across these two tables: 352 void PackageEntryTable::verify() { 353 int element_count = 0; 354 int max_bucket_count = 0; 355 for (int index = 0; index < table_size(); index++) { 356 int bucket_count = 0; 357 for (PackageEntry* probe = bucket(index); 358 probe != NULL; 359 probe = probe->next()) { 360 probe->verify(); 361 element_count++; 362 bucket_count++; 363 } 364 max_bucket_count = MAX2(max_bucket_count, bucket_count); 365 } 366 guarantee(number_of_entries() == element_count, 367 "Verify of Package Entry Table failed"); 368 DEBUG_ONLY(verify_lookup_length(max_bucket_count, "Package Entry Table")); 369 } 509 void ModuleEntryTable::verify() { 510 int element_count = 0; 511 int max_bucket_count = 0; 512 for (int i = 0; i < table_size(); i++) { 513 int bucket_count = 0; 514 for (ModuleEntry* probe = bucket(i); 515 probe != NULL; 516 probe = probe->next()) { 517 probe->verify(); 518 element_count++; 519 bucket_count++; 520 } 521 max_bucket_count = MAX2(max_bucket_count, bucket_count); 522 } 523 guarantee(number_of_entries() == element_count, 524 "Verify of Module Entry Table failed"); 525 DEBUG_ONLY(verify_lookup_length(max_bucket_count, "Module Entry Table")); 526 } and void Dictionary::verify() only has a bit more code inside the inner loop. How about using a template here. The following is probably not the best way to do it (but it compiles :-) #include class Hashtable { public: template void do_verify() { for (int i=0; i<10; i++) { // pseudo code for the table iteration void *probe = NULL; // pseudo code for getting an entry Table::verify_entry((Entry*)probe); } } }; class MyHashentry {/*...*/}; class MyHashtable : public Hashtable { public: void verify() { do_verify(); } static void verify_entry(MyHashentry *probe) { // ... } }; Thanks - Ioi On 5/15/17 5:30 AM, coleen.phillimore at oracle.com wrote: > >> Summary: Check instead that a bucket isn't 10x the average >> >> See bug and linked bugs for more details. >> >> Tested with RBT nightly tests (tier2-5). >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8166848.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8166848 >> >> Thanks, >> Coleen > From coleen.phillimore at oracle.com Wed May 17 18:10:05 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 17 May 2017 14:10:05 -0400 Subject: RFR 8166848: Performance bug: SystemDictionary - optimization In-Reply-To: <591C7DAA.6070801@oracle.com> References: <249af7dc-a3a4-52b4-ca6d-ffdb7ec13764@oracle.com> <1cc33d02-6fca-7f67-34ff-0fcc8c309cad@oracle.com> <591C7DAA.6070801@oracle.com> Message-ID: <5d7a8a9f-8359-6007-fe53-e4a6d76b80ed@oracle.com> On 5/17/17 12:43 PM, Ioi Lam wrote: > I know many people are not fans of macros or templates, but the > hashtable iteration code is repeated in many places. I know. I am not pleased with this. What I wanted to do was have Hashtable::verify() call probe->verify() and each Entry type have a verify function, like in C++, but the *Entry types aren't constructed with vtables. Let me see what I can do with template tricks. Coleen > > For example, the verification code is the same across these two tables: > > 352 void PackageEntryTable::verify() { > 353 int element_count = 0; > 354 int max_bucket_count = 0; > 355 for (int index = 0; index < table_size(); index++) { > 356 int bucket_count = 0; > 357 for (PackageEntry* probe = bucket(index); > 358 probe != NULL; > 359 probe = probe->next()) { > 360 probe->verify(); > 361 element_count++; > 362 bucket_count++; > 363 } > 364 max_bucket_count = MAX2(max_bucket_count, bucket_count); > 365 } > 366 guarantee(number_of_entries() == element_count, > 367 "Verify of Package Entry Table failed"); > 368 DEBUG_ONLY(verify_lookup_length(max_bucket_count, "Package > Entry Table")); > 369 } > > 509 void ModuleEntryTable::verify() { > 510 int element_count = 0; > 511 int max_bucket_count = 0; > 512 for (int i = 0; i < table_size(); i++) { > 513 int bucket_count = 0; > 514 for (ModuleEntry* probe = bucket(i); > 515 probe != NULL; > 516 probe = probe->next()) { > 517 probe->verify(); > 518 element_count++; > 519 bucket_count++; > 520 } > 521 max_bucket_count = MAX2(max_bucket_count, bucket_count); > 522 } > 523 guarantee(number_of_entries() == element_count, > 524 "Verify of Module Entry Table failed"); > 525 DEBUG_ONLY(verify_lookup_length(max_bucket_count, "Module Entry > Table")); > 526 } > > > and void Dictionary::verify() only has a bit more code inside the > inner loop. How about using a template here. The following is probably > not the best way to do it (but it compiles :-) > > #include > > class Hashtable { > public: > template void do_verify() { > for (int i=0; i<10; i++) { // pseudo code for the table iteration > void *probe = NULL; // pseudo code for getting an entry > Table::verify_entry((Entry*)probe); > } > } > }; > > class MyHashentry {/*...*/}; > > class MyHashtable : public Hashtable { > public: > void verify() { > do_verify(); > } > static void verify_entry(MyHashentry *probe) { > // ... > } > }; > > > Thanks > - Ioi > > > On 5/15/17 5:30 AM, coleen.phillimore at oracle.com wrote: >> >>> Summary: Check instead that a bucket isn't 10x the average >>> >>> See bug and linked bugs for more details. >>> >>> Tested with RBT nightly tests (tier2-5). >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8166848.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8166848 >>> >>> Thanks, >>> Coleen >> > From serguei.spitsyn at oracle.com Wed May 17 18:17:11 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 17 May 2017 11:17:11 -0700 Subject: RFR 8166848: Performance bug: SystemDictionary - optimization In-Reply-To: <14bc424a-08e0-18c0-80c7-d3678130c8ae@oracle.com> References: <249af7dc-a3a4-52b4-ca6d-ffdb7ec13764@oracle.com> <1cc33d02-6fca-7f67-34ff-0fcc8c309cad@oracle.com> <14bc424a-08e0-18c0-80c7-d3678130c8ae@oracle.com> Message-ID: Hi Coleen, Looks good. Thank you for the update! Thanks, Serguei On 5/17/17 08:45, coleen.phillimore at oracle.com wrote: > > Hi Serguei, thank you for reviewing. > > On 5/17/17 12:21 AM, serguei.spitsyn at oracle.com wrote: >> Hi Coleen, >> >> It looks good to me. >> Nice simplification! >> >> One tip is to use the bucket_count for incrementing the element_count: >> element_count += bucket_count; >> >> out of the internal for-loop. >> It is to underline the relationship between the two counters. > > This is a good suggestion. I made this change and retested: > > open webrev at http://cr.openjdk.java.net/~coleenp/8166848.02/webrev > > Thanks! > Coleen >> >> Thanks, >> Serguei >> >> >> >> On 5/15/17 05:30, coleen.phillimore at oracle.com wrote: >>> >>>> Summary: Check instead that a bucket isn't 10x the average >>>> >>>> See bug and linked bugs for more details. >>>> >>>> Tested with RBT nightly tests (tier2-5). >>>> >>>> open webrev at http://cr.openjdk.java.net/~coleenp/8166848.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8166848 >>>> >>>> Thanks, >>>> Coleen >>> >> > From robbin.ehn at oracle.com Wed May 17 19:57:18 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 17 May 2017 21:57:18 +0200 Subject: RFR (jdk10): JDK-8180175: ObjectSynchronizer only needs to iterate in-use monitors In-Reply-To: <03378add-534e-bbd1-23cf-5e496786347c@redhat.com> References: <89399a72-0263-5ebd-5fda-82f404fbe2ca@redhat.com> <6b162e43-3b0f-9e23-e63f-c844b0bc21da@oracle.com> <03378add-534e-bbd1-23cf-5e496786347c@redhat.com> Message-ID: <525fce11-7055-4221-4690-074f1c2aa338@oracle.com> On 05/17/2017 05:57 PM, Roman Kennke wrote: > Am 16.05.2017 um 09:01 schrieb Robbin Ehn: >> On 05/15/2017 11:01 PM, Roman Kennke wrote: >>> Ping? >> >> Looks ok, will look good when we can drop MonitorInUseLists branches. >> >> Are you handling the deprecation of MonitorInUseLists when CSR becomes >> available? > > We don't have to stall this RFR until CSR becomes available though, right? Right, create a new enhancement for that and when CSR becomes available you should be able create a CSR in jira for that via some menu. And first changeset for that is just to add an deprecation warning if someone does -XX:-MonitorInUseLists. /Robbin > > Any reviewer wants to give it a review? > > Cheers, Roman > From chenyt at cs.sjtu.edu.cn Wed May 17 20:13:16 2017 From: chenyt at cs.sjtu.edu.cn (=?gb2312?B?s8LT6s2k?=) Date: Wed, 17 May 2017 13:13:16 -0700 Subject: HotSpot and IBM's J9 behave quite differently when processing monitorenters and monitorexits Message-ID: <004501d2cf4a$060b7770$12226650$@cs.sjtu.edu.cn> I have tested several programs (in Jimple) and found that HotSpot and J9 match monitorenters and monitorexits quite differently. Verifiers should play more important roles here. (1) Test the next program (r2 is not initizlied) on HotSpot and J9. J9 throw out a verifier error, while HotSpot does not. It seems that HotSpot's verifier forgets to check whether a monitored object is initialized. public class Search extends java.lang.Object { public void () { Search r0; r0 := @this: Search; specialinvoke r0.()>(); return; } public void main(java.lang.String[]) throws java.lang.Exception { Search r0; Search r2; java.lang.String[] r1; r0 := @this: Search; r1 := @parameter0: java.lang.String[]; r2 = new Search; entermonitor r2; entermonitor r0; exitmonitor r2; exitmonitor r0; return; } } (2) Test the next program on HotSpot and J9, and both do not report any errors. However, I guess the order in the program (entermonitor r2; => entermonitor r0; => exitmonitor r2; => exitmonitor r0;) violates the situation of "structured locking" (Structured locking is the situation when, during a method invocation, every exit on a given monitor matches a preceding entry on that monitor, see the specification https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-2.html#jvms-2.11.10) ? Actually, the words (every exit on a given monitor matches a preceding entry on that monitor) are not quite clear as for me. Otherwise the first rule (The number of monitor entries performed by T on M during a method invocation must equal the number of monitor exits performed by T on M during the method invocation whether the method invocation completes normally or abruptly.) is sufficient. public class Search extends java.lang.Object { public void () { Search r0; r0 := @this: Search; specialinvoke r0.()>(); return; } public void main(java.lang.String[]) throws java.lang.Exception { Search r0; Search r2; java.lang.String[] r1; r0 := @this: Search; r1 := @parameter0: java.lang.String[]; r2 = new Search; specialinvoke r2.()>(); entermonitor r2; entermonitor r0; exitmonitor r2; exitmonitor r0; return; } } (3) The next program enters monitor in and exits it in main(). HotSpot throws a runtime exception, while J9 does not. Should this program be rejected by the verifiers? public class Search extends java.lang.Object { public void () { Search r0; r0 := @this: Search; specialinvoke r0.()>(); entermonitor r0; return; } public void main(java.lang.String[]) throws java.lang.Exception { Search r0; Search r2; java.lang.String[] r1; r0 := @this: Search; r1 := @parameter0: java.lang.String[]; r2 = new Search; specialinvoke r2.()>(); exitmonitor r2; exitmonitor r0; return; } } From chenyt at cs.sjtu.edu.cn Wed May 17 20:51:36 2017 From: chenyt at cs.sjtu.edu.cn (=?gb2312?B?s8LT6s2k?=) Date: Wed, 17 May 2017 13:51:36 -0700 Subject: HotSpot and IBM's J9 behave quite differently when processing monitorenters and monitorexits Message-ID: <004b01d2cf4f$60f3eb90$22dbc2b0$@cs.sjtu.edu.cn> The differences are there with updated JRE versions: HotSpot java version "1.8.0_131" Java(TM) SE Runtime Environment (build 1.8.0_131-b11) Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode) J9, java version "1.8.0" Java(TM) SE Runtime Environment (build pxa6480sr4fp5-20170421_01(SR4 FP5)) IBM J9 VM (build 2.8, JRE 1.8.0 Linux amd64-64 Compressed References 20170419_344392 (JIT enabled, AOT enabled) J9VM - R28_20170419_1004_B344392 JIT - tr.r14.java_20170419_344392 GC - R28_20170419_1004_B344392_CMPRSS J9CL - 20170419_344392) JCL - 20170420_01 based on Oracle jdk8u131-b11 ??: HotSpot and IBM's J9 behave quite differently when processing monitorenters and monitorexits I have tested several programs (in Jimple) and found that HotSpot and J9 match monitorenters and monitorexits quite differently. Verifiers should play more important roles here. (1) Test the next program (r2 is not initizlied) on HotSpot and J9. J9 throw out a verifier error, while HotSpot does not. It seems that HotSpot's verifier forgets to check whether a monitored object is initialized. public class Search extends java.lang.Object { public void () { Search r0; r0 := @this: Search; specialinvoke r0.()>(); return; } public void main(java.lang.String[]) throws java.lang.Exception { Search r0; Search r2; java.lang.String[] r1; r0 := @this: Search; r1 := @parameter0: java.lang.String[]; r2 = new Search; entermonitor r2; entermonitor r0; exitmonitor r2; exitmonitor r0; return; } } (2) Test the next program on HotSpot and J9, and both do not report any errors. However, I guess the order in the program (entermonitor r2; => entermonitor r0; => exitmonitor r2; => exitmonitor r0;) violates the situation of "structured locking" (Structured locking is the situation when, during a method invocation, every exit on a given monitor matches a preceding entry on that monitor, see the specification https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-2.html#jvms-2.11.10) ? Actually, the words (every exit on a given monitor matches a preceding entry on that monitor) are not quite clear as for me. Otherwise the first rule (The number of monitor entries performed by T on M during a method invocation must equal the number of monitor exits performed by T on M during the method invocation whether the method invocation completes normally or abruptly.) is sufficient. public class Search extends java.lang.Object { public void () { Search r0; r0 := @this: Search; specialinvoke r0.()>(); return; } public void main(java.lang.String[]) throws java.lang.Exception { Search r0; Search r2; java.lang.String[] r1; r0 := @this: Search; r1 := @parameter0: java.lang.String[]; r2 = new Search; specialinvoke r2.()>(); entermonitor r2; entermonitor r0; exitmonitor r2; exitmonitor r0; return; } } (3) The next program enters monitor in and exits it in main(). HotSpot throws a runtime exception, while J9 does not. Should this program be rejected by the verifiers? public class Search extends java.lang.Object { public void () { Search r0; r0 := @this: Search; specialinvoke r0.()>(); entermonitor r0; return; } public void main(java.lang.String[]) throws java.lang.Exception { Search r0; Search r2; java.lang.String[] r1; r0 := @this: Search; r1 := @parameter0: java.lang.String[]; r2 = new Search; specialinvoke r2.()>(); exitmonitor r2; exitmonitor r0; return; } } From zgu at redhat.com Wed May 17 20:52:37 2017 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 17 May 2017 16:52:37 -0400 Subject: RFR (jdk10): JDK-8180175: ObjectSynchronizer only needs to iterate in-use monitors In-Reply-To: <89399a72-0263-5ebd-5fda-82f404fbe2ca@redhat.com> References: <89399a72-0263-5ebd-5fda-82f404fbe2ca@redhat.com> Message-ID: <8d75eef3-c143-dd75-6362-6203c7fb175f@redhat.com> Looks good to me. Some comments will be nice. Thanks, -Zhengyu On 05/11/2017 06:08 AM, Roman Kennke wrote: > Hi, > > ObjectSynchronizer::oops_do() currently iterates all monitor blocks, > even free ones. With -XX:+MonitorInUseLists, each thread has its own > in-use monitors list, and ObjectSynchronizer has a global gOmInUseList > for moribund threads. Iterating over in-use monitors only significantly > inmproves scanning time for monitors, because it avoids scanning free > blocks, improves caching (no padding, and better chances to have > thread-local oops in cache already). > > Performance-wise it makes a very significant difference (running > gc-bench's roots.Sync test, which exaggerates synchronizer usage, with > Shenandoah): > > baseline: > S: Thread Roots 37748 us > S: Synchronizer Roots 15115 us > UR: Thread Roots 24967 us > UR: Synchronizer Roots 11906 us > > patched: > S: Thread Roots 40365 us > S: Synchronizer Roots 0 us > UR: Thread Roots 24459 us > UR: Synchronizer Roots 0 us > > Testing: hotspot_gc, specjvm, jcstress -m quick > > http://cr.openjdk.java.net/~rkennke/8180175/webrev.00/ > > > Roman > > From coleen.phillimore at oracle.com Wed May 17 21:02:25 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 17 May 2017 17:02:25 -0400 Subject: RFR 8166848: Performance bug: SystemDictionary - optimization In-Reply-To: <591C7DAA.6070801@oracle.com> References: <249af7dc-a3a4-52b4-ca6d-ffdb7ec13764@oracle.com> <1cc33d02-6fca-7f67-34ff-0fcc8c309cad@oracle.com> <591C7DAA.6070801@oracle.com> Message-ID: <1eb0f3aa-965a-62ab-cd59-b4ea44a81db2@oracle.com> Ioi, Thanks for pushing me to remove this duplication. How's this? I reran with runThese and -XX:+VerifyBeforeGC, and built product. http://cr.openjdk.java.net/~coleenp/8166848.03/webrev thanks, Coleen On 5/17/17 12:43 PM, Ioi Lam wrote: > I know many people are not fans of macros or templates, but the > hashtable iteration code is repeated in many places. > > For example, the verification code is the same across these two tables: > > 352 void PackageEntryTable::verify() { > 353 int element_count = 0; > 354 int max_bucket_count = 0; > 355 for (int index = 0; index < table_size(); index++) { > 356 int bucket_count = 0; > 357 for (PackageEntry* probe = bucket(index); > 358 probe != NULL; > 359 probe = probe->next()) { > 360 probe->verify(); > 361 element_count++; > 362 bucket_count++; > 363 } > 364 max_bucket_count = MAX2(max_bucket_count, bucket_count); > 365 } > 366 guarantee(number_of_entries() == element_count, > 367 "Verify of Package Entry Table failed"); > 368 DEBUG_ONLY(verify_lookup_length(max_bucket_count, "Package > Entry Table")); > 369 } > > 509 void ModuleEntryTable::verify() { > 510 int element_count = 0; > 511 int max_bucket_count = 0; > 512 for (int i = 0; i < table_size(); i++) { > 513 int bucket_count = 0; > 514 for (ModuleEntry* probe = bucket(i); > 515 probe != NULL; > 516 probe = probe->next()) { > 517 probe->verify(); > 518 element_count++; > 519 bucket_count++; > 520 } > 521 max_bucket_count = MAX2(max_bucket_count, bucket_count); > 522 } > 523 guarantee(number_of_entries() == element_count, > 524 "Verify of Module Entry Table failed"); > 525 DEBUG_ONLY(verify_lookup_length(max_bucket_count, "Module Entry > Table")); > 526 } > > > and void Dictionary::verify() only has a bit more code inside the > inner loop. How about using a template here. The following is probably > not the best way to do it (but it compiles :-) > > #include > > class Hashtable { > public: > template void do_verify() { > for (int i=0; i<10; i++) { // pseudo code for the table iteration > void *probe = NULL; // pseudo code for getting an entry > Table::verify_entry((Entry*)probe); > } > } > }; > > class MyHashentry {/*...*/}; > > class MyHashtable : public Hashtable { > public: > void verify() { > do_verify(); > } > static void verify_entry(MyHashentry *probe) { > // ... > } > }; > > > Thanks > - Ioi > > > On 5/15/17 5:30 AM, coleen.phillimore at oracle.com wrote: >> >>> Summary: Check instead that a bucket isn't 10x the average >>> >>> See bug and linked bugs for more details. >>> >>> Tested with RBT nightly tests (tier2-5). >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8166848.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8166848 >>> >>> Thanks, >>> Coleen >> > From chenyt at cs.sjtu.edu.cn Wed May 17 21:23:08 2017 From: chenyt at cs.sjtu.edu.cn (=?gb2312?B?s8LT6s2k?=) Date: Wed, 17 May 2017 14:23:08 -0700 Subject: HotSpot and IBM's J9 behave quite differently when processing monitorenters and monitorexits Message-ID: <005101d2cf53$c8c16b40$5a4441c0$@cs.sjtu.edu.cn> Am I wrong? The byte code for main() in case 1 is as follows. The strange thing is that NullPointerException is also not thrown at runtime. public void main(java.lang.String[]) throws java.lang.Exception; descriptor: ([Ljava/lang/String;)V flags: ACC_PUBLIC Code: stack=3, locals=2, args_size=2 0: new #2 // class Search 3: dup 4: aload_0 5: monitorenter 6: monitorenter 7: monitorexit 8: aload_0 9: monitorexit 10: return Exceptions: throws java.lang.Exception ??: HotSpot and IBM's J9 behave quite differently when processing monitorenters and monitorexits I have tested several programs (in Jimple) and found that HotSpot and J9 match monitorenters and monitorexits quite differently. Verifiers should play more important roles here. (1) Test the next program (r2 is not initizlied) on HotSpot and J9. J9 throw out a verifier error, while HotSpot does not. It seems that HotSpot's verifier forgets to check whether a monitored object is initialized. public class Search extends java.lang.Object { public void () { Search r0; r0 := @this: Search; specialinvoke r0.()>(); return; } public void main(java.lang.String[]) throws java.lang.Exception { Search r0; Search r2; java.lang.String[] r1; r0 := @this: Search; r1 := @parameter0: java.lang.String[]; r2 = new Search; entermonitor r2; entermonitor r0; exitmonitor r2; exitmonitor r0; return; } } (2) Test the next program on HotSpot and J9, and both do not report any errors. However, I guess the order in the program (entermonitor r2; => entermonitor r0; => exitmonitor r2; => exitmonitor r0;) violates the situation of "structured locking" (Structured locking is the situation when, during a method invocation, every exit on a given monitor matches a preceding entry on that monitor, see the specification https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-2.html#jvms-2.11.10) ? Actually, the words (every exit on a given monitor matches a preceding entry on that monitor) are not quite clear as for me. Otherwise the first rule (The number of monitor entries performed by T on M during a method invocation must equal the number of monitor exits performed by T on M during the method invocation whether the method invocation completes normally or abruptly.) is sufficient. public class Search extends java.lang.Object { public void () { Search r0; r0 := @this: Search; specialinvoke r0.()>(); return; } public void main(java.lang.String[]) throws java.lang.Exception { Search r0; Search r2; java.lang.String[] r1; r0 := @this: Search; r1 := @parameter0: java.lang.String[]; r2 = new Search; specialinvoke r2.()>(); entermonitor r2; entermonitor r0; exitmonitor r2; exitmonitor r0; return; } } (3) The next program enters monitor in and exits it in main(). HotSpot throws a runtime exception, while J9 does not. Should this program be rejected by the verifiers? public class Search extends java.lang.Object { public void () { Search r0; r0 := @this: Search; specialinvoke r0.()>(); entermonitor r0; return; } public void main(java.lang.String[]) throws java.lang.Exception { Search r0; Search r2; java.lang.String[] r1; r0 := @this: Search; r1 := @parameter0: java.lang.String[]; r2 = new Search; specialinvoke r2.()>(); exitmonitor r2; exitmonitor r0; return; } } From rkennke at redhat.com Wed May 17 21:25:46 2017 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 17 May 2017 23:25:46 +0200 Subject: RFR (jdk10): JDK-8180175: ObjectSynchronizer only needs to iterate in-use monitors In-Reply-To: <8d75eef3-c143-dd75-6362-6203c7fb175f@redhat.com> References: <89399a72-0263-5ebd-5fda-82f404fbe2ca@redhat.com> <8d75eef3-c143-dd75-6362-6203c7fb175f@redhat.com> Message-ID: > Looks good to me. Some comments will be nice. Like this: http://cr.openjdk.java.net/~rkennke/8180175/webrev.01/ ? Roman > > Thanks, > > -Zhengyu > > On 05/11/2017 06:08 AM, Roman Kennke wrote: >> Hi, >> >> ObjectSynchronizer::oops_do() currently iterates all monitor blocks, >> even free ones. With -XX:+MonitorInUseLists, each thread has its own >> in-use monitors list, and ObjectSynchronizer has a global gOmInUseList >> for moribund threads. Iterating over in-use monitors only significantly >> inmproves scanning time for monitors, because it avoids scanning free >> blocks, improves caching (no padding, and better chances to have >> thread-local oops in cache already). >> >> Performance-wise it makes a very significant difference (running >> gc-bench's roots.Sync test, which exaggerates synchronizer usage, with >> Shenandoah): >> >> baseline: >> S: Thread Roots 37748 us >> S: Synchronizer Roots 15115 us >> UR: Thread Roots 24967 us >> UR: Synchronizer Roots 11906 us >> >> patched: >> S: Thread Roots 40365 us >> S: Synchronizer Roots 0 us >> UR: Thread Roots 24459 us >> UR: Synchronizer Roots 0 us >> >> Testing: hotspot_gc, specjvm, jcstress -m quick >> >> http://cr.openjdk.java.net/~rkennke/8180175/webrev.00/ >> >> >> Roman >> >> From david.holmes at oracle.com Wed May 17 21:27:10 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 18 May 2017 07:27:10 +1000 Subject: RFR (jdk10): JDK-8180175: ObjectSynchronizer only needs to iterate in-use monitors In-Reply-To: <525fce11-7055-4221-4690-074f1c2aa338@oracle.com> References: <89399a72-0263-5ebd-5fda-82f404fbe2ca@redhat.com> <6b162e43-3b0f-9e23-e63f-c844b0bc21da@oracle.com> <03378add-534e-bbd1-23cf-5e496786347c@redhat.com> <525fce11-7055-4221-4690-074f1c2aa338@oracle.com> Message-ID: <8faa78ba-60c2-b0a5-7f40-eb7e15aff03a@oracle.com> On 18/05/2017 5:57 AM, Robbin Ehn wrote: > On 05/17/2017 05:57 PM, Roman Kennke wrote: >> Am 16.05.2017 um 09:01 schrieb Robbin Ehn: >>> On 05/15/2017 11:01 PM, Roman Kennke wrote: >>>> Ping? >>> >>> Looks ok, will look good when we can drop MonitorInUseLists branches. >>> >>> Are you handling the deprecation of MonitorInUseLists when CSR becomes >>> available? >> >> We don't have to stall this RFR until CSR becomes available though, >> right? > > Right, create a new enhancement for that and when CSR becomes available > you should be able create a CSR in jira for that via some menu. > And first changeset for that is just to add an deprecation warning if > someone does -XX:-MonitorInUseLists. Not sure we need to rush in to deprecating the flag until we have seen in practice how this works in the wild. If there are cases where we still want to not use MonitorInUseLists then we need to keep the flag live. >> Any reviewer wants to give it a review? I've looked at it and it seems reasonable. Thanks, David >> >> Cheers, Roman >> From david.holmes at oracle.com Wed May 17 21:35:18 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 18 May 2017 07:35:18 +1000 Subject: RFR (jdk10): JDK-8180175: ObjectSynchronizer only needs to iterate in-use monitors In-Reply-To: References: <89399a72-0263-5ebd-5fda-82f404fbe2ca@redhat.com> <8d75eef3-c143-dd75-6362-6203c7fb175f@redhat.com> Message-ID: <3a0f4490-646c-8004-6272-180ef198b67e@oracle.com> On 18/05/2017 7:25 AM, Roman Kennke wrote: > >> Looks good to me. Some comments will be nice. > Like this: > > http://cr.openjdk.java.net/~rkennke/8180175/webrev.01/ > + // and the remaing global monitors in ObjectSynchronizer::oops_do(). Typo: remaing David > ? > > Roman > >> >> Thanks, >> >> -Zhengyu >> >> On 05/11/2017 06:08 AM, Roman Kennke wrote: >>> Hi, >>> >>> ObjectSynchronizer::oops_do() currently iterates all monitor blocks, >>> even free ones. With -XX:+MonitorInUseLists, each thread has its own >>> in-use monitors list, and ObjectSynchronizer has a global gOmInUseList >>> for moribund threads. Iterating over in-use monitors only significantly >>> inmproves scanning time for monitors, because it avoids scanning free >>> blocks, improves caching (no padding, and better chances to have >>> thread-local oops in cache already). >>> >>> Performance-wise it makes a very significant difference (running >>> gc-bench's roots.Sync test, which exaggerates synchronizer usage, with >>> Shenandoah): >>> >>> baseline: >>> S: Thread Roots 37748 us >>> S: Synchronizer Roots 15115 us >>> UR: Thread Roots 24967 us >>> UR: Synchronizer Roots 11906 us >>> >>> patched: >>> S: Thread Roots 40365 us >>> S: Synchronizer Roots 0 us >>> UR: Thread Roots 24459 us >>> UR: Synchronizer Roots 0 us >>> >>> Testing: hotspot_gc, specjvm, jcstress -m quick >>> >>> http://cr.openjdk.java.net/~rkennke/8180175/webrev.00/ >>> >>> >>> Roman >>> >>> > From rkennke at redhat.com Wed May 17 21:38:20 2017 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 17 May 2017 23:38:20 +0200 Subject: RFR (jdk10): JDK-8180175: ObjectSynchronizer only needs to iterate in-use monitors In-Reply-To: <3a0f4490-646c-8004-6272-180ef198b67e@oracle.com> References: <89399a72-0263-5ebd-5fda-82f404fbe2ca@redhat.com> <8d75eef3-c143-dd75-6362-6203c7fb175f@redhat.com> <3a0f4490-646c-8004-6272-180ef198b67e@oracle.com> Message-ID: <13484ee1-5c37-6971-028d-2ddf0b0f2845@redhat.com> Am 17.05.2017 um 23:35 schrieb David Holmes: > On 18/05/2017 7:25 AM, Roman Kennke wrote: >> >>> Looks good to me. Some comments will be nice. >> Like this: >> >> http://cr.openjdk.java.net/~rkennke/8180175/webrev.01/ >> > > + // and the remaing global monitors in > ObjectSynchronizer::oops_do(). > > Typo: remaing Ok. Fixed. Final webrev: http://cr.openjdk.java.net/~rkennke/8180175/webrev.02/ If it's ok, I'm going to need a sponsor. ? Thanks, Roman From david.holmes at oracle.com Wed May 17 21:40:58 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 18 May 2017 07:40:58 +1000 Subject: HotSpot and IBM's J9 behave quite differently when processing monitorenters and monitorexits In-Reply-To: <005101d2cf53$c8c16b40$5a4441c0$@cs.sjtu.edu.cn> References: <005101d2cf53$c8c16b40$5a4441c0$@cs.sjtu.edu.cn> Message-ID: <656dcb18-e65a-4c48-a6bc-cf00fe021af5@oracle.com> Hi, On 18/05/2017 7:23 AM, ??? wrote: > Am I wrong? I will look at each situation in detail when I get a chance but structured locking enforcement is optional. Also balancing the number of locks and unlocks in a frame does not mean they can't be locked and unlocked in a non-nested fashion - just that by the end the number of unlocks matches the number of locks. BTW the way you respond to these emails, as if having a conversation with yourself, makes it difficult to respond as we can't readily see what is the new email and what is the original. Cheers, David > The byte code for main() in case 1 is as follows. The strange thing is that > NullPointerException is also not thrown at runtime. > > public void main(java.lang.String[]) throws java.lang.Exception; > descriptor: ([Ljava/lang/String;)V > flags: ACC_PUBLIC > Code: > stack=3, locals=2, args_size=2 > 0: new #2 // class Search > 3: dup > 4: aload_0 > 5: monitorenter > 6: monitorenter > 7: monitorexit > 8: aload_0 > 9: monitorexit > 10: return > Exceptions: > throws java.lang.Exception > > ??: HotSpot and IBM's J9 behave quite differently when processing > monitorenters and monitorexits > > I have tested several programs (in Jimple) and found that HotSpot and J9 > match monitorenters and monitorexits quite differently. Verifiers should > play more important roles here. > > (1) Test the next program (r2 is not initizlied) on HotSpot and J9. > J9 throw out a verifier error, while HotSpot does not. It seems that > HotSpot's verifier forgets to check whether a monitored object is > initialized. > > public class Search extends java.lang.Object { public void () > { > Search r0; > r0 := @this: Search; > specialinvoke r0.()>(); > return; > } > public void main(java.lang.String[]) throws java.lang.Exception > { > Search r0; > Search r2; > java.lang.String[] r1; > r0 := @this: Search; > r1 := @parameter0: java.lang.String[]; > r2 = new Search; > > entermonitor r2; > entermonitor r0; > exitmonitor r2; > exitmonitor r0; > return; > } > } > > (2) Test the next program on HotSpot and J9, and both do not report any > errors. However, I guess the order in the program (entermonitor r2; => > entermonitor r0; => exitmonitor r2; => exitmonitor r0;) violates the > situation of "structured locking" (Structured locking is the situation when, > during a method invocation, every exit on a given monitor matches a > preceding entry on that monitor, see the specification > https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-2.html#jvms-2.11.10) > ? > Actually, the words (every exit on a given monitor matches a preceding entry > on that monitor) are not quite clear as for me. Otherwise the first rule > (The number of monitor entries performed by T on M during a method > invocation must equal the number of monitor exits performed by T on M during > the method invocation whether the method invocation completes normally or > abruptly.) is sufficient. > > public class Search extends java.lang.Object { > > public void () > { > Search r0; > r0 := @this: Search; > specialinvoke r0.()>(); > return; > } > > public void main(java.lang.String[]) throws java.lang.Exception > { > Search r0; > Search r2; > java.lang.String[] r1; > r0 := @this: Search; > r1 := @parameter0: java.lang.String[]; > r2 = new Search; > specialinvoke r2.()>(); > entermonitor r2; > entermonitor r0; > exitmonitor r2; > exitmonitor r0; > return; > } > } > > (3) The next program enters monitor in and exits it in main(). > HotSpot throws a runtime exception, while J9 does not. Should this program > be rejected by the verifiers? > > public class Search extends java.lang.Object { > > public void () > { > Search r0; > r0 := @this: Search; > specialinvoke r0.()>(); > entermonitor r0; > return; > } > > public void main(java.lang.String[]) throws java.lang.Exception > { > Search r0; > Search r2; > java.lang.String[] r1; > r0 := @this: Search; > r1 := @parameter0: java.lang.String[]; > r2 = new Search; > specialinvoke r2.()>(); > exitmonitor r2; > exitmonitor r0; > return; > } > } > > From david.holmes at oracle.com Wed May 17 21:45:21 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 18 May 2017 07:45:21 +1000 Subject: RFR (jdk10): JDK-8180175: ObjectSynchronizer only needs to iterate in-use monitors In-Reply-To: <13484ee1-5c37-6971-028d-2ddf0b0f2845@redhat.com> References: <89399a72-0263-5ebd-5fda-82f404fbe2ca@redhat.com> <8d75eef3-c143-dd75-6362-6203c7fb175f@redhat.com> <3a0f4490-646c-8004-6272-180ef198b67e@oracle.com> <13484ee1-5c37-6971-028d-2ddf0b0f2845@redhat.com> Message-ID: Hi Roman, I can sponsor this for you. BTW Robbin should also be listed as a reviewer - I'll add him. David On 18/05/2017 7:38 AM, Roman Kennke wrote: > Am 17.05.2017 um 23:35 schrieb David Holmes: >> On 18/05/2017 7:25 AM, Roman Kennke wrote: >>> >>>> Looks good to me. Some comments will be nice. >>> Like this: >>> >>> http://cr.openjdk.java.net/~rkennke/8180175/webrev.01/ >>> >> >> + // and the remaing global monitors in >> ObjectSynchronizer::oops_do(). >> >> Typo: remaing > > Ok. Fixed. > Final webrev: > > http://cr.openjdk.java.net/~rkennke/8180175/webrev.02/ > > > If it's ok, I'm going to need a sponsor. ? > > Thanks, Roman > From chenyt at cs.sjtu.edu.cn Wed May 17 22:12:22 2017 From: chenyt at cs.sjtu.edu.cn (=?gb2312?B?s8LT6s2k?=) Date: Wed, 17 May 2017 15:12:22 -0700 Subject: =?gb2312?B?tPC4tDogSG90U3BvdCBhbmQgSUJNJ3MgSjkgYmVoYXZlIHF1aXRlIA==?= =?gb2312?B?ZGlmZmVyZW50bHkgd2hlbiBwcm9jZXNzaW5nIG1vbml0b3JlbnRlcnMgYW4=?= =?gb2312?B?ZCBtb25pdG9yZXhpdHM=?= In-Reply-To: <656dcb18-e65a-4c48-a6bc-cf00fe021af5@oracle.com> References: <005101d2cf53$c8c16b40$5a4441c0$@cs.sjtu.edu.cn> <656dcb18-e65a-4c48-a6bc-cf00fe021af5@oracle.com> Message-ID: <005201d2cf5a$aa2e8f30$fe8bad90$@cs.sjtu.edu.cn> Thank you, David. I have seen from the specification that structured locking enforcement is optional. The second and the third ones are cases of structured/nested lockings. Will non-nested locking sequences raise deadlocks? Of course it is a different topic, while it might be better if it can be kicked out earlier from the specification/JVM. The first example is still a problem. It seems that HotSpot allows to monitor a pure object reference without initialized (Is it true? How can this checking be omitted?). J9 reports a verifyerror as follows. Exception in thread "main" java.lang.VerifyError: JVMVRFY012 stack shape inconsistent; class=Search, method=main([Ljava/lang/String;)V, pc=6 Exception Details: Location: Search.main([Ljava/lang/String;)V @6: JBmonitorenter Reason: Type 'uninitialized' (current frame, stack[1]) is not assignable to 'java/lang/Object' Current Frame: bci: @6 flags: { } locals: { 'Search', '[Ljava/lang/String;' } stack: { 'uninitialized', 'uninitialized' } at T.main(T.java:4) -----????----- ???: David Holmes [mailto:david.holmes at oracle.com] ????: 2017?5?17? 14:41 ???: ??? ; hotspot-runtime-dev at openjdk.java.net ??: Re: HotSpot and IBM's J9 behave quite differently when processing monitorenters and monitorexits Hi, On 18/05/2017 7:23 AM, ??? wrote: > Am I wrong? I will look at each situation in detail when I get a chance but structured locking enforcement is optional. Also balancing the number of locks and unlocks in a frame does not mean they can't be locked and unlocked in a non-nested fashion - just that by the end the number of unlocks matches the number of locks. BTW the way you respond to these emails, as if having a conversation with yourself, makes it difficult to respond as we can't readily see what is the new email and what is the original. Cheers, David > The byte code for main() in case 1 is as follows. The strange thing is > that NullPointerException is also not thrown at runtime. > > public void main(java.lang.String[]) throws java.lang.Exception; > descriptor: ([Ljava/lang/String;)V > flags: ACC_PUBLIC > Code: > stack=3, locals=2, args_size=2 > 0: new #2 // class Search > 3: dup > 4: aload_0 > 5: monitorenter > 6: monitorenter > 7: monitorexit > 8: aload_0 > 9: monitorexit > 10: return > Exceptions: > throws java.lang.Exception > > ??: HotSpot and IBM's J9 behave quite differently when processing > monitorenters and monitorexits > > I have tested several programs (in Jimple) and found that HotSpot and > J9 match monitorenters and monitorexits quite differently. Verifiers > should play more important roles here. > > (1) Test the next program (r2 is not initizlied) on HotSpot and J9. > J9 throw out a verifier error, while HotSpot does not. It seems that > HotSpot's verifier forgets to check whether a monitored object is > initialized. > > public class Search extends java.lang.Object { public void () > { > Search r0; > r0 := @this: Search; > specialinvoke r0.()>(); > return; > } > public void main(java.lang.String[]) throws java.lang.Exception > { > Search r0; > Search r2; > java.lang.String[] r1; > r0 := @this: Search; > r1 := @parameter0: java.lang.String[]; > r2 = new Search; > > entermonitor r2; > entermonitor r0; > exitmonitor r2; > exitmonitor r0; > return; > } > } > > (2) Test the next program on HotSpot and J9, and both do not report > any errors. However, I guess the order in the program (entermonitor > r2; => entermonitor r0; => exitmonitor r2; => exitmonitor r0;) > violates the situation of "structured locking" (Structured locking is > the situation when, during a method invocation, every exit on a given > monitor matches a preceding entry on that monitor, see the > specification > https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-2.html#jvms-2. > 11.10) > ? > Actually, the words (every exit on a given monitor matches a preceding > entry on that monitor) are not quite clear as for me. Otherwise the > first rule (The number of monitor entries performed by T on M during a > method invocation must equal the number of monitor exits performed by > T on M during the method invocation whether the method invocation > completes normally or > abruptly.) is sufficient. > > public class Search extends java.lang.Object { > > public void () > { > Search r0; > r0 := @this: Search; > specialinvoke r0.()>(); > return; > } > > public void main(java.lang.String[]) throws java.lang.Exception > { > Search r0; > Search r2; > java.lang.String[] r1; > r0 := @this: Search; > r1 := @parameter0: java.lang.String[]; > r2 = new Search; > specialinvoke r2.()>(); > entermonitor r2; > entermonitor r0; > exitmonitor r2; > exitmonitor r0; > return; > } > } > > (3) The next program enters monitor in and exits it in main(). > HotSpot throws a runtime exception, while J9 does not. Should this > program be rejected by the verifiers? > > public class Search extends java.lang.Object { > > public void () > { > Search r0; > r0 := @this: Search; > specialinvoke r0.()>(); > entermonitor r0; > return; > } > > public void main(java.lang.String[]) throws java.lang.Exception > { > Search r0; > Search r2; > java.lang.String[] r1; > r0 := @this: Search; > r1 := @parameter0: java.lang.String[]; > r2 = new Search; > specialinvoke r2.()>(); > exitmonitor r2; > exitmonitor r0; > return; > } > } > > From ioi.lam at oracle.com Wed May 17 23:53:38 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 17 May 2017 16:53:38 -0700 Subject: RFR 8166848: Performance bug: SystemDictionary - optimization In-Reply-To: <1eb0f3aa-965a-62ab-cd59-b4ea44a81db2@oracle.com> References: <249af7dc-a3a4-52b4-ca6d-ffdb7ec13764@oracle.com> <1cc33d02-6fca-7f67-34ff-0fcc8c309cad@oracle.com> <591C7DAA.6070801@oracle.com> <1eb0f3aa-965a-62ab-cd59-b4ea44a81db2@oracle.com> Message-ID: <591CE282.4010607@oracle.com> Hi Coleen, This looks much cleaner. Thanks for doing it. - Ioi On 5/17/17 2:02 PM, coleen.phillimore at oracle.com wrote: > > Ioi, Thanks for pushing me to remove this duplication. How's this? > I reran with runThese and -XX:+VerifyBeforeGC, and built product. > > http://cr.openjdk.java.net/~coleenp/8166848.03/webrev > > thanks, > Coleen > > On 5/17/17 12:43 PM, Ioi Lam wrote: >> I know many people are not fans of macros or templates, but the >> hashtable iteration code is repeated in many places. >> >> For example, the verification code is the same across these two tables: >> >> 352 void PackageEntryTable::verify() { >> 353 int element_count = 0; >> 354 int max_bucket_count = 0; >> 355 for (int index = 0; index < table_size(); index++) { >> 356 int bucket_count = 0; >> 357 for (PackageEntry* probe = bucket(index); >> 358 probe != NULL; >> 359 probe = probe->next()) { >> 360 probe->verify(); >> 361 element_count++; >> 362 bucket_count++; >> 363 } >> 364 max_bucket_count = MAX2(max_bucket_count, bucket_count); >> 365 } >> 366 guarantee(number_of_entries() == element_count, >> 367 "Verify of Package Entry Table failed"); >> 368 DEBUG_ONLY(verify_lookup_length(max_bucket_count, "Package >> Entry Table")); >> 369 } >> >> 509 void ModuleEntryTable::verify() { >> 510 int element_count = 0; >> 511 int max_bucket_count = 0; >> 512 for (int i = 0; i < table_size(); i++) { >> 513 int bucket_count = 0; >> 514 for (ModuleEntry* probe = bucket(i); >> 515 probe != NULL; >> 516 probe = probe->next()) { >> 517 probe->verify(); >> 518 element_count++; >> 519 bucket_count++; >> 520 } >> 521 max_bucket_count = MAX2(max_bucket_count, bucket_count); >> 522 } >> 523 guarantee(number_of_entries() == element_count, >> 524 "Verify of Module Entry Table failed"); >> 525 DEBUG_ONLY(verify_lookup_length(max_bucket_count, "Module >> Entry Table")); >> 526 } >> >> >> and void Dictionary::verify() only has a bit more code inside the >> inner loop. How about using a template here. The following is >> probably not the best way to do it (but it compiles :-) >> >> #include >> >> class Hashtable { >> public: >> template void do_verify() { >> for (int i=0; i<10; i++) { // pseudo code for the table iteration >> void *probe = NULL; // pseudo code for getting an entry >> Table::verify_entry((Entry*)probe); >> } >> } >> }; >> >> class MyHashentry {/*...*/}; >> >> class MyHashtable : public Hashtable { >> public: >> void verify() { >> do_verify(); >> } >> static void verify_entry(MyHashentry *probe) { >> // ... >> } >> }; >> >> >> Thanks >> - Ioi >> >> >> On 5/15/17 5:30 AM, coleen.phillimore at oracle.com wrote: >>> >>>> Summary: Check instead that a bucket isn't 10x the average >>>> >>>> See bug and linked bugs for more details. >>>> >>>> Tested with RBT nightly tests (tier2-5). >>>> >>>> open webrev at http://cr.openjdk.java.net/~coleenp/8166848.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8166848 >>>> >>>> Thanks, >>>> Coleen >>> >> > From coleen.phillimore at oracle.com Thu May 18 00:00:50 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 17 May 2017 20:00:50 -0400 Subject: RFR 8166848: Performance bug: SystemDictionary - optimization In-Reply-To: <591CE282.4010607@oracle.com> References: <249af7dc-a3a4-52b4-ca6d-ffdb7ec13764@oracle.com> <1cc33d02-6fca-7f67-34ff-0fcc8c309cad@oracle.com> <591C7DAA.6070801@oracle.com> <1eb0f3aa-965a-62ab-cd59-b4ea44a81db2@oracle.com> <591CE282.4010607@oracle.com> Message-ID: <81de812f-8077-d42c-276b-f19205f5c774@oracle.com> Thanks, Ioi. Coleen On 5/17/17 7:53 PM, Ioi Lam wrote: > Hi Coleen, > > This looks much cleaner. Thanks for doing it. > > - Ioi > > On 5/17/17 2:02 PM, coleen.phillimore at oracle.com wrote: >> >> Ioi, Thanks for pushing me to remove this duplication. How's this? >> I reran with runThese and -XX:+VerifyBeforeGC, and built product. >> >> http://cr.openjdk.java.net/~coleenp/8166848.03/webrev >> >> thanks, >> Coleen >> >> On 5/17/17 12:43 PM, Ioi Lam wrote: >>> I know many people are not fans of macros or templates, but the >>> hashtable iteration code is repeated in many places. >>> >>> For example, the verification code is the same across these two tables: >>> >>> 352 void PackageEntryTable::verify() { >>> 353 int element_count = 0; >>> 354 int max_bucket_count = 0; >>> 355 for (int index = 0; index < table_size(); index++) { >>> 356 int bucket_count = 0; >>> 357 for (PackageEntry* probe = bucket(index); >>> 358 probe != NULL; >>> 359 probe = probe->next()) { >>> 360 probe->verify(); >>> 361 element_count++; >>> 362 bucket_count++; >>> 363 } >>> 364 max_bucket_count = MAX2(max_bucket_count, bucket_count); >>> 365 } >>> 366 guarantee(number_of_entries() == element_count, >>> 367 "Verify of Package Entry Table failed"); >>> 368 DEBUG_ONLY(verify_lookup_length(max_bucket_count, "Package >>> Entry Table")); >>> 369 } >>> >>> 509 void ModuleEntryTable::verify() { >>> 510 int element_count = 0; >>> 511 int max_bucket_count = 0; >>> 512 for (int i = 0; i < table_size(); i++) { >>> 513 int bucket_count = 0; >>> 514 for (ModuleEntry* probe = bucket(i); >>> 515 probe != NULL; >>> 516 probe = probe->next()) { >>> 517 probe->verify(); >>> 518 element_count++; >>> 519 bucket_count++; >>> 520 } >>> 521 max_bucket_count = MAX2(max_bucket_count, bucket_count); >>> 522 } >>> 523 guarantee(number_of_entries() == element_count, >>> 524 "Verify of Module Entry Table failed"); >>> 525 DEBUG_ONLY(verify_lookup_length(max_bucket_count, "Module >>> Entry Table")); >>> 526 } >>> >>> >>> and void Dictionary::verify() only has a bit more code inside the >>> inner loop. How about using a template here. The following is >>> probably not the best way to do it (but it compiles :-) >>> >>> #include >>> >>> class Hashtable { >>> public: >>> template void do_verify() { >>> for (int i=0; i<10; i++) { // pseudo code for the table iteration >>> void *probe = NULL; // pseudo code for getting an entry >>> Table::verify_entry((Entry*)probe); >>> } >>> } >>> }; >>> >>> class MyHashentry {/*...*/}; >>> >>> class MyHashtable : public Hashtable { >>> public: >>> void verify() { >>> do_verify(); >>> } >>> static void verify_entry(MyHashentry *probe) { >>> // ... >>> } >>> }; >>> >>> >>> Thanks >>> - Ioi >>> >>> >>> On 5/15/17 5:30 AM, coleen.phillimore at oracle.com wrote: >>>> >>>>> Summary: Check instead that a bucket isn't 10x the average >>>>> >>>>> See bug and linked bugs for more details. >>>>> >>>>> Tested with RBT nightly tests (tier2-5). >>>>> >>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8166848.01/webrev >>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8166848 >>>>> >>>>> Thanks, >>>>> Coleen >>>> >>> >> > From david.holmes at oracle.com Thu May 18 00:29:00 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 18 May 2017 10:29:00 +1000 Subject: =?UTF-8?Q?Re:_=e7=ad=94=e5=a4=8d:_HotSpot_and_IBM's_J9_behave_quite?= =?UTF-8?Q?_differently_when_processing_monitorenters_and_monitorexits?= In-Reply-To: <005201d2cf5a$aa2e8f30$fe8bad90$@cs.sjtu.edu.cn> References: <005101d2cf53$c8c16b40$5a4441c0$@cs.sjtu.edu.cn> <656dcb18-e65a-4c48-a6bc-cf00fe021af5@oracle.com> <005201d2cf5a$aa2e8f30$fe8bad90$@cs.sjtu.edu.cn> Message-ID: On 18/05/2017 8:12 AM, ??? wrote: > Thank you, David. I have seen from the specification that structured locking > enforcement is optional. The second and the third ones are cases of > structured/nested lockings. Will non-nested locking sequences raise > deadlocks? Of course it is a different topic, while it might be better if it > can be kicked out earlier from the specification/JVM. Non-nested locking doesn't necessarily lead to deadlocks - you just need a different locking order for that (even if properly nested). There are locking patterns that rely on the ability to lock and unlock in different order ie chained-locking for walking linked-lists A->B->C->D: - lock A, lock B, unlock A, lock C, unlock B, lock D, unlock C ... The VM spec allows a little flexibility in how the monitor bytecodes can be used compared to the Java programming language. That's not something that will change. > The first example is still a problem. It seems that HotSpot allows to > monitor a pure object reference without initialized (Is it true? How can > this checking be omitted?). J9 reports a verifyerror as follows. > > Exception in thread "main" java.lang.VerifyError: JVMVRFY012 stack shape > inconsistent; class=Search, method=main([Ljava/lang/String;)V, pc=6 > Exception Details: > Location: > Search.main([Ljava/lang/String;)V @6: JBmonitorenter > Reason: > Type 'uninitialized' (current frame, stack[1]) is not assignable to > 'java/lang/Object' > Current Frame: > bci: @6 > flags: { } > locals: { 'Search', '[Ljava/lang/String;' } > stack: { 'uninitialized', 'uninitialized' } > at T.main(T.java:4) Yes I think this may be a bug in hotspot. The type-checking for the monitor bytecodes requires a matching type of reference on the operand stack - but "uninitialized" does not match Object, as J9 reports. But I'm not an expert on this aspect of verification so I may not be interpreting it correctly. More below ... > -----????----- > ???: David Holmes [mailto:david.holmes at oracle.com] > ????: 2017?5?17? 14:41 > ???: ??? ; hotspot-runtime-dev at openjdk.java.net > ??: Re: HotSpot and IBM's J9 behave quite differently when processing > monitorenters and monitorexits > > Hi, > On 18/05/2017 7:23 AM, ??? wrote: >> Am I wrong? > > I will look at each situation in detail when I get a chance but structured > locking enforcement is optional. Also balancing the number of locks and > unlocks in a frame does not mean they can't be locked and unlocked in a > non-nested fashion - just that by the end the number of unlocks matches the > number of locks. > > BTW the way you respond to these emails, as if having a conversation with > yourself, makes it difficult to respond as we can't readily see what is the > new email and what is the original. > > Cheers, > David > >> The byte code for main() in case 1 is as follows. The strange thing is >> that NullPointerException is also not thrown at runtime. That is strange as it does for the normal obvious case of using synchronized(o) when o is null. >> >> public void main(java.lang.String[]) throws java.lang.Exception; >> descriptor: ([Ljava/lang/String;)V >> flags: ACC_PUBLIC >> Code: >> stack=3, locals=2, args_size=2 >> 0: new #2 // class Search >> 3: dup >> 4: aload_0 >> 5: monitorenter >> 6: monitorenter >> 7: monitorexit >> 8: aload_0 >> 9: monitorexit >> 10: return >> Exceptions: >> throws java.lang.Exception >> >> ??: HotSpot and IBM's J9 behave quite differently when processing >> monitorenters and monitorexits >> >> I have tested several programs (in Jimple) and found that HotSpot and >> J9 match monitorenters and monitorexits quite differently. Verifiers >> should play more important roles here. The job of the verifier is to establish some basic guarantees for the JVM to then operate under. The verifier plays no role in checking how monitorenter/exit are used in combination, only that each individual bytecode meets some basic type constraints. >> >> (1) Test the next program (r2 is not initizlied) on HotSpot and J9. >> J9 throw out a verifier error, while HotSpot does not. It seems that >> HotSpot's verifier forgets to check whether a monitored object is >> initialized. >> >> public class Search extends java.lang.Object { public void () >> { >> Search r0; >> r0 := @this: Search; >> specialinvoke r0.()>(); >> return; >> } >> public void main(java.lang.String[]) throws java.lang.Exception >> { >> Search r0; >> Search r2; >> java.lang.String[] r1; >> r0 := @this: Search; >> r1 := @parameter0: java.lang.String[]; >> r2 = new Search; >> >> entermonitor r2; >> entermonitor r0; >> exitmonitor r2; >> exitmonitor r0; >> return; >> } >> } Verification was covered above. >> >> (2) Test the next program on HotSpot and J9, and both do not report >> any errors. However, I guess the order in the program (entermonitor >> r2; => entermonitor r0; => exitmonitor r2; => exitmonitor r0;) >> violates the situation of "structured locking" (Structured locking is >> the situation when, during a method invocation, every exit on a given >> monitor matches a preceding entry on that monitor, see the >> specification >> https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-2.html#jvms-2. >> 11.10) >> ? No it doesn't violate structured locking as the number of enters and exits match, and there is always an enter before an exit. >> Actually, the words (every exit on a given monitor matches a preceding >> entry on that monitor) are not quite clear as for me. Otherwise the >> first rule (The number of monitor entries performed by T on M during a >> method invocation must equal the number of monitor exits performed by >> T on M during the method invocation whether the method invocation >> completes normally or abruptly.) is sufficient. The number of enters and exits must not only match/balance, but there must be an enter before a corresponding exit. >> >> public class Search extends java.lang.Object { >> >> public void () >> { >> Search r0; >> r0 := @this: Search; >> specialinvoke r0.()>(); >> return; >> } >> >> public void main(java.lang.String[]) throws java.lang.Exception >> { >> Search r0; >> Search r2; >> java.lang.String[] r1; >> r0 := @this: Search; >> r1 := @parameter0: java.lang.String[]; >> r2 = new Search; >> specialinvoke r2.()>(); >> entermonitor r2; >> entermonitor r0; >> exitmonitor r2; >> exitmonitor r0; >> return; >> } >> } >> >> (3) The next program enters monitor in and exits it in main(). >> HotSpot throws a runtime exception, while J9 does not. Should this >> program be rejected by the verifiers? No this does not violate any verification rules. The runtime behaviour depends on whether structured locking is enforced or not.(Even in hotspot there can be differences between interpreted and jitted code). Hope that helps clarify things. David ----- >> >> public class Search extends java.lang.Object { >> >> public void () >> { >> Search r0; >> r0 := @this: Search; >> specialinvoke r0.()>(); >> entermonitor r0; >> return; >> } >> >> public void main(java.lang.String[]) throws java.lang.Exception >> { >> Search r0; >> Search r2; >> java.lang.String[] r1; >> r0 := @this: Search; >> r1 := @parameter0: java.lang.String[]; >> r2 = new Search; >> specialinvoke r2.()>(); >> exitmonitor r2; >> exitmonitor r0; >> return; >> } >> } >> >> > From david.holmes at oracle.com Thu May 18 02:07:25 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 18 May 2017 12:07:25 +1000 Subject: RFR (jdk10): JDK-8180175: ObjectSynchronizer only needs to iterate in-use monitors In-Reply-To: References: <89399a72-0263-5ebd-5fda-82f404fbe2ca@redhat.com> <8d75eef3-c143-dd75-6362-6203c7fb175f@redhat.com> <3a0f4490-646c-8004-6272-180ef198b67e@oracle.com> <13484ee1-5c37-6971-028d-2ddf0b0f2845@redhat.com> Message-ID: Roman, I just pushed this but now I think there is a problem with this optimization. When a thread terminates, omFlush is used to transfer the thread's inUseList to the global inUseList. This happens after the thread has been removed from the _threads_list and can run concurrently with a safepoint. This means that there is a window of time where the monitors in the thread's inUseList will not be seen by the safepoint oops_do processing: - the thread oops_do won't be called because the thread is not in the thread's list - the monitors are not yet in the global in-use-list Did I overlook something? David On 18/05/2017 7:45 AM, David Holmes wrote: > Hi Roman, > > I can sponsor this for you. BTW Robbin should also be listed as a > reviewer - I'll add him. > > David > > On 18/05/2017 7:38 AM, Roman Kennke wrote: >> Am 17.05.2017 um 23:35 schrieb David Holmes: >>> On 18/05/2017 7:25 AM, Roman Kennke wrote: >>>> >>>>> Looks good to me. Some comments will be nice. >>>> Like this: >>>> >>>> http://cr.openjdk.java.net/~rkennke/8180175/webrev.01/ >>>> >>> >>> + // and the remaing global monitors in >>> ObjectSynchronizer::oops_do(). >>> >>> Typo: remaing >> >> Ok. Fixed. >> Final webrev: >> >> http://cr.openjdk.java.net/~rkennke/8180175/webrev.02/ >> >> >> If it's ok, I'm going to need a sponsor. ? >> >> Thanks, Roman >> From serguei.spitsyn at oracle.com Thu May 18 02:08:29 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 17 May 2017 19:08:29 -0700 Subject: RFR 8166848: Performance bug: SystemDictionary - optimization In-Reply-To: <591CE282.4010607@oracle.com> References: <249af7dc-a3a4-52b4-ca6d-ffdb7ec13764@oracle.com> <1cc33d02-6fca-7f67-34ff-0fcc8c309cad@oracle.com> <591C7DAA.6070801@oracle.com> <1eb0f3aa-965a-62ab-cd59-b4ea44a81db2@oracle.com> <591CE282.4010607@oracle.com> Message-ID: <0ed9426f-e0bd-af45-3d52-4df2912ea44e@oracle.com> +1 Thanks, Serguei On 5/17/17 16:53, Ioi Lam wrote: > Hi Coleen, > > This looks much cleaner. Thanks for doing it. > > - Ioi > > On 5/17/17 2:02 PM, coleen.phillimore at oracle.com wrote: >> >> Ioi, Thanks for pushing me to remove this duplication. How's this? >> I reran with runThese and -XX:+VerifyBeforeGC, and built product. >> >> http://cr.openjdk.java.net/~coleenp/8166848.03/webrev >> >> thanks, >> Coleen >> >> On 5/17/17 12:43 PM, Ioi Lam wrote: >>> I know many people are not fans of macros or templates, but the >>> hashtable iteration code is repeated in many places. >>> >>> For example, the verification code is the same across these two tables: >>> >>> 352 void PackageEntryTable::verify() { >>> 353 int element_count = 0; >>> 354 int max_bucket_count = 0; >>> 355 for (int index = 0; index < table_size(); index++) { >>> 356 int bucket_count = 0; >>> 357 for (PackageEntry* probe = bucket(index); >>> 358 probe != NULL; >>> 359 probe = probe->next()) { >>> 360 probe->verify(); >>> 361 element_count++; >>> 362 bucket_count++; >>> 363 } >>> 364 max_bucket_count = MAX2(max_bucket_count, bucket_count); >>> 365 } >>> 366 guarantee(number_of_entries() == element_count, >>> 367 "Verify of Package Entry Table failed"); >>> 368 DEBUG_ONLY(verify_lookup_length(max_bucket_count, "Package >>> Entry Table")); >>> 369 } >>> >>> 509 void ModuleEntryTable::verify() { >>> 510 int element_count = 0; >>> 511 int max_bucket_count = 0; >>> 512 for (int i = 0; i < table_size(); i++) { >>> 513 int bucket_count = 0; >>> 514 for (ModuleEntry* probe = bucket(i); >>> 515 probe != NULL; >>> 516 probe = probe->next()) { >>> 517 probe->verify(); >>> 518 element_count++; >>> 519 bucket_count++; >>> 520 } >>> 521 max_bucket_count = MAX2(max_bucket_count, bucket_count); >>> 522 } >>> 523 guarantee(number_of_entries() == element_count, >>> 524 "Verify of Module Entry Table failed"); >>> 525 DEBUG_ONLY(verify_lookup_length(max_bucket_count, "Module >>> Entry Table")); >>> 526 } >>> >>> >>> and void Dictionary::verify() only has a bit more code inside the >>> inner loop. How about using a template here. The following is >>> probably not the best way to do it (but it compiles :-) >>> >>> #include >>> >>> class Hashtable { >>> public: >>> template void do_verify() { >>> for (int i=0; i<10; i++) { // pseudo code for the table iteration >>> void *probe = NULL; // pseudo code for getting an entry >>> Table::verify_entry((Entry*)probe); >>> } >>> } >>> }; >>> >>> class MyHashentry {/*...*/}; >>> >>> class MyHashtable : public Hashtable { >>> public: >>> void verify() { >>> do_verify(); >>> } >>> static void verify_entry(MyHashentry *probe) { >>> // ... >>> } >>> }; >>> >>> >>> Thanks >>> - Ioi >>> >>> >>> On 5/15/17 5:30 AM, coleen.phillimore at oracle.com wrote: >>>> >>>>> Summary: Check instead that a bucket isn't 10x the average >>>>> >>>>> See bug and linked bugs for more details. >>>>> >>>>> Tested with RBT nightly tests (tier2-5). >>>>> >>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8166848.01/webrev >>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8166848 >>>>> >>>>> Thanks, >>>>> Coleen >>>> >>> >> > From david.holmes at oracle.com Thu May 18 03:23:09 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 18 May 2017 13:23:09 +1000 Subject: =?UTF-8?Q?Re:_=e7=ad=94=e5=a4=8d:_HotSpot_and_IBM's_J9_behave_quite?= =?UTF-8?Q?_differently_when_processing_monitorenters_and_monitorexits?= In-Reply-To: References: <005101d2cf53$c8c16b40$5a4441c0$@cs.sjtu.edu.cn> <656dcb18-e65a-4c48-a6bc-cf00fe021af5@oracle.com> <005201d2cf5a$aa2e8f30$fe8bad90$@cs.sjtu.edu.cn> Message-ID: <5a6a2de2-eb43-985d-bfc7-18c3fa7ab0ee@oracle.com> One correction ... On 18/05/2017 10:29 AM, David Holmes wrote: > On 18/05/2017 8:12 AM, ??? wrote: >> Thank you, David. I have seen from the specification that structured >> locking >> enforcement is optional. The second and the third ones are cases of >> structured/nested lockings. Will non-nested locking sequences raise >> deadlocks? Of course it is a different topic, while it might be better >> if it >> can be kicked out earlier from the specification/JVM. > > Non-nested locking doesn't necessarily lead to deadlocks - you just need > a different locking order for that (even if properly nested). There are > locking patterns that rely on the ability to lock and unlock in > different order ie chained-locking for walking linked-lists A->B->C->D: > - lock A, lock B, unlock A, lock C, unlock B, lock D, unlock C ... > > The VM spec allows a little flexibility in how the monitor bytecodes can > be used compared to the Java programming language. That's not something > that will change. > >> The first example is still a problem. It seems that HotSpot allows to >> monitor a pure object reference without initialized (Is it true? How can >> this checking be omitted?). J9 reports a verifyerror as follows. >> >> Exception in thread "main" java.lang.VerifyError: JVMVRFY012 stack shape >> inconsistent; class=Search, method=main([Ljava/lang/String;)V, pc=6 >> Exception Details: >> Location: >> Search.main([Ljava/lang/String;)V @6: JBmonitorenter >> Reason: >> Type 'uninitialized' (current frame, stack[1]) is not assignable to >> 'java/lang/Object' >> Current Frame: >> bci: @6 >> flags: { } >> locals: { 'Search', '[Ljava/lang/String;' } >> stack: { 'uninitialized', 'uninitialized' } >> at T.main(T.java:4) > > Yes I think this may be a bug in hotspot. The type-checking for the > monitor bytecodes requires a matching type of reference on the operand > stack - but "uninitialized" does not match Object, as J9 reports. But > I'm not an expert on this aspect of verification so I may not be > interpreting it correctly. > > More below ... > >> -----????----- >> ???: David Holmes [mailto:david.holmes at oracle.com] >> ????: 2017?5?17? 14:41 >> ???: ??? ; >> hotspot-runtime-dev at openjdk.java.net >> ??: Re: HotSpot and IBM's J9 behave quite differently when processing >> monitorenters and monitorexits >> >> Hi, >> On 18/05/2017 7:23 AM, ??? wrote: >>> Am I wrong? >> >> I will look at each situation in detail when I get a chance but >> structured >> locking enforcement is optional. Also balancing the number of locks and >> unlocks in a frame does not mean they can't be locked and unlocked in a >> non-nested fashion - just that by the end the number of unlocks >> matches the >> number of locks. >> >> BTW the way you respond to these emails, as if having a conversation with >> yourself, makes it difficult to respond as we can't readily see what >> is the >> new email and what is the original. >> >> Cheers, >> David >> >>> The byte code for main() in case 1 is as follows. The strange thing is >>> that NullPointerException is also not thrown at runtime. > > That is strange as it does for the normal obvious case of using > synchronized(o) when o is null. Ah - it isn't null it just an object for which the constructor has not been run. The runtime can't tell the difference between a pointer to a valid initialized object, and a pointer to an uninitialized chunk of memory. >>> >>> public void main(java.lang.String[]) throws java.lang.Exception; >>> descriptor: ([Ljava/lang/String;)V >>> flags: ACC_PUBLIC >>> Code: >>> stack=3, locals=2, args_size=2 >>> 0: new #2 // class Search This allocated an object - hence no null reference. But this is what verification should have complained about. David ----- >>> 3: dup >>> 4: aload_0 >>> 5: monitorenter >>> 6: monitorenter >>> 7: monitorexit >>> 8: aload_0 >>> 9: monitorexit >>> 10: return >>> Exceptions: >>> throws java.lang.Exception >>> >>> ??: HotSpot and IBM's J9 behave quite differently when processing >>> monitorenters and monitorexits >>> >>> I have tested several programs (in Jimple) and found that HotSpot and >>> J9 match monitorenters and monitorexits quite differently. Verifiers >>> should play more important roles here. > > The job of the verifier is to establish some basic guarantees for the > JVM to then operate under. The verifier plays no role in checking how > monitorenter/exit are used in combination, only that each individual > bytecode meets some basic type constraints. > >>> >>> (1) Test the next program (r2 is not initizlied) on HotSpot and J9. >>> J9 throw out a verifier error, while HotSpot does not. It seems that >>> HotSpot's verifier forgets to check whether a monitored object is >>> initialized. >>> >>> public class Search extends java.lang.Object { public void () >>> { >>> Search r0; >>> r0 := @this: Search; >>> specialinvoke r0.()>(); >>> return; >>> } >>> public void main(java.lang.String[]) throws java.lang.Exception >>> { >>> Search r0; >>> Search r2; >>> java.lang.String[] r1; >>> r0 := @this: Search; >>> r1 := @parameter0: java.lang.String[]; >>> r2 = new Search; >>> >>> entermonitor r2; >>> entermonitor r0; >>> exitmonitor r2; >>> exitmonitor r0; >>> return; >>> } >>> } > > Verification was covered above. > >>> >>> (2) Test the next program on HotSpot and J9, and both do not report >>> any errors. However, I guess the order in the program (entermonitor >>> r2; => entermonitor r0; => exitmonitor r2; => exitmonitor r0;) >>> violates the situation of "structured locking" (Structured locking is >>> the situation when, during a method invocation, every exit on a given >>> monitor matches a preceding entry on that monitor, see the >>> specification >>> https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-2.html#jvms-2. >>> 11.10) >>> ? > > No it doesn't violate structured locking as the number of enters and > exits match, and there is always an enter before an exit. > >>> Actually, the words (every exit on a given monitor matches a preceding >>> entry on that monitor) are not quite clear as for me. Otherwise the >>> first rule (The number of monitor entries performed by T on M during a >>> method invocation must equal the number of monitor exits performed by >>> T on M during the method invocation whether the method invocation >>> completes normally or abruptly.) is sufficient. > > The number of enters and exits must not only match/balance, but there > must be an enter before a corresponding exit. > >>> >>> public class Search extends java.lang.Object { >>> >>> public void () >>> { >>> Search r0; >>> r0 := @this: Search; >>> specialinvoke r0.()>(); >>> return; >>> } >>> >>> public void main(java.lang.String[]) throws java.lang.Exception >>> { >>> Search r0; >>> Search r2; >>> java.lang.String[] r1; >>> r0 := @this: Search; >>> r1 := @parameter0: java.lang.String[]; >>> r2 = new Search; >>> specialinvoke r2.()>(); >>> entermonitor r2; >>> entermonitor r0; >>> exitmonitor r2; >>> exitmonitor r0; >>> return; >>> } >>> } >>> >>> (3) The next program enters monitor in and exits it in main(). >>> HotSpot throws a runtime exception, while J9 does not. Should this >>> program be rejected by the verifiers? > > No this does not violate any verification rules. The runtime behaviour > depends on whether structured locking is enforced or not.(Even in > hotspot there can be differences between interpreted and jitted code). > > Hope that helps clarify things. > > David > ----- > >>> >>> public class Search extends java.lang.Object { >>> >>> public void () >>> { >>> Search r0; >>> r0 := @this: Search; >>> specialinvoke r0.()>(); >>> entermonitor r0; >>> return; >>> } >>> >>> public void main(java.lang.String[]) throws java.lang.Exception >>> { >>> Search r0; >>> Search r2; >>> java.lang.String[] r1; >>> r0 := @this: Search; >>> r1 := @parameter0: java.lang.String[]; >>> r2 = new Search; >>> specialinvoke r2.()>(); >>> exitmonitor r2; >>> exitmonitor r0; >>> return; >>> } >>> } >>> >>> >> From chenyt at cs.sjtu.edu.cn Thu May 18 04:18:18 2017 From: chenyt at cs.sjtu.edu.cn (=?gb2312?B?s8LT6s2k?=) Date: Wed, 17 May 2017 21:18:18 -0700 Subject: =?gb2312?B?tPC4tDogtPC4tDogSG90U3BvdCBhbmQgSUJNJ3MgSjkgYmVoYXZlIA==?= =?gb2312?B?cXVpdGUgZGlmZmVyZW50bHkgd2hlbiBwcm9jZXNzaW5nIG1vbml0bw==?= =?gb2312?B?cmVudGVycyBhbmQgbW9uaXRvcmV4aXRz?= In-Reply-To: <5a6a2de2-eb43-985d-bfc7-18c3fa7ab0ee@oracle.com> References: <005101d2cf53$c8c16b40$5a4441c0$@cs.sjtu.edu.cn> <656dcb18-e65a-4c48-a6bc-cf00fe021af5@oracle.com> <005201d2cf5a$aa2e8f30$fe8bad90$@cs.sjtu.edu.cn> <5a6a2de2-eb43-985d-bfc7-18c3fa7ab0ee@oracle.com> Message-ID: <000401d2cf8d$c8352470$589f6d50$@cs.sjtu.edu.cn> Hi, David, I still did not catch why it happens... I just read the specification and wrote some tests in order to understand some paragraphs better, and then had questions. Will the HotSpot's verifier detect such kind of uninitialized monitored objects in future? Regards, Yuting -----????----- ???: David Holmes [mailto:david.holmes at oracle.com] ????: 2017?5?17? 20:23 ???: ??? ; hotspot-runtime-dev at openjdk.java.net ??: Re: ??: HotSpot and IBM's J9 behave quite differently when processing monitorenters and monitorexits One correction ... On 18/05/2017 10:29 AM, David Holmes wrote: > On 18/05/2017 8:12 AM, ??? wrote: >> Thank you, David. I have seen from the specification that structured >> locking enforcement is optional. The second and the third ones are >> cases of structured/nested lockings. Will non-nested locking >> sequences raise deadlocks? Of course it is a different topic, while >> it might be better if it can be kicked out earlier from the >> specification/JVM. > > Non-nested locking doesn't necessarily lead to deadlocks - you just > need a different locking order for that (even if properly nested). > There are locking patterns that rely on the ability to lock and unlock > in different order ie chained-locking for walking linked-lists A->B->C->D: > - lock A, lock B, unlock A, lock C, unlock B, lock D, unlock C ... > > The VM spec allows a little flexibility in how the monitor bytecodes > can be used compared to the Java programming language. That's not > something that will change. > >> The first example is still a problem. It seems that HotSpot allows to >> monitor a pure object reference without initialized (Is it true? How >> can this checking be omitted?). J9 reports a verifyerror as follows. >> >> Exception in thread "main" java.lang.VerifyError: JVMVRFY012 stack >> shape inconsistent; class=Search, method=main([Ljava/lang/String;)V, >> pc=6 Exception Details: >> Location: >> Search.main([Ljava/lang/String;)V @6: JBmonitorenter >> Reason: >> Type 'uninitialized' (current frame, stack[1]) is not assignable >> to 'java/lang/Object' >> Current Frame: >> bci: @6 >> flags: { } >> locals: { 'Search', '[Ljava/lang/String;' } >> stack: { 'uninitialized', 'uninitialized' } >> at T.main(T.java:4) > > Yes I think this may be a bug in hotspot. The type-checking for the > monitor bytecodes requires a matching type of reference on the operand > stack - but "uninitialized" does not match Object, as J9 reports. But > I'm not an expert on this aspect of verification so I may not be > interpreting it correctly. > > More below ... > >> -----????----- >> ???: David Holmes [mailto:david.holmes at oracle.com] >> ????: 2017?5?17? 14:41 >> ???: ??? ; >> hotspot-runtime-dev at openjdk.java.net >> ??: Re: HotSpot and IBM's J9 behave quite differently when processing >> monitorenters and monitorexits >> >> Hi, >> On 18/05/2017 7:23 AM, ??? wrote: >>> Am I wrong? >> >> I will look at each situation in detail when I get a chance but >> structured locking enforcement is optional. Also balancing the number >> of locks and unlocks in a frame does not mean they can't be locked >> and unlocked in a non-nested fashion - just that by the end the >> number of unlocks matches the number of locks. >> >> BTW the way you respond to these emails, as if having a conversation >> with yourself, makes it difficult to respond as we can't readily see >> what is the new email and what is the original. >> >> Cheers, >> David >> >>> The byte code for main() in case 1 is as follows. The strange thing >>> is that NullPointerException is also not thrown at runtime. > > That is strange as it does for the normal obvious case of using > synchronized(o) when o is null. Ah - it isn't null it just an object for which the constructor has not been run. The runtime can't tell the difference between a pointer to a valid initialized object, and a pointer to an uninitialized chunk of memory. >>> >>> public void main(java.lang.String[]) throws java.lang.Exception; >>> descriptor: ([Ljava/lang/String;)V >>> flags: ACC_PUBLIC >>> Code: >>> stack=3, locals=2, args_size=2 >>> 0: new #2 // class Search This allocated an object - hence no null reference. But this is what verification should have complained about. David ----- >>> 3: dup >>> 4: aload_0 >>> 5: monitorenter >>> 6: monitorenter >>> 7: monitorexit >>> 8: aload_0 >>> 9: monitorexit >>> 10: return >>> Exceptions: >>> throws java.lang.Exception >>> >>> ??: HotSpot and IBM's J9 behave quite differently when processing >>> monitorenters and monitorexits >>> >>> I have tested several programs (in Jimple) and found that HotSpot >>> and >>> J9 match monitorenters and monitorexits quite differently. Verifiers >>> should play more important roles here. > > The job of the verifier is to establish some basic guarantees for the > JVM to then operate under. The verifier plays no role in checking how > monitorenter/exit are used in combination, only that each individual > bytecode meets some basic type constraints. > >>> >>> (1) Test the next program (r2 is not initizlied) on HotSpot and J9. >>> J9 throw out a verifier error, while HotSpot does not. It seems that >>> HotSpot's verifier forgets to check whether a monitored object is >>> initialized. >>> >>> public class Search extends java.lang.Object { public void () >>> { >>> Search r0; >>> r0 := @this: Search; >>> specialinvoke r0.()>(); >>> return; >>> } >>> public void main(java.lang.String[]) throws java.lang.Exception >>> { >>> Search r0; >>> Search r2; >>> java.lang.String[] r1; >>> r0 := @this: Search; >>> r1 := @parameter0: java.lang.String[]; >>> r2 = new Search; >>> >>> entermonitor r2; >>> entermonitor r0; >>> exitmonitor r2; >>> exitmonitor r0; >>> return; >>> } >>> } > > Verification was covered above. > >>> >>> (2) Test the next program on HotSpot and J9, and both do not report >>> any errors. However, I guess the order in the program (entermonitor >>> r2; => entermonitor r0; => exitmonitor r2; => exitmonitor r0;) >>> violates the situation of "structured locking" (Structured locking >>> is the situation when, during a method invocation, every exit on a >>> given monitor matches a preceding entry on that monitor, see the >>> specification >>> https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-2.html#jvms-2. >>> 11.10) >>> ? > > No it doesn't violate structured locking as the number of enters and > exits match, and there is always an enter before an exit. > >>> Actually, the words (every exit on a given monitor matches a >>> preceding entry on that monitor) are not quite clear as for me. >>> Otherwise the first rule (The number of monitor entries performed by >>> T on M during a method invocation must equal the number of monitor >>> exits performed by T on M during the method invocation whether the >>> method invocation completes normally or abruptly.) is sufficient. > > The number of enters and exits must not only match/balance, but there > must be an enter before a corresponding exit. > >>> >>> public class Search extends java.lang.Object { >>> >>> public void () >>> { >>> Search r0; >>> r0 := @this: Search; >>> specialinvoke r0.()>(); >>> return; >>> } >>> >>> public void main(java.lang.String[]) throws java.lang.Exception >>> { >>> Search r0; >>> Search r2; >>> java.lang.String[] r1; >>> r0 := @this: Search; >>> r1 := @parameter0: java.lang.String[]; >>> r2 = new Search; >>> specialinvoke r2.()>(); >>> entermonitor r2; >>> entermonitor r0; >>> exitmonitor r2; >>> exitmonitor r0; >>> return; >>> } >>> } >>> >>> (3) The next program enters monitor in and exits it in main(). >>> HotSpot throws a runtime exception, while J9 does not. Should this >>> program be rejected by the verifiers? > > No this does not violate any verification rules. The runtime behaviour > depends on whether structured locking is enforced or not.(Even in > hotspot there can be differences between interpreted and jitted code). > > Hope that helps clarify things. > > David > ----- > >>> >>> public class Search extends java.lang.Object { >>> >>> public void () >>> { >>> Search r0; >>> r0 := @this: Search; >>> specialinvoke r0.()>(); >>> entermonitor r0; >>> return; >>> } >>> >>> public void main(java.lang.String[]) throws java.lang.Exception >>> { >>> Search r0; >>> Search r2; >>> java.lang.String[] r1; >>> r0 := @this: Search; >>> r1 := @parameter0: java.lang.String[]; >>> r2 = new Search; >>> specialinvoke r2.()>(); >>> exitmonitor r2; >>> exitmonitor r0; >>> return; >>> } >>> } >>> >>> >> From chenyt at cs.sjtu.edu.cn Thu May 18 04:49:53 2017 From: chenyt at cs.sjtu.edu.cn (chenyt) Date: Thu, 18 May 2017 12:49:53 +0800 Subject: =?UTF-8?Q?=E7=AD=94=E5=A4=8D=3A=20HotSpot=20and=20IBM=27s=20J?= =?UTF-8?Q?=39=20behave=20quite=20differently=20when=20processing=20monito?= =?UTF-8?Q?renters=20and=20monitorexits?= In-Reply-To: <5a6a2de2-eb43-985d-bfc7-18c3fa7ab0ee@oracle.com> References: <005101d2cf53$c8c16b40$5a4441c0$@cs.sjtu.edu.cn> <656dcb18-e65a-4c48-a6bc-cf00fe021af5@oracle.com> <005201d2cf5a$aa2e8f30$fe8bad90$@cs.sjtu.edu.cn> <5a6a2de2-eb43-985d-bfc7-18c3fa7ab0ee@oracle.com> Message-ID: Hi, David, I tested the next program (let one object to be monitored be null) using J9 and HotSpot. HotSpot does not throw a NullPointerException, as the specification says. J9 looks fine (throw a NullPointerException). Am I wrong here? It is very interesting that I have found so many unexpected behaviors here. public void main(java.lang.String[]) throws java.lang.Exception; descriptor: ([Ljava/lang/String;)V flags: ACC_PUBLIC Code: stack=1, locals=2, args_size=2 0: aload_0 1: monitorenter 2: aconst_null 3: monitorenter 4: aconst_null 5: monitorexit 6: aload_0 7: monitorexit 8: return Exceptions: throws java.lang.Exception Jimple code is given as follows: public void main(java.lang.String[]) throws java.lang.Exception { Search r0; Search r2; java.lang.String[] r1; r0 := @this: Search; r1 := @parameter0: java.lang.String[]; r2 = null; entermonitor r0; entermonitor r2; exitmonitor r2; exitmonitor r0; return; } Wishes, Yuting On Thu, 18 May 2017 13:23:09 +1000, David Holmes wrote: > One correction ... > > On 18/05/2017 10:29 AM, David Holmes wrote: >> On 18/05/2017 8:12 AM, ??? wrote: >>> Thank you, David. I have seen from the specification that >>> structured >>> locking >>> enforcement is optional. The second and the third ones are cases of >>> structured/nested lockings. Will non-nested locking sequences raise >>> deadlocks? Of course it is a different topic, while it might be >>> better >>> if it >>> can be kicked out earlier from the specification/JVM. >> >> Non-nested locking doesn't necessarily lead to deadlocks - you just >> need >> a different locking order for that (even if properly nested). There >> are >> locking patterns that rely on the ability to lock and unlock in >> different order ie chained-locking for walking linked-lists >> A->B->C->D: >> - lock A, lock B, unlock A, lock C, unlock B, lock D, unlock C ... >> >> The VM spec allows a little flexibility in how the monitor bytecodes >> can >> be used compared to the Java programming language. That's not >> something >> that will change. >> >>> The first example is still a problem. It seems that HotSpot allows >>> to >>> monitor a pure object reference without initialized (Is it true? >>> How can >>> this checking be omitted?). J9 reports a verifyerror as follows. >>> >>> Exception in thread "main" java.lang.VerifyError: JVMVRFY012 stack >>> shape >>> inconsistent; class=Search, method=main([Ljava/lang/String;)V, pc=6 >>> Exception Details: >>> Location: >>> Search.main([Ljava/lang/String;)V @6: JBmonitorenter >>> Reason: >>> Type 'uninitialized' (current frame, stack[1]) is not >>> assignable to >>> 'java/lang/Object' >>> Current Frame: >>> bci: @6 >>> flags: { } >>> locals: { 'Search', '[Ljava/lang/String;' } >>> stack: { 'uninitialized', 'uninitialized' } >>> at T.main(T.java:4) >> >> Yes I think this may be a bug in hotspot. The type-checking for the >> monitor bytecodes requires a matching type of reference on the >> operand >> stack - but "uninitialized" does not match Object, as J9 reports. >> But >> I'm not an expert on this aspect of verification so I may not be >> interpreting it correctly. >> >> More below ... >> >>> -----????----- >>> ???: David Holmes [mailto:david.holmes at oracle.com] >>> ????: 2017?5?17? 14:41 >>> ???: ??? ; >>> hotspot-runtime-dev at openjdk.java.net >>> ??: Re: HotSpot and IBM's J9 behave quite differently when >>> processing >>> monitorenters and monitorexits >>> >>> Hi, >>> On 18/05/2017 7:23 AM, ??? wrote: >>>> Am I wrong? >>> >>> I will look at each situation in detail when I get a chance but >>> structured >>> locking enforcement is optional. Also balancing the number of locks >>> and >>> unlocks in a frame does not mean they can't be locked and unlocked >>> in a >>> non-nested fashion - just that by the end the number of unlocks >>> matches the >>> number of locks. >>> >>> BTW the way you respond to these emails, as if having a >>> conversation with >>> yourself, makes it difficult to respond as we can't readily see >>> what >>> is the >>> new email and what is the original. >>> >>> Cheers, >>> David >>> >>>> The byte code for main() in case 1 is as follows. The strange >>>> thing is >>>> that NullPointerException is also not thrown at runtime. >> >> That is strange as it does for the normal obvious case of using >> synchronized(o) when o is null. > > Ah - it isn't null it just an object for which the constructor has > not been run. The runtime can't tell the difference between a pointer > to a valid initialized object, and a pointer to an uninitialized > chunk > of memory. > >>>> >>>> public void main(java.lang.String[]) throws java.lang.Exception; >>>> descriptor: ([Ljava/lang/String;)V >>>> flags: ACC_PUBLIC >>>> Code: >>>> stack=3, locals=2, args_size=2 >>>> 0: new #2 // class Search > > This allocated an object - hence no null reference. But this is what > verification should have complained about. > > David > ----- > >>>> 3: dup >>>> 4: aload_0 >>>> 5: monitorenter >>>> 6: monitorenter >>>> 7: monitorexit >>>> 8: aload_0 >>>> 9: monitorexit >>>> 10: return >>>> Exceptions: >>>> throws java.lang.Exception >>>> >>>> ??: HotSpot and IBM's J9 behave quite differently when processing >>>> monitorenters and monitorexits >>>> >>>> I have tested several programs (in Jimple) and found that HotSpot >>>> and >>>> J9 match monitorenters and monitorexits quite differently. >>>> Verifiers >>>> should play more important roles here. >> >> The job of the verifier is to establish some basic guarantees for >> the >> JVM to then operate under. The verifier plays no role in checking >> how >> monitorenter/exit are used in combination, only that each individual >> bytecode meets some basic type constraints. >> >>>> >>>> (1) Test the next program (r2 is not initizlied) on HotSpot and >>>> J9. >>>> J9 throw out a verifier error, while HotSpot does not. It seems >>>> that >>>> HotSpot's verifier forgets to check whether a monitored object is >>>> initialized. >>>> >>>> public class Search extends java.lang.Object { public void >>>> () >>>> { >>>> Search r0; >>>> r0 := @this: Search; >>>> specialinvoke r0.()>(); >>>> return; >>>> } >>>> public void main(java.lang.String[]) throws java.lang.Exception >>>> { >>>> Search r0; >>>> Search r2; >>>> java.lang.String[] r1; >>>> r0 := @this: Search; >>>> r1 := @parameter0: java.lang.String[]; >>>> r2 = new Search; >>>> >>>> entermonitor r2; >>>> entermonitor r0; >>>> exitmonitor r2; >>>> exitmonitor r0; >>>> return; >>>> } >>>> } >> >> Verification was covered above. >> >>>> >>>> (2) Test the next program on HotSpot and J9, and both do not >>>> report >>>> any errors. However, I guess the order in the program >>>> (entermonitor >>>> r2; => entermonitor r0; => exitmonitor r2; => exitmonitor r0;) >>>> violates the situation of "structured locking" (Structured locking >>>> is >>>> the situation when, during a method invocation, every exit on a >>>> given >>>> monitor matches a preceding entry on that monitor, see the >>>> specification >>>> >>>> https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-2.html#jvms-2. >>>> 11.10) >>>> ? >> >> No it doesn't violate structured locking as the number of enters and >> exits match, and there is always an enter before an exit. >> >>>> Actually, the words (every exit on a given monitor matches a >>>> preceding >>>> entry on that monitor) are not quite clear as for me. Otherwise >>>> the >>>> first rule (The number of monitor entries performed by T on M >>>> during a >>>> method invocation must equal the number of monitor exits performed >>>> by >>>> T on M during the method invocation whether the method invocation >>>> completes normally or abruptly.) is sufficient. >> >> The number of enters and exits must not only match/balance, but >> there >> must be an enter before a corresponding exit. >> >>>> >>>> public class Search extends java.lang.Object { >>>> >>>> public void () >>>> { >>>> Search r0; >>>> r0 := @this: Search; >>>> specialinvoke r0.()>(); >>>> return; >>>> } >>>> >>>> public void main(java.lang.String[]) throws java.lang.Exception >>>> { >>>> Search r0; >>>> Search r2; >>>> java.lang.String[] r1; >>>> r0 := @this: Search; >>>> r1 := @parameter0: java.lang.String[]; >>>> r2 = new Search; >>>> specialinvoke r2.()>(); >>>> entermonitor r2; >>>> entermonitor r0; >>>> exitmonitor r2; >>>> exitmonitor r0; >>>> return; >>>> } >>>> } >>>> >>>> (3) The next program enters monitor in and exits it in >>>> main(). >>>> HotSpot throws a runtime exception, while J9 does not. Should this >>>> program be rejected by the verifiers? >> >> No this does not violate any verification rules. The runtime >> behaviour >> depends on whether structured locking is enforced or not.(Even in >> hotspot there can be differences between interpreted and jitted >> code). >> >> Hope that helps clarify things. >> >> David >> ----- >> >>>> >>>> public class Search extends java.lang.Object { >>>> >>>> public void () >>>> { >>>> Search r0; >>>> r0 := @this: Search; >>>> specialinvoke r0.()>(); >>>> entermonitor r0; >>>> return; >>>> } >>>> >>>> public void main(java.lang.String[]) throws java.lang.Exception >>>> { >>>> Search r0; >>>> Search r2; >>>> java.lang.S From david.holmes at oracle.com Thu May 18 04:57:03 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 18 May 2017 14:57:03 +1000 Subject: =?UTF-8?B?UmU6IOetlOWkjTog562U5aSNOiBIb3RTcG90IGFuZCBJQk0ncyBKOSBi?= =?UTF-8?Q?ehave_quite_differently_when_processing_monitorenters_and_monitor?= =?UTF-8?Q?exits?= In-Reply-To: <000401d2cf8d$c8352470$589f6d50$@cs.sjtu.edu.cn> References: <005101d2cf53$c8c16b40$5a4441c0$@cs.sjtu.edu.cn> <656dcb18-e65a-4c48-a6bc-cf00fe021af5@oracle.com> <005201d2cf5a$aa2e8f30$fe8bad90$@cs.sjtu.edu.cn> <5a6a2de2-eb43-985d-bfc7-18c3fa7ab0ee@oracle.com> <000401d2cf8d$c8352470$589f6d50$@cs.sjtu.edu.cn> Message-ID: <18003d32-08ad-4891-cf2a-4ba0d71a2bb0@oracle.com> On 18/05/2017 2:18 PM, ??? wrote: > Hi, David, > > I still did not catch why it happens... I just read the specification and > wrote some tests in order to understand some paragraphs better, and then had > questions. > > Will the HotSpot's verifier detect such kind of uninitialized monitored > objects in future? I have filed a bug so this can be examined more closely. https://bugs.openjdk.java.net/browse/JDK-8180581 Thanks, David > Regards, > Yuting > > -----????----- > ???: David Holmes [mailto:david.holmes at oracle.com] > ????: 2017?5?17? 20:23 > ???: ??? ; hotspot-runtime-dev at openjdk.java.net > ??: Re: ??: HotSpot and IBM's J9 behave quite differently when > processing monitorenters and monitorexits > > One correction ... > > On 18/05/2017 10:29 AM, David Holmes wrote: >> On 18/05/2017 8:12 AM, ??? wrote: >>> Thank you, David. I have seen from the specification that structured >>> locking enforcement is optional. The second and the third ones are >>> cases of structured/nested lockings. Will non-nested locking >>> sequences raise deadlocks? Of course it is a different topic, while >>> it might be better if it can be kicked out earlier from the >>> specification/JVM. >> >> Non-nested locking doesn't necessarily lead to deadlocks - you just >> need a different locking order for that (even if properly nested). >> There are locking patterns that rely on the ability to lock and unlock >> in different order ie chained-locking for walking linked-lists A->B->C->D: >> - lock A, lock B, unlock A, lock C, unlock B, lock D, unlock C ... >> >> The VM spec allows a little flexibility in how the monitor bytecodes >> can be used compared to the Java programming language. That's not >> something that will change. >> >>> The first example is still a problem. It seems that HotSpot allows to >>> monitor a pure object reference without initialized (Is it true? How >>> can this checking be omitted?). J9 reports a verifyerror as follows. >>> >>> Exception in thread "main" java.lang.VerifyError: JVMVRFY012 stack >>> shape inconsistent; class=Search, method=main([Ljava/lang/String;)V, >>> pc=6 Exception Details: >>> Location: >>> Search.main([Ljava/lang/String;)V @6: JBmonitorenter >>> Reason: >>> Type 'uninitialized' (current frame, stack[1]) is not assignable >>> to 'java/lang/Object' >>> Current Frame: >>> bci: @6 >>> flags: { } >>> locals: { 'Search', '[Ljava/lang/String;' } >>> stack: { 'uninitialized', 'uninitialized' } >>> at T.main(T.java:4) >> >> Yes I think this may be a bug in hotspot. The type-checking for the >> monitor bytecodes requires a matching type of reference on the operand >> stack - but "uninitialized" does not match Object, as J9 reports. But >> I'm not an expert on this aspect of verification so I may not be >> interpreting it correctly. >> >> More below ... >> >>> -----????----- >>> ???: David Holmes [mailto:david.holmes at oracle.com] >>> ????: 2017?5?17? 14:41 >>> ???: ??? ; >>> hotspot-runtime-dev at openjdk.java.net >>> ??: Re: HotSpot and IBM's J9 behave quite differently when processing >>> monitorenters and monitorexits >>> >>> Hi, >>> On 18/05/2017 7:23 AM, ??? wrote: >>>> Am I wrong? >>> >>> I will look at each situation in detail when I get a chance but >>> structured locking enforcement is optional. Also balancing the number >>> of locks and unlocks in a frame does not mean they can't be locked >>> and unlocked in a non-nested fashion - just that by the end the >>> number of unlocks matches the number of locks. >>> >>> BTW the way you respond to these emails, as if having a conversation >>> with yourself, makes it difficult to respond as we can't readily see >>> what is the new email and what is the original. >>> >>> Cheers, >>> David >>> >>>> The byte code for main() in case 1 is as follows. The strange thing >>>> is that NullPointerException is also not thrown at runtime. >> >> That is strange as it does for the normal obvious case of using >> synchronized(o) when o is null. > > Ah - it isn't null it just an object for which the constructor has not been > run. The runtime can't tell the difference between a pointer to a valid > initialized object, and a pointer to an uninitialized chunk of memory. > >>>> >>>> public void main(java.lang.String[]) throws java.lang.Exception; >>>> descriptor: ([Ljava/lang/String;)V >>>> flags: ACC_PUBLIC >>>> Code: >>>> stack=3, locals=2, args_size=2 >>>> 0: new #2 // class Search > > This allocated an object - hence no null reference. But this is what > verification should have complained about. > > David > ----- > >>>> 3: dup >>>> 4: aload_0 >>>> 5: monitorenter >>>> 6: monitorenter >>>> 7: monitorexit >>>> 8: aload_0 >>>> 9: monitorexit >>>> 10: return >>>> Exceptions: >>>> throws java.lang.Exception >>>> >>>> ??: HotSpot and IBM's J9 behave quite differently when processing >>>> monitorenters and monitorexits >>>> >>>> I have tested several programs (in Jimple) and found that HotSpot >>>> and >>>> J9 match monitorenters and monitorexits quite differently. Verifiers >>>> should play more important roles here. >> >> The job of the verifier is to establish some basic guarantees for the >> JVM to then operate under. The verifier plays no role in checking how >> monitorenter/exit are used in combination, only that each individual >> bytecode meets some basic type constraints. >> >>>> >>>> (1) Test the next program (r2 is not initizlied) on HotSpot and J9. >>>> J9 throw out a verifier error, while HotSpot does not. It seems that >>>> HotSpot's verifier forgets to check whether a monitored object is >>>> initialized. >>>> >>>> public class Search extends java.lang.Object { public void () >>>> { >>>> Search r0; >>>> r0 := @this: Search; >>>> specialinvoke r0.()>(); >>>> return; >>>> } >>>> public void main(java.lang.String[]) throws java.lang.Exception >>>> { >>>> Search r0; >>>> Search r2; >>>> java.lang.String[] r1; >>>> r0 := @this: Search; >>>> r1 := @parameter0: java.lang.String[]; >>>> r2 = new Search; >>>> >>>> entermonitor r2; >>>> entermonitor r0; >>>> exitmonitor r2; >>>> exitmonitor r0; >>>> return; >>>> } >>>> } >> >> Verification was covered above. >> >>>> >>>> (2) Test the next program on HotSpot and J9, and both do not report >>>> any errors. However, I guess the order in the program (entermonitor >>>> r2; => entermonitor r0; => exitmonitor r2; => exitmonitor r0;) >>>> violates the situation of "structured locking" (Structured locking >>>> is the situation when, during a method invocation, every exit on a >>>> given monitor matches a preceding entry on that monitor, see the >>>> specification >>>> https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-2.html#jvms-2. >>>> 11.10) >>>> ? >> >> No it doesn't violate structured locking as the number of enters and >> exits match, and there is always an enter before an exit. >> >>>> Actually, the words (every exit on a given monitor matches a >>>> preceding entry on that monitor) are not quite clear as for me. >>>> Otherwise the first rule (The number of monitor entries performed by >>>> T on M during a method invocation must equal the number of monitor >>>> exits performed by T on M during the method invocation whether the >>>> method invocation completes normally or abruptly.) is sufficient. >> >> The number of enters and exits must not only match/balance, but there >> must be an enter before a corresponding exit. >> >>>> >>>> public class Search extends java.lang.Object { >>>> >>>> public void () >>>> { >>>> Search r0; >>>> r0 := @this: Search; >>>> specialinvoke r0.()>(); >>>> return; >>>> } >>>> >>>> public void main(java.lang.String[]) throws java.lang.Exception >>>> { >>>> Search r0; >>>> Search r2; >>>> java.lang.String[] r1; >>>> r0 := @this: Search; >>>> r1 := @parameter0: java.lang.String[]; >>>> r2 = new Search; >>>> specialinvoke r2.()>(); >>>> entermonitor r2; >>>> entermonitor r0; >>>> exitmonitor r2; >>>> exitmonitor r0; >>>> return; >>>> } >>>> } >>>> >>>> (3) The next program enters monitor in and exits it in main(). >>>> HotSpot throws a runtime exception, while J9 does not. Should this >>>> program be rejected by the verifiers? >> >> No this does not violate any verification rules. The runtime behaviour >> depends on whether structured locking is enforced or not.(Even in >> hotspot there can be differences between interpreted and jitted code). >> >> Hope that helps clarify things. >> >> David >> ----- >> >>>> >>>> public class Search extends java.lang.Object { >>>> >>>> public void () >>>> { >>>> Search r0; >>>> r0 := @this: Search; >>>> specialinvoke r0.()>(); >>>> entermonitor r0; >>>> return; >>>> } >>>> >>>> public void main(java.lang.String[]) throws java.lang.Exception >>>> { >>>> Search r0; >>>> Search r2; >>>> java.lang.String[] r1; >>>> r0 := @this: Search; >>>> r1 := @parameter0: java.lang.String[]; >>>> r2 = new Search; >>>> specialinvoke r2.()>(); >>>> exitmonitor r2; >>>> exitmonitor r0; >>>> return; >>>> } >>>> } >>>> >>>> >>> > From chenyt at cs.sjtu.edu.cn Thu May 18 05:15:20 2017 From: chenyt at cs.sjtu.edu.cn (chenyt) Date: Thu, 18 May 2017 13:15:20 +0800 Subject: =?UTF-8?Q?=E7=AD=94=E5=A4=8D=3A=20=E7=AD=94=E5=A4=8D=3A=20Hot?= =?UTF-8?Q?Spot=20and=20IBM=27s=20J=39=20behave=20quite=20differently=20wh?= =?UTF-8?Q?en=20processing=20monitorenters=20and=20monitorexits?= In-Reply-To: <18003d32-08ad-4891-cf2a-4ba0d71a2bb0@oracle.com> References: <005101d2cf53$c8c16b40$5a4441c0$@cs.sjtu.edu.cn> <656dcb18-e65a-4c48-a6bc-cf00fe021af5@oracle.com> <005201d2cf5a$aa2e8f30$fe8bad90$@cs.sjtu.edu.cn> <5a6a2de2-eb43-985d-bfc7-18c3fa7ab0ee@oracle.com> <000401d2cf8d$c8352470$589f6d50$@cs.sjtu.edu.cn> <18003d32-08ad-4891-cf2a-4ba0d71a2bb0@oracle.com> Message-ID: <9a93ed9268233cf5847156fc7ceef5d5@cs.sjtu.edu.cn> Hi, David, I tested the next Jimple programs (let one object under monitoring be null) using J9 and HotSpot. I did not see NullPointerException here (the specification does tell me so). Am I wrong again? Hope the next two cases will help check. (1) case 1: HotSpot: IllegalMonitorStateException J9: NullPointerException r0 := @this: Search; r1 := @parameter0: java.lang.String[]; r2 = null; entermonitor r0; entermonitor r2; exitmonitor r2; exitmonitor r0; (2) case 2: HotSpot: IllegalMonitorStateException J9: VerifyError r1 := @parameter0: java.lang.String[]; r2 = new Search; entermonitor r0; entermonitor r2; r2=null; exitmonitor r2; exitmonitor r0; It is very interesting that I can still find some unexpected behaviors here. Wishes, Yuting On Thu, 18 May 2017 14:57:03 +1000, David Holmes wrote: > On 18/05/2017 2:18 PM, ??? wrote: >> Hi, David, >> >> I still did not catch why it happens... I just read the >> specification and >> wrote some tests in order to understand some paragraphs better, and >> then had >> questions. >> >> Will the HotSpot's verifier detect such kind of uninitialized >> monitored >> objects in future? > > I have filed a bug so this can be examined more closely. > > https://bugs.openjdk.java.net/browse/JDK-8180581 > > Thanks, > David > >> Regards, >> Yuting >> >> -----????----- >> ???: David Holmes [mailto:david.holmes at oracle.com] >> ????: 2017?5?17? 20:23 >> ???: ??? ; >> hotspot-runtime-dev at openjdk.java.net >> ??: Re: ??: HotSpot and IBM's J9 behave quite differently when >> processing monitorenters and monitorexits >> >> One correction ... >> >> On 18/05/2017 10:29 AM, David Holmes wrote: >>> On 18/05/2017 8:12 AM, ??? wrote: >>>> Thank you, David. I have seen from the specification that >>>> structured >>>> locking enforcement is optional. The second and the third ones are >>>> cases of structured/nested lockings. Will non-nested locking >>>> sequences raise deadlocks? Of course it is a different topic, >>>> while >>>> it might be better if it can be kicked out earlier from the >>>> specification/JVM. >>> >>> Non-nested locking doesn't necessarily lead to deadlocks - you just >>> need a different locking order for that (even if properly nested). >>> There are locking patterns that rely on the ability to lock and >>> unlock >>> in different order ie chained-locking for walking linked-lists >>> A->B->C->D: >>> - lock A, lock B, unlock A, lock C, unlock B, lock D, unlock C ... >>> >>> The VM spec allows a little flexibility in how the monitor >>> bytecodes >>> can be used compared to the Java programming language. That's not >>> something that will change. >>> >>>> The first example is still a problem. It seems that HotSpot allows >>>> to >>>> monitor a pure object reference without initialized (Is it true? >>>> How >>>> can this checking be omitted?). J9 reports a verifyerror as >>>> follows. >>>> >>>> Exception in thread "main" java.lang.VerifyError: JVMVRFY012 stack >>>> shape inconsistent; class=Search, >>>> method=main([Ljava/lang/String;)V, >>>> pc=6 Exception Details: >>>> Location: >>>> Search.main([Ljava/lang/String;)V @6: JBmonitorenter >>>> Reason: >>>> Type 'uninitialized' (current frame, stack[1]) is not >>>> assignable >>>> to 'java/lang/Object' >>>> Current Frame: >>>> bci: @6 >>>> flags: { } >>>> locals: { 'Search', '[Ljava/lang/String;' } >>>> stack: { 'uninitialized', 'uninitialized' } >>>> at T.main(T.java:4) >>> >>> Yes I think this may be a bug in hotspot. The type-checking for the >>> monitor bytecodes requires a matching type of reference on the >>> operand >>> stack - but "uninitialized" does not match Object, as J9 reports. >>> But >>> I'm not an expert on this aspect of verification so I may not be >>> interpreting it correctly. >>> >>> More below ... >>> >>>> -----????----- >>>> ???: David Holmes [mailto:david.holmes at oracle.com] >>>> ????: 2017?5?17? 14:41 >>>> ???: ??? ; >>>> hotspot-runtime-dev at openjdk.java.net >>>> ??: Re: HotSpot and IBM's J9 behave quite differently when >>>> processing >>>> monitorenters and monitorexits >>>> >>>> Hi, >>>> On 18/05/2017 7:23 AM, ??? wrote: >>>>> Am I wrong? >>>> >>>> I will look at each situation in detail when I get a chance but >>>> structured locking enforcement is optional. Also balancing the >>>> number >>>> of locks and unlocks in a frame does not mean they can't be locked >>>> and unlocked in a non-nested fashion - just that by the end the >>>> number of unlocks matches the number of locks. >>>> >>>> BTW the way you respond to these emails, as if having a >>>> conversation >>>> with yourself, makes it difficult to respond as we can't readily >>>> see >>>> what is the new email and what is the original. >>>> >>>> Cheers, >>>> David >>>> >>>>> The byte code for main() in case 1 is as follows. The strange >>>>> thing >>>>> is that NullPointerException is also not thrown at runtime. >>> >>> That is strange as it does for the normal obvious case of using >>> synchronized(o) when o is null. >> >> Ah - it isn't null it just an object for which the constructor has >> not been >> run. The runtime can't tell the difference between a pointer to a >> valid >> initialized object, and a pointer to an uninitialized chunk of >> memory. >> >>>>> >>>>> public void main(java.lang.String[]) throws java.lang.Exception; >>>>> descriptor: ([Ljava/lang/String;)V >>>>> flags: ACC_PUBLIC >>>>> Code: >>>>> stack=3, locals=2, args_size=2 >>>>> 0: new #2 // class Search >> >> This allocated an object - hence no null reference. But this is what >> verification should have complained about. >> >> David >> ----- >> >>>>> 3: dup >>>>> 4: aload_0 >>>>> 5: monitorenter >>>>> 6: monitorenter >>>>> 7: monitorexit >>>>> 8: aload_0 >>>>> 9: monitorexit >>>>> 10: return >>>>> Exceptions: >>>>> throws java.lang.Exception >>>>> >>>>> ??: HotSpot and IBM's J9 behave quite differently when processing >>>>> monitorenters and monitorexits >>>>> >>>>> I have tested several programs (in Jimple) and found that HotSpot >>>>> and >>>>> J9 match monitorenters and monitorexits quite differently. >>>>> Verifiers >>>>> should play more important roles here. >>> >>> The job of the verifier is to establish some basic guarantees for >>> the >>> JVM to then operate under. The verifier plays no role in checking >>> how >>> monitorenter/exit are used in combination, only that each >>> individual >>> bytecode meets some basic type constraints. >>> >>>>> >>>>> (1) Test the next program (r2 is not initizlied) on HotSpot and >>>>> J9. >>>>> J9 throw out a verifier error, while HotSpot does not. It seems >>>>> that >>>>> HotSpot's verifier forgets to check whether a monitored object is >>>>> initialized. >>>>> >>>>> public class Search extends java.lang.Object { public void >>>>> () >>>>> { >>>>> Search r0; >>>>> r0 := @this: Search; >>>>> specialinvoke r0.()>(); >>>>> return; >>>>> } >>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>> { >>>>> Search r0; >>>>> Search r2; >>>>> java.lang.String[] r1; >>>>> r0 := @this: Search; >>>>> r1 := @parameter0: java.lang.String[]; >>>>> r2 = new Search; >>>>> >>>>> entermonitor r2; >>>>> entermonitor r0; >>>>> exitmonitor r2; >>>>> exitmonitor r0; >>>>> return; >>>>> } >>>>> } >>> >>> Verification was covered above. >>> >>>>> >>>>> (2) Test the next program on HotSpot and J9, and both do not >>>>> report >>>>> any errors. However, I guess the order in the program >>>>> (entermonitor >>>>> r2; => entermonitor r0; => exitmonitor r2; => exitmonitor r0;) >>>>> violates the situation of "structured locking" (Structured >>>>> locking >>>>> is the situation when, during a method invocation, every exit on >>>>> a >>>>> given monitor matches a preceding entry on that monitor, see the >>>>> specification >>>>> >>>>> https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-2.html#jvms-2. >>>>> 11.10) >>>>> ? >>> >>> No it doesn't violate structured locking as the number of enters >>> and >>> exits match, and there is always an enter before an exit. >>> >>>>> Actually, the words (every exit on a given monitor matches a >>>>> preceding entry on that monitor) are not quite clear as for me. >>>>> Otherwise the first rule (The number of monitor entries performed >>>>> by >>>>> T on M during a method invocation must equal the number of >>>>> monitor >>>>> exits performed by T on M during the method invocation whether >>>>> the >>>>> method invocation completes normally or abruptly.) is >>>>> sufficient. >>> >>> The number of enters and exits must not only match/balance, but >>> there >>> must be an enter before a corresponding exit. >>> >>>>> >>>>> public class Search extends java.lang.Object { >>>>> >>>>> public void () >>>>> { >>>>> Search r0; >>>>> r0 := @this: Search; >>>>> specialinvoke r0.()>(); >>>>> return; >>>>> } >>>>> >>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>> { >>>>> Search r0; >>>>> Search r2; >>>>> java.lang.String[] r1; >>>>> r0 := @this: Search; >>>>> r1 := @parameter0: java.lang.String[]; >>>>> r2 = new Search; >>>>> specialinvoke r2.()>(); >>>>> entermonitor r2; >>>>> entermonitor r0; >>>>> exitmonitor r2; >>>>> exitmonitor r0; >>>>> return; >>>>> } >>>>> } >>>>> >>>>> (3) The next program enters monitor in and exits it in >>>>> main(). >>>>> HotSpot throws a runtime exception, while J9 does not. Should >>>>> this >>>>> program be rejected by the verifiers? >>> >>> No this does not violate any verification rules. The runtime >>> behaviour >>> depends on whether structured locking is enforced or not.(Even in >>> hotspot there can be differences between interpreted and jitted >>> code). >>> >>> Hope that helps clarify things. >>> >>> David >>> ----- >>> >>>>> >>>>> public class Search extends java.lang.Object { >>>>> >>>>> public void () >>>>> { >>>>> Search r0; >>>>> r0 := @this: Search; >>>>> specialinvoke r0.()>(); >>>>> entermonitor r0; >>> From david.holmes at oracle.com Thu May 18 05:25:57 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 18 May 2017 15:25:57 +1000 Subject: =?UTF-8?Q?Re:_=e7=ad=94=e5=a4=8d:_HotSpot_and_IBM's_J9_behave_quite?= =?UTF-8?Q?_differently_when_processing_monitorenters_and_monitorexits?= In-Reply-To: References: <005101d2cf53$c8c16b40$5a4441c0$@cs.sjtu.edu.cn> <656dcb18-e65a-4c48-a6bc-cf00fe021af5@oracle.com> <005201d2cf5a$aa2e8f30$fe8bad90$@cs.sjtu.edu.cn> <5a6a2de2-eb43-985d-bfc7-18c3fa7ab0ee@oracle.com> Message-ID: <24a31d5c-8185-e55d-481f-719f3949bf7d@oracle.com> Hi Yuting, On 18/05/2017 2:49 PM, chenyt wrote: > Hi, David, > > I tested the next program (let one object to be monitored be null) using > J9 and HotSpot. HotSpot does not throw a NullPointerException, as the > specification says. J9 looks fine (throw a NullPointerException). Am I > wrong here? I can't readily test this as I don't have Jimple nor quick and easy access to bytecode assemblers. It is strange though as the interpreter code contains this: CASE(_monitorenter): { oop lockee = STACK_OBJECT(-1); // derefing's lockee ought to provoke implicit null check CHECK_NULL(lockee); If I can find some time I will try to test this myself. > It is very interesting that I have found so many unexpected behaviors here. Well you only found two and they are related. :) Manually assembled monitor code is not something very many (any?) people care about or do - other than emulating correct language usage. Cheers, David > public void main(java.lang.String[]) throws java.lang.Exception; > descriptor: ([Ljava/lang/String;)V > flags: ACC_PUBLIC > Code: > stack=1, locals=2, args_size=2 > 0: aload_0 > 1: monitorenter > 2: aconst_null > 3: monitorenter > 4: aconst_null > 5: monitorexit > 6: aload_0 > 7: monitorexit > 8: return > Exceptions: > throws java.lang.Exception > > Jimple code is given as follows: > public void main(java.lang.String[]) throws java.lang.Exception > { > Search r0; > Search r2; > java.lang.String[] r1; > > r0 := @this: Search; > r1 := @parameter0: java.lang.String[]; > r2 = null; > entermonitor r0; > entermonitor r2; > exitmonitor r2; > exitmonitor r0; > > return; > } > > Wishes, > Yuting > > > On Thu, 18 May 2017 13:23:09 +1000, David Holmes wrote: >> One correction ... >> >> On 18/05/2017 10:29 AM, David Holmes wrote: >>> On 18/05/2017 8:12 AM, ??? wrote: >>>> Thank you, David. I have seen from the specification that structured >>>> locking >>>> enforcement is optional. The second and the third ones are cases of >>>> structured/nested lockings. Will non-nested locking sequences raise >>>> deadlocks? Of course it is a different topic, while it might be better >>>> if it >>>> can be kicked out earlier from the specification/JVM. >>> >>> Non-nested locking doesn't necessarily lead to deadlocks - you just need >>> a different locking order for that (even if properly nested). There are >>> locking patterns that rely on the ability to lock and unlock in >>> different order ie chained-locking for walking linked-lists A->B->C->D: >>> - lock A, lock B, unlock A, lock C, unlock B, lock D, unlock C ... >>> >>> The VM spec allows a little flexibility in how the monitor bytecodes can >>> be used compared to the Java programming language. That's not something >>> that will change. >>> >>>> The first example is still a problem. It seems that HotSpot allows to >>>> monitor a pure object reference without initialized (Is it true? How >>>> can >>>> this checking be omitted?). J9 reports a verifyerror as follows. >>>> >>>> Exception in thread "main" java.lang.VerifyError: JVMVRFY012 stack >>>> shape >>>> inconsistent; class=Search, method=main([Ljava/lang/String;)V, pc=6 >>>> Exception Details: >>>> Location: >>>> Search.main([Ljava/lang/String;)V @6: JBmonitorenter >>>> Reason: >>>> Type 'uninitialized' (current frame, stack[1]) is not assignable to >>>> 'java/lang/Object' >>>> Current Frame: >>>> bci: @6 >>>> flags: { } >>>> locals: { 'Search', '[Ljava/lang/String;' } >>>> stack: { 'uninitialized', 'uninitialized' } >>>> at T.main(T.java:4) >>> >>> Yes I think this may be a bug in hotspot. The type-checking for the >>> monitor bytecodes requires a matching type of reference on the operand >>> stack - but "uninitialized" does not match Object, as J9 reports. But >>> I'm not an expert on this aspect of verification so I may not be >>> interpreting it correctly. >>> >>> More below ... >>> >>>> -----????----- >>>> ???: David Holmes [mailto:david.holmes at oracle.com] >>>> ????: 2017?5?17? 14:41 >>>> ???: ??? ; >>>> hotspot-runtime-dev at openjdk.java.net >>>> ??: Re: HotSpot and IBM's J9 behave quite differently when processing >>>> monitorenters and monitorexits >>>> >>>> Hi, >>>> On 18/05/2017 7:23 AM, ??? wrote: >>>>> Am I wrong? >>>> >>>> I will look at each situation in detail when I get a chance but >>>> structured >>>> locking enforcement is optional. Also balancing the number of locks and >>>> unlocks in a frame does not mean they can't be locked and unlocked in a >>>> non-nested fashion - just that by the end the number of unlocks >>>> matches the >>>> number of locks. >>>> >>>> BTW the way you respond to these emails, as if having a conversation >>>> with >>>> yourself, makes it difficult to respond as we can't readily see what >>>> is the >>>> new email and what is the original. >>>> >>>> Cheers, >>>> David >>>> >>>>> The byte code for main() in case 1 is as follows. The strange thing is >>>>> that NullPointerException is also not thrown at runtime. >>> >>> That is strange as it does for the normal obvious case of using >>> synchronized(o) when o is null. >> >> Ah - it isn't null it just an object for which the constructor has >> not been run. The runtime can't tell the difference between a pointer >> to a valid initialized object, and a pointer to an uninitialized chunk >> of memory. >> >>>>> >>>>> public void main(java.lang.String[]) throws java.lang.Exception; >>>>> descriptor: ([Ljava/lang/String;)V >>>>> flags: ACC_PUBLIC >>>>> Code: >>>>> stack=3, locals=2, args_size=2 >>>>> 0: new #2 // class Search >> >> This allocated an object - hence no null reference. But this is what >> verification should have complained about. >> >> David >> ----- >> >>>>> 3: dup >>>>> 4: aload_0 >>>>> 5: monitorenter >>>>> 6: monitorenter >>>>> 7: monitorexit >>>>> 8: aload_0 >>>>> 9: monitorexit >>>>> 10: return >>>>> Exceptions: >>>>> throws java.lang.Exception >>>>> >>>>> ??: HotSpot and IBM's J9 behave quite differently when processing >>>>> monitorenters and monitorexits >>>>> >>>>> I have tested several programs (in Jimple) and found that HotSpot and >>>>> J9 match monitorenters and monitorexits quite differently. Verifiers >>>>> should play more important roles here. >>> >>> The job of the verifier is to establish some basic guarantees for the >>> JVM to then operate under. The verifier plays no role in checking how >>> monitorenter/exit are used in combination, only that each individual >>> bytecode meets some basic type constraints. >>> >>>>> >>>>> (1) Test the next program (r2 is not initizlied) on HotSpot and J9. >>>>> J9 throw out a verifier error, while HotSpot does not. It seems that >>>>> HotSpot's verifier forgets to check whether a monitored object is >>>>> initialized. >>>>> >>>>> public class Search extends java.lang.Object { public void () >>>>> { >>>>> Search r0; >>>>> r0 := @this: Search; >>>>> specialinvoke r0.()>(); >>>>> return; >>>>> } >>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>> { >>>>> Search r0; >>>>> Search r2; >>>>> java.lang.String[] r1; >>>>> r0 := @this: Search; >>>>> r1 := @parameter0: java.lang.String[]; >>>>> r2 = new Search; >>>>> >>>>> entermonitor r2; >>>>> entermonitor r0; >>>>> exitmonitor r2; >>>>> exitmonitor r0; >>>>> return; >>>>> } >>>>> } >>> >>> Verification was covered above. >>> >>>>> >>>>> (2) Test the next program on HotSpot and J9, and both do not report >>>>> any errors. However, I guess the order in the program (entermonitor >>>>> r2; => entermonitor r0; => exitmonitor r2; => exitmonitor r0;) >>>>> violates the situation of "structured locking" (Structured locking is >>>>> the situation when, during a method invocation, every exit on a given >>>>> monitor matches a preceding entry on that monitor, see the >>>>> specification >>>>> >>>>> https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-2.html#jvms-2. >>>>> 11.10) >>>>> ? >>> >>> No it doesn't violate structured locking as the number of enters and >>> exits match, and there is always an enter before an exit. >>> >>>>> Actually, the words (every exit on a given monitor matches a preceding >>>>> entry on that monitor) are not quite clear as for me. Otherwise the >>>>> first rule (The number of monitor entries performed by T on M during a >>>>> method invocation must equal the number of monitor exits performed by >>>>> T on M during the method invocation whether the method invocation >>>>> completes normally or abruptly.) is sufficient. >>> >>> The number of enters and exits must not only match/balance, but there >>> must be an enter before a corresponding exit. >>> >>>>> >>>>> public class Search extends java.lang.Object { >>>>> >>>>> public void () >>>>> { >>>>> Search r0; >>>>> r0 := @this: Search; >>>>> specialinvoke r0.()>(); >>>>> return; >>>>> } >>>>> >>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>> { >>>>> Search r0; >>>>> Search r2; >>>>> java.lang.String[] r1; >>>>> r0 := @this: Search; >>>>> r1 := @parameter0: java.lang.String[]; >>>>> r2 = new Search; >>>>> specialinvoke r2.()>(); >>>>> entermonitor r2; >>>>> entermonitor r0; >>>>> exitmonitor r2; >>>>> exitmonitor r0; >>>>> return; >>>>> } >>>>> } >>>>> >>>>> (3) The next program enters monitor in and exits it in main(). >>>>> HotSpot throws a runtime exception, while J9 does not. Should this >>>>> program be rejected by the verifiers? >>> >>> No this does not violate any verification rules. The runtime behaviour >>> depends on whether structured locking is enforced or not.(Even in >>> hotspot there can be differences between interpreted and jitted code). >>> >>> Hope that helps clarify things. >>> >>> David >>> ----- >>> >>>>> >>>>> public class Search extends java.lang.Object { >>>>> >>>>> public void () >>>>> { >>>>> Search r0; >>>>> r0 := @this: Search; >>>>> specialinvoke r0.()>(); >>>>> entermonitor r0; >>>>> return; >>>>> } >>>>> >>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>> { >>>>> Search r0; >>>>> Search r2; >>>>> java.lang.S From chenyt at cs.sjtu.edu.cn Thu May 18 05:26:39 2017 From: chenyt at cs.sjtu.edu.cn (=?gb2312?B?s8LT6s2k?=) Date: Wed, 17 May 2017 22:26:39 -0700 Subject: =?gb2312?B?tPC4tDogtPC4tDogSG90U3BvdCBhbmQgSUJNJ3MgSjkgYmVoYXZlIA==?= =?gb2312?B?cXVpdGUgZGlmZmVyZW50bHkgd2hlbiBwcm9jZXNzaW5nIG1vbml0bw==?= =?gb2312?B?cmVudGVycyBhbmQgbW9uaXRvcmV4aXRz?= References: <005101d2cf53$c8c16b40$5a4441c0$@cs.sjtu.edu.cn> <656dcb18-e65a-4c48-a6bc-cf00fe021af5@oracle.com> <005201d2cf5a$aa2e8f30$fe8bad90$@cs.sjtu.edu.cn> <5a6a2de2-eb43-985d-bfc7-18c3fa7ab0ee@oracle.com> Message-ID: <000001d2cf97$54ba6a00$fe2f3e00$@cs.sjtu.edu.cn> Hi, David, I tested the next Jimple programs (let one object under monitoring be null) using J9 and HotSpot. I did not see NullPointerException here (the specification does tell me so when an object reference is null). Am I wrong again? Hope the next two cases will help check. (1) case 1: HotSpot: IllegalMonitorStateException J9: NullPointerException r0 := @this: Search; r1 := @parameter0: java.lang.String[]; r2 = null; entermonitor r0; entermonitor r2; exitmonitor r2; exitmonitor r0; (2) case 2: HotSpot: IllegalMonitorStateException J9: VerifyError r1 := @parameter0: java.lang.String[]; r2 = new Search; entermonitor r0; entermonitor r2; r2=null; exitmonitor r2; exitmonitor r0; It is very interesting that I can still find some unexpected behaviors here. Wishes, Yuting-----????----- ???: ??? [mailto:chenyt at cs.sjtu.edu.cn] ????: 2017?5?17? 21:18 ???: 'David Holmes' ; 'hotspot-runtime-dev at openjdk.java.net' ??: ??: ??: HotSpot and IBM's J9 behave quite differently when processing monitorenters and monitorexits Hi, David, I still did not catch why it happens... I just read the specification and wrote some tests in order to understand some paragraphs better, and then had questions. Will the HotSpot's verifier detect such kind of uninitialized monitored objects in future? Regards, Yuting -----????----- ???: David Holmes [mailto:david.holmes at oracle.com] ????: 2017?5?17? 20:23 ???: ??? ; hotspot-runtime-dev at openjdk.java.net ??: Re: ??: HotSpot and IBM's J9 behave quite differently when processing monitorenters and monitorexits One correction ... On 18/05/2017 10:29 AM, David Holmes wrote: > On 18/05/2017 8:12 AM, ??? wrote: >> Thank you, David. I have seen from the specification that structured >> locking enforcement is optional. The second and the third ones are >> cases of structured/nested lockings. Will non-nested locking >> sequences raise deadlocks? Of course it is a different topic, while >> it might be better if it can be kicked out earlier from the >> specification/JVM. > > Non-nested locking doesn't necessarily lead to deadlocks - you just > need a different locking order for that (even if properly nested). > There are locking patterns that rely on the ability to lock and unlock > in different order ie chained-locking for walking linked-lists A->B->C->D: > - lock A, lock B, unlock A, lock C, unlock B, lock D, unlock C ... > > The VM spec allows a little flexibility in how the monitor bytecodes > can be used compared to the Java programming language. That's not > something that will change. > >> The first example is still a problem. It seems that HotSpot allows to >> monitor a pure object reference without initialized (Is it true? How >> can this checking be omitted?). J9 reports a verifyerror as follows. >> >> Exception in thread "main" java.lang.VerifyError: JVMVRFY012 stack >> shape inconsistent; class=Search, method=main([Ljava/lang/String;)V, >> pc=6 Exception Details: >> Location: >> Search.main([Ljava/lang/String;)V @6: JBmonitorenter >> Reason: >> Type 'uninitialized' (current frame, stack[1]) is not assignable >> to 'java/lang/Object' >> Current Frame: >> bci: @6 >> flags: { } >> locals: { 'Search', '[Ljava/lang/String;' } >> stack: { 'uninitialized', 'uninitialized' } >> at T.main(T.java:4) > > Yes I think this may be a bug in hotspot. The type-checking for the > monitor bytecodes requires a matching type of reference on the operand > stack - but "uninitialized" does not match Object, as J9 reports. But > I'm not an expert on this aspect of verification so I may not be > interpreting it correctly. > > More below ... > >> -----????----- >> ???: David Holmes [mailto:david.holmes at oracle.com] >> ????: 2017?5?17? 14:41 >> ???: ??? ; >> hotspot-runtime-dev at openjdk.java.net >> ??: Re: HotSpot and IBM's J9 behave quite differently when processing >> monitorenters and monitorexits >> >> Hi, >> On 18/05/2017 7:23 AM, ??? wrote: >>> Am I wrong? >> >> I will look at each situation in detail when I get a chance but >> structured locking enforcement is optional. Also balancing the number >> of locks and unlocks in a frame does not mean they can't be locked >> and unlocked in a non-nested fashion - just that by the end the >> number of unlocks matches the number of locks. >> >> BTW the way you respond to these emails, as if having a conversation >> with yourself, makes it difficult to respond as we can't readily see >> what is the new email and what is the original. >> >> Cheers, >> David >> >>> The byte code for main() in case 1 is as follows. The strange thing >>> is that NullPointerException is also not thrown at runtime. > > That is strange as it does for the normal obvious case of using > synchronized(o) when o is null. Ah - it isn't null it just an object for which the constructor has not been run. The runtime can't tell the difference between a pointer to a valid initialized object, and a pointer to an uninitialized chunk of memory. >>> >>> public void main(java.lang.String[]) throws java.lang.Exception; >>> descriptor: ([Ljava/lang/String;)V >>> flags: ACC_PUBLIC >>> Code: >>> stack=3, locals=2, args_size=2 >>> 0: new #2 // class Search This allocated an object - hence no null reference. But this is what verification should have complained about. David ----- >>> 3: dup >>> 4: aload_0 >>> 5: monitorenter >>> 6: monitorenter >>> 7: monitorexit >>> 8: aload_0 >>> 9: monitorexit >>> 10: return >>> Exceptions: >>> throws java.lang.Exception >>> >>> ??: HotSpot and IBM's J9 behave quite differently when processing >>> monitorenters and monitorexits >>> >>> I have tested several programs (in Jimple) and found that HotSpot >>> and >>> J9 match monitorenters and monitorexits quite differently. Verifiers >>> should play more important roles here. > > The job of the verifier is to establish some basic guarantees for the > JVM to then operate under. The verifier plays no role in checking how > monitorenter/exit are used in combination, only that each individual > bytecode meets some basic type constraints. > >>> >>> (1) Test the next program (r2 is not initizlied) on HotSpot and J9. >>> J9 throw out a verifier error, while HotSpot does not. It seems that >>> HotSpot's verifier forgets to check whether a monitored object is >>> initialized. >>> >>> public class Search extends java.lang.Object { public void () >>> { >>> Search r0; >>> r0 := @this: Search; >>> specialinvoke r0.()>(); >>> return; >>> } >>> public void main(java.lang.String[]) throws java.lang.Exception >>> { >>> Search r0; >>> Search r2; >>> java.lang.String[] r1; >>> r0 := @this: Search; >>> r1 := @parameter0: java.lang.String[]; >>> r2 = new Search; >>> >>> entermonitor r2; >>> entermonitor r0; >>> exitmonitor r2; >>> exitmonitor r0; >>> return; >>> } >>> } > > Verification was covered above. > >>> >>> (2) Test the next program on HotSpot and J9, and both do not report >>> any errors. However, I guess the order in the program (entermonitor >>> r2; => entermonitor r0; => exitmonitor r2; => exitmonitor r0;) >>> violates the situation of "structured locking" (Structured locking >>> is the situation when, during a method invocation, every exit on a >>> given monitor matches a preceding entry on that monitor, see the >>> specification >>> https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-2.html#jvms-2. >>> 11.10) >>> ? > > No it doesn't violate structured locking as the number of enters and > exits match, and there is always an enter before an exit. > >>> Actually, the words (every exit on a given monitor matches a >>> preceding entry on that monitor) are not quite clear as for me. >>> Otherwise the first rule (The number of monitor entries performed by >>> T on M during a method invocation must equal the number of monitor >>> exits performed by T on M during the method invocation whether the >>> method invocation completes normally or abruptly.) is sufficient. > > The number of enters and exits must not only match/balance, but there > must be an enter before a corresponding exit. > >>> >>> public class Search extends java.lang.Object { >>> >>> public void () >>> { >>> Search r0; >>> r0 := @this: Search; >>> specialinvoke r0.()>(); >>> return; >>> } >>> >>> public void main(java.lang.String[]) throws java.lang.Exception >>> { >>> Search r0; >>> Search r2; >>> java.lang.String[] r1; >>> r0 := @this: Search; >>> r1 := @parameter0: java.lang.String[]; >>> r2 = new Search; >>> specialinvoke r2.()>(); >>> entermonitor r2; >>> entermonitor r0; >>> exitmonitor r2; >>> exitmonitor r0; >>> return; >>> } >>> } >>> >>> (3) The next program enters monitor in and exits it in main(). >>> HotSpot throws a runtime exception, while J9 does not. Should this >>> program be rejected by the verifiers? > > No this does not violate any verification rules. The runtime behaviour > depends on whether structured locking is enforced or not.(Even in > hotspot there can be differences between interpreted and jitted code). > > Hope that helps clarify things. > > David > ----- > >>> >>> public class Search extends java.lang.Object { >>> >>> public void () >>> { >>> Search r0; >>> r0 := @this: Search; >>> specialinvoke r0.()>(); >>> entermonitor r0; >>> return; >>> } >>> >>> public void main(java.lang.String[]) throws java.lang.Exception >>> { >>> Search r0; >>> Search r2; >>> java.lang.String[] r1; >>> r0 := @this: Search; >>> r1 := @parameter0: java.lang.String[]; >>> r2 = new Search; >>> specialinvoke r2.()>(); >>> exitmonitor r2; >>> exitmonitor r0; >>> return; >>> } >>> } >>> >>> >> From david.holmes at oracle.com Thu May 18 05:29:15 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 18 May 2017 15:29:15 +1000 Subject: =?UTF-8?B?UmU6IOetlOWkjTog562U5aSNOiBIb3RTcG90IGFuZCBJQk0ncyBKOSBi?= =?UTF-8?Q?ehave_quite_differently_when_processing_monitorenters_and_monitor?= =?UTF-8?Q?exits?= In-Reply-To: <9a93ed9268233cf5847156fc7ceef5d5@cs.sjtu.edu.cn> References: <005101d2cf53$c8c16b40$5a4441c0$@cs.sjtu.edu.cn> <656dcb18-e65a-4c48-a6bc-cf00fe021af5@oracle.com> <005201d2cf5a$aa2e8f30$fe8bad90$@cs.sjtu.edu.cn> <5a6a2de2-eb43-985d-bfc7-18c3fa7ab0ee@oracle.com> <000401d2cf8d$c8352470$589f6d50$@cs.sjtu.edu.cn> <18003d32-08ad-4891-cf2a-4ba0d71a2bb0@oracle.com> <9a93ed9268233cf5847156fc7ceef5d5@cs.sjtu.edu.cn> Message-ID: Is this any different from what you previously reported?? Sorry I'm having a hard time tracking the scenarios. As far as I can tell there are two cases: - new'd but uninitialized objectref This should be a verification error in which case runtime behaviour is unspecified. - null reference This should throw NullPointerException, and as per last email the interpreter explicitly has a null check, so I don't know why this wouldn't throw. David On 18/05/2017 3:15 PM, chenyt wrote: > Hi, David, > > I tested the next Jimple programs (let one object under monitoring be > null) using J9 and HotSpot. I did not see NullPointerException here (the > specification does tell me so). Am I wrong again? Hope the next two > cases will help check. > > (1) case 1: HotSpot: IllegalMonitorStateException J9: NullPointerException > r0 := @this: Search; > r1 := @parameter0: java.lang.String[]; > r2 = null; > entermonitor r0; > entermonitor r2; > exitmonitor r2; > exitmonitor r0; > (2) case 2: HotSpot: IllegalMonitorStateException J9: VerifyError > r1 := @parameter0: java.lang.String[]; > r2 = new Search; > entermonitor r0; > entermonitor r2; > r2=null; > exitmonitor r2; > exitmonitor r0; > It is very interesting that I can still find some unexpected behaviors > here. > > Wishes, > Yuting > > > On Thu, 18 May 2017 14:57:03 +1000, David Holmes wrote: >> On 18/05/2017 2:18 PM, ??? wrote: >>> Hi, David, >>> >>> I still did not catch why it happens... I just read the specification >>> and >>> wrote some tests in order to understand some paragraphs better, and >>> then had >>> questions. >>> >>> Will the HotSpot's verifier detect such kind of uninitialized monitored >>> objects in future? >> >> I have filed a bug so this can be examined more closely. >> >> https://bugs.openjdk.java.net/browse/JDK-8180581 >> >> Thanks, >> David >> >>> Regards, >>> Yuting >>> >>> -----????----- >>> ???: David Holmes [mailto:david.holmes at oracle.com] >>> ????: 2017?5?17? 20:23 >>> ???: ??? ; >>> hotspot-runtime-dev at openjdk.java.net >>> ??: Re: ??: HotSpot and IBM's J9 behave quite differently when >>> processing monitorenters and monitorexits >>> >>> One correction ... >>> >>> On 18/05/2017 10:29 AM, David Holmes wrote: >>>> On 18/05/2017 8:12 AM, ??? wrote: >>>>> Thank you, David. I have seen from the specification that structured >>>>> locking enforcement is optional. The second and the third ones are >>>>> cases of structured/nested lockings. Will non-nested locking >>>>> sequences raise deadlocks? Of course it is a different topic, while >>>>> it might be better if it can be kicked out earlier from the >>>>> specification/JVM. >>>> >>>> Non-nested locking doesn't necessarily lead to deadlocks - you just >>>> need a different locking order for that (even if properly nested). >>>> There are locking patterns that rely on the ability to lock and unlock >>>> in different order ie chained-locking for walking linked-lists >>>> A->B->C->D: >>>> - lock A, lock B, unlock A, lock C, unlock B, lock D, unlock C ... >>>> >>>> The VM spec allows a little flexibility in how the monitor bytecodes >>>> can be used compared to the Java programming language. That's not >>>> something that will change. >>>> >>>>> The first example is still a problem. It seems that HotSpot allows to >>>>> monitor a pure object reference without initialized (Is it true? How >>>>> can this checking be omitted?). J9 reports a verifyerror as follows. >>>>> >>>>> Exception in thread "main" java.lang.VerifyError: JVMVRFY012 stack >>>>> shape inconsistent; class=Search, method=main([Ljava/lang/String;)V, >>>>> pc=6 Exception Details: >>>>> Location: >>>>> Search.main([Ljava/lang/String;)V @6: JBmonitorenter >>>>> Reason: >>>>> Type 'uninitialized' (current frame, stack[1]) is not assignable >>>>> to 'java/lang/Object' >>>>> Current Frame: >>>>> bci: @6 >>>>> flags: { } >>>>> locals: { 'Search', '[Ljava/lang/String;' } >>>>> stack: { 'uninitialized', 'uninitialized' } >>>>> at T.main(T.java:4) >>>> >>>> Yes I think this may be a bug in hotspot. The type-checking for the >>>> monitor bytecodes requires a matching type of reference on the operand >>>> stack - but "uninitialized" does not match Object, as J9 reports. But >>>> I'm not an expert on this aspect of verification so I may not be >>>> interpreting it correctly. >>>> >>>> More below ... >>>> >>>>> -----????----- >>>>> ???: David Holmes [mailto:david.holmes at oracle.com] >>>>> ????: 2017?5?17? 14:41 >>>>> ???: ??? ; >>>>> hotspot-runtime-dev at openjdk.java.net >>>>> ??: Re: HotSpot and IBM's J9 behave quite differently when >>>>> processing >>>>> monitorenters and monitorexits >>>>> >>>>> Hi, >>>>> On 18/05/2017 7:23 AM, ??? wrote: >>>>>> Am I wrong? >>>>> >>>>> I will look at each situation in detail when I get a chance but >>>>> structured locking enforcement is optional. Also balancing the number >>>>> of locks and unlocks in a frame does not mean they can't be locked >>>>> and unlocked in a non-nested fashion - just that by the end the >>>>> number of unlocks matches the number of locks. >>>>> >>>>> BTW the way you respond to these emails, as if having a conversation >>>>> with yourself, makes it difficult to respond as we can't readily see >>>>> what is the new email and what is the original. >>>>> >>>>> Cheers, >>>>> David >>>>> >>>>>> The byte code for main() in case 1 is as follows. The strange thing >>>>>> is that NullPointerException is also not thrown at runtime. >>>> >>>> That is strange as it does for the normal obvious case of using >>>> synchronized(o) when o is null. >>> >>> Ah - it isn't null it just an object for which the constructor has >>> not been >>> run. The runtime can't tell the difference between a pointer to a valid >>> initialized object, and a pointer to an uninitialized chunk of memory. >>> >>>>>> >>>>>> public void main(java.lang.String[]) throws java.lang.Exception; >>>>>> descriptor: ([Ljava/lang/String;)V >>>>>> flags: ACC_PUBLIC >>>>>> Code: >>>>>> stack=3, locals=2, args_size=2 >>>>>> 0: new #2 // class Search >>> >>> This allocated an object - hence no null reference. But this is what >>> verification should have complained about. >>> >>> David >>> ----- >>> >>>>>> 3: dup >>>>>> 4: aload_0 >>>>>> 5: monitorenter >>>>>> 6: monitorenter >>>>>> 7: monitorexit >>>>>> 8: aload_0 >>>>>> 9: monitorexit >>>>>> 10: return >>>>>> Exceptions: >>>>>> throws java.lang.Exception >>>>>> >>>>>> ??: HotSpot and IBM's J9 behave quite differently when processing >>>>>> monitorenters and monitorexits >>>>>> >>>>>> I have tested several programs (in Jimple) and found that HotSpot >>>>>> and >>>>>> J9 match monitorenters and monitorexits quite differently. Verifiers >>>>>> should play more important roles here. >>>> >>>> The job of the verifier is to establish some basic guarantees for the >>>> JVM to then operate under. The verifier plays no role in checking how >>>> monitorenter/exit are used in combination, only that each individual >>>> bytecode meets some basic type constraints. >>>> >>>>>> >>>>>> (1) Test the next program (r2 is not initizlied) on HotSpot and J9. >>>>>> J9 throw out a verifier error, while HotSpot does not. It seems that >>>>>> HotSpot's verifier forgets to check whether a monitored object is >>>>>> initialized. >>>>>> >>>>>> public class Search extends java.lang.Object { public void () >>>>>> { >>>>>> Search r0; >>>>>> r0 := @this: Search; >>>>>> specialinvoke r0.()>(); >>>>>> return; >>>>>> } >>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>> { >>>>>> Search r0; >>>>>> Search r2; >>>>>> java.lang.String[] r1; >>>>>> r0 := @this: Search; >>>>>> r1 := @parameter0: java.lang.String[]; >>>>>> r2 = new Search; >>>>>> >>>>>> entermonitor r2; >>>>>> entermonitor r0; >>>>>> exitmonitor r2; >>>>>> exitmonitor r0; >>>>>> return; >>>>>> } >>>>>> } >>>> >>>> Verification was covered above. >>>> >>>>>> >>>>>> (2) Test the next program on HotSpot and J9, and both do not report >>>>>> any errors. However, I guess the order in the program (entermonitor >>>>>> r2; => entermonitor r0; => exitmonitor r2; => exitmonitor r0;) >>>>>> violates the situation of "structured locking" (Structured locking >>>>>> is the situation when, during a method invocation, every exit on a >>>>>> given monitor matches a preceding entry on that monitor, see the >>>>>> specification >>>>>> >>>>>> https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-2.html#jvms-2. >>>>>> >>>>>> 11.10) >>>>>> ? >>>> >>>> No it doesn't violate structured locking as the number of enters and >>>> exits match, and there is always an enter before an exit. >>>> >>>>>> Actually, the words (every exit on a given monitor matches a >>>>>> preceding entry on that monitor) are not quite clear as for me. >>>>>> Otherwise the first rule (The number of monitor entries performed by >>>>>> T on M during a method invocation must equal the number of monitor >>>>>> exits performed by T on M during the method invocation whether the >>>>>> method invocation completes normally or abruptly.) is sufficient. >>>> >>>> The number of enters and exits must not only match/balance, but there >>>> must be an enter before a corresponding exit. >>>> >>>>>> >>>>>> public class Search extends java.lang.Object { >>>>>> >>>>>> public void () >>>>>> { >>>>>> Search r0; >>>>>> r0 := @this: Search; >>>>>> specialinvoke r0.()>(); >>>>>> return; >>>>>> } >>>>>> >>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>> { >>>>>> Search r0; >>>>>> Search r2; >>>>>> java.lang.String[] r1; >>>>>> r0 := @this: Search; >>>>>> r1 := @parameter0: java.lang.String[]; >>>>>> r2 = new Search; >>>>>> specialinvoke r2.()>(); >>>>>> entermonitor r2; >>>>>> entermonitor r0; >>>>>> exitmonitor r2; >>>>>> exitmonitor r0; >>>>>> return; >>>>>> } >>>>>> } >>>>>> >>>>>> (3) The next program enters monitor in and exits it in main(). >>>>>> HotSpot throws a runtime exception, while J9 does not. Should this >>>>>> program be rejected by the verifiers? >>>> >>>> No this does not violate any verification rules. The runtime behaviour >>>> depends on whether structured locking is enforced or not.(Even in >>>> hotspot there can be differences between interpreted and jitted code). >>>> >>>> Hope that helps clarify things. >>>> >>>> David >>>> ----- >>>> >>>>>> >>>>>> public class Search extends java.lang.Object { >>>>>> >>>>>> public void () >>>>>> { >>>>>> Search r0; >>>>>> r0 := @this: Search; >>>>>> specialinvoke r0.()>(); >>>>>> entermonitor r0; >>>> From chenyt at cs.sjtu.edu.cn Thu May 18 05:35:47 2017 From: chenyt at cs.sjtu.edu.cn (=?UTF-8?B?6ZmI6Zuo5Lqt?=) Date: Wed, 17 May 2017 22:35:47 -0700 Subject: =?UTF-8?Q?=E7=AD=94=E5=A4=8D:_=E7=AD=94=E5=A4=8D:_HotSpot_and_IBM's_J9_beh?= =?UTF-8?Q?ave_quite_differently_when_proce?= =?UTF-8?Q?ssing_monitorenters_and_monitore?= =?UTF-8?Q?xits?= In-Reply-To: <24a31d5c-8185-e55d-481f-719f3949bf7d@oracle.com> References: <005101d2cf53$c8c16b40$5a4441c0$@cs.sjtu.edu.cn> <656dcb18-e65a-4c48-a6bc-cf00fe021af5@oracle.com> <005201d2cf5a$aa2e8f30$fe8bad90$@cs.sjtu.edu.cn> <5a6a2de2-eb43-985d-bfc7-18c3fa7ab0ee@oracle.com> <24a31d5c-8185-e55d-481f-719f3949bf7d@oracle.com> Message-ID: <000101d2cf98$9b9f9070$d2deb150$@cs.sjtu.edu.cn> Sorry for my poor mailbox. The bytecode of the previous two programs. Hope that they will help find identify the reasons. (1) public class Search minor version: 0 major version: 52 flags: ACC_PUBLIC, ACC_SUPER Constant pool: #1 = Utf8 Search #2 = Class #1 // Search #3 = Utf8 java/lang/Object #4 = Class #3 // java/lang/Object #5 = Utf8 #6 = Utf8 ()V #7 = Utf8 #8 = NameAndType #7:#6 // "":()V #9 = Methodref #4.#8 // java/lang/Object."":()V #10 = Utf8 main #11 = Utf8 ([Ljava/lang/String;)V #12 = Utf8 java/lang/Exception #13 = Class #12 // java/lang/Exception #14 = Utf8 Code #15 = Utf8 Exceptions { public static {}; descriptor: ()V flags: ACC_PUBLIC, ACC_STATIC Code: stack=0, locals=0, args_size=0 0: return public Search(); descriptor: ()V flags: ACC_PUBLIC Code: stack=1, locals=1, args_size=1 0: aload_0 1: invokespecial #9 // Method java/lang/Object."":()V 4: return public void main(java.lang.String[]) throws java.lang.Exception; descriptor: ([Ljava/lang/String;)V flags: ACC_PUBLIC Code: stack=1, locals=2, args_size=2 0: aload_0 1: monitorenter 2: aconst_null 3: monitorenter 4: aconst_null 5: monitorexit 6: aload_0 7: monitorexit 8: return Exceptions: throws java.lang.Exception } (2) Classfile /home/yuting/Desktop/Search.class Last modified May 17, 2017; size 276 bytes MD5 checksum 7fdacbbecd7521380b74b755be7acd14 public class Search minor version: 0 major version: 52 flags: ACC_PUBLIC, ACC_SUPER Constant pool: #1 = Utf8 Search #2 = Class #1 // Search #3 = Utf8 java/lang/Object #4 = Class #3 // java/lang/Object #5 = Utf8 #6 = Utf8 ()V #7 = Utf8 #8 = NameAndType #7:#6 // "":()V #9 = Methodref #4.#8 // java/lang/Object."":()V #10 = Utf8 main #11 = Utf8 ([Ljava/lang/String;)V #12 = Utf8 java/lang/Exception #13 = Class #12 // java/lang/Exception #14 = Utf8 Code #15 = Utf8 Exceptions { public static {}; descriptor: ()V flags: ACC_PUBLIC, ACC_STATIC Code: stack=0, locals=0, args_size=0 0: return public Search(); descriptor: ()V flags: ACC_PUBLIC Code: stack=1, locals=1, args_size=1 0: aload_0 1: invokespecial #9 // Method java/lang/Object."":()V 4: return public void main(java.lang.String[]) throws java.lang.Exception; descriptor: ([Ljava/lang/String;)V flags: ACC_PUBLIC Code: stack=2, locals=2, args_size=2 0: new #2 // class Search 3: aload_0 4: monitorenter 5: monitorenter 6: aconst_null 7: monitorexit 8: aload_0 9: monitorexit 10: return Exceptions: throws java.lang.Exception } -----????----- ???: David Holmes [mailto:david.holmes at oracle.com] ????: 2017?5?17? 22:26 ???: chenyt ??: hotspot-runtime-dev at openjdk.java.net ??: Re: ??: HotSpot and IBM's J9 behave quite differently when processing monitorenters and monitorexits Hi Yuting, On 18/05/2017 2:49 PM, chenyt wrote: > Hi, David, > > I tested the next program (let one object to be monitored be null) > using > J9 and HotSpot. HotSpot does not throw a NullPointerException, as the > specification says. J9 looks fine (throw a NullPointerException). Am I > wrong here? I can't readily test this as I don't have Jimple nor quick and easy access to bytecode assemblers. It is strange though as the interpreter code contains this: CASE(_monitorenter): { oop lockee = STACK_OBJECT(-1); // derefing's lockee ought to provoke implicit null check CHECK_NULL(lockee); If I can find some time I will try to test this myself. > It is very interesting that I have found so many unexpected behaviors here. Well you only found two and they are related. :) Manually assembled monitor code is not something very many (any?) people care about or do - other than emulating correct language usage. Cheers, David > public void main(java.lang.String[]) throws java.lang.Exception; > descriptor: ([Ljava/lang/String;)V > flags: ACC_PUBLIC > Code: > stack=1, locals=2, args_size=2 > 0: aload_0 > 1: monitorenter > 2: aconst_null > 3: monitorenter > 4: aconst_null > 5: monitorexit > 6: aload_0 > 7: monitorexit > 8: return > Exceptions: > throws java.lang.Exception > > Jimple code is given as follows: > public void main(java.lang.String[]) throws java.lang.Exception > { > Search r0; > Search r2; > java.lang.String[] r1; > > r0 := @this: Search; > r1 := @parameter0: java.lang.String[]; > r2 = null; > entermonitor r0; > entermonitor r2; > exitmonitor r2; > exitmonitor r0; > > return; > } > > Wishes, > Yuting > > > On Thu, 18 May 2017 13:23:09 +1000, David Holmes wrote: >> One correction ... >> >> On 18/05/2017 10:29 AM, David Holmes wrote: >>> On 18/05/2017 8:12 AM, ??? wrote: >>>> Thank you, David. I have seen from the specification that >>>> structured locking enforcement is optional. The second and the >>>> third ones are cases of structured/nested lockings. Will non-nested >>>> locking sequences raise deadlocks? Of course it is a different >>>> topic, while it might be better if it can be kicked out earlier >>>> from the specification/JVM. >>> >>> Non-nested locking doesn't necessarily lead to deadlocks - you just >>> need a different locking order for that (even if properly nested). >>> There are locking patterns that rely on the ability to lock and >>> unlock in different order ie chained-locking for walking linked-lists A->B->C->D: >>> - lock A, lock B, unlock A, lock C, unlock B, lock D, unlock C ... >>> >>> The VM spec allows a little flexibility in how the monitor bytecodes >>> can be used compared to the Java programming language. That's not >>> something that will change. >>> >>>> The first example is still a problem. It seems that HotSpot allows >>>> to monitor a pure object reference without initialized (Is it true? >>>> How can this checking be omitted?). J9 reports a verifyerror as >>>> follows. >>>> >>>> Exception in thread "main" java.lang.VerifyError: JVMVRFY012 stack >>>> shape inconsistent; class=Search, >>>> method=main([Ljava/lang/String;)V, pc=6 Exception Details: >>>> Location: >>>> Search.main([Ljava/lang/String;)V @6: JBmonitorenter >>>> Reason: >>>> Type 'uninitialized' (current frame, stack[1]) is not >>>> assignable to 'java/lang/Object' >>>> Current Frame: >>>> bci: @6 >>>> flags: { } >>>> locals: { 'Search', '[Ljava/lang/String;' } >>>> stack: { 'uninitialized', 'uninitialized' } >>>> at T.main(T.java:4) >>> >>> Yes I think this may be a bug in hotspot. The type-checking for the >>> monitor bytecodes requires a matching type of reference on the >>> operand stack - but "uninitialized" does not match Object, as J9 >>> reports. But I'm not an expert on this aspect of verification so I >>> may not be interpreting it correctly. >>> >>> More below ... >>> >>>> -----????----- >>>> ???: David Holmes [mailto:david.holmes at oracle.com] >>>> ????: 2017?5?17? 14:41 >>>> ???: ??? ; >>>> hotspot-runtime-dev at openjdk.java.net >>>> ??: Re: HotSpot and IBM's J9 behave quite differently when >>>> processing monitorenters and monitorexits >>>> >>>> Hi, >>>> On 18/05/2017 7:23 AM, ??? wrote: >>>>> Am I wrong? >>>> >>>> I will look at each situation in detail when I get a chance but >>>> structured locking enforcement is optional. Also balancing the >>>> number of locks and unlocks in a frame does not mean they can't be >>>> locked and unlocked in a non-nested fashion - just that by the end >>>> the number of unlocks matches the number of locks. >>>> >>>> BTW the way you respond to these emails, as if having a >>>> conversation with yourself, makes it difficult to respond as we >>>> can't readily see what is the new email and what is the original. >>>> >>>> Cheers, >>>> David >>>> >>>>> The byte code for main() in case 1 is as follows. The strange >>>>> thing is that NullPointerException is also not thrown at runtime. >>> >>> That is strange as it does for the normal obvious case of using >>> synchronized(o) when o is null. >> >> Ah - it isn't null it just an object for which the constructor has >> not been run. The runtime can't tell the difference between a pointer >> to a valid initialized object, and a pointer to an uninitialized >> chunk of memory. >> >>>>> >>>>> public void main(java.lang.String[]) throws java.lang.Exception; >>>>> descriptor: ([Ljava/lang/String;)V >>>>> flags: ACC_PUBLIC >>>>> Code: >>>>> stack=3, locals=2, args_size=2 >>>>> 0: new #2 // class Search >> >> This allocated an object - hence no null reference. But this is what >> verification should have complained about. >> >> David >> ----- >> >>>>> 3: dup >>>>> 4: aload_0 >>>>> 5: monitorenter >>>>> 6: monitorenter >>>>> 7: monitorexit >>>>> 8: aload_0 >>>>> 9: monitorexit >>>>> 10: return >>>>> Exceptions: >>>>> throws java.lang.Exception >>>>> >>>>> ??: HotSpot and IBM's J9 behave quite differently when processing >>>>> monitorenters and monitorexits >>>>> >>>>> I have tested several programs (in Jimple) and found that HotSpot >>>>> and >>>>> J9 match monitorenters and monitorexits quite differently. >>>>> Verifiers should play more important roles here. >>> >>> The job of the verifier is to establish some basic guarantees for >>> the JVM to then operate under. The verifier plays no role in >>> checking how monitorenter/exit are used in combination, only that >>> each individual bytecode meets some basic type constraints. >>> >>>>> >>>>> (1) Test the next program (r2 is not initizlied) on HotSpot and J9. >>>>> J9 throw out a verifier error, while HotSpot does not. It seems >>>>> that HotSpot's verifier forgets to check whether a monitored >>>>> object is initialized. >>>>> >>>>> public class Search extends java.lang.Object { public void () >>>>> { >>>>> Search r0; >>>>> r0 := @this: Search; >>>>> specialinvoke r0.()>(); >>>>> return; >>>>> } >>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>> { >>>>> Search r0; >>>>> Search r2; >>>>> java.lang.String[] r1; >>>>> r0 := @this: Search; >>>>> r1 := @parameter0: java.lang.String[]; >>>>> r2 = new Search; >>>>> >>>>> entermonitor r2; >>>>> entermonitor r0; >>>>> exitmonitor r2; >>>>> exitmonitor r0; >>>>> return; >>>>> } >>>>> } >>> >>> Verification was covered above. >>> >>>>> >>>>> (2) Test the next program on HotSpot and J9, and both do not >>>>> report any errors. However, I guess the order in the program >>>>> (entermonitor r2; => entermonitor r0; => exitmonitor r2; => >>>>> exitmonitor r0;) violates the situation of "structured locking" >>>>> (Structured locking is the situation when, during a method >>>>> invocation, every exit on a given monitor matches a preceding >>>>> entry on that monitor, see the specification >>>>> >>>>> https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-2.html#jvms-2. >>>>> 11.10) >>>>> ? >>> >>> No it doesn't violate structured locking as the number of enters and >>> exits match, and there is always an enter before an exit. >>> >>>>> Actually, the words (every exit on a given monitor matches a >>>>> preceding entry on that monitor) are not quite clear as for me. >>>>> Otherwise the first rule (The number of monitor entries performed >>>>> by T on M during a method invocation must equal the number of >>>>> monitor exits performed by T on M during the method invocation >>>>> whether the method invocation completes normally or abruptly.) is sufficient. >>> >>> The number of enters and exits must not only match/balance, but >>> there must be an enter before a corresponding exit. >>> >>>>> >>>>> public class Search extends java.lang.Object { >>>>> >>>>> public void () >>>>> { >>>>> Search r0; >>>>> r0 := @this: Search; >>>>> specialinvoke r0.()>(); >>>>> return; >>>>> } >>>>> >>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>> { >>>>> Search r0; >>>>> Search r2; >>>>> java.lang.String[] r1; >>>>> r0 := @this: Search; >>>>> r1 := @parameter0: java.lang.String[]; >>>>> r2 = new Search; >>>>> specialinvoke r2.()>(); >>>>> entermonitor r2; >>>>> entermonitor r0; >>>>> exitmonitor r2; >>>>> exitmonitor r0; >>>>> return; >>>>> } >>>>> } >>>>> >>>>> (3) The next program enters monitor in and exits it in main(). >>>>> HotSpot throws a runtime exception, while J9 does not. Should this >>>>> program be rejected by the verifiers? >>> >>> No this does not violate any verification rules. The runtime >>> behaviour depends on whether structured locking is enforced or >>> not.(Even in hotspot there can be differences between interpreted and jitted code). >>> >>> Hope that helps clarify things. >>> >>> David >>> ----- >>> >>>>> >>>>> public class Search extends java.lang.Object { >>>>> >>>>> public void () >>>>> { >>>>> Search r0; >>>>> r0 := @this: Search; >>>>> specialinvoke r0.()>(); >>>>> entermonitor r0; >>>>> return; >>>>> } >>>>> >>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>> { >>>>> Search r0; >>>>> Search r2; >>>>> java.lang.S From chenyt at cs.sjtu.edu.cn Thu May 18 06:02:44 2017 From: chenyt at cs.sjtu.edu.cn (=?UTF-8?B?6ZmI6Zuo5Lqt?=) Date: Wed, 17 May 2017 23:02:44 -0700 Subject: =?UTF-8?Q?=E7=AD=94=E5=A4=8D:_=E7=AD=94=E5=A4=8D:_=E7=AD=94=E5=A4=8D:_HotS?= =?UTF-8?Q?pot_and_IBM's_J9_behave_quite_di?= =?UTF-8?Q?fferently_when_processing_monito?= =?UTF-8?Q?renters_and_monitorexits?= In-Reply-To: References: <005101d2cf53$c8c16b40$5a4441c0$@cs.sjtu.edu.cn> <656dcb18-e65a-4c48-a6bc-cf00fe021af5@oracle.com> <005201d2cf5a$aa2e8f30$fe8bad90$@cs.sjtu.edu.cn> <5a6a2de2-eb43-985d-bfc7-18c3fa7ab0ee@oracle.com> <000401d2cf8d$c8352470$589f6d50$@cs.sjtu.edu.cn> <18003d32-08ad-4891-cf2a-4ba0d71a2bb0@oracle.com> <9a93ed9268233cf5847156fc7ceef5d5@cs.sjtu.edu.cn> Message-ID: <000301d2cf9c$5f4d3970$1de7ac50$@cs.sjtu.edu.cn> I agree. There may be a typo when an exception needs to be thrown,:) https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-6.html#jvms-6.5.monitorenter Yuting -----????----- ???: David Holmes [mailto:david.holmes at oracle.com] ????: 2017?5?17? 22:29 ???: chenyt ??: hotspot-runtime-dev at openjdk.java.net ??: Re: ??: ??: HotSpot and IBM's J9 behave quite differently when processing monitorenters and monitorexits Is this any different from what you previously reported?? Sorry I'm having a hard time tracking the scenarios. As far as I can tell there are two cases: - new'd but uninitialized objectref This should be a verification error in which case runtime behaviour is unspecified. - null reference This should throw NullPointerException, and as per last email the interpreter explicitly has a null check, so I don't know why this wouldn't throw. David On 18/05/2017 3:15 PM, chenyt wrote: > Hi, David, > > I tested the next Jimple programs (let one object under monitoring be > null) using J9 and HotSpot. I did not see NullPointerException here > (the specification does tell me so). Am I wrong again? Hope the next > two cases will help check. > > (1) case 1: HotSpot: IllegalMonitorStateException J9: NullPointerException > r0 := @this: Search; > r1 := @parameter0: java.lang.String[]; > r2 = null; > entermonitor r0; > entermonitor r2; > exitmonitor r2; > exitmonitor r0; > (2) case 2: HotSpot: IllegalMonitorStateException J9: VerifyError > r1 := @parameter0: java.lang.String[]; > r2 = new Search; > entermonitor r0; > entermonitor r2; > r2=null; > exitmonitor r2; > exitmonitor r0; > It is very interesting that I can still find some unexpected behaviors > here. > > Wishes, > Yuting > > > On Thu, 18 May 2017 14:57:03 +1000, David Holmes wrote: >> On 18/05/2017 2:18 PM, ??? wrote: >>> Hi, David, >>> >>> I still did not catch why it happens... I just read the >>> specification and wrote some tests in order to understand some >>> paragraphs better, and then had questions. >>> >>> Will the HotSpot's verifier detect such kind of uninitialized >>> monitored objects in future? >> >> I have filed a bug so this can be examined more closely. >> >> https://bugs.openjdk.java.net/browse/JDK-8180581 >> >> Thanks, >> David >> >>> Regards, >>> Yuting >>> >>> -----????----- >>> ???: David Holmes [mailto:david.holmes at oracle.com] >>> ????: 2017?5?17? 20:23 >>> ???: ??? ; >>> hotspot-runtime-dev at openjdk.java.net >>> ??: Re: ??: HotSpot and IBM's J9 behave quite differently when >>> processing monitorenters and monitorexits >>> >>> One correction ... >>> >>> On 18/05/2017 10:29 AM, David Holmes wrote: >>>> On 18/05/2017 8:12 AM, ??? wrote: >>>>> Thank you, David. I have seen from the specification that >>>>> structured locking enforcement is optional. The second and the >>>>> third ones are cases of structured/nested lockings. Will >>>>> non-nested locking sequences raise deadlocks? Of course it is a >>>>> different topic, while it might be better if it can be kicked out >>>>> earlier from the specification/JVM. >>>> >>>> Non-nested locking doesn't necessarily lead to deadlocks - you just >>>> need a different locking order for that (even if properly nested). >>>> There are locking patterns that rely on the ability to lock and >>>> unlock in different order ie chained-locking for walking >>>> linked-lists >>>> A->B->C->D: >>>> - lock A, lock B, unlock A, lock C, unlock B, lock D, unlock C ... >>>> >>>> The VM spec allows a little flexibility in how the monitor >>>> bytecodes can be used compared to the Java programming language. >>>> That's not something that will change. >>>> >>>>> The first example is still a problem. It seems that HotSpot allows >>>>> to monitor a pure object reference without initialized (Is it >>>>> true? How can this checking be omitted?). J9 reports a verifyerror as follows. >>>>> >>>>> Exception in thread "main" java.lang.VerifyError: JVMVRFY012 stack >>>>> shape inconsistent; class=Search, >>>>> method=main([Ljava/lang/String;)V, >>>>> pc=6 Exception Details: >>>>> Location: >>>>> Search.main([Ljava/lang/String;)V @6: JBmonitorenter >>>>> Reason: >>>>> Type 'uninitialized' (current frame, stack[1]) is not >>>>> assignable to 'java/lang/Object' >>>>> Current Frame: >>>>> bci: @6 >>>>> flags: { } >>>>> locals: { 'Search', '[Ljava/lang/String;' } >>>>> stack: { 'uninitialized', 'uninitialized' } >>>>> at T.main(T.java:4) >>>> >>>> Yes I think this may be a bug in hotspot. The type-checking for the >>>> monitor bytecodes requires a matching type of reference on the >>>> operand stack - but "uninitialized" does not match Object, as J9 >>>> reports. But I'm not an expert on this aspect of verification so I >>>> may not be interpreting it correctly. >>>> >>>> More below ... >>>> >>>>> -----????----- >>>>> ???: David Holmes [mailto:david.holmes at oracle.com] >>>>> ????: 2017?5?17? 14:41 >>>>> ???: ??? ; >>>>> hotspot-runtime-dev at openjdk.java.net >>>>> ??: Re: HotSpot and IBM's J9 behave quite differently when >>>>> processing monitorenters and monitorexits >>>>> >>>>> Hi, >>>>> On 18/05/2017 7:23 AM, ??? wrote: >>>>>> Am I wrong? >>>>> >>>>> I will look at each situation in detail when I get a chance but >>>>> structured locking enforcement is optional. Also balancing the >>>>> number of locks and unlocks in a frame does not mean they can't be >>>>> locked and unlocked in a non-nested fashion - just that by the end >>>>> the number of unlocks matches the number of locks. >>>>> >>>>> BTW the way you respond to these emails, as if having a >>>>> conversation with yourself, makes it difficult to respond as we >>>>> can't readily see what is the new email and what is the original. >>>>> >>>>> Cheers, >>>>> David >>>>> >>>>>> The byte code for main() in case 1 is as follows. The strange >>>>>> thing is that NullPointerException is also not thrown at runtime. >>>> >>>> That is strange as it does for the normal obvious case of using >>>> synchronized(o) when o is null. >>> >>> Ah - it isn't null it just an object for which the constructor has >>> not been run. The runtime can't tell the difference between a >>> pointer to a valid initialized object, and a pointer to an >>> uninitialized chunk of memory. >>> >>>>>> >>>>>> public void main(java.lang.String[]) throws java.lang.Exception; >>>>>> descriptor: ([Ljava/lang/String;)V >>>>>> flags: ACC_PUBLIC >>>>>> Code: >>>>>> stack=3, locals=2, args_size=2 >>>>>> 0: new #2 // class Search >>> >>> This allocated an object - hence no null reference. But this is what >>> verification should have complained about. >>> >>> David >>> ----- >>> >>>>>> 3: dup >>>>>> 4: aload_0 >>>>>> 5: monitorenter >>>>>> 6: monitorenter >>>>>> 7: monitorexit >>>>>> 8: aload_0 >>>>>> 9: monitorexit >>>>>> 10: return >>>>>> Exceptions: >>>>>> throws java.lang.Exception >>>>>> >>>>>> ??: HotSpot and IBM's J9 behave quite differently when processing >>>>>> monitorenters and monitorexits >>>>>> >>>>>> I have tested several programs (in Jimple) and found that HotSpot >>>>>> and >>>>>> J9 match monitorenters and monitorexits quite differently. >>>>>> Verifiers should play more important roles here. >>>> >>>> The job of the verifier is to establish some basic guarantees for >>>> the JVM to then operate under. The verifier plays no role in >>>> checking how monitorenter/exit are used in combination, only that >>>> each individual bytecode meets some basic type constraints. >>>> >>>>>> >>>>>> (1) Test the next program (r2 is not initizlied) on HotSpot and J9. >>>>>> J9 throw out a verifier error, while HotSpot does not. It seems >>>>>> that HotSpot's verifier forgets to check whether a monitored >>>>>> object is initialized. >>>>>> >>>>>> public class Search extends java.lang.Object { public void () >>>>>> { >>>>>> Search r0; >>>>>> r0 := @this: Search; >>>>>> specialinvoke r0.()>(); >>>>>> return; >>>>>> } >>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>> { >>>>>> Search r0; >>>>>> Search r2; >>>>>> java.lang.String[] r1; >>>>>> r0 := @this: Search; >>>>>> r1 := @parameter0: java.lang.String[]; >>>>>> r2 = new Search; >>>>>> >>>>>> entermonitor r2; >>>>>> entermonitor r0; >>>>>> exitmonitor r2; >>>>>> exitmonitor r0; >>>>>> return; >>>>>> } >>>>>> } >>>> >>>> Verification was covered above. >>>> >>>>>> >>>>>> (2) Test the next program on HotSpot and J9, and both do not >>>>>> report any errors. However, I guess the order in the program >>>>>> (entermonitor r2; => entermonitor r0; => exitmonitor r2; => >>>>>> exitmonitor r0;) violates the situation of "structured locking" >>>>>> (Structured locking is the situation when, during a method >>>>>> invocation, every exit on a given monitor matches a preceding >>>>>> entry on that monitor, see the specification >>>>>> >>>>>> https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-2.html#jvms-2. >>>>>> >>>>>> 11.10) >>>>>> ? >>>> >>>> No it doesn't violate structured locking as the number of enters >>>> and exits match, and there is always an enter before an exit. >>>> >>>>>> Actually, the words (every exit on a given monitor matches a >>>>>> preceding entry on that monitor) are not quite clear as for me. >>>>>> Otherwise the first rule (The number of monitor entries performed >>>>>> by T on M during a method invocation must equal the number of >>>>>> monitor exits performed by T on M during the method invocation >>>>>> whether the method invocation completes normally or abruptly.) is sufficient. >>>> >>>> The number of enters and exits must not only match/balance, but >>>> there must be an enter before a corresponding exit. >>>> >>>>>> >>>>>> public class Search extends java.lang.Object { >>>>>> >>>>>> public void () >>>>>> { >>>>>> Search r0; >>>>>> r0 := @this: Search; >>>>>> specialinvoke r0.()>(); >>>>>> return; >>>>>> } >>>>>> >>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>> { >>>>>> Search r0; >>>>>> Search r2; >>>>>> java.lang.String[] r1; >>>>>> r0 := @this: Search; >>>>>> r1 := @parameter0: java.lang.String[]; >>>>>> r2 = new Search; >>>>>> specialinvoke r2.()>(); >>>>>> entermonitor r2; >>>>>> entermonitor r0; >>>>>> exitmonitor r2; >>>>>> exitmonitor r0; >>>>>> return; >>>>>> } >>>>>> } >>>>>> >>>>>> (3) The next program enters monitor in and exits it in main(). >>>>>> HotSpot throws a runtime exception, while J9 does not. Should >>>>>> this program be rejected by the verifiers? >>>> >>>> No this does not violate any verification rules. The runtime >>>> behaviour depends on whether structured locking is enforced or >>>> not.(Even in hotspot there can be differences between interpreted and jitted code). >>>> >>>> Hope that helps clarify things. >>>> >>>> David >>>> ----- >>>> >>>>>> >>>>>> public class Search extends java.lang.Object { >>>>>> >>>>>> public void () >>>>>> { >>>>>> Search r0; >>>>>> r0 := @this: Search; >>>>>> specialinvoke r0.()>(); >>>>>> entermonitor r0; >>>> From rkennke at redhat.com Thu May 18 07:54:08 2017 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 18 May 2017 09:54:08 +0200 Subject: RFR (jdk10): JDK-8180175: ObjectSynchronizer only needs to iterate in-use monitors In-Reply-To: References: <89399a72-0263-5ebd-5fda-82f404fbe2ca@redhat.com> <8d75eef3-c143-dd75-6362-6203c7fb175f@redhat.com> <3a0f4490-646c-8004-6272-180ef198b67e@oracle.com> <13484ee1-5c37-6971-028d-2ddf0b0f2845@redhat.com> Message-ID: Am 18.05.2017 um 04:07 schrieb David Holmes: > Roman, > > I just pushed this but now I think there is a problem with this > optimization. > > When a thread terminates, omFlush is used to transfer the thread's > inUseList to the global inUseList. This happens after the thread has > been removed from the _threads_list and can run concurrently with a > safepoint. This means that there is a window of time where the > monitors in the thread's inUseList will not be seen by the safepoint > oops_do processing: > - the thread oops_do won't be called because the thread is not in the > thread's list > - the monitors are not yet in the global in-use-list > > Did I overlook something? Ugh. How ugly :-) Is there a reason why omFlush cannot be called from the same block where the thread is removed from _thread_list, under the Threads_lock ? I need to think about this a little more... Roman From david.holmes at oracle.com Thu May 18 08:03:15 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 18 May 2017 18:03:15 +1000 Subject: RFR (jdk10): JDK-8180175: ObjectSynchronizer only needs to iterate in-use monitors In-Reply-To: References: <89399a72-0263-5ebd-5fda-82f404fbe2ca@redhat.com> <8d75eef3-c143-dd75-6362-6203c7fb175f@redhat.com> <3a0f4490-646c-8004-6272-180ef198b67e@oracle.com> <13484ee1-5c37-6971-028d-2ddf0b0f2845@redhat.com> Message-ID: <4e5f18fb-972b-82ad-b84b-feddc6b25d4e@oracle.com> On 18/05/2017 5:54 PM, Roman Kennke wrote: > Am 18.05.2017 um 04:07 schrieb David Holmes: >> Roman, >> >> I just pushed this but now I think there is a problem with this >> optimization. >> >> When a thread terminates, omFlush is used to transfer the thread's >> inUseList to the global inUseList. This happens after the thread has >> been removed from the _threads_list and can run concurrently with a >> safepoint. This means that there is a window of time where the >> monitors in the thread's inUseList will not be seen by the safepoint >> oops_do processing: >> - the thread oops_do won't be called because the thread is not in the >> thread's list >> - the monitors are not yet in the global in-use-list >> >> Did I overlook something? > > Ugh. How ugly :-) > > Is there a reason why omFlush cannot be called from the same block where > the thread is removed from _thread_list, under the Threads_lock ? I think the explicit intent is to let it run independent of safepoints (which holding the Threads_lock would affect). I guess it depends how long it takes to run in general ... and we'd need to check that the code executed while holding the gListLock is truly leaf code. David > I need to think about this a little more... > > Roman > From rkennke at redhat.com Thu May 18 08:08:57 2017 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 18 May 2017 10:08:57 +0200 Subject: RFR (jdk10): JDK-8180175: ObjectSynchronizer only needs to iterate in-use monitors In-Reply-To: <4e5f18fb-972b-82ad-b84b-feddc6b25d4e@oracle.com> References: <89399a72-0263-5ebd-5fda-82f404fbe2ca@redhat.com> <8d75eef3-c143-dd75-6362-6203c7fb175f@redhat.com> <3a0f4490-646c-8004-6272-180ef198b67e@oracle.com> <13484ee1-5c37-6971-028d-2ddf0b0f2845@redhat.com> <4e5f18fb-972b-82ad-b84b-feddc6b25d4e@oracle.com> Message-ID: Am 18.05.2017 um 10:03 schrieb David Holmes: > On 18/05/2017 5:54 PM, Roman Kennke wrote: >> Am 18.05.2017 um 04:07 schrieb David Holmes: >>> Roman, >>> >>> I just pushed this but now I think there is a problem with this >>> optimization. >>> >>> When a thread terminates, omFlush is used to transfer the thread's >>> inUseList to the global inUseList. This happens after the thread has >>> been removed from the _threads_list and can run concurrently with a >>> safepoint. This means that there is a window of time where the >>> monitors in the thread's inUseList will not be seen by the safepoint >>> oops_do processing: >>> - the thread oops_do won't be called because the thread is not in the >>> thread's list >>> - the monitors are not yet in the global in-use-list >>> >>> Did I overlook something? >> >> Ugh. How ugly :-) >> >> Is there a reason why omFlush cannot be called from the same block where >> the thread is removed from _thread_list, under the Threads_lock ? > > I think the explicit intent is to let it run independent of safepoints > (which holding the Threads_lock would affect). But Thread::remove() *is* going to block on Threads_lock. If I read the code correctly, omFlush() is called right after we give up Threads_lock (thread_main_inner()). Stuff under gListLock in omFlush() is only doing some simple re-linking of the linked lists. I'll give that a try with some very careful checking ... Roman From david.holmes at oracle.com Thu May 18 08:10:54 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 18 May 2017 18:10:54 +1000 Subject: RFR (jdk10): JDK-8180175: ObjectSynchronizer only needs to iterate in-use monitors In-Reply-To: References: <89399a72-0263-5ebd-5fda-82f404fbe2ca@redhat.com> <8d75eef3-c143-dd75-6362-6203c7fb175f@redhat.com> <3a0f4490-646c-8004-6272-180ef198b67e@oracle.com> <13484ee1-5c37-6971-028d-2ddf0b0f2845@redhat.com> <4e5f18fb-972b-82ad-b84b-feddc6b25d4e@oracle.com> Message-ID: On 18/05/2017 6:08 PM, Roman Kennke wrote: > Am 18.05.2017 um 10:03 schrieb David Holmes: >> On 18/05/2017 5:54 PM, Roman Kennke wrote: >>> Am 18.05.2017 um 04:07 schrieb David Holmes: >>>> Roman, >>>> >>>> I just pushed this but now I think there is a problem with this >>>> optimization. >>>> >>>> When a thread terminates, omFlush is used to transfer the thread's >>>> inUseList to the global inUseList. This happens after the thread has >>>> been removed from the _threads_list and can run concurrently with a >>>> safepoint. This means that there is a window of time where the >>>> monitors in the thread's inUseList will not be seen by the safepoint >>>> oops_do processing: >>>> - the thread oops_do won't be called because the thread is not in the >>>> thread's list >>>> - the monitors are not yet in the global in-use-list >>>> >>>> Did I overlook something? >>> >>> Ugh. How ugly :-) >>> >>> Is there a reason why omFlush cannot be called from the same block where >>> the thread is removed from _thread_list, under the Threads_lock ? >> >> I think the explicit intent is to let it run independent of safepoints >> (which holding the Threads_lock would affect). > > But Thread::remove() *is* going to block on Threads_lock. If I read the > code correctly, omFlush() is called right after we give up Threads_lock Right - in the spirit of "critical sections should be as short as possible" the omFlush is done outside of the holding of the threads_lock. > (thread_main_inner()). Stuff under gListLock in omFlush() is only doing > some simple re-linking of the linked lists. I'll give that a try with > some very careful checking ... It's the other uses of the gListlock that need to be examined. David > Roman > From robbin.ehn at oracle.com Thu May 18 09:00:31 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Thu, 18 May 2017 11:00:31 +0200 Subject: RFR (jdk10): JDK-8180175: ObjectSynchronizer only needs to iterate in-use monitors In-Reply-To: References: <89399a72-0263-5ebd-5fda-82f404fbe2ca@redhat.com> <8d75eef3-c143-dd75-6362-6203c7fb175f@redhat.com> <3a0f4490-646c-8004-6272-180ef198b67e@oracle.com> <13484ee1-5c37-6971-028d-2ddf0b0f2845@redhat.com> <4e5f18fb-972b-82ad-b84b-feddc6b25d4e@oracle.com> Message-ID: <0497b570-32ba-ef04-bfc0-32c4d6817f07@oracle.com> Hi, On 05/18/2017 10:10 AM, David Holmes wrote: > On 18/05/2017 6:08 PM, Roman Kennke wrote: >> Am 18.05.2017 um 10:03 schrieb David Holmes: >>> On 18/05/2017 5:54 PM, Roman Kennke wrote: >>>> Am 18.05.2017 um 04:07 schrieb David Holmes: >>>>> Roman, >>>>> >>>>> I just pushed this but now I think there is a problem with this >>>>> optimization. >>>>> >>>>> When a thread terminates, omFlush is used to transfer the thread's >>>>> inUseList to the global inUseList. This happens after the thread has >>>>> been removed from the _threads_list and can run concurrently with a >>>>> safepoint. This means that there is a window of time where the >>>>> monitors in the thread's inUseList will not be seen by the safepoint >>>>> oops_do processing: >>>>> - the thread oops_do won't be called because the thread is not in the >>>>> thread's list >>>>> - the monitors are not yet in the global in-use-list >>>>> >>>>> Did I overlook something? Good catch! >>>> >>>> Ugh. How ugly :-) >>>> >>>> Is there a reason why omFlush cannot be called from the same block where >>>> the thread is removed from _thread_list, under the Threads_lock ? Couldn't omFlush be done before Threads_lock in Thread::remove, while the thread is still on threads list? I do not see the reason why you want to hold the Threads_lock? To me it looks like gListLock is enough? /Robbin >>> >>> I think the explicit intent is to let it run independent of safepoints >>> (which holding the Threads_lock would affect). >> >> But Thread::remove() *is* going to block on Threads_lock. If I read the >> code correctly, omFlush() is called right after we give up Threads_lock > > Right - in the spirit of "critical sections should be as short as possible" the omFlush is done outside of the holding of the threads_lock. > >> (thread_main_inner()). Stuff under gListLock in omFlush() is only doing >> some simple re-linking of the linked lists. I'll give that a try with >> some very careful checking ... > > It's the other uses of the gListlock that need to be examined. > > David > >> Roman >> From david.holmes at oracle.com Thu May 18 09:26:19 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 18 May 2017 19:26:19 +1000 Subject: RFR (jdk10): JDK-8180175: ObjectSynchronizer only needs to iterate in-use monitors In-Reply-To: <0497b570-32ba-ef04-bfc0-32c4d6817f07@oracle.com> References: <89399a72-0263-5ebd-5fda-82f404fbe2ca@redhat.com> <8d75eef3-c143-dd75-6362-6203c7fb175f@redhat.com> <3a0f4490-646c-8004-6272-180ef198b67e@oracle.com> <13484ee1-5c37-6971-028d-2ddf0b0f2845@redhat.com> <4e5f18fb-972b-82ad-b84b-feddc6b25d4e@oracle.com> <0497b570-32ba-ef04-bfc0-32c4d6817f07@oracle.com> Message-ID: On 18/05/2017 7:00 PM, Robbin Ehn wrote: > Hi, > > On 05/18/2017 10:10 AM, David Holmes wrote: >> On 18/05/2017 6:08 PM, Roman Kennke wrote: >>> Am 18.05.2017 um 10:03 schrieb David Holmes: >>>> On 18/05/2017 5:54 PM, Roman Kennke wrote: >>>>> Am 18.05.2017 um 04:07 schrieb David Holmes: >>>>>> Roman, >>>>>> >>>>>> I just pushed this but now I think there is a problem with this >>>>>> optimization. >>>>>> >>>>>> When a thread terminates, omFlush is used to transfer the thread's >>>>>> inUseList to the global inUseList. This happens after the thread has >>>>>> been removed from the _threads_list and can run concurrently with a >>>>>> safepoint. This means that there is a window of time where the >>>>>> monitors in the thread's inUseList will not be seen by the safepoint >>>>>> oops_do processing: >>>>>> - the thread oops_do won't be called because the thread is not in the >>>>>> thread's list >>>>>> - the monitors are not yet in the global in-use-list >>>>>> >>>>>> Did I overlook something? > > Good catch! > >>>>> >>>>> Ugh. How ugly :-) >>>>> >>>>> Is there a reason why omFlush cannot be called from the same block >>>>> where >>>>> the thread is removed from _thread_list, under the Threads_lock ? > > Couldn't omFlush be done before Threads_lock in Thread::remove, while > the thread is still on threads list? > I do not see the reason why you want to hold the Threads_lock? To me it > looks like gListLock is enough? Not sure it makes any practical difference. If you do it before then the thread is still part of safepoint protocol, so no safepoint can occur while calling omFlush. If you do it while holding Threads_lock then no safepoint can occur while doing omFlush. David > /Robbin > >>>> >>>> I think the explicit intent is to let it run independent of safepoints >>>> (which holding the Threads_lock would affect). >>> >>> But Thread::remove() *is* going to block on Threads_lock. If I read the >>> code correctly, omFlush() is called right after we give up Threads_lock >> >> Right - in the spirit of "critical sections should be as short as >> possible" the omFlush is done outside of the holding of the threads_lock. >> >>> (thread_main_inner()). Stuff under gListLock in omFlush() is only doing >>> some simple re-linking of the linked lists. I'll give that a try with >>> some very careful checking ... >> >> It's the other uses of the gListlock that need to be examined. >> >> David >> >>> Roman >>> From robbin.ehn at oracle.com Thu May 18 09:33:38 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Thu, 18 May 2017 11:33:38 +0200 Subject: RFR (jdk10): JDK-8180175: ObjectSynchronizer only needs to iterate in-use monitors In-Reply-To: References: <89399a72-0263-5ebd-5fda-82f404fbe2ca@redhat.com> <8d75eef3-c143-dd75-6362-6203c7fb175f@redhat.com> <3a0f4490-646c-8004-6272-180ef198b67e@oracle.com> <13484ee1-5c37-6971-028d-2ddf0b0f2845@redhat.com> <4e5f18fb-972b-82ad-b84b-feddc6b25d4e@oracle.com> <0497b570-32ba-ef04-bfc0-32c4d6817f07@oracle.com> Message-ID: <6d67ac4c-51ef-b5d9-09e8-dfa24e63e792@oracle.com> > > Not sure it makes any practical difference. If you do it before then the thread is still part of safepoint protocol, so no safepoint can occur while calling omFlush. If you > do it while holding Threads_lock then no safepoint can occur while doing omFlush. New people to the code would assume there is a point doing it under the lock. I think the code also would a bit nicer doing it outside and as you said: "critical sections should be as short as possible" Since Threads_lock is used e.g. every-time we loop the threads list, so that point is still totally valid. /Robbin > > David > From rkennke at redhat.com Thu May 18 09:57:53 2017 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 18 May 2017 11:57:53 +0200 Subject: RFR (jdk10): JDK-8180175: ObjectSynchronizer only needs to iterate in-use monitors In-Reply-To: <0497b570-32ba-ef04-bfc0-32c4d6817f07@oracle.com> References: <89399a72-0263-5ebd-5fda-82f404fbe2ca@redhat.com> <8d75eef3-c143-dd75-6362-6203c7fb175f@redhat.com> <3a0f4490-646c-8004-6272-180ef198b67e@oracle.com> <13484ee1-5c37-6971-028d-2ddf0b0f2845@redhat.com> <4e5f18fb-972b-82ad-b84b-feddc6b25d4e@oracle.com> <0497b570-32ba-ef04-bfc0-32c4d6817f07@oracle.com> Message-ID: <1474021a-0eae-09c0-cbc2-b41e0503c624@redhat.com> Am 18.05.2017 um 11:00 schrieb Robbin Ehn: > Hi, > > On 05/18/2017 10:10 AM, David Holmes wrote: >> On 18/05/2017 6:08 PM, Roman Kennke wrote: >>> Am 18.05.2017 um 10:03 schrieb David Holmes: >>>> On 18/05/2017 5:54 PM, Roman Kennke wrote: >>>>> Am 18.05.2017 um 04:07 schrieb David Holmes: >>>>>> Roman, >>>>>> >>>>>> I just pushed this but now I think there is a problem with this >>>>>> optimization. >>>>>> >>>>>> When a thread terminates, omFlush is used to transfer the thread's >>>>>> inUseList to the global inUseList. This happens after the thread has >>>>>> been removed from the _threads_list and can run concurrently with a >>>>>> safepoint. This means that there is a window of time where the >>>>>> monitors in the thread's inUseList will not be seen by the safepoint >>>>>> oops_do processing: >>>>>> - the thread oops_do won't be called because the thread is not in >>>>>> the >>>>>> thread's list >>>>>> - the monitors are not yet in the global in-use-list >>>>>> >>>>>> Did I overlook something? > > Good catch! > >>>>> >>>>> Ugh. How ugly :-) >>>>> >>>>> Is there a reason why omFlush cannot be called from the same block >>>>> where >>>>> the thread is removed from _thread_list, under the Threads_lock ? > > Couldn't omFlush be done before Threads_lock in Thread::remove, while > the thread is still on threads list? > I do not see the reason why you want to hold the Threads_lock? To me > it looks like gListLock is enough? I was thinking the same. Currently testing this. I suppose a new bug needs to be filed for this? Roman From david.holmes at oracle.com Thu May 18 10:52:36 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 18 May 2017 20:52:36 +1000 Subject: RFR (jdk10): JDK-8180175: ObjectSynchronizer only needs to iterate in-use monitors In-Reply-To: <1474021a-0eae-09c0-cbc2-b41e0503c624@redhat.com> References: <89399a72-0263-5ebd-5fda-82f404fbe2ca@redhat.com> <8d75eef3-c143-dd75-6362-6203c7fb175f@redhat.com> <3a0f4490-646c-8004-6272-180ef198b67e@oracle.com> <13484ee1-5c37-6971-028d-2ddf0b0f2845@redhat.com> <4e5f18fb-972b-82ad-b84b-feddc6b25d4e@oracle.com> <0497b570-32ba-ef04-bfc0-32c4d6817f07@oracle.com> <1474021a-0eae-09c0-cbc2-b41e0503c624@redhat.com> Message-ID: Hi Roman, Yes a new bug must be filed for this. Sorry I didn't catch it before the push. Thanks, David On 18/05/2017 7:57 PM, Roman Kennke wrote: > Am 18.05.2017 um 11:00 schrieb Robbin Ehn: >> Hi, >> >> On 05/18/2017 10:10 AM, David Holmes wrote: >>> On 18/05/2017 6:08 PM, Roman Kennke wrote: >>>> Am 18.05.2017 um 10:03 schrieb David Holmes: >>>>> On 18/05/2017 5:54 PM, Roman Kennke wrote: >>>>>> Am 18.05.2017 um 04:07 schrieb David Holmes: >>>>>>> Roman, >>>>>>> >>>>>>> I just pushed this but now I think there is a problem with this >>>>>>> optimization. >>>>>>> >>>>>>> When a thread terminates, omFlush is used to transfer the thread's >>>>>>> inUseList to the global inUseList. This happens after the thread has >>>>>>> been removed from the _threads_list and can run concurrently with a >>>>>>> safepoint. This means that there is a window of time where the >>>>>>> monitors in the thread's inUseList will not be seen by the safepoint >>>>>>> oops_do processing: >>>>>>> - the thread oops_do won't be called because the thread is not in >>>>>>> the >>>>>>> thread's list >>>>>>> - the monitors are not yet in the global in-use-list >>>>>>> >>>>>>> Did I overlook something? >> >> Good catch! >> >>>>>> >>>>>> Ugh. How ugly :-) >>>>>> >>>>>> Is there a reason why omFlush cannot be called from the same block >>>>>> where >>>>>> the thread is removed from _thread_list, under the Threads_lock ? >> >> Couldn't omFlush be done before Threads_lock in Thread::remove, while >> the thread is still on threads list? >> I do not see the reason why you want to hold the Threads_lock? To me >> it looks like gListLock is enough? > > I was thinking the same. Currently testing this. > > I suppose a new bug needs to be filed for this? > > Roman > From mikael.gerdin at oracle.com Thu May 18 11:55:02 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Thu, 18 May 2017 13:55:02 +0200 Subject: =?UTF-8?Q?Re:_=e7=ad=94=e5=a4=8d:_HotSpot_and_IBM's_J9_behave_quite?= =?UTF-8?Q?_differently_when_processing_monitorenters_and_monitorexits?= In-Reply-To: <24a31d5c-8185-e55d-481f-719f3949bf7d@oracle.com> References: <005101d2cf53$c8c16b40$5a4441c0$@cs.sjtu.edu.cn> <656dcb18-e65a-4c48-a6bc-cf00fe021af5@oracle.com> <005201d2cf5a$aa2e8f30$fe8bad90$@cs.sjtu.edu.cn> <5a6a2de2-eb43-985d-bfc7-18c3fa7ab0ee@oracle.com> <24a31d5c-8185-e55d-481f-719f3949bf7d@oracle.com> Message-ID: <77c31484-6c76-4b36-671b-71061c98f1ce@oracle.com> On 2017-05-18 07:25, David Holmes wrote: > Hi Yuting, > > On 18/05/2017 2:49 PM, chenyt wrote: >> Hi, David, >> >> I tested the next program (let one object to be monitored be null) using >> J9 and HotSpot. HotSpot does not throw a NullPointerException, as the >> specification says. J9 looks fine (throw a NullPointerException). Am I >> wrong here? > > I can't readily test this as I don't have Jimple nor quick and easy > access to bytecode assemblers. It is strange though as the interpreter > code contains this: > > CASE(_monitorenter): { > oop lockee = STACK_OBJECT(-1); > // derefing's lockee ought to provoke implicit null check > CHECK_NULL(lockee); In the x86 templateTable we do void TemplateTable::monitorenter() { transition(atos, vtos); // check for NULL object __ null_check(rax); but looking in macroAssembler_x86.cpp null_check only does a cmp (setting the eflags) but there's no condtional branch on the Z flag? It feels like I'm crazy but several of these __ null_check() calls seem broken to me. /Mikael > > If I can find some time I will try to test this myself. > >> It is very interesting that I have found so many unexpected behaviors >> here. > > Well you only found two and they are related. :) Manually assembled > monitor code is not something very many (any?) people care about or do - > other than emulating correct language usage. > > Cheers, > David > >> public void main(java.lang.String[]) throws java.lang.Exception; >> descriptor: ([Ljava/lang/String;)V >> flags: ACC_PUBLIC >> Code: >> stack=1, locals=2, args_size=2 >> 0: aload_0 >> 1: monitorenter >> 2: aconst_null >> 3: monitorenter >> 4: aconst_null >> 5: monitorexit >> 6: aload_0 >> 7: monitorexit >> 8: return >> Exceptions: >> throws java.lang.Exception >> >> Jimple code is given as follows: >> public void main(java.lang.String[]) throws java.lang.Exception >> { >> Search r0; >> Search r2; >> java.lang.String[] r1; >> >> r0 := @this: Search; >> r1 := @parameter0: java.lang.String[]; >> r2 = null; >> entermonitor r0; >> entermonitor r2; >> exitmonitor r2; >> exitmonitor r0; >> >> return; >> } >> >> Wishes, >> Yuting >> >> >> On Thu, 18 May 2017 13:23:09 +1000, David Holmes wrote: >>> One correction ... >>> >>> On 18/05/2017 10:29 AM, David Holmes wrote: >>>> On 18/05/2017 8:12 AM, ??? wrote: >>>>> Thank you, David. I have seen from the specification that structured >>>>> locking >>>>> enforcement is optional. The second and the third ones are cases of >>>>> structured/nested lockings. Will non-nested locking sequences raise >>>>> deadlocks? Of course it is a different topic, while it might be better >>>>> if it >>>>> can be kicked out earlier from the specification/JVM. >>>> >>>> Non-nested locking doesn't necessarily lead to deadlocks - you just >>>> need >>>> a different locking order for that (even if properly nested). There are >>>> locking patterns that rely on the ability to lock and unlock in >>>> different order ie chained-locking for walking linked-lists A->B->C->D: >>>> - lock A, lock B, unlock A, lock C, unlock B, lock D, unlock C ... >>>> >>>> The VM spec allows a little flexibility in how the monitor bytecodes >>>> can >>>> be used compared to the Java programming language. That's not something >>>> that will change. >>>> >>>>> The first example is still a problem. It seems that HotSpot allows to >>>>> monitor a pure object reference without initialized (Is it true? How >>>>> can >>>>> this checking be omitted?). J9 reports a verifyerror as follows. >>>>> >>>>> Exception in thread "main" java.lang.VerifyError: JVMVRFY012 stack >>>>> shape >>>>> inconsistent; class=Search, method=main([Ljava/lang/String;)V, pc=6 >>>>> Exception Details: >>>>> Location: >>>>> Search.main([Ljava/lang/String;)V @6: JBmonitorenter >>>>> Reason: >>>>> Type 'uninitialized' (current frame, stack[1]) is not >>>>> assignable to >>>>> 'java/lang/Object' >>>>> Current Frame: >>>>> bci: @6 >>>>> flags: { } >>>>> locals: { 'Search', '[Ljava/lang/String;' } >>>>> stack: { 'uninitialized', 'uninitialized' } >>>>> at T.main(T.java:4) >>>> >>>> Yes I think this may be a bug in hotspot. The type-checking for the >>>> monitor bytecodes requires a matching type of reference on the operand >>>> stack - but "uninitialized" does not match Object, as J9 reports. But >>>> I'm not an expert on this aspect of verification so I may not be >>>> interpreting it correctly. >>>> >>>> More below ... >>>> >>>>> -----????----- >>>>> ???: David Holmes [mailto:david.holmes at oracle.com] >>>>> ????: 2017?5?17? 14:41 >>>>> ???: ??? ; >>>>> hotspot-runtime-dev at openjdk.java.net >>>>> ??: Re: HotSpot and IBM's J9 behave quite differently when >>>>> processing >>>>> monitorenters and monitorexits >>>>> >>>>> Hi, >>>>> On 18/05/2017 7:23 AM, ??? wrote: >>>>>> Am I wrong? >>>>> >>>>> I will look at each situation in detail when I get a chance but >>>>> structured >>>>> locking enforcement is optional. Also balancing the number of locks >>>>> and >>>>> unlocks in a frame does not mean they can't be locked and unlocked >>>>> in a >>>>> non-nested fashion - just that by the end the number of unlocks >>>>> matches the >>>>> number of locks. >>>>> >>>>> BTW the way you respond to these emails, as if having a conversation >>>>> with >>>>> yourself, makes it difficult to respond as we can't readily see what >>>>> is the >>>>> new email and what is the original. >>>>> >>>>> Cheers, >>>>> David >>>>> >>>>>> The byte code for main() in case 1 is as follows. The strange >>>>>> thing is >>>>>> that NullPointerException is also not thrown at runtime. >>>> >>>> That is strange as it does for the normal obvious case of using >>>> synchronized(o) when o is null. >>> >>> Ah - it isn't null it just an object for which the constructor has >>> not been run. The runtime can't tell the difference between a pointer >>> to a valid initialized object, and a pointer to an uninitialized chunk >>> of memory. >>> >>>>>> >>>>>> public void main(java.lang.String[]) throws java.lang.Exception; >>>>>> descriptor: ([Ljava/lang/String;)V >>>>>> flags: ACC_PUBLIC >>>>>> Code: >>>>>> stack=3, locals=2, args_size=2 >>>>>> 0: new #2 // class Search >>> >>> This allocated an object - hence no null reference. But this is what >>> verification should have complained about. >>> >>> David >>> ----- >>> >>>>>> 3: dup >>>>>> 4: aload_0 >>>>>> 5: monitorenter >>>>>> 6: monitorenter >>>>>> 7: monitorexit >>>>>> 8: aload_0 >>>>>> 9: monitorexit >>>>>> 10: return >>>>>> Exceptions: >>>>>> throws java.lang.Exception >>>>>> >>>>>> ??: HotSpot and IBM's J9 behave quite differently when processing >>>>>> monitorenters and monitorexits >>>>>> >>>>>> I have tested several programs (in Jimple) and found that HotSpot and >>>>>> J9 match monitorenters and monitorexits quite differently. Verifiers >>>>>> should play more important roles here. >>>> >>>> The job of the verifier is to establish some basic guarantees for the >>>> JVM to then operate under. The verifier plays no role in checking how >>>> monitorenter/exit are used in combination, only that each individual >>>> bytecode meets some basic type constraints. >>>> >>>>>> >>>>>> (1) Test the next program (r2 is not initizlied) on HotSpot and J9. >>>>>> J9 throw out a verifier error, while HotSpot does not. It seems that >>>>>> HotSpot's verifier forgets to check whether a monitored object is >>>>>> initialized. >>>>>> >>>>>> public class Search extends java.lang.Object { public void () >>>>>> { >>>>>> Search r0; >>>>>> r0 := @this: Search; >>>>>> specialinvoke r0.()>(); >>>>>> return; >>>>>> } >>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>> { >>>>>> Search r0; >>>>>> Search r2; >>>>>> java.lang.String[] r1; >>>>>> r0 := @this: Search; >>>>>> r1 := @parameter0: java.lang.String[]; >>>>>> r2 = new Search; >>>>>> >>>>>> entermonitor r2; >>>>>> entermonitor r0; >>>>>> exitmonitor r2; >>>>>> exitmonitor r0; >>>>>> return; >>>>>> } >>>>>> } >>>> >>>> Verification was covered above. >>>> >>>>>> >>>>>> (2) Test the next program on HotSpot and J9, and both do not report >>>>>> any errors. However, I guess the order in the program (entermonitor >>>>>> r2; => entermonitor r0; => exitmonitor r2; => exitmonitor r0;) >>>>>> violates the situation of "structured locking" (Structured locking is >>>>>> the situation when, during a method invocation, every exit on a given >>>>>> monitor matches a preceding entry on that monitor, see the >>>>>> specification >>>>>> >>>>>> https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-2.html#jvms-2. >>>>>> >>>>>> 11.10) >>>>>> ? >>>> >>>> No it doesn't violate structured locking as the number of enters and >>>> exits match, and there is always an enter before an exit. >>>> >>>>>> Actually, the words (every exit on a given monitor matches a >>>>>> preceding >>>>>> entry on that monitor) are not quite clear as for me. Otherwise the >>>>>> first rule (The number of monitor entries performed by T on M >>>>>> during a >>>>>> method invocation must equal the number of monitor exits performed by >>>>>> T on M during the method invocation whether the method invocation >>>>>> completes normally or abruptly.) is sufficient. >>>> >>>> The number of enters and exits must not only match/balance, but there >>>> must be an enter before a corresponding exit. >>>> >>>>>> >>>>>> public class Search extends java.lang.Object { >>>>>> >>>>>> public void () >>>>>> { >>>>>> Search r0; >>>>>> r0 := @this: Search; >>>>>> specialinvoke r0.()>(); >>>>>> return; >>>>>> } >>>>>> >>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>> { >>>>>> Search r0; >>>>>> Search r2; >>>>>> java.lang.String[] r1; >>>>>> r0 := @this: Search; >>>>>> r1 := @parameter0: java.lang.String[]; >>>>>> r2 = new Search; >>>>>> specialinvoke r2.()>(); >>>>>> entermonitor r2; >>>>>> entermonitor r0; >>>>>> exitmonitor r2; >>>>>> exitmonitor r0; >>>>>> return; >>>>>> } >>>>>> } >>>>>> >>>>>> (3) The next program enters monitor in and exits it in main(). >>>>>> HotSpot throws a runtime exception, while J9 does not. Should this >>>>>> program be rejected by the verifiers? >>>> >>>> No this does not violate any verification rules. The runtime behaviour >>>> depends on whether structured locking is enforced or not.(Even in >>>> hotspot there can be differences between interpreted and jitted code). >>>> >>>> Hope that helps clarify things. >>>> >>>> David >>>> ----- >>>> >>>>>> >>>>>> public class Search extends java.lang.Object { >>>>>> >>>>>> public void () >>>>>> { >>>>>> Search r0; >>>>>> r0 := @this: Search; >>>>>> specialinvoke r0.()>(); >>>>>> entermonitor r0; >>>>>> return; >>>>>> } >>>>>> >>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>> { >>>>>> Search r0; >>>>>> Search r2; >>>>>> java.lang.S From rkennke at redhat.com Thu May 18 12:16:45 2017 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 18 May 2017 14:16:45 +0200 Subject: RFR: JDK-8180599: Possibly miss to iterate monitors on thread exit Message-ID: As David pointed out here: http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2017-May/023468.html The fix to JDK-8180175: ObjectSynchronizer only needs to iterate in-use monitors introduced a bug where we could potentially miss to iterate oops in monitors of exiting threads. The proposed fix is as Robbin Ehn suggested to move the call to omFlush() before we remove the thread from _thread_list: http://cr.openjdk.java.net/~rkennke/8180599/webrev.00/ This solves the problem because there's no window where monitors would not appear in any list. I tested a number of programs and jtreg hotspot_gc tests and it doesn't show any ill effects. Ok? Roman From chenyt at cs.sjtu.edu.cn Thu May 18 14:22:37 2017 From: chenyt at cs.sjtu.edu.cn (=?UTF-8?B?6ZmI6Zuo5Lqt?=) Date: Thu, 18 May 2017 07:22:37 -0700 Subject: =?UTF-8?Q?=E7=AD=94=E5=A4=8D:_=E7=AD=94=E5=A4=8D:_HotSpot_and_IBM's_J9_beh?= =?UTF-8?Q?ave_quite_differently_when_proce?= =?UTF-8?Q?ssing_monitorenters_and_monitore?= =?UTF-8?Q?xits?= In-Reply-To: <77c31484-6c76-4b36-671b-71061c98f1ce@oracle.com> References: <005101d2cf53$c8c16b40$5a4441c0$@cs.sjtu.edu.cn> <656dcb18-e65a-4c48-a6bc-cf00fe021af5@oracle.com> <005201d2cf5a$aa2e8f30$fe8bad90$@cs.sjtu.edu.cn> <5a6a2de2-eb43-985d-bfc7-18c3fa7ab0ee@oracle.com> <24a31d5c-8185-e55d-481f-719f3949bf7d@oracle.com> <77c31484-6c76-4b36-671b-71061c98f1ce@oracle.com> Message-ID: <000e01d2cfe2$374939b0$a5dbad10$@cs.sjtu.edu.cn> HotSpot throws a NullPointerException correctly when an initialized object as set as null, as follows. public void main(java.lang.String[]) throws java.lang.Exception; descriptor: ([Ljava/lang/String;)V flags: ACC_PUBLIC Code: stack=1, locals=2, args_size=2 0: new #2 // class Search 3: invokespecial #14 // Method "":()V 6: aconst_null 7: monitorenter 8: aconst_null 9: monitorexit 10: return Exceptions: throws java.lang.Exception -----????----- ???: Mikael Gerdin [mailto:mikael.gerdin at oracle.com] ????: 2017?5?18? 4:55 ???: David Holmes ; chenyt ??: hotspot-runtime-dev at openjdk.java.net ??: Re: ??: HotSpot and IBM's J9 behave quite differently when processing monitorenters and monitorexits On 2017-05-18 07:25, David Holmes wrote: > Hi Yuting, > > On 18/05/2017 2:49 PM, chenyt wrote: >> Hi, David, >> >> I tested the next program (let one object to be monitored be null) >> using >> J9 and HotSpot. HotSpot does not throw a NullPointerException, as the >> specification says. J9 looks fine (throw a NullPointerException). Am >> I wrong here? > > I can't readily test this as I don't have Jimple nor quick and easy > access to bytecode assemblers. It is strange though as the interpreter > code contains this: > > CASE(_monitorenter): { > oop lockee = STACK_OBJECT(-1); > // derefing's lockee ought to provoke implicit null check > CHECK_NULL(lockee); In the x86 templateTable we do void TemplateTable::monitorenter() { transition(atos, vtos); // check for NULL object __ null_check(rax); but looking in macroAssembler_x86.cpp null_check only does a cmp (setting the eflags) but there's no condtional branch on the Z flag? It feels like I'm crazy but several of these __ null_check() calls seem broken to me. /Mikael > > If I can find some time I will try to test this myself. > >> It is very interesting that I have found so many unexpected behaviors >> here. > > Well you only found two and they are related. :) Manually assembled > monitor code is not something very many (any?) people care about or do > - other than emulating correct language usage. > > Cheers, > David > >> public void main(java.lang.String[]) throws java.lang.Exception; >> descriptor: ([Ljava/lang/String;)V >> flags: ACC_PUBLIC >> Code: >> stack=1, locals=2, args_size=2 >> 0: aload_0 >> 1: monitorenter >> 2: aconst_null >> 3: monitorenter >> 4: aconst_null >> 5: monitorexit >> 6: aload_0 >> 7: monitorexit >> 8: return >> Exceptions: >> throws java.lang.Exception >> >> Jimple code is given as follows: >> public void main(java.lang.String[]) throws java.lang.Exception >> { >> Search r0; >> Search r2; >> java.lang.String[] r1; >> >> r0 := @this: Search; >> r1 := @parameter0: java.lang.String[]; >> r2 = null; >> entermonitor r0; >> entermonitor r2; >> exitmonitor r2; >> exitmonitor r0; >> >> return; >> } >> >> Wishes, >> Yuting >> >> >> On Thu, 18 May 2017 13:23:09 +1000, David Holmes wrote: >>> One correction ... >>> >>> On 18/05/2017 10:29 AM, David Holmes wrote: >>>> On 18/05/2017 8:12 AM, ??? wrote: >>>>> Thank you, David. I have seen from the specification that >>>>> structured locking enforcement is optional. The second and the >>>>> third ones are cases of structured/nested lockings. Will >>>>> non-nested locking sequences raise deadlocks? Of course it is a >>>>> different topic, while it might be better if it can be kicked out >>>>> earlier from the specification/JVM. >>>> >>>> Non-nested locking doesn't necessarily lead to deadlocks - you just >>>> need a different locking order for that (even if properly nested). >>>> There are locking patterns that rely on the ability to lock and >>>> unlock in different order ie chained-locking for walking >>>> linked-lists A->B->C->D: >>>> - lock A, lock B, unlock A, lock C, unlock B, lock D, unlock C ... >>>> >>>> The VM spec allows a little flexibility in how the monitor >>>> bytecodes can be used compared to the Java programming language. >>>> That's not something that will change. >>>> >>>>> The first example is still a problem. It seems that HotSpot allows >>>>> to monitor a pure object reference without initialized (Is it >>>>> true? How can this checking be omitted?). J9 reports a verifyerror >>>>> as follows. >>>>> >>>>> Exception in thread "main" java.lang.VerifyError: JVMVRFY012 stack >>>>> shape inconsistent; class=Search, >>>>> method=main([Ljava/lang/String;)V, pc=6 Exception Details: >>>>> Location: >>>>> Search.main([Ljava/lang/String;)V @6: JBmonitorenter >>>>> Reason: >>>>> Type 'uninitialized' (current frame, stack[1]) is not >>>>> assignable to 'java/lang/Object' >>>>> Current Frame: >>>>> bci: @6 >>>>> flags: { } >>>>> locals: { 'Search', '[Ljava/lang/String;' } >>>>> stack: { 'uninitialized', 'uninitialized' } >>>>> at T.main(T.java:4) >>>> >>>> Yes I think this may be a bug in hotspot. The type-checking for the >>>> monitor bytecodes requires a matching type of reference on the >>>> operand stack - but "uninitialized" does not match Object, as J9 >>>> reports. But I'm not an expert on this aspect of verification so I >>>> may not be interpreting it correctly. >>>> >>>> More below ... >>>> >>>>> -----????----- >>>>> ???: David Holmes [mailto:david.holmes at oracle.com] >>>>> ????: 2017?5?17? 14:41 >>>>> ???: ??? ; >>>>> hotspot-runtime-dev at openjdk.java.net >>>>> ??: Re: HotSpot and IBM's J9 behave quite differently when >>>>> processing monitorenters and monitorexits >>>>> >>>>> Hi, >>>>> On 18/05/2017 7:23 AM, ??? wrote: >>>>>> Am I wrong? >>>>> >>>>> I will look at each situation in detail when I get a chance but >>>>> structured locking enforcement is optional. Also balancing the >>>>> number of locks and unlocks in a frame does not mean they can't be >>>>> locked and unlocked in a non-nested fashion - just that by the end >>>>> the number of unlocks matches the number of locks. >>>>> >>>>> BTW the way you respond to these emails, as if having a >>>>> conversation with yourself, makes it difficult to respond as we >>>>> can't readily see what is the new email and what is the original. >>>>> >>>>> Cheers, >>>>> David >>>>> >>>>>> The byte code for main() in case 1 is as follows. The strange >>>>>> thing is that NullPointerException is also not thrown at runtime. >>>> >>>> That is strange as it does for the normal obvious case of using >>>> synchronized(o) when o is null. >>> >>> Ah - it isn't null it just an object for which the constructor has >>> not been run. The runtime can't tell the difference between a >>> pointer to a valid initialized object, and a pointer to an >>> uninitialized chunk of memory. >>> >>>>>> >>>>>> public void main(java.lang.String[]) throws java.lang.Exception; >>>>>> descriptor: ([Ljava/lang/String;)V >>>>>> flags: ACC_PUBLIC >>>>>> Code: >>>>>> stack=3, locals=2, args_size=2 >>>>>> 0: new #2 // class Search >>> >>> This allocated an object - hence no null reference. But this is what >>> verification should have complained about. >>> >>> David >>> ----- >>> >>>>>> 3: dup >>>>>> 4: aload_0 >>>>>> 5: monitorenter >>>>>> 6: monitorenter >>>>>> 7: monitorexit >>>>>> 8: aload_0 >>>>>> 9: monitorexit >>>>>> 10: return >>>>>> Exceptions: >>>>>> throws java.lang.Exception >>>>>> >>>>>> ??: HotSpot and IBM's J9 behave quite differently when processing >>>>>> monitorenters and monitorexits >>>>>> >>>>>> I have tested several programs (in Jimple) and found that HotSpot >>>>>> and >>>>>> J9 match monitorenters and monitorexits quite differently. >>>>>> Verifiers should play more important roles here. >>>> >>>> The job of the verifier is to establish some basic guarantees for >>>> the JVM to then operate under. The verifier plays no role in >>>> checking how monitorenter/exit are used in combination, only that >>>> each individual bytecode meets some basic type constraints. >>>> >>>>>> >>>>>> (1) Test the next program (r2 is not initizlied) on HotSpot and J9. >>>>>> J9 throw out a verifier error, while HotSpot does not. It seems >>>>>> that HotSpot's verifier forgets to check whether a monitored >>>>>> object is initialized. >>>>>> >>>>>> public class Search extends java.lang.Object { public void () >>>>>> { >>>>>> Search r0; >>>>>> r0 := @this: Search; >>>>>> specialinvoke r0.()>(); >>>>>> return; >>>>>> } >>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>> { >>>>>> Search r0; >>>>>> Search r2; >>>>>> java.lang.String[] r1; >>>>>> r0 := @this: Search; >>>>>> r1 := @parameter0: java.lang.String[]; >>>>>> r2 = new Search; >>>>>> >>>>>> entermonitor r2; >>>>>> entermonitor r0; >>>>>> exitmonitor r2; >>>>>> exitmonitor r0; >>>>>> return; >>>>>> } >>>>>> } >>>> >>>> Verification was covered above. >>>> >>>>>> >>>>>> (2) Test the next program on HotSpot and J9, and both do not >>>>>> report any errors. However, I guess the order in the program >>>>>> (entermonitor r2; => entermonitor r0; => exitmonitor r2; => >>>>>> exitmonitor r0;) violates the situation of "structured locking" >>>>>> (Structured locking is the situation when, during a method >>>>>> invocation, every exit on a given monitor matches a preceding >>>>>> entry on that monitor, see the specification >>>>>> >>>>>> https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-2.html#jvms-2. >>>>>> >>>>>> 11.10) >>>>>> ? >>>> >>>> No it doesn't violate structured locking as the number of enters >>>> and exits match, and there is always an enter before an exit. >>>> >>>>>> Actually, the words (every exit on a given monitor matches a >>>>>> preceding entry on that monitor) are not quite clear as for me. >>>>>> Otherwise the first rule (The number of monitor entries performed >>>>>> by T on M during a method invocation must equal the number of >>>>>> monitor exits performed by T on M during the method invocation >>>>>> whether the method invocation completes normally or abruptly.) >>>>>> is sufficient. >>>> >>>> The number of enters and exits must not only match/balance, but >>>> there must be an enter before a corresponding exit. >>>> >>>>>> >>>>>> public class Search extends java.lang.Object { >>>>>> >>>>>> public void () >>>>>> { >>>>>> Search r0; >>>>>> r0 := @this: Search; >>>>>> specialinvoke r0.()>(); >>>>>> return; >>>>>> } >>>>>> >>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>> { >>>>>> Search r0; >>>>>> Search r2; >>>>>> java.lang.String[] r1; >>>>>> r0 := @this: Search; >>>>>> r1 := @parameter0: java.lang.String[]; >>>>>> r2 = new Search; >>>>>> specialinvoke r2.()>(); >>>>>> entermonitor r2; >>>>>> entermonitor r0; >>>>>> exitmonitor r2; >>>>>> exitmonitor r0; >>>>>> return; >>>>>> } >>>>>> } >>>>>> >>>>>> (3) The next program enters monitor in and exits it in main(). >>>>>> HotSpot throws a runtime exception, while J9 does not. Should >>>>>> this program be rejected by the verifiers? >>>> >>>> No this does not violate any verification rules. The runtime >>>> behaviour depends on whether structured locking is enforced or >>>> not.(Even in hotspot there can be differences between interpreted and jitted code). >>>> >>>> Hope that helps clarify things. >>>> >>>> David >>>> ----- >>>> >>>>>> >>>>>> public class Search extends java.lang.Object { >>>>>> >>>>>> public void () >>>>>> { >>>>>> Search r0; >>>>>> r0 := @this: Search; >>>>>> specialinvoke r0.()>(); >>>>>> entermonitor r0; >>>>>> return; >>>>>> } >>>>>> >>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>> { >>>>>> Search r0; >>>>>> Search r2; >>>>>> java.lang.S From mikael.gerdin at oracle.com Thu May 18 14:25:45 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Thu, 18 May 2017 16:25:45 +0200 Subject: =?UTF-8?B?UmU6IOetlOWkjTog562U5aSNOiBIb3RTcG90IGFuZCBJQk0ncyBKOSBi?= =?UTF-8?Q?ehave_quite_differently_when_processing_monitorenters_and_monitor?= =?UTF-8?Q?exits?= In-Reply-To: <000e01d2cfe2$374939b0$a5dbad10$@cs.sjtu.edu.cn> References: <005101d2cf53$c8c16b40$5a4441c0$@cs.sjtu.edu.cn> <656dcb18-e65a-4c48-a6bc-cf00fe021af5@oracle.com> <005201d2cf5a$aa2e8f30$fe8bad90$@cs.sjtu.edu.cn> <5a6a2de2-eb43-985d-bfc7-18c3fa7ab0ee@oracle.com> <24a31d5c-8185-e55d-481f-719f3949bf7d@oracle.com> <77c31484-6c76-4b36-671b-71061c98f1ce@oracle.com> <000e01d2cfe2$374939b0$a5dbad10$@cs.sjtu.edu.cn> Message-ID: Hi, On 2017-05-18 16:22, ??? wrote: > HotSpot throws a NullPointerException correctly when an initialized object as set as null, as follows. > > public void main(java.lang.String[]) throws java.lang.Exception; > descriptor: ([Ljava/lang/String;)V > flags: ACC_PUBLIC > Code: > stack=1, locals=2, args_size=2 > 0: new #2 // class Search > 3: invokespecial #14 // Method "":()V > 6: aconst_null > 7: monitorenter > 8: aconst_null > 9: monitorexit > 10: return > Exceptions: > throws java.lang.Exception > Right. I just realized I missed the fact that the code did end up dereferencing the object but the code which actually makes that happen is not particularly easy to follow. Sorry for the noise. /Mikael > -----????----- > ???: Mikael Gerdin [mailto:mikael.gerdin at oracle.com] > ????: 2017?5?18? 4:55 > ???: David Holmes ; chenyt > ??: hotspot-runtime-dev at openjdk.java.net > ??: Re: ??: HotSpot and IBM's J9 behave quite differently when processing monitorenters and monitorexits > > > > On 2017-05-18 07:25, David Holmes wrote: >> Hi Yuting, >> >> On 18/05/2017 2:49 PM, chenyt wrote: >>> Hi, David, >>> >>> I tested the next program (let one object to be monitored be null) >>> using >>> J9 and HotSpot. HotSpot does not throw a NullPointerException, as the >>> specification says. J9 looks fine (throw a NullPointerException). Am >>> I wrong here? >> >> I can't readily test this as I don't have Jimple nor quick and easy >> access to bytecode assemblers. It is strange though as the interpreter >> code contains this: >> >> CASE(_monitorenter): { >> oop lockee = STACK_OBJECT(-1); >> // derefing's lockee ought to provoke implicit null check >> CHECK_NULL(lockee); > > In the x86 templateTable we do > > void TemplateTable::monitorenter() { > transition(atos, vtos); > > // check for NULL object > __ null_check(rax); > > > but looking in macroAssembler_x86.cpp null_check only does a cmp (setting the eflags) but there's no condtional branch on the Z flag? > > It feels like I'm crazy but several of these __ null_check() calls seem broken to me. > > /Mikael > >> >> If I can find some time I will try to test this myself. >> >>> It is very interesting that I have found so many unexpected behaviors >>> here. >> >> Well you only found two and they are related. :) Manually assembled >> monitor code is not something very many (any?) people care about or do >> - other than emulating correct language usage. >> >> Cheers, >> David >> >>> public void main(java.lang.String[]) throws java.lang.Exception; >>> descriptor: ([Ljava/lang/String;)V >>> flags: ACC_PUBLIC >>> Code: >>> stack=1, locals=2, args_size=2 >>> 0: aload_0 >>> 1: monitorenter >>> 2: aconst_null >>> 3: monitorenter >>> 4: aconst_null >>> 5: monitorexit >>> 6: aload_0 >>> 7: monitorexit >>> 8: return >>> Exceptions: >>> throws java.lang.Exception >>> >>> Jimple code is given as follows: >>> public void main(java.lang.String[]) throws java.lang.Exception >>> { >>> Search r0; >>> Search r2; >>> java.lang.String[] r1; >>> >>> r0 := @this: Search; >>> r1 := @parameter0: java.lang.String[]; >>> r2 = null; >>> entermonitor r0; >>> entermonitor r2; >>> exitmonitor r2; >>> exitmonitor r0; >>> >>> return; >>> } >>> >>> Wishes, >>> Yuting >>> >>> >>> On Thu, 18 May 2017 13:23:09 +1000, David Holmes wrote: >>>> One correction ... >>>> >>>> On 18/05/2017 10:29 AM, David Holmes wrote: >>>>> On 18/05/2017 8:12 AM, ??? wrote: >>>>>> Thank you, David. I have seen from the specification that >>>>>> structured locking enforcement is optional. The second and the >>>>>> third ones are cases of structured/nested lockings. Will >>>>>> non-nested locking sequences raise deadlocks? Of course it is a >>>>>> different topic, while it might be better if it can be kicked out >>>>>> earlier from the specification/JVM. >>>>> >>>>> Non-nested locking doesn't necessarily lead to deadlocks - you just >>>>> need a different locking order for that (even if properly nested). >>>>> There are locking patterns that rely on the ability to lock and >>>>> unlock in different order ie chained-locking for walking >>>>> linked-lists A->B->C->D: >>>>> - lock A, lock B, unlock A, lock C, unlock B, lock D, unlock C ... >>>>> >>>>> The VM spec allows a little flexibility in how the monitor >>>>> bytecodes can be used compared to the Java programming language. >>>>> That's not something that will change. >>>>> >>>>>> The first example is still a problem. It seems that HotSpot allows >>>>>> to monitor a pure object reference without initialized (Is it >>>>>> true? How can this checking be omitted?). J9 reports a verifyerror >>>>>> as follows. >>>>>> >>>>>> Exception in thread "main" java.lang.VerifyError: JVMVRFY012 stack >>>>>> shape inconsistent; class=Search, >>>>>> method=main([Ljava/lang/String;)V, pc=6 Exception Details: >>>>>> Location: >>>>>> Search.main([Ljava/lang/String;)V @6: JBmonitorenter >>>>>> Reason: >>>>>> Type 'uninitialized' (current frame, stack[1]) is not >>>>>> assignable to 'java/lang/Object' >>>>>> Current Frame: >>>>>> bci: @6 >>>>>> flags: { } >>>>>> locals: { 'Search', '[Ljava/lang/String;' } >>>>>> stack: { 'uninitialized', 'uninitialized' } >>>>>> at T.main(T.java:4) >>>>> >>>>> Yes I think this may be a bug in hotspot. The type-checking for the >>>>> monitor bytecodes requires a matching type of reference on the >>>>> operand stack - but "uninitialized" does not match Object, as J9 >>>>> reports. But I'm not an expert on this aspect of verification so I >>>>> may not be interpreting it correctly. >>>>> >>>>> More below ... >>>>> >>>>>> -----????----- >>>>>> ???: David Holmes [mailto:david.holmes at oracle.com] >>>>>> ????: 2017?5?17? 14:41 >>>>>> ???: ??? ; >>>>>> hotspot-runtime-dev at openjdk.java.net >>>>>> ??: Re: HotSpot and IBM's J9 behave quite differently when >>>>>> processing monitorenters and monitorexits >>>>>> >>>>>> Hi, >>>>>> On 18/05/2017 7:23 AM, ??? wrote: >>>>>>> Am I wrong? >>>>>> >>>>>> I will look at each situation in detail when I get a chance but >>>>>> structured locking enforcement is optional. Also balancing the >>>>>> number of locks and unlocks in a frame does not mean they can't be >>>>>> locked and unlocked in a non-nested fashion - just that by the end >>>>>> the number of unlocks matches the number of locks. >>>>>> >>>>>> BTW the way you respond to these emails, as if having a >>>>>> conversation with yourself, makes it difficult to respond as we >>>>>> can't readily see what is the new email and what is the original. >>>>>> >>>>>> Cheers, >>>>>> David >>>>>> >>>>>>> The byte code for main() in case 1 is as follows. The strange >>>>>>> thing is that NullPointerException is also not thrown at runtime. >>>>> >>>>> That is strange as it does for the normal obvious case of using >>>>> synchronized(o) when o is null. >>>> >>>> Ah - it isn't null it just an object for which the constructor has >>>> not been run. The runtime can't tell the difference between a >>>> pointer to a valid initialized object, and a pointer to an >>>> uninitialized chunk of memory. >>>> >>>>>>> >>>>>>> public void main(java.lang.String[]) throws java.lang.Exception; >>>>>>> descriptor: ([Ljava/lang/String;)V >>>>>>> flags: ACC_PUBLIC >>>>>>> Code: >>>>>>> stack=3, locals=2, args_size=2 >>>>>>> 0: new #2 // class Search >>>> >>>> This allocated an object - hence no null reference. But this is what >>>> verification should have complained about. >>>> >>>> David >>>> ----- >>>> >>>>>>> 3: dup >>>>>>> 4: aload_0 >>>>>>> 5: monitorenter >>>>>>> 6: monitorenter >>>>>>> 7: monitorexit >>>>>>> 8: aload_0 >>>>>>> 9: monitorexit >>>>>>> 10: return >>>>>>> Exceptions: >>>>>>> throws java.lang.Exception >>>>>>> >>>>>>> ??: HotSpot and IBM's J9 behave quite differently when processing >>>>>>> monitorenters and monitorexits >>>>>>> >>>>>>> I have tested several programs (in Jimple) and found that HotSpot >>>>>>> and >>>>>>> J9 match monitorenters and monitorexits quite differently. >>>>>>> Verifiers should play more important roles here. >>>>> >>>>> The job of the verifier is to establish some basic guarantees for >>>>> the JVM to then operate under. The verifier plays no role in >>>>> checking how monitorenter/exit are used in combination, only that >>>>> each individual bytecode meets some basic type constraints. >>>>> >>>>>>> >>>>>>> (1) Test the next program (r2 is not initizlied) on HotSpot and J9. >>>>>>> J9 throw out a verifier error, while HotSpot does not. It seems >>>>>>> that HotSpot's verifier forgets to check whether a monitored >>>>>>> object is initialized. >>>>>>> >>>>>>> public class Search extends java.lang.Object { public void () >>>>>>> { >>>>>>> Search r0; >>>>>>> r0 := @this: Search; >>>>>>> specialinvoke r0.()>(); >>>>>>> return; >>>>>>> } >>>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>>> { >>>>>>> Search r0; >>>>>>> Search r2; >>>>>>> java.lang.String[] r1; >>>>>>> r0 := @this: Search; >>>>>>> r1 := @parameter0: java.lang.String[]; >>>>>>> r2 = new Search; >>>>>>> >>>>>>> entermonitor r2; >>>>>>> entermonitor r0; >>>>>>> exitmonitor r2; >>>>>>> exitmonitor r0; >>>>>>> return; >>>>>>> } >>>>>>> } >>>>> >>>>> Verification was covered above. >>>>> >>>>>>> >>>>>>> (2) Test the next program on HotSpot and J9, and both do not >>>>>>> report any errors. However, I guess the order in the program >>>>>>> (entermonitor r2; => entermonitor r0; => exitmonitor r2; => >>>>>>> exitmonitor r0;) violates the situation of "structured locking" >>>>>>> (Structured locking is the situation when, during a method >>>>>>> invocation, every exit on a given monitor matches a preceding >>>>>>> entry on that monitor, see the specification >>>>>>> >>>>>>> https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-2.html#jvms-2. >>>>>>> >>>>>>> 11.10) >>>>>>> ? >>>>> >>>>> No it doesn't violate structured locking as the number of enters >>>>> and exits match, and there is always an enter before an exit. >>>>> >>>>>>> Actually, the words (every exit on a given monitor matches a >>>>>>> preceding entry on that monitor) are not quite clear as for me. >>>>>>> Otherwise the first rule (The number of monitor entries performed >>>>>>> by T on M during a method invocation must equal the number of >>>>>>> monitor exits performed by T on M during the method invocation >>>>>>> whether the method invocation completes normally or abruptly.) >>>>>>> is sufficient. >>>>> >>>>> The number of enters and exits must not only match/balance, but >>>>> there must be an enter before a corresponding exit. >>>>> >>>>>>> >>>>>>> public class Search extends java.lang.Object { >>>>>>> >>>>>>> public void () >>>>>>> { >>>>>>> Search r0; >>>>>>> r0 := @this: Search; >>>>>>> specialinvoke r0.()>(); >>>>>>> return; >>>>>>> } >>>>>>> >>>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>>> { >>>>>>> Search r0; >>>>>>> Search r2; >>>>>>> java.lang.String[] r1; >>>>>>> r0 := @this: Search; >>>>>>> r1 := @parameter0: java.lang.String[]; >>>>>>> r2 = new Search; >>>>>>> specialinvoke r2.()>(); >>>>>>> entermonitor r2; >>>>>>> entermonitor r0; >>>>>>> exitmonitor r2; >>>>>>> exitmonitor r0; >>>>>>> return; >>>>>>> } >>>>>>> } >>>>>>> >>>>>>> (3) The next program enters monitor in and exits it in main(). >>>>>>> HotSpot throws a runtime exception, while J9 does not. Should >>>>>>> this program be rejected by the verifiers? >>>>> >>>>> No this does not violate any verification rules. The runtime >>>>> behaviour depends on whether structured locking is enforced or >>>>> not.(Even in hotspot there can be differences between interpreted and jitted code). >>>>> >>>>> Hope that helps clarify things. >>>>> >>>>> David >>>>> ----- >>>>> >>>>>>> >>>>>>> public class Search extends java.lang.Object { >>>>>>> >>>>>>> public void () >>>>>>> { >>>>>>> Search r0; >>>>>>> r0 := @this: Search; >>>>>>> specialinvoke r0.()>(); >>>>>>> entermonitor r0; >>>>>>> return; >>>>>>> } >>>>>>> >>>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>>> { >>>>>>> Search r0; >>>>>>> Search r2; >>>>>>> java.lang.S > From chenyt at cs.sjtu.edu.cn Thu May 18 14:55:51 2017 From: chenyt at cs.sjtu.edu.cn (=?UTF-8?B?6ZmI6Zuo5Lqt?=) Date: Thu, 18 May 2017 07:55:51 -0700 Subject: =?UTF-8?Q?=E7=AD=94=E5=A4=8D:_=E7=AD=94=E5=A4=8D:_=E7=AD=94=E5=A4=8D:_HotS?= =?UTF-8?Q?pot_and_IBM's_J9_behave_quite_di?= =?UTF-8?Q?fferently_when_processing_monito?= =?UTF-8?Q?renters_and_monitorexits?= In-Reply-To: References: <005101d2cf53$c8c16b40$5a4441c0$@cs.sjtu.edu.cn> <656dcb18-e65a-4c48-a6bc-cf00fe021af5@oracle.com> <005201d2cf5a$aa2e8f30$fe8bad90$@cs.sjtu.edu.cn> <5a6a2de2-eb43-985d-bfc7-18c3fa7ab0ee@oracle.com> <24a31d5c-8185-e55d-481f-719f3949bf7d@oracle.com> <77c31484-6c76-4b36-671b-71061c98f1ce@oracle.com> <000e01d2cfe2$374939b0$a5dbad10$@cs.sjtu.edu.cn> Message-ID: <001701d2cfe6$dc5918e0$950b4aa0$@cs.sjtu.edu.cn> Is it caused by the locking structure? I tested the next two programs, and saw a difference was raised by the locking structure. The first program throws an IllegalMonitorStateException and the second one throws the NullPointerException. We just monitored another object in the first program. public void main(java.lang.String[]) throws java.lang.Exception; descriptor: ([Ljava/lang/String;)V flags: ACC_PUBLIC Code: stack=1, locals=2, args_size=2 0: aload_0 1: monitorenter 2: aconst_null 3: monitorenter 4: aconst_null 5: monitorexit 6: aload_0 7: monitorexit 8: return Exceptions: throws java.lang.Exception public void main(java.lang.String[]) throws java.lang.Exception; descriptor: ([Ljava/lang/String;)V flags: ACC_PUBLIC Code: stack=1, locals=2, args_size=2 0: aconst_null 1: monitorenter 2: aconst_null 3: monitorexit 4: return Exceptions: throws java.lang.Exception -----????----- ???: Mikael Gerdin [mailto:mikael.gerdin at oracle.com] ????: 2017?5?18? 7:26 ???: ??? ; 'David Holmes' ??: hotspot-runtime-dev at openjdk.java.net ??: Re: ??: ??: HotSpot and IBM's J9 behave quite differently when processing monitorenters and monitorexits Hi, On 2017-05-18 16:22, ??? wrote: > HotSpot throws a NullPointerException correctly when an initialized object as set as null, as follows. > > public void main(java.lang.String[]) throws java.lang.Exception; > descriptor: ([Ljava/lang/String;)V > flags: ACC_PUBLIC > Code: > stack=1, locals=2, args_size=2 > 0: new #2 // class Search > 3: invokespecial #14 // Method "":()V > 6: aconst_null > 7: monitorenter > 8: aconst_null > 9: monitorexit > 10: return > Exceptions: > throws java.lang.Exception > Right. I just realized I missed the fact that the code did end up dereferencing the object but the code which actually makes that happen is not particularly easy to follow. Sorry for the noise. /Mikael > -----????----- > ???: Mikael Gerdin [mailto:mikael.gerdin at oracle.com] > ????: 2017?5?18? 4:55 > ???: David Holmes ; chenyt > > ??: hotspot-runtime-dev at openjdk.java.net > ??: Re: ??: HotSpot and IBM's J9 behave quite differently when > processing monitorenters and monitorexits > > > > On 2017-05-18 07:25, David Holmes wrote: >> Hi Yuting, >> >> On 18/05/2017 2:49 PM, chenyt wrote: >>> Hi, David, >>> >>> I tested the next program (let one object to be monitored be null) >>> using >>> J9 and HotSpot. HotSpot does not throw a NullPointerException, as >>> the specification says. J9 looks fine (throw a >>> NullPointerException). Am I wrong here? >> >> I can't readily test this as I don't have Jimple nor quick and easy >> access to bytecode assemblers. It is strange though as the >> interpreter code contains this: >> >> CASE(_monitorenter): { >> oop lockee = STACK_OBJECT(-1); >> // derefing's lockee ought to provoke implicit null check >> CHECK_NULL(lockee); > > In the x86 templateTable we do > > void TemplateTable::monitorenter() { > transition(atos, vtos); > > // check for NULL object > __ null_check(rax); > > > but looking in macroAssembler_x86.cpp null_check only does a cmp (setting the eflags) but there's no condtional branch on the Z flag? > > It feels like I'm crazy but several of these __ null_check() calls seem broken to me. > > /Mikael > >> >> If I can find some time I will try to test this myself. >> >>> It is very interesting that I have found so many unexpected >>> behaviors here. >> >> Well you only found two and they are related. :) Manually assembled >> monitor code is not something very many (any?) people care about or >> do >> - other than emulating correct language usage. >> >> Cheers, >> David >> >>> public void main(java.lang.String[]) throws java.lang.Exception; >>> descriptor: ([Ljava/lang/String;)V >>> flags: ACC_PUBLIC >>> Code: >>> stack=1, locals=2, args_size=2 >>> 0: aload_0 >>> 1: monitorenter >>> 2: aconst_null >>> 3: monitorenter >>> 4: aconst_null >>> 5: monitorexit >>> 6: aload_0 >>> 7: monitorexit >>> 8: return >>> Exceptions: >>> throws java.lang.Exception >>> >>> Jimple code is given as follows: >>> public void main(java.lang.String[]) throws java.lang.Exception >>> { >>> Search r0; >>> Search r2; >>> java.lang.String[] r1; >>> >>> r0 := @this: Search; >>> r1 := @parameter0: java.lang.String[]; >>> r2 = null; >>> entermonitor r0; >>> entermonitor r2; >>> exitmonitor r2; >>> exitmonitor r0; >>> >>> return; >>> } >>> >>> Wishes, >>> Yuting >>> >>> >>> On Thu, 18 May 2017 13:23:09 +1000, David Holmes wrote: >>>> One correction ... >>>> >>>> On 18/05/2017 10:29 AM, David Holmes wrote: >>>>> On 18/05/2017 8:12 AM, ??? wrote: >>>>>> Thank you, David. I have seen from the specification that >>>>>> structured locking enforcement is optional. The second and the >>>>>> third ones are cases of structured/nested lockings. Will >>>>>> non-nested locking sequences raise deadlocks? Of course it is a >>>>>> different topic, while it might be better if it can be kicked out >>>>>> earlier from the specification/JVM. >>>>> >>>>> Non-nested locking doesn't necessarily lead to deadlocks - you >>>>> just need a different locking order for that (even if properly nested). >>>>> There are locking patterns that rely on the ability to lock and >>>>> unlock in different order ie chained-locking for walking >>>>> linked-lists A->B->C->D: >>>>> - lock A, lock B, unlock A, lock C, unlock B, lock D, unlock C ... >>>>> >>>>> The VM spec allows a little flexibility in how the monitor >>>>> bytecodes can be used compared to the Java programming language. >>>>> That's not something that will change. >>>>> >>>>>> The first example is still a problem. It seems that HotSpot >>>>>> allows to monitor a pure object reference without initialized (Is >>>>>> it true? How can this checking be omitted?). J9 reports a >>>>>> verifyerror as follows. >>>>>> >>>>>> Exception in thread "main" java.lang.VerifyError: JVMVRFY012 >>>>>> stack shape inconsistent; class=Search, >>>>>> method=main([Ljava/lang/String;)V, pc=6 Exception Details: >>>>>> Location: >>>>>> Search.main([Ljava/lang/String;)V @6: JBmonitorenter >>>>>> Reason: >>>>>> Type 'uninitialized' (current frame, stack[1]) is not >>>>>> assignable to 'java/lang/Object' >>>>>> Current Frame: >>>>>> bci: @6 >>>>>> flags: { } >>>>>> locals: { 'Search', '[Ljava/lang/String;' } >>>>>> stack: { 'uninitialized', 'uninitialized' } >>>>>> at T.main(T.java:4) >>>>> >>>>> Yes I think this may be a bug in hotspot. The type-checking for >>>>> the monitor bytecodes requires a matching type of reference on the >>>>> operand stack - but "uninitialized" does not match Object, as J9 >>>>> reports. But I'm not an expert on this aspect of verification so I >>>>> may not be interpreting it correctly. >>>>> >>>>> More below ... >>>>> >>>>>> -----????----- >>>>>> ???: David Holmes [mailto:david.holmes at oracle.com] >>>>>> ????: 2017?5?17? 14:41 >>>>>> ???: ??? ; >>>>>> hotspot-runtime-dev at openjdk.java.net >>>>>> ??: Re: HotSpot and IBM's J9 behave quite differently when >>>>>> processing monitorenters and monitorexits >>>>>> >>>>>> Hi, >>>>>> On 18/05/2017 7:23 AM, ??? wrote: >>>>>>> Am I wrong? >>>>>> >>>>>> I will look at each situation in detail when I get a chance but >>>>>> structured locking enforcement is optional. Also balancing the >>>>>> number of locks and unlocks in a frame does not mean they can't >>>>>> be locked and unlocked in a non-nested fashion - just that by the >>>>>> end the number of unlocks matches the number of locks. >>>>>> >>>>>> BTW the way you respond to these emails, as if having a >>>>>> conversation with yourself, makes it difficult to respond as we >>>>>> can't readily see what is the new email and what is the original. >>>>>> >>>>>> Cheers, >>>>>> David >>>>>> >>>>>>> The byte code for main() in case 1 is as follows. The strange >>>>>>> thing is that NullPointerException is also not thrown at runtime. >>>>> >>>>> That is strange as it does for the normal obvious case of using >>>>> synchronized(o) when o is null. >>>> >>>> Ah - it isn't null it just an object for which the constructor has >>>> not been run. The runtime can't tell the difference between a >>>> pointer to a valid initialized object, and a pointer to an >>>> uninitialized chunk of memory. >>>> >>>>>>> >>>>>>> public void main(java.lang.String[]) throws java.lang.Exception; >>>>>>> descriptor: ([Ljava/lang/String;)V >>>>>>> flags: ACC_PUBLIC >>>>>>> Code: >>>>>>> stack=3, locals=2, args_size=2 >>>>>>> 0: new #2 // class Search >>>> >>>> This allocated an object - hence no null reference. But this is >>>> what verification should have complained about. >>>> >>>> David >>>> ----- >>>> >>>>>>> 3: dup >>>>>>> 4: aload_0 >>>>>>> 5: monitorenter >>>>>>> 6: monitorenter >>>>>>> 7: monitorexit >>>>>>> 8: aload_0 >>>>>>> 9: monitorexit >>>>>>> 10: return >>>>>>> Exceptions: >>>>>>> throws java.lang.Exception >>>>>>> >>>>>>> ??: HotSpot and IBM's J9 behave quite differently when >>>>>>> processing monitorenters and monitorexits >>>>>>> >>>>>>> I have tested several programs (in Jimple) and found that >>>>>>> HotSpot and >>>>>>> J9 match monitorenters and monitorexits quite differently. >>>>>>> Verifiers should play more important roles here. >>>>> >>>>> The job of the verifier is to establish some basic guarantees for >>>>> the JVM to then operate under. The verifier plays no role in >>>>> checking how monitorenter/exit are used in combination, only that >>>>> each individual bytecode meets some basic type constraints. >>>>> >>>>>>> >>>>>>> (1) Test the next program (r2 is not initizlied) on HotSpot and J9. >>>>>>> J9 throw out a verifier error, while HotSpot does not. It seems >>>>>>> that HotSpot's verifier forgets to check whether a monitored >>>>>>> object is initialized. >>>>>>> >>>>>>> public class Search extends java.lang.Object { public void () >>>>>>> { >>>>>>> Search r0; >>>>>>> r0 := @this: Search; >>>>>>> specialinvoke r0.()>(); >>>>>>> return; >>>>>>> } >>>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>>> { >>>>>>> Search r0; >>>>>>> Search r2; >>>>>>> java.lang.String[] r1; >>>>>>> r0 := @this: Search; >>>>>>> r1 := @parameter0: java.lang.String[]; >>>>>>> r2 = new Search; >>>>>>> >>>>>>> entermonitor r2; >>>>>>> entermonitor r0; >>>>>>> exitmonitor r2; >>>>>>> exitmonitor r0; >>>>>>> return; >>>>>>> } >>>>>>> } >>>>> >>>>> Verification was covered above. >>>>> >>>>>>> >>>>>>> (2) Test the next program on HotSpot and J9, and both do not >>>>>>> report any errors. However, I guess the order in the program >>>>>>> (entermonitor r2; => entermonitor r0; => exitmonitor r2; => >>>>>>> exitmonitor r0;) violates the situation of "structured locking" >>>>>>> (Structured locking is the situation when, during a method >>>>>>> invocation, every exit on a given monitor matches a preceding >>>>>>> entry on that monitor, see the specification >>>>>>> >>>>>>> https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-2.html#jvms-2. >>>>>>> >>>>>>> 11.10) >>>>>>> ? >>>>> >>>>> No it doesn't violate structured locking as the number of enters >>>>> and exits match, and there is always an enter before an exit. >>>>> >>>>>>> Actually, the words (every exit on a given monitor matches a >>>>>>> preceding entry on that monitor) are not quite clear as for me. >>>>>>> Otherwise the first rule (The number of monitor entries >>>>>>> performed by T on M during a method invocation must equal the >>>>>>> number of monitor exits performed by T on M during the method >>>>>>> invocation whether the method invocation completes normally or >>>>>>> abruptly.) is sufficient. >>>>> >>>>> The number of enters and exits must not only match/balance, but >>>>> there must be an enter before a corresponding exit. >>>>> >>>>>>> >>>>>>> public class Search extends java.lang.Object { >>>>>>> >>>>>>> public void () >>>>>>> { >>>>>>> Search r0; >>>>>>> r0 := @this: Search; >>>>>>> specialinvoke r0.()>(); >>>>>>> return; >>>>>>> } >>>>>>> >>>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>>> { >>>>>>> Search r0; >>>>>>> Search r2; >>>>>>> java.lang.String[] r1; >>>>>>> r0 := @this: Search; >>>>>>> r1 := @parameter0: java.lang.String[]; >>>>>>> r2 = new Search; >>>>>>> specialinvoke r2.()>(); >>>>>>> entermonitor r2; >>>>>>> entermonitor r0; >>>>>>> exitmonitor r2; >>>>>>> exitmonitor r0; >>>>>>> return; >>>>>>> } >>>>>>> } >>>>>>> >>>>>>> (3) The next program enters monitor in and exits it in main(). >>>>>>> HotSpot throws a runtime exception, while J9 does not. Should >>>>>>> this program be rejected by the verifiers? >>>>> >>>>> No this does not violate any verification rules. The runtime >>>>> behaviour depends on whether structured locking is enforced or >>>>> not.(Even in hotspot there can be differences between interpreted and jitted code). >>>>> >>>>> Hope that helps clarify things. >>>>> >>>>> David >>>>> ----- >>>>> >>>>>>> >>>>>>> public class Search extends java.lang.Object { >>>>>>> >>>>>>> public void () >>>>>>> { >>>>>>> Search r0; >>>>>>> r0 := @this: Search; >>>>>>> specialinvoke r0.()>(); >>>>>>> entermonitor r0; >>>>>>> return; >>>>>>> } >>>>>>> >>>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>>> { >>>>>>> Search r0; >>>>>>> Search r2; >>>>>>> java.lang.S > From chenyt at cs.sjtu.edu.cn Thu May 18 15:15:20 2017 From: chenyt at cs.sjtu.edu.cn (=?UTF-8?B?6ZmI6Zuo5Lqt?=) Date: Thu, 18 May 2017 08:15:20 -0700 Subject: =?UTF-8?Q?=E7=AD=94=E5=A4=8D:_=E7=AD=94=E5=A4=8D:_=E7=AD=94=E5=A4=8D:_HotS?= =?UTF-8?Q?pot_and_IBM's_J9_behave_quite_di?= =?UTF-8?Q?fferently_when_processing_monito?= =?UTF-8?Q?renters_and_monitorexits?= References: <005101d2cf53$c8c16b40$5a4441c0$@cs.sjtu.edu.cn> <656dcb18-e65a-4c48-a6bc-cf00fe021af5@oracle.com> <005201d2cf5a$aa2e8f30$fe8bad90$@cs.sjtu.edu.cn> <5a6a2de2-eb43-985d-bfc7-18c3fa7ab0ee@oracle.com> <24a31d5c-8185-e55d-481f-719f3949bf7d@oracle.com> <77c31484-6c76-4b36-671b-71061c98f1ce@oracle.com> <000e01d2cfe2$374939b0$a5dbad10$@cs.sjtu.edu.cn> Message-ID: <001801d2cfe9$94e5e670$beb1b350$@cs.sjtu.edu.cn> HotSpot still misses catching the NullPointerException for the next program (only one pair of entermonitor and monitorexit). It only performs checks when the function exits and throw an IllegalMonitorStateException. We should enumerate more cases to test the JVMs. public void main(java.lang.String[]) throws java.lang.Exception; descriptor: ([Ljava/lang/String;)V flags: ACC_PUBLIC Code: stack=1, locals=2, args_size=2 0: aload_0 1: monitorenter 2: aconst_null 3: monitorexit 4: return Exceptions: throws java.lang.Exception -----????----- ???: ??? [mailto:chenyt at cs.sjtu.edu.cn] ????: 2017?5?18? 7:56 ???: 'Mikael Gerdin' ; 'David Holmes' ??: 'hotspot-runtime-dev at openjdk.java.net' ??: ??: ??: ??: HotSpot and IBM's J9 behave quite differently when processing monitorenters and monitorexits Is it caused by the locking structure? I tested the next two programs, and saw a difference was raised by the locking structure. The first program throws an IllegalMonitorStateException and the second one throws the NullPointerException. We just monitored another object in the first program. public void main(java.lang.String[]) throws java.lang.Exception; descriptor: ([Ljava/lang/String;)V flags: ACC_PUBLIC Code: stack=1, locals=2, args_size=2 0: aload_0 1: monitorenter 2: aconst_null 3: monitorenter 4: aconst_null 5: monitorexit 6: aload_0 7: monitorexit 8: return Exceptions: throws java.lang.Exception public void main(java.lang.String[]) throws java.lang.Exception; descriptor: ([Ljava/lang/String;)V flags: ACC_PUBLIC Code: stack=1, locals=2, args_size=2 0: aconst_null 1: monitorenter 2: aconst_null 3: monitorexit 4: return Exceptions: throws java.lang.Exception -----????----- ???: Mikael Gerdin [mailto:mikael.gerdin at oracle.com] ????: 2017?5?18? 7:26 ???: ??? ; 'David Holmes' ??: hotspot-runtime-dev at openjdk.java.net ??: Re: ??: ??: HotSpot and IBM's J9 behave quite differently when processing monitorenters and monitorexits Hi, On 2017-05-18 16:22, ??? wrote: > HotSpot throws a NullPointerException correctly when an initialized object as set as null, as follows. > > public void main(java.lang.String[]) throws java.lang.Exception; > descriptor: ([Ljava/lang/String;)V > flags: ACC_PUBLIC > Code: > stack=1, locals=2, args_size=2 > 0: new #2 // class Search > 3: invokespecial #14 // Method "":()V > 6: aconst_null > 7: monitorenter > 8: aconst_null > 9: monitorexit > 10: return > Exceptions: > throws java.lang.Exception > Right. I just realized I missed the fact that the code did end up dereferencing the object but the code which actually makes that happen is not particularly easy to follow. Sorry for the noise. /Mikael > -----????----- > ???: Mikael Gerdin [mailto:mikael.gerdin at oracle.com] > ????: 2017?5?18? 4:55 > ???: David Holmes ; chenyt > > ??: hotspot-runtime-dev at openjdk.java.net > ??: Re: ??: HotSpot and IBM's J9 behave quite differently when > processing monitorenters and monitorexits > > > > On 2017-05-18 07:25, David Holmes wrote: >> Hi Yuting, >> >> On 18/05/2017 2:49 PM, chenyt wrote: >>> Hi, David, >>> >>> I tested the next program (let one object to be monitored be null) >>> using >>> J9 and HotSpot. HotSpot does not throw a NullPointerException, as >>> the specification says. J9 looks fine (throw a >>> NullPointerException). Am I wrong here? >> >> I can't readily test this as I don't have Jimple nor quick and easy >> access to bytecode assemblers. It is strange though as the >> interpreter code contains this: >> >> CASE(_monitorenter): { >> oop lockee = STACK_OBJECT(-1); >> // derefing's lockee ought to provoke implicit null check >> CHECK_NULL(lockee); > > In the x86 templateTable we do > > void TemplateTable::monitorenter() { > transition(atos, vtos); > > // check for NULL object > __ null_check(rax); > > > but looking in macroAssembler_x86.cpp null_check only does a cmp (setting the eflags) but there's no condtional branch on the Z flag? > > It feels like I'm crazy but several of these __ null_check() calls seem broken to me. > > /Mikael > >> >> If I can find some time I will try to test this myself. >> >>> It is very interesting that I have found so many unexpected >>> behaviors here. >> >> Well you only found two and they are related. :) Manually assembled >> monitor code is not something very many (any?) people care about or >> do >> - other than emulating correct language usage. >> >> Cheers, >> David >> >>> public void main(java.lang.String[]) throws java.lang.Exception; >>> descriptor: ([Ljava/lang/String;)V >>> flags: ACC_PUBLIC >>> Code: >>> stack=1, locals=2, args_size=2 >>> 0: aload_0 >>> 1: monitorenter >>> 2: aconst_null >>> 3: monitorenter >>> 4: aconst_null >>> 5: monitorexit >>> 6: aload_0 >>> 7: monitorexit >>> 8: return >>> Exceptions: >>> throws java.lang.Exception >>> >>> Jimple code is given as follows: >>> public void main(java.lang.String[]) throws java.lang.Exception >>> { >>> Search r0; >>> Search r2; >>> java.lang.String[] r1; >>> >>> r0 := @this: Search; >>> r1 := @parameter0: java.lang.String[]; >>> r2 = null; >>> entermonitor r0; >>> entermonitor r2; >>> exitmonitor r2; >>> exitmonitor r0; >>> >>> return; >>> } >>> >>> Wishes, >>> Yuting >>> >>> >>> On Thu, 18 May 2017 13:23:09 +1000, David Holmes wrote: >>>> One correction ... >>>> >>>> On 18/05/2017 10:29 AM, David Holmes wrote: >>>>> On 18/05/2017 8:12 AM, ??? wrote: >>>>>> Thank you, David. I have seen from the specification that >>>>>> structured locking enforcement is optional. The second and the >>>>>> third ones are cases of structured/nested lockings. Will >>>>>> non-nested locking sequences raise deadlocks? Of course it is a >>>>>> different topic, while it might be better if it can be kicked out >>>>>> earlier from the specification/JVM. >>>>> >>>>> Non-nested locking doesn't necessarily lead to deadlocks - you >>>>> just need a different locking order for that (even if properly nested). >>>>> There are locking patterns that rely on the ability to lock and >>>>> unlock in different order ie chained-locking for walking >>>>> linked-lists A->B->C->D: >>>>> - lock A, lock B, unlock A, lock C, unlock B, lock D, unlock C ... >>>>> >>>>> The VM spec allows a little flexibility in how the monitor >>>>> bytecodes can be used compared to the Java programming language. >>>>> That's not something that will change. >>>>> >>>>>> The first example is still a problem. It seems that HotSpot >>>>>> allows to monitor a pure object reference without initialized (Is >>>>>> it true? How can this checking be omitted?). J9 reports a >>>>>> verifyerror as follows. >>>>>> >>>>>> Exception in thread "main" java.lang.VerifyError: JVMVRFY012 >>>>>> stack shape inconsistent; class=Search, >>>>>> method=main([Ljava/lang/String;)V, pc=6 Exception Details: >>>>>> Location: >>>>>> Search.main([Ljava/lang/String;)V @6: JBmonitorenter >>>>>> Reason: >>>>>> Type 'uninitialized' (current frame, stack[1]) is not >>>>>> assignable to 'java/lang/Object' >>>>>> Current Frame: >>>>>> bci: @6 >>>>>> flags: { } >>>>>> locals: { 'Search', '[Ljava/lang/String;' } >>>>>> stack: { 'uninitialized', 'uninitialized' } >>>>>> at T.main(T.java:4) >>>>> >>>>> Yes I think this may be a bug in hotspot. The type-checking for >>>>> the monitor bytecodes requires a matching type of reference on the >>>>> operand stack - but "uninitialized" does not match Object, as J9 >>>>> reports. But I'm not an expert on this aspect of verification so I >>>>> may not be interpreting it correctly. >>>>> >>>>> More below ... >>>>> >>>>>> -----????----- >>>>>> ???: David Holmes [mailto:david.holmes at oracle.com] >>>>>> ????: 2017?5?17? 14:41 >>>>>> ???: ??? ; >>>>>> hotspot-runtime-dev at openjdk.java.net >>>>>> ??: Re: HotSpot and IBM's J9 behave quite differently when >>>>>> processing monitorenters and monitorexits >>>>>> >>>>>> Hi, >>>>>> On 18/05/2017 7:23 AM, ??? wrote: >>>>>>> Am I wrong? >>>>>> >>>>>> I will look at each situation in detail when I get a chance but >>>>>> structured locking enforcement is optional. Also balancing the >>>>>> number of locks and unlocks in a frame does not mean they can't >>>>>> be locked and unlocked in a non-nested fashion - just that by the >>>>>> end the number of unlocks matches the number of locks. >>>>>> >>>>>> BTW the way you respond to these emails, as if having a >>>>>> conversation with yourself, makes it difficult to respond as we >>>>>> can't readily see what is the new email and what is the original. >>>>>> >>>>>> Cheers, >>>>>> David >>>>>> >>>>>>> The byte code for main() in case 1 is as follows. The strange >>>>>>> thing is that NullPointerException is also not thrown at runtime. >>>>> >>>>> That is strange as it does for the normal obvious case of using >>>>> synchronized(o) when o is null. >>>> >>>> Ah - it isn't null it just an object for which the constructor has >>>> not been run. The runtime can't tell the difference between a >>>> pointer to a valid initialized object, and a pointer to an >>>> uninitialized chunk of memory. >>>> >>>>>>> >>>>>>> public void main(java.lang.String[]) throws java.lang.Exception; >>>>>>> descriptor: ([Ljava/lang/String;)V >>>>>>> flags: ACC_PUBLIC >>>>>>> Code: >>>>>>> stack=3, locals=2, args_size=2 >>>>>>> 0: new #2 // class Search >>>> >>>> This allocated an object - hence no null reference. But this is >>>> what verification should have complained about. >>>> >>>> David >>>> ----- >>>> >>>>>>> 3: dup >>>>>>> 4: aload_0 >>>>>>> 5: monitorenter >>>>>>> 6: monitorenter >>>>>>> 7: monitorexit >>>>>>> 8: aload_0 >>>>>>> 9: monitorexit >>>>>>> 10: return >>>>>>> Exceptions: >>>>>>> throws java.lang.Exception >>>>>>> >>>>>>> ??: HotSpot and IBM's J9 behave quite differently when >>>>>>> processing monitorenters and monitorexits >>>>>>> >>>>>>> I have tested several programs (in Jimple) and found that >>>>>>> HotSpot and >>>>>>> J9 match monitorenters and monitorexits quite differently. >>>>>>> Verifiers should play more important roles here. >>>>> >>>>> The job of the verifier is to establish some basic guarantees for >>>>> the JVM to then operate under. The verifier plays no role in >>>>> checking how monitorenter/exit are used in combination, only that >>>>> each individual bytecode meets some basic type constraints. >>>>> >>>>>>> >>>>>>> (1) Test the next program (r2 is not initizlied) on HotSpot and J9. >>>>>>> J9 throw out a verifier error, while HotSpot does not. It seems >>>>>>> that HotSpot's verifier forgets to check whether a monitored >>>>>>> object is initialized. >>>>>>> >>>>>>> public class Search extends java.lang.Object { public void () >>>>>>> { >>>>>>> Search r0; >>>>>>> r0 := @this: Search; >>>>>>> specialinvoke r0.()>(); >>>>>>> return; >>>>>>> } >>>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>>> { >>>>>>> Search r0; >>>>>>> Search r2; >>>>>>> java.lang.String[] r1; >>>>>>> r0 := @this: Search; >>>>>>> r1 := @parameter0: java.lang.String[]; >>>>>>> r2 = new Search; >>>>>>> >>>>>>> entermonitor r2; >>>>>>> entermonitor r0; >>>>>>> exitmonitor r2; >>>>>>> exitmonitor r0; >>>>>>> return; >>>>>>> } >>>>>>> } >>>>> >>>>> Verification was covered above. >>>>> >>>>>>> >>>>>>> (2) Test the next program on HotSpot and J9, and both do not >>>>>>> report any errors. However, I guess the order in the program >>>>>>> (entermonitor r2; => entermonitor r0; => exitmonitor r2; => >>>>>>> exitmonitor r0;) violates the situation of "structured locking" >>>>>>> (Structured locking is the situation when, during a method >>>>>>> invocation, every exit on a given monitor matches a preceding >>>>>>> entry on that monitor, see the specification >>>>>>> >>>>>>> https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-2.html#jvms-2. >>>>>>> >>>>>>> 11.10) >>>>>>> ? >>>>> >>>>> No it doesn't violate structured locking as the number of enters >>>>> and exits match, and there is always an enter before an exit. >>>>> >>>>>>> Actually, the words (every exit on a given monitor matches a >>>>>>> preceding entry on that monitor) are not quite clear as for me. >>>>>>> Otherwise the first rule (The number of monitor entries >>>>>>> performed by T on M during a method invocation must equal the >>>>>>> number of monitor exits performed by T on M during the method >>>>>>> invocation whether the method invocation completes normally or >>>>>>> abruptly.) is sufficient. >>>>> >>>>> The number of enters and exits must not only match/balance, but >>>>> there must be an enter before a corresponding exit. >>>>> >>>>>>> >>>>>>> public class Search extends java.lang.Object { >>>>>>> >>>>>>> public void () >>>>>>> { >>>>>>> Search r0; >>>>>>> r0 := @this: Search; >>>>>>> specialinvoke r0.()>(); >>>>>>> return; >>>>>>> } >>>>>>> >>>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>>> { >>>>>>> Search r0; >>>>>>> Search r2; >>>>>>> java.lang.String[] r1; >>>>>>> r0 := @this: Search; >>>>>>> r1 := @parameter0: java.lang.String[]; >>>>>>> r2 = new Search; >>>>>>> specialinvoke r2.()>(); >>>>>>> entermonitor r2; >>>>>>> entermonitor r0; >>>>>>> exitmonitor r2; >>>>>>> exitmonitor r0; >>>>>>> return; >>>>>>> } >>>>>>> } >>>>>>> >>>>>>> (3) The next program enters monitor in and exits it in main(). >>>>>>> HotSpot throws a runtime exception, while J9 does not. Should >>>>>>> this program be rejected by the verifiers? >>>>> >>>>> No this does not violate any verification rules. The runtime >>>>> behaviour depends on whether structured locking is enforced or >>>>> not.(Even in hotspot there can be differences between interpreted and jitted code). >>>>> >>>>> Hope that helps clarify things. >>>>> >>>>> David >>>>> ----- >>>>> >>>>>>> >>>>>>> public class Search extends java.lang.Object { >>>>>>> >>>>>>> public void () >>>>>>> { >>>>>>> Search r0; >>>>>>> r0 := @this: Search; >>>>>>> specialinvoke r0.()>(); >>>>>>> entermonitor r0; >>>>>>> return; >>>>>>> } >>>>>>> >>>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>>> { >>>>>>> Search r0; >>>>>>> Search r2; >>>>>>> java.lang.S > From claes.redestad at oracle.com Thu May 18 16:35:38 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 18 May 2017 18:35:38 +0200 Subject: RFR[10]: 8180614: Skip range and constraint checks on non-existent flags Message-ID: <09110ab8-9538-d8c2-08ab-f8cb145c4595@oracle.com> Hi, analyzing the remaining overhead in command line flag checking after JDK-8178991, I found that we're taking a runtime cost scanning for flags that doesn't exist in the build, e.g., develop flags in a product build. Since such flags are simply emitted as constants, they can't[1] be in violation of the constraints, thus I think we can safely skip such checks altogether. This removes ~500k instructions from being executed for no reason during VM startup on a 64-bit JVM. Bug: https://bugs.openjdk.java.net/browse/JDK-8180614 Webrev: http://cr.openjdk.java.net/~redestad/8180614/hotspot.00/ Thanks! /Claes [1] Unless the default value is invalid, which is a potential programmer error that would be caught in debug builds anyhow. From ioi.lam at oracle.com Thu May 18 16:47:12 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 18 May 2017 09:47:12 -0700 Subject: =?UTF-8?B?562U5aSNOiDnrZTlpI06IOetlOWkjTogSG90U3BvdCBhbmQgSUI=?= =?UTF-8?B?TSdzIEo5IGJlaGF2ZSBxdWl0ZSBkaWZmZXJlbnRseSB3aGVuIHByb2Nlc3Npbmc=?= =?UTF-8?B?IG1vbml0b3JlbnRlcnMgYW5kIG1vbml0b3JleGl0cw==?= In-Reply-To: <001801d2cfe9$94e5e670$beb1b350$@cs.sjtu.edu.cn> References: <005101d2cf53$c8c16b40$5a4441c0$@cs.sjtu.edu.cn> <656dcb18-e65a-4c48-a6bc-cf00fe021af5@oracle.com> <005201d2cf5a$aa2e8f30$fe8bad90$@cs.sjtu.edu.cn> <5a6a2de2-eb43-985d-bfc7-18c3fa7ab0ee@oracle.com> <24a31d5c-8185-e55d-481f-719f3949bf7d@oracle.com> <77c31484-6c76-4b36-671b-71061c98f1ce@oracle.com> <000e01d2cfe2$374939b0$a5dbad10$@cs.sjtu.edu.cn> <001801d2cfe9$94e5e670$beb1b350$@cs.sjtu.edu.cn> Message-ID: <591DD010.3030101@oracle.com> Hi ??, Thanks for the bug report. I have created a new bug, https://bugs.openjdk.java.net/browse/JDK-8180615 and attached your test case. JDK-8180615 - monitorenter on null object produces unexpected IllegalMonitorStateException - Ioi On 5/18/17 8:15 AM, ??? wrote: > HotSpot still misses catching the NullPointerException for the next program (only one pair of entermonitor and monitorexit). It only performs checks when the function exits and throw an IllegalMonitorStateException. We should enumerate more cases to test the JVMs. > > public void main(java.lang.String[]) throws java.lang.Exception; > descriptor: ([Ljava/lang/String;)V > flags: ACC_PUBLIC > Code: > stack=1, locals=2, args_size=2 > 0: aload_0 > 1: monitorenter > 2: aconst_null > 3: monitorexit > 4: return > Exceptions: > throws java.lang.Exception > > -----????----- > ???: ??? [mailto:chenyt at cs.sjtu.edu.cn] > ????: 2017?5?18? 7:56 > ???: 'Mikael Gerdin' ; 'David Holmes' > ??: 'hotspot-runtime-dev at openjdk.java.net' > ??: ??: ??: ??: HotSpot and IBM's J9 behave quite differently when processing monitorenters and monitorexits > > Is it caused by the locking structure? I tested the next two programs, and saw a difference was raised by the locking structure. The first program throws an IllegalMonitorStateException and the second one throws the NullPointerException. We just monitored another object in the first program. > > public void main(java.lang.String[]) throws java.lang.Exception; > descriptor: ([Ljava/lang/String;)V > flags: ACC_PUBLIC > Code: > stack=1, locals=2, args_size=2 > 0: aload_0 > 1: monitorenter > 2: aconst_null > 3: monitorenter > 4: aconst_null > 5: monitorexit > 6: aload_0 > 7: monitorexit > 8: return > Exceptions: > throws java.lang.Exception > > > public void main(java.lang.String[]) throws java.lang.Exception; > descriptor: ([Ljava/lang/String;)V > flags: ACC_PUBLIC > Code: > stack=1, locals=2, args_size=2 > 0: aconst_null > 1: monitorenter > 2: aconst_null > 3: monitorexit > 4: return > Exceptions: > throws java.lang.Exception > > -----????----- > ???: Mikael Gerdin [mailto:mikael.gerdin at oracle.com] > ????: 2017?5?18? 7:26 > ???: ??? ; 'David Holmes' > ??: hotspot-runtime-dev at openjdk.java.net > ??: Re: ??: ??: HotSpot and IBM's J9 behave quite differently when processing monitorenters and monitorexits > > Hi, > > On 2017-05-18 16:22, ??? wrote: >> HotSpot throws a NullPointerException correctly when an initialized object as set as null, as follows. >> >> public void main(java.lang.String[]) throws java.lang.Exception; >> descriptor: ([Ljava/lang/String;)V >> flags: ACC_PUBLIC >> Code: >> stack=1, locals=2, args_size=2 >> 0: new #2 // class Search >> 3: invokespecial #14 // Method "":()V >> 6: aconst_null >> 7: monitorenter >> 8: aconst_null >> 9: monitorexit >> 10: return >> Exceptions: >> throws java.lang.Exception >> > Right. I just realized I missed the fact that the code did end up dereferencing the object but the code which actually makes that happen is not particularly easy to follow. > Sorry for the noise. > /Mikael > >> -----????----- >> ???: Mikael Gerdin [mailto:mikael.gerdin at oracle.com] >> ????: 2017?5?18? 4:55 >> ???: David Holmes ; chenyt >> >> ??: hotspot-runtime-dev at openjdk.java.net >> ??: Re: ??: HotSpot and IBM's J9 behave quite differently when >> processing monitorenters and monitorexits >> >> >> >> On 2017-05-18 07:25, David Holmes wrote: >>> Hi Yuting, >>> >>> On 18/05/2017 2:49 PM, chenyt wrote: >>>> Hi, David, >>>> >>>> I tested the next program (let one object to be monitored be null) >>>> using >>>> J9 and HotSpot. HotSpot does not throw a NullPointerException, as >>>> the specification says. J9 looks fine (throw a >>>> NullPointerException). Am I wrong here? >>> I can't readily test this as I don't have Jimple nor quick and easy >>> access to bytecode assemblers. It is strange though as the >>> interpreter code contains this: >>> >>> CASE(_monitorenter): { >>> oop lockee = STACK_OBJECT(-1); >>> // derefing's lockee ought to provoke implicit null check >>> CHECK_NULL(lockee); >> In the x86 templateTable we do >> >> void TemplateTable::monitorenter() { >> transition(atos, vtos); >> >> // check for NULL object >> __ null_check(rax); >> >> >> but looking in macroAssembler_x86.cpp null_check only does a cmp (setting the eflags) but there's no condtional branch on the Z flag? >> >> It feels like I'm crazy but several of these __ null_check() calls seem broken to me. >> >> /Mikael >> >>> If I can find some time I will try to test this myself. >>> >>>> It is very interesting that I have found so many unexpected >>>> behaviors here. >>> Well you only found two and they are related. :) Manually assembled >>> monitor code is not something very many (any?) people care about or >>> do >>> - other than emulating correct language usage. >>> >>> Cheers, >>> David >>> >>>> public void main(java.lang.String[]) throws java.lang.Exception; >>>> descriptor: ([Ljava/lang/String;)V >>>> flags: ACC_PUBLIC >>>> Code: >>>> stack=1, locals=2, args_size=2 >>>> 0: aload_0 >>>> 1: monitorenter >>>> 2: aconst_null >>>> 3: monitorenter >>>> 4: aconst_null >>>> 5: monitorexit >>>> 6: aload_0 >>>> 7: monitorexit >>>> 8: return >>>> Exceptions: >>>> throws java.lang.Exception >>>> >>>> Jimple code is given as follows: >>>> public void main(java.lang.String[]) throws java.lang.Exception >>>> { >>>> Search r0; >>>> Search r2; >>>> java.lang.String[] r1; >>>> >>>> r0 := @this: Search; >>>> r1 := @parameter0: java.lang.String[]; >>>> r2 = null; >>>> entermonitor r0; >>>> entermonitor r2; >>>> exitmonitor r2; >>>> exitmonitor r0; >>>> >>>> return; >>>> } >>>> >>>> Wishes, >>>> Yuting >>>> >>>> >>>> On Thu, 18 May 2017 13:23:09 +1000, David Holmes wrote: >>>>> One correction ... >>>>> >>>>> On 18/05/2017 10:29 AM, David Holmes wrote: >>>>>> On 18/05/2017 8:12 AM, ??? wrote: >>>>>>> Thank you, David. I have seen from the specification that >>>>>>> structured locking enforcement is optional. The second and the >>>>>>> third ones are cases of structured/nested lockings. Will >>>>>>> non-nested locking sequences raise deadlocks? Of course it is a >>>>>>> different topic, while it might be better if it can be kicked out >>>>>>> earlier from the specification/JVM. >>>>>> Non-nested locking doesn't necessarily lead to deadlocks - you >>>>>> just need a different locking order for that (even if properly nested). >>>>>> There are locking patterns that rely on the ability to lock and >>>>>> unlock in different order ie chained-locking for walking >>>>>> linked-lists A->B->C->D: >>>>>> - lock A, lock B, unlock A, lock C, unlock B, lock D, unlock C ... >>>>>> >>>>>> The VM spec allows a little flexibility in how the monitor >>>>>> bytecodes can be used compared to the Java programming language. >>>>>> That's not something that will change. >>>>>> >>>>>>> The first example is still a problem. It seems that HotSpot >>>>>>> allows to monitor a pure object reference without initialized (Is >>>>>>> it true? How can this checking be omitted?). J9 reports a >>>>>>> verifyerror as follows. >>>>>>> >>>>>>> Exception in thread "main" java.lang.VerifyError: JVMVRFY012 >>>>>>> stack shape inconsistent; class=Search, >>>>>>> method=main([Ljava/lang/String;)V, pc=6 Exception Details: >>>>>>> Location: >>>>>>> Search.main([Ljava/lang/String;)V @6: JBmonitorenter >>>>>>> Reason: >>>>>>> Type 'uninitialized' (current frame, stack[1]) is not >>>>>>> assignable to 'java/lang/Object' >>>>>>> Current Frame: >>>>>>> bci: @6 >>>>>>> flags: { } >>>>>>> locals: { 'Search', '[Ljava/lang/String;' } >>>>>>> stack: { 'uninitialized', 'uninitialized' } >>>>>>> at T.main(T.java:4) >>>>>> Yes I think this may be a bug in hotspot. The type-checking for >>>>>> the monitor bytecodes requires a matching type of reference on the >>>>>> operand stack - but "uninitialized" does not match Object, as J9 >>>>>> reports. But I'm not an expert on this aspect of verification so I >>>>>> may not be interpreting it correctly. >>>>>> >>>>>> More below ... >>>>>> >>>>>>> -----????----- >>>>>>> ???: David Holmes [mailto:david.holmes at oracle.com] >>>>>>> ????: 2017?5?17? 14:41 >>>>>>> ???: ??? ; >>>>>>> hotspot-runtime-dev at openjdk.java.net >>>>>>> ??: Re: HotSpot and IBM's J9 behave quite differently when >>>>>>> processing monitorenters and monitorexits >>>>>>> >>>>>>> Hi, >>>>>>> On 18/05/2017 7:23 AM, ??? wrote: >>>>>>>> Am I wrong? >>>>>>> I will look at each situation in detail when I get a chance but >>>>>>> structured locking enforcement is optional. Also balancing the >>>>>>> number of locks and unlocks in a frame does not mean they can't >>>>>>> be locked and unlocked in a non-nested fashion - just that by the >>>>>>> end the number of unlocks matches the number of locks. >>>>>>> >>>>>>> BTW the way you respond to these emails, as if having a >>>>>>> conversation with yourself, makes it difficult to respond as we >>>>>>> can't readily see what is the new email and what is the original. >>>>>>> >>>>>>> Cheers, >>>>>>> David >>>>>>> >>>>>>>> The byte code for main() in case 1 is as follows. The strange >>>>>>>> thing is that NullPointerException is also not thrown at runtime. >>>>>> That is strange as it does for the normal obvious case of using >>>>>> synchronized(o) when o is null. >>>>> Ah - it isn't null it just an object for which the constructor has >>>>> not been run. The runtime can't tell the difference between a >>>>> pointer to a valid initialized object, and a pointer to an >>>>> uninitialized chunk of memory. >>>>> >>>>>>>> public void main(java.lang.String[]) throws java.lang.Exception; >>>>>>>> descriptor: ([Ljava/lang/String;)V >>>>>>>> flags: ACC_PUBLIC >>>>>>>> Code: >>>>>>>> stack=3, locals=2, args_size=2 >>>>>>>> 0: new #2 // class Search >>>>> This allocated an object - hence no null reference. But this is >>>>> what verification should have complained about. >>>>> >>>>> David >>>>> ----- >>>>> >>>>>>>> 3: dup >>>>>>>> 4: aload_0 >>>>>>>> 5: monitorenter >>>>>>>> 6: monitorenter >>>>>>>> 7: monitorexit >>>>>>>> 8: aload_0 >>>>>>>> 9: monitorexit >>>>>>>> 10: return >>>>>>>> Exceptions: >>>>>>>> throws java.lang.Exception >>>>>>>> >>>>>>>> ??: HotSpot and IBM's J9 behave quite differently when >>>>>>>> processing monitorenters and monitorexits >>>>>>>> >>>>>>>> I have tested several programs (in Jimple) and found that >>>>>>>> HotSpot and >>>>>>>> J9 match monitorenters and monitorexits quite differently. >>>>>>>> Verifiers should play more important roles here. >>>>>> The job of the verifier is to establish some basic guarantees for >>>>>> the JVM to then operate under. The verifier plays no role in >>>>>> checking how monitorenter/exit are used in combination, only that >>>>>> each individual bytecode meets some basic type constraints. >>>>>> >>>>>>>> (1) Test the next program (r2 is not initizlied) on HotSpot and J9. >>>>>>>> J9 throw out a verifier error, while HotSpot does not. It seems >>>>>>>> that HotSpot's verifier forgets to check whether a monitored >>>>>>>> object is initialized. >>>>>>>> >>>>>>>> public class Search extends java.lang.Object { public void () >>>>>>>> { >>>>>>>> Search r0; >>>>>>>> r0 := @this: Search; >>>>>>>> specialinvoke r0.()>(); >>>>>>>> return; >>>>>>>> } >>>>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>>>> { >>>>>>>> Search r0; >>>>>>>> Search r2; >>>>>>>> java.lang.String[] r1; >>>>>>>> r0 := @this: Search; >>>>>>>> r1 := @parameter0: java.lang.String[]; >>>>>>>> r2 = new Search; >>>>>>>> >>>>>>>> entermonitor r2; >>>>>>>> entermonitor r0; >>>>>>>> exitmonitor r2; >>>>>>>> exitmonitor r0; >>>>>>>> return; >>>>>>>> } >>>>>>>> } >>>>>> Verification was covered above. >>>>>> >>>>>>>> (2) Test the next program on HotSpot and J9, and both do not >>>>>>>> report any errors. However, I guess the order in the program >>>>>>>> (entermonitor r2; => entermonitor r0; => exitmonitor r2; => >>>>>>>> exitmonitor r0;) violates the situation of "structured locking" >>>>>>>> (Structured locking is the situation when, during a method >>>>>>>> invocation, every exit on a given monitor matches a preceding >>>>>>>> entry on that monitor, see the specification >>>>>>>> >>>>>>>> https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-2.html#jvms-2. >>>>>>>> >>>>>>>> 11.10) >>>>>>>> ? >>>>>> No it doesn't violate structured locking as the number of enters >>>>>> and exits match, and there is always an enter before an exit. >>>>>> >>>>>>>> Actually, the words (every exit on a given monitor matches a >>>>>>>> preceding entry on that monitor) are not quite clear as for me. >>>>>>>> Otherwise the first rule (The number of monitor entries >>>>>>>> performed by T on M during a method invocation must equal the >>>>>>>> number of monitor exits performed by T on M during the method >>>>>>>> invocation whether the method invocation completes normally or >>>>>>>> abruptly.) is sufficient. >>>>>> The number of enters and exits must not only match/balance, but >>>>>> there must be an enter before a corresponding exit. >>>>>> >>>>>>>> public class Search extends java.lang.Object { >>>>>>>> >>>>>>>> public void () >>>>>>>> { >>>>>>>> Search r0; >>>>>>>> r0 := @this: Search; >>>>>>>> specialinvoke r0.()>(); >>>>>>>> return; >>>>>>>> } >>>>>>>> >>>>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>>>> { >>>>>>>> Search r0; >>>>>>>> Search r2; >>>>>>>> java.lang.String[] r1; >>>>>>>> r0 := @this: Search; >>>>>>>> r1 := @parameter0: java.lang.String[]; >>>>>>>> r2 = new Search; >>>>>>>> specialinvoke r2.()>(); >>>>>>>> entermonitor r2; >>>>>>>> entermonitor r0; >>>>>>>> exitmonitor r2; >>>>>>>> exitmonitor r0; >>>>>>>> return; >>>>>>>> } >>>>>>>> } >>>>>>>> >>>>>>>> (3) The next program enters monitor in and exits it in main(). >>>>>>>> HotSpot throws a runtime exception, while J9 does not. Should >>>>>>>> this program be rejected by the verifiers? >>>>>> No this does not violate any verification rules. The runtime >>>>>> behaviour depends on whether structured locking is enforced or >>>>>> not.(Even in hotspot there can be differences between interpreted and jitted code). >>>>>> >>>>>> Hope that helps clarify things. >>>>>> >>>>>> David >>>>>> ----- >>>>>> >>>>>>>> public class Search extends java.lang.Object { >>>>>>>> >>>>>>>> public void () >>>>>>>> { >>>>>>>> Search r0; >>>>>>>> r0 := @this: Search; >>>>>>>> specialinvoke r0.()>(); >>>>>>>> entermonitor r0; >>>>>>>> return; >>>>>>>> } >>>>>>>> >>>>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>>>> { >>>>>>>> Search r0; >>>>>>>> Search r2; >>>>>>>> java.lang.S From ioi.lam at oracle.com Thu May 18 17:42:03 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 18 May 2017 10:42:03 -0700 Subject: RFR[10]: 8180614: Skip range and constraint checks on non-existent flags In-Reply-To: <09110ab8-9538-d8c2-08ab-f8cb145c4595@oracle.com> References: <09110ab8-9538-d8c2-08ab-f8cb145c4595@oracle.com> Message-ID: <591DDCEB.5020800@oracle.com> Hi Claes, This looks good to me. As a further optimization, I wonder if we can get rid of all the calls CommandLineFlagRangeList::check_ranges() -> Flag::find_flag() This can be done by changing void emit_range_intx(const char* name, int min, intx max) { CommandLineFlagRangeList::add(new CommandLineFlagRange_int(name, min, max)); } => void emit_range_intx(const char* name, const intx* ptr, int min, intx max) { CommandLineFlagRangeList::add(new CommandLineFlagRange_int(name, ptr, min, max)); } class CommandLineFlagRange_intx { const int* _ptr; .... - Flag::Error check_int(int value, bool verbose = true) { + Flag::Error check_int(bool verbose = true) { + int value = *_ptr ...... } Then, the main loop can look like this: for (int i=0; iname(); - Flag* flag = Flag::find_flag(name, strlen(name), true, true); - // We must check for NULL here as lp64_product flags on 32 bit architecture - // can generate range check (despite that they are declared as constants), - // but they will not be returned by Flag::find_flag() - if (flag != NULL) { - if (flag->is_int()) { - int value = flag->get_int(); - if (range->check_int(value, true) != Flag::SUCCESS) status = false; + switch(range->type()) { + case CommandLineFlagRange::INT + if (range->check_int(true) != Flag::SUCCESS) status = false; Now, we will do more range checks than before, including flags that won't be returned by Flag::find_flag, like these guys: Flag* Flag::find_flag(const char* name, size_t length, bool allow_locked, bool return_flag) { for (Flag* current = &flagTable[0]; current->_name != NULL; current++) { if (str_equal(current->_name, current->get_name_length(), name, length)) { ... if (!(current->is_unlocked() || current->is_unlocker())) { if (!allow_locked) { // disable use of locked flags, e.g. diagnostic, experimental, // commercial... until they are explicitly unlocked HERE>>>> return NULL; But I think this is better, as we can even find errors in the ergo code that might have programmatically modified a diagnostic flag, for example. Or, we can even go one step further, add the ranges into the Flag::flags table directly. Then, we just need to walk this table sequentially. We can get rid of all the malloc inside CommandLineFlagRangeList::init() <-- how many cycles are spent here? => try building commandLineFlagRangeList.o with --save-temps and look at the file commandLineFlagRangeList.ii :-) And, we can sort Flag::flags alphabetically (by generating this table using a C program, which includes globals.hpp, during build time). That way Flag::find_flag can be searched on O(log n) time. What do you think? Thanks - Ioi On 5/18/17 9:35 AM, Claes Redestad wrote: > Hi, > > analyzing the remaining overhead in command line flag > checking after JDK-8178991, I found that we're taking a > runtime cost scanning for flags that doesn't exist in the > build, e.g., develop flags in a product build. > > Since such flags are simply emitted as constants, they > can't[1] be in violation of the constraints, thus I think > we can safely skip such checks altogether. > > This removes ~500k instructions from being executed for > no reason during VM startup on a 64-bit JVM. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8180614 > Webrev: http://cr.openjdk.java.net/~redestad/8180614/hotspot.00/ > > Thanks! > > /Claes > > [1] Unless the default value is invalid, which is a potential > programmer error that would be caught in debug builds > anyhow. From mikhailo.seledtsov at oracle.com Fri May 19 00:30:06 2017 From: mikhailo.seledtsov at oracle.com (Mikhailo Seledtsov) Date: Thu, 18 May 2017 17:30:06 -0700 Subject: RFR(S): 8180631: [TESTBUG] CDS tests should use CDSTestUtils.executeAndLog whenever spawning sub processes Message-ID: <591E3C8E.8060305@oracle.com> Please review this simple change that uses CDSTestUtils.executeAndLog() for starting child processes in CDS tests, in places that did not use this method before. In short, this change brings benefits of recording the output of child processes thus aiding in troubleshooting of failures. For more details see bug description. JBS: https://bugs.openjdk.java.net/browse/JDK-8180631/ Webrev: http://cr.openjdk.java.net/~mseledtsov/8180631.01/ Testing: CDS tests on Linux-x64 (local) - PASS CDS tests via automated multi-platform test system - in Progress Thank you, Misha From ioi.lam at oracle.com Fri May 19 04:50:07 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 18 May 2017 21:50:07 -0700 Subject: RFR(S): 8180631: [TESTBUG] CDS tests should use CDSTestUtils.executeAndLog whenever spawning sub processes In-Reply-To: <591E3C8E.8060305@oracle.com> References: <591E3C8E.8060305@oracle.com> Message-ID: <591E797F.5080703@oracle.com> Hi Misha, This looks good! Thanks - Ioi On 5/18/17 5:30 PM, Mikhailo Seledtsov wrote: > Please review this simple change that uses > CDSTestUtils.executeAndLog() for starting child > processes in CDS tests, in places that did not use this method before. > In short, this change brings > benefits of recording the output of child processes thus aiding in > troubleshooting of failures. > For more details see bug description. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8180631/ > Webrev: http://cr.openjdk.java.net/~mseledtsov/8180631.01/ > Testing: > CDS tests on Linux-x64 (local) - PASS > CDS tests via automated multi-platform test system - in Progress > > Thank you, > Misha From david.holmes at oracle.com Fri May 19 07:32:42 2017 From: david.holmes at oracle.com (David Holmes) Date: Fri, 19 May 2017 17:32:42 +1000 Subject: RFR: JDK-8180599: Possibly miss to iterate monitors on thread exit In-Reply-To: References: Message-ID: Hi Roman, On 18/05/2017 10:16 PM, Roman Kennke wrote: > As David pointed out here: > > http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2017-May/023468.html > > The fix to JDK-8180175: ObjectSynchronizer only needs to iterate in-use > monitors introduced a bug where we could potentially miss to iterate > oops in monitors of exiting threads. The proposed fix is as Robbin Ehn > suggested to move the call to omFlush() before we remove the thread from > _thread_list: > > http://cr.openjdk.java.net/~rkennke/8180599/webrev.00/ > > > This solves the problem because there's no window where monitors would > not appear in any list. > > I tested a number of programs and jtreg hotspot_gc tests and it doesn't > show any ill effects. > > Ok? I think this is okay as an approach. There is one issue I can see this will expose and that is that omFlush doesn't zero omFreeCount. With the current change we may go to a safepoint when acquiring the Threads_lock after calling omFlush and that could lead to monitor deflating and (if enabled) a call to verifyInUse, where we would fail this assert: assert(free_tally == Self->omFreeCount, "free count off"); but the fix is trivial: just zero omFreeCount the same way we already zero omInUseCount. And we should also have a similar assert that tally==omFreeCount. I also verified that there are no thread inspection mechanisms that might be surprised if they find a thread with its monitor lists nulled out. (Again due to a safepoint being taken immediately after omFlush.) The comment: // Reclaim the objectmonitors from the omFreeList of the moribund thread. was already inaccurate since in-use lists were added, and should be updated to reflect the transfer of free or in-use monitors to the global lists. Also the comment block before ObjectSynchronizer::omFlush in synchronizer.cpp needs to be updated to reflect this change. Thanks, David > Roman > From rkennke at redhat.com Fri May 19 10:26:28 2017 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 19 May 2017 12:26:28 +0200 Subject: RFR: JDK-8180599: Possibly miss to iterate monitors on thread exit In-Reply-To: References: Message-ID: Am 19.05.2017 um 09:32 schrieb David Holmes: > Hi Roman, > > On 18/05/2017 10:16 PM, Roman Kennke wrote: >> As David pointed out here: >> >> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2017-May/023468.html >> >> >> The fix to JDK-8180175: ObjectSynchronizer only needs to iterate in-use >> monitors introduced a bug where we could potentially miss to iterate >> oops in monitors of exiting threads. The proposed fix is as Robbin Ehn >> suggested to move the call to omFlush() before we remove the thread from >> _thread_list: >> >> http://cr.openjdk.java.net/~rkennke/8180599/webrev.00/ >> >> >> This solves the problem because there's no window where monitors would >> not appear in any list. >> >> I tested a number of programs and jtreg hotspot_gc tests and it doesn't >> show any ill effects. >> >> Ok? > > I think this is okay as an approach. > > There is one issue I can see this will expose and that is that omFlush > doesn't zero omFreeCount. With the current change we may go to a > safepoint when acquiring the Threads_lock after calling omFlush and > that could lead to monitor deflating and (if enabled) a call to > verifyInUse, where we would fail this assert: > > assert(free_tally == Self->omFreeCount, "free count off"); > > but the fix is trivial: just zero omFreeCount the same way we already > zero omInUseCount. And we should also have a similar assert that > tally==omFreeCount. > > I also verified that there are no thread inspection mechanisms that > might be surprised if they find a thread with its monitor lists nulled > out. (Again due to a safepoint being taken immediately after omFlush.) > > The comment: > > // Reclaim the objectmonitors from the omFreeList of the moribund thread. > > was already inaccurate since in-use lists were added, and should be > updated to reflect the transfer of free or in-use monitors to the > global lists. > > Also the comment block before ObjectSynchronizer::omFlush in > synchronizer.cpp needs to be updated to reflect this change. Hi David, Thanks for reviewing! http://cr.openjdk.java.net/~rkennke/8180599/webrev.01/ ? Does this require a 2nd reviewer? Roman From claes.redestad at oracle.com Fri May 19 11:40:32 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 19 May 2017 13:40:32 +0200 Subject: RFR[10]: 8180614: Skip range and constraint checks on non-existent flags In-Reply-To: <591DDCEB.5020800@oracle.com> References: <09110ab8-9538-d8c2-08ab-f8cb145c4595@oracle.com> <591DDCEB.5020800@oracle.com> Message-ID: <80b917d1-3396-1c28-fd28-b0186dd84018@oracle.com> Hi Ioi, On 05/18/2017 07:42 PM, Ioi Lam wrote: > Hi Claes, > > > This looks good to me. thanks! > > As a further optimization, I wonder if we can get rid of all the calls > > CommandLineFlagRangeList::check_ranges() -> Flag::find_flag() Yes, I was considering approaches to improve this even further as a follow up, but it turns out it was easier than I had envisioned with some helpful pointers... ;-) > > Now, we will do more range checks than before, including flags that > won't be returned by Flag::find_flag, like these guys: > > Flag* Flag::find_flag(const char* name, size_t length, bool > allow_locked, bool return_flag) { > for (Flag* current = &flagTable[0]; current->_name != NULL; > current++) { > if (str_equal(current->_name, current->get_name_length(), name, > length)) { > ... > if (!(current->is_unlocked() || current->is_unlocker())) { > if (!allow_locked) { > // disable use of locked flags, e.g. diagnostic, experimental, > // commercial... until they are explicitly unlocked > HERE>>>> return NULL; > > > But I think this is better, as we can even find errors in the ergo > code that might have programmatically modified a diagnostic flag, for > example. Yes, this definitely seems like an improvement. Updated webrev: http://cr.openjdk.java.net/~redestad/8180614/hotspot.01/ We discussed offline if all the constraint and range classes could be turned into templates, and while it superficially seems easy there are some uses of constraint methods from globals.cpp that needs special care, so I decided to defer that cleanup. > > Or, we can even go one step further, add the ranges into the > Flag::flags table directly. Then, we just need to walk this table > sequentially. We can get rid of all the malloc inside > CommandLineFlagRangeList::init() <-- how many cycles are spent here? > > => try building commandLineFlagRangeList.o with --save-temps and look > at the file commandLineFlagRangeList.ii :-) > > > And, we can sort Flag::flags alphabetically (by generating this table > using a C program, which includes globals.hpp, during build time). > That way Flag::find_flag can be searched on O(log n) time. > > What do you think? Might be a lot of effort for no measurable startup gain (since the above solution gets rid of calls to most calls to find_flag), and other uses of find_flag doesn't seem very performance critical. Cleaning up the magic numbers in commandLineFlag*List.cpp would be nice, though... Thanks! /Claes > > > Thanks > - Ioi > > > On 5/18/17 9:35 AM, Claes Redestad wrote: >> Hi, >> >> analyzing the remaining overhead in command line flag >> checking after JDK-8178991, I found that we're taking a >> runtime cost scanning for flags that doesn't exist in the >> build, e.g., develop flags in a product build. >> >> Since such flags are simply emitted as constants, they >> can't[1] be in violation of the constraints, thus I think >> we can safely skip such checks altogether. >> >> This removes ~500k instructions from being executed for >> no reason during VM startup on a 64-bit JVM. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8180614 >> Webrev: http://cr.openjdk.java.net/~redestad/8180614/hotspot.00/ >> >> Thanks! >> >> /Claes >> >> [1] Unless the default value is invalid, which is a potential >> programmer error that would be caught in debug builds >> anyhow. > From robbin.ehn at oracle.com Fri May 19 13:04:29 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Fri, 19 May 2017 15:04:29 +0200 Subject: RFR: JDK-8180599: Possibly miss to iterate monitors on thread exit In-Reply-To: References: Message-ID: <221d8619-87cb-a218-02b2-4d1ab222ecee@oracle.com> Hi, On 05/19/2017 12:26 PM, Roman Kennke wrote: > > http://cr.openjdk.java.net/~rkennke/8180599/webrev.01/ > > > ? > > Does this require a 2nd reviewer? One Reviewer + one reviewer at minimum. Looks good! /Robbin > > Roman > From ioi.lam at oracle.com Fri May 19 14:26:28 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 19 May 2017 07:26:28 -0700 Subject: RFR[10]: 8180614: Skip range and constraint checks on non-existent flags In-Reply-To: <80b917d1-3396-1c28-fd28-b0186dd84018@oracle.com> References: <09110ab8-9538-d8c2-08ab-f8cb145c4595@oracle.com> <591DDCEB.5020800@oracle.com> <80b917d1-3396-1c28-fd28-b0186dd84018@oracle.com> Message-ID: <591F0094.9070401@oracle.com> Hi Claes, The new webrev looks good. Thanks! - Ioi On 5/19/17 4:40 AM, Claes Redestad wrote: > Hi Ioi, > > On 05/18/2017 07:42 PM, Ioi Lam wrote: >> Hi Claes, >> >> >> This looks good to me. > > thanks! > >> >> As a further optimization, I wonder if we can get rid of all the calls >> >> CommandLineFlagRangeList::check_ranges() -> Flag::find_flag() > > Yes, I was considering approaches to improve this even further as a > follow up, but it turns out it was easier than I had envisioned with > some helpful pointers... ;-) > >> >> Now, we will do more range checks than before, including flags that >> won't be returned by Flag::find_flag, like these guys: >> >> Flag* Flag::find_flag(const char* name, size_t length, bool >> allow_locked, bool return_flag) { >> for (Flag* current = &flagTable[0]; current->_name != NULL; >> current++) { >> if (str_equal(current->_name, current->get_name_length(), name, >> length)) { >> ... >> if (!(current->is_unlocked() || current->is_unlocker())) { >> if (!allow_locked) { >> // disable use of locked flags, e.g. diagnostic, experimental, >> // commercial... until they are explicitly unlocked >> HERE>>>> return NULL; >> >> >> But I think this is better, as we can even find errors in the ergo >> code that might have programmatically modified a diagnostic flag, for >> example. > > Yes, this definitely seems like an improvement. > > Updated webrev: http://cr.openjdk.java.net/~redestad/8180614/hotspot.01/ > > We discussed offline if all the constraint and range classes could be > turned into templates, and while it superficially seems easy there are > some uses of constraint methods from globals.cpp that needs special > care, so I decided to defer that cleanup. > > >> >> Or, we can even go one step further, add the ranges into the >> Flag::flags table directly. Then, we just need to walk this table >> sequentially. We can get rid of all the malloc inside >> CommandLineFlagRangeList::init() <-- how many cycles are spent here? >> >> => try building commandLineFlagRangeList.o with --save-temps and look >> at the file commandLineFlagRangeList.ii :-) >> >> >> And, we can sort Flag::flags alphabetically (by generating this table >> using a C program, which includes globals.hpp, during build time). >> That way Flag::find_flag can be searched on O(log n) time. >> >> What do you think? > > Might be a lot of effort for no measurable startup gain (since the > above solution gets rid of calls to most calls to find_flag), and > other uses of find_flag doesn't seem very performance critical. > > Cleaning up the magic numbers in commandLineFlag*List.cpp would be > nice, though... > > Thanks! > > /Claes > >> >> >> Thanks >> - Ioi >> >> >> On 5/18/17 9:35 AM, Claes Redestad wrote: >>> Hi, >>> >>> analyzing the remaining overhead in command line flag >>> checking after JDK-8178991, I found that we're taking a >>> runtime cost scanning for flags that doesn't exist in the >>> build, e.g., develop flags in a product build. >>> >>> Since such flags are simply emitted as constants, they >>> can't[1] be in violation of the constraints, thus I think >>> we can safely skip such checks altogether. >>> >>> This removes ~500k instructions from being executed for >>> no reason during VM startup on a 64-bit JVM. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8180614 >>> Webrev: http://cr.openjdk.java.net/~redestad/8180614/hotspot.00/ >>> >>> Thanks! >>> >>> /Claes >>> >>> [1] Unless the default value is invalid, which is a potential >>> programmer error that would be caught in debug builds >>> anyhow. >> > From claes.redestad at oracle.com Fri May 19 14:44:53 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 19 May 2017 16:44:53 +0200 Subject: RFR[10]: 8180614: Skip range and constraint checks on non-existent flags In-Reply-To: <591F0094.9070401@oracle.com> References: <09110ab8-9538-d8c2-08ab-f8cb145c4595@oracle.com> <591DDCEB.5020800@oracle.com> <80b917d1-3396-1c28-fd28-b0186dd84018@oracle.com> <591F0094.9070401@oracle.com> Message-ID: <25a69081-85a1-758f-9084-a821dc652491@oracle.com> Hi Ioi, On 2017-05-19 16:26, Ioi Lam wrote: > Hi Claes, > > The new webrev looks good. Thanks! Thanks! For reference this gets down the cost of checking constraints and ranges from ~3.5M instructions (~1ms) to less than 200k (<0.1ms). /Claes > > - Ioi > > On 5/19/17 4:40 AM, Claes Redestad wrote: >> Hi Ioi, >> >> On 05/18/2017 07:42 PM, Ioi Lam wrote: >>> Hi Claes, >>> >>> >>> This looks good to me. >> >> thanks! >> >>> >>> As a further optimization, I wonder if we can get rid of all the calls >>> >>> CommandLineFlagRangeList::check_ranges() -> Flag::find_flag() >> >> Yes, I was considering approaches to improve this even further as a >> follow up, but it turns out it was easier than I had envisioned with >> some helpful pointers... ;-) >> >>> >>> Now, we will do more range checks than before, including flags that >>> won't be returned by Flag::find_flag, like these guys: >>> >>> Flag* Flag::find_flag(const char* name, size_t length, bool >>> allow_locked, bool return_flag) { >>> for (Flag* current = &flagTable[0]; current->_name != NULL; >>> current++) { >>> if (str_equal(current->_name, current->get_name_length(), name, >>> length)) { >>> ... >>> if (!(current->is_unlocked() || current->is_unlocker())) { >>> if (!allow_locked) { >>> // disable use of locked flags, e.g. diagnostic, >>> experimental, >>> // commercial... until they are explicitly unlocked >>> HERE>>>> return NULL; >>> >>> >>> But I think this is better, as we can even find errors in the ergo >>> code that might have programmatically modified a diagnostic flag, >>> for example. >> >> Yes, this definitely seems like an improvement. >> >> Updated webrev: http://cr.openjdk.java.net/~redestad/8180614/hotspot.01/ >> >> We discussed offline if all the constraint and range classes could be >> turned into templates, and while it superficially seems easy there >> are some uses of constraint methods from globals.cpp that needs >> special care, so I decided to defer that cleanup. >> >> >>> >>> Or, we can even go one step further, add the ranges into the >>> Flag::flags table directly. Then, we just need to walk this table >>> sequentially. We can get rid of all the malloc inside >>> CommandLineFlagRangeList::init() <-- how many cycles are spent here? >>> >>> => try building commandLineFlagRangeList.o with --save-temps and >>> look at the file commandLineFlagRangeList.ii :-) >>> >>> >>> And, we can sort Flag::flags alphabetically (by generating this >>> table using a C program, which includes globals.hpp, during build >>> time). That way Flag::find_flag can be searched on O(log n) time. >>> >>> What do you think? >> >> Might be a lot of effort for no measurable startup gain (since the >> above solution gets rid of calls to most calls to find_flag), and >> other uses of find_flag doesn't seem very performance critical. >> >> Cleaning up the magic numbers in commandLineFlag*List.cpp would be >> nice, though... >> >> Thanks! >> >> /Claes >> >>> >>> >>> Thanks >>> - Ioi >>> >>> >>> On 5/18/17 9:35 AM, Claes Redestad wrote: >>>> Hi, >>>> >>>> analyzing the remaining overhead in command line flag >>>> checking after JDK-8178991, I found that we're taking a >>>> runtime cost scanning for flags that doesn't exist in the >>>> build, e.g., develop flags in a product build. >>>> >>>> Since such flags are simply emitted as constants, they >>>> can't[1] be in violation of the constraints, thus I think >>>> we can safely skip such checks altogether. >>>> >>>> This removes ~500k instructions from being executed for >>>> no reason during VM startup on a 64-bit JVM. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8180614 >>>> Webrev: http://cr.openjdk.java.net/~redestad/8180614/hotspot.00/ >>>> >>>> Thanks! >>>> >>>> /Claes >>>> >>>> [1] Unless the default value is invalid, which is a potential >>>> programmer error that would be caught in debug builds >>>> anyhow. >>> >> > From gerard.ziemski at oracle.com Fri May 19 16:07:10 2017 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Fri, 19 May 2017 11:07:10 -0500 Subject: RFR[10]: 8180614: Skip range and constraint checks on non-existent flags In-Reply-To: <25a69081-85a1-758f-9084-a821dc652491@oracle.com> References: <09110ab8-9538-d8c2-08ab-f8cb145c4595@oracle.com> <591DDCEB.5020800@oracle.com> <80b917d1-3396-1c28-fd28-b0186dd84018@oracle.com> <591F0094.9070401@oracle.com> <25a69081-85a1-758f-9084-a821dc652491@oracle.com> Message-ID: Thank you Claes and Ioi for 2 very nice optimizations! Looks good! ps. Claes, did you run "hotspot/test/runtime/CommandLine/OptionsValidation? with this change? cheers > On May 19, 2017, at 9:44 AM, Claes Redestad wrote: > > Hi Ioi, > > On 2017-05-19 16:26, Ioi Lam wrote: >> Hi Claes, >> >> The new webrev looks good. Thanks! > > Thanks! > > For reference this gets down the cost of checking constraints and ranges from ~3.5M instructions (~1ms) to less than 200k (<0.1ms). > > /Claes > >> >> - Ioi >> >> On 5/19/17 4:40 AM, Claes Redestad wrote: >>> Hi Ioi, >>> >>> On 05/18/2017 07:42 PM, Ioi Lam wrote: >>>> Hi Claes, >>>> >>>> >>>> This looks good to me. >>> >>> thanks! >>> >>>> >>>> As a further optimization, I wonder if we can get rid of all the calls >>>> >>>> CommandLineFlagRangeList::check_ranges() -> Flag::find_flag() >>> >>> Yes, I was considering approaches to improve this even further as a follow up, but it turns out it was easier than I had envisioned with some helpful pointers... ;-) >>> >>>> >>>> Now, we will do more range checks than before, including flags that won't be returned by Flag::find_flag, like these guys: >>>> >>>> Flag* Flag::find_flag(const char* name, size_t length, bool allow_locked, bool return_flag) { >>>> for (Flag* current = &flagTable[0]; current->_name != NULL; current++) { >>>> if (str_equal(current->_name, current->get_name_length(), name, length)) { >>>> ... >>>> if (!(current->is_unlocked() || current->is_unlocker())) { >>>> if (!allow_locked) { >>>> // disable use of locked flags, e.g. diagnostic, experimental, >>>> // commercial... until they are explicitly unlocked >>>> HERE>>>> return NULL; >>>> >>>> >>>> But I think this is better, as we can even find errors in the ergo code that might have programmatically modified a diagnostic flag, for example. >>> >>> Yes, this definitely seems like an improvement. >>> >>> Updated webrev: http://cr.openjdk.java.net/~redestad/8180614/hotspot.01/ >>> >>> We discussed offline if all the constraint and range classes could be turned into templates, and while it superficially seems easy there are some uses of constraint methods from globals.cpp that needs special care, so I decided to defer that cleanup. >>> >>> >>>> >>>> Or, we can even go one step further, add the ranges into the Flag::flags table directly. Then, we just need to walk this table sequentially. We can get rid of all the malloc inside CommandLineFlagRangeList::init() <-- how many cycles are spent here? >>>> >>>> => try building commandLineFlagRangeList.o with --save-temps and look at the file commandLineFlagRangeList.ii :-) >>>> >>>> >>>> And, we can sort Flag::flags alphabetically (by generating this table using a C program, which includes globals.hpp, during build time). That way Flag::find_flag can be searched on O(log n) time. >>>> >>>> What do you think? >>> >>> Might be a lot of effort for no measurable startup gain (since the above solution gets rid of calls to most calls to find_flag), and other uses of find_flag doesn't seem very performance critical. >>> >>> Cleaning up the magic numbers in commandLineFlag*List.cpp would be nice, though... >>> >>> Thanks! >>> >>> /Claes >>> >>>> >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>> On 5/18/17 9:35 AM, Claes Redestad wrote: >>>>> Hi, >>>>> >>>>> analyzing the remaining overhead in command line flag >>>>> checking after JDK-8178991, I found that we're taking a >>>>> runtime cost scanning for flags that doesn't exist in the >>>>> build, e.g., develop flags in a product build. >>>>> >>>>> Since such flags are simply emitted as constants, they >>>>> can't[1] be in violation of the constraints, thus I think >>>>> we can safely skip such checks altogether. >>>>> >>>>> This removes ~500k instructions from being executed for >>>>> no reason during VM startup on a 64-bit JVM. >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8180614 >>>>> Webrev: http://cr.openjdk.java.net/~redestad/8180614/hotspot.00/ >>>>> >>>>> Thanks! >>>>> >>>>> /Claes >>>>> >>>>> [1] Unless the default value is invalid, which is a potential >>>>> programmer error that would be caught in debug builds >>>>> anyhow. >>>> >>> >> > From chenyt at cs.sjtu.edu.cn Fri May 19 21:10:47 2017 From: chenyt at cs.sjtu.edu.cn (=?utf-8?B?6ZmI6Zuo5Lqt?=) Date: Fri, 19 May 2017 14:10:47 -0700 Subject: =?utf-8?Q?=E7=AD=94=E5=A4=8D:_=E7=AD=94=E5=A4=8D:_=E7=AD=94=E5=A4=8D:_HotS?= =?utf-8?Q?pot_and_IBM's_J9_behave_quite_di?= =?utf-8?Q?fferently_when_processing_monito?= =?utf-8?Q?renters_and_monitorexits?= In-Reply-To: References: <005101d2cf53$c8c16b40$5a4441c0$@cs.sjtu.edu.cn> <656dcb18-e65a-4c48-a6bc-cf00fe021af5@oracle.com> <005201d2cf5a$aa2e8f30$fe8bad90$@cs.sjtu.edu.cn> <5a6a2de2-eb43-985d-bfc7-18c3fa7ab0ee@oracle.com> <000401d2cf8d$c8352470$589f6d50$@cs.sjtu.edu.cn> <18003d32-08ad-4891-cf2a-4ba0d71a2bb0@oracle.com> <9a93ed9268233cf5847156fc7ceef5d5@cs.sjtu.edu.cn> Message-ID: <000001d2d0e4$64446310$2ccd2930$@cs.sjtu.edu.cn> Hi, David, Hi, I saw the discussions (about the verification) at https://bugs.openjdk.java.net/browse/JDK-8180581. If the specification needs to be clarified, please also include a similar example. Similarly, an object is not initialized before the " invokeinterface" instruction. At this time, HotSpot's verifier reports a verifyerror, while J9 normally runs the code. A verifier takes a def-use analysis to verify the bytecode, while when an object is really ready to use is not very clear. public void loadData() throws java.lang.Exception; descriptor: ()V flags: ACC_PUBLIC Code: stack=1, locals=1, args_size=1 0: new #19 // class org/apache/derbyTesting/system/oe/load/ThreadInsert 3: invokeinterface #24, 1 // InterfaceMethod org/apache/derbyTesting/system/oe/client/Load.populateAllTables:()V 8: return Exceptions: throws java.lang.Exception Regards, Yuting ----------------------- public class org.dacapo.h2.TPCC minor version: 0 major version: 52 flags: ACC_PUBLIC, ACC_SUPER Constant pool: #1 = Utf8 org/dacapo/h2/TPCC #2 = Class #1 // org/dacapo/h2/TPCC #3 = Utf8 java/lang/Object #4 = Class #3 // java/lang/Object #5 = Utf8 #6 = Utf8 ()V #7 = Utf8 #8 = Utf8 (Lorg/dacapo/parser/Config;Ljava/io/File;ZZ)V #9 = Utf8 java/lang/Exception #10 = Class #9 // java/lang/Exception #11 = NameAndType #7:#6 // "":()V #12 = Methodref #4.#11 // java/lang/Object."":()V #13 = Utf8 loadData #14 = NameAndType #13:#6 // loadData:()V #15 = Methodref #2.#14 // org/dacapo/h2/TPCC.loadData:()V #16 = Utf8 iteration #17 = Utf8 (Ljava/lang/String;)V #18 = Utf8 org/apache/derbyTesting/system/oe/load/ThreadInsert #19 = Class #18 // org/apache/derbyTesting/system/oe/load/ThreadInsert #20 = Utf8 org/apache/derbyTesting/system/oe/client/Load #21 = Class #20 // org/apache/derbyTesting/system/oe/client/Load #22 = Utf8 populateAllTables #23 = NameAndType #22:#6 // populateAllTables:()V #24 = InterfaceMethodref #21.#23 // org/apache/derbyTesting/system/oe/client/Load.populateAllTables:()V #25 = Utf8 make #26 = Utf8 (Lorg/dacapo/parser/Config;Ljava/io/File;Ljava/lang/Boolean;Ljava/lang/Boolean;)Lorg/dacapo/h2/TPCC; #27 = NameAndType #7:#8 // "":(Lorg/dacapo/parser/Config;Ljava/io/File;ZZ)V #28 = Methodref #2.#27 // org/dacapo/h2/TPCC."":(Lorg/dacapo/parser/Config;Ljava/io/File;ZZ)V #29 = Utf8 postIteration #30 = Utf8 preIteration #31 = Utf8 Code #32 = Utf8 Exceptions { static {}; descriptor: ()V flags: ACC_STATIC Code: stack=0, locals=0, args_size=0 0: return public org.dacapo.h2.TPCC(org.dacapo.parser.Config, java.io.File, boolean, boolean) throws java.lang.Exception; descriptor: (Lorg/dacapo/parser/Config;Ljava/io/File;ZZ)V flags: ACC_PUBLIC Code: stack=1, locals=5, args_size=5 0: aload_0 1: invokespecial #12 // Method java/lang/Object."":()V 4: aload_0 5: invokespecial #15 // Method loadData:()V 8: return Exceptions: throws java.lang.Exception public void iteration(java.lang.String) throws java.lang.Exception; descriptor: (Ljava/lang/String;)V flags: ACC_PUBLIC Code: stack=0, locals=2, args_size=2 0: return Exceptions: throws java.lang.Exception public void loadData() throws java.lang.Exception; descriptor: ()V flags: ACC_PUBLIC Code: stack=1, locals=1, args_size=1 0: new #19 // class org/apache/derbyTesting/system/oe/load/ThreadInsert 3: invokeinterface #24, 1 // InterfaceMethod org/apache/derbyTesting/system/oe/client/Load.populateAllTables:()V 8: return Exceptions: throws java.lang.Exception public static org.dacapo.h2.TPCC make(org.dacapo.parser.Config, java.io.File, java.lang.Boolean, java.lang.Boolean) throws java.lang.Exception; descriptor: (Lorg/dacapo/parser/Config;Ljava/io/File;Ljava/lang/Boolean;Ljava/lang/Boolean;)Lorg/dacapo/h2/TPCC; flags: ACC_PUBLIC, ACC_STATIC Code: stack=6, locals=4, args_size=4 0: new #2 // class org/dacapo/h2/TPCC 3: dup 4: aload_0 5: aload_1 6: iconst_1 7: iconst_1 8: invokespecial #28 // Method "":(Lorg/dacapo/parser/Config;Ljava/io/File;ZZ)V 11: areturn Exceptions: throws java.lang.Exception public void postIteration(java.lang.String) throws java.lang.Exception; descriptor: (Ljava/lang/String;)V flags: ACC_PUBLIC Code: stack=0, locals=2, args_size=2 0: return Exceptions: throws java.lang.Exception public void preIteration(java.lang.String) throws java.lang.Exception; descriptor: (Ljava/lang/String;)V flags: ACC_PUBLIC Code: stack=0, locals=2, args_size=2 0: return Exceptions: throws java.lang.Exception } From chenyt at cs.sjtu.edu.cn Fri May 19 21:25:31 2017 From: chenyt at cs.sjtu.edu.cn (=?utf-8?B?6ZmI6Zuo5Lqt?=) Date: Fri, 19 May 2017 14:25:31 -0700 Subject: =?utf-8?Q?=E7=AD=94=E5=A4=8D:_=E7=AD=94=E5=A4=8D:_=E7=AD=94=E5=A4=8D:_HotS?= =?utf-8?Q?pot_and_IBM's_J9_behave_quite_di?= =?utf-8?Q?fferently_when_processing_monito?= =?utf-8?Q?renters_and_monitorexits?= References: <005101d2cf53$c8c16b40$5a4441c0$@cs.sjtu.edu.cn> <656dcb18-e65a-4c48-a6bc-cf00fe021af5@oracle.com> <005201d2cf5a$aa2e8f30$fe8bad90$@cs.sjtu.edu.cn> <5a6a2de2-eb43-985d-bfc7-18c3fa7ab0ee@oracle.com> <000401d2cf8d$c8352470$589f6d50$@cs.sjtu.edu.cn> <18003d32-08ad-4891-cf2a-4ba0d71a2bb0@oracle.com> <9a93ed9268233cf5847156fc7ceef5d5@cs.sjtu.edu.cn> Message-ID: <000101d2d0e6$735dfa80$5a19ef80$@cs.sjtu.edu.cn> HotSpot's verifyerror is given here. I read the specification (https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.10.1.9.invokeinterface) and did not get the hints. Not quite sure whether isAssignable should be enforced for verifying the invokeinterface instruction. Location: org/dacapo/h2/TPCC.loadData()V @3: invokeinterface Reason: Type uninitialized 0 (current frame, stack[0]) is not assignable to 'org/apache/derbyTesting/system/oe/client/Load' Current Frame: bci: @3 flags: { } locals: { 'org/dacapo/h2/TPCC' } stack: { uninitialized 0 } Bytecode: 0x0000000: bb00 13b9 0018 0100 b1 -----????----- ???: ??? [mailto:chenyt at cs.sjtu.edu.cn] ????: 2017?5?19? 14:11 ???: 'David Holmes' ??: 'hotspot-runtime-dev at openjdk.java.net' ??: ??: ??: ??: HotSpot and IBM's J9 behave quite differently when processing monitorenters and monitorexits Hi, David, Hi, I saw the discussions (about the verification) at https://bugs.openjdk.java.net/browse/JDK-8180581. If the specification needs to be clarified, please also include a similar example. Similarly, an object is not initialized before the " invokeinterface" instruction. At this time, HotSpot's verifier reports a verifyerror, while J9 normally runs the code. A verifier takes a def-use analysis to verify the bytecode, while when an object is really ready to use is not very clear. public void loadData() throws java.lang.Exception; descriptor: ()V flags: ACC_PUBLIC Code: stack=1, locals=1, args_size=1 0: new #19 // class org/apache/derbyTesting/system/oe/load/ThreadInsert 3: invokeinterface #24, 1 // InterfaceMethod org/apache/derbyTesting/system/oe/client/Load.populateAllTables:()V 8: return Exceptions: throws java.lang.Exception Regards, Yuting ----------------------- public class org.dacapo.h2.TPCC minor version: 0 major version: 52 flags: ACC_PUBLIC, ACC_SUPER Constant pool: #1 = Utf8 org/dacapo/h2/TPCC #2 = Class #1 // org/dacapo/h2/TPCC #3 = Utf8 java/lang/Object #4 = Class #3 // java/lang/Object #5 = Utf8 #6 = Utf8 ()V #7 = Utf8 #8 = Utf8 (Lorg/dacapo/parser/Config;Ljava/io/File;ZZ)V #9 = Utf8 java/lang/Exception #10 = Class #9 // java/lang/Exception #11 = NameAndType #7:#6 // "":()V #12 = Methodref #4.#11 // java/lang/Object."":()V #13 = Utf8 loadData #14 = NameAndType #13:#6 // loadData:()V #15 = Methodref #2.#14 // org/dacapo/h2/TPCC.loadData:()V #16 = Utf8 iteration #17 = Utf8 (Ljava/lang/String;)V #18 = Utf8 org/apache/derbyTesting/system/oe/load/ThreadInsert #19 = Class #18 // org/apache/derbyTesting/system/oe/load/ThreadInsert #20 = Utf8 org/apache/derbyTesting/system/oe/client/Load #21 = Class #20 // org/apache/derbyTesting/system/oe/client/Load #22 = Utf8 populateAllTables #23 = NameAndType #22:#6 // populateAllTables:()V #24 = InterfaceMethodref #21.#23 // org/apache/derbyTesting/system/oe/client/Load.populateAllTables:()V #25 = Utf8 make #26 = Utf8 (Lorg/dacapo/parser/Config;Ljava/io/File;Ljava/lang/Boolean;Ljava/lang/Boolean;)Lorg/dacapo/h2/TPCC; #27 = NameAndType #7:#8 // "":(Lorg/dacapo/parser/Config;Ljava/io/File;ZZ)V #28 = Methodref #2.#27 // org/dacapo/h2/TPCC."":(Lorg/dacapo/parser/Config;Ljava/io/File;ZZ)V #29 = Utf8 postIteration #30 = Utf8 preIteration #31 = Utf8 Code #32 = Utf8 Exceptions { static {}; descriptor: ()V flags: ACC_STATIC Code: stack=0, locals=0, args_size=0 0: return public org.dacapo.h2.TPCC(org.dacapo.parser.Config, java.io.File, boolean, boolean) throws java.lang.Exception; descriptor: (Lorg/dacapo/parser/Config;Ljava/io/File;ZZ)V flags: ACC_PUBLIC Code: stack=1, locals=5, args_size=5 0: aload_0 1: invokespecial #12 // Method java/lang/Object."":()V 4: aload_0 5: invokespecial #15 // Method loadData:()V 8: return Exceptions: throws java.lang.Exception public void iteration(java.lang.String) throws java.lang.Exception; descriptor: (Ljava/lang/String;)V flags: ACC_PUBLIC Code: stack=0, locals=2, args_size=2 0: return Exceptions: throws java.lang.Exception public void loadData() throws java.lang.Exception; descriptor: ()V flags: ACC_PUBLIC Code: stack=1, locals=1, args_size=1 0: new #19 // class org/apache/derbyTesting/system/oe/load/ThreadInsert 3: invokeinterface #24, 1 // InterfaceMethod org/apache/derbyTesting/system/oe/client/Load.populateAllTables:()V 8: return Exceptions: throws java.lang.Exception public static org.dacapo.h2.TPCC make(org.dacapo.parser.Config, java.io.File, java.lang.Boolean, java.lang.Boolean) throws java.lang.Exception; descriptor: (Lorg/dacapo/parser/Config;Ljava/io/File;Ljava/lang/Boolean;Ljava/lang/Boolean;)Lorg/dacapo/h2/TPCC; flags: ACC_PUBLIC, ACC_STATIC Code: stack=6, locals=4, args_size=4 0: new #2 // class org/dacapo/h2/TPCC 3: dup 4: aload_0 5: aload_1 6: iconst_1 7: iconst_1 8: invokespecial #28 // Method "":(Lorg/dacapo/parser/Config;Ljava/io/File;ZZ)V 11: areturn Exceptions: throws java.lang.Exception public void postIteration(java.lang.String) throws java.lang.Exception; descriptor: (Ljava/lang/String;)V flags: ACC_PUBLIC Code: stack=0, locals=2, args_size=2 0: return Exceptions: throws java.lang.Exception public void preIteration(java.lang.String) throws java.lang.Exception; descriptor: (Ljava/lang/String;)V flags: ACC_PUBLIC Code: stack=0, locals=2, args_size=2 0: return Exceptions: throws java.lang.Exception } From claes.redestad at oracle.com Sun May 21 21:08:18 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Sun, 21 May 2017 23:08:18 +0200 Subject: RFR[10]: 8180614: Skip range and constraint checks on non-existent flags In-Reply-To: References: <09110ab8-9538-d8c2-08ab-f8cb145c4595@oracle.com> <591DDCEB.5020800@oracle.com> <80b917d1-3396-1c28-fd28-b0186dd84018@oracle.com> <591F0094.9070401@oracle.com> <25a69081-85a1-758f-9084-a821dc652491@oracle.com> Message-ID: Hi Gerard, On 2017-05-19 18:07, Gerard Ziemski wrote: > Thank you Claes and Ioi for 2 very nice optimizations! > > Looks good! Thanks! > > ps. Claes, did you run "hotspot/test/runtime/CommandLine/OptionsValidation? with this change? Yes - all pass. /Claes > > > cheers > >> On May 19, 2017, at 9:44 AM, Claes Redestad wrote: >> >> Hi Ioi, >> >> On 2017-05-19 16:26, Ioi Lam wrote: >>> Hi Claes, >>> >>> The new webrev looks good. Thanks! >> Thanks! >> >> For reference this gets down the cost of checking constraints and ranges from ~3.5M instructions (~1ms) to less than 200k (<0.1ms). >> >> /Claes >> >>> - Ioi >>> >>> On 5/19/17 4:40 AM, Claes Redestad wrote: >>>> Hi Ioi, >>>> >>>> On 05/18/2017 07:42 PM, Ioi Lam wrote: >>>>> Hi Claes, >>>>> >>>>> >>>>> This looks good to me. >>>> thanks! >>>> >>>>> As a further optimization, I wonder if we can get rid of all the calls >>>>> >>>>> CommandLineFlagRangeList::check_ranges() -> Flag::find_flag() >>>> Yes, I was considering approaches to improve this even further as a follow up, but it turns out it was easier than I had envisioned with some helpful pointers... ;-) >>>> >>>>> Now, we will do more range checks than before, including flags that won't be returned by Flag::find_flag, like these guys: >>>>> >>>>> Flag* Flag::find_flag(const char* name, size_t length, bool allow_locked, bool return_flag) { >>>>> for (Flag* current = &flagTable[0]; current->_name != NULL; current++) { >>>>> if (str_equal(current->_name, current->get_name_length(), name, length)) { >>>>> ... >>>>> if (!(current->is_unlocked() || current->is_unlocker())) { >>>>> if (!allow_locked) { >>>>> // disable use of locked flags, e.g. diagnostic, experimental, >>>>> // commercial... until they are explicitly unlocked >>>>> HERE>>>> return NULL; >>>>> >>>>> >>>>> But I think this is better, as we can even find errors in the ergo code that might have programmatically modified a diagnostic flag, for example. >>>> Yes, this definitely seems like an improvement. >>>> >>>> Updated webrev: http://cr.openjdk.java.net/~redestad/8180614/hotspot.01/ >>>> >>>> We discussed offline if all the constraint and range classes could be turned into templates, and while it superficially seems easy there are some uses of constraint methods from globals.cpp that needs special care, so I decided to defer that cleanup. >>>> >>>> >>>>> Or, we can even go one step further, add the ranges into the Flag::flags table directly. Then, we just need to walk this table sequentially. We can get rid of all the malloc inside CommandLineFlagRangeList::init() <-- how many cycles are spent here? >>>>> >>>>> => try building commandLineFlagRangeList.o with --save-temps and look at the file commandLineFlagRangeList.ii :-) >>>>> >>>>> >>>>> And, we can sort Flag::flags alphabetically (by generating this table using a C program, which includes globals.hpp, during build time). That way Flag::find_flag can be searched on O(log n) time. >>>>> >>>>> What do you think? >>>> Might be a lot of effort for no measurable startup gain (since the above solution gets rid of calls to most calls to find_flag), and other uses of find_flag doesn't seem very performance critical. >>>> >>>> Cleaning up the magic numbers in commandLineFlag*List.cpp would be nice, though... >>>> >>>> Thanks! >>>> >>>> /Claes >>>> >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> >>>>> On 5/18/17 9:35 AM, Claes Redestad wrote: >>>>>> Hi, >>>>>> >>>>>> analyzing the remaining overhead in command line flag >>>>>> checking after JDK-8178991, I found that we're taking a >>>>>> runtime cost scanning for flags that doesn't exist in the >>>>>> build, e.g., develop flags in a product build. >>>>>> >>>>>> Since such flags are simply emitted as constants, they >>>>>> can't[1] be in violation of the constraints, thus I think >>>>>> we can safely skip such checks altogether. >>>>>> >>>>>> This removes ~500k instructions from being executed for >>>>>> no reason during VM startup on a 64-bit JVM. >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8180614 >>>>>> Webrev: http://cr.openjdk.java.net/~redestad/8180614/hotspot.00/ >>>>>> >>>>>> Thanks! >>>>>> >>>>>> /Claes >>>>>> >>>>>> [1] Unless the default value is invalid, which is a potential >>>>>> programmer error that would be caught in debug builds >>>>>> anyhow. From david.holmes at oracle.com Mon May 22 00:18:56 2017 From: david.holmes at oracle.com (David Holmes) Date: Mon, 22 May 2017 10:18:56 +1000 Subject: =?UTF-8?B?UmU6IOetlOWkjTog562U5aSNOiDnrZTlpI06IEhvdFNwb3QgYW5kIElC?= =?UTF-8?Q?M's_J9_behave_quite_differently_when_processing_monitorenters_and?= =?UTF-8?Q?_monitorexits?= In-Reply-To: <591DD010.3030101@oracle.com> References: <005101d2cf53$c8c16b40$5a4441c0$@cs.sjtu.edu.cn> <656dcb18-e65a-4c48-a6bc-cf00fe021af5@oracle.com> <005201d2cf5a$aa2e8f30$fe8bad90$@cs.sjtu.edu.cn> <5a6a2de2-eb43-985d-bfc7-18c3fa7ab0ee@oracle.com> <24a31d5c-8185-e55d-481f-719f3949bf7d@oracle.com> <77c31484-6c76-4b36-671b-71061c98f1ce@oracle.com> <000e01d2cfe2$374939b0$a5dbad10$@cs.sjtu.edu.cn> <001801d2cfe9$94e5e670$beb1b350$@cs.sjtu.edu.cn> <591DD010.3030101@oracle.com> Message-ID: Ioi, Yuting, After close examination I have closed this bug (8180615) as not an issue. What is happening is that we have a successful monitorenter followed by an invalid monitorenter. The invalid monitorenter throws NPE as expected and the call stack unwinds. As we leave the method frame the system can see that we still have a locked monitor, which means we have skipped an unlock and so have violated the "structured locking" requirements that a VM can choose to enforce. Because of that we throw the IllegalMonitorStateException, which replaces the NullPointerException that was in the process of being thrown. If you change the examples to only test the invalid enter/exits, and ensure all entered monitors are exited properly then you will see the results you expect with regard to the NullPointerException. Thanks, David On 19/05/2017 2:47 AM, Ioi Lam wrote: > Hi ??, > > Thanks for the bug report. I have created a new bug, > https://bugs.openjdk.java.net/browse/JDK-8180615 and attached your test > case. > > JDK-8180615 - monitorenter on null object produces unexpected > IllegalMonitorStateException > > - Ioi > > On 5/18/17 8:15 AM, ??? wrote: >> HotSpot still misses catching the NullPointerException for the next >> program (only one pair of entermonitor and monitorexit). It only >> performs checks when the function exits and throw an >> IllegalMonitorStateException. We should enumerate more cases to test >> the JVMs. >> >> public void main(java.lang.String[]) throws java.lang.Exception; >> descriptor: ([Ljava/lang/String;)V >> flags: ACC_PUBLIC >> Code: >> stack=1, locals=2, args_size=2 >> 0: aload_0 >> 1: monitorenter >> 2: aconst_null >> 3: monitorexit >> 4: return >> Exceptions: >> throws java.lang.Exception >> >> -----????----- >> ???: ??? [mailto:chenyt at cs.sjtu.edu.cn] >> ????: 2017?5?18? 7:56 >> ???: 'Mikael Gerdin' ; 'David Holmes' >> >> ??: 'hotspot-runtime-dev at openjdk.java.net' >> >> ??: ??: ??: ??: HotSpot and IBM's J9 behave quite differently >> when processing monitorenters and monitorexits >> >> Is it caused by the locking structure? I tested the next two programs, >> and saw a difference was raised by the locking structure. The first >> program throws an IllegalMonitorStateException and the second one >> throws the NullPointerException. We just monitored another object in >> the first program. >> >> public void main(java.lang.String[]) throws java.lang.Exception; >> descriptor: ([Ljava/lang/String;)V >> flags: ACC_PUBLIC >> Code: >> stack=1, locals=2, args_size=2 >> 0: aload_0 >> 1: monitorenter >> 2: aconst_null >> 3: monitorenter >> 4: aconst_null >> 5: monitorexit >> 6: aload_0 >> 7: monitorexit >> 8: return >> Exceptions: >> throws java.lang.Exception >> >> >> public void main(java.lang.String[]) throws java.lang.Exception; >> descriptor: ([Ljava/lang/String;)V >> flags: ACC_PUBLIC >> Code: >> stack=1, locals=2, args_size=2 >> 0: aconst_null >> 1: monitorenter >> 2: aconst_null >> 3: monitorexit >> 4: return >> Exceptions: >> throws java.lang.Exception >> >> -----????----- >> ???: Mikael Gerdin [mailto:mikael.gerdin at oracle.com] >> ????: 2017?5?18? 7:26 >> ???: ??? ; 'David Holmes' >> >> ??: hotspot-runtime-dev at openjdk.java.net >> ??: Re: ??: ??: HotSpot and IBM's J9 behave quite differently >> when processing monitorenters and monitorexits >> >> Hi, >> >> On 2017-05-18 16:22, ??? wrote: >>> HotSpot throws a NullPointerException correctly when an initialized >>> object as set as null, as follows. >>> >>> public void main(java.lang.String[]) throws java.lang.Exception; >>> descriptor: ([Ljava/lang/String;)V >>> flags: ACC_PUBLIC >>> Code: >>> stack=1, locals=2, args_size=2 >>> 0: new #2 // class Search >>> 3: invokespecial #14 // Method "":()V >>> 6: aconst_null >>> 7: monitorenter >>> 8: aconst_null >>> 9: monitorexit >>> 10: return >>> Exceptions: >>> throws java.lang.Exception >>> >> Right. I just realized I missed the fact that the code did end up >> dereferencing the object but the code which actually makes that happen >> is not particularly easy to follow. >> Sorry for the noise. >> /Mikael >> >>> -----????----- >>> ???: Mikael Gerdin [mailto:mikael.gerdin at oracle.com] >>> ????: 2017?5?18? 4:55 >>> ???: David Holmes ; chenyt >>> >>> ??: hotspot-runtime-dev at openjdk.java.net >>> ??: Re: ??: HotSpot and IBM's J9 behave quite differently when >>> processing monitorenters and monitorexits >>> >>> >>> >>> On 2017-05-18 07:25, David Holmes wrote: >>>> Hi Yuting, >>>> >>>> On 18/05/2017 2:49 PM, chenyt wrote: >>>>> Hi, David, >>>>> >>>>> I tested the next program (let one object to be monitored be null) >>>>> using >>>>> J9 and HotSpot. HotSpot does not throw a NullPointerException, as >>>>> the specification says. J9 looks fine (throw a >>>>> NullPointerException). Am I wrong here? >>>> I can't readily test this as I don't have Jimple nor quick and easy >>>> access to bytecode assemblers. It is strange though as the >>>> interpreter code contains this: >>>> >>>> CASE(_monitorenter): { >>>> oop lockee = STACK_OBJECT(-1); >>>> // derefing's lockee ought to provoke implicit null check >>>> CHECK_NULL(lockee); >>> In the x86 templateTable we do >>> >>> void TemplateTable::monitorenter() { >>> transition(atos, vtos); >>> >>> // check for NULL object >>> __ null_check(rax); >>> >>> >>> but looking in macroAssembler_x86.cpp null_check only does a cmp >>> (setting the eflags) but there's no condtional branch on the Z flag? >>> >>> It feels like I'm crazy but several of these __ null_check() calls >>> seem broken to me. >>> >>> /Mikael >>> >>>> If I can find some time I will try to test this myself. >>>> >>>>> It is very interesting that I have found so many unexpected >>>>> behaviors here. >>>> Well you only found two and they are related. :) Manually assembled >>>> monitor code is not something very many (any?) people care about or >>>> do >>>> - other than emulating correct language usage. >>>> >>>> Cheers, >>>> David >>>> >>>>> public void main(java.lang.String[]) throws java.lang.Exception; >>>>> descriptor: ([Ljava/lang/String;)V >>>>> flags: ACC_PUBLIC >>>>> Code: >>>>> stack=1, locals=2, args_size=2 >>>>> 0: aload_0 >>>>> 1: monitorenter >>>>> 2: aconst_null >>>>> 3: monitorenter >>>>> 4: aconst_null >>>>> 5: monitorexit >>>>> 6: aload_0 >>>>> 7: monitorexit >>>>> 8: return >>>>> Exceptions: >>>>> throws java.lang.Exception >>>>> >>>>> Jimple code is given as follows: >>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>> { >>>>> Search r0; >>>>> Search r2; >>>>> java.lang.String[] r1; >>>>> >>>>> r0 := @this: Search; >>>>> r1 := @parameter0: java.lang.String[]; >>>>> r2 = null; >>>>> entermonitor r0; >>>>> entermonitor r2; >>>>> exitmonitor r2; >>>>> exitmonitor r0; >>>>> >>>>> return; >>>>> } >>>>> >>>>> Wishes, >>>>> Yuting >>>>> >>>>> >>>>> On Thu, 18 May 2017 13:23:09 +1000, David Holmes wrote: >>>>>> One correction ... >>>>>> >>>>>> On 18/05/2017 10:29 AM, David Holmes wrote: >>>>>>> On 18/05/2017 8:12 AM, ??? wrote: >>>>>>>> Thank you, David. I have seen from the specification that >>>>>>>> structured locking enforcement is optional. The second and the >>>>>>>> third ones are cases of structured/nested lockings. Will >>>>>>>> non-nested locking sequences raise deadlocks? Of course it is a >>>>>>>> different topic, while it might be better if it can be kicked out >>>>>>>> earlier from the specification/JVM. >>>>>>> Non-nested locking doesn't necessarily lead to deadlocks - you >>>>>>> just need a different locking order for that (even if properly >>>>>>> nested). >>>>>>> There are locking patterns that rely on the ability to lock and >>>>>>> unlock in different order ie chained-locking for walking >>>>>>> linked-lists A->B->C->D: >>>>>>> - lock A, lock B, unlock A, lock C, unlock B, lock D, unlock C ... >>>>>>> >>>>>>> The VM spec allows a little flexibility in how the monitor >>>>>>> bytecodes can be used compared to the Java programming language. >>>>>>> That's not something that will change. >>>>>>> >>>>>>>> The first example is still a problem. It seems that HotSpot >>>>>>>> allows to monitor a pure object reference without initialized (Is >>>>>>>> it true? How can this checking be omitted?). J9 reports a >>>>>>>> verifyerror as follows. >>>>>>>> >>>>>>>> Exception in thread "main" java.lang.VerifyError: JVMVRFY012 >>>>>>>> stack shape inconsistent; class=Search, >>>>>>>> method=main([Ljava/lang/String;)V, pc=6 Exception Details: >>>>>>>> Location: >>>>>>>> Search.main([Ljava/lang/String;)V @6: JBmonitorenter >>>>>>>> Reason: >>>>>>>> Type 'uninitialized' (current frame, stack[1]) is not >>>>>>>> assignable to 'java/lang/Object' >>>>>>>> Current Frame: >>>>>>>> bci: @6 >>>>>>>> flags: { } >>>>>>>> locals: { 'Search', '[Ljava/lang/String;' } >>>>>>>> stack: { 'uninitialized', 'uninitialized' } >>>>>>>> at T.main(T.java:4) >>>>>>> Yes I think this may be a bug in hotspot. The type-checking for >>>>>>> the monitor bytecodes requires a matching type of reference on the >>>>>>> operand stack - but "uninitialized" does not match Object, as J9 >>>>>>> reports. But I'm not an expert on this aspect of verification so I >>>>>>> may not be interpreting it correctly. >>>>>>> >>>>>>> More below ... >>>>>>> >>>>>>>> -----????----- >>>>>>>> ???: David Holmes [mailto:david.holmes at oracle.com] >>>>>>>> ????: 2017?5?17? 14:41 >>>>>>>> ???: ??? ; >>>>>>>> hotspot-runtime-dev at openjdk.java.net >>>>>>>> ??: Re: HotSpot and IBM's J9 behave quite differently when >>>>>>>> processing monitorenters and monitorexits >>>>>>>> >>>>>>>> Hi, >>>>>>>> On 18/05/2017 7:23 AM, ??? wrote: >>>>>>>>> Am I wrong? >>>>>>>> I will look at each situation in detail when I get a chance but >>>>>>>> structured locking enforcement is optional. Also balancing the >>>>>>>> number of locks and unlocks in a frame does not mean they can't >>>>>>>> be locked and unlocked in a non-nested fashion - just that by the >>>>>>>> end the number of unlocks matches the number of locks. >>>>>>>> >>>>>>>> BTW the way you respond to these emails, as if having a >>>>>>>> conversation with yourself, makes it difficult to respond as we >>>>>>>> can't readily see what is the new email and what is the original. >>>>>>>> >>>>>>>> Cheers, >>>>>>>> David >>>>>>>> >>>>>>>>> The byte code for main() in case 1 is as follows. The strange >>>>>>>>> thing is that NullPointerException is also not thrown at runtime. >>>>>>> That is strange as it does for the normal obvious case of using >>>>>>> synchronized(o) when o is null. >>>>>> Ah - it isn't null it just an object for which the constructor has >>>>>> not been run. The runtime can't tell the difference between a >>>>>> pointer to a valid initialized object, and a pointer to an >>>>>> uninitialized chunk of memory. >>>>>> >>>>>>>>> public void main(java.lang.String[]) throws java.lang.Exception; >>>>>>>>> descriptor: ([Ljava/lang/String;)V >>>>>>>>> flags: ACC_PUBLIC >>>>>>>>> Code: >>>>>>>>> stack=3, locals=2, args_size=2 >>>>>>>>> 0: new #2 // class Search >>>>>> This allocated an object - hence no null reference. But this is >>>>>> what verification should have complained about. >>>>>> >>>>>> David >>>>>> ----- >>>>>> >>>>>>>>> 3: dup >>>>>>>>> 4: aload_0 >>>>>>>>> 5: monitorenter >>>>>>>>> 6: monitorenter >>>>>>>>> 7: monitorexit >>>>>>>>> 8: aload_0 >>>>>>>>> 9: monitorexit >>>>>>>>> 10: return >>>>>>>>> Exceptions: >>>>>>>>> throws java.lang.Exception >>>>>>>>> >>>>>>>>> ??: HotSpot and IBM's J9 behave quite differently when >>>>>>>>> processing monitorenters and monitorexits >>>>>>>>> >>>>>>>>> I have tested several programs (in Jimple) and found that >>>>>>>>> HotSpot and >>>>>>>>> J9 match monitorenters and monitorexits quite differently. >>>>>>>>> Verifiers should play more important roles here. >>>>>>> The job of the verifier is to establish some basic guarantees for >>>>>>> the JVM to then operate under. The verifier plays no role in >>>>>>> checking how monitorenter/exit are used in combination, only that >>>>>>> each individual bytecode meets some basic type constraints. >>>>>>> >>>>>>>>> (1) Test the next program (r2 is not initizlied) on HotSpot and >>>>>>>>> J9. >>>>>>>>> J9 throw out a verifier error, while HotSpot does not. It seems >>>>>>>>> that HotSpot's verifier forgets to check whether a monitored >>>>>>>>> object is initialized. >>>>>>>>> >>>>>>>>> public class Search extends java.lang.Object { public void >>>>>>>>> () >>>>>>>>> { >>>>>>>>> Search r0; >>>>>>>>> r0 := @this: Search; >>>>>>>>> specialinvoke r0.()>(); >>>>>>>>> return; >>>>>>>>> } >>>>>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>>>>> { >>>>>>>>> Search r0; >>>>>>>>> Search r2; >>>>>>>>> java.lang.String[] r1; >>>>>>>>> r0 := @this: Search; >>>>>>>>> r1 := @parameter0: java.lang.String[]; >>>>>>>>> r2 = new Search; >>>>>>>>> >>>>>>>>> entermonitor r2; >>>>>>>>> entermonitor r0; >>>>>>>>> exitmonitor r2; >>>>>>>>> exitmonitor r0; >>>>>>>>> return; >>>>>>>>> } >>>>>>>>> } >>>>>>> Verification was covered above. >>>>>>> >>>>>>>>> (2) Test the next program on HotSpot and J9, and both do not >>>>>>>>> report any errors. However, I guess the order in the program >>>>>>>>> (entermonitor r2; => entermonitor r0; => exitmonitor r2; => >>>>>>>>> exitmonitor r0;) violates the situation of "structured locking" >>>>>>>>> (Structured locking is the situation when, during a method >>>>>>>>> invocation, every exit on a given monitor matches a preceding >>>>>>>>> entry on that monitor, see the specification >>>>>>>>> >>>>>>>>> https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-2.html#jvms-2. >>>>>>>>> >>>>>>>>> >>>>>>>>> 11.10) >>>>>>>>> ? >>>>>>> No it doesn't violate structured locking as the number of enters >>>>>>> and exits match, and there is always an enter before an exit. >>>>>>> >>>>>>>>> Actually, the words (every exit on a given monitor matches a >>>>>>>>> preceding entry on that monitor) are not quite clear as for me. >>>>>>>>> Otherwise the first rule (The number of monitor entries >>>>>>>>> performed by T on M during a method invocation must equal the >>>>>>>>> number of monitor exits performed by T on M during the method >>>>>>>>> invocation whether the method invocation completes normally or >>>>>>>>> abruptly.) is sufficient. >>>>>>> The number of enters and exits must not only match/balance, but >>>>>>> there must be an enter before a corresponding exit. >>>>>>> >>>>>>>>> public class Search extends java.lang.Object { >>>>>>>>> >>>>>>>>> public void () >>>>>>>>> { >>>>>>>>> Search r0; >>>>>>>>> r0 := @this: Search; >>>>>>>>> specialinvoke r0.()>(); >>>>>>>>> return; >>>>>>>>> } >>>>>>>>> >>>>>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>>>>> { >>>>>>>>> Search r0; >>>>>>>>> Search r2; >>>>>>>>> java.lang.String[] r1; >>>>>>>>> r0 := @this: Search; >>>>>>>>> r1 := @parameter0: java.lang.String[]; >>>>>>>>> r2 = new Search; >>>>>>>>> specialinvoke r2.()>(); >>>>>>>>> entermonitor r2; >>>>>>>>> entermonitor r0; >>>>>>>>> exitmonitor r2; >>>>>>>>> exitmonitor r0; >>>>>>>>> return; >>>>>>>>> } >>>>>>>>> } >>>>>>>>> >>>>>>>>> (3) The next program enters monitor in and exits it in >>>>>>>>> main(). >>>>>>>>> HotSpot throws a runtime exception, while J9 does not. Should >>>>>>>>> this program be rejected by the verifiers? >>>>>>> No this does not violate any verification rules. The runtime >>>>>>> behaviour depends on whether structured locking is enforced or >>>>>>> not.(Even in hotspot there can be differences between interpreted >>>>>>> and jitted code). >>>>>>> >>>>>>> Hope that helps clarify things. >>>>>>> >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>>>> public class Search extends java.lang.Object { >>>>>>>>> >>>>>>>>> public void () >>>>>>>>> { >>>>>>>>> Search r0; >>>>>>>>> r0 := @this: Search; >>>>>>>>> specialinvoke r0.()>(); >>>>>>>>> entermonitor r0; >>>>>>>>> return; >>>>>>>>> } >>>>>>>>> >>>>>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>>>>> { >>>>>>>>> Search r0; >>>>>>>>> Search r2; >>>>>>>>> java.lang.S > From david.holmes at oracle.com Mon May 22 00:28:25 2017 From: david.holmes at oracle.com (David Holmes) Date: Mon, 22 May 2017 10:28:25 +1000 Subject: =?UTF-8?B?UmU6IOetlOWkjTog562U5aSNOiDnrZTlpI06IEhvdFNwb3QgYW5kIElC?= =?UTF-8?Q?M's_J9_behave_quite_differently_when_processing_monitorenters_and?= =?UTF-8?Q?_monitorexits?= In-Reply-To: <000101d2d0e6$735dfa80$5a19ef80$@cs.sjtu.edu.cn> References: <005101d2cf53$c8c16b40$5a4441c0$@cs.sjtu.edu.cn> <656dcb18-e65a-4c48-a6bc-cf00fe021af5@oracle.com> <005201d2cf5a$aa2e8f30$fe8bad90$@cs.sjtu.edu.cn> <5a6a2de2-eb43-985d-bfc7-18c3fa7ab0ee@oracle.com> <000401d2cf8d$c8352470$589f6d50$@cs.sjtu.edu.cn> <18003d32-08ad-4891-cf2a-4ba0d71a2bb0@oracle.com> <9a93ed9268233cf5847156fc7ceef5d5@cs.sjtu.edu.cn> <000101d2d0e6$735dfa80$5a19ef80$@cs.sjtu.edu.cn> Message-ID: <50a38be3-59ab-eb5c-312b-a519d02404e9@oracle.com> Hi Yuting, This invokeinterface issue really should be discussed in a separate thread - this email chain is already quite unstructured. On 20/05/2017 7:25 AM, ??? wrote: > HotSpot's verifyerror is given here. I read the specification (https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.10.1.9.invokeinterface) and did not get the hints. Not quite sure whether isAssignable should be enforced for verifying the invokeinterface instruction. What version of the JDK did you use to check this? There have been some very recent changes with regard to verification of invokeinterface. More generally these kinds of verifier queries, and discussions of difference between verifiers should be taken to jls-jvms-spec-comments at openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/jls-jvms-spec-comments I am not fluent at reading and understanding the prolog rules for this, and this is an area where we need to be very precise. Thanks, David > > Location: > org/dacapo/h2/TPCC.loadData()V @3: invokeinterface > Reason: > Type uninitialized 0 (current frame, stack[0]) is not assignable to 'org/apache/derbyTesting/system/oe/client/Load' > Current Frame: > bci: @3 > flags: { } > locals: { 'org/dacapo/h2/TPCC' } > stack: { uninitialized 0 } > Bytecode: > 0x0000000: bb00 13b9 0018 0100 b1 > > -----????----- > ???: ??? [mailto:chenyt at cs.sjtu.edu.cn] > ????: 2017?5?19? 14:11 > ???: 'David Holmes' > ??: 'hotspot-runtime-dev at openjdk.java.net' > ??: ??: ??: ??: HotSpot and IBM's J9 behave quite differently when processing monitorenters and monitorexits > > Hi, David, > Hi, I saw the discussions (about the verification) at https://bugs.openjdk.java.net/browse/JDK-8180581. If the specification needs to be clarified, please also include a similar example. Similarly, an object is not initialized before the " invokeinterface" instruction. At this time, HotSpot's verifier reports a verifyerror, while J9 normally runs the code. > > A verifier takes a def-use analysis to verify the bytecode, while when an object is really ready to use is not very clear. > > public void loadData() throws java.lang.Exception; > descriptor: ()V > flags: ACC_PUBLIC > Code: > stack=1, locals=1, args_size=1 > 0: new #19 // class org/apache/derbyTesting/system/oe/load/ThreadInsert > 3: invokeinterface #24, 1 // InterfaceMethod org/apache/derbyTesting/system/oe/client/Load.populateAllTables:()V > 8: return > Exceptions: > throws java.lang.Exception > > Regards, > Yuting > > ----------------------- > public class org.dacapo.h2.TPCC > minor version: 0 > major version: 52 > flags: ACC_PUBLIC, ACC_SUPER > Constant pool: > #1 = Utf8 org/dacapo/h2/TPCC > #2 = Class #1 // org/dacapo/h2/TPCC > #3 = Utf8 java/lang/Object > #4 = Class #3 // java/lang/Object > #5 = Utf8 > #6 = Utf8 ()V > #7 = Utf8 > #8 = Utf8 (Lorg/dacapo/parser/Config;Ljava/io/File;ZZ)V > #9 = Utf8 java/lang/Exception > #10 = Class #9 // java/lang/Exception > #11 = NameAndType #7:#6 // "":()V > #12 = Methodref #4.#11 // java/lang/Object."":()V > #13 = Utf8 loadData > #14 = NameAndType #13:#6 // loadData:()V > #15 = Methodref #2.#14 // org/dacapo/h2/TPCC.loadData:()V > #16 = Utf8 iteration > #17 = Utf8 (Ljava/lang/String;)V > #18 = Utf8 org/apache/derbyTesting/system/oe/load/ThreadInsert > #19 = Class #18 // org/apache/derbyTesting/system/oe/load/ThreadInsert > #20 = Utf8 org/apache/derbyTesting/system/oe/client/Load > #21 = Class #20 // org/apache/derbyTesting/system/oe/client/Load > #22 = Utf8 populateAllTables > #23 = NameAndType #22:#6 // populateAllTables:()V > #24 = InterfaceMethodref #21.#23 // org/apache/derbyTesting/system/oe/client/Load.populateAllTables:()V > #25 = Utf8 make > #26 = Utf8 (Lorg/dacapo/parser/Config;Ljava/io/File;Ljava/lang/Boolean;Ljava/lang/Boolean;)Lorg/dacapo/h2/TPCC; > #27 = NameAndType #7:#8 // "":(Lorg/dacapo/parser/Config;Ljava/io/File;ZZ)V > #28 = Methodref #2.#27 // org/dacapo/h2/TPCC."":(Lorg/dacapo/parser/Config;Ljava/io/File;ZZ)V > #29 = Utf8 postIteration > #30 = Utf8 preIteration > #31 = Utf8 Code > #32 = Utf8 Exceptions > { > static {}; > descriptor: ()V > flags: ACC_STATIC > Code: > stack=0, locals=0, args_size=0 > 0: return > > public org.dacapo.h2.TPCC(org.dacapo.parser.Config, java.io.File, boolean, boolean) throws java.lang.Exception; > descriptor: (Lorg/dacapo/parser/Config;Ljava/io/File;ZZ)V > flags: ACC_PUBLIC > Code: > stack=1, locals=5, args_size=5 > 0: aload_0 > 1: invokespecial #12 // Method java/lang/Object."":()V > 4: aload_0 > 5: invokespecial #15 // Method loadData:()V > 8: return > Exceptions: > throws java.lang.Exception > > public void iteration(java.lang.String) throws java.lang.Exception; > descriptor: (Ljava/lang/String;)V > flags: ACC_PUBLIC > Code: > stack=0, locals=2, args_size=2 > 0: return > Exceptions: > throws java.lang.Exception > > public void loadData() throws java.lang.Exception; > descriptor: ()V > flags: ACC_PUBLIC > Code: > stack=1, locals=1, args_size=1 > 0: new #19 // class org/apache/derbyTesting/system/oe/load/ThreadInsert > 3: invokeinterface #24, 1 // InterfaceMethod org/apache/derbyTesting/system/oe/client/Load.populateAllTables:()V > 8: return > Exceptions: > throws java.lang.Exception > > public static org.dacapo.h2.TPCC make(org.dacapo.parser.Config, java.io.File, java.lang.Boolean, java.lang.Boolean) throws java.lang.Exception; > descriptor: (Lorg/dacapo/parser/Config;Ljava/io/File;Ljava/lang/Boolean;Ljava/lang/Boolean;)Lorg/dacapo/h2/TPCC; > flags: ACC_PUBLIC, ACC_STATIC > Code: > stack=6, locals=4, args_size=4 > 0: new #2 // class org/dacapo/h2/TPCC > 3: dup > 4: aload_0 > 5: aload_1 > 6: iconst_1 > 7: iconst_1 > 8: invokespecial #28 // Method "":(Lorg/dacapo/parser/Config;Ljava/io/File;ZZ)V > 11: areturn > Exceptions: > throws java.lang.Exception > > public void postIteration(java.lang.String) throws java.lang.Exception; > descriptor: (Ljava/lang/String;)V > flags: ACC_PUBLIC > Code: > stack=0, locals=2, args_size=2 > 0: return > Exceptions: > throws java.lang.Exception > > public void preIteration(java.lang.String) throws java.lang.Exception; > descriptor: (Ljava/lang/String;)V > flags: ACC_PUBLIC > Code: > stack=0, locals=2, args_size=2 > 0: return > Exceptions: > throws java.lang.Exception > } > From chenyt at cs.sjtu.edu.cn Mon May 22 01:09:30 2017 From: chenyt at cs.sjtu.edu.cn (=?UTF-8?B?6ZmI6Zuo5Lqt?=) Date: Sun, 21 May 2017 18:09:30 -0700 Subject: =?UTF-8?Q?=E7=AD=94=E5=A4=8D:_=E7=AD=94=E5=A4=8D:_=E7=AD=94=E5=A4=8D:_?= =?UTF-8?Q?=E7=AD=94=E5=A4=8D:_HotSpot_and_IBM's_J9_behave_?= =?UTF-8?Q?quite_differently_when_processin?= =?UTF-8?Q?g_monitorenters_and_monitorexits?= In-Reply-To: References: <005101d2cf53$c8c16b40$5a4441c0$@cs.sjtu.edu.cn> <656dcb18-e65a-4c48-a6bc-cf00fe021af5@oracle.com> <005201d2cf5a$aa2e8f30$fe8bad90$@cs.sjtu.edu.cn> <5a6a2de2-eb43-985d-bfc7-18c3fa7ab0ee@oracle.com> <24a31d5c-8185-e55d-481f-719f3949bf7d@oracle.com> <77c31484-6c76-4b36-671b-71061c98f1ce@oracle.com> <000e01d2cfe2$374939b0$a5dbad10$@cs.sjtu.edu.cn> <001801d2cfe9$94e5e670$beb1b350$@cs.sjtu.edu.cn> <591DD010.3030101@oracle.com> Message-ID: <000001d2d298$1275fcf0$3761f6d0$@cs.sjtu.edu.cn> Thank you, David. I have noticed that the policy is nothing different from that for processing an athrow instruction: "Otherwise, if the Java Virtual Machine implementation enforces the rules on structured locking described in ?2.11.10 and if the first of those rules is violated during invocation of the current method, then athrow throws an IllegalMonitorStateException instead of the object previously being thrown." I agree that it is not an issue. A quick suggestion that may help understand the replacement: the specification may say one more sentence (e.g., the runtime processes a runtime exception using the strategy for processing an athrow instruction) in $2.11.9. Regards, Yuting -----????----- ???: David Holmes [mailto:david.holmes at oracle.com] ????: 2017?5?21? 17:19 ???: Ioi Lam ; hotspot-runtime-dev at openjdk.java.net; chenyt at cs.sjtu.edu.cn ??: Re: ??: ??: ??: HotSpot and IBM's J9 behave quite differently when processing monitorenters and monitorexits Ioi, Yuting, After close examination I have closed this bug (8180615) as not an issue. What is happening is that we have a successful monitorenter followed by an invalid monitorenter. The invalid monitorenter throws NPE as expected and the call stack unwinds. As we leave the method frame the system can see that we still have a locked monitor, which means we have skipped an unlock and so have violated the "structured locking" requirements that a VM can choose to enforce. Because of that we throw the IllegalMonitorStateException, which replaces the NullPointerException that was in the process of being thrown. If you change the examples to only test the invalid enter/exits, and ensure all entered monitors are exited properly then you will see the results you expect with regard to the NullPointerException. Thanks, David On 19/05/2017 2:47 AM, Ioi Lam wrote: > Hi ??, > > Thanks for the bug report. I have created a new bug, > https://bugs.openjdk.java.net/browse/JDK-8180615 and attached your > test case. > > JDK-8180615 - monitorenter on null object produces unexpected > IllegalMonitorStateException > > - Ioi > > On 5/18/17 8:15 AM, ??? wrote: >> HotSpot still misses catching the NullPointerException for the next >> program (only one pair of entermonitor and monitorexit). It only >> performs checks when the function exits and throw an >> IllegalMonitorStateException. We should enumerate more cases to test >> the JVMs. >> >> public void main(java.lang.String[]) throws java.lang.Exception; >> descriptor: ([Ljava/lang/String;)V >> flags: ACC_PUBLIC >> Code: >> stack=1, locals=2, args_size=2 >> 0: aload_0 >> 1: monitorenter >> 2: aconst_null >> 3: monitorexit >> 4: return >> Exceptions: >> throws java.lang.Exception >> >> -----????----- >> ???: ??? [mailto:chenyt at cs.sjtu.edu.cn] >> ????: 2017?5?18? 7:56 >> ???: 'Mikael Gerdin' ; 'David Holmes' >> >> ??: 'hotspot-runtime-dev at openjdk.java.net' >> >> ??: ??: ??: ??: HotSpot and IBM's J9 behave quite differently when >> processing monitorenters and monitorexits >> >> Is it caused by the locking structure? I tested the next two >> programs, and saw a difference was raised by the locking structure. >> The first program throws an IllegalMonitorStateException and the >> second one throws the NullPointerException. We just monitored another >> object in the first program. >> >> public void main(java.lang.String[]) throws java.lang.Exception; >> descriptor: ([Ljava/lang/String;)V >> flags: ACC_PUBLIC >> Code: >> stack=1, locals=2, args_size=2 >> 0: aload_0 >> 1: monitorenter >> 2: aconst_null >> 3: monitorenter >> 4: aconst_null >> 5: monitorexit >> 6: aload_0 >> 7: monitorexit >> 8: return >> Exceptions: >> throws java.lang.Exception >> >> >> public void main(java.lang.String[]) throws java.lang.Exception; >> descriptor: ([Ljava/lang/String;)V >> flags: ACC_PUBLIC >> Code: >> stack=1, locals=2, args_size=2 >> 0: aconst_null >> 1: monitorenter >> 2: aconst_null >> 3: monitorexit >> 4: return >> Exceptions: >> throws java.lang.Exception >> >> -----????----- >> ???: Mikael Gerdin [mailto:mikael.gerdin at oracle.com] >> ????: 2017?5?18? 7:26 >> ???: ??? ; 'David Holmes' >> >> ??: hotspot-runtime-dev at openjdk.java.net >> ??: Re: ??: ??: HotSpot and IBM's J9 behave quite differently when >> processing monitorenters and monitorexits >> >> Hi, >> >> On 2017-05-18 16:22, ??? wrote: >>> HotSpot throws a NullPointerException correctly when an initialized >>> object as set as null, as follows. >>> >>> public void main(java.lang.String[]) throws java.lang.Exception; >>> descriptor: ([Ljava/lang/String;)V >>> flags: ACC_PUBLIC >>> Code: >>> stack=1, locals=2, args_size=2 >>> 0: new #2 // class Search >>> 3: invokespecial #14 // Method "":()V >>> 6: aconst_null >>> 7: monitorenter >>> 8: aconst_null >>> 9: monitorexit >>> 10: return >>> Exceptions: >>> throws java.lang.Exception >>> >> Right. I just realized I missed the fact that the code did end up >> dereferencing the object but the code which actually makes that >> happen is not particularly easy to follow. >> Sorry for the noise. >> /Mikael >> >>> -----????----- >>> ???: Mikael Gerdin [mailto:mikael.gerdin at oracle.com] >>> ????: 2017?5?18? 4:55 >>> ???: David Holmes ; chenyt >>> >>> ??: hotspot-runtime-dev at openjdk.java.net >>> ??: Re: ??: HotSpot and IBM's J9 behave quite differently when >>> processing monitorenters and monitorexits >>> >>> >>> >>> On 2017-05-18 07:25, David Holmes wrote: >>>> Hi Yuting, >>>> >>>> On 18/05/2017 2:49 PM, chenyt wrote: >>>>> Hi, David, >>>>> >>>>> I tested the next program (let one object to be monitored be null) >>>>> using >>>>> J9 and HotSpot. HotSpot does not throw a NullPointerException, as >>>>> the specification says. J9 looks fine (throw a >>>>> NullPointerException). Am I wrong here? >>>> I can't readily test this as I don't have Jimple nor quick and easy >>>> access to bytecode assemblers. It is strange though as the >>>> interpreter code contains this: >>>> >>>> CASE(_monitorenter): { >>>> oop lockee = STACK_OBJECT(-1); >>>> // derefing's lockee ought to provoke implicit null check >>>> CHECK_NULL(lockee); >>> In the x86 templateTable we do >>> >>> void TemplateTable::monitorenter() { >>> transition(atos, vtos); >>> >>> // check for NULL object >>> __ null_check(rax); >>> >>> >>> but looking in macroAssembler_x86.cpp null_check only does a cmp >>> (setting the eflags) but there's no condtional branch on the Z flag? >>> >>> It feels like I'm crazy but several of these __ null_check() calls >>> seem broken to me. >>> >>> /Mikael >>> >>>> If I can find some time I will try to test this myself. >>>> >>>>> It is very interesting that I have found so many unexpected >>>>> behaviors here. >>>> Well you only found two and they are related. :) Manually assembled >>>> monitor code is not something very many (any?) people care about or >>>> do >>>> - other than emulating correct language usage. >>>> >>>> Cheers, >>>> David >>>> >>>>> public void main(java.lang.String[]) throws java.lang.Exception; >>>>> descriptor: ([Ljava/lang/String;)V >>>>> flags: ACC_PUBLIC >>>>> Code: >>>>> stack=1, locals=2, args_size=2 >>>>> 0: aload_0 >>>>> 1: monitorenter >>>>> 2: aconst_null >>>>> 3: monitorenter >>>>> 4: aconst_null >>>>> 5: monitorexit >>>>> 6: aload_0 >>>>> 7: monitorexit >>>>> 8: return >>>>> Exceptions: >>>>> throws java.lang.Exception >>>>> >>>>> Jimple code is given as follows: >>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>> { >>>>> Search r0; >>>>> Search r2; >>>>> java.lang.String[] r1; >>>>> >>>>> r0 := @this: Search; >>>>> r1 := @parameter0: java.lang.String[]; >>>>> r2 = null; >>>>> entermonitor r0; >>>>> entermonitor r2; >>>>> exitmonitor r2; >>>>> exitmonitor r0; >>>>> >>>>> return; >>>>> } >>>>> >>>>> Wishes, >>>>> Yuting >>>>> >>>>> >>>>> On Thu, 18 May 2017 13:23:09 +1000, David Holmes wrote: >>>>>> One correction ... >>>>>> >>>>>> On 18/05/2017 10:29 AM, David Holmes wrote: >>>>>>> On 18/05/2017 8:12 AM, ??? wrote: >>>>>>>> Thank you, David. I have seen from the specification that >>>>>>>> structured locking enforcement is optional. The second and the >>>>>>>> third ones are cases of structured/nested lockings. Will >>>>>>>> non-nested locking sequences raise deadlocks? Of course it is a >>>>>>>> different topic, while it might be better if it can be kicked >>>>>>>> out earlier from the specification/JVM. >>>>>>> Non-nested locking doesn't necessarily lead to deadlocks - you >>>>>>> just need a different locking order for that (even if properly >>>>>>> nested). >>>>>>> There are locking patterns that rely on the ability to lock and >>>>>>> unlock in different order ie chained-locking for walking >>>>>>> linked-lists A->B->C->D: >>>>>>> - lock A, lock B, unlock A, lock C, unlock B, lock D, unlock C ... >>>>>>> >>>>>>> The VM spec allows a little flexibility in how the monitor >>>>>>> bytecodes can be used compared to the Java programming language. >>>>>>> That's not something that will change. >>>>>>> >>>>>>>> The first example is still a problem. It seems that HotSpot >>>>>>>> allows to monitor a pure object reference without initialized >>>>>>>> (Is it true? How can this checking be omitted?). J9 reports a >>>>>>>> verifyerror as follows. >>>>>>>> >>>>>>>> Exception in thread "main" java.lang.VerifyError: JVMVRFY012 >>>>>>>> stack shape inconsistent; class=Search, >>>>>>>> method=main([Ljava/lang/String;)V, pc=6 Exception Details: >>>>>>>> Location: >>>>>>>> Search.main([Ljava/lang/String;)V @6: JBmonitorenter >>>>>>>> Reason: >>>>>>>> Type 'uninitialized' (current frame, stack[1]) is not >>>>>>>> assignable to 'java/lang/Object' >>>>>>>> Current Frame: >>>>>>>> bci: @6 >>>>>>>> flags: { } >>>>>>>> locals: { 'Search', '[Ljava/lang/String;' } >>>>>>>> stack: { 'uninitialized', 'uninitialized' } >>>>>>>> at T.main(T.java:4) >>>>>>> Yes I think this may be a bug in hotspot. The type-checking for >>>>>>> the monitor bytecodes requires a matching type of reference on >>>>>>> the operand stack - but "uninitialized" does not match Object, >>>>>>> as J9 reports. But I'm not an expert on this aspect of >>>>>>> verification so I may not be interpreting it correctly. >>>>>>> >>>>>>> More below ... >>>>>>> >>>>>>>> -----????----- >>>>>>>> ???: David Holmes [mailto:david.holmes at oracle.com] >>>>>>>> ????: 2017?5?17? 14:41 >>>>>>>> ???: ??? ; >>>>>>>> hotspot-runtime-dev at openjdk.java.net >>>>>>>> ??: Re: HotSpot and IBM's J9 behave quite differently when >>>>>>>> processing monitorenters and monitorexits >>>>>>>> >>>>>>>> Hi, >>>>>>>> On 18/05/2017 7:23 AM, ??? wrote: >>>>>>>>> Am I wrong? >>>>>>>> I will look at each situation in detail when I get a chance but >>>>>>>> structured locking enforcement is optional. Also balancing the >>>>>>>> number of locks and unlocks in a frame does not mean they can't >>>>>>>> be locked and unlocked in a non-nested fashion - just that by >>>>>>>> the end the number of unlocks matches the number of locks. >>>>>>>> >>>>>>>> BTW the way you respond to these emails, as if having a >>>>>>>> conversation with yourself, makes it difficult to respond as we >>>>>>>> can't readily see what is the new email and what is the original. >>>>>>>> >>>>>>>> Cheers, >>>>>>>> David >>>>>>>> >>>>>>>>> The byte code for main() in case 1 is as follows. The strange >>>>>>>>> thing is that NullPointerException is also not thrown at runtime. >>>>>>> That is strange as it does for the normal obvious case of using >>>>>>> synchronized(o) when o is null. >>>>>> Ah - it isn't null it just an object for which the constructor >>>>>> has not been run. The runtime can't tell the difference between a >>>>>> pointer to a valid initialized object, and a pointer to an >>>>>> uninitialized chunk of memory. >>>>>> >>>>>>>>> public void main(java.lang.String[]) throws java.lang.Exception; >>>>>>>>> descriptor: ([Ljava/lang/String;)V >>>>>>>>> flags: ACC_PUBLIC >>>>>>>>> Code: >>>>>>>>> stack=3, locals=2, args_size=2 >>>>>>>>> 0: new #2 // class Search >>>>>> This allocated an object - hence no null reference. But this is >>>>>> what verification should have complained about. >>>>>> >>>>>> David >>>>>> ----- >>>>>> >>>>>>>>> 3: dup >>>>>>>>> 4: aload_0 >>>>>>>>> 5: monitorenter >>>>>>>>> 6: monitorenter >>>>>>>>> 7: monitorexit >>>>>>>>> 8: aload_0 >>>>>>>>> 9: monitorexit >>>>>>>>> 10: return >>>>>>>>> Exceptions: >>>>>>>>> throws java.lang.Exception >>>>>>>>> >>>>>>>>> ??: HotSpot and IBM's J9 behave quite differently when >>>>>>>>> processing monitorenters and monitorexits >>>>>>>>> >>>>>>>>> I have tested several programs (in Jimple) and found that >>>>>>>>> HotSpot and >>>>>>>>> J9 match monitorenters and monitorexits quite differently. >>>>>>>>> Verifiers should play more important roles here. >>>>>>> The job of the verifier is to establish some basic guarantees >>>>>>> for the JVM to then operate under. The verifier plays no role in >>>>>>> checking how monitorenter/exit are used in combination, only >>>>>>> that each individual bytecode meets some basic type constraints. >>>>>>> >>>>>>>>> (1) Test the next program (r2 is not initizlied) on HotSpot >>>>>>>>> and J9. >>>>>>>>> J9 throw out a verifier error, while HotSpot does not. It >>>>>>>>> seems that HotSpot's verifier forgets to check whether a >>>>>>>>> monitored object is initialized. >>>>>>>>> >>>>>>>>> public class Search extends java.lang.Object { public void >>>>>>>>> () >>>>>>>>> { >>>>>>>>> Search r0; >>>>>>>>> r0 := @this: Search; >>>>>>>>> specialinvoke r0.()>(); >>>>>>>>> return; >>>>>>>>> } >>>>>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>>>>> { >>>>>>>>> Search r0; >>>>>>>>> Search r2; >>>>>>>>> java.lang.String[] r1; >>>>>>>>> r0 := @this: Search; >>>>>>>>> r1 := @parameter0: java.lang.String[]; >>>>>>>>> r2 = new Search; >>>>>>>>> >>>>>>>>> entermonitor r2; >>>>>>>>> entermonitor r0; >>>>>>>>> exitmonitor r2; >>>>>>>>> exitmonitor r0; >>>>>>>>> return; >>>>>>>>> } >>>>>>>>> } >>>>>>> Verification was covered above. >>>>>>> >>>>>>>>> (2) Test the next program on HotSpot and J9, and both do not >>>>>>>>> report any errors. However, I guess the order in the program >>>>>>>>> (entermonitor r2; => entermonitor r0; => exitmonitor r2; => >>>>>>>>> exitmonitor r0;) violates the situation of "structured locking" >>>>>>>>> (Structured locking is the situation when, during a method >>>>>>>>> invocation, every exit on a given monitor matches a preceding >>>>>>>>> entry on that monitor, see the specification >>>>>>>>> >>>>>>>>> https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-2.html#jvms-2. >>>>>>>>> >>>>>>>>> >>>>>>>>> 11.10) >>>>>>>>> ? >>>>>>> No it doesn't violate structured locking as the number of enters >>>>>>> and exits match, and there is always an enter before an exit. >>>>>>> >>>>>>>>> Actually, the words (every exit on a given monitor matches a >>>>>>>>> preceding entry on that monitor) are not quite clear as for me. >>>>>>>>> Otherwise the first rule (The number of monitor entries >>>>>>>>> performed by T on M during a method invocation must equal the >>>>>>>>> number of monitor exits performed by T on M during the method >>>>>>>>> invocation whether the method invocation completes normally >>>>>>>>> or >>>>>>>>> abruptly.) is sufficient. >>>>>>> The number of enters and exits must not only match/balance, but >>>>>>> there must be an enter before a corresponding exit. >>>>>>> >>>>>>>>> public class Search extends java.lang.Object { >>>>>>>>> >>>>>>>>> public void () >>>>>>>>> { >>>>>>>>> Search r0; >>>>>>>>> r0 := @this: Search; >>>>>>>>> specialinvoke r0.()>(); >>>>>>>>> return; >>>>>>>>> } >>>>>>>>> >>>>>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>>>>> { >>>>>>>>> Search r0; >>>>>>>>> Search r2; >>>>>>>>> java.lang.String[] r1; >>>>>>>>> r0 := @this: Search; >>>>>>>>> r1 := @parameter0: java.lang.String[]; >>>>>>>>> r2 = new Search; >>>>>>>>> specialinvoke r2.()>(); >>>>>>>>> entermonitor r2; >>>>>>>>> entermonitor r0; >>>>>>>>> exitmonitor r2; >>>>>>>>> exitmonitor r0; >>>>>>>>> return; >>>>>>>>> } >>>>>>>>> } >>>>>>>>> >>>>>>>>> (3) The next program enters monitor in and exits it in >>>>>>>>> main(). >>>>>>>>> HotSpot throws a runtime exception, while J9 does not. Should >>>>>>>>> this program be rejected by the verifiers? >>>>>>> No this does not violate any verification rules. The runtime >>>>>>> behaviour depends on whether structured locking is enforced or >>>>>>> not.(Even in hotspot there can be differences between >>>>>>> interpreted and jitted code). >>>>>>> >>>>>>> Hope that helps clarify things. >>>>>>> >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>>>> public class Search extends java.lang.Object { >>>>>>>>> >>>>>>>>> public void () >>>>>>>>> { >>>>>>>>> Search r0; >>>>>>>>> r0 := @this: Search; >>>>>>>>> specialinvoke r0.()>(); >>>>>>>>> entermonitor r0; >>>>>>>>> return; >>>>>>>>> } >>>>>>>>> >>>>>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>>>>> { >>>>>>>>> Search r0; >>>>>>>>> Search r2; >>>>>>>>> java.lang.S > From david.holmes at oracle.com Mon May 22 01:25:34 2017 From: david.holmes at oracle.com (David Holmes) Date: Mon, 22 May 2017 11:25:34 +1000 Subject: =?UTF-8?B?UmU6IOetlOWkjTog562U5aSNOiDnrZTlpI06IOetlOWkjTogSG90U3Bv?= =?UTF-8?Q?t_and_IBM's_J9_behave_quite_differently_when_processing_monitoren?= =?UTF-8?Q?ters_and_monitorexits?= In-Reply-To: <000001d2d298$1275fcf0$3761f6d0$@cs.sjtu.edu.cn> References: <005101d2cf53$c8c16b40$5a4441c0$@cs.sjtu.edu.cn> <656dcb18-e65a-4c48-a6bc-cf00fe021af5@oracle.com> <005201d2cf5a$aa2e8f30$fe8bad90$@cs.sjtu.edu.cn> <5a6a2de2-eb43-985d-bfc7-18c3fa7ab0ee@oracle.com> <24a31d5c-8185-e55d-481f-719f3949bf7d@oracle.com> <77c31484-6c76-4b36-671b-71061c98f1ce@oracle.com> <000e01d2cfe2$374939b0$a5dbad10$@cs.sjtu.edu.cn> <001801d2cfe9$94e5e670$beb1b350$@cs.sjtu.edu.cn> <591DD010.3030101@oracle.com> <000001d2d298$1275fcf0$3761f6d0$@cs.sjtu.edu.cn> Message-ID: <681be1e1-fa7f-44d2-a602-1acdeb4d963d@oracle.com> On 22/05/2017 11:09 AM, ??? wrote: > Thank you, David. I have noticed that the policy is nothing different from that for processing an athrow instruction: "Otherwise, if the Java Virtual Machine implementation enforces the rules on structured locking described in ?2.11.10 and if the first of those rules is violated during invocation of the current method, then athrow throws an IllegalMonitorStateException instead of the object previously being thrown." > > I agree that it is not an issue. A quick suggestion that may help understand the replacement: the specification may say one more sentence (e.g., the runtime processes a runtime exception using the strategy for processing an athrow instruction) in $2.11.9. I think this would need to be explained in some more general description of how exceptions are processed overall - but that doesn't seem to exist explicitly in JVMS. If I were specifying this today I might suggest that the original exception be kept as a suppressed-exception of the IMSE. However looking at the actual runtime implementation it seems very difficult to connect the two. David > Regards, > Yuting > > > -----????----- > ???: David Holmes [mailto:david.holmes at oracle.com] > ????: 2017?5?21? 17:19 > ???: Ioi Lam ; hotspot-runtime-dev at openjdk.java.net; chenyt at cs.sjtu.edu.cn > ??: Re: ??: ??: ??: HotSpot and IBM's J9 behave quite differently when processing monitorenters and monitorexits > > Ioi, Yuting, > > After close examination I have closed this bug (8180615) as not an issue. > > What is happening is that we have a successful monitorenter followed by an invalid monitorenter. The invalid monitorenter throws NPE as expected and the call stack unwinds. As we leave the method frame the system can see that we still have a locked monitor, which means we have skipped an unlock and so have violated the "structured locking" requirements that a VM can choose to enforce. Because of that we throw the IllegalMonitorStateException, which replaces the NullPointerException that was in the process of being thrown. > > If you change the examples to only test the invalid enter/exits, and ensure all entered monitors are exited properly then you will see the results you expect with regard to the NullPointerException. > > Thanks, > David > > On 19/05/2017 2:47 AM, Ioi Lam wrote: >> Hi ??, >> >> Thanks for the bug report. I have created a new bug, >> https://bugs.openjdk.java.net/browse/JDK-8180615 and attached your >> test case. >> >> JDK-8180615 - monitorenter on null object produces unexpected >> IllegalMonitorStateException >> >> - Ioi >> >> On 5/18/17 8:15 AM, ??? wrote: >>> HotSpot still misses catching the NullPointerException for the next >>> program (only one pair of entermonitor and monitorexit). It only >>> performs checks when the function exits and throw an >>> IllegalMonitorStateException. We should enumerate more cases to test >>> the JVMs. >>> >>> public void main(java.lang.String[]) throws java.lang.Exception; >>> descriptor: ([Ljava/lang/String;)V >>> flags: ACC_PUBLIC >>> Code: >>> stack=1, locals=2, args_size=2 >>> 0: aload_0 >>> 1: monitorenter >>> 2: aconst_null >>> 3: monitorexit >>> 4: return >>> Exceptions: >>> throws java.lang.Exception >>> >>> -----????----- >>> ???: ??? [mailto:chenyt at cs.sjtu.edu.cn] >>> ????: 2017?5?18? 7:56 >>> ???: 'Mikael Gerdin' ; 'David Holmes' >>> >>> ??: 'hotspot-runtime-dev at openjdk.java.net' >>> >>> ??: ??: ??: ??: HotSpot and IBM's J9 behave quite differently when >>> processing monitorenters and monitorexits >>> >>> Is it caused by the locking structure? I tested the next two >>> programs, and saw a difference was raised by the locking structure. >>> The first program throws an IllegalMonitorStateException and the >>> second one throws the NullPointerException. We just monitored another >>> object in the first program. >>> >>> public void main(java.lang.String[]) throws java.lang.Exception; >>> descriptor: ([Ljava/lang/String;)V >>> flags: ACC_PUBLIC >>> Code: >>> stack=1, locals=2, args_size=2 >>> 0: aload_0 >>> 1: monitorenter >>> 2: aconst_null >>> 3: monitorenter >>> 4: aconst_null >>> 5: monitorexit >>> 6: aload_0 >>> 7: monitorexit >>> 8: return >>> Exceptions: >>> throws java.lang.Exception >>> >>> >>> public void main(java.lang.String[]) throws java.lang.Exception; >>> descriptor: ([Ljava/lang/String;)V >>> flags: ACC_PUBLIC >>> Code: >>> stack=1, locals=2, args_size=2 >>> 0: aconst_null >>> 1: monitorenter >>> 2: aconst_null >>> 3: monitorexit >>> 4: return >>> Exceptions: >>> throws java.lang.Exception >>> >>> -----????----- >>> ???: Mikael Gerdin [mailto:mikael.gerdin at oracle.com] >>> ????: 2017?5?18? 7:26 >>> ???: ??? ; 'David Holmes' >>> >>> ??: hotspot-runtime-dev at openjdk.java.net >>> ??: Re: ??: ??: HotSpot and IBM's J9 behave quite differently when >>> processing monitorenters and monitorexits >>> >>> Hi, >>> >>> On 2017-05-18 16:22, ??? wrote: >>>> HotSpot throws a NullPointerException correctly when an initialized >>>> object as set as null, as follows. >>>> >>>> public void main(java.lang.String[]) throws java.lang.Exception; >>>> descriptor: ([Ljava/lang/String;)V >>>> flags: ACC_PUBLIC >>>> Code: >>>> stack=1, locals=2, args_size=2 >>>> 0: new #2 // class Search >>>> 3: invokespecial #14 // Method "":()V >>>> 6: aconst_null >>>> 7: monitorenter >>>> 8: aconst_null >>>> 9: monitorexit >>>> 10: return >>>> Exceptions: >>>> throws java.lang.Exception >>>> >>> Right. I just realized I missed the fact that the code did end up >>> dereferencing the object but the code which actually makes that >>> happen is not particularly easy to follow. >>> Sorry for the noise. >>> /Mikael >>> >>>> -----????----- >>>> ???: Mikael Gerdin [mailto:mikael.gerdin at oracle.com] >>>> ????: 2017?5?18? 4:55 >>>> ???: David Holmes ; chenyt >>>> >>>> ??: hotspot-runtime-dev at openjdk.java.net >>>> ??: Re: ??: HotSpot and IBM's J9 behave quite differently when >>>> processing monitorenters and monitorexits >>>> >>>> >>>> >>>> On 2017-05-18 07:25, David Holmes wrote: >>>>> Hi Yuting, >>>>> >>>>> On 18/05/2017 2:49 PM, chenyt wrote: >>>>>> Hi, David, >>>>>> >>>>>> I tested the next program (let one object to be monitored be null) >>>>>> using >>>>>> J9 and HotSpot. HotSpot does not throw a NullPointerException, as >>>>>> the specification says. J9 looks fine (throw a >>>>>> NullPointerException). Am I wrong here? >>>>> I can't readily test this as I don't have Jimple nor quick and easy >>>>> access to bytecode assemblers. It is strange though as the >>>>> interpreter code contains this: >>>>> >>>>> CASE(_monitorenter): { >>>>> oop lockee = STACK_OBJECT(-1); >>>>> // derefing's lockee ought to provoke implicit null check >>>>> CHECK_NULL(lockee); >>>> In the x86 templateTable we do >>>> >>>> void TemplateTable::monitorenter() { >>>> transition(atos, vtos); >>>> >>>> // check for NULL object >>>> __ null_check(rax); >>>> >>>> >>>> but looking in macroAssembler_x86.cpp null_check only does a cmp >>>> (setting the eflags) but there's no condtional branch on the Z flag? >>>> >>>> It feels like I'm crazy but several of these __ null_check() calls >>>> seem broken to me. >>>> >>>> /Mikael >>>> >>>>> If I can find some time I will try to test this myself. >>>>> >>>>>> It is very interesting that I have found so many unexpected >>>>>> behaviors here. >>>>> Well you only found two and they are related. :) Manually assembled >>>>> monitor code is not something very many (any?) people care about or >>>>> do >>>>> - other than emulating correct language usage. >>>>> >>>>> Cheers, >>>>> David >>>>> >>>>>> public void main(java.lang.String[]) throws java.lang.Exception; >>>>>> descriptor: ([Ljava/lang/String;)V >>>>>> flags: ACC_PUBLIC >>>>>> Code: >>>>>> stack=1, locals=2, args_size=2 >>>>>> 0: aload_0 >>>>>> 1: monitorenter >>>>>> 2: aconst_null >>>>>> 3: monitorenter >>>>>> 4: aconst_null >>>>>> 5: monitorexit >>>>>> 6: aload_0 >>>>>> 7: monitorexit >>>>>> 8: return >>>>>> Exceptions: >>>>>> throws java.lang.Exception >>>>>> >>>>>> Jimple code is given as follows: >>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>> { >>>>>> Search r0; >>>>>> Search r2; >>>>>> java.lang.String[] r1; >>>>>> >>>>>> r0 := @this: Search; >>>>>> r1 := @parameter0: java.lang.String[]; >>>>>> r2 = null; >>>>>> entermonitor r0; >>>>>> entermonitor r2; >>>>>> exitmonitor r2; >>>>>> exitmonitor r0; >>>>>> >>>>>> return; >>>>>> } >>>>>> >>>>>> Wishes, >>>>>> Yuting >>>>>> >>>>>> >>>>>> On Thu, 18 May 2017 13:23:09 +1000, David Holmes wrote: >>>>>>> One correction ... >>>>>>> >>>>>>> On 18/05/2017 10:29 AM, David Holmes wrote: >>>>>>>> On 18/05/2017 8:12 AM, ??? wrote: >>>>>>>>> Thank you, David. I have seen from the specification that >>>>>>>>> structured locking enforcement is optional. The second and the >>>>>>>>> third ones are cases of structured/nested lockings. Will >>>>>>>>> non-nested locking sequences raise deadlocks? Of course it is a >>>>>>>>> different topic, while it might be better if it can be kicked >>>>>>>>> out earlier from the specification/JVM. >>>>>>>> Non-nested locking doesn't necessarily lead to deadlocks - you >>>>>>>> just need a different locking order for that (even if properly >>>>>>>> nested). >>>>>>>> There are locking patterns that rely on the ability to lock and >>>>>>>> unlock in different order ie chained-locking for walking >>>>>>>> linked-lists A->B->C->D: >>>>>>>> - lock A, lock B, unlock A, lock C, unlock B, lock D, unlock C ... >>>>>>>> >>>>>>>> The VM spec allows a little flexibility in how the monitor >>>>>>>> bytecodes can be used compared to the Java programming language. >>>>>>>> That's not something that will change. >>>>>>>> >>>>>>>>> The first example is still a problem. It seems that HotSpot >>>>>>>>> allows to monitor a pure object reference without initialized >>>>>>>>> (Is it true? How can this checking be omitted?). J9 reports a >>>>>>>>> verifyerror as follows. >>>>>>>>> >>>>>>>>> Exception in thread "main" java.lang.VerifyError: JVMVRFY012 >>>>>>>>> stack shape inconsistent; class=Search, >>>>>>>>> method=main([Ljava/lang/String;)V, pc=6 Exception Details: >>>>>>>>> Location: >>>>>>>>> Search.main([Ljava/lang/String;)V @6: JBmonitorenter >>>>>>>>> Reason: >>>>>>>>> Type 'uninitialized' (current frame, stack[1]) is not >>>>>>>>> assignable to 'java/lang/Object' >>>>>>>>> Current Frame: >>>>>>>>> bci: @6 >>>>>>>>> flags: { } >>>>>>>>> locals: { 'Search', '[Ljava/lang/String;' } >>>>>>>>> stack: { 'uninitialized', 'uninitialized' } >>>>>>>>> at T.main(T.java:4) >>>>>>>> Yes I think this may be a bug in hotspot. The type-checking for >>>>>>>> the monitor bytecodes requires a matching type of reference on >>>>>>>> the operand stack - but "uninitialized" does not match Object, >>>>>>>> as J9 reports. But I'm not an expert on this aspect of >>>>>>>> verification so I may not be interpreting it correctly. >>>>>>>> >>>>>>>> More below ... >>>>>>>> >>>>>>>>> -----????----- >>>>>>>>> ???: David Holmes [mailto:david.holmes at oracle.com] >>>>>>>>> ????: 2017?5?17? 14:41 >>>>>>>>> ???: ??? ; >>>>>>>>> hotspot-runtime-dev at openjdk.java.net >>>>>>>>> ??: Re: HotSpot and IBM's J9 behave quite differently when >>>>>>>>> processing monitorenters and monitorexits >>>>>>>>> >>>>>>>>> Hi, >>>>>>>>> On 18/05/2017 7:23 AM, ??? wrote: >>>>>>>>>> Am I wrong? >>>>>>>>> I will look at each situation in detail when I get a chance but >>>>>>>>> structured locking enforcement is optional. Also balancing the >>>>>>>>> number of locks and unlocks in a frame does not mean they can't >>>>>>>>> be locked and unlocked in a non-nested fashion - just that by >>>>>>>>> the end the number of unlocks matches the number of locks. >>>>>>>>> >>>>>>>>> BTW the way you respond to these emails, as if having a >>>>>>>>> conversation with yourself, makes it difficult to respond as we >>>>>>>>> can't readily see what is the new email and what is the original. >>>>>>>>> >>>>>>>>> Cheers, >>>>>>>>> David >>>>>>>>> >>>>>>>>>> The byte code for main() in case 1 is as follows. The strange >>>>>>>>>> thing is that NullPointerException is also not thrown at runtime. >>>>>>>> That is strange as it does for the normal obvious case of using >>>>>>>> synchronized(o) when o is null. >>>>>>> Ah - it isn't null it just an object for which the constructor >>>>>>> has not been run. The runtime can't tell the difference between a >>>>>>> pointer to a valid initialized object, and a pointer to an >>>>>>> uninitialized chunk of memory. >>>>>>> >>>>>>>>>> public void main(java.lang.String[]) throws java.lang.Exception; >>>>>>>>>> descriptor: ([Ljava/lang/String;)V >>>>>>>>>> flags: ACC_PUBLIC >>>>>>>>>> Code: >>>>>>>>>> stack=3, locals=2, args_size=2 >>>>>>>>>> 0: new #2 // class Search >>>>>>> This allocated an object - hence no null reference. But this is >>>>>>> what verification should have complained about. >>>>>>> >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>>>>> 3: dup >>>>>>>>>> 4: aload_0 >>>>>>>>>> 5: monitorenter >>>>>>>>>> 6: monitorenter >>>>>>>>>> 7: monitorexit >>>>>>>>>> 8: aload_0 >>>>>>>>>> 9: monitorexit >>>>>>>>>> 10: return >>>>>>>>>> Exceptions: >>>>>>>>>> throws java.lang.Exception >>>>>>>>>> >>>>>>>>>> ??: HotSpot and IBM's J9 behave quite differently when >>>>>>>>>> processing monitorenters and monitorexits >>>>>>>>>> >>>>>>>>>> I have tested several programs (in Jimple) and found that >>>>>>>>>> HotSpot and >>>>>>>>>> J9 match monitorenters and monitorexits quite differently. >>>>>>>>>> Verifiers should play more important roles here. >>>>>>>> The job of the verifier is to establish some basic guarantees >>>>>>>> for the JVM to then operate under. The verifier plays no role in >>>>>>>> checking how monitorenter/exit are used in combination, only >>>>>>>> that each individual bytecode meets some basic type constraints. >>>>>>>> >>>>>>>>>> (1) Test the next program (r2 is not initizlied) on HotSpot >>>>>>>>>> and J9. >>>>>>>>>> J9 throw out a verifier error, while HotSpot does not. It >>>>>>>>>> seems that HotSpot's verifier forgets to check whether a >>>>>>>>>> monitored object is initialized. >>>>>>>>>> >>>>>>>>>> public class Search extends java.lang.Object { public void >>>>>>>>>> () >>>>>>>>>> { >>>>>>>>>> Search r0; >>>>>>>>>> r0 := @this: Search; >>>>>>>>>> specialinvoke r0.()>(); >>>>>>>>>> return; >>>>>>>>>> } >>>>>>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>>>>>> { >>>>>>>>>> Search r0; >>>>>>>>>> Search r2; >>>>>>>>>> java.lang.String[] r1; >>>>>>>>>> r0 := @this: Search; >>>>>>>>>> r1 := @parameter0: java.lang.String[]; >>>>>>>>>> r2 = new Search; >>>>>>>>>> >>>>>>>>>> entermonitor r2; >>>>>>>>>> entermonitor r0; >>>>>>>>>> exitmonitor r2; >>>>>>>>>> exitmonitor r0; >>>>>>>>>> return; >>>>>>>>>> } >>>>>>>>>> } >>>>>>>> Verification was covered above. >>>>>>>> >>>>>>>>>> (2) Test the next program on HotSpot and J9, and both do not >>>>>>>>>> report any errors. However, I guess the order in the program >>>>>>>>>> (entermonitor r2; => entermonitor r0; => exitmonitor r2; => >>>>>>>>>> exitmonitor r0;) violates the situation of "structured locking" >>>>>>>>>> (Structured locking is the situation when, during a method >>>>>>>>>> invocation, every exit on a given monitor matches a preceding >>>>>>>>>> entry on that monitor, see the specification >>>>>>>>>> >>>>>>>>>> https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-2.html#jvms-2. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 11.10) >>>>>>>>>> ? >>>>>>>> No it doesn't violate structured locking as the number of enters >>>>>>>> and exits match, and there is always an enter before an exit. >>>>>>>> >>>>>>>>>> Actually, the words (every exit on a given monitor matches a >>>>>>>>>> preceding entry on that monitor) are not quite clear as for me. >>>>>>>>>> Otherwise the first rule (The number of monitor entries >>>>>>>>>> performed by T on M during a method invocation must equal the >>>>>>>>>> number of monitor exits performed by T on M during the method >>>>>>>>>> invocation whether the method invocation completes normally >>>>>>>>>> or >>>>>>>>>> abruptly.) is sufficient. >>>>>>>> The number of enters and exits must not only match/balance, but >>>>>>>> there must be an enter before a corresponding exit. >>>>>>>> >>>>>>>>>> public class Search extends java.lang.Object { >>>>>>>>>> >>>>>>>>>> public void () >>>>>>>>>> { >>>>>>>>>> Search r0; >>>>>>>>>> r0 := @this: Search; >>>>>>>>>> specialinvoke r0.()>(); >>>>>>>>>> return; >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>>>>>> { >>>>>>>>>> Search r0; >>>>>>>>>> Search r2; >>>>>>>>>> java.lang.String[] r1; >>>>>>>>>> r0 := @this: Search; >>>>>>>>>> r1 := @parameter0: java.lang.String[]; >>>>>>>>>> r2 = new Search; >>>>>>>>>> specialinvoke r2.()>(); >>>>>>>>>> entermonitor r2; >>>>>>>>>> entermonitor r0; >>>>>>>>>> exitmonitor r2; >>>>>>>>>> exitmonitor r0; >>>>>>>>>> return; >>>>>>>>>> } >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> (3) The next program enters monitor in and exits it in >>>>>>>>>> main(). >>>>>>>>>> HotSpot throws a runtime exception, while J9 does not. Should >>>>>>>>>> this program be rejected by the verifiers? >>>>>>>> No this does not violate any verification rules. The runtime >>>>>>>> behaviour depends on whether structured locking is enforced or >>>>>>>> not.(Even in hotspot there can be differences between >>>>>>>> interpreted and jitted code). >>>>>>>> >>>>>>>> Hope that helps clarify things. >>>>>>>> >>>>>>>> David >>>>>>>> ----- >>>>>>>> >>>>>>>>>> public class Search extends java.lang.Object { >>>>>>>>>> >>>>>>>>>> public void () >>>>>>>>>> { >>>>>>>>>> Search r0; >>>>>>>>>> r0 := @this: Search; >>>>>>>>>> specialinvoke r0.()>(); >>>>>>>>>> entermonitor r0; >>>>>>>>>> return; >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>>>>>>> { >>>>>>>>>> Search r0; >>>>>>>>>> Search r2; >>>>>>>>>> java.lang.S >> > From david.holmes at oracle.com Mon May 22 01:27:41 2017 From: david.holmes at oracle.com (David Holmes) Date: Mon, 22 May 2017 11:27:41 +1000 Subject: =?UTF-8?B?UmU6IOetlOWkjTog562U5aSNOiBIb3RTcG90IGFuZCBJQk0ncyBKOSBi?= =?UTF-8?Q?ehave_quite_differently_when_processing_monitorenters_and_monitor?= =?UTF-8?Q?exits?= In-Reply-To: <18003d32-08ad-4891-cf2a-4ba0d71a2bb0@oracle.com> References: <005101d2cf53$c8c16b40$5a4441c0$@cs.sjtu.edu.cn> <656dcb18-e65a-4c48-a6bc-cf00fe021af5@oracle.com> <005201d2cf5a$aa2e8f30$fe8bad90$@cs.sjtu.edu.cn> <5a6a2de2-eb43-985d-bfc7-18c3fa7ab0ee@oracle.com> <000401d2cf8d$c8352470$589f6d50$@cs.sjtu.edu.cn> <18003d32-08ad-4891-cf2a-4ba0d71a2bb0@oracle.com> Message-ID: <081a90d2-389d-439e-5e90-4c8369f03b0b@oracle.com> On 18/05/2017 2:57 PM, David Holmes wrote: > On 18/05/2017 2:18 PM, ??? wrote: >> Hi, David, >> >> I still did not catch why it happens... I just read the specification and >> wrote some tests in order to understand some paragraphs better, and >> then had >> questions. >> >> Will the HotSpot's verifier detect such kind of uninitialized monitored >> objects in future? > > I have filed a bug so this can be examined more closely. > > https://bugs.openjdk.java.net/browse/JDK-8180581 This bug has now been closed as "not an issue". Please see the bug report for full details. In short the verifier is not required to reject "uninitialized" instances. You would have to contact the J9 folk to ask them why they do what they do. Thanks, David > Thanks, > David > >> Regards, >> Yuting >> >> -----????----- >> ???: David Holmes [mailto:david.holmes at oracle.com] >> ????: 2017?5?17? 20:23 >> ???: ??? ; >> hotspot-runtime-dev at openjdk.java.net >> ??: Re: ??: HotSpot and IBM's J9 behave quite differently when >> processing monitorenters and monitorexits >> >> One correction ... >> >> On 18/05/2017 10:29 AM, David Holmes wrote: >>> On 18/05/2017 8:12 AM, ??? wrote: >>>> Thank you, David. I have seen from the specification that structured >>>> locking enforcement is optional. The second and the third ones are >>>> cases of structured/nested lockings. Will non-nested locking >>>> sequences raise deadlocks? Of course it is a different topic, while >>>> it might be better if it can be kicked out earlier from the >>>> specification/JVM. >>> >>> Non-nested locking doesn't necessarily lead to deadlocks - you just >>> need a different locking order for that (even if properly nested). >>> There are locking patterns that rely on the ability to lock and unlock >>> in different order ie chained-locking for walking linked-lists >>> A->B->C->D: >>> - lock A, lock B, unlock A, lock C, unlock B, lock D, unlock C ... >>> >>> The VM spec allows a little flexibility in how the monitor bytecodes >>> can be used compared to the Java programming language. That's not >>> something that will change. >>> >>>> The first example is still a problem. It seems that HotSpot allows to >>>> monitor a pure object reference without initialized (Is it true? How >>>> can this checking be omitted?). J9 reports a verifyerror as follows. >>>> >>>> Exception in thread "main" java.lang.VerifyError: JVMVRFY012 stack >>>> shape inconsistent; class=Search, method=main([Ljava/lang/String;)V, >>>> pc=6 Exception Details: >>>> Location: >>>> Search.main([Ljava/lang/String;)V @6: JBmonitorenter >>>> Reason: >>>> Type 'uninitialized' (current frame, stack[1]) is not assignable >>>> to 'java/lang/Object' >>>> Current Frame: >>>> bci: @6 >>>> flags: { } >>>> locals: { 'Search', '[Ljava/lang/String;' } >>>> stack: { 'uninitialized', 'uninitialized' } >>>> at T.main(T.java:4) >>> >>> Yes I think this may be a bug in hotspot. The type-checking for the >>> monitor bytecodes requires a matching type of reference on the operand >>> stack - but "uninitialized" does not match Object, as J9 reports. But >>> I'm not an expert on this aspect of verification so I may not be >>> interpreting it correctly. >>> >>> More below ... >>> >>>> -----????----- >>>> ???: David Holmes [mailto:david.holmes at oracle.com] >>>> ????: 2017?5?17? 14:41 >>>> ???: ??? ; >>>> hotspot-runtime-dev at openjdk.java.net >>>> ??: Re: HotSpot and IBM's J9 behave quite differently when processing >>>> monitorenters and monitorexits >>>> >>>> Hi, >>>> On 18/05/2017 7:23 AM, ??? wrote: >>>>> Am I wrong? >>>> >>>> I will look at each situation in detail when I get a chance but >>>> structured locking enforcement is optional. Also balancing the number >>>> of locks and unlocks in a frame does not mean they can't be locked >>>> and unlocked in a non-nested fashion - just that by the end the >>>> number of unlocks matches the number of locks. >>>> >>>> BTW the way you respond to these emails, as if having a conversation >>>> with yourself, makes it difficult to respond as we can't readily see >>>> what is the new email and what is the original. >>>> >>>> Cheers, >>>> David >>>> >>>>> The byte code for main() in case 1 is as follows. The strange thing >>>>> is that NullPointerException is also not thrown at runtime. >>> >>> That is strange as it does for the normal obvious case of using >>> synchronized(o) when o is null. >> >> Ah - it isn't null it just an object for which the constructor has not >> been >> run. The runtime can't tell the difference between a pointer to a valid >> initialized object, and a pointer to an uninitialized chunk of memory. >> >>>>> >>>>> public void main(java.lang.String[]) throws java.lang.Exception; >>>>> descriptor: ([Ljava/lang/String;)V >>>>> flags: ACC_PUBLIC >>>>> Code: >>>>> stack=3, locals=2, args_size=2 >>>>> 0: new #2 // class Search >> >> This allocated an object - hence no null reference. But this is what >> verification should have complained about. >> >> David >> ----- >> >>>>> 3: dup >>>>> 4: aload_0 >>>>> 5: monitorenter >>>>> 6: monitorenter >>>>> 7: monitorexit >>>>> 8: aload_0 >>>>> 9: monitorexit >>>>> 10: return >>>>> Exceptions: >>>>> throws java.lang.Exception >>>>> >>>>> ??: HotSpot and IBM's J9 behave quite differently when processing >>>>> monitorenters and monitorexits >>>>> >>>>> I have tested several programs (in Jimple) and found that HotSpot >>>>> and >>>>> J9 match monitorenters and monitorexits quite differently. Verifiers >>>>> should play more important roles here. >>> >>> The job of the verifier is to establish some basic guarantees for the >>> JVM to then operate under. The verifier plays no role in checking how >>> monitorenter/exit are used in combination, only that each individual >>> bytecode meets some basic type constraints. >>> >>>>> >>>>> (1) Test the next program (r2 is not initizlied) on HotSpot and J9. >>>>> J9 throw out a verifier error, while HotSpot does not. It seems that >>>>> HotSpot's verifier forgets to check whether a monitored object is >>>>> initialized. >>>>> >>>>> public class Search extends java.lang.Object { public void () >>>>> { >>>>> Search r0; >>>>> r0 := @this: Search; >>>>> specialinvoke r0.()>(); >>>>> return; >>>>> } >>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>> { >>>>> Search r0; >>>>> Search r2; >>>>> java.lang.String[] r1; >>>>> r0 := @this: Search; >>>>> r1 := @parameter0: java.lang.String[]; >>>>> r2 = new Search; >>>>> >>>>> entermonitor r2; >>>>> entermonitor r0; >>>>> exitmonitor r2; >>>>> exitmonitor r0; >>>>> return; >>>>> } >>>>> } >>> >>> Verification was covered above. >>> >>>>> >>>>> (2) Test the next program on HotSpot and J9, and both do not report >>>>> any errors. However, I guess the order in the program (entermonitor >>>>> r2; => entermonitor r0; => exitmonitor r2; => exitmonitor r0;) >>>>> violates the situation of "structured locking" (Structured locking >>>>> is the situation when, during a method invocation, every exit on a >>>>> given monitor matches a preceding entry on that monitor, see the >>>>> specification >>>>> https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-2.html#jvms-2. >>>>> 11.10) >>>>> ? >>> >>> No it doesn't violate structured locking as the number of enters and >>> exits match, and there is always an enter before an exit. >>> >>>>> Actually, the words (every exit on a given monitor matches a >>>>> preceding entry on that monitor) are not quite clear as for me. >>>>> Otherwise the first rule (The number of monitor entries performed by >>>>> T on M during a method invocation must equal the number of monitor >>>>> exits performed by T on M during the method invocation whether the >>>>> method invocation completes normally or abruptly.) is sufficient. >>> >>> The number of enters and exits must not only match/balance, but there >>> must be an enter before a corresponding exit. >>> >>>>> >>>>> public class Search extends java.lang.Object { >>>>> >>>>> public void () >>>>> { >>>>> Search r0; >>>>> r0 := @this: Search; >>>>> specialinvoke r0.()>(); >>>>> return; >>>>> } >>>>> >>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>> { >>>>> Search r0; >>>>> Search r2; >>>>> java.lang.String[] r1; >>>>> r0 := @this: Search; >>>>> r1 := @parameter0: java.lang.String[]; >>>>> r2 = new Search; >>>>> specialinvoke r2.()>(); >>>>> entermonitor r2; >>>>> entermonitor r0; >>>>> exitmonitor r2; >>>>> exitmonitor r0; >>>>> return; >>>>> } >>>>> } >>>>> >>>>> (3) The next program enters monitor in and exits it in main(). >>>>> HotSpot throws a runtime exception, while J9 does not. Should this >>>>> program be rejected by the verifiers? >>> >>> No this does not violate any verification rules. The runtime behaviour >>> depends on whether structured locking is enforced or not.(Even in >>> hotspot there can be differences between interpreted and jitted code). >>> >>> Hope that helps clarify things. >>> >>> David >>> ----- >>> >>>>> >>>>> public class Search extends java.lang.Object { >>>>> >>>>> public void () >>>>> { >>>>> Search r0; >>>>> r0 := @this: Search; >>>>> specialinvoke r0.()>(); >>>>> entermonitor r0; >>>>> return; >>>>> } >>>>> >>>>> public void main(java.lang.String[]) throws java.lang.Exception >>>>> { >>>>> Search r0; >>>>> Search r2; >>>>> java.lang.String[] r1; >>>>> r0 := @this: Search; >>>>> r1 := @parameter0: java.lang.String[]; >>>>> r2 = new Search; >>>>> specialinvoke r2.()>(); >>>>> exitmonitor r2; >>>>> exitmonitor r0; >>>>> return; >>>>> } >>>>> } >>>>> >>>>> >>>> >> From david.holmes at oracle.com Mon May 22 03:31:03 2017 From: david.holmes at oracle.com (David Holmes) Date: Mon, 22 May 2017 13:31:03 +1000 Subject: RFR: JDK-8180599: Possibly miss to iterate monitors on thread exit In-Reply-To: References: Message-ID: <8189ce07-13ab-59ca-090b-6243db692ecc@oracle.com> Hi Roman, This looks fine to me. I will sponsor this with myself and Robbin as reviewers. Thanks. David ----- On 19/05/2017 8:26 PM, Roman Kennke wrote: > Am 19.05.2017 um 09:32 schrieb David Holmes: >> Hi Roman, >> >> On 18/05/2017 10:16 PM, Roman Kennke wrote: >>> As David pointed out here: >>> >>> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2017-May/023468.html >>> >>> >>> The fix to JDK-8180175: ObjectSynchronizer only needs to iterate in-use >>> monitors introduced a bug where we could potentially miss to iterate >>> oops in monitors of exiting threads. The proposed fix is as Robbin Ehn >>> suggested to move the call to omFlush() before we remove the thread from >>> _thread_list: >>> >>> http://cr.openjdk.java.net/~rkennke/8180599/webrev.00/ >>> >>> >>> This solves the problem because there's no window where monitors would >>> not appear in any list. >>> >>> I tested a number of programs and jtreg hotspot_gc tests and it doesn't >>> show any ill effects. >>> >>> Ok? >> >> I think this is okay as an approach. >> >> There is one issue I can see this will expose and that is that omFlush >> doesn't zero omFreeCount. With the current change we may go to a >> safepoint when acquiring the Threads_lock after calling omFlush and >> that could lead to monitor deflating and (if enabled) a call to >> verifyInUse, where we would fail this assert: >> >> assert(free_tally == Self->omFreeCount, "free count off"); >> >> but the fix is trivial: just zero omFreeCount the same way we already >> zero omInUseCount. And we should also have a similar assert that >> tally==omFreeCount. >> >> I also verified that there are no thread inspection mechanisms that >> might be surprised if they find a thread with its monitor lists nulled >> out. (Again due to a safepoint being taken immediately after omFlush.) >> >> The comment: >> >> // Reclaim the objectmonitors from the omFreeList of the moribund thread. >> >> was already inaccurate since in-use lists were added, and should be >> updated to reflect the transfer of free or in-use monitors to the >> global lists. >> >> Also the comment block before ObjectSynchronizer::omFlush in >> synchronizer.cpp needs to be updated to reflect this change. > > Hi David, > > Thanks for reviewing! > > http://cr.openjdk.java.net/~rkennke/8180599/webrev.01/ > > > ? > > Does this require a 2nd reviewer? > > Roman > From zgu at redhat.com Mon May 22 14:34:30 2017 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 22 May 2017 10:34:30 -0400 Subject: RFR(S): 8139673: NMT stack traces in output should show mtcomponent Message-ID: <8f6634ad-b8af-e50b-c980-7edf1a07c0f0@redhat.com> I re-open this bug, as I received a new request for this functionality. The most of work was done by Max Ockner (http://cr.openjdk.java.net/~mockner/8139673/). I only made to a few twists to his patch to create malloc callsite for each memory type, if the callsite is shared by multiple memory types, as Coleen suggested. Bug: https://bugs.openjdk.java.net/browse/JDK-8139673 Webrev: http://cr.openjdk.java.net/~zgu/8139673/webrev.00/ Thanks, -Zhengyu From claes.redestad at oracle.com Mon May 22 15:03:56 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 22 May 2017 17:03:56 +0200 Subject: RFR: 8180763: Improve inlining of Symbol::equals(char*,int) into CompactHashtable Message-ID: <840ef4f9-bdec-3717-7d0a-852f538013d5@oracle.com> Hi, it appears that CompactHashtable suffers slightly from failure to completely inline Symbol::equals, and this trivial patch to move the implementation into the header helps to reduce instructions spent in Symbol::lookup when CDS is enabled by 7.5%: Bug: https://bugs.openjdk.java.net/browse/JDK-8180763 Webrev: http://cr.openjdk.java.net/~redestad/8180763/hotspot.00/ Thanks! /Claes From thomas.schatzl at oracle.com Mon May 22 15:39:53 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 22 May 2017 17:39:53 +0200 Subject: RFR (S): 8180755: Remove use of bitMap.inline.hpp include from instanceKlass.hpp and c1_ValueSet.hpp Message-ID: <1495467593.2573.82.camel@oracle.com> Hi all, ? can I have reviews for this change that removes the use of bitMap.inline.hpp from instanceKlass.hpp (which does not use bitMaps at all) and?c1_ValueSet.hpp as per guidelines? This is only seemingly a large change the reason is the required move of methods that use the BitMap class in .hpp files into .inline.hpp files (GC and compiler only). Also, this touches files from GC, runtime and compiler, so I sent this RFR to all corresponding lists to get reviews from every group for their respective files. It's mostly GC changes though. CR: https://bugs.openjdk.java.net/browse/JDK-8180755 Webrev: http://cr.openjdk.java.net/~tschatzl/8180755/webrev/ Testing: jprt Thanks, ? Thomas From rkennke at redhat.com Mon May 22 15:46:20 2017 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 22 May 2017 17:46:20 +0200 Subject: RFR (jdk10): JDK-8180175: ObjectSynchronizer only needs to iterate in-use monitors In-Reply-To: <525fce11-7055-4221-4690-074f1c2aa338@oracle.com> References: <89399a72-0263-5ebd-5fda-82f404fbe2ca@redhat.com> <6b162e43-3b0f-9e23-e63f-c844b0bc21da@oracle.com> <03378add-534e-bbd1-23cf-5e496786347c@redhat.com> <525fce11-7055-4221-4690-074f1c2aa338@oracle.com> Message-ID: <48227873-9429-45bf-63fc-2f1f10f9d3b7@redhat.com> >>> Looks ok, will look good when we can drop MonitorInUseLists branches. >>> >>> Are you handling the deprecation of MonitorInUseLists when CSR becomes >>> available? >> >> We don't have to stall this RFR until CSR becomes available though, >> right? > > Right, create a new enhancement for that Done: https://bugs.openjdk.java.net/browse/JDK-8180768 > and when CSR becomes available you should be able create a CSR in jira > for that via some menu. > And first changeset for that is just to add an deprecation warning if > someone does -XX:-MonitorInUseLists. Okidoki, will do. Let's see if I can make that the first CSR to go through the system :-P Roman From jiangli.zhou at oracle.com Mon May 22 17:24:23 2017 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Mon, 22 May 2017 10:24:23 -0700 Subject: RFR: 8180763: Improve inlining of Symbol::equals(char*, int) into CompactHashtable In-Reply-To: <840ef4f9-bdec-3717-7d0a-852f538013d5@oracle.com> References: <840ef4f9-bdec-3717-7d0a-852f538013d5@oracle.com> Message-ID: <6E008582-65D9-42CF-923B-2BA5FEA8D641@oracle.com> Hi Claes, Looks good. Thanks, Jiangli > On May 22, 2017, at 8:03 AM, Claes Redestad wrote: > > Hi, > > it appears that CompactHashtable suffers slightly from failure to completely > inline Symbol::equals, and this trivial patch to move the implementation into > the header helps to reduce instructions spent in Symbol::lookup when CDS is > enabled by 7.5%: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8180763 > Webrev: http://cr.openjdk.java.net/~redestad/8180763/hotspot.00/ > > Thanks! > > /Claes From ioi.lam at oracle.com Mon May 22 18:22:30 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 22 May 2017 11:22:30 -0700 Subject: RFR: 8180763: Improve inlining of Symbol::equals(char*, int) into CompactHashtable In-Reply-To: <840ef4f9-bdec-3717-7d0a-852f538013d5@oracle.com> References: <840ef4f9-bdec-3717-7d0a-852f538013d5@oracle.com> Message-ID: <59232C66.2010002@oracle.com> Hi Claes, This looks good to me. Thanks! - Ioi On 5/22/17 8:03 AM, Claes Redestad wrote: > Hi, > > it appears that CompactHashtable suffers slightly from failure to > completely > inline Symbol::equals, and this trivial patch to move the > implementation into > the header helps to reduce instructions spent in Symbol::lookup when > CDS is > enabled by 7.5%: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8180763 > Webrev: http://cr.openjdk.java.net/~redestad/8180763/hotspot.00/ > > Thanks! > > /Claes From david.holmes at oracle.com Tue May 23 01:38:14 2017 From: david.holmes at oracle.com (David Holmes) Date: Tue, 23 May 2017 11:38:14 +1000 Subject: RFR (S): 8180755: Remove use of bitMap.inline.hpp include from instanceKlass.hpp and c1_ValueSet.hpp In-Reply-To: <1495467593.2573.82.camel@oracle.com> References: <1495467593.2573.82.camel@oracle.com> Message-ID: <722e9800-5d1b-bc9e-7bff-16a228d367c2@oracle.com> Hi Thomas, This looks okay to me. A couple of comments: src/share/vm/oops/generateOopMap.cpp inline void GenerateOopMap::set_bbmark_bit(int bci) { _bb_hdr_bits.at_put(bci, true); } Does "inline" serve any purpose here? --- src/share/vm/oops/generateOopMap.hpp - void set_bbmark_bit (int bci) { - _bb_hdr_bits.at_put(bci, true); - } + inline void set_bbmark_bit (int bci); void clear_bbmark_bit (int bci) { _bb_hdr_bits.at_put(bci, false); } I don't understand why set_bbmark_bit had to be moved out but clear_bbmark_bit remains ?? Thanks, David ----- On 23/05/2017 1:39 AM, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this change that removes the use of > bitMap.inline.hpp from instanceKlass.hpp (which does not use bitMaps at > all) and c1_ValueSet.hpp as per guidelines? > > This is only seemingly a large change the reason is the required move > of methods that use the BitMap class in .hpp files into .inline.hpp > files (GC and compiler only). > > Also, this touches files from GC, runtime and compiler, so I sent this > RFR to all corresponding lists to get reviews from every group for > their respective files. It's mostly GC changes though. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8180755 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8180755/webrev/ > Testing: > jprt > > Thanks, > Thomas > From david.holmes at oracle.com Tue May 23 01:46:17 2017 From: david.holmes at oracle.com (David Holmes) Date: Tue, 23 May 2017 11:46:17 +1000 Subject: RFR(S): 8139673: NMT stack traces in output should show mtcomponent In-Reply-To: <8f6634ad-b8af-e50b-c980-7edf1a07c0f0@redhat.com> References: <8f6634ad-b8af-e50b-c980-7edf1a07c0f0@redhat.com> Message-ID: On 23/05/2017 12:34 AM, Zhengyu Gu wrote: > I re-open this bug, as I received a new request for this functionality. > > The most of work was done by Max Ockner > (http://cr.openjdk.java.net/~mockner/8139673/). I only made to a few > twists to his patch to create malloc callsite for each memory type, if > the callsite is shared by multiple memory types, as Coleen suggested. > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8139673 > Webrev: http://cr.openjdk.java.net/~zgu/8139673/webrev.00/ src/share/vm/services/memReporter.cpp Can print_malloc(size_t amount, size_t count) not delegate to print_malloc(size_t amount, size_t count, MEMFLAGS flag) - can we have a default value for "flag"? --- Otherwise seems okay. Copyright years need updating. Thanks, David > > Thanks, > > -Zhengyu > From goetz.lindenmaier at sap.com Tue May 23 06:23:03 2017 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 23 May 2017 06:23:03 +0000 Subject: RFR(S): 8179953: [ppc] TLABWasteIncrement not loaded correctly In-Reply-To: <55e4d2ca-5496-c232-bcf4-a8ba5999b4f8@oracle.com> References: <00a572990741428e9b1935f77c536f07@sap.com> <55e4d2ca-5496-c232-bcf4-a8ba5999b4f8@oracle.com> Message-ID: <1194337ef0bd4ffea9bb28ffe8e59083@sap.com> Hi David, I made a new webrev which contains a proper patch with all the change information: http://cr.openjdk.java.net/~goetz/wr17/8179953-ppc_flag/webrev.02/ Sorry I didn't come back to this more early. Thanks for sponsoring! Best regards, Goetz. > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Mittwoch, 17. Mai 2017 11:50 > To: Lindenmaier, Goetz ; hotspot-runtime- > dev at openjdk.java.net > Subject: Re: RFR(S): 8179953: [ppc] TLABWasteIncrement not loaded correctly > > Okay testing passed. > > If you finalize the changeset and send me a link I will sponsor it for you. > > Thanks, > David > > On 17/05/2017 5:01 PM, David Holmes wrote: > > Hi Goetz, > > > > On 15/05/2017 8:31 PM, Lindenmaier, Goetz wrote: > >> Hi David, > >> > >> yes, what you are saying is correct, and I share your concerns about the > >> design of JVMOptionsUtils. > >> Actually I think a test is considered as passed if the VM issues the > >> message > >> that gc flags arecontradictory. Thus, if you run with > >> GCType=ParallelGC those > >> flags that set G1GC explicitly are not properly tested. This is what > >> first > >> happened with my fix as we test with default settings which is G1. > >> And here I didn't consider that I might change test behavior for the > >> other > >> flags. But if this raises new errors, they should be addressed anyways. > > > > I'm trying to test this change with each explicit GC flag but the thing > > is so slow that it keeps timing out on me. Will take about 5 or 6 hours > > to test at this rate. > > > > David > > > >> Best regards, > >> Goetz. > >> > >> > >>> -----Original Message----- > >>> From: David Holmes [mailto:david.holmes at oracle.com] > >>> Sent: Montag, 15. Mai 2017 06:25 > >>> To: Lindenmaier, Goetz ; hotspot-runtime- > >>> dev at openjdk.java.net > >>> Subject: Re: RFR(S): 8179953: [ppc] TLABWasteIncrement not loaded > >>> correctly > >>> > >>> Hi Goetz, > >>> > >>> On 12/05/2017 4:45 PM, Lindenmaier, Goetz wrote: > >>>> Hi David, > >>>> > >>>> the broken code is protected by > >>>> if (allow_shared_alloc) { > >>>> which evaluates to true with ParallelGC but not with G1GC. > >>>> It might be true with other GCs, too, but the design of > >>>> JVMOptionsUtils:addNameDependency() is quite simple, > >>>> and I just did it as for other flags. > >>>> I could put in more logic into that case there, like > >>>> if (GCType == G1GC || GCTyep == ... ) { > >>>> option.addPrepend("-XX:+UseParallelGC"); > >>>> } > >>>> enumerating all GCs that don't support TLABWastIncrement. > >>>> But I thought one test case for the flag is fine. The way it is > >>>> we missed the error, so I would like to fix the test. > >>> > >>> So if I understand correctly you just want to test use of > >>> TLABWasteIncrement, and you know ParallelGC uses it, so you added > >>> -XX:+UseParallelGC to the test options. Okay I get that part. > >> Yes. > >> > >>> However the more I look at JVMOptionsUtils.java the less I understand > >>> how it expects things to work. I don't find addNameDependency simple, > >>> but confused! It looks for things like G1* and so adds UseG1GC, but > >>> completely ignores that GCType may already be set - but that works > >>> because AFAICS when it finds a G1* option in the existing set of > >>> options, it has to be that G1GC is already the current GC type - > >>> otherwise the test would fail - no? > >>> > >>> Looking at the history of this code it never set the GC so would use the > >>> default, unless addNameDependency explicitly set one. That worked fine. > >>> Then 8144578 took issue with the fact the test only ran with the default > >>> GC and so introduced GCType to run the test using the same GC as was > >>> used to launch the main test. But AFAICS that completely overlooked the > >>> fact that addNameDependency might set an explicit GC! Mea culpa as I was > >>> one of the reviewers of that change and it was obviously incomplete! :( > >>> It also seem there are mails missing from the archives as you find now > >>> mail from me here: > >>> > >>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2016- > January/021390.html > >>> > >>> > >>> BTW in that review it refers to TLABWasteIncrement as a "non-dedicated > >>> gc flag". That suggests to me that testing of this flag was expected to > >>> be GC agnostic. ?? > >>> > >>> Anyway the question is, what is the right/best way to reconcile the use > >>> of GCType with explicit setting of GC selection flags? Your fix is one > >>> way. Another option would be to clear GCType when requiring a specifc > >>> GC. Not sure either way is significantly better - pros and cons to both. > >>> > >>> My concern is whether this change may have an unexpected side-effects > >>> given that I don't see how it can have been working correctly in the > >>> first place. > >>> > >>> Unfortunately the primary author of that code is no longer around. > >>> > >>> David > >>> > >>> > >>>> Yes, in JVMOption I must exclude setting G1GC else I get > >>>> an error claiming that two different GCs are set. > >>>> > >>>> Best regards, > >>>> Goetz. > >>>> > >>>> > >>>> > >>>> > >>>>> -----Original Message----- > >>>>> From: David Holmes [mailto:david.holmes at oracle.com] > >>>>> Sent: Wednesday, May 10, 2017 5:19 AM > >>>>> To: Lindenmaier, Goetz ; hotspot-runtime- > >>>>> dev at openjdk.java.net > >>>>> Subject: Re: RFR(S): 8179953: [ppc] TLABWasteIncrement not loaded > >>>>> correctly > >>>>> > >>>>> Hi Goetz, > >>>>> > >>>>> On 10/05/2017 12:56 AM, Lindenmaier, Goetz wrote: > >>>>>> Hi, > >>>>>> > >>>>>> Please review this change. As I fix the test, I please need a > >>>>>> sponsor. > >>>>>> http://cr.openjdk.java.net/~goetz/wr17/8179953-ppc_flag/webrev.01/ > >>>>> > >>>>> I'm unclear on the test changes. It isn't obvious to me that > >>>>> TLABWasteIncrement is a ParallelGC only flag. By adding that as an > >>>>> additional flag you then had to exclude adding the GCType (if any GC > >>>>> selector is given) to avoid conflicting options - is that right? > >>>>> > >>>>> Thanks, > >>>>> David > >>>>> > >>>>>> On ppc, the mentioned flag is loaded as a 16 bit immediate, while > >>>>>> it can > >>>>> have > >>>>>> bigger values. > >>>>>> I also adapt TestOptionWithRanges, which did not show the bug > because > >>>>>> the flag has no effect with G1. > >>>>>> > >>>>>> Best regards, > >>>>>> Goetz. > >>>>>> > >>>>>> From david.holmes at oracle.com Tue May 23 07:53:44 2017 From: david.holmes at oracle.com (David Holmes) Date: Tue, 23 May 2017 17:53:44 +1000 Subject: RFR(S): 8179953: [ppc] TLABWasteIncrement not loaded correctly In-Reply-To: <1194337ef0bd4ffea9bb28ffe8e59083@sap.com> References: <00a572990741428e9b1935f77c536f07@sap.com> <55e4d2ca-5496-c232-bcf4-a8ba5999b4f8@oracle.com> <1194337ef0bd4ffea9bb28ffe8e59083@sap.com> Message-ID: <98b7d02c-af58-b606-216b-21f43a0d91a5@oracle.com> Hi Goetz, We need a second reviewer. :( David On 23/05/2017 4:23 PM, Lindenmaier, Goetz wrote: > Hi David, > > I made a new webrev which contains a proper patch with all the > change information: > http://cr.openjdk.java.net/~goetz/wr17/8179953-ppc_flag/webrev.02/ > > Sorry I didn't come back to this more early. Thanks for sponsoring! > > Best regards, > Goetz. > > >> -----Original Message----- >> From: David Holmes [mailto:david.holmes at oracle.com] >> Sent: Mittwoch, 17. Mai 2017 11:50 >> To: Lindenmaier, Goetz ; hotspot-runtime- >> dev at openjdk.java.net >> Subject: Re: RFR(S): 8179953: [ppc] TLABWasteIncrement not loaded correctly >> >> Okay testing passed. >> >> If you finalize the changeset and send me a link I will sponsor it for you. >> >> Thanks, >> David >> >> On 17/05/2017 5:01 PM, David Holmes wrote: >>> Hi Goetz, >>> >>> On 15/05/2017 8:31 PM, Lindenmaier, Goetz wrote: >>>> Hi David, >>>> >>>> yes, what you are saying is correct, and I share your concerns about the >>>> design of JVMOptionsUtils. >>>> Actually I think a test is considered as passed if the VM issues the >>>> message >>>> that gc flags arecontradictory. Thus, if you run with >>>> GCType=ParallelGC those >>>> flags that set G1GC explicitly are not properly tested. This is what >>>> first >>>> happened with my fix as we test with default settings which is G1. >>>> And here I didn't consider that I might change test behavior for the >>>> other >>>> flags. But if this raises new errors, they should be addressed anyways. >>> >>> I'm trying to test this change with each explicit GC flag but the thing >>> is so slow that it keeps timing out on me. Will take about 5 or 6 hours >>> to test at this rate. >>> >>> David >>> >>>> Best regards, >>>> Goetz. >>>> >>>> >>>>> -----Original Message----- >>>>> From: David Holmes [mailto:david.holmes at oracle.com] >>>>> Sent: Montag, 15. Mai 2017 06:25 >>>>> To: Lindenmaier, Goetz ; hotspot-runtime- >>>>> dev at openjdk.java.net >>>>> Subject: Re: RFR(S): 8179953: [ppc] TLABWasteIncrement not loaded >>>>> correctly >>>>> >>>>> Hi Goetz, >>>>> >>>>> On 12/05/2017 4:45 PM, Lindenmaier, Goetz wrote: >>>>>> Hi David, >>>>>> >>>>>> the broken code is protected by >>>>>> if (allow_shared_alloc) { >>>>>> which evaluates to true with ParallelGC but not with G1GC. >>>>>> It might be true with other GCs, too, but the design of >>>>>> JVMOptionsUtils:addNameDependency() is quite simple, >>>>>> and I just did it as for other flags. >>>>>> I could put in more logic into that case there, like >>>>>> if (GCType == G1GC || GCTyep == ... ) { >>>>>> option.addPrepend("-XX:+UseParallelGC"); >>>>>> } >>>>>> enumerating all GCs that don't support TLABWastIncrement. >>>>>> But I thought one test case for the flag is fine. The way it is >>>>>> we missed the error, so I would like to fix the test. >>>>> >>>>> So if I understand correctly you just want to test use of >>>>> TLABWasteIncrement, and you know ParallelGC uses it, so you added >>>>> -XX:+UseParallelGC to the test options. Okay I get that part. >>>> Yes. >>>> >>>>> However the more I look at JVMOptionsUtils.java the less I understand >>>>> how it expects things to work. I don't find addNameDependency simple, >>>>> but confused! It looks for things like G1* and so adds UseG1GC, but >>>>> completely ignores that GCType may already be set - but that works >>>>> because AFAICS when it finds a G1* option in the existing set of >>>>> options, it has to be that G1GC is already the current GC type - >>>>> otherwise the test would fail - no? >>>>> >>>>> Looking at the history of this code it never set the GC so would use the >>>>> default, unless addNameDependency explicitly set one. That worked fine. >>>>> Then 8144578 took issue with the fact the test only ran with the default >>>>> GC and so introduced GCType to run the test using the same GC as was >>>>> used to launch the main test. But AFAICS that completely overlooked the >>>>> fact that addNameDependency might set an explicit GC! Mea culpa as I was >>>>> one of the reviewers of that change and it was obviously incomplete! :( >>>>> It also seem there are mails missing from the archives as you find now >>>>> mail from me here: >>>>> >>>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2016- >> January/021390.html >>>>> >>>>> >>>>> BTW in that review it refers to TLABWasteIncrement as a "non-dedicated >>>>> gc flag". That suggests to me that testing of this flag was expected to >>>>> be GC agnostic. ?? >>>>> >>>>> Anyway the question is, what is the right/best way to reconcile the use >>>>> of GCType with explicit setting of GC selection flags? Your fix is one >>>>> way. Another option would be to clear GCType when requiring a specifc >>>>> GC. Not sure either way is significantly better - pros and cons to both. >>>>> >>>>> My concern is whether this change may have an unexpected side-effects >>>>> given that I don't see how it can have been working correctly in the >>>>> first place. >>>>> >>>>> Unfortunately the primary author of that code is no longer around. >>>>> >>>>> David >>>>> >>>>> >>>>>> Yes, in JVMOption I must exclude setting G1GC else I get >>>>>> an error claiming that two different GCs are set. >>>>>> >>>>>> Best regards, >>>>>> Goetz. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>> -----Original Message----- >>>>>>> From: David Holmes [mailto:david.holmes at oracle.com] >>>>>>> Sent: Wednesday, May 10, 2017 5:19 AM >>>>>>> To: Lindenmaier, Goetz ; hotspot-runtime- >>>>>>> dev at openjdk.java.net >>>>>>> Subject: Re: RFR(S): 8179953: [ppc] TLABWasteIncrement not loaded >>>>>>> correctly >>>>>>> >>>>>>> Hi Goetz, >>>>>>> >>>>>>> On 10/05/2017 12:56 AM, Lindenmaier, Goetz wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> Please review this change. As I fix the test, I please need a >>>>>>>> sponsor. >>>>>>>> http://cr.openjdk.java.net/~goetz/wr17/8179953-ppc_flag/webrev.01/ >>>>>>> >>>>>>> I'm unclear on the test changes. It isn't obvious to me that >>>>>>> TLABWasteIncrement is a ParallelGC only flag. By adding that as an >>>>>>> additional flag you then had to exclude adding the GCType (if any GC >>>>>>> selector is given) to avoid conflicting options - is that right? >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> >>>>>>>> On ppc, the mentioned flag is loaded as a 16 bit immediate, while >>>>>>>> it can >>>>>>> have >>>>>>>> bigger values. >>>>>>>> I also adapt TestOptionWithRanges, which did not show the bug >> because >>>>>>>> the flag has no effect with G1. >>>>>>>> >>>>>>>> Best regards, >>>>>>>> Goetz. >>>>>>>> >>>>>>>> From shade at redhat.com Tue May 23 07:59:08 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 23 May 2017 09:59:08 +0200 Subject: RFR(S): 8179953: [ppc] TLABWasteIncrement not loaded correctly In-Reply-To: <98b7d02c-af58-b606-216b-21f43a0d91a5@oracle.com> References: <00a572990741428e9b1935f77c536f07@sap.com> <55e4d2ca-5496-c232-bcf4-a8ba5999b4f8@oracle.com> <1194337ef0bd4ffea9bb28ffe8e59083@sap.com> <98b7d02c-af58-b606-216b-21f43a0d91a5@oracle.com> Message-ID: On 05/23/2017 09:53 AM, David Holmes wrote: > We need a second reviewer. :( Non-runtime Reviewer is fine? > On 23/05/2017 4:23 PM, Lindenmaier, Goetz wrote: >> Hi David, >> >> I made a new webrev which contains a proper patch with all the >> change information: >> http://cr.openjdk.java.net/~goetz/wr17/8179953-ppc_flag/webrev.02/ This looks good. Thanks, -Aleksey From david.holmes at oracle.com Tue May 23 08:47:29 2017 From: david.holmes at oracle.com (David Holmes) Date: Tue, 23 May 2017 18:47:29 +1000 Subject: RFR(S): 8179953: [ppc] TLABWasteIncrement not loaded correctly In-Reply-To: References: <00a572990741428e9b1935f77c536f07@sap.com> <55e4d2ca-5496-c232-bcf4-a8ba5999b4f8@oracle.com> <1194337ef0bd4ffea9bb28ffe8e59083@sap.com> <98b7d02c-af58-b606-216b-21f43a0d91a5@oracle.com> Message-ID: <03adce34-cf96-40ac-b56b-9407c2081802@oracle.com> On 23/05/2017 5:59 PM, Aleksey Shipilev wrote: > On 05/23/2017 09:53 AM, David Holmes wrote: >> We need a second reviewer. :( > > Non-runtime Reviewer is fine? Yes - thanks. Only need one runtime reviewer and one Reviewer. :) David >> On 23/05/2017 4:23 PM, Lindenmaier, Goetz wrote: >>> Hi David, >>> >>> I made a new webrev which contains a proper patch with all the >>> change information: >>> http://cr.openjdk.java.net/~goetz/wr17/8179953-ppc_flag/webrev.02/ > > This looks good. > > Thanks, > -Aleksey > > From thomas.schatzl at oracle.com Tue May 23 10:36:51 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 23 May 2017 12:36:51 +0200 Subject: RFR (S): 8180755: Remove use of bitMap.inline.hpp include from instanceKlass.hpp and c1_ValueSet.hpp In-Reply-To: <722e9800-5d1b-bc9e-7bff-16a228d367c2@oracle.com> References: <1495467593.2573.82.camel@oracle.com> <722e9800-5d1b-bc9e-7bff-16a228d367c2@oracle.com> Message-ID: <1495535811.2781.1.camel@oracle.com> Hi David, ? thanks for your review. On Tue, 2017-05-23 at 11:38 +1000, David Holmes wrote: > Hi Thomas, > > This looks okay to me. A couple of comments: > > src/share/vm/oops/generateOopMap.cpp > > inline void GenerateOopMap::set_bbmark_bit(int bci) { > ???_bb_hdr_bits.at_put(bci, true); > } > > Does "inline" serve any purpose here? Removed. > > --- > > src/share/vm/oops/generateOopMap.hpp > > -??void??????????set_bbmark_bit??????????????(int bci) { > -????_bb_hdr_bits.at_put(bci, true); > -??} > +??inline void???set_bbmark_bit??????????????(int bci); > ????void??????????clear_bbmark_bit????????????(int bci) { > ??????_bb_hdr_bits.at_put(bci, false); > ????} > > I don't understand why set_bbmark_bit had to be moved out but? > clear_bbmark_bit remains ?? > It is never referenced. Removed the method. http://cr.openjdk.java.net/~tschatzl/8180755/webrev.0_to_1 (diff) http://cr.openjdk.java.net/~tschatzl/8180755/webrev.1 (full) Thanks, ? Thomas From david.holmes at oracle.com Tue May 23 12:31:35 2017 From: david.holmes at oracle.com (David Holmes) Date: Tue, 23 May 2017 22:31:35 +1000 Subject: RFR (S): 8180755: Remove use of bitMap.inline.hpp include from instanceKlass.hpp and c1_ValueSet.hpp In-Reply-To: <1495535811.2781.1.camel@oracle.com> References: <1495467593.2573.82.camel@oracle.com> <722e9800-5d1b-bc9e-7bff-16a228d367c2@oracle.com> <1495535811.2781.1.camel@oracle.com> Message-ID: <1ac8dcac-b750-9e60-addc-0f80b2644943@oracle.com> On 23/05/2017 8:36 PM, Thomas Schatzl wrote: > Hi David, > > thanks for your review. > > On Tue, 2017-05-23 at 11:38 +1000, David Holmes wrote: >> Hi Thomas, >> >> This looks okay to me. A couple of comments: >> >> src/share/vm/oops/generateOopMap.cpp >> >> inline void GenerateOopMap::set_bbmark_bit(int bci) { >> _bb_hdr_bits.at_put(bci, true); >> } >> >> Does "inline" serve any purpose here? > > Removed. It is still on the declaration in the header file: inline void set_bbmark_bit (int bci); >> >> --- >> >> src/share/vm/oops/generateOopMap.hpp >> >> - void set_bbmark_bit (int bci) { >> - _bb_hdr_bits.at_put(bci, true); >> - } >> + inline void set_bbmark_bit (int bci); >> void clear_bbmark_bit (int bci) { >> _bb_hdr_bits.at_put(bci, false); >> } >> >> I don't understand why set_bbmark_bit had to be moved out but >> clear_bbmark_bit remains ?? >> > > It is never referenced. Removed the method. Ok. Thanks, David > http://cr.openjdk.java.net/~tschatzl/8180755/webrev.0_to_1 (diff) > http://cr.openjdk.java.net/~tschatzl/8180755/webrev.1 (full) > > Thanks, > Thomas > From thomas.schatzl at oracle.com Tue May 23 13:01:53 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 23 May 2017 15:01:53 +0200 Subject: RFR (S): 8180755: Remove use of bitMap.inline.hpp include from instanceKlass.hpp and c1_ValueSet.hpp In-Reply-To: <1ac8dcac-b750-9e60-addc-0f80b2644943@oracle.com> References: <1495467593.2573.82.camel@oracle.com> <722e9800-5d1b-bc9e-7bff-16a228d367c2@oracle.com> <1495535811.2781.1.camel@oracle.com> <1ac8dcac-b750-9e60-addc-0f80b2644943@oracle.com> Message-ID: <1495544513.2781.44.camel@oracle.com> Hi David, On Tue, 2017-05-23 at 22:31 +1000, David Holmes wrote: > On 23/05/2017 8:36 PM, Thomas Schatzl wrote: > > > > Hi David, > > > > ???thanks for your review. > > > > On Tue, 2017-05-23 at 11:38 +1000, David Holmes wrote: > > > > > > Hi Thomas, > > > > > > This looks okay to me. A couple of comments: > > > > > > src/share/vm/oops/generateOopMap.cpp > > > > > > inline void GenerateOopMap::set_bbmark_bit(int bci) { > > > ????_bb_hdr_bits.at_put(bci, true); > > > } > > > > > > Does "inline" serve any purpose here? > > Removed. > It is still on the declaration in the header file: > > ?????inline void???set_bbmark_bit??????????????(int bci); ? I updated the webrev. > > http://cr.openjdk.java.net/~tschatzl/8180755/webrev.0_to_1 (diff) > > http://cr.openjdk.java.net/~tschatzl/8180755/webrev.1 (full) > > Thanks, ? Thomas From zgu at redhat.com Tue May 23 13:33:19 2017 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 23 May 2017 09:33:19 -0400 Subject: RFR(S): 8139673: NMT stack traces in output should show mtcomponent In-Reply-To: References: <8f6634ad-b8af-e50b-c980-7edf1a07c0f0@redhat.com> Message-ID: <1ee9293f-91bb-ff6f-2335-db0b0cca6d1a@redhat.com> Hi David, Thanks for the review. I updated webrev according to your comments. Webrev: http://cr.openjdk.java.net/~zgu/8139673/webrev.01/index.html -Zhengyu On 05/22/2017 09:46 PM, David Holmes wrote: > On 23/05/2017 12:34 AM, Zhengyu Gu wrote: >> I re-open this bug, as I received a new request for this functionality. >> >> The most of work was done by Max Ockner >> (http://cr.openjdk.java.net/~mockner/8139673/). I only made to a few >> twists to his patch to create malloc callsite for each memory type, if >> the callsite is shared by multiple memory types, as Coleen suggested. >> >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8139673 >> Webrev: http://cr.openjdk.java.net/~zgu/8139673/webrev.00/ > > > src/share/vm/services/memReporter.cpp > > Can print_malloc(size_t amount, size_t count) not delegate to > print_malloc(size_t amount, size_t count, MEMFLAGS flag) - can we have a > default value for "flag"? > > --- > > Otherwise seems okay. Copyright years need updating. > > Thanks, > David > >> >> Thanks, >> >> -Zhengyu >> From coleen.phillimore at oracle.com Tue May 23 14:15:51 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 23 May 2017 10:15:51 -0400 Subject: RFR (S): 8180755: Remove use of bitMap.inline.hpp include from instanceKlass.hpp and c1_ValueSet.hpp In-Reply-To: <1495544513.2781.44.camel@oracle.com> References: <1495467593.2573.82.camel@oracle.com> <722e9800-5d1b-bc9e-7bff-16a228d367c2@oracle.com> <1495535811.2781.1.camel@oracle.com> <1ac8dcac-b750-9e60-addc-0f80b2644943@oracle.com> <1495544513.2781.44.camel@oracle.com> Message-ID: <534a0cbf-ad68-f76b-678d-b3764a68c00b@oracle.com> http://cr.openjdk.java.net/~tschatzl/8180755/webrev.1/src/share/vm/oops/generateOopMap.cpp.udiff.html I don't think this should include methodHandles.hpp (I see you've rearranged it, can you try to remove it?) The rest of this looks fine to me. Coleen On 5/23/17 9:01 AM, Thomas Schatzl wrote: > Hi David, > > On Tue, 2017-05-23 at 22:31 +1000, David Holmes wrote: >> On 23/05/2017 8:36 PM, Thomas Schatzl wrote: >>> Hi David, >>> >>> thanks for your review. >>> >>> On Tue, 2017-05-23 at 11:38 +1000, David Holmes wrote: >>>> Hi Thomas, >>>> >>>> This looks okay to me. A couple of comments: >>>> >>>> src/share/vm/oops/generateOopMap.cpp >>>> >>>> inline void GenerateOopMap::set_bbmark_bit(int bci) { >>>> _bb_hdr_bits.at_put(bci, true); >>>> } >>>> >>>> Does "inline" serve any purpose here? >>> Removed. >> It is still on the declaration in the header file: >> >> inline void set_bbmark_bit (int bci); > I updated the webrev. > >>> http://cr.openjdk.java.net/~tschatzl/8180755/webrev.0_to_1 (diff) >>> http://cr.openjdk.java.net/~tschatzl/8180755/webrev.1 (full) >>> > Thanks, > Thomas > From shade at redhat.com Tue May 23 15:05:40 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 23 May 2017 17:05:40 +0200 Subject: RFR(S): 8139673: NMT stack traces in output should show mtcomponent In-Reply-To: <1ee9293f-91bb-ff6f-2335-db0b0cca6d1a@redhat.com> References: <8f6634ad-b8af-e50b-c980-7edf1a07c0f0@redhat.com> <1ee9293f-91bb-ff6f-2335-db0b0cca6d1a@redhat.com> Message-ID: <7357dd9b-18f0-e931-05e2-f67e716d6128@redhat.com> On 05/23/2017 03:33 PM, Zhengyu Gu wrote: > Webrev: http://cr.openjdk.java.net/~zgu/8139673/webrev.01/index.html Looks good. Minor nit: there are some double spaces, fix them before pushing? 71 _malloc_site(stack, flags), _next(NULL) { ... 81 bool ret = MallocSiteTable::allocation_at(stack, size, bucket_idx, pos_idx, flags); Thanks, -Aleksey From vladimir.kozlov at oracle.com Tue May 23 15:13:42 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 23 May 2017 08:13:42 -0700 Subject: RFR (S): 8180755: Remove use of bitMap.inline.hpp include from instanceKlass.hpp and c1_ValueSet.hpp In-Reply-To: <1495544513.2781.44.camel@oracle.com> References: <1495467593.2573.82.camel@oracle.com> <722e9800-5d1b-bc9e-7bff-16a228d367c2@oracle.com> <1495535811.2781.1.camel@oracle.com> <1ac8dcac-b750-9e60-addc-0f80b2644943@oracle.com> <1495544513.2781.44.camel@oracle.com> Message-ID: <1bb9fc26-8d99-0660-6ef8-5b45226a7b0a@oracle.com> C1 changes are fine. Thanks, Vladimir On 5/23/17 6:01 AM, Thomas Schatzl wrote: > Hi David, > > On Tue, 2017-05-23 at 22:31 +1000, David Holmes wrote: >> On 23/05/2017 8:36 PM, Thomas Schatzl wrote: >>> >>> Hi David, >>> >>> thanks for your review. >>> >>> On Tue, 2017-05-23 at 11:38 +1000, David Holmes wrote: >>>> >>>> Hi Thomas, >>>> >>>> This looks okay to me. A couple of comments: >>>> >>>> src/share/vm/oops/generateOopMap.cpp >>>> >>>> inline void GenerateOopMap::set_bbmark_bit(int bci) { >>>> _bb_hdr_bits.at_put(bci, true); >>>> } >>>> >>>> Does "inline" serve any purpose here? >>> Removed. >> It is still on the declaration in the header file: >> >> inline void set_bbmark_bit (int bci); > > I updated the webrev. > >>> http://cr.openjdk.java.net/~tschatzl/8180755/webrev.0_to_1 (diff) >>> http://cr.openjdk.java.net/~tschatzl/8180755/webrev.1 (full) >>> > > Thanks, > Thomas > From thomas.schatzl at oracle.com Tue May 23 15:31:26 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 23 May 2017 17:31:26 +0200 Subject: RFR (S): 8180755: Remove use of bitMap.inline.hpp include from instanceKlass.hpp and c1_ValueSet.hpp In-Reply-To: <534a0cbf-ad68-f76b-678d-b3764a68c00b@oracle.com> References: <1495467593.2573.82.camel@oracle.com> <722e9800-5d1b-bc9e-7bff-16a228d367c2@oracle.com> <1495535811.2781.1.camel@oracle.com> <1ac8dcac-b750-9e60-addc-0f80b2644943@oracle.com> <1495544513.2781.44.camel@oracle.com> <534a0cbf-ad68-f76b-678d-b3764a68c00b@oracle.com> Message-ID: <1495553486.2781.48.camel@oracle.com> Hi Coleen, On Tue, 2017-05-23 at 10:15 -0400, coleen.phillimore at oracle.com wrote: > http://cr.openjdk.java.net/~tschatzl/8180755/webrev.1/src/share/vm/oo > ps/generateOopMap.cpp.udiff.html > > I don't think this should include methodHandles.hpp (I see you've? > rearranged it, can you try to remove it?) > > The rest of this looks fine to me. ? Done, still passes build-only jprt. New webrevs: http://cr.openjdk.java.net/~tschatzl/8180755/webrev.1_to_2 (diff) http://cr.openjdk.java.net/~tschatzl/8180755/webrev.2 (full) Thanks, ? Thomas From zgu at redhat.com Tue May 23 16:05:07 2017 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 23 May 2017 12:05:07 -0400 Subject: RFR(S): 8139673: NMT stack traces in output should show mtcomponent In-Reply-To: <7357dd9b-18f0-e931-05e2-f67e716d6128@redhat.com> References: <8f6634ad-b8af-e50b-c980-7edf1a07c0f0@redhat.com> <1ee9293f-91bb-ff6f-2335-db0b0cca6d1a@redhat.com> <7357dd9b-18f0-e931-05e2-f67e716d6128@redhat.com> Message-ID: <95dba880-4d17-7fad-c700-d540bfb0c48f@redhat.com> Hi Aleksey, Thanks for the review. I fixed double spaces in new webrev: http://cr.openjdk.java.net/~zgu/8139673/webrev.02/ I do need sponsor the this, and please also add Max Ockner (mockner) as an author. Thanks, -Zhengyu On 05/23/2017 11:05 AM, Aleksey Shipilev wrote: > On 05/23/2017 03:33 PM, Zhengyu Gu wrote: >> Webrev: http://cr.openjdk.java.net/~zgu/8139673/webrev.01/index.html > > Looks good. > > Minor nit: there are some double spaces, fix them before pushing? > > 71 _malloc_site(stack, flags), _next(NULL) { > > ... > > 81 bool ret = MallocSiteTable::allocation_at(stack, size, bucket_idx, > pos_idx, flags); > > Thanks, > -Aleksey > > > From coleen.phillimore at oracle.com Tue May 23 16:54:34 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 23 May 2017 12:54:34 -0400 Subject: RFR (S): 8180755: Remove use of bitMap.inline.hpp include from instanceKlass.hpp and c1_ValueSet.hpp In-Reply-To: <1495553486.2781.48.camel@oracle.com> References: <1495467593.2573.82.camel@oracle.com> <722e9800-5d1b-bc9e-7bff-16a228d367c2@oracle.com> <1495535811.2781.1.camel@oracle.com> <1ac8dcac-b750-9e60-addc-0f80b2644943@oracle.com> <1495544513.2781.44.camel@oracle.com> <534a0cbf-ad68-f76b-678d-b3764a68c00b@oracle.com> <1495553486.2781.48.camel@oracle.com> Message-ID: <016ce562-4c05-c851-fe2f-f8b5ab38527e@oracle.com> Thanks! Coleen On 5/23/17 11:31 AM, Thomas Schatzl wrote: > Hi Coleen, > > On Tue, 2017-05-23 at 10:15 -0400, coleen.phillimore at oracle.com wrote: >> http://cr.openjdk.java.net/~tschatzl/8180755/webrev.1/src/share/vm/oo >> ps/generateOopMap.cpp.udiff.html >> >> I don't think this should include methodHandles.hpp (I see you've >> rearranged it, can you try to remove it?) >> >> The rest of this looks fine to me. > Done, still passes build-only jprt. New webrevs: > > http://cr.openjdk.java.net/~tschatzl/8180755/webrev.1_to_2 (diff) > http://cr.openjdk.java.net/~tschatzl/8180755/webrev.2 (full) > > Thanks, > Thomas > From coleen.phillimore at oracle.com Tue May 23 16:56:58 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 23 May 2017 12:56:58 -0400 Subject: RFR(S): 8139673: NMT stack traces in output should show mtcomponent In-Reply-To: <95dba880-4d17-7fad-c700-d540bfb0c48f@redhat.com> References: <8f6634ad-b8af-e50b-c980-7edf1a07c0f0@redhat.com> <1ee9293f-91bb-ff6f-2335-db0b0cca6d1a@redhat.com> <7357dd9b-18f0-e931-05e2-f67e716d6128@redhat.com> <95dba880-4d17-7fad-c700-d540bfb0c48f@redhat.com> Message-ID: Zhengyu, Thank you for completing this change! I'll sponsor it and make both you and Max authors, and add myself as reviewer. Thanks, Coleen On 5/23/17 12:05 PM, Zhengyu Gu wrote: > Hi Aleksey, > > Thanks for the review. I fixed double spaces in new webrev: > > http://cr.openjdk.java.net/~zgu/8139673/webrev.02/ > > I do need sponsor the this, and please also add Max Ockner (mockner) > as an author. > > Thanks, > > -Zhengyu > > > On 05/23/2017 11:05 AM, Aleksey Shipilev wrote: >> On 05/23/2017 03:33 PM, Zhengyu Gu wrote: >>> Webrev: http://cr.openjdk.java.net/~zgu/8139673/webrev.01/index.html >> >> Looks good. >> >> Minor nit: there are some double spaces, fix them before pushing? >> >> 71 _malloc_site(stack, flags), _next(NULL) { >> >> ... >> >> 81 bool ret = MallocSiteTable::allocation_at(stack, size, >> bucket_idx, >> pos_idx, flags); >> >> Thanks, >> -Aleksey >> >> >> From robbin.ehn at oracle.com Tue May 23 18:31:35 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 23 May 2017 20:31:35 +0200 Subject: RFR: 8152953: ForceSafepoint operations should be more specific Message-ID: <44fc57e6-d22f-bbca-d262-d8a31f0f60e5@oracle.com> Hi all, please review. The empty ForceSafepoint operation is now specified by a subclass. Webrev: http://cr.openjdk.java.net/~rehn/8152953/ Bug: https://bugs.openjdk.java.net/browse/JDK-8152953 Passes hotspot_tier1 Thanks, Robbin From robbin.ehn at oracle.com Tue May 23 18:32:10 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 23 May 2017 20:32:10 +0200 Subject: RFR: 8152955: Many safepoints of "no vm operation" kind Message-ID: Hi all, please review. When cleanup, safepointalot or vm halt starts a safepoint without an operation we now set a reason string for. Webrev: http://cr.openjdk.java.net/~rehn/8152955/ Bug: https://bugs.openjdk.java.net/browse/JDK-8152955 Passes hotspot_tier1 Thanks, Robbin From calvin.cheung at oracle.com Tue May 23 18:43:03 2017 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Tue, 23 May 2017 11:43:03 -0700 Subject: RFR(S): 8180631: [TESTBUG] CDS tests should use CDSTestUtils.executeAndLog whenever spawning sub processes In-Reply-To: <591E3C8E.8060305@oracle.com> References: <591E3C8E.8060305@oracle.com> Message-ID: <592482B7.5040001@oracle.com> Hi Misha, Just one comment on LimitedSharedSizes.java: 171 try { 172 CDSTestUtils.executeAndLog(pb, "use" + counter) 173 .shouldContain("archive is valid"); Do you need to cache the output in line 172 since you're checking it in the catch block? thanks, Calvin On 5/18/17, 5:30 PM, Mikhailo Seledtsov wrote: > Please review this simple change that uses > CDSTestUtils.executeAndLog() for starting child > processes in CDS tests, in places that did not use this method before. > In short, this change brings > benefits of recording the output of child processes thus aiding in > troubleshooting of failures. > For more details see bug description. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8180631/ > Webrev: http://cr.openjdk.java.net/~mseledtsov/8180631.01/ > Testing: > CDS tests on Linux-x64 (local) - PASS > CDS tests via automated multi-platform test system - in Progress > > Thank you, > Misha From mikhailo.seledtsov at oracle.com Tue May 23 19:30:45 2017 From: mikhailo.seledtsov at oracle.com (Mikhailo Seledtsov) Date: Tue, 23 May 2017 12:30:45 -0700 Subject: RFR(S): 8180631: [TESTBUG] CDS tests should use CDSTestUtils.executeAndLog whenever spawning sub processes In-Reply-To: <592482B7.5040001@oracle.com> References: <591E3C8E.8060305@oracle.com> <592482B7.5040001@oracle.com> Message-ID: <59248DE5.1000804@oracle.com> Hi Calvin, Thank you for review. Good catch. I have updated this code block to properly use the utility method to determine whether the mapping failed: output = CDSTestUtils.executeAndLog(pb, "use" + counter); if(CDSTestUtils.isUnableToMap(output)) { System.out.println("Unable to use shared archive: " + "test not executed; assumed passed"); continue; } else { output.shouldHaveExitValue(0); } Here is the updated webrev: http://cr.openjdk.java.net/~mseledtsov/8180631.05/ Misha On 5/23/17, 11:43 AM, Calvin Cheung wrote: > Hi Misha, > > Just one comment on LimitedSharedSizes.java: > > 171 try { > 172 CDSTestUtils.executeAndLog(pb, "use" + counter) > 173 .shouldContain("archive is valid"); > > Do you need to cache the output in line 172 since you're checking it > in the catch block? > > thanks, > Calvin > > On 5/18/17, 5:30 PM, Mikhailo Seledtsov wrote: >> Please review this simple change that uses >> CDSTestUtils.executeAndLog() for starting child >> processes in CDS tests, in places that did not use this method >> before. In short, this change brings >> benefits of recording the output of child processes thus aiding in >> troubleshooting of failures. >> For more details see bug description. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8180631/ >> Webrev: http://cr.openjdk.java.net/~mseledtsov/8180631.01/ >> Testing: >> CDS tests on Linux-x64 (local) - PASS >> CDS tests via automated multi-platform test system - in Progress >> >> Thank you, >> Misha From coleen.phillimore at oracle.com Tue May 23 20:00:40 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 23 May 2017 16:00:40 -0400 Subject: RFR: 8152953: ForceSafepoint operations should be more specific In-Reply-To: <44fc57e6-d22f-bbca-d262-d8a31f0f60e5@oracle.com> References: <44fc57e6-d22f-bbca-d262-d8a31f0f60e5@oracle.com> Message-ID: <7887a60d-f1b5-50aa-7468-416584822469@oracle.com> Hi Robbin, This looks good. Coleen On 5/23/17 2:31 PM, Robbin Ehn wrote: > Hi all, please review. > > The empty ForceSafepoint operation is now specified by a subclass. > > Webrev: http://cr.openjdk.java.net/~rehn/8152953/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8152953 > > Passes hotspot_tier1 > > Thanks, Robbin From rkennke at redhat.com Tue May 23 20:05:55 2017 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 23 May 2017 22:05:55 +0200 Subject: RFR: 8152955: Many safepoints of "no vm operation" kind In-Reply-To: References: Message-ID: Am 23.05.2017 um 20:32 schrieb Robbin Ehn: > Hi all, please review. > > When cleanup, safepointalot or vm halt starts a safepoint without an > operation we now set a reason string for. > > Webrev: http://cr.openjdk.java.net/~rehn/8152955/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8152955 > > Passes hotspot_tier1 > > Thanks, Robbin This looks reasonable to me. (Not a Reviewer) Roman From rkennke at redhat.com Tue May 23 20:08:34 2017 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 23 May 2017 22:08:34 +0200 Subject: RFR: 8152953: ForceSafepoint operations should be more specific In-Reply-To: <44fc57e6-d22f-bbca-d262-d8a31f0f60e5@oracle.com> References: <44fc57e6-d22f-bbca-d262-d8a31f0f60e5@oracle.com> Message-ID: Am 23.05.2017 um 20:31 schrieb Robbin Ehn: > Hi all, please review. > > The empty ForceSafepoint operation is now specified by a subclass. > > Webrev: http://cr.openjdk.java.net/~rehn/8152953/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8152953 > > Passes hotspot_tier1 > > Thanks, Robbin Looks reasonable to me too. Roman (still not a Reviewer ;-) ) From calvin.cheung at oracle.com Tue May 23 20:34:20 2017 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Tue, 23 May 2017 13:34:20 -0700 Subject: RFR(S): 8180631: [TESTBUG] CDS tests should use CDSTestUtils.executeAndLog whenever spawning sub processes In-Reply-To: <59248DE5.1000804@oracle.com> References: <591E3C8E.8060305@oracle.com> <592482B7.5040001@oracle.com> <59248DE5.1000804@oracle.com> Message-ID: <59249CCC.9060906@oracle.com> Looks good. thanks, Calvin On 5/23/17, 12:30 PM, Mikhailo Seledtsov wrote: > Hi Calvin, > > Thank you for review. Good catch. > > I have updated this code block to properly use the utility method to > determine whether the mapping failed: > > output = CDSTestUtils.executeAndLog(pb, "use" + > counter); > if(CDSTestUtils.isUnableToMap(output)) { > System.out.println("Unable to use shared > archive: " + > "test not executed; > assumed passed"); > continue; > } else { > output.shouldHaveExitValue(0); > } > > Here is the updated webrev: > http://cr.openjdk.java.net/~mseledtsov/8180631.05/ > > Misha > > On 5/23/17, 11:43 AM, Calvin Cheung wrote: >> Hi Misha, >> >> Just one comment on LimitedSharedSizes.java: >> >> 171 try { >> 172 CDSTestUtils.executeAndLog(pb, "use" + counter) >> 173 .shouldContain("archive is valid"); >> >> Do you need to cache the output in line 172 since you're checking it >> in the catch block? >> >> thanks, >> Calvin >> >> On 5/18/17, 5:30 PM, Mikhailo Seledtsov wrote: >>> Please review this simple change that uses >>> CDSTestUtils.executeAndLog() for starting child >>> processes in CDS tests, in places that did not use this method >>> before. In short, this change brings >>> benefits of recording the output of child processes thus aiding in >>> troubleshooting of failures. >>> For more details see bug description. >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8180631/ >>> Webrev: http://cr.openjdk.java.net/~mseledtsov/8180631.01/ >>> Testing: >>> CDS tests on Linux-x64 (local) - PASS >>> CDS tests via automated multi-platform test system - in >>> Progress >>> >>> Thank you, >>> Misha From mikhailo.seledtsov at oracle.com Tue May 23 21:11:37 2017 From: mikhailo.seledtsov at oracle.com (Mikhailo Seledtsov) Date: Tue, 23 May 2017 14:11:37 -0700 Subject: RFR(S): 8180631: [TESTBUG] CDS tests should use CDSTestUtils.executeAndLog whenever spawning sub processes In-Reply-To: <59249CCC.9060906@oracle.com> References: <591E3C8E.8060305@oracle.com> <592482B7.5040001@oracle.com> <59248DE5.1000804@oracle.com> <59249CCC.9060906@oracle.com> Message-ID: <5924A589.4000701@oracle.com> Thank you, Misha On 5/23/17, 1:34 PM, Calvin Cheung wrote: > Looks good. > > thanks, > Calvin > > On 5/23/17, 12:30 PM, Mikhailo Seledtsov wrote: >> Hi Calvin, >> >> Thank you for review. Good catch. >> >> I have updated this code block to properly use the utility method to >> determine whether the mapping failed: >> >> output = CDSTestUtils.executeAndLog(pb, "use" + >> counter); >> if(CDSTestUtils.isUnableToMap(output)) { >> System.out.println("Unable to use shared >> archive: " + >> "test not executed; >> assumed passed"); >> continue; >> } else { >> output.shouldHaveExitValue(0); >> } >> >> Here is the updated webrev: >> http://cr.openjdk.java.net/~mseledtsov/8180631.05/ >> >> Misha >> >> On 5/23/17, 11:43 AM, Calvin Cheung wrote: >>> Hi Misha, >>> >>> Just one comment on LimitedSharedSizes.java: >>> >>> 171 try { >>> 172 CDSTestUtils.executeAndLog(pb, "use" + counter) >>> 173 .shouldContain("archive is valid"); >>> >>> Do you need to cache the output in line 172 since you're checking it >>> in the catch block? >>> >>> thanks, >>> Calvin >>> >>> On 5/18/17, 5:30 PM, Mikhailo Seledtsov wrote: >>>> Please review this simple change that uses >>>> CDSTestUtils.executeAndLog() for starting child >>>> processes in CDS tests, in places that did not use this method >>>> before. In short, this change brings >>>> benefits of recording the output of child processes thus aiding in >>>> troubleshooting of failures. >>>> For more details see bug description. >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8180631/ >>>> Webrev: http://cr.openjdk.java.net/~mseledtsov/8180631.01/ >>>> Testing: >>>> CDS tests on Linux-x64 (local) - PASS >>>> CDS tests via automated multi-platform test system - in >>>> Progress >>>> >>>> Thank you, >>>> Misha From david.holmes at oracle.com Wed May 24 01:27:11 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 24 May 2017 11:27:11 +1000 Subject: RFR (S): 8180755: Remove use of bitMap.inline.hpp include from instanceKlass.hpp and c1_ValueSet.hpp In-Reply-To: <1495553486.2781.48.camel@oracle.com> References: <1495467593.2573.82.camel@oracle.com> <722e9800-5d1b-bc9e-7bff-16a228d367c2@oracle.com> <1495535811.2781.1.camel@oracle.com> <1ac8dcac-b750-9e60-addc-0f80b2644943@oracle.com> <1495544513.2781.44.camel@oracle.com> <534a0cbf-ad68-f76b-678d-b3764a68c00b@oracle.com> <1495553486.2781.48.camel@oracle.com> Message-ID: On 24/05/2017 1:31 AM, Thomas Schatzl wrote: > Hi Coleen, > > On Tue, 2017-05-23 at 10:15 -0400, coleen.phillimore at oracle.com wrote: >> http://cr.openjdk.java.net/~tschatzl/8180755/webrev.1/src/share/vm/oo >> ps/generateOopMap.cpp.udiff.html >> >> I don't think this should include methodHandles.hpp (I see you've >> rearranged it, can you try to remove it?) Why should it be removed? The file uses things from methodHandles. David >> >> The rest of this looks fine to me. > > Done, still passes build-only jprt. New webrevs: > > http://cr.openjdk.java.net/~tschatzl/8180755/webrev.1_to_2 (diff) > http://cr.openjdk.java.net/~tschatzl/8180755/webrev.2 (full) > > Thanks, > Thomas > From david.holmes at oracle.com Wed May 24 01:44:52 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 24 May 2017 11:44:52 +1000 Subject: RFR(S): 8139673: NMT stack traces in output should show mtcomponent In-Reply-To: <1ee9293f-91bb-ff6f-2335-db0b0cca6d1a@redhat.com> References: <8f6634ad-b8af-e50b-c980-7edf1a07c0f0@redhat.com> <1ee9293f-91bb-ff6f-2335-db0b0cca6d1a@redhat.com> Message-ID: On 23/05/2017 11:33 PM, Zhengyu Gu wrote: > Hi David, > > Thanks for the review. I updated webrev according to your comments. > > Webrev: http://cr.openjdk.java.net/~zgu/8139673/webrev.01/index.html This: if (flag != mtNone) { out->print("(malloc=" SIZE_FORMAT "%s type=%s", amount_in_current_scale(amount), scale, NMTUtil::flag_to_name(flag)); } else { out->print("(malloc=" SIZE_FORMAT "%s", amount_in_current_scale(amount), scale); } could have simply been: out->print("(malloc=" SIZE_FORMAT "%s", amount_in_current_scale(amount), scale); if (flag != mtNone) { out->print(" type=%s", NMTUtil::flag_to_name(flag)); } But I see it has been pushed. Thanks, David ----- > -Zhengyu > > > On 05/22/2017 09:46 PM, David Holmes wrote: >> On 23/05/2017 12:34 AM, Zhengyu Gu wrote: >>> I re-open this bug, as I received a new request for this functionality. >>> >>> The most of work was done by Max Ockner >>> (http://cr.openjdk.java.net/~mockner/8139673/). I only made to a few >>> twists to his patch to create malloc callsite for each memory type, if >>> the callsite is shared by multiple memory types, as Coleen suggested. >>> >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8139673 >>> Webrev: http://cr.openjdk.java.net/~zgu/8139673/webrev.00/ >> >> >> src/share/vm/services/memReporter.cpp >> >> Can print_malloc(size_t amount, size_t count) not delegate to >> print_malloc(size_t amount, size_t count, MEMFLAGS flag) - can we have a >> default value for "flag"? >> >> --- >> >> Otherwise seems okay. Copyright years need updating. >> >> Thanks, >> David >> >>> >>> Thanks, >>> >>> -Zhengyu >>> From david.holmes at oracle.com Wed May 24 03:48:03 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 24 May 2017 13:48:03 +1000 Subject: RFR: 8152953: ForceSafepoint operations should be more specific In-Reply-To: <44fc57e6-d22f-bbca-d262-d8a31f0f60e5@oracle.com> References: <44fc57e6-d22f-bbca-d262-d8a31f0f60e5@oracle.com> Message-ID: Hi Robbin, On 24/05/2017 4:31 AM, Robbin Ehn wrote: > Hi all, please review. > > The empty ForceSafepoint operation is now specified by a subclass. > > Webrev: http://cr.openjdk.java.net/~rehn/8152953/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8152953 Looks good. A few typos in the comments: 259 // empty vm op, when forcing a safepoint due to ctw threshold is reach for sweeper is reach -> is reached for sweeper -> for the sweeper 271 // empty vm op, when forcing a safepoint due to inline cache buffers full either "being full" or "due to full inline cache buffers" 277 // empty vm op, asynchrone forced safepoint for scavenge monitors typo: asynchrone Suggest: // empty asynchronous vm op, when forcing a safepoint to scavenge monitors Thanks, David > Passes hotspot_tier1 > > Thanks, Robbin From david.holmes at oracle.com Wed May 24 04:07:05 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 24 May 2017 14:07:05 +1000 Subject: RFR: 8152955: Many safepoints of "no vm operation" kind In-Reply-To: References: Message-ID: Hi Robbin, On 24/05/2017 4:32 AM, Robbin Ehn wrote: > Hi all, please review. > > When cleanup, safepointalot or vm halt starts a safepoint without an > operation we now set a reason string for. > > Webrev: http://cr.openjdk.java.net/~rehn/8152955/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8152955 This looks good. A couple of minor comments: + bool VMThread::no_op_safepoint_needed(bool check_time) { + _no_op_reason = "no operation"; // This value should never be printed Not sure why you set "no operation". Harmless but ... 396 bool max_time_exceeded = GuaranteedSafepointInterval != 0 && (interval > GuaranteedSafepointInterval); Can you split into two lines please. No need to see update, whatever you choose to do. Thanks, David > Passes hotspot_tier1 > > Thanks, Robbin From thomas.schatzl at oracle.com Wed May 24 06:34:10 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 24 May 2017 08:34:10 +0200 Subject: RFR (S): 8180755: Remove use of bitMap.inline.hpp include from instanceKlass.hpp and c1_ValueSet.hpp In-Reply-To: References: <1495467593.2573.82.camel@oracle.com> <722e9800-5d1b-bc9e-7bff-16a228d367c2@oracle.com> <1495535811.2781.1.camel@oracle.com> <1ac8dcac-b750-9e60-addc-0f80b2644943@oracle.com> <1495544513.2781.44.camel@oracle.com> <534a0cbf-ad68-f76b-678d-b3764a68c00b@oracle.com> <1495553486.2781.48.camel@oracle.com> Message-ID: <1495607650.2894.13.camel@oracle.com> Hi, On Wed, 2017-05-24 at 11:27 +1000, David Holmes wrote: > On 24/05/2017 1:31 AM, Thomas Schatzl wrote: > > > > Hi Coleen, > > > > On Tue, 2017-05-23 at 10:15 -0400, coleen.phillimore at oracle.com > > wrote: > > > > > > http://cr.openjdk.java.net/~tschatzl/8180755/webrev.1/src/share/v > > > m/oo > > > ps/generateOopMap.cpp.udiff.html > > > > > > I don't think this should include methodHandles.hpp (I see you've > > > rearranged it, can you try to remove it?) > Why should it be removed? The file uses things from methodHandles. ? I will revert this change. Thanks for pointing this out. Thanks, ? Thomas From robbin.ehn at oracle.com Wed May 24 07:20:35 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 24 May 2017 09:20:35 +0200 Subject: RFR: 8152955: Many safepoints of "no vm operation" kind In-Reply-To: References: Message-ID: Hi David, thanks for looking at this. On 05/24/2017 06:07 AM, David Holmes wrote: > Hi Robbin, > > On 24/05/2017 4:32 AM, Robbin Ehn wrote: >> Hi all, please review. >> >> When cleanup, safepointalot or vm halt starts a safepoint without an operation we now set a reason string for. >> >> Webrev: http://cr.openjdk.java.net/~rehn/8152955/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8152955 > > This looks good. A couple of minor comments: > > + bool VMThread::no_op_safepoint_needed(bool check_time) { > + _no_op_reason = "no operation"; // This value should never be printed > > Not sure why you set "no operation". Harmless but ... This was more a debug thing, removed. I also removed: -const char* VMThread::_no_op_reason = "no operation"; +const char* VMThread::_no_op_reason = NULL; > > 396 bool max_time_exceeded = GuaranteedSafepointInterval != 0 && (interval > GuaranteedSafepointInterval); > > Can you split into two lines please. Fixed Thanks, Robbin > > No need to see update, whatever you choose to do. > > Thanks, > David > >> Passes hotspot_tier1 >> >> Thanks, Robbin From shade at redhat.com Wed May 24 07:23:36 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 24 May 2017 09:23:36 +0200 Subject: RFR: 8152955: Many safepoints of "no vm operation" kind In-Reply-To: References: Message-ID: <54d90ccc-99f0-c4ef-4706-d18a057f2394@redhat.com> On 05/23/2017 08:32 PM, Robbin Ehn wrote: > Webrev: http://cr.openjdk.java.net/~rehn/8152955/ I like this a lot, "no vm operation" is sure confusing. Minor nit: Space after "if" here: 391 if(!SafepointSynchronize::is_cleanup_needed()) { > Bug: https://bugs.openjdk.java.net/browse/JDK-8152955 Permission violation. Please consider making it public? Seems harmless. Thanks, -Aleksey From robbin.ehn at oracle.com Wed May 24 07:24:25 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 24 May 2017 09:24:25 +0200 Subject: RFR: 8152953: ForceSafepoint operations should be more specific In-Reply-To: References: <44fc57e6-d22f-bbca-d262-d8a31f0f60e5@oracle.com> Message-ID: <0402cd57-5ad2-b756-81b4-68bc2ee3f445@oracle.com> Hi David, fixed the 3 typos with your suggestions. Thanks, Robbin On 05/24/2017 05:48 AM, David Holmes wrote: > Hi Robbin, > > On 24/05/2017 4:31 AM, Robbin Ehn wrote: >> Hi all, please review. >> >> The empty ForceSafepoint operation is now specified by a subclass. >> >> Webrev: http://cr.openjdk.java.net/~rehn/8152953/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8152953 > > Looks good. A few typos in the comments: > > 259 // empty vm op, when forcing a safepoint due to ctw threshold is reach for sweeper > > is reach -> is reached > for sweeper -> for the sweeper > > 271 // empty vm op, when forcing a safepoint due to inline cache buffers full > > either "being full" or "due to full inline cache buffers" > > 277 // empty vm op, asynchrone forced safepoint for scavenge monitors > > typo: asynchrone > > Suggest: > > // empty asynchronous vm op, when forcing a safepoint to scavenge monitors > > Thanks, > David > >> Passes hotspot_tier1 >> >> Thanks, Robbin From robbin.ehn at oracle.com Wed May 24 07:30:52 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 24 May 2017 09:30:52 +0200 Subject: RFR: 8152955: Many safepoints of "no vm operation" kind In-Reply-To: <54d90ccc-99f0-c4ef-4706-d18a057f2394@redhat.com> References: <54d90ccc-99f0-c4ef-4706-d18a057f2394@redhat.com> Message-ID: <87790139-0533-1ecc-3f41-09f21ec0f7f5@oracle.com> Hi Aleksey, On 05/24/2017 09:23 AM, Aleksey Shipilev wrote: > On 05/23/2017 08:32 PM, Robbin Ehn wrote: >> Webrev: http://cr.openjdk.java.net/~rehn/8152955/ > > I like this a lot, "no vm operation" is sure confusing. Great! > > Minor nit: > > Space after "if" here: > 391 if(!SafepointSynchronize::is_cleanup_needed()) { Fixed! > >> Bug: https://bugs.openjdk.java.net/browse/JDK-8152955 > > Permission violation. Please consider making it public? Seems harmless. There is an attachment from customer and I'm not allowed to remove that attachment. A quick scan shows no sensitive information in it, but I might have missed something. Sorry.... Thanks, Robbin > > Thanks, > -Aleksey > > From robbin.ehn at oracle.com Wed May 24 07:31:27 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 24 May 2017 09:31:27 +0200 Subject: RFR: 8152955: Many safepoints of "no vm operation" kind In-Reply-To: References: Message-ID: <5662fef7-4710-61e1-929f-61d5084e77a9@oracle.com> Thanks Roman! /Robbin On 05/23/2017 10:05 PM, Roman Kennke wrote: > Am 23.05.2017 um 20:32 schrieb Robbin Ehn: >> Hi all, please review. >> >> When cleanup, safepointalot or vm halt starts a safepoint without an >> operation we now set a reason string for. >> >> Webrev: http://cr.openjdk.java.net/~rehn/8152955/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8152955 >> >> Passes hotspot_tier1 >> >> Thanks, Robbin > > This looks reasonable to me. > > (Not a Reviewer) > > Roman > From robbin.ehn at oracle.com Wed May 24 07:31:34 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 24 May 2017 09:31:34 +0200 Subject: RFR: 8152953: ForceSafepoint operations should be more specific In-Reply-To: References: <44fc57e6-d22f-bbca-d262-d8a31f0f60e5@oracle.com> Message-ID: Thanks Roman! /Robbin On 05/23/2017 10:08 PM, Roman Kennke wrote: > Am 23.05.2017 um 20:31 schrieb Robbin Ehn: >> Hi all, please review. >> >> The empty ForceSafepoint operation is now specified by a subclass. >> >> Webrev: http://cr.openjdk.java.net/~rehn/8152953/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8152953 >> >> Passes hotspot_tier1 >> >> Thanks, Robbin > > Looks reasonable to me too. > > Roman (still not a Reviewer ;-) ) > From robbin.ehn at oracle.com Wed May 24 07:32:23 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 24 May 2017 09:32:23 +0200 Subject: RFR: 8152953: ForceSafepoint operations should be more specific In-Reply-To: <7887a60d-f1b5-50aa-7468-416584822469@oracle.com> References: <44fc57e6-d22f-bbca-d262-d8a31f0f60e5@oracle.com> <7887a60d-f1b5-50aa-7468-416584822469@oracle.com> Message-ID: Thanks Coleen! /Robbin On 05/23/2017 10:00 PM, coleen.phillimore at oracle.com wrote: > Hi Robbin, > This looks good. > Coleen > > On 5/23/17 2:31 PM, Robbin Ehn wrote: >> Hi all, please review. >> >> The empty ForceSafepoint operation is now specified by a subclass. >> >> Webrev: http://cr.openjdk.java.net/~rehn/8152953/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8152953 >> >> Passes hotspot_tier1 >> >> Thanks, Robbin > From serguei.spitsyn at oracle.com Wed May 24 09:47:00 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 24 May 2017 02:47:00 -0700 Subject: RFR: 8152953: ForceSafepoint operations should be more specific In-Reply-To: <44fc57e6-d22f-bbca-d262-d8a31f0f60e5@oracle.com> References: <44fc57e6-d22f-bbca-d262-d8a31f0f60e5@oracle.com> Message-ID: <6f0b25df-72db-ed7a-0cf0-927f70aed280@oracle.com> Hi Robbin, It looks good modulo David's comments. The bug has to be targeted to 10, not 11. But it should not be a problem. :) Thanks, Serguei On 5/23/17 11:31, Robbin Ehn wrote: > Hi all, please review. > > The empty ForceSafepoint operation is now specified by a subclass. > > Webrev: http://cr.openjdk.java.net/~rehn/8152953/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8152953 > > Passes hotspot_tier1 > > Thanks, Robbin From david.holmes at oracle.com Wed May 24 10:10:31 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 24 May 2017 20:10:31 +1000 Subject: RFR: 8152953: ForceSafepoint operations should be more specific In-Reply-To: <6f0b25df-72db-ed7a-0cf0-927f70aed280@oracle.com> References: <44fc57e6-d22f-bbca-d262-d8a31f0f60e5@oracle.com> <6f0b25df-72db-ed7a-0cf0-927f70aed280@oracle.com> Message-ID: On 24/05/2017 7:47 PM, serguei.spitsyn at oracle.com wrote: > Hi Robbin, > > It looks good modulo David's comments. > The bug has to be targeted to 10, not 11. Good catch. It must be targeted to 10 else a backport to 10 will be created! Thanks, David > But it should not be a problem. :) > > Thanks, > Serguei > > > On 5/23/17 11:31, Robbin Ehn wrote: >> Hi all, please review. >> >> The empty ForceSafepoint operation is now specified by a subclass. >> >> Webrev: http://cr.openjdk.java.net/~rehn/8152953/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8152953 >> >> Passes hotspot_tier1 >> >> Thanks, Robbin > From rkennke at redhat.com Wed May 24 10:43:44 2017 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 24 May 2017 12:43:44 +0200 Subject: RFR: Parallelize safepoint cleanup Message-ID: Some operations in safepoint cleanup have been observed to (sometimes) take significant time. Most notably, idle monitor deflation and nmethod marking stick out in some popular applications and benchmarks. I propose to: - parallelize safepoint cleanup processing - enable to hook up idle monitor deflation and nmethod marking to GC VM ops, if GC can support it (resulting in even more efficient deflation/nmethod marking) In some of my measurements this resulted in much improved pause times. For example, in one popular benchmark on a server-class machine, I got total average pause time down from ~80ms to ~30ms. In none of my measurements has this resulted in decreased performance (although it may be possible to construct something. For example, it may not be worth to spin up worker threads if there's no work to do.) Some implementation notes: I introduced a dedicated worker thread pool in SafepointSynchronize. This is only initialized when -XX:+ParallelSafepointCleanup is enabled, and uses -XX:ParallelSafepointCleanupThreads=X threads, defaulting to 8 threads (just a wild guess, open for discussion. With -XX:-ParallelSafepointCleanup turned off (the default) it will use the old serial safepoint cleanup processing (with optional GC hooks, see below). Parallel processing first lets all worker threads scan threads and thereby deflate idle monitors and mark nmethods (in one pass). The rest of the cleanup work is divided into claimed chunks by using SubTasksDone (like, e.g., in G1RootProcessor). Notice that I tried a bunch of other alternatives: - First I tried to let Java threads deflate their own monitors on safepoint arrival. This did not work out, because deflation (currently) depends on all Java threads having arrived. Adding another sync point there would have defeated the purpose. - Then I tried to always use workers of the current GC. This did not work either, because the GC may be using them, for example if a cleanup safepoint is happening during concurrent marking. - Then I gave SafepointSynchronize its own workers, and I now think this is the best solution: indepdent of the GC and relatively isolated code-wise. The other big thing in this change is the possibility to let the GC take over deflation and nmethod marking. The motivation for this is simple: GCs often scan threads themselves, and when they do, they can just as well also do the deflation and nmethod marking. This is more efficient, because it's better on caches, and because it parallelizes better (other GC workers can do other GC stuff while some are still busy with the threads). Notice that this change only provides convenient APIs for the GCs to consume, but no actual implementation. More specifically: - Deflation of idle monitors can be enabled for GC ops by overriding VM_Operation::deflates_idle_monitors() to return true. This automatically makes Threads::oops_do() and friends to deflate idle monitors. CAUTION: this only works if the GC leaves oop's mark words alone. Unfortunately, I think all GCs currently in OpenJDK preserve the mark word and temporarily use it as forwarding pointer, and thus this optimization is not possible. I have done it successfully in Shenandoah GC. GC devs need to evaluate this. - NMethod marking can be enabled by overriding VM_Operation::marks_nmethods() to return true. In order to mark nmethods during GC thread scanning, one has to call NMethodSweeper::prepare_mark_active_nmethods() and pass the returned CodeBlobClosure to Thread::oops_do() or Threads::possibly_parallel_oops_do(). This is relatively simple and should work for all existing GCs. Again, I have done it successfully in Shenandoah GC. - Hooking up deflation and nmethod marking also works with serial safepoint cleanup. This may be useful for workloads where it's not worth to spin up additional worker threads. They would still benefit from improved cleanup at GC pauses. Webrev: http://cr.openjdk.java.net/~rkennke/8180932/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8180932 Testing: specjvm, specjbb, hotspot_gc I suppose this requires CSR for the new options? Opinions? Roman From robbin.ehn at oracle.com Wed May 24 12:06:37 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 24 May 2017 14:06:37 +0200 Subject: RFR: 8152953: ForceSafepoint operations should be more specific In-Reply-To: <6f0b25df-72db-ed7a-0cf0-927f70aed280@oracle.com> References: <44fc57e6-d22f-bbca-d262-d8a31f0f60e5@oracle.com> <6f0b25df-72db-ed7a-0cf0-927f70aed280@oracle.com> Message-ID: Thanks Serguei! /Robbin On 05/24/2017 11:47 AM, serguei.spitsyn at oracle.com wrote: > Hi Robbin, > > It looks good modulo David's comments. > The bug has to be targeted to 10, not 11. > But it should not be a problem. :) > > Thanks, > Serguei > > > On 5/23/17 11:31, Robbin Ehn wrote: >> Hi all, please review. >> >> The empty ForceSafepoint operation is now specified by a subclass. >> >> Webrev: http://cr.openjdk.java.net/~rehn/8152953/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8152953 >> >> Passes hotspot_tier1 >> >> Thanks, Robbin > From claes.redestad at oracle.com Wed May 24 13:26:36 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 24 May 2017 15:26:36 +0200 Subject: RFR[10]: 8180614: Skip range and constraint checks on non-existent flags In-Reply-To: References: <09110ab8-9538-d8c2-08ab-f8cb145c4595@oracle.com> <591DDCEB.5020800@oracle.com> <80b917d1-3396-1c28-fd28-b0186dd84018@oracle.com> <591F0094.9070401@oracle.com> <25a69081-85a1-758f-9084-a821dc652491@oracle.com> Message-ID: Hi again, On 05/21/2017 11:08 PM, Claes Redestad wrote: >> ps. Claes, did you run >> "hotspot/test/runtime/CommandLine/OptionsValidation? with this change? > > Yes - all pass. I did find another tests that failed, though: hotspot/test/runtime/CompressedOops/ObjectAlignment.java Turns out this was due to me incorrectly checking #ifdef LP64 instead of #ifdef _LP64 I've updated the webrev in place with this typo/bug fixed. Thanks! /Claes From coleen.phillimore at oracle.com Wed May 24 13:28:19 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 24 May 2017 09:28:19 -0400 Subject: RFR (S): 8180755: Remove use of bitMap.inline.hpp include from instanceKlass.hpp and c1_ValueSet.hpp In-Reply-To: <1495607650.2894.13.camel@oracle.com> References: <1495467593.2573.82.camel@oracle.com> <722e9800-5d1b-bc9e-7bff-16a228d367c2@oracle.com> <1495535811.2781.1.camel@oracle.com> <1ac8dcac-b750-9e60-addc-0f80b2644943@oracle.com> <1495544513.2781.44.camel@oracle.com> <534a0cbf-ad68-f76b-678d-b3764a68c00b@oracle.com> <1495553486.2781.48.camel@oracle.com> <1495607650.2894.13.camel@oracle.com> Message-ID: <9bf70b22-109c-99c5-b31f-a91dbc22f306@oracle.com> On 5/24/17 2:34 AM, Thomas Schatzl wrote: > Hi, > > On Wed, 2017-05-24 at 11:27 +1000, David Holmes wrote: >> On 24/05/2017 1:31 AM, Thomas Schatzl wrote: >>> Hi Coleen, >>> >>> On Tue, 2017-05-23 at 10:15 -0400, coleen.phillimore at oracle.com >>> wrote: >>>> http://cr.openjdk.java.net/~tschatzl/8180755/webrev.1/src/share/v >>>> m/oo >>>> ps/generateOopMap.cpp.udiff.html >>>> >>>> I don't think this should include methodHandles.hpp (I see you've >>>> rearranged it, can you try to remove it?) >> Why should it be removed? The file uses things from methodHandles. I'm looking for this, but I don't see it. methodHandles.hpp is for JSR292 MethodHandles not the handle type for Method*. Coleen > I will revert this change. Thanks for pointing this out. > > Thanks, > Thomas > From rkennke at redhat.com Wed May 24 14:40:45 2017 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 24 May 2017 16:40:45 +0200 Subject: RFR: Parallelize safepoint cleanup In-Reply-To: References: Message-ID: <801ce7a8-ff52-a92a-3878-afb30e7e3683@redhat.com> Erik Helin asked me on IRC to trim down the scope of this change and split up the big patch into 3: the parallel cleanup, the GC hookup for deflation, and the GC hookup for nmethods marking. So here comes the first part: http://cr.openjdk.java.net/~rkennke/8180932/webrev.01/ The description for that part still applies. Should be simpler to review this way. Will file 2 more enhancement-bugs for the other two parts. Roman > Some operations in safepoint cleanup have been observed to (sometimes) > take significant time. Most notably, idle monitor deflation and nmethod > marking stick out in some popular applications and benchmarks. > > I propose to: > - parallelize safepoint cleanup processing > - enable to hook up idle monitor deflation and nmethod marking to GC VM > ops, if GC can support it (resulting in even more efficient > deflation/nmethod marking) > > In some of my measurements this resulted in much improved pause times. > For example, in one popular benchmark on a server-class machine, I got > total average pause time down from ~80ms to ~30ms. In none of my > measurements has this resulted in decreased performance (although it may > be possible to construct something. For example, it may not be worth to > spin up worker threads if there's no work to do.) > > Some implementation notes: > > I introduced a dedicated worker thread pool in SafepointSynchronize. > This is only initialized when -XX:+ParallelSafepointCleanup is enabled, > and uses -XX:ParallelSafepointCleanupThreads=X threads, defaulting to 8 > threads (just a wild guess, open for discussion. With > -XX:-ParallelSafepointCleanup turned off (the default) it will use the > old serial safepoint cleanup processing (with optional GC hooks, see below). > > Parallel processing first lets all worker threads scan threads and > thereby deflate idle monitors and mark nmethods (in one pass). The rest > of the cleanup work is divided into claimed chunks by using SubTasksDone > (like, e.g., in G1RootProcessor). > > Notice that I tried a bunch of other alternatives: > > - First I tried to let Java threads deflate their own monitors on > safepoint arrival. This did not work out, because deflation (currently) > depends on all Java threads having arrived. Adding another sync point > there would have defeated the purpose. > > - Then I tried to always use workers of the current GC. This did not > work either, because the GC may be using them, for example if a cleanup > safepoint is happening during concurrent marking. > > - Then I gave SafepointSynchronize its own workers, and I now think this > is the best solution: indepdent of the GC and relatively isolated code-wise. > > > The other big thing in this change is the possibility to let the GC take > over deflation and nmethod marking. The motivation for this is simple: > GCs often scan threads themselves, and when they do, they can just as > well also do the deflation and nmethod marking. This is more efficient, > because it's better on caches, and because it parallelizes better (other > GC workers can do other GC stuff while some are still busy with the > threads). Notice that this change only provides convenient APIs for the > GCs to consume, but no actual implementation. More specifically: > > - Deflation of idle monitors can be enabled for GC ops by overriding > VM_Operation::deflates_idle_monitors() to return true. This > automatically makes Threads::oops_do() and friends to deflate idle > monitors. CAUTION: this only works if the GC leaves oop's mark words > alone. Unfortunately, I think all GCs currently in OpenJDK preserve the > mark word and temporarily use it as forwarding pointer, and thus this > optimization is not possible. I have done it successfully in Shenandoah > GC. GC devs need to evaluate this. > > - NMethod marking can be enabled by overriding > VM_Operation::marks_nmethods() to return true. In order to mark nmethods > during GC thread scanning, one has to call > NMethodSweeper::prepare_mark_active_nmethods() and pass the returned > CodeBlobClosure to Thread::oops_do() or > Threads::possibly_parallel_oops_do(). This is relatively simple and > should work for all existing GCs. Again, I have done it successfully in > Shenandoah GC. > > - Hooking up deflation and nmethod marking also works with serial > safepoint cleanup. This may be useful for workloads where it's not worth > to spin up additional worker threads. They would still benefit from > improved cleanup at GC pauses. > > Webrev: > http://cr.openjdk.java.net/~rkennke/8180932/webrev.00/ > > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8180932 > > Testing: specjvm, specjbb, hotspot_gc > > I suppose this requires CSR for the new options? > > Opinions? > > Roman > From george.triantafillou at oracle.com Wed May 24 15:17:38 2017 From: george.triantafillou at oracle.com (George Triantafillou) Date: Wed, 24 May 2017 11:17:38 -0400 Subject: RFR JDK-8179861: Missing copyrights in some hotspot files Message-ID: Please review this small fix to include missing copyrights in some hotspot files: JBS: https://bugs.openjdk.java.net/browse/JDK-8179861 webrev: http://cr.openjdk.java.net/~gtriantafill/8179861-webrev/webrev/index.html Built and tested on Linux x64. Thanks. -George From george.triantafillou at oracle.com Wed May 24 15:20:00 2017 From: george.triantafillou at oracle.com (George Triantafillou) Date: Wed, 24 May 2017 11:20:00 -0400 Subject: RFR JDK-8180853: Remove bitrotted Windows IA64 code in src/share/vm/oops/oop.inline.hpp Message-ID: Please review this small fix to remove bitrotted Windows IA64 code: JBS: https://bugs.openjdk.java.net/browse/JDK-8180853 webrev: http://cr.openjdk.java.net/~gtriantafill/8180853-webrev/webrev/ Built and tested on Windows x64. Thanks. -George From coleen.phillimore at oracle.com Wed May 24 15:28:37 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 24 May 2017 11:28:37 -0400 Subject: RFR JDK-8180853: Remove bitrotted Windows IA64 code in src/share/vm/oops/oop.inline.hpp In-Reply-To: References: Message-ID: <6b288e52-4e83-2a71-8b86-7d07f5df06c6@oracle.com> Thank you for making this change. It looks good to me. Coleen On 5/24/17 11:20 AM, George Triantafillou wrote: > Please review this small fix to remove bitrotted Windows IA64 code: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8180853 > webrev: > http://cr.openjdk.java.net/~gtriantafill/8180853-webrev/webrev/ > > > Built and tested on Windows x64. > > Thanks. > > -George > From george.triantafillou at oracle.com Wed May 24 15:31:08 2017 From: george.triantafillou at oracle.com (George Triantafillou) Date: Wed, 24 May 2017 11:31:08 -0400 Subject: RFR JDK-8180853: Remove bitrotted Windows IA64 code in src/share/vm/oops/oop.inline.hpp In-Reply-To: <6b288e52-4e83-2a71-8b86-7d07f5df06c6@oracle.com> References: <6b288e52-4e83-2a71-8b86-7d07f5df06c6@oracle.com> Message-ID: <0c7bb849-6d9c-2518-9900-0e2d6285fb4f@oracle.com> Hi Coleen, Thanks for the review. -George On 5/24/2017 11:28 AM, coleen.phillimore at oracle.com wrote: > > Thank you for making this change. It looks good to me. > Coleen > > On 5/24/17 11:20 AM, George Triantafillou wrote: >> Please review this small fix to remove bitrotted Windows IA64 code: >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8180853 >> webrev: >> http://cr.openjdk.java.net/~gtriantafill/8180853-webrev/webrev/ >> >> >> Built and tested on Windows x64. >> >> Thanks. >> >> -George >> > From volker.simonis at gmail.com Wed May 24 15:52:18 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 24 May 2017 17:52:18 +0200 Subject: RFR JDK-8180853: Remove bitrotted Windows IA64 code in src/share/vm/oops/oop.inline.hpp In-Reply-To: References: Message-ID: Hi George, the change looks good. I know we insisted on this code for a long time but we really don't need it any more :) Thank you and best regards, Volker On Wed, May 24, 2017 at 5:20 PM, George Triantafillou wrote: > Please review this small fix to remove bitrotted Windows IA64 code: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8180853 > webrev: http://cr.openjdk.java.net/~gtriantafill/8180853-webrev/webrev/ > > > Built and tested on Windows x64. > > Thanks. > > -George > From coleen.phillimore at oracle.com Wed May 24 15:56:54 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 24 May 2017 11:56:54 -0400 Subject: RFR JDK-8179861: Missing copyrights in some hotspot files In-Reply-To: References: Message-ID: Reviewed, looks good. Thanks, Coleen On 5/24/17 11:17 AM, George Triantafillou wrote: > Please review this small fix to include missing copyrights in some > hotspot files: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8179861 > webrev: > http://cr.openjdk.java.net/~gtriantafill/8179861-webrev/webrev/index.html > > > > Built and tested on Linux x64. > > Thanks. > > -George > From george.triantafillou at oracle.com Wed May 24 15:57:56 2017 From: george.triantafillou at oracle.com (George Triantafillou) Date: Wed, 24 May 2017 11:57:56 -0400 Subject: RFR JDK-8180853: Remove bitrotted Windows IA64 code in src/share/vm/oops/oop.inline.hpp In-Reply-To: References: Message-ID: <2d94b292-0133-f2d7-5359-aece6adf47da@oracle.com> Thanks Volker! -George On 5/24/2017 11:52 AM, Volker Simonis wrote: > Hi George, > > the change looks good. > I know we insisted on this code for a long time but we really don't > need it any more :) > > Thank you and best regards, > Volker > > > On Wed, May 24, 2017 at 5:20 PM, George Triantafillou > wrote: >> Please review this small fix to remove bitrotted Windows IA64 code: >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8180853 >> webrev: http://cr.openjdk.java.net/~gtriantafill/8180853-webrev/webrev/ >> >> >> Built and tested on Windows x64. >> >> Thanks. >> >> -George >> From george.triantafillou at oracle.com Wed May 24 15:58:32 2017 From: george.triantafillou at oracle.com (George Triantafillou) Date: Wed, 24 May 2017 11:58:32 -0400 Subject: RFR JDK-8179861: Missing copyrights in some hotspot files In-Reply-To: References: Message-ID: Thanks Coleen. -George On 5/24/2017 11:56 AM, coleen.phillimore at oracle.com wrote: > Reviewed, looks good. > Thanks, > Coleen > > On 5/24/17 11:17 AM, George Triantafillou wrote: >> Please review this small fix to include missing copyrights in some >> hotspot files: >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8179861 >> webrev: >> http://cr.openjdk.java.net/~gtriantafill/8179861-webrev/webrev/index.html >> >> >> >> Built and tested on Linux x64. >> >> Thanks. >> >> -George >> > From vladimir.kozlov at oracle.com Wed May 24 16:18:11 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 24 May 2017 09:18:11 -0700 Subject: RFR JDK-8180853: Remove bitrotted Windows IA64 code in src/share/vm/oops/oop.inline.hpp In-Reply-To: References: Message-ID: <3b1ffd9b-1c53-6015-3374-4fb12f9b61d3@oracle.com> Finally ;) In such case we should remove all ia64 related code in Hotspot. Thanks, Vladimir On 5/24/17 8:52 AM, Volker Simonis wrote: > Hi George, > > the change looks good. > I know we insisted on this code for a long time but we really don't > need it any more :) > > Thank you and best regards, > Volker > > > On Wed, May 24, 2017 at 5:20 PM, George Triantafillou > wrote: >> Please review this small fix to remove bitrotted Windows IA64 code: >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8180853 >> webrev: http://cr.openjdk.java.net/~gtriantafill/8180853-webrev/webrev/ >> >> >> Built and tested on Windows x64. >> >> Thanks. >> >> -George >> From volker.simonis at gmail.com Wed May 24 16:20:47 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 24 May 2017 18:20:47 +0200 Subject: RFR JDK-8180853: Remove bitrotted Windows IA64 code in src/share/vm/oops/oop.inline.hpp In-Reply-To: <3b1ffd9b-1c53-6015-3374-4fb12f9b61d3@oracle.com> References: <3b1ffd9b-1c53-6015-3374-4fb12f9b61d3@oracle.com> Message-ID: Yes, that would be OK for us. On Wed, May 24, 2017 at 6:18 PM, Vladimir Kozlov wrote: > Finally ;) > > In such case we should remove all ia64 related code in Hotspot. > > Thanks, > Vladimir > > > On 5/24/17 8:52 AM, Volker Simonis wrote: >> >> Hi George, >> >> the change looks good. >> I know we insisted on this code for a long time but we really don't >> need it any more :) >> >> Thank you and best regards, >> Volker >> >> >> On Wed, May 24, 2017 at 5:20 PM, George Triantafillou >> wrote: >>> >>> Please review this small fix to remove bitrotted Windows IA64 code: >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8180853 >>> webrev: http://cr.openjdk.java.net/~gtriantafill/8180853-webrev/webrev/ >>> >>> >>> Built and tested on Windows x64. >>> >>> Thanks. >>> >>> -George >>> > From erik.osterlund at oracle.com Wed May 24 16:46:06 2017 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Wed, 24 May 2017 18:46:06 +0200 Subject: RFR: Parallelize safepoint cleanup In-Reply-To: <801ce7a8-ff52-a92a-3878-afb30e7e3683@redhat.com> References: <801ce7a8-ff52-a92a-3878-afb30e7e3683@redhat.com> Message-ID: <3EB5E102-02B8-469E-B989-52A922140BB3@oracle.com> Hi Roman, Just something to keep in mind when letting the GC steal safepoint cleanup tasks from the safepoint synchronizer: When the cleanup has been skipped, it is crucial that the GC operation indeed does keep to the contract. So if for example a JNI locker blocks the GC, one has to still perform the cleanup tasks before exiting the VM operation. Unless of course the order was inversed so that cleanup runs after the VM operation and could check what is left to be done. But I am not certain what the consequences would be of doing that. Just something to keep in mind. Thanks, /Erik > On 24 May 2017, at 16:40, Roman Kennke wrote: > > Erik Helin asked me on IRC to trim down the scope of this change and > split up the big patch into 3: the parallel cleanup, the GC hookup for > deflation, and the GC hookup for nmethods marking. So here comes the > first part: > > http://cr.openjdk.java.net/~rkennke/8180932/webrev.01/ > > > The description for that part still applies. Should be simpler to review > this way. Will file 2 more enhancement-bugs for the other two parts. > > Roman > >> Some operations in safepoint cleanup have been observed to (sometimes) >> take significant time. Most notably, idle monitor deflation and nmethod >> marking stick out in some popular applications and benchmarks. >> >> I propose to: >> - parallelize safepoint cleanup processing >> - enable to hook up idle monitor deflation and nmethod marking to GC VM >> ops, if GC can support it (resulting in even more efficient >> deflation/nmethod marking) >> >> In some of my measurements this resulted in much improved pause times. >> For example, in one popular benchmark on a server-class machine, I got >> total average pause time down from ~80ms to ~30ms. In none of my >> measurements has this resulted in decreased performance (although it may >> be possible to construct something. For example, it may not be worth to >> spin up worker threads if there's no work to do.) >> >> Some implementation notes: >> >> I introduced a dedicated worker thread pool in SafepointSynchronize. >> This is only initialized when -XX:+ParallelSafepointCleanup is enabled, >> and uses -XX:ParallelSafepointCleanupThreads=X threads, defaulting to 8 >> threads (just a wild guess, open for discussion. With >> -XX:-ParallelSafepointCleanup turned off (the default) it will use the >> old serial safepoint cleanup processing (with optional GC hooks, see below). >> >> Parallel processing first lets all worker threads scan threads and >> thereby deflate idle monitors and mark nmethods (in one pass). The rest >> of the cleanup work is divided into claimed chunks by using SubTasksDone >> (like, e.g., in G1RootProcessor). >> >> Notice that I tried a bunch of other alternatives: >> >> - First I tried to let Java threads deflate their own monitors on >> safepoint arrival. This did not work out, because deflation (currently) >> depends on all Java threads having arrived. Adding another sync point >> there would have defeated the purpose. >> >> - Then I tried to always use workers of the current GC. This did not >> work either, because the GC may be using them, for example if a cleanup >> safepoint is happening during concurrent marking. >> >> - Then I gave SafepointSynchronize its own workers, and I now think this >> is the best solution: indepdent of the GC and relatively isolated code-wise. >> >> >> The other big thing in this change is the possibility to let the GC take >> over deflation and nmethod marking. The motivation for this is simple: >> GCs often scan threads themselves, and when they do, they can just as >> well also do the deflation and nmethod marking. This is more efficient, >> because it's better on caches, and because it parallelizes better (other >> GC workers can do other GC stuff while some are still busy with the >> threads). Notice that this change only provides convenient APIs for the >> GCs to consume, but no actual implementation. More specifically: >> >> - Deflation of idle monitors can be enabled for GC ops by overriding >> VM_Operation::deflates_idle_monitors() to return true. This >> automatically makes Threads::oops_do() and friends to deflate idle >> monitors. CAUTION: this only works if the GC leaves oop's mark words >> alone. Unfortunately, I think all GCs currently in OpenJDK preserve the >> mark word and temporarily use it as forwarding pointer, and thus this >> optimization is not possible. I have done it successfully in Shenandoah >> GC. GC devs need to evaluate this. >> >> - NMethod marking can be enabled by overriding >> VM_Operation::marks_nmethods() to return true. In order to mark nmethods >> during GC thread scanning, one has to call >> NMethodSweeper::prepare_mark_active_nmethods() and pass the returned >> CodeBlobClosure to Thread::oops_do() or >> Threads::possibly_parallel_oops_do(). This is relatively simple and >> should work for all existing GCs. Again, I have done it successfully in >> Shenandoah GC. >> >> - Hooking up deflation and nmethod marking also works with serial >> safepoint cleanup. This may be useful for workloads where it's not worth >> to spin up additional worker threads. They would still benefit from >> improved cleanup at GC pauses. >> >> Webrev: >> http://cr.openjdk.java.net/~rkennke/8180932/webrev.00/ >> >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8180932 >> >> Testing: specjvm, specjbb, hotspot_gc >> >> I suppose this requires CSR for the new options? >> >> Opinions? >> >> Roman >> > From rkennke at redhat.com Wed May 24 17:32:05 2017 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 24 May 2017 19:32:05 +0200 Subject: RFR: Parallelize safepoint cleanup In-Reply-To: <3EB5E102-02B8-469E-B989-52A922140BB3@oracle.com> References: <801ce7a8-ff52-a92a-3878-afb30e7e3683@redhat.com> <3EB5E102-02B8-469E-B989-52A922140BB3@oracle.com> Message-ID: <68ae3346-b2f1-eaad-d013-7677f56ceab9@redhat.com> Am 24.05.2017 um 18:46 schrieb Erik Osterlund: > Hi Roman, > > Just something to keep in mind when letting the GC steal safepoint cleanup tasks from the safepoint synchronizer: > > When the cleanup has been skipped, it is crucial that the GC operation indeed does keep to the contract. So if for example a JNI locker blocks the GC, one has to still perform the cleanup tasks before exiting the VM operation. Unless of course the order was inversed so that cleanup runs after the VM operation and could check what is left to be done. But I am not certain what the consequences would be of doing that. > > Just something to keep in mind. Very good point! Thanks for pointing it out. Luckily (for me), Shenandoah does not need to skip/stall GC due to GC locker ;-) Cheers, Roman > > Thanks, > /Erik > >> On 24 May 2017, at 16:40, Roman Kennke wrote: >> >> Erik Helin asked me on IRC to trim down the scope of this change and >> split up the big patch into 3: the parallel cleanup, the GC hookup for >> deflation, and the GC hookup for nmethods marking. So here comes the >> first part: >> >> http://cr.openjdk.java.net/~rkennke/8180932/webrev.01/ >> >> >> The description for that part still applies. Should be simpler to review >> this way. Will file 2 more enhancement-bugs for the other two parts. >> >> Roman >> >>> Some operations in safepoint cleanup have been observed to (sometimes) >>> take significant time. Most notably, idle monitor deflation and nmethod >>> marking stick out in some popular applications and benchmarks. >>> >>> I propose to: >>> - parallelize safepoint cleanup processing >>> - enable to hook up idle monitor deflation and nmethod marking to GC VM >>> ops, if GC can support it (resulting in even more efficient >>> deflation/nmethod marking) >>> >>> In some of my measurements this resulted in much improved pause times. >>> For example, in one popular benchmark on a server-class machine, I got >>> total average pause time down from ~80ms to ~30ms. In none of my >>> measurements has this resulted in decreased performance (although it may >>> be possible to construct something. For example, it may not be worth to >>> spin up worker threads if there's no work to do.) >>> >>> Some implementation notes: >>> >>> I introduced a dedicated worker thread pool in SafepointSynchronize. >>> This is only initialized when -XX:+ParallelSafepointCleanup is enabled, >>> and uses -XX:ParallelSafepointCleanupThreads=X threads, defaulting to 8 >>> threads (just a wild guess, open for discussion. With >>> -XX:-ParallelSafepointCleanup turned off (the default) it will use the >>> old serial safepoint cleanup processing (with optional GC hooks, see below). >>> >>> Parallel processing first lets all worker threads scan threads and >>> thereby deflate idle monitors and mark nmethods (in one pass). The rest >>> of the cleanup work is divided into claimed chunks by using SubTasksDone >>> (like, e.g., in G1RootProcessor). >>> >>> Notice that I tried a bunch of other alternatives: >>> >>> - First I tried to let Java threads deflate their own monitors on >>> safepoint arrival. This did not work out, because deflation (currently) >>> depends on all Java threads having arrived. Adding another sync point >>> there would have defeated the purpose. >>> >>> - Then I tried to always use workers of the current GC. This did not >>> work either, because the GC may be using them, for example if a cleanup >>> safepoint is happening during concurrent marking. >>> >>> - Then I gave SafepointSynchronize its own workers, and I now think this >>> is the best solution: indepdent of the GC and relatively isolated code-wise. >>> >>> >>> The other big thing in this change is the possibility to let the GC take >>> over deflation and nmethod marking. The motivation for this is simple: >>> GCs often scan threads themselves, and when they do, they can just as >>> well also do the deflation and nmethod marking. This is more efficient, >>> because it's better on caches, and because it parallelizes better (other >>> GC workers can do other GC stuff while some are still busy with the >>> threads). Notice that this change only provides convenient APIs for the >>> GCs to consume, but no actual implementation. More specifically: >>> >>> - Deflation of idle monitors can be enabled for GC ops by overriding >>> VM_Operation::deflates_idle_monitors() to return true. This >>> automatically makes Threads::oops_do() and friends to deflate idle >>> monitors. CAUTION: this only works if the GC leaves oop's mark words >>> alone. Unfortunately, I think all GCs currently in OpenJDK preserve the >>> mark word and temporarily use it as forwarding pointer, and thus this >>> optimization is not possible. I have done it successfully in Shenandoah >>> GC. GC devs need to evaluate this. >>> >>> - NMethod marking can be enabled by overriding >>> VM_Operation::marks_nmethods() to return true. In order to mark nmethods >>> during GC thread scanning, one has to call >>> NMethodSweeper::prepare_mark_active_nmethods() and pass the returned >>> CodeBlobClosure to Thread::oops_do() or >>> Threads::possibly_parallel_oops_do(). This is relatively simple and >>> should work for all existing GCs. Again, I have done it successfully in >>> Shenandoah GC. >>> >>> - Hooking up deflation and nmethod marking also works with serial >>> safepoint cleanup. This may be useful for workloads where it's not worth >>> to spin up additional worker threads. They would still benefit from >>> improved cleanup at GC pauses. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~rkennke/8180932/webrev.00/ >>> >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8180932 >>> >>> Testing: specjvm, specjbb, hotspot_gc >>> >>> I suppose this requires CSR for the new options? >>> >>> Opinions? >>> >>> Roman >>> From rkennke at redhat.com Wed May 24 18:43:10 2017 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 24 May 2017 20:43:10 +0200 Subject: RFR: Parallelize safepoint cleanup In-Reply-To: <3EB5E102-02B8-469E-B989-52A922140BB3@oracle.com> References: <801ce7a8-ff52-a92a-3878-afb30e7e3683@redhat.com> <3EB5E102-02B8-469E-B989-52A922140BB3@oracle.com> Message-ID: <58fddd9b-1598-283c-3259-c264255ced65@redhat.com> Thought about it a little more, and probably warrants some comments: - I am not sure that it 'is crucial'. not deflating idle monitors or marking nmethods is not very nice, but is not catastrophic. It just means we cling to monitors or nmethods longer than we should (and probably induce an artificial cleanup safepoint). That being said, I would not encourage to not clean up if you say you clean up (hey, I have 3 kids ;-) ) - Inversing the order and doing cleanup at the end of the pause is probably not a good idea, at least for deflating monitors. It means the GC would have to scan all those idle monitors that would otherwise have been deflated before. And some programs generate a lot of them! Cheers, Roman > Hi Roman, > > Just something to keep in mind when letting the GC steal safepoint cleanup tasks from the safepoint synchronizer: > > When the cleanup has been skipped, it is crucial that the GC operation indeed does keep to the contract. So if for example a JNI locker blocks the GC, one has to still perform the cleanup tasks before exiting the VM operation. Unless of course the order was inversed so that cleanup runs after the VM operation and could check what is left to be done. But I am not certain what the consequences would be of doing that. > > Just something to keep in mind. > > Thanks, > /Erik > >> On 24 May 2017, at 16:40, Roman Kennke wrote: >> >> Erik Helin asked me on IRC to trim down the scope of this change and >> split up the big patch into 3: the parallel cleanup, the GC hookup for >> deflation, and the GC hookup for nmethods marking. So here comes the >> first part: >> >> http://cr.openjdk.java.net/~rkennke/8180932/webrev.01/ >> >> >> The description for that part still applies. Should be simpler to review >> this way. Will file 2 more enhancement-bugs for the other two parts. >> >> Roman >> >>> Some operations in safepoint cleanup have been observed to (sometimes) >>> take significant time. Most notably, idle monitor deflation and nmethod >>> marking stick out in some popular applications and benchmarks. >>> >>> I propose to: >>> - parallelize safepoint cleanup processing >>> - enable to hook up idle monitor deflation and nmethod marking to GC VM >>> ops, if GC can support it (resulting in even more efficient >>> deflation/nmethod marking) >>> >>> In some of my measurements this resulted in much improved pause times. >>> For example, in one popular benchmark on a server-class machine, I got >>> total average pause time down from ~80ms to ~30ms. In none of my >>> measurements has this resulted in decreased performance (although it may >>> be possible to construct something. For example, it may not be worth to >>> spin up worker threads if there's no work to do.) >>> >>> Some implementation notes: >>> >>> I introduced a dedicated worker thread pool in SafepointSynchronize. >>> This is only initialized when -XX:+ParallelSafepointCleanup is enabled, >>> and uses -XX:ParallelSafepointCleanupThreads=X threads, defaulting to 8 >>> threads (just a wild guess, open for discussion. With >>> -XX:-ParallelSafepointCleanup turned off (the default) it will use the >>> old serial safepoint cleanup processing (with optional GC hooks, see below). >>> >>> Parallel processing first lets all worker threads scan threads and >>> thereby deflate idle monitors and mark nmethods (in one pass). The rest >>> of the cleanup work is divided into claimed chunks by using SubTasksDone >>> (like, e.g., in G1RootProcessor). >>> >>> Notice that I tried a bunch of other alternatives: >>> >>> - First I tried to let Java threads deflate their own monitors on >>> safepoint arrival. This did not work out, because deflation (currently) >>> depends on all Java threads having arrived. Adding another sync point >>> there would have defeated the purpose. >>> >>> - Then I tried to always use workers of the current GC. This did not >>> work either, because the GC may be using them, for example if a cleanup >>> safepoint is happening during concurrent marking. >>> >>> - Then I gave SafepointSynchronize its own workers, and I now think this >>> is the best solution: indepdent of the GC and relatively isolated code-wise. >>> >>> >>> The other big thing in this change is the possibility to let the GC take >>> over deflation and nmethod marking. The motivation for this is simple: >>> GCs often scan threads themselves, and when they do, they can just as >>> well also do the deflation and nmethod marking. This is more efficient, >>> because it's better on caches, and because it parallelizes better (other >>> GC workers can do other GC stuff while some are still busy with the >>> threads). Notice that this change only provides convenient APIs for the >>> GCs to consume, but no actual implementation. More specifically: >>> >>> - Deflation of idle monitors can be enabled for GC ops by overriding >>> VM_Operation::deflates_idle_monitors() to return true. This >>> automatically makes Threads::oops_do() and friends to deflate idle >>> monitors. CAUTION: this only works if the GC leaves oop's mark words >>> alone. Unfortunately, I think all GCs currently in OpenJDK preserve the >>> mark word and temporarily use it as forwarding pointer, and thus this >>> optimization is not possible. I have done it successfully in Shenandoah >>> GC. GC devs need to evaluate this. >>> >>> - NMethod marking can be enabled by overriding >>> VM_Operation::marks_nmethods() to return true. In order to mark nmethods >>> during GC thread scanning, one has to call >>> NMethodSweeper::prepare_mark_active_nmethods() and pass the returned >>> CodeBlobClosure to Thread::oops_do() or >>> Threads::possibly_parallel_oops_do(). This is relatively simple and >>> should work for all existing GCs. Again, I have done it successfully in >>> Shenandoah GC. >>> >>> - Hooking up deflation and nmethod marking also works with serial >>> safepoint cleanup. This may be useful for workloads where it's not worth >>> to spin up additional worker threads. They would still benefit from >>> improved cleanup at GC pauses. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~rkennke/8180932/webrev.00/ >>> >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8180932 >>> >>> Testing: specjvm, specjbb, hotspot_gc >>> >>> I suppose this requires CSR for the new options? >>> >>> Opinions? >>> >>> Roman >>> From erik.osterlund at oracle.com Wed May 24 18:57:35 2017 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Wed, 24 May 2017 20:57:35 +0200 Subject: RFR: Parallelize safepoint cleanup In-Reply-To: <58fddd9b-1598-283c-3259-c264255ced65@redhat.com> References: <801ce7a8-ff52-a92a-3878-afb30e7e3683@redhat.com> <3EB5E102-02B8-469E-B989-52A922140BB3@oracle.com> <58fddd9b-1598-283c-3259-c264255ced65@redhat.com> Message-ID: <434E74E1-8E2F-4FFE-8D92-5274DA6379B6@oracle.com> Hi Roman, > On 24 May 2017, at 20:43, Roman Kennke wrote: > > Thought about it a little more, and probably warrants some comments: > > - I am not sure that it 'is crucial'. not deflating idle monitors or > marking nmethods is not very nice, but is not catastrophic. It just > means we cling to monitors or nmethods longer than we should (and > probably induce an artificial cleanup safepoint). I think the sweeper thread expects marking to be done in the next safepoint. If it isn't done, things will not work. As for deflation, you might be right. Although if ForceSafepoint issued specifically to scavenge monitors is coalesced in the same safepoint as the GC, then I am not 100% convinced things will work properly if it was skipped. It might, but I can't convince myself yet. > That being said, I > would not encourage to not clean up if you say you clean up (hey, I have > 3 kids ;-) ) Wow! > - Inversing the order and doing cleanup at the end of the pause is > probably not a good idea, at least for deflating monitors. It means the > GC would have to scan all those idle monitors that would otherwise have > been deflated before. And some programs generate a lot of them! Well, you did propose to merge them... ;) Then once you get to the end of the safepoint, they are "ticked" as done and skipped! /Erik > Cheers, Roman > >> Hi Roman, >> >> Just something to keep in mind when letting the GC steal safepoint cleanup tasks from the safepoint synchronizer: >> >> When the cleanup has been skipped, it is crucial that the GC operation indeed does keep to the contract. So if for example a JNI locker blocks the GC, one has to still perform the cleanup tasks before exiting the VM operation. Unless of course the order was inversed so that cleanup runs after the VM operation and could check what is left to be done. But I am not certain what the consequences would be of doing that. >> >> Just something to keep in mind. >> >> Thanks, >> /Erik >> >>> On 24 May 2017, at 16:40, Roman Kennke wrote: >>> >>> Erik Helin asked me on IRC to trim down the scope of this change and >>> split up the big patch into 3: the parallel cleanup, the GC hookup for >>> deflation, and the GC hookup for nmethods marking. So here comes the >>> first part: >>> >>> http://cr.openjdk.java.net/~rkennke/8180932/webrev.01/ >>> >>> >>> The description for that part still applies. Should be simpler to review >>> this way. Will file 2 more enhancement-bugs for the other two parts. >>> >>> Roman >>> >>>> Some operations in safepoint cleanup have been observed to (sometimes) >>>> take significant time. Most notably, idle monitor deflation and nmethod >>>> marking stick out in some popular applications and benchmarks. >>>> >>>> I propose to: >>>> - parallelize safepoint cleanup processing >>>> - enable to hook up idle monitor deflation and nmethod marking to GC VM >>>> ops, if GC can support it (resulting in even more efficient >>>> deflation/nmethod marking) >>>> >>>> In some of my measurements this resulted in much improved pause times. >>>> For example, in one popular benchmark on a server-class machine, I got >>>> total average pause time down from ~80ms to ~30ms. In none of my >>>> measurements has this resulted in decreased performance (although it may >>>> be possible to construct something. For example, it may not be worth to >>>> spin up worker threads if there's no work to do.) >>>> >>>> Some implementation notes: >>>> >>>> I introduced a dedicated worker thread pool in SafepointSynchronize. >>>> This is only initialized when -XX:+ParallelSafepointCleanup is enabled, >>>> and uses -XX:ParallelSafepointCleanupThreads=X threads, defaulting to 8 >>>> threads (just a wild guess, open for discussion. With >>>> -XX:-ParallelSafepointCleanup turned off (the default) it will use the >>>> old serial safepoint cleanup processing (with optional GC hooks, see below). >>>> >>>> Parallel processing first lets all worker threads scan threads and >>>> thereby deflate idle monitors and mark nmethods (in one pass). The rest >>>> of the cleanup work is divided into claimed chunks by using SubTasksDone >>>> (like, e.g., in G1RootProcessor). >>>> >>>> Notice that I tried a bunch of other alternatives: >>>> >>>> - First I tried to let Java threads deflate their own monitors on >>>> safepoint arrival. This did not work out, because deflation (currently) >>>> depends on all Java threads having arrived. Adding another sync point >>>> there would have defeated the purpose. >>>> >>>> - Then I tried to always use workers of the current GC. This did not >>>> work either, because the GC may be using them, for example if a cleanup >>>> safepoint is happening during concurrent marking. >>>> >>>> - Then I gave SafepointSynchronize its own workers, and I now think this >>>> is the best solution: indepdent of the GC and relatively isolated code-wise. >>>> >>>> >>>> The other big thing in this change is the possibility to let the GC take >>>> over deflation and nmethod marking. The motivation for this is simple: >>>> GCs often scan threads themselves, and when they do, they can just as >>>> well also do the deflation and nmethod marking. This is more efficient, >>>> because it's better on caches, and because it parallelizes better (other >>>> GC workers can do other GC stuff while some are still busy with the >>>> threads). Notice that this change only provides convenient APIs for the >>>> GCs to consume, but no actual implementation. More specifically: >>>> >>>> - Deflation of idle monitors can be enabled for GC ops by overriding >>>> VM_Operation::deflates_idle_monitors() to return true. This >>>> automatically makes Threads::oops_do() and friends to deflate idle >>>> monitors. CAUTION: this only works if the GC leaves oop's mark words >>>> alone. Unfortunately, I think all GCs currently in OpenJDK preserve the >>>> mark word and temporarily use it as forwarding pointer, and thus this >>>> optimization is not possible. I have done it successfully in Shenandoah >>>> GC. GC devs need to evaluate this. >>>> >>>> - NMethod marking can be enabled by overriding >>>> VM_Operation::marks_nmethods() to return true. In order to mark nmethods >>>> during GC thread scanning, one has to call >>>> NMethodSweeper::prepare_mark_active_nmethods() and pass the returned >>>> CodeBlobClosure to Thread::oops_do() or >>>> Threads::possibly_parallel_oops_do(). This is relatively simple and >>>> should work for all existing GCs. Again, I have done it successfully in >>>> Shenandoah GC. >>>> >>>> - Hooking up deflation and nmethod marking also works with serial >>>> safepoint cleanup. This may be useful for workloads where it's not worth >>>> to spin up additional worker threads. They would still benefit from >>>> improved cleanup at GC pauses. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~rkennke/8180932/webrev.00/ >>>> >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8180932 >>>> >>>> Testing: specjvm, specjbb, hotspot_gc >>>> >>>> I suppose this requires CSR for the new options? >>>> >>>> Opinions? >>>> >>>> Roman >>>> > From kirk at kodewerk.com Thu May 25 05:55:25 2017 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Thu, 25 May 2017 07:55:25 +0200 Subject: RFR: Parallelize safepoint cleanup In-Reply-To: <801ce7a8-ff52-a92a-3878-afb30e7e3683@redhat.com> References: <801ce7a8-ff52-a92a-3878-afb30e7e3683@redhat.com> Message-ID: Hi Roman, I also see opportunities for some wins with this enhancement. >> >> >> >> - Then I gave SafepointSynchronize its own workers, and I now think this >> is the best solution: indepdent of the GC and relatively isolated code-wise. This seems like the best option to me as everyone would benefit irregardless of choice of collector. For example, serial is still by far the best option for smaller devices/heaps. Additionally, I?d be hestiant to place more load on the GC threads. Kind regards, Kirk From david.holmes at oracle.com Thu May 25 07:35:28 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 25 May 2017 17:35:28 +1000 Subject: RFR: Parallelize safepoint cleanup In-Reply-To: <801ce7a8-ff52-a92a-3878-afb30e7e3683@redhat.com> References: <801ce7a8-ff52-a92a-3878-afb30e7e3683@redhat.com> Message-ID: <80ded02f-8475-22a0-9c62-213e82ad453f@oracle.com> Hi Roman, I will look at this when I get time. As always this needs a lot of careful consideration and is not something I would want to rush into. Putting it under an experimental opt-in flag initially would avoid the need for a (possibly premature) CSR request. Thanks, David On 25/05/2017 12:40 AM, Roman Kennke wrote: > Erik Helin asked me on IRC to trim down the scope of this change and > split up the big patch into 3: the parallel cleanup, the GC hookup for > deflation, and the GC hookup for nmethods marking. So here comes the > first part: > > http://cr.openjdk.java.net/~rkennke/8180932/webrev.01/ > > > The description for that part still applies. Should be simpler to review > this way. Will file 2 more enhancement-bugs for the other two parts. > > Roman > >> Some operations in safepoint cleanup have been observed to (sometimes) >> take significant time. Most notably, idle monitor deflation and nmethod >> marking stick out in some popular applications and benchmarks. >> >> I propose to: >> - parallelize safepoint cleanup processing >> - enable to hook up idle monitor deflation and nmethod marking to GC VM >> ops, if GC can support it (resulting in even more efficient >> deflation/nmethod marking) >> >> In some of my measurements this resulted in much improved pause times. >> For example, in one popular benchmark on a server-class machine, I got >> total average pause time down from ~80ms to ~30ms. In none of my >> measurements has this resulted in decreased performance (although it may >> be possible to construct something. For example, it may not be worth to >> spin up worker threads if there's no work to do.) >> >> Some implementation notes: >> >> I introduced a dedicated worker thread pool in SafepointSynchronize. >> This is only initialized when -XX:+ParallelSafepointCleanup is enabled, >> and uses -XX:ParallelSafepointCleanupThreads=X threads, defaulting to 8 >> threads (just a wild guess, open for discussion. With >> -XX:-ParallelSafepointCleanup turned off (the default) it will use the >> old serial safepoint cleanup processing (with optional GC hooks, see below). >> >> Parallel processing first lets all worker threads scan threads and >> thereby deflate idle monitors and mark nmethods (in one pass). The rest >> of the cleanup work is divided into claimed chunks by using SubTasksDone >> (like, e.g., in G1RootProcessor). >> >> Notice that I tried a bunch of other alternatives: >> >> - First I tried to let Java threads deflate their own monitors on >> safepoint arrival. This did not work out, because deflation (currently) >> depends on all Java threads having arrived. Adding another sync point >> there would have defeated the purpose. >> >> - Then I tried to always use workers of the current GC. This did not >> work either, because the GC may be using them, for example if a cleanup >> safepoint is happening during concurrent marking. >> >> - Then I gave SafepointSynchronize its own workers, and I now think this >> is the best solution: indepdent of the GC and relatively isolated code-wise. >> >> >> The other big thing in this change is the possibility to let the GC take >> over deflation and nmethod marking. The motivation for this is simple: >> GCs often scan threads themselves, and when they do, they can just as >> well also do the deflation and nmethod marking. This is more efficient, >> because it's better on caches, and because it parallelizes better (other >> GC workers can do other GC stuff while some are still busy with the >> threads). Notice that this change only provides convenient APIs for the >> GCs to consume, but no actual implementation. More specifically: >> >> - Deflation of idle monitors can be enabled for GC ops by overriding >> VM_Operation::deflates_idle_monitors() to return true. This >> automatically makes Threads::oops_do() and friends to deflate idle >> monitors. CAUTION: this only works if the GC leaves oop's mark words >> alone. Unfortunately, I think all GCs currently in OpenJDK preserve the >> mark word and temporarily use it as forwarding pointer, and thus this >> optimization is not possible. I have done it successfully in Shenandoah >> GC. GC devs need to evaluate this. >> >> - NMethod marking can be enabled by overriding >> VM_Operation::marks_nmethods() to return true. In order to mark nmethods >> during GC thread scanning, one has to call >> NMethodSweeper::prepare_mark_active_nmethods() and pass the returned >> CodeBlobClosure to Thread::oops_do() or >> Threads::possibly_parallel_oops_do(). This is relatively simple and >> should work for all existing GCs. Again, I have done it successfully in >> Shenandoah GC. >> >> - Hooking up deflation and nmethod marking also works with serial >> safepoint cleanup. This may be useful for workloads where it's not worth >> to spin up additional worker threads. They would still benefit from >> improved cleanup at GC pauses. >> >> Webrev: >> http://cr.openjdk.java.net/~rkennke/8180932/webrev.00/ >> >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8180932 >> >> Testing: specjvm, specjbb, hotspot_gc >> >> I suppose this requires CSR for the new options? >> >> Opinions? >> >> Roman >> > From david.holmes at oracle.com Thu May 25 08:01:36 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 25 May 2017 18:01:36 +1000 Subject: RFR (S): 8180755: Remove use of bitMap.inline.hpp include from instanceKlass.hpp and c1_ValueSet.hpp In-Reply-To: <9bf70b22-109c-99c5-b31f-a91dbc22f306@oracle.com> References: <1495467593.2573.82.camel@oracle.com> <722e9800-5d1b-bc9e-7bff-16a228d367c2@oracle.com> <1495535811.2781.1.camel@oracle.com> <1ac8dcac-b750-9e60-addc-0f80b2644943@oracle.com> <1495544513.2781.44.camel@oracle.com> <534a0cbf-ad68-f76b-678d-b3764a68c00b@oracle.com> <1495553486.2781.48.camel@oracle.com> <1495607650.2894.13.camel@oracle.com> <9bf70b22-109c-99c5-b31f-a91dbc22f306@oracle.com> Message-ID: <9f65c230-2f42-c908-8b42-5f3b3a3dad9c@oracle.com> On 24/05/2017 11:28 PM, coleen.phillimore at oracle.com wrote: > > > On 5/24/17 2:34 AM, Thomas Schatzl wrote: >> Hi, >> >> On Wed, 2017-05-24 at 11:27 +1000, David Holmes wrote: >>> On 24/05/2017 1:31 AM, Thomas Schatzl wrote: >>>> Hi Coleen, >>>> >>>> On Tue, 2017-05-23 at 10:15 -0400, coleen.phillimore at oracle.com >>>> wrote: >>>>> http://cr.openjdk.java.net/~tschatzl/8180755/webrev.1/src/share/v >>>>> m/oops/generateOopMap.cpp.udiff.html >>>>> >>>>> I don't think this should include methodHandles.hpp (I see you've >>>>> rearranged it, can you try to remove it?) >>> Why should it be removed? The file uses things from methodHandles. > > I'm looking for this, but I don't see it. methodHandles.hpp is for > JSR292 MethodHandles not the handle type for Method*. Sorry - what terrible naming! David > Coleen > >> I will revert this change. Thanks for pointing this out. >> >> Thanks, >> Thomas >> > From david.holmes at oracle.com Thu May 25 08:38:17 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 25 May 2017 18:38:17 +1000 Subject: RFR JDK-8179861: Missing copyrights in some hotspot files In-Reply-To: References: Message-ID: <3ed6be62-00b0-8b4c-be34-6b55b8ac97b8@oracle.com> Hi George, On 25/05/2017 1:17 AM, George Triantafillou wrote: > Please review this small fix to include missing copyrights in some > hotspot files: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8179861 > webrev: > http://cr.openjdk.java.net/~gtriantafill/8179861-webrev/webrev/index.html test/runtime/6626217/Test6626217.sh You can't put a Java comment block in a shell script. Also note that the last copyright year reflects the most recent substantive change to a file. Adding the copyright notice is not such a change, so these don't all automatically get 2017 as the second year. Cheers, David > > Built and tested on Linux x64. > > Thanks. > > -George > From george.triantafillou at oracle.com Thu May 25 14:49:52 2017 From: george.triantafillou at oracle.com (George Triantafillou) Date: Thu, 25 May 2017 10:49:52 -0400 Subject: RFR JDK-8180853: Remove bitrotted Windows IA64 code in src/share/vm/oops/oop.inline.hpp In-Reply-To: References: <3b1ffd9b-1c53-6015-3374-4fb12f9b61d3@oracle.com> Message-ID: <465c2835-194d-83ea-c973-e3e8d2f21b81@oracle.com> Based on Volker's feedback, I'm withdrawing this fix in its current form. -George On 5/24/2017 12:20 PM, Volker Simonis wrote: > Yes, that would be OK for us. > > On Wed, May 24, 2017 at 6:18 PM, Vladimir Kozlov > wrote: >> Finally ;) >> >> In such case we should remove all ia64 related code in Hotspot. >> >> Thanks, >> Vladimir >> >> >> On 5/24/17 8:52 AM, Volker Simonis wrote: >>> Hi George, >>> >>> the change looks good. >>> I know we insisted on this code for a long time but we really don't >>> need it any more :) >>> >>> Thank you and best regards, >>> Volker >>> >>> >>> On Wed, May 24, 2017 at 5:20 PM, George Triantafillou >>> wrote: >>>> Please review this small fix to remove bitrotted Windows IA64 code: >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8180853 >>>> webrev: http://cr.openjdk.java.net/~gtriantafill/8180853-webrev/webrev/ >>>> >>>> >>>> Built and tested on Windows x64. >>>> >>>> Thanks. >>>> >>>> -George >>>> From george.triantafillou at oracle.com Thu May 25 21:07:26 2017 From: george.triantafillou at oracle.com (George Triantafillou) Date: Thu, 25 May 2017 17:07:26 -0400 Subject: RFR JDK-8179861: Missing copyrights in some hotspot files In-Reply-To: <3ed6be62-00b0-8b4c-be34-6b55b8ac97b8@oracle.com> References: <3ed6be62-00b0-8b4c-be34-6b55b8ac97b8@oracle.com> Message-ID: <68651c85-d827-56e1-c7e0-071e17a59f95@oracle.com> Hi David, Here's a new webrev addressing the issues you raised: http://cr.openjdk.java.net/~gtriantafill/8179861-webrev/webrev.01/index.html Comments inline: On 5/25/2017 4:38 AM, David Holmes wrote: > Hi George, > > On 25/05/2017 1:17 AM, George Triantafillou wrote: >> Please review this small fix to include missing copyrights in some >> hotspot files: >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8179861 >> webrev: >> http://cr.openjdk.java.net/~gtriantafill/8179861-webrev/webrev/index.html >> > > > test/runtime/6626217/Test6626217.sh > > You can't put a Java comment block in a shell script. That change was inadvertent and has been removed. > > Also note that the last copyright year reflects the most recent > substantive change to a file. Adding the copyright notice is not such > a change, so these don't all automatically get 2017 as the second year. All copyrights have been updated with the dates from hg log. Thanks for the review. -George > > Cheers, > David > >> >> Built and tested on Linux x64. >> >> Thanks. >> >> -George >> From serguei.spitsyn at oracle.com Fri May 26 00:08:07 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 25 May 2017 17:08:07 -0700 Subject: RFR JDK-8179861: Missing copyrights in some hotspot files In-Reply-To: <68651c85-d827-56e1-c7e0-071e17a59f95@oracle.com> References: <3ed6be62-00b0-8b4c-be34-6b55b8ac97b8@oracle.com> <68651c85-d827-56e1-c7e0-071e17a59f95@oracle.com> Message-ID: <6e40e52d-a992-01fe-c99d-88d41463be54@oracle.com> Hi George, I've compared the hg logs with the copyright dates. It looks good in general. A question is about two files in: src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/utilities They both have one changeset dated by 2015 but the copyright headers have also the 2012 year. Should they have just 2015 or I'm miss something? Thanks, Serguei On 5/25/17 14:07, George Triantafillou wrote: > Hi David, > > Here's a new webrev addressing the issues you raised: > > http://cr.openjdk.java.net/~gtriantafill/8179861-webrev/webrev.01/index.html > > > > Comments inline: > > On 5/25/2017 4:38 AM, David Holmes wrote: >> Hi George, >> >> On 25/05/2017 1:17 AM, George Triantafillou wrote: >>> Please review this small fix to include missing copyrights in some >>> hotspot files: >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8179861 >>> webrev: >>> http://cr.openjdk.java.net/~gtriantafill/8179861-webrev/webrev/index.html >>> >> >> >> >> test/runtime/6626217/Test6626217.sh >> >> You can't put a Java comment block in a shell script. > That change was inadvertent and has been removed. >> >> Also note that the last copyright year reflects the most recent >> substantive change to a file. Adding the copyright notice is not such >> a change, so these don't all automatically get 2017 as the second year. > All copyrights have been updated with the dates from hg log. > > Thanks for the review. > > -George >> >> Cheers, >> David >> >>> >>> Built and tested on Linux x64. >>> >>> Thanks. >>> >>> -George >>> > From david.holmes at oracle.com Fri May 26 00:31:26 2017 From: david.holmes at oracle.com (David Holmes) Date: Fri, 26 May 2017 10:31:26 +1000 Subject: RFR JDK-8179861: Missing copyrights in some hotspot files In-Reply-To: <6e40e52d-a992-01fe-c99d-88d41463be54@oracle.com> References: <3ed6be62-00b0-8b4c-be34-6b55b8ac97b8@oracle.com> <68651c85-d827-56e1-c7e0-071e17a59f95@oracle.com> <6e40e52d-a992-01fe-c99d-88d41463be54@oracle.com> Message-ID: <31d6bb06-3d23-51f8-09b0-3ad238c08db4@oracle.com> Hi Serguei, On 26/05/2017 10:08 AM, serguei.spitsyn at oracle.com wrote: > Hi George, > > I've compared the hg logs with the copyright dates. > It looks good in general. > > A question is about two files in: > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/utilities > > They both have one changeset dated by 2015 but the copyright headers > have also the 2012 year. > Should they have just 2015 or I'm miss something? These files were added in 2012 by: 6964458: Reimplement class meta-data storage to use native memory ? eg. for IntArray ... Changeset: 3602 (da91efe96a93) 6964458: Reimplement class meta-data storage to use native memory ? User: coleenp Date: 2012-09-01 13:25:18 -0400 (2012-09-01) Parent: 3601 (36d1d483d5d6) 7195615: new hotspot build - hs25-b01 ? Child: 3603 (03049e0e8544) 7195823: NPG: CMS reserved() doesn't match _rs.base(). ? David ----- > Thanks, > Serguei > > > > On 5/25/17 14:07, George Triantafillou wrote: >> Hi David, >> >> Here's a new webrev addressing the issues you raised: >> >> http://cr.openjdk.java.net/~gtriantafill/8179861-webrev/webrev.01/index.html >> >> >> >> Comments inline: >> >> On 5/25/2017 4:38 AM, David Holmes wrote: >>> Hi George, >>> >>> On 25/05/2017 1:17 AM, George Triantafillou wrote: >>>> Please review this small fix to include missing copyrights in some >>>> hotspot files: >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8179861 >>>> webrev: >>>> http://cr.openjdk.java.net/~gtriantafill/8179861-webrev/webrev/index.html >>>> >>> >>> >>> >>> >>> test/runtime/6626217/Test6626217.sh >>> >>> You can't put a Java comment block in a shell script. >> That change was inadvertent and has been removed. >>> >>> Also note that the last copyright year reflects the most recent >>> substantive change to a file. Adding the copyright notice is not such >>> a change, so these don't all automatically get 2017 as the second year. >> All copyrights have been updated with the dates from hg log. >> >> Thanks for the review. >> >> -George >>> >>> Cheers, >>> David >>> >>>> >>>> Built and tested on Linux x64. >>>> >>>> Thanks. >>>> >>>> -George >>>> >> > From david.holmes at oracle.com Fri May 26 00:33:34 2017 From: david.holmes at oracle.com (David Holmes) Date: Fri, 26 May 2017 10:33:34 +1000 Subject: RFR JDK-8179861: Missing copyrights in some hotspot files In-Reply-To: <68651c85-d827-56e1-c7e0-071e17a59f95@oracle.com> References: <3ed6be62-00b0-8b4c-be34-6b55b8ac97b8@oracle.com> <68651c85-d827-56e1-c7e0-071e17a59f95@oracle.com> Message-ID: <79fb6b5e-4641-e2ba-69c8-e88f921c3048@oracle.com> Hi George, On 26/05/2017 7:07 AM, George Triantafillou wrote: > Hi David, > > Here's a new webrev addressing the issues you raised: > > http://cr.openjdk.java.net/~gtriantafill/8179861-webrev/webrev.01/index.html > Looks fine. Thanks, David > > Comments inline: > > On 5/25/2017 4:38 AM, David Holmes wrote: >> Hi George, >> >> On 25/05/2017 1:17 AM, George Triantafillou wrote: >>> Please review this small fix to include missing copyrights in some >>> hotspot files: >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8179861 >>> webrev: >>> http://cr.openjdk.java.net/~gtriantafill/8179861-webrev/webrev/index.html >>> >> >> >> >> test/runtime/6626217/Test6626217.sh >> >> You can't put a Java comment block in a shell script. > That change was inadvertent and has been removed. >> >> Also note that the last copyright year reflects the most recent >> substantive change to a file. Adding the copyright notice is not such >> a change, so these don't all automatically get 2017 as the second year. > All copyrights have been updated with the dates from hg log. > > Thanks for the review. > > -George >> >> Cheers, >> David >> >>> >>> Built and tested on Linux x64. >>> >>> Thanks. >>> >>> -George >>> > From serguei.spitsyn at oracle.com Fri May 26 00:39:50 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 25 May 2017 17:39:50 -0700 Subject: RFR JDK-8179861: Missing copyrights in some hotspot files In-Reply-To: <31d6bb06-3d23-51f8-09b0-3ad238c08db4@oracle.com> References: <3ed6be62-00b0-8b4c-be34-6b55b8ac97b8@oracle.com> <68651c85-d827-56e1-c7e0-071e17a59f95@oracle.com> <6e40e52d-a992-01fe-c99d-88d41463be54@oracle.com> <31d6bb06-3d23-51f8-09b0-3ad238c08db4@oracle.com> Message-ID: Hi David, Ok, thanks! Serguei On 5/25/17 17:31, David Holmes wrote: > Hi Serguei, > > On 26/05/2017 10:08 AM, serguei.spitsyn at oracle.com wrote: >> Hi George, >> >> I've compared the hg logs with the copyright dates. >> It looks good in general. >> >> A question is about two files in: >> src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/utilities >> >> They both have one changeset dated by 2015 but the copyright headers >> have also the 2012 year. >> Should they have just 2015 or I'm miss something? > > These files were added in 2012 by: > > 6964458: Reimplement class meta-data storage to use native memory ? > > eg. for IntArray ... > > Changeset: 3602 (da91efe96a93) 6964458: Reimplement class meta-data > storage to use native memory ? > User: coleenp > Date: 2012-09-01 13:25:18 -0400 (2012-09-01) > Parent: 3601 (36d1d483d5d6) 7195615: new hotspot build - hs25-b01 ? > Child: 3603 (03049e0e8544) 7195823: NPG: CMS reserved() doesn't match > _rs.base(). ? > > David > ----- > > >> Thanks, >> Serguei >> >> >> >> On 5/25/17 14:07, George Triantafillou wrote: >>> Hi David, >>> >>> Here's a new webrev addressing the issues you raised: >>> >>> http://cr.openjdk.java.net/~gtriantafill/8179861-webrev/webrev.01/index.html >>> >>> >>> >>> Comments inline: >>> >>> On 5/25/2017 4:38 AM, David Holmes wrote: >>>> Hi George, >>>> >>>> On 25/05/2017 1:17 AM, George Triantafillou wrote: >>>>> Please review this small fix to include missing copyrights in some >>>>> hotspot files: >>>>> >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8179861 >>>>> webrev: >>>>> http://cr.openjdk.java.net/~gtriantafill/8179861-webrev/webrev/index.html >>>>> >>>> >>>> >>>> >>>> >>>> >>>> test/runtime/6626217/Test6626217.sh >>>> >>>> You can't put a Java comment block in a shell script. >>> That change was inadvertent and has been removed. >>>> >>>> Also note that the last copyright year reflects the most recent >>>> substantive change to a file. Adding the copyright notice is not >>>> such a change, so these don't all automatically get 2017 as the >>>> second year. >>> All copyrights have been updated with the dates from hg log. >>> >>> Thanks for the review. >>> >>> -George >>>> >>>> Cheers, >>>> David >>>> >>>>> >>>>> Built and tested on Linux x64. >>>>> >>>>> Thanks. >>>>> >>>>> -George >>>>> >>> >> From thomas.schatzl at oracle.com Fri May 26 07:57:23 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 26 May 2017 09:57:23 +0200 Subject: RFR (S): 8180755: Remove use of bitMap.inline.hpp include from instanceKlass.hpp and c1_ValueSet.hpp In-Reply-To: <9f65c230-2f42-c908-8b42-5f3b3a3dad9c@oracle.com> References: <1495467593.2573.82.camel@oracle.com> <722e9800-5d1b-bc9e-7bff-16a228d367c2@oracle.com> <1495535811.2781.1.camel@oracle.com> <1ac8dcac-b750-9e60-addc-0f80b2644943@oracle.com> <1495544513.2781.44.camel@oracle.com> <534a0cbf-ad68-f76b-678d-b3764a68c00b@oracle.com> <1495553486.2781.48.camel@oracle.com> <1495607650.2894.13.camel@oracle.com> <9bf70b22-109c-99c5-b31f-a91dbc22f306@oracle.com> <9f65c230-2f42-c908-8b42-5f3b3a3dad9c@oracle.com> Message-ID: <1495785443.3436.2.camel@oracle.com> Hi all, On Thu, 2017-05-25 at 18:01 +1000, David Holmes wrote: > On 24/05/2017 11:28 PM, coleen.phillimore at oracle.com wrote: > > > > On 5/24/17 2:34 AM, Thomas Schatzl wrote: > > > > > > Hi, > > > > > > On Wed, 2017-05-24 at 11:27 +1000, David Holmes wrote: > > > > > > > > On 24/05/2017 1:31 AM, Thomas Schatzl wrote: > > > > > > > > > > Hi Coleen, > > > > > > > > > > On Tue, 2017-05-23 at 10:15 -0400, coleen.phillimore at oracle.c > > > > > om > > > > > wrote: > > > > > > > > > > > > http://cr.openjdk.java.net/~tschatzl/8180755/webrev.1/src/s > > > > > > hare/vm/oops/generateOopMap.cpp.udiff.html > > > > > > > > > > > > I don't think this should include methodHandles.hpp (I see > > > > > > you've > > > > > > rearranged it, can you try to remove it?) > > > > Why should it be removed? The file uses things from > > > > methodHandles. > > I'm looking for this, but I don't see it.???methodHandles.hpp is > > for? > > JSR292 MethodHandles not the handle type for Method*. > Sorry - what terrible naming! > I reinstated the change, updated? http://cr.openjdk.java.net/~tschatzl/8180755/webrev.2 (full) http://cr.openjdk.java.net/~tschatzl/8180755/webrev.1_to_2 (diff) Could I get reviews from the GC team? Thanks, ? Thomas From erik.helin at oracle.com Fri May 26 09:14:23 2017 From: erik.helin at oracle.com (Erik Helin) Date: Fri, 26 May 2017 11:14:23 +0200 Subject: RFR (S): 8180755: Remove use of bitMap.inline.hpp include from instanceKlass.hpp and c1_ValueSet.hpp In-Reply-To: <1495785443.3436.2.camel@oracle.com> References: <1495467593.2573.82.camel@oracle.com> <722e9800-5d1b-bc9e-7bff-16a228d367c2@oracle.com> <1495535811.2781.1.camel@oracle.com> <1ac8dcac-b750-9e60-addc-0f80b2644943@oracle.com> <1495544513.2781.44.camel@oracle.com> <534a0cbf-ad68-f76b-678d-b3764a68c00b@oracle.com> <1495553486.2781.48.camel@oracle.com> <1495607650.2894.13.camel@oracle.com> <9bf70b22-109c-99c5-b31f-a91dbc22f306@oracle.com> <9f65c230-2f42-c908-8b42-5f3b3a3dad9c@oracle.com> <1495785443.3436.2.camel@oracle.com> Message-ID: <72e6091b-0631-0989-f1b1-a37dab0098f4@oracle.com> On 05/26/2017 09:57 AM, Thomas Schatzl wrote: > I reinstated the change, updated > http://cr.openjdk.java.net/~tschatzl/8180755/webrev.2 (full) > http://cr.openjdk.java.net/~tschatzl/8180755/webrev.1_to_2 (diff) > > Could I get reviews from the GC team? The GC parts looks good, thanks for cleaning this up! Erik > Thanks, > Thomas > From thomas.schatzl at oracle.com Fri May 26 11:44:21 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 26 May 2017 13:44:21 +0200 Subject: RFR (S): 8180755: Remove use of bitMap.inline.hpp include from instanceKlass.hpp and c1_ValueSet.hpp In-Reply-To: <72e6091b-0631-0989-f1b1-a37dab0098f4@oracle.com> References: <1495467593.2573.82.camel@oracle.com> <722e9800-5d1b-bc9e-7bff-16a228d367c2@oracle.com> <1495535811.2781.1.camel@oracle.com> <1ac8dcac-b750-9e60-addc-0f80b2644943@oracle.com> <1495544513.2781.44.camel@oracle.com> <534a0cbf-ad68-f76b-678d-b3764a68c00b@oracle.com> <1495553486.2781.48.camel@oracle.com> <1495607650.2894.13.camel@oracle.com> <9bf70b22-109c-99c5-b31f-a91dbc22f306@oracle.com> <9f65c230-2f42-c908-8b42-5f3b3a3dad9c@oracle.com> <1495785443.3436.2.camel@oracle.com> <72e6091b-0631-0989-f1b1-a37dab0098f4@oracle.com> Message-ID: <1495799061.3436.57.camel@oracle.com> Hi Erik (and Coleen and David), On Fri, 2017-05-26 at 11:14 +0200, Erik Helin wrote: > On 05/26/2017 09:57 AM, Thomas Schatzl wrote: > > > > I reinstated the change, updated? > > http://cr.openjdk.java.net/~tschatzl/8180755/webrev.2 (full) > > http://cr.openjdk.java.net/~tschatzl/8180755/webrev.1_to_2 (diff) > > > > Could I get reviews from the GC team? > The GC parts looks good, thanks for cleaning this up! > ? thanks for your reviews. Thomas From george.triantafillou at oracle.com Fri May 26 13:42:35 2017 From: george.triantafillou at oracle.com (George Triantafillou) Date: Fri, 26 May 2017 09:42:35 -0400 Subject: RFR JDK-8179861: Missing copyrights in some hotspot files In-Reply-To: <79fb6b5e-4641-e2ba-69c8-e88f921c3048@oracle.com> References: <3ed6be62-00b0-8b4c-be34-6b55b8ac97b8@oracle.com> <68651c85-d827-56e1-c7e0-071e17a59f95@oracle.com> <79fb6b5e-4641-e2ba-69c8-e88f921c3048@oracle.com> Message-ID: <82a033c4-b539-ef37-c71c-54a26368f166@oracle.com> Thanks David. -George On 5/25/2017 8:33 PM, David Holmes wrote: > Hi George, > > On 26/05/2017 7:07 AM, George Triantafillou wrote: >> Hi David, >> >> Here's a new webrev addressing the issues you raised: >> >> http://cr.openjdk.java.net/~gtriantafill/8179861-webrev/webrev.01/index.html >> > > > Looks fine. > > Thanks, > David > >> >> Comments inline: >> >> On 5/25/2017 4:38 AM, David Holmes wrote: >>> Hi George, >>> >>> On 25/05/2017 1:17 AM, George Triantafillou wrote: >>>> Please review this small fix to include missing copyrights in some >>>> hotspot files: >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8179861 >>>> webrev: >>>> http://cr.openjdk.java.net/~gtriantafill/8179861-webrev/webrev/index.html >>>> >>> >>> >>> >>> >>> test/runtime/6626217/Test6626217.sh >>> >>> You can't put a Java comment block in a shell script. >> That change was inadvertent and has been removed. >>> >>> Also note that the last copyright year reflects the most recent >>> substantive change to a file. Adding the copyright notice is not >>> such a change, so these don't all automatically get 2017 as the >>> second year. >> All copyrights have been updated with the dates from hg log. >> >> Thanks for the review. >> >> -George >>> >>> Cheers, >>> David >>> >>>> >>>> Built and tested on Linux x64. >>>> >>>> Thanks. >>>> >>>> -George >>>> >> From mikael.vidstedt at oracle.com Fri May 26 21:22:34 2017 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Fri, 26 May 2017 14:22:34 -0700 Subject: RFR(S): 8180184: Add DATA and FSIZE to os::Posix::print_rlimit_info Message-ID: <17B78E0D-C38B-4C8C-9F52-FA572563D485@oracle.com> Please review the following fix which adds RLIMIT_DATA and RLIMIT_FISZE to the rlimit related data in the crash dump, and cleans up/unifies some of the related code. Bug: https://bugs.openjdk.java.net/browse/JDK-8180184 Webrev: http://cr.openjdk.java.net/~mikael/webrevs/8180184/webrev.00/hotspot/webrev/ Tested using JPRT. Manually verified that the crash dump contains the expected information. Thanks to Thomas for helping verify that the change works as expected on AIX as well! Cheers, Mikael From david.holmes at oracle.com Mon May 29 06:54:10 2017 From: david.holmes at oracle.com (David Holmes) Date: Mon, 29 May 2017 16:54:10 +1000 Subject: RFR: Parallelize safepoint cleanup In-Reply-To: References: Message-ID: <486b5a72-bef8-4ebc-2729-3fe3aa3ab3b9@oracle.com> Hi Roman, Has a RFE been filed for this yet? On 24/05/2017 8:43 PM, Roman Kennke wrote: > Some operations in safepoint cleanup have been observed to (sometimes) > take significant time. Most notably, idle monitor deflation and nmethod > marking stick out in some popular applications and benchmarks. > > I propose to: > - parallelize safepoint cleanup processing > - enable to hook up idle monitor deflation and nmethod marking to GC VM > ops, if GC can support it (resulting in even more efficient > deflation/nmethod marking) This seems reasonable in principle. Though I do cringe at creating yet another set of VM threads! But as it is opt-in that is not so bad. This will need to be reworked a little to fit in with some recent changes done under: - 8152953: ForceSafepoint operations should be more specific - 8152955: Many safepoints of "no vm operation" kind > In some of my measurements this resulted in much improved pause times. > For example, in one popular benchmark on a server-class machine, I got > total average pause time down from ~80ms to ~30ms. In none of my > measurements has this resulted in decreased performance (although it may > be possible to construct something. For example, it may not be worth to > spin up worker threads if there's no work to do.) The impact on startup time should be measured. For opt-in this is not a big deal, but if this were to be the default someday then it does have to be considered. > Some implementation notes: > > I introduced a dedicated worker thread pool in SafepointSynchronize. > This is only initialized when -XX:+ParallelSafepointCleanup is enabled, > and uses -XX:ParallelSafepointCleanupThreads=X threads, defaulting to 8 > threads (just a wild guess, open for discussion. With > -XX:-ParallelSafepointCleanup turned off (the default) it will use the > old serial safepoint cleanup processing (with optional GC hooks, see below). I think the default needs to be ergonomically set, as for other "thread pools". Both flags should be experimental at this stage. > Parallel processing first lets all worker threads scan threads and > thereby deflate idle monitors and mark nmethods (in one pass). The rest > of the cleanup work is divided into claimed chunks by using SubTasksDone > (like, e.g., in G1RootProcessor). > > Notice that I tried a bunch of other alternatives: > > - First I tried to let Java threads deflate their own monitors on > safepoint arrival. This did not work out, because deflation (currently) > depends on all Java threads having arrived. Adding another sync point > there would have defeated the purpose. Right, deflation by its very nature must occur at a safepoint regardless of which thread(s) actually does the work. > - Then I tried to always use workers of the current GC. This did not > work either, because the GC may be using them, for example if a cleanup > safepoint is happening during concurrent marking. > > - Then I gave SafepointSynchronize its own workers, and I now think this > is the best solution: indepdent of the GC and relatively isolated code-wise. > > > The other big thing in this change is the possibility to let the GC take > over deflation and nmethod marking. The motivation for this is simple: > GCs often scan threads themselves, and when they do, they can just as > well also do the deflation and nmethod marking. This is more efficient, > because it's better on caches, and because it parallelizes better (other > GC workers can do other GC stuff while some are still busy with the > threads). Notice that this change only provides convenient APIs for the > GCs to consume, but no actual implementation. More specifically: > > - Deflation of idle monitors can be enabled for GC ops by overriding > VM_Operation::deflates_idle_monitors() to return true. This > automatically makes Threads::oops_do() and friends to deflate idle > monitors. CAUTION: this only works if the GC leaves oop's mark words > alone. Unfortunately, I think all GCs currently in OpenJDK preserve the > mark word and temporarily use it as forwarding pointer, and thus this > optimization is not possible. I have done it successfully in Shenandoah > GC. GC devs need to evaluate this. > > - NMethod marking can be enabled by overriding > VM_Operation::marks_nmethods() to return true. In order to mark nmethods > during GC thread scanning, one has to call > NMethodSweeper::prepare_mark_active_nmethods() and pass the returned > CodeBlobClosure to Thread::oops_do() or > Threads::possibly_parallel_oops_do(). This is relatively simple and > should work for all existing GCs. Again, I have done it successfully in > Shenandoah GC. > > - Hooking up deflation and nmethod marking also works with serial > safepoint cleanup. This may be useful for workloads where it's not worth > to spin up additional worker threads. They would still benefit from > improved cleanup at GC pauses. That sounds reasonable, but the devil is in the details and the GC (and compiler?) folk would need to give that due consideration. > Webrev: > http://cr.openjdk.java.net/~rkennke/8180932/webrev.00/ > A few specific, minor comments: src/share/vm/runtime/safepoint.cpp 549 // If op does both deflating and nmethod marking, we don't bother firing up 550 // the workers. 551 bool op_does_cleanup = op != NULL && op->marks_nmethods() && op->deflates_idle_monitors(); 552 if (ParallelSafepointCleanup && ! op_does_cleanup) { I'd prefer to see the op_does_cleanup logic evaluated only when ParallelSafepointCleanup is true. --- src/share/vm/runtime/sweeper.cpp 205 // TODO: Is this really needed? 206 OrderAccess::storestore(); I can't see what following store the original code is trying to maintain order with. ?? Compiler folk would need to chime in on that I think. --- src/share/vm/runtime/synchronizer.cpp 1675 if (obj != NULL && cl != NULL) { 1676 cl->do_oop((oop*) mid->object_addr()); 1677 } What is this logic doing? I'm unclear why we need to do something to a monitor we could not deflate. 1796 thread->omInUseCount-= deflated_count; Nit: space needed before -= --- src/share/vm/runtime/vm_operations.hpp 222 // a CodeBlobClosure*, which then needs to be passes as nmethods_cl argument Typo: passes as -> passed as the --- Thanks, David ----- > Bug: > https://bugs.openjdk.java.net/browse/JDK-8180932 > > Testing: specjvm, specjbb, hotspot_gc > > I suppose this requires CSR for the new options? > > Opinions? > > Roman > From david.holmes at oracle.com Mon May 29 07:10:52 2017 From: david.holmes at oracle.com (David Holmes) Date: Mon, 29 May 2017 17:10:52 +1000 Subject: RFR(S): 8180184: Add DATA and FSIZE to os::Posix::print_rlimit_info In-Reply-To: <17B78E0D-C38B-4C8C-9F52-FA572563D485@oracle.com> References: <17B78E0D-C38B-4C8C-9F52-FA572563D485@oracle.com> Message-ID: <92b540d7-dc21-047a-a1e8-8c191a3f7b74@oracle.com> Hi Mikael, Looks okay - good to see the code sharing. I wonder if the C compiler converts x/20124 into x >>10 ? :) Cheers, David On 27/05/2017 7:22 AM, Mikael Vidstedt wrote: > > Please review the following fix which adds RLIMIT_DATA and RLIMIT_FISZE to the rlimit related data in the crash dump, and cleans up/unifies some of the related code. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8180184 > Webrev: http://cr.openjdk.java.net/~mikael/webrevs/8180184/webrev.00/hotspot/webrev/ > > Tested using JPRT. Manually verified that the crash dump contains the expected information. > > Thanks to Thomas for helping verify that the change works as expected on AIX as well! > > Cheers, > Mikael > From david.holmes at oracle.com Mon May 29 07:12:06 2017 From: david.holmes at oracle.com (David Holmes) Date: Mon, 29 May 2017 17:12:06 +1000 Subject: RFR(S): 8180184: Add DATA and FSIZE to os::Posix::print_rlimit_info In-Reply-To: <92b540d7-dc21-047a-a1e8-8c191a3f7b74@oracle.com> References: <17B78E0D-C38B-4C8C-9F52-FA572563D485@oracle.com> <92b540d7-dc21-047a-a1e8-8c191a3f7b74@oracle.com> Message-ID: <9e60e467-6cc1-62a9-f84d-7fc4cd03fc0e@oracle.com> On 29/05/2017 5:10 PM, David Holmes wrote: > Hi Mikael, > > Looks okay - good to see the code sharing. > > I wonder if the C compiler converts x/20124 into x >>10 ? :) Don't know what happened there: x/1024 into x>>10 :) David > Cheers, > David > > On 27/05/2017 7:22 AM, Mikael Vidstedt wrote: >> >> Please review the following fix which adds RLIMIT_DATA and >> RLIMIT_FISZE to the rlimit related data in the crash dump, and cleans >> up/unifies some of the related code. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8180184 >> >> Webrev: >> http://cr.openjdk.java.net/~mikael/webrevs/8180184/webrev.00/hotspot/webrev/ >> >> >> >> Tested using JPRT. Manually verified that the crash dump contains the >> expected information. >> >> Thanks to Thomas for helping verify that the change works as expected >> on AIX as well! >> >> Cheers, >> Mikael >> From rkennke at redhat.com Mon May 29 07:13:03 2017 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 29 May 2017 09:13:03 +0200 Subject: RFR: Parallelize safepoint cleanup In-Reply-To: <486b5a72-bef8-4ebc-2729-3fe3aa3ab3b9@oracle.com> References: <486b5a72-bef8-4ebc-2729-3fe3aa3ab3b9@oracle.com> Message-ID: <94c81c33-6af3-9e06-43a6-0e565e040075@redhat.com> Hi David, > Has a RFE been filed for this yet? https://bugs.openjdk.java.net/browse/JDK-8180932 Also noticed that you reviewed the first version of the patch. Erik Helin asked me to split out the GC related parts and only propose/review the parallel safepoint parts for now: http://cr.openjdk.java.net/~rkennke/8180932/webrev.01/ This makes most of your comments go away for now :-) Let me reply to some of them anyway: > On 24/05/2017 8:43 PM, Roman Kennke wrote: >> Some operations in safepoint cleanup have been observed to (sometimes) >> take significant time. Most notably, idle monitor deflation and nmethod >> marking stick out in some popular applications and benchmarks. >> >> I propose to: >> - parallelize safepoint cleanup processing >> - enable to hook up idle monitor deflation and nmethod marking to GC VM >> ops, if GC can support it (resulting in even more efficient >> deflation/nmethod marking) > > This seems reasonable in principle. Though I do cringe at creating yet > another set of VM threads! But as it is opt-in that is not so bad. > > This will need to be reworked a little to fit in with some recent > changes done under: > - 8152953: ForceSafepoint operations should be more specific > - 8152955: Many safepoints of "no vm operation" kind Ok will do. >> In some of my measurements this resulted in much improved pause times. >> For example, in one popular benchmark on a server-class machine, I got >> total average pause time down from ~80ms to ~30ms. In none of my >> measurements has this resulted in decreased performance (although it may >> be possible to construct something. For example, it may not be worth to >> spin up worker threads if there's no work to do.) > > The impact on startup time should be measured. For opt-in this is not > a big deal, but if this were to be the default someday then it does > have to be considered. Do we have a test for this? Or should I try to fire up HelloWorld 100x and see how well that performs with and without the patch? >> Some implementation notes: >> >> I introduced a dedicated worker thread pool in SafepointSynchronize. >> This is only initialized when -XX:+ParallelSafepointCleanup is enabled, >> and uses -XX:ParallelSafepointCleanupThreads=X threads, defaulting to 8 >> threads (just a wild guess, open for discussion. With >> -XX:-ParallelSafepointCleanup turned off (the default) it will use the >> old serial safepoint cleanup processing (with optional GC hooks, see >> below). > > I think the default needs to be ergonomically set, as for other > "thread pools". Both flags should be experimental at this stage. Ok >> - Then I tried to always use workers of the current GC. This did not >> work either, because the GC may be using them, for example if a cleanup >> safepoint is happening during concurrent marking. >> >> - Then I gave SafepointSynchronize its own workers, and I now think this >> is the best solution: indepdent of the GC and relatively isolated >> code-wise. >> >> >> The other big thing in this change is the possibility to let the GC take >> over deflation and nmethod marking. The motivation for this is simple: >> GCs often scan threads themselves, and when they do, they can just as >> well also do the deflation and nmethod marking. This is more efficient, >> because it's better on caches, and because it parallelizes better (other >> GC workers can do other GC stuff while some are still busy with the >> threads). Notice that this change only provides convenient APIs for the >> GCs to consume, but no actual implementation. More specifically: >> >> - Deflation of idle monitors can be enabled for GC ops by overriding >> VM_Operation::deflates_idle_monitors() to return true. This >> automatically makes Threads::oops_do() and friends to deflate idle >> monitors. CAUTION: this only works if the GC leaves oop's mark words >> alone. Unfortunately, I think all GCs currently in OpenJDK preserve the >> mark word and temporarily use it as forwarding pointer, and thus this >> optimization is not possible. I have done it successfully in Shenandoah >> GC. GC devs need to evaluate this. >> >> - NMethod marking can be enabled by overriding >> VM_Operation::marks_nmethods() to return true. In order to mark nmethods >> during GC thread scanning, one has to call >> NMethodSweeper::prepare_mark_active_nmethods() and pass the returned >> CodeBlobClosure to Thread::oops_do() or >> Threads::possibly_parallel_oops_do(). This is relatively simple and >> should work for all existing GCs. Again, I have done it successfully in >> Shenandoah GC. >> >> - Hooking up deflation and nmethod marking also works with serial >> safepoint cleanup. This may be useful for workloads where it's not worth >> to spin up additional worker threads. They would still benefit from >> improved cleanup at GC pauses. > > That sounds reasonable, but the devil is in the details and the GC > (and compiler?) folk would need to give that due consideration. Those GC related parts are exluded for now, but I'm in contact with them to work out the details. > src/share/vm/runtime/safepoint.cpp > > 549 // If op does both deflating and nmethod marking, we don't > bother firing up > 550 // the workers. > 551 bool op_does_cleanup = op != NULL && op->marks_nmethods() && > op->deflates_idle_monitors(); > 552 if (ParallelSafepointCleanup && ! op_does_cleanup) { > > I'd prefer to see the op_does_cleanup logic evaluated only when > ParallelSafepointCleanup is true. Right. > src/share/vm/runtime/sweeper.cpp > > 205 // TODO: Is this really needed? > 206 OrderAccess::storestore(); > > I can't see what following store the original code is trying to > maintain order with. ?? Compiler folk would need to chime in on that I > think. The only one possibly being affected by this (as far as I can tell) is the NMethodSweeper thread and the compiler threads, and I think both only resume work after the safepoint, so should get a fence() in any case. Right? > --- > > src/share/vm/runtime/synchronizer.cpp > > 1675 if (obj != NULL && cl != NULL) { > 1676 cl->do_oop((oop*) mid->object_addr()); > 1677 } > > What is this logic doing? I'm unclear why we need to do something to a > monitor we could not deflate. This part is out for now, but it's basically a combined deflation/iteration pass over the monitors. It scans all of the thread's monitors and either deflates them or else calls the closure to, e.g., mark its oop (or whatever the GC wants it to do). The idea is that when the GC needs to scan the monitors anyway, we don't need two passes over it, but can deflate and apply the closure in one pass. I will update my patch to latest jdk10-hs and incorporate the changes that you suggested and send the updated patch for review soon. Thanks for reviewing! Roman From thomas.stuefe at gmail.com Mon May 29 08:22:39 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 29 May 2017 10:22:39 +0200 Subject: RFR(S): 8180184: Add DATA and FSIZE to os::Posix::print_rlimit_info In-Reply-To: <17B78E0D-C38B-4C8C-9F52-FA572563D485@oracle.com> References: <17B78E0D-C38B-4C8C-9F52-FA572563D485@oracle.com> Message-ID: Hi Mikael, looks fine. Small nit, we never seem to check the return value of getrlimit(). But seeing that the only way to fail getrlimit() would be to specify an invalid limit constant, maybe this is ok. Best Regards, Thomas On Fri, May 26, 2017 at 11:22 PM, Mikael Vidstedt < mikael.vidstedt at oracle.com> wrote: > > Please review the following fix which adds RLIMIT_DATA and RLIMIT_FISZE to > the rlimit related data in the crash dump, and cleans up/unifies some of > the related code. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8180184 > Webrev: http://cr.openjdk.java.net/~mikael/webrevs/ > 8180184/webrev.00/hotspot/webrev/ > > Tested using JPRT. Manually verified that the crash dump contains the > expected information. > > Thanks to Thomas for helping verify that the change works as expected on > AIX as well! > > Cheers, > Mikael > > From rkennke at redhat.com Mon May 29 08:23:45 2017 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 29 May 2017 10:23:45 +0200 Subject: RFR: Parallelize safepoint cleanup In-Reply-To: <486b5a72-bef8-4ebc-2729-3fe3aa3ab3b9@oracle.com> References: <486b5a72-bef8-4ebc-2729-3fe3aa3ab3b9@oracle.com> Message-ID: Hi David, the patch applied cleanly, i.e. was not affected by any hotspot changes. Most of your suggestions applied to the GC part that I excluded from the scope of this change, except for the following: >> I introduced a dedicated worker thread pool in SafepointSynchronize. >> This is only initialized when -XX:+ParallelSafepointCleanup is enabled, >> and uses -XX:ParallelSafepointCleanupThreads=X threads, defaulting to 8 >> threads (just a wild guess, open for discussion. With >> -XX:-ParallelSafepointCleanup turned off (the default) it will use the >> old serial safepoint cleanup processing (with optional GC hooks, see >> below). > > I think the default needs to be ergonomically set, as for other > "thread pools". Both flags should be experimental at this stage. I implemented this using the following: if (ParallelSafepointCleanup && FLAG_IS_DEFAULT(ParallelSafepointCleanupThreads)) { // This will pick up ParallelGCThreads if set... FLAG_SET_DEFAULT(ParallelSafepointCleanupThreads, Abstract_VM_Version::parallel_worker_threads()); } As noted, it will pick up ParallelGCThreads, which might be a reasonable number. Otherwise we'd need to re-work the logic in vm_version.cpp to calculate the number of worker threads without considering ParallelGCThreads. http://cr.openjdk.java.net/~rkennke/8180932/webrev.02/ What do you think? Roman From david.holmes at oracle.com Mon May 29 08:53:30 2017 From: david.holmes at oracle.com (David Holmes) Date: Mon, 29 May 2017 18:53:30 +1000 Subject: RFR: Parallelize safepoint cleanup In-Reply-To: References: <486b5a72-bef8-4ebc-2729-3fe3aa3ab3b9@oracle.com> Message-ID: <329520f4-4c87-b826-7d9f-6e8adc04f843@oracle.com> Hi Roman, Very quick response... On 29/05/2017 6:23 PM, Roman Kennke wrote: > Hi David, > > the patch applied cleanly, i.e. was not affected by any hotspot changes. I don't think those changes have been pushed yet. David ----- > Most of your suggestions applied to the GC part that I excluded from the > scope of this change, except for the following: > >>> I introduced a dedicated worker thread pool in SafepointSynchronize. >>> This is only initialized when -XX:+ParallelSafepointCleanup is enabled, >>> and uses -XX:ParallelSafepointCleanupThreads=X threads, defaulting to 8 >>> threads (just a wild guess, open for discussion. With >>> -XX:-ParallelSafepointCleanup turned off (the default) it will use the >>> old serial safepoint cleanup processing (with optional GC hooks, see >>> below). >> >> I think the default needs to be ergonomically set, as for other >> "thread pools". Both flags should be experimental at this stage. > > I implemented this using the following: > > if (ParallelSafepointCleanup && > FLAG_IS_DEFAULT(ParallelSafepointCleanupThreads)) { > // This will pick up ParallelGCThreads if set... > FLAG_SET_DEFAULT(ParallelSafepointCleanupThreads, > Abstract_VM_Version::parallel_worker_threads()); > } > > As noted, it will pick up ParallelGCThreads, which might be a reasonable > number. Otherwise we'd need to re-work the logic in vm_version.cpp to > calculate the number of worker threads without considering > ParallelGCThreads. > > http://cr.openjdk.java.net/~rkennke/8180932/webrev.02/ > > > What do you think? > > Roman > From rkennke at redhat.com Mon May 29 10:46:36 2017 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 29 May 2017 12:46:36 +0200 Subject: RFR: Parallelize safepoint cleanup In-Reply-To: <329520f4-4c87-b826-7d9f-6e8adc04f843@oracle.com> References: <486b5a72-bef8-4ebc-2729-3fe3aa3ab3b9@oracle.com> <329520f4-4c87-b826-7d9f-6e8adc04f843@oracle.com> Message-ID: <5812fbba-b3ea-a15d-ad08-810306ab06e8@redhat.com> Am 29.05.2017 um 10:53 schrieb David Holmes: > Hi Roman, > > Very quick response... > > On 29/05/2017 6:23 PM, Roman Kennke wrote: >> Hi David, >> >> the patch applied cleanly, i.e. was not affected by any hotspot changes. > > I don't think those changes have been pushed yet. I just checked. Both: - 8152953: ForceSafepoint operations should be more specific - 8152955: Many safepoints of "no vm operation" kind are in jdk10-hs Roman From kim.barrett at oracle.com Mon May 29 10:47:10 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 29 May 2017 06:47:10 -0400 Subject: RFR: Parallelize safepoint cleanup In-Reply-To: References: <486b5a72-bef8-4ebc-2729-3fe3aa3ab3b9@oracle.com> Message-ID: > On May 29, 2017, at 4:23 AM, Roman Kennke wrote: > I implemented this using the following: > > if (ParallelSafepointCleanup && > FLAG_IS_DEFAULT(ParallelSafepointCleanupThreads)) { > // This will pick up ParallelGCThreads if set... > FLAG_SET_DEFAULT(ParallelSafepointCleanupThreads, > Abstract_VM_Version::parallel_worker_threads()); > } Just following along, and not presently planning to do a real review, but the above caught my eye. Clients of the VM_Version infrastructure should never directly reference Abstract_VM_Version::, as doing so may bypass platform-specific specializations. I thought I?d filed a bug about that sort of thing, but apparently that got pushed deep onto my todo list. I?ll go take care of that now. From david.holmes at oracle.com Mon May 29 11:25:15 2017 From: david.holmes at oracle.com (David Holmes) Date: Mon, 29 May 2017 21:25:15 +1000 Subject: RFR: Parallelize safepoint cleanup In-Reply-To: <5812fbba-b3ea-a15d-ad08-810306ab06e8@redhat.com> References: <486b5a72-bef8-4ebc-2729-3fe3aa3ab3b9@oracle.com> <329520f4-4c87-b826-7d9f-6e8adc04f843@oracle.com> <5812fbba-b3ea-a15d-ad08-810306ab06e8@redhat.com> Message-ID: <6cc3a0de-bf2a-bf83-1d04-5b2b1c2e6947@oracle.com> On 29/05/2017 8:46 PM, Roman Kennke wrote: > Am 29.05.2017 um 10:53 schrieb David Holmes: >> Hi Roman, >> >> Very quick response... >> >> On 29/05/2017 6:23 PM, Roman Kennke wrote: >>> Hi David, >>> >>> the patch applied cleanly, i.e. was not affected by any hotspot changes. >> >> I don't think those changes have been pushed yet. > > I just checked. Both: > > - 8152953: ForceSafepoint operations should be more specific > - 8152955: Many safepoints of "no vm operation" kind > > are in jdk10-hs Okay. I'm surprised there was no conflict with your changes. David ----- > Roman > From david.holmes at oracle.com Mon May 29 11:27:33 2017 From: david.holmes at oracle.com (David Holmes) Date: Mon, 29 May 2017 21:27:33 +1000 Subject: RFR: Parallelize safepoint cleanup In-Reply-To: References: <486b5a72-bef8-4ebc-2729-3fe3aa3ab3b9@oracle.com> Message-ID: <81f51f96-01c7-dafc-9486-ed04932363c1@oracle.com> On 29/05/2017 8:47 PM, Kim Barrett wrote: >> On May 29, 2017, at 4:23 AM, Roman Kennke wrote: >> I implemented this using the following: >> >> if (ParallelSafepointCleanup && >> FLAG_IS_DEFAULT(ParallelSafepointCleanupThreads)) { >> // This will pick up ParallelGCThreads if set... >> FLAG_SET_DEFAULT(ParallelSafepointCleanupThreads, >> Abstract_VM_Version::parallel_worker_threads()); >> } > > Just following along, and not presently planning to do a real review, but the > above caught my eye. Clients of the VM_Version infrastructure should never > directly reference Abstract_VM_Version::, as doing so may bypass > platform-specific specializations. > > I thought I?d filed a bug about that sort of thing, but apparently that got pushed > deep onto my todo list. I?ll go take care of that now. Beat you to it by six years: https://bugs.openjdk.java.net/browse/JDK-7041262 That's a pretty sad statement. Fortunately most things in Abstract_VM_Version are not overridden. David From kim.barrett at oracle.com Mon May 29 12:27:15 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 29 May 2017 08:27:15 -0400 Subject: RFR: Parallelize safepoint cleanup In-Reply-To: <81f51f96-01c7-dafc-9486-ed04932363c1@oracle.com> References: <486b5a72-bef8-4ebc-2729-3fe3aa3ab3b9@oracle.com> <81f51f96-01c7-dafc-9486-ed04932363c1@oracle.com> Message-ID: <672A3CE9-A060-4CB8-A772-87F9308DB9A9@oracle.com> > On May 29, 2017, at 7:27 AM, David Holmes wrote: > > On 29/05/2017 8:47 PM, Kim Barrett wrote: >>> On May 29, 2017, at 4:23 AM, Roman Kennke wrote: >>> I implemented this using the following: >>> >>> if (ParallelSafepointCleanup && >>> FLAG_IS_DEFAULT(ParallelSafepointCleanupThreads)) { >>> // This will pick up ParallelGCThreads if set... >>> FLAG_SET_DEFAULT(ParallelSafepointCleanupThreads, >>> Abstract_VM_Version::parallel_worker_threads()); >>> } >> Just following along, and not presently planning to do a real review, but the >> above caught my eye. Clients of the VM_Version infrastructure should never >> directly reference Abstract_VM_Version::, as doing so may bypass >> platform-specific specializations. >> I thought I?d filed a bug about that sort of thing, but apparently that got pushed >> deep onto my todo list. I?ll go take care of that now. > > Beat you to it by six years: > > https://bugs.openjdk.java.net/browse/JDK-7041262 > > That's a pretty sad statement. Fortunately most things in Abstract_VM_Version are not overridden. > > David I was sure I?d looked for an existing bug, but obviously not! It looks like there are some worms under that rock. The initialize issue looks like a case where there should be more than one stage to the initialization. In fact, there?s already been some work done in that direction, e.g. we now have early_initialize() and init_before_ergo(). I?ll go add some comments to the bug. From robbin.ehn at oracle.com Mon May 29 12:53:22 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Mon, 29 May 2017 14:53:22 +0200 Subject: RFR: Parallelize safepoint cleanup In-Reply-To: References: <486b5a72-bef8-4ebc-2729-3fe3aa3ab3b9@oracle.com> Message-ID: <0ccc1ec4-9866-cd2a-c6bc-a32c2da8cb3d@oracle.com> Hi Roman, On 05/29/2017 10:23 AM, Roman Kennke wrote: > Hi David, > > the patch applied cleanly, i.e. was not affected by any hotspot changes. > Most of your suggestions applied to the GC part that I excluded from the > scope of this change, except for the following: > >>> I introduced a dedicated worker thread pool in SafepointSynchronize. >>> This is only initialized when -XX:+ParallelSafepointCleanup is enabled, >>> and uses -XX:ParallelSafepointCleanupThreads=X threads, defaulting to 8 >>> threads (just a wild guess, open for discussion. With >>> -XX:-ParallelSafepointCleanup turned off (the default) it will use the >>> old serial safepoint cleanup processing (with optional GC hooks, see >>> below). >> >> I think the default needs to be ergonomically set, as for other >> "thread pools". Both flags should be experimental at this stage. > > I implemented this using the following: > > if (ParallelSafepointCleanup && > FLAG_IS_DEFAULT(ParallelSafepointCleanupThreads)) { > // This will pick up ParallelGCThreads if set... > FLAG_SET_DEFAULT(ParallelSafepointCleanupThreads, > Abstract_VM_Version::parallel_worker_threads()); > } > > As noted, it will pick up ParallelGCThreads, which might be a reasonable > number. Otherwise we'd need to re-work the logic in vm_version.cpp to > calculate the number of worker threads without considering > ParallelGCThreads. > > http://cr.openjdk.java.net/~rkennke/8180932/webrev.02/ > > This worries me because heuristics will be very hard to the average user. 1885 if (ParallelSafepointCleanup && FLAG_IS_DEFAULT(ParallelSafepointCleanupThreads)) { 1886 // This will pick up ParallelGCThreads if set... 1887 FLAG_SET_DEFAULT(ParallelSafepointCleanupThreads, Abstract_VM_Version::parallel_worker_threads()); 1888 } So on a 8 vcpu machine we could be running 8 threads cleanup and 8 workers threads might be marking? "and I now think this is the best solution" I don't agree that is the the best solution and I'm not in favor of a second thread pool. If you want short safepoints it seem much better to use available cpu time finishing the cleanup. Maybe steal or reserve worker threads... just keep 1 pool of thread which we can do decent sizing for. "For example, in one popular benchmark on a server-class machine, I got total average pause time down from ~80ms to ~30ms." What's the numbers for a 4 core/8 threads machine? And other GCs? Looked through the patch quick, I think you should do some abstraction. Now this code is already lacking abstraction so I understand why you did it this way. For example: void ObjectSynchronizer::deflate_idle_monitors(bool deflate_thread_local_monitors) ... if (MonitorInUseLists) { if (deflate_thread_local_monitors) { You have mandatory parameter which is ignore by a global flag. Right now there are 4 different cases in the same code path. Does ParallelSafepointCleanup work with -MonitorInUseLists ? Please create some kind of abstraction for this and set it a startup. I do not see any reason for duplicating the SafepointSynchronize::serial_cleanup(), just create proper abstraction so it can be used in both cases? Thanks! /Robbin > What do you think? > > Roman > From erik.osterlund at oracle.com Mon May 29 13:39:14 2017 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 29 May 2017 15:39:14 +0200 Subject: RFR: Parallelize safepoint cleanup In-Reply-To: <0ccc1ec4-9866-cd2a-c6bc-a32c2da8cb3d@oracle.com> References: <486b5a72-bef8-4ebc-2729-3fe3aa3ab3b9@oracle.com> <0ccc1ec4-9866-cd2a-c6bc-a32c2da8cb3d@oracle.com> Message-ID: <592C2482.9020400@oracle.com> Hi, How do we feel about reusing the Parallel GC Threads work gang during safepoint cleanup? We know that the Parallel GC Threads worker gang is idle then. This saves creating another gazillion threads and another JVM flag to keep track of. Thanks, /Erik On 2017-05-29 14:53, Robbin Ehn wrote: > Hi Roman, > > On 05/29/2017 10:23 AM, Roman Kennke wrote: >> Hi David, >> >> the patch applied cleanly, i.e. was not affected by any hotspot changes. >> Most of your suggestions applied to the GC part that I excluded from the >> scope of this change, except for the following: >> >>>> I introduced a dedicated worker thread pool in SafepointSynchronize. >>>> This is only initialized when -XX:+ParallelSafepointCleanup is >>>> enabled, >>>> and uses -XX:ParallelSafepointCleanupThreads=X threads, defaulting >>>> to 8 >>>> threads (just a wild guess, open for discussion. With >>>> -XX:-ParallelSafepointCleanup turned off (the default) it will use the >>>> old serial safepoint cleanup processing (with optional GC hooks, see >>>> below). >>> >>> I think the default needs to be ergonomically set, as for other >>> "thread pools". Both flags should be experimental at this stage. >> >> I implemented this using the following: >> >> if (ParallelSafepointCleanup && >> FLAG_IS_DEFAULT(ParallelSafepointCleanupThreads)) { >> // This will pick up ParallelGCThreads if set... >> FLAG_SET_DEFAULT(ParallelSafepointCleanupThreads, >> Abstract_VM_Version::parallel_worker_threads()); >> } >> >> As noted, it will pick up ParallelGCThreads, which might be a reasonable >> number. Otherwise we'd need to re-work the logic in vm_version.cpp to >> calculate the number of worker threads without considering >> ParallelGCThreads. >> >> http://cr.openjdk.java.net/~rkennke/8180932/webrev.02/ >> >> > > This worries me because heuristics will be very hard to the average user. > > 1885 if (ParallelSafepointCleanup && > FLAG_IS_DEFAULT(ParallelSafepointCleanupThreads)) { > 1886 // This will pick up ParallelGCThreads if set... > 1887 FLAG_SET_DEFAULT(ParallelSafepointCleanupThreads, > Abstract_VM_Version::parallel_worker_threads()); > 1888 } > > So on a 8 vcpu machine we could be running 8 threads cleanup and 8 > workers threads might be marking? > > "and I now think this is the best solution" > > I don't agree that is the the best solution and I'm not in favor of a > second thread pool. > If you want short safepoints it seem much better to use available cpu > time finishing the cleanup. > Maybe steal or reserve worker threads... just keep 1 pool of thread > which we can do decent sizing for. > > "For example, in one popular benchmark on a server-class machine, I got > total average pause time down from ~80ms to ~30ms." > > What's the numbers for a 4 core/8 threads machine? And other GCs? > > Looked through the patch quick, I think you should do some abstraction. > Now this code is already lacking abstraction so I understand why you > did it this way. > > For example: > void ObjectSynchronizer::deflate_idle_monitors(bool > deflate_thread_local_monitors) > ... > if (MonitorInUseLists) { > if (deflate_thread_local_monitors) { > > You have mandatory parameter which is ignore by a global flag. > Right now there are 4 different cases in the same code path. > Does ParallelSafepointCleanup work with -MonitorInUseLists ? > Please create some kind of abstraction for this and set it a startup. > > I do not see any reason for duplicating the > SafepointSynchronize::serial_cleanup(), just create proper abstraction > so it can be used in both cases? > > Thanks! > > /Robbin > >> What do you think? >> >> Roman >> From rkennke at redhat.com Mon May 29 14:08:27 2017 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 29 May 2017 16:08:27 +0200 Subject: RFR: Parallelize safepoint cleanup In-Reply-To: <592C2482.9020400@oracle.com> References: <486b5a72-bef8-4ebc-2729-3fe3aa3ab3b9@oracle.com> <0ccc1ec4-9866-cd2a-c6bc-a32c2da8cb3d@oracle.com> <592C2482.9020400@oracle.com> Message-ID: <05a8fb6c-6c49-b5aa-b8ba-38166ae09d6d@redhat.com> Am 29.05.2017 um 15:39 schrieb Erik ?sterlund: > Hi, > > How do we feel about reusing the Parallel GC Threads work gang during > safepoint cleanup? I actually tried that, but failed, at least with Shenandoah, because... > We know that the Parallel GC Threads worker gang is idle then. Not Shenandoah :-) Shenandoah GC threads are currently happy to continue working during (non-GC) safepoints. One exception is the heap dumper, which currently requires some extra synchronization. If you know of any other safepoints that would require stopping concurrent GC threads (marking, evacuating or updating references) let me know. I haven't found any. Parallelizing safepoint cleanup might be a reason to implement GC thread safepointing though. Another problem with this approach is that it's clearly GC dependend, and requires support (i.e. a worker thread pool) and coordination (i.e. pause-resume GC work) by the GC. For example, as Kirk pointed out, serial gc couldn't support that. > This saves creating another gazillion threads and another JVM flag to > keep track of. It would only be a few threads. There's just not that much work to do. The prime scenario for this seems to be many- (like hundreds or thousands of) threads work loads, in which case it hardly matters to add some more to support more efficient cleanup. Roman From rkennke at redhat.com Mon May 29 14:16:16 2017 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 29 May 2017 16:16:16 +0200 Subject: RFR: Parallelize safepoint cleanup In-Reply-To: <0ccc1ec4-9866-cd2a-c6bc-a32c2da8cb3d@oracle.com> References: <486b5a72-bef8-4ebc-2729-3fe3aa3ab3b9@oracle.com> <0ccc1ec4-9866-cd2a-c6bc-a32c2da8cb3d@oracle.com> Message-ID: <9c019b11-0c78-2649-d3bd-cd02fd999e68@redhat.com> Am 29.05.2017 um 14:53 schrieb Robbin Ehn: > Hi Roman, > > On 05/29/2017 10:23 AM, Roman Kennke wrote: >> Hi David, >> >> the patch applied cleanly, i.e. was not affected by any hotspot changes. >> Most of your suggestions applied to the GC part that I excluded from the >> scope of this change, except for the following: >> >>>> I introduced a dedicated worker thread pool in SafepointSynchronize. >>>> This is only initialized when -XX:+ParallelSafepointCleanup is >>>> enabled, >>>> and uses -XX:ParallelSafepointCleanupThreads=X threads, defaulting >>>> to 8 >>>> threads (just a wild guess, open for discussion. With >>>> -XX:-ParallelSafepointCleanup turned off (the default) it will use the >>>> old serial safepoint cleanup processing (with optional GC hooks, see >>>> below). >>> >>> I think the default needs to be ergonomically set, as for other >>> "thread pools". Both flags should be experimental at this stage. >> >> I implemented this using the following: >> >> if (ParallelSafepointCleanup && >> FLAG_IS_DEFAULT(ParallelSafepointCleanupThreads)) { >> // This will pick up ParallelGCThreads if set... >> FLAG_SET_DEFAULT(ParallelSafepointCleanupThreads, >> Abstract_VM_Version::parallel_worker_threads()); >> } >> >> As noted, it will pick up ParallelGCThreads, which might be a reasonable >> number. Otherwise we'd need to re-work the logic in vm_version.cpp to >> calculate the number of worker threads without considering >> ParallelGCThreads. >> >> http://cr.openjdk.java.net/~rkennke/8180932/webrev.02/ >> >> > > This worries me because heuristics will be very hard to the average user. > > 1885 if (ParallelSafepointCleanup && > FLAG_IS_DEFAULT(ParallelSafepointCleanupThreads)) { > 1886 // This will pick up ParallelGCThreads if set... > 1887 FLAG_SET_DEFAULT(ParallelSafepointCleanupThreads, > Abstract_VM_Version::parallel_worker_threads()); > 1888 } > > So on a 8 vcpu machine we could be running 8 threads cleanup and 8 > workers threads might be marking? > > "and I now think this is the best solution" > > I don't agree that is the the best solution and I'm not in favor of a > second thread pool. > If you want short safepoints it seem much better to use available cpu > time finishing the cleanup. > Maybe steal or reserve worker threads... just keep 1 pool of thread > which we can do decent sizing for. > > "For example, in one popular benchmark on a server-class machine, I got > total average pause time down from ~80ms to ~30ms." > > What's the numbers for a 4 core/8 threads machine? And other GCs? There are workloads and machine configs that are not thread-heavy, and in no case have I seen regressions in pause time. I agree that having a single pool would be good. The current WorkGang doesn't do it though, because we can't borrow threads while the GC is doing work, at least not in a way that is GC agnostic (see my reply to Robbin). > Looked through the patch quick, I think you should do some abstraction. > Now this code is already lacking abstraction so I understand why you > did it this way. > > For example: > void ObjectSynchronizer::deflate_idle_monitors(bool > deflate_thread_local_monitors) > ... > if (MonitorInUseLists) { > if (deflate_thread_local_monitors) { > > You have mandatory parameter which is ignore by a global flag. > Right now there are 4 different cases in the same code path. > Does ParallelSafepointCleanup work with -MonitorInUseLists ? > Please create some kind of abstraction for this and set it a startup. Yeah. In the meantime, I reduced the scope of this patch (no longer including the above), see current webrev: http://cr.openjdk.java.net/~rkennke/8180932/webrev.02/ I agree that better abstraction would be good. > I do not see any reason for duplicating the > SafepointSynchronize::serial_cleanup(), just create proper abstraction > so it can be used in both cases? Yeah, I will think about it. The easiest way out would be to treat the serial case as 1-thread-MT case (called by VMThread), but this also means we can't easily keep the existing code path as-is while having a new separate code path. Maybe give the new code path some testing and time to iron out bugs (if any) and later switch the old path to a degenerated 1-threaded-MT path? Roman From erik.osterlund at oracle.com Mon May 29 15:25:41 2017 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 29 May 2017 17:25:41 +0200 Subject: RFR: Parallelize safepoint cleanup In-Reply-To: <05a8fb6c-6c49-b5aa-b8ba-38166ae09d6d@redhat.com> References: <486b5a72-bef8-4ebc-2729-3fe3aa3ab3b9@oracle.com> <0ccc1ec4-9866-cd2a-c6bc-a32c2da8cb3d@oracle.com> <592C2482.9020400@oracle.com> <05a8fb6c-6c49-b5aa-b8ba-38166ae09d6d@redhat.com> Message-ID: <592C3D75.9060500@oracle.com> On 2017-05-29 16:08, Roman Kennke wrote: > Am 29.05.2017 um 15:39 schrieb Erik ?sterlund: >> Hi, >> >> How do we feel about reusing the Parallel GC Threads work gang during >> safepoint cleanup? > I actually tried that, but failed, at least with Shenandoah, because... > >> We know that the Parallel GC Threads worker gang is idle then. > Not Shenandoah :-) Shenandoah GC threads are currently happy to continue > working during (non-GC) safepoints. One exception is the heap dumper, > which currently requires some extra synchronization. If you know of any > other safepoints that would require stopping concurrent GC threads > (marking, evacuating or updating references) let me know. I haven't > found any. Parallelizing safepoint cleanup might be a reason to > implement GC thread safepointing though. So perhaps one solution would be to have a SafepointSynchronize::worker_gang() that is initialized during bootstrapping. Then the GC can decide whether it wants to share its ParallelGCThreads worker gang, or do something else like creating a new worker gang. > Another problem with this approach is that it's clearly GC dependend, > and requires support (i.e. a worker thread pool) and coordination (i.e. > pause-resume GC work) by the GC. For example, as Kirk pointed out, > serial gc couldn't support that. You typically use Serial GC because you don't want a bunch of worker gangs. I don't think safepoint cleanup is an exception to that. > >> This saves creating another gazillion threads and another JVM flag to >> keep track of. > It would only be a few threads. There's just not that much work to do. > The prime scenario for this seems to be many- (like hundreds or > thousands of) threads work loads, in which case it hardly matters to add > some more to support more efficient cleanup. I see your point. I am just a bit hesitant to adding new worker gangs with user configurable knobs unless there is no other way. Thanks, /Erik > Roman > From david.holmes at oracle.com Tue May 30 01:59:07 2017 From: david.holmes at oracle.com (David Holmes) Date: Tue, 30 May 2017 11:59:07 +1000 Subject: RFR: Parallelize safepoint cleanup In-Reply-To: <592C3D75.9060500@oracle.com> References: <486b5a72-bef8-4ebc-2729-3fe3aa3ab3b9@oracle.com> <0ccc1ec4-9866-cd2a-c6bc-a32c2da8cb3d@oracle.com> <592C2482.9020400@oracle.com> <05a8fb6c-6c49-b5aa-b8ba-38166ae09d6d@redhat.com> <592C3D75.9060500@oracle.com> Message-ID: <36f8b0e6-2b83-60ff-0a0f-c8362d9611f7@oracle.com> I missed this before replying earlier up the thread ... On 30/05/2017 1:25 AM, Erik ?sterlund wrote: > > > On 2017-05-29 16:08, Roman Kennke wrote: >> Am 29.05.2017 um 15:39 schrieb Erik ?sterlund: >>> Hi, >>> >>> How do we feel about reusing the Parallel GC Threads work gang during >>> safepoint cleanup? >> I actually tried that, but failed, at least with Shenandoah, because... >> >>> We know that the Parallel GC Threads worker gang is idle then. >> Not Shenandoah :-) Shenandoah GC threads are currently happy to continue >> working during (non-GC) safepoints. One exception is the heap dumper, >> which currently requires some extra synchronization. If you know of any >> other safepoints that would require stopping concurrent GC threads >> (marking, evacuating or updating references) let me know. I haven't >> found any. Parallelizing safepoint cleanup might be a reason to >> implement GC thread safepointing though. > > So perhaps one solution would be to have a > SafepointSynchronize::worker_gang() that is initialized during > bootstrapping. Then the GC can decide whether it wants to share its > ParallelGCThreads worker gang, or do something else like creating a new > worker gang. I think the best approach would be to factor out the "worker gang" into a more general purpose thread pool (of non-JavaThreads) which can be dynamically sized, buffer tasks during high demand etc ( as per java.util.concurrent.ThreadPoolExecutor). Thereby isolating the worker threads from any specific task they may be engaged for. Cheers, David ----- >> Another problem with this approach is that it's clearly GC dependend, >> and requires support (i.e. a worker thread pool) and coordination (i.e. >> pause-resume GC work) by the GC. For example, as Kirk pointed out, >> serial gc couldn't support that. > > You typically use Serial GC because you don't want a bunch of worker > gangs. I don't think safepoint cleanup is an exception to that. > >> >>> This saves creating another gazillion threads and another JVM flag to >>> keep track of. >> It would only be a few threads. There's just not that much work to do. >> The prime scenario for this seems to be many- (like hundreds or >> thousands of) threads work loads, in which case it hardly matters to add >> some more to support more efficient cleanup. > > I see your point. I am just a bit hesitant to adding new worker gangs > with user configurable knobs unless there is no other way. > > Thanks, > /Erik > >> Roman >> > From david.holmes at oracle.com Tue May 30 01:59:17 2017 From: david.holmes at oracle.com (David Holmes) Date: Tue, 30 May 2017 11:59:17 +1000 Subject: RFR: Parallelize safepoint cleanup In-Reply-To: References: <486b5a72-bef8-4ebc-2729-3fe3aa3ab3b9@oracle.com> Message-ID: Hi Roman, On 29/05/2017 6:23 PM, Roman Kennke wrote: > Hi David, > > the patch applied cleanly, i.e. was not affected by any hotspot changes. > Most of your suggestions applied to the GC part that I excluded from the > scope of this change, except for the following: > >>> I introduced a dedicated worker thread pool in SafepointSynchronize. >>> This is only initialized when -XX:+ParallelSafepointCleanup is enabled, >>> and uses -XX:ParallelSafepointCleanupThreads=X threads, defaulting to 8 >>> threads (just a wild guess, open for discussion. With >>> -XX:-ParallelSafepointCleanup turned off (the default) it will use the >>> old serial safepoint cleanup processing (with optional GC hooks, see >>> below). >> >> I think the default needs to be ergonomically set, as for other >> "thread pools". Both flags should be experimental at this stage. > > I implemented this using the following: > > if (ParallelSafepointCleanup && > FLAG_IS_DEFAULT(ParallelSafepointCleanupThreads)) { > // This will pick up ParallelGCThreads if set... > FLAG_SET_DEFAULT(ParallelSafepointCleanupThreads, > Abstract_VM_Version::parallel_worker_threads()); > } > > As noted, it will pick up ParallelGCThreads, which might be a reasonable > number. Otherwise we'd need to re-work the logic in vm_version.cpp to > calculate the number of worker threads without considering > ParallelGCThreads. I think this is way too big a number of threads to consider for this situation. GC has a huge amount of work to do, but the safepoint cleanup should be much less work. I'm more inclined now to set a small hard-wired default initially - say 4 - and for future work look at generalising the existing fixed GC thread pool into a more general purpose, dynamic thread pool (something similar to java.util.concurrent.ThreadPoolExecutor.) I note others object to a dedicated thread pool for this, so generalising the existing one seems to be a reasonable future direction, whilst avoiding tying this to specific GCs. Thanks, David ----- > http://cr.openjdk.java.net/~rkennke/8180932/webrev.02/ > > > What do you think? > > Roman > From robbin.ehn at oracle.com Tue May 30 12:57:23 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 30 May 2017 14:57:23 +0200 Subject: RFR: Parallelize safepoint cleanup In-Reply-To: <9c019b11-0c78-2649-d3bd-cd02fd999e68@redhat.com> References: <486b5a72-bef8-4ebc-2729-3fe3aa3ab3b9@oracle.com> <0ccc1ec4-9866-cd2a-c6bc-a32c2da8cb3d@oracle.com> <9c019b11-0c78-2649-d3bd-cd02fd999e68@redhat.com> Message-ID: <1cd887ad-8fa9-02f6-8d88-d995c1f56031@oracle.com> Hi Roman, On 05/29/2017 04:16 PM, Roman Kennke wrote: > > I agree that having a single pool would be good. The current WorkGang > doesn't do it though, because we can't borrow threads while the GC is > doing work, at least not in a way that is GC agnostic (see my reply to > Robbin). Correct me if I'm wrong, using WorkGang works for all the GCs that we have in JDK10. (serial I would assume just use the VmThread) If Shenandoah needs modification to the WorkGang then you would be-able to have them in your repo. If the modification is very large alternative you could use separate thread pool for this in your Shenandoah repo. Since for vanilla JDK 10, as now, I don't see we need it? > > Yeah, I will think about it. The easiest way out would be to treat the > serial case as 1-thread-MT case (called by VMThread), but this also > means we can't easily keep the existing code path as-is while having a > new separate code path. Maybe give the new code path some testing and > time to iron out bugs (if any) and later switch the old path to a > degenerated 1-threaded-MT path? This sounds good, I think you could get away with doing just 1 case and we will iron out bugs faster I guess. Thanks Robbin > > Roman > From rkennke at redhat.com Tue May 30 13:09:28 2017 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 30 May 2017 15:09:28 +0200 Subject: RFR: Parallelize safepoint cleanup In-Reply-To: <1cd887ad-8fa9-02f6-8d88-d995c1f56031@oracle.com> References: <486b5a72-bef8-4ebc-2729-3fe3aa3ab3b9@oracle.com> <0ccc1ec4-9866-cd2a-c6bc-a32c2da8cb3d@oracle.com> <9c019b11-0c78-2649-d3bd-cd02fd999e68@redhat.com> <1cd887ad-8fa9-02f6-8d88-d995c1f56031@oracle.com> Message-ID: Hi Robbin, I'm currently testing an abstraction that allows to use GC workers, and also makes serial and parallel cleanup to use the same code path. Stay tuned ;-) /Roman Am 30. Mai 2017 14:57:23 MESZ schrieb Robbin Ehn : >Hi Roman, > >On 05/29/2017 04:16 PM, Roman Kennke wrote: >> >> I agree that having a single pool would be good. The current WorkGang >> doesn't do it though, because we can't borrow threads while the GC is >> doing work, at least not in a way that is GC agnostic (see my reply >to >> Robbin). > >Correct me if I'm wrong, using WorkGang works for all the GCs that we >have in JDK10. >(serial I would assume just use the VmThread) > >If Shenandoah needs modification to the WorkGang then you would be-able >to have them in your repo. >If the modification is very large alternative you could use separate >thread pool for this in your Shenandoah repo. >Since for vanilla JDK 10, as now, I don't see we need it? > >> >> Yeah, I will think about it. The easiest way out would be to treat >the >> serial case as 1-thread-MT case (called by VMThread), but this also >> means we can't easily keep the existing code path as-is while having >a >> new separate code path. Maybe give the new code path some testing and >> time to iron out bugs (if any) and later switch the old path to a >> degenerated 1-threaded-MT path? > >This sounds good, I think you could get away with doing just 1 case and >we will iron out bugs faster I guess. > >Thanks Robbin > >> >> Roman >> -- Sent from my FairPhone From rkennke at redhat.com Tue May 30 16:51:48 2017 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 30 May 2017 18:51:48 +0200 Subject: RFR: Parallelize safepoint cleanup In-Reply-To: References: <486b5a72-bef8-4ebc-2729-3fe3aa3ab3b9@oracle.com> Message-ID: <2f399b0d-5e68-9861-43bc-023dcba1acb1@redhat.com> Am 30.05.2017 um 03:59 schrieb David Holmes: > Hi Roman, > > On 29/05/2017 6:23 PM, Roman Kennke wrote: >> Hi David, >> >> the patch applied cleanly, i.e. was not affected by any hotspot changes. >> Most of your suggestions applied to the GC part that I excluded from the >> scope of this change, except for the following: >> >>>> I introduced a dedicated worker thread pool in SafepointSynchronize. >>>> This is only initialized when -XX:+ParallelSafepointCleanup is >>>> enabled, >>>> and uses -XX:ParallelSafepointCleanupThreads=X threads, defaulting >>>> to 8 >>>> threads (just a wild guess, open for discussion. With >>>> -XX:-ParallelSafepointCleanup turned off (the default) it will use the >>>> old serial safepoint cleanup processing (with optional GC hooks, see >>>> below). >>> >>> I think the default needs to be ergonomically set, as for other >>> "thread pools". Both flags should be experimental at this stage. >> >> I implemented this using the following: >> >> if (ParallelSafepointCleanup && >> FLAG_IS_DEFAULT(ParallelSafepointCleanupThreads)) { >> // This will pick up ParallelGCThreads if set... >> FLAG_SET_DEFAULT(ParallelSafepointCleanupThreads, >> Abstract_VM_Version::parallel_worker_threads()); >> } >> >> As noted, it will pick up ParallelGCThreads, which might be a reasonable >> number. Otherwise we'd need to re-work the logic in vm_version.cpp to >> calculate the number of worker threads without considering >> ParallelGCThreads. > > I think this is way too big a number of threads to consider for this > situation. GC has a huge amount of work to do, but the safepoint > cleanup should be much less work. > > I'm more inclined now to set a small hard-wired default initially - > say 4 - and for future work look at generalising the existing fixed GC > thread pool into a more general purpose, dynamic thread pool > (something similar to java.util.concurrent.ThreadPoolExecutor.) I note > others object to a dedicated thread pool for this, so generalising the > existing one seems to be a reasonable future direction, whilst > avoiding tying this to specific GCs. ok, I changed the following: - default to 4 safepoint cleanup threads, no ergo - added a hook into the GC, which can now share its worker threads with safepoint cleanup. works for CMS and G1, Parallel does not use WorkGang, Serial doesn't use multithreads at all. GC can return NULL, which lets the safepoint code create its own worker threads (when ParallelSafepointCleanup is requested). When using GC workers, only min(ParallelSafepointCleanupThread, ParallelGCThreads) are used. - serial and parallel code paths are now the same. Serial is calling the task from the VMThread. Pro: no rarely tested code paths, con: serial uses claiming logic. I'm undecided: if you're favoring to keep separate code paths, let me know. Testing: hotspot_gc all passes with release and fastdebug. no regressions with specjvm http://cr.openjdk.java.net/~rkennke/8180932/webrev.03/ Roman From mikael.vidstedt at oracle.com Wed May 31 00:20:43 2017 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Tue, 30 May 2017 17:20:43 -0700 Subject: RFR(S): 8180184: Add DATA and FSIZE to os::Posix::print_rlimit_info In-Reply-To: References: <17B78E0D-C38B-4C8C-9F52-FA572563D485@oracle.com> Message-ID: David, I assume that the C++ compilers will all convert it to a simple shift by ten, but I?m not going to verify it :) Thomas, agree that in theory the return value from getrlimit should be checked, but chose to not make any further modifications as part of this change. Thanks to both of you for the reviews! Cheers, Mikael > On May 29, 2017, at 1:22 AM, Thomas St?fe wrote: > > Hi Mikael, > > looks fine. > > Small nit, we never seem to check the return value of getrlimit(). But seeing that the only way to fail getrlimit() would be to specify an invalid limit constant, maybe this is ok. > > Best Regards, Thomas > > On Fri, May 26, 2017 at 11:22 PM, Mikael Vidstedt > wrote: > > Please review the following fix which adds RLIMIT_DATA and RLIMIT_FISZE to the rlimit related data in the crash dump, and cleans up/unifies some of the related code. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8180184 > Webrev: http://cr.openjdk.java.net/~mikael/webrevs/8180184/webrev.00/hotspot/webrev/ > > Tested using JPRT. Manually verified that the crash dump contains the expected information. > > Thanks to Thomas for helping verify that the change works as expected on AIX as well! > > Cheers, > Mikael > > From rkennke at redhat.com Wed May 31 08:27:27 2017 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 31 May 2017 10:27:27 +0200 Subject: RFR: Parallelize safepoint cleanup In-Reply-To: References: <486b5a72-bef8-4ebc-2729-3fe3aa3ab3b9@oracle.com> Message-ID: <5c80f8df-27c9-f9a9-dc6d-47f9c6019a61@redhat.com> I realized that sharing workers with GC is not so easy. We need to be able to use the workers at a safepoint during concurrent GC work (which also uses the same workers). This does not only require that those workers be suspended, like e.g. SuspendibleThreadSet::yield(), but they need to be idle, i.e. have finished their tasks. This needs some careful handling to work without races: it requires a SuspendibleThreadSetJoiner around the corresponding run_task() call and also the tasks themselves need to join the STS and handle requests for safepoints not by yielding, but by leaving the task. This is far too peculiar for me to make the call to hook up GC workers for safepoint cleanup, and I thus removed those parts. I left the API in CollectedHeap in place. I think GC devs who know better about G1 and CMS should make that call, or else just use a separate thread pool. http://cr.openjdk.java.net/~rkennke/8180932/webrev.05/ Is it ok now? Roman Am 30.05.2017 um 03:59 schrieb David Holmes: > Hi Roman, > > On 29/05/2017 6:23 PM, Roman Kennke wrote: >> Hi David, >> >> the patch applied cleanly, i.e. was not affected by any hotspot changes. >> Most of your suggestions applied to the GC part that I excluded from the >> scope of this change, except for the following: >> >>>> I introduced a dedicated worker thread pool in SafepointSynchronize. >>>> This is only initialized when -XX:+ParallelSafepointCleanup is >>>> enabled, >>>> and uses -XX:ParallelSafepointCleanupThreads=X threads, defaulting >>>> to 8 >>>> threads (just a wild guess, open for discussion. With >>>> -XX:-ParallelSafepointCleanup turned off (the default) it will use the >>>> old serial safepoint cleanup processing (with optional GC hooks, see >>>> below). >>> >>> I think the default needs to be ergonomically set, as for other >>> "thread pools". Both flags should be experimental at this stage. >> >> I implemented this using the following: >> >> if (ParallelSafepointCleanup && >> FLAG_IS_DEFAULT(ParallelSafepointCleanupThreads)) { >> // This will pick up ParallelGCThreads if set... >> FLAG_SET_DEFAULT(ParallelSafepointCleanupThreads, >> Abstract_VM_Version::parallel_worker_threads()); >> } >> >> As noted, it will pick up ParallelGCThreads, which might be a reasonable >> number. Otherwise we'd need to re-work the logic in vm_version.cpp to >> calculate the number of worker threads without considering >> ParallelGCThreads. > > I think this is way too big a number of threads to consider for this > situation. GC has a huge amount of work to do, but the safepoint > cleanup should be much less work. > > I'm more inclined now to set a small hard-wired default initially - > say 4 - and for future work look at generalising the existing fixed GC > thread pool into a more general purpose, dynamic thread pool > (something similar to java.util.concurrent.ThreadPoolExecutor.) I note > others object to a dedicated thread pool for this, so generalising the > existing one seems to be a reasonable future direction, whilst > avoiding tying this to specific GCs. > > Thanks, > David > ----- > >> http://cr.openjdk.java.net/~rkennke/8180932/webrev.02/ >> >> >> What do you think? >> >> Roman >> From per.liden at oracle.com Wed May 31 11:59:30 2017 From: per.liden at oracle.com (Per Liden) Date: Wed, 31 May 2017 13:59:30 +0200 Subject: RFR(s): 8181319: Make os::Linux::sched_getcpu() available even when UseNUMA is false Message-ID: Hi, The initialization/availability of os::Linux::sched_getcpu() is today guarded by UseNUMA. While NUMA-related code is currently the only user of sched_getcpu(), this function is useful in many other situations as well. I propose that we initialize os::Linux::sched_getcpu() regardless of whether UseNUMA is enabled or not. Bug: https://bugs.openjdk.java.net/browse/JDK-8181319 Webrev: http://cr.openjdk.java.net/~pliden/8181319/webrev.0/ Testing: JPRT, ad-hoc runs using ParallelGC with and without UseNUMA cheers, Per From robbin.ehn at oracle.com Wed May 31 20:06:26 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 31 May 2017 22:06:26 +0200 Subject: RFR: Parallelize safepoint cleanup In-Reply-To: <5c80f8df-27c9-f9a9-dc6d-47f9c6019a61@redhat.com> References: <486b5a72-bef8-4ebc-2729-3fe3aa3ab3b9@oracle.com> <5c80f8df-27c9-f9a9-dc6d-47f9c6019a61@redhat.com> Message-ID: <46ad874e-eb41-7927-265a-40dea92dfe1e@oracle.com> Hi Roman, I agree that is really needed but: On 05/31/2017 10:27 AM, Roman Kennke wrote: > I realized that sharing workers with GC is not so easy. > > We need to be able to use the workers at a safepoint during concurrent > GC work (which also uses the same workers). This does not only require > that those workers be suspended, like e.g. > SuspendibleThreadSet::yield(), but they need to be idle, i.e. have > finished their tasks. This needs some careful handling to work without > races: it requires a SuspendibleThreadSetJoiner around the corresponding > run_task() call and also the tasks themselves need to join the STS and > handle requests for safepoints not by yielding, but by leaving the task. > This is far too peculiar for me to make the call to hook up GC workers > for safepoint cleanup, and I thus removed those parts. I left the API in > CollectedHeap in place. I think GC devs who know better about G1 and CMS > should make that call, or else just use a separate thread pool. > > http://cr.openjdk.java.net/~rkennke/8180932/webrev.05/ > > > Is it ok now? I still think you should put the "Parallel Safepoint Cleanup" workers inside Shenandoah, so the SafepointSynchronizer only calls get_safepoint_workers, e.g.: _cleanup_workers = heap->get_safepoint_workers(); _num_cleanup_workers = _cleanup_workers != NULL ? _cleanup_workers->total_workers() : 1; ParallelSPCleanupTask cleanup(_cleanup_subtasks); StrongRootsScope srs(_num_cleanup_workers); if (_cleanup_workers != NULL) { _cleanup_workers->run_task(&cleanup, _num_cleanup_workers); } else { cleanup.work(0); } That way you don't even need your new flags, but it will be up to the other GCs to make their worker available or cheat with a separate workgang. Otherwise this looking reasonable. Thanks for doing this! /Robbin > > Roman > > Am 30.05.2017 um 03:59 schrieb David Holmes: >> Hi Roman, >> >> On 29/05/2017 6:23 PM, Roman Kennke wrote: >>> Hi David, >>> >>> the patch applied cleanly, i.e. was not affected by any hotspot changes. >>> Most of your suggestions applied to the GC part that I excluded from the >>> scope of this change, except for the following: >>> >>>>> I introduced a dedicated worker thread pool in SafepointSynchronize. >>>>> This is only initialized when -XX:+ParallelSafepointCleanup is >>>>> enabled, >>>>> and uses -XX:ParallelSafepointCleanupThreads=X threads, defaulting >>>>> to 8 >>>>> threads (just a wild guess, open for discussion. With >>>>> -XX:-ParallelSafepointCleanup turned off (the default) it will use the >>>>> old serial safepoint cleanup processing (with optional GC hooks, see >>>>> below). >>>> >>>> I think the default needs to be ergonomically set, as for other >>>> "thread pools". Both flags should be experimental at this stage. >>> >>> I implemented this using the following: >>> >>> if (ParallelSafepointCleanup && >>> FLAG_IS_DEFAULT(ParallelSafepointCleanupThreads)) { >>> // This will pick up ParallelGCThreads if set... >>> FLAG_SET_DEFAULT(ParallelSafepointCleanupThreads, >>> Abstract_VM_Version::parallel_worker_threads()); >>> } >>> >>> As noted, it will pick up ParallelGCThreads, which might be a reasonable >>> number. Otherwise we'd need to re-work the logic in vm_version.cpp to >>> calculate the number of worker threads without considering >>> ParallelGCThreads. >> >> I think this is way too big a number of threads to consider for this >> situation. GC has a huge amount of work to do, but the safepoint >> cleanup should be much less work. >> >> I'm more inclined now to set a small hard-wired default initially - >> say 4 - and for future work look at generalising the existing fixed GC >> thread pool into a more general purpose, dynamic thread pool >> (something similar to java.util.concurrent.ThreadPoolExecutor.) I note >> others object to a dedicated thread pool for this, so generalising the >> existing one seems to be a reasonable future direction, whilst >> avoiding tying this to specific GCs. >> >> Thanks, >> David >> ----- >> >>> http://cr.openjdk.java.net/~rkennke/8180932/webrev.02/ >>> >>> >>> What do you think? >>> >>> Roman >>> > From mikael.vidstedt at oracle.com Wed May 31 21:48:46 2017 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Wed, 31 May 2017 14:48:46 -0700 Subject: RFR(XS): 8181377: Capture underlying type for unsafe/unaligned pointers in ClassFileParser Message-ID: Bug: https://bugs.openjdk.java.net/browse/JDK-8181377 Webrev: http://cr.openjdk.java.net/~mikael/webrevs/8181377/webrev.00/hotspot/webrev/ Copied from the issue description: As part of JDK-8180032 I fixed a problem where ClassFileParser passed around potentially unaligned pointers to various class file data structures, and in some cases dereferenced those unaligned pointers which is not valid in C++. As part of the fix I chose to switch from using typed pointers (like u2*) in favor of void*, to signal that the pointer is not (necessarily) possible to dereference directly. However, by switching to void* the information about the underlying type (u2) is lost, which is unfortunate. The information about the type should be maintained in some way. I?ve been playing around with a more generic representation of ?unsafe? pointers, but I think it?s a slightly bigger project and meanwhile I don?t want to ?lose" the type information, so the suggested change at least addresses the ?regression? in information. Cheers, Mikael