From jaroslav.bachorik at oracle.com Mon Dec 9 03:46:03 2013 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Mon, 09 Dec 2013 12:46:03 +0100 Subject: jmx-dev RFR 6309226: TEST: java/lang/management/ThreadMXBean/SynchronizationStatistics.java didn't check Thread.sleep In-Reply-To: <528BB41A.4070107@oracle.com> References: <52651646.4050705@oracle.com> <5289E4F5.1080807@oracle.com> <528A8084.5060406@oracle.com> <528B7906.7010708@oracle.com> <528BB41A.4070107@oracle.com> Message-ID: <52A5AD7B.1060403@oracle.com> Hi David, On 19.11.2013 19:55, David Holmes wrote: > On 20/11/2013 12:43 AM, Jaroslav Bachorik wrote: >> Hi David, >> >> On 18.11.2013 22:03, David Holmes wrote: >>> Hi Jaroslav, >>> >>> I think your phaser usage is incorrect: >>> >>> 88 public void run() { >>> 89 p.arriveAndAwaitAdvance(); // phase[1] >>> 90 synchronized(lock1) { >>> 91 System.out.println("[LockerThread obtained >>> Lock1]"); >>> 92 p.arrive(); // phase[2] >>> 93 } >>> 94 p.arriveAndAwaitAdvance(); // phase[3] >>> 95 } >>> >>> Here the current thread can release itself at line 94. You have assumed >>> that the phase[2] arrive will be a trigger to release the main thread >>> but it may not have reached its arriveAndAwaitAdvance phase[2] statement >>> yet, so the current thread arrives then does arrive-and-wait but the >>> number of arrivals is 2 so it doesn't wait. >>> >>> A CyclicBarrier per phase may be clearer. >> >> Yes, thanks for pointing this out. But, wouldn't >> p.arriveAndAwaitAdvance() instead of p.arrive() suffice? I've tried to >> replace Phaser with CyclicBarrier but while it seems to work as expected >> it pollutes code with the necessity to catch various exceptions >> (InterruptedException, BarrierException etc.). So, if the simple fix >> with Phaser would do the trick I would better use that. > > In this case yes arriveAndAwaitAdvance would fix it. I don't know if > other tests have the same issue - the concern would be ensuring there's > no possibility of deadlocks in general. I've updated the webrev with the one using p.arriveAndAwaitAdvance() at line 92 - http://cr.openjdk.java.net/~jbachorik/6309226/webrev.04 Are you ok with this change then? Thanks, -JB- > > Thanks, > David > >> -JB- >> >>> >>> David >>> >>> On 18/11/2013 7:59 PM, Jaroslav Bachorik wrote: >>>> Hi, >>>> >>>> after discussing this with Mandy I've rewritten the test to use the >>>> j.u.concurrent for synchronization - this also makes it much easier to >>>> follow the test logic. >>>> >>>> The waited time, the blocked time and the waited counts are only >>>> checked >>>> for sanity (increasing values) since it is not possible to do the >>>> reliable checks against hard numbers. >>>> >>>> I ran the test in a tight loop for 1500 times using -Xcomp and -Xint >>>> and >>>> the test seems to pass constantly. >>>> >>>> New webrev: http://cr.openjdk.java.net/~jbachorik/6309226/webrev.03 >>>> >>>> Thanks, >>>> >>>> -JB- >>>> >>>> >>>> On 21.10.2013 13:55, Jaroslav Bachorik wrote: >>>>> Please, review this small patch for a test failing due to the updated >>>>> implementation in JDK6. >>>>> >>>>> Issue: https://bugs.openjdk.java.net/browse/JDK-6309226 >>>>> Webrev: http://cr.openjdk.java.net/~jbachorik/6309226/webrev.00/ >>>>> >>>>> The test fails due to the change in mustang where >>>>> ThreadMXBean.getThreadInfo().getWaitedTime() and >>>>> ThreadMXBean.getThreadInfo().getWaitedCount() include Thread.sleep() >>>>> too. Unfortunately, Thread.sleep() is used throughout the test for >>>>> synchronization purposes and this breaks the test. >>>>> >>>>> In the patch I propose to replace Thread.sleep() with busy wait and >>>>> hinting the scheduler by Thread.yield(). While not very elegant it >>>>> successfully works around inclusion of unknown number of >>>>> Thread.sleep()s >>>>> (they are called in loop). >>>>> >>>>> Thanks, >>>>> >>>>> -JB- >>>> >> From jaroslav.bachorik at oracle.com Tue Dec 17 03:41:12 2013 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Tue, 17 Dec 2013 12:41:12 +0100 Subject: jmx-dev [ping] Re: RFR 8022221: Intermittent test failures in sun/management/jmxremote/startstop/JMXStartStopTest.sh In-Reply-To: <52862EDC.4080703@oracle.com> References: <52862EDC.4080703@oracle.com> Message-ID: <52B03858.7040409@oracle.com> Anyone? -JB- On 15.11.2013 15:25, Jaroslav Bachorik wrote: > Please, review this test fix. > > Issue : https://bugs.openjdk.java.net/browse/JDK-8022221 > Webrev: http://cr.openjdk.java.net/~jbachorik/8022221/webrev.00 > > The test was facing intermittent failures due to not 100% failproof > interprocess synchronization using lock files. The solution is rewriting > the shell test to pure Java and use stdout/stderr processing for the > application started by the test to assess its status. > > A part of the change is also few improvements to the > jdk.testlibrary.ProcessTools. > > Thanks, > > -JB- From jaroslav.bachorik at oracle.com Tue Dec 17 03:48:00 2013 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Tue, 17 Dec 2013 12:48:00 +0100 Subject: jmx-dev [ping] Re: RFR 6309226: TEST: java/lang/management/ThreadMXBean/SynchronizationStatistics.java didn't check Thread.sleep In-Reply-To: <52A5AD7B.1060403@oracle.com> References: <52651646.4050705@oracle.com> <5289E4F5.1080807@oracle.com> <528A8084.5060406@oracle.com> <528B7906.7010708@oracle.com> <528BB41A.4070107@oracle.com> <52A5AD7B.1060403@oracle.com> Message-ID: <52B039F0.4020507@oracle.com> Hi, an updated webrev with added comments and removed the commented out debug code is here - http://cr.openjdk.java.net/~jbachorik/6309226/webrev.05 -JB- On 9.12.2013 12:46, Jaroslav Bachorik wrote: > Hi David, > > On 19.11.2013 19:55, David Holmes wrote: >> On 20/11/2013 12:43 AM, Jaroslav Bachorik wrote: >>> Hi David, >>> >>> On 18.11.2013 22:03, David Holmes wrote: >>>> Hi Jaroslav, >>>> >>>> I think your phaser usage is incorrect: >>>> >>>> 88 public void run() { >>>> 89 p.arriveAndAwaitAdvance(); // phase[1] >>>> 90 synchronized(lock1) { >>>> 91 System.out.println("[LockerThread obtained >>>> Lock1]"); >>>> 92 p.arrive(); // phase[2] >>>> 93 } >>>> 94 p.arriveAndAwaitAdvance(); // phase[3] >>>> 95 } >>>> >>>> Here the current thread can release itself at line 94. You have assumed >>>> that the phase[2] arrive will be a trigger to release the main thread >>>> but it may not have reached its arriveAndAwaitAdvance phase[2] >>>> statement >>>> yet, so the current thread arrives then does arrive-and-wait but the >>>> number of arrivals is 2 so it doesn't wait. >>>> >>>> A CyclicBarrier per phase may be clearer. >>> >>> Yes, thanks for pointing this out. But, wouldn't >>> p.arriveAndAwaitAdvance() instead of p.arrive() suffice? I've tried to >>> replace Phaser with CyclicBarrier but while it seems to work as expected >>> it pollutes code with the necessity to catch various exceptions >>> (InterruptedException, BarrierException etc.). So, if the simple fix >>> with Phaser would do the trick I would better use that. >> >> In this case yes arriveAndAwaitAdvance would fix it. I don't know if >> other tests have the same issue - the concern would be ensuring there's >> no possibility of deadlocks in general. > > I've updated the webrev with the one using p.arriveAndAwaitAdvance() at > line 92 - http://cr.openjdk.java.net/~jbachorik/6309226/webrev.04 > > Are you ok with this change then? > > Thanks, > > -JB- > > >> >> Thanks, >> David >> >>> -JB- >>> >>>> >>>> David >>>> >>>> On 18/11/2013 7:59 PM, Jaroslav Bachorik wrote: >>>>> Hi, >>>>> >>>>> after discussing this with Mandy I've rewritten the test to use the >>>>> j.u.concurrent for synchronization - this also makes it much easier to >>>>> follow the test logic. >>>>> >>>>> The waited time, the blocked time and the waited counts are only >>>>> checked >>>>> for sanity (increasing values) since it is not possible to do the >>>>> reliable checks against hard numbers. >>>>> >>>>> I ran the test in a tight loop for 1500 times using -Xcomp and -Xint >>>>> and >>>>> the test seems to pass constantly. >>>>> >>>>> New webrev: http://cr.openjdk.java.net/~jbachorik/6309226/webrev.03 >>>>> >>>>> Thanks, >>>>> >>>>> -JB- >>>>> >>>>> >>>>> On 21.10.2013 13:55, Jaroslav Bachorik wrote: >>>>>> Please, review this small patch for a test failing due to the updated >>>>>> implementation in JDK6. >>>>>> >>>>>> Issue: https://bugs.openjdk.java.net/browse/JDK-6309226 >>>>>> Webrev: http://cr.openjdk.java.net/~jbachorik/6309226/webrev.00/ >>>>>> >>>>>> The test fails due to the change in mustang where >>>>>> ThreadMXBean.getThreadInfo().getWaitedTime() and >>>>>> ThreadMXBean.getThreadInfo().getWaitedCount() include Thread.sleep() >>>>>> too. Unfortunately, Thread.sleep() is used throughout the test for >>>>>> synchronization purposes and this breaks the test. >>>>>> >>>>>> In the patch I propose to replace Thread.sleep() with busy wait and >>>>>> hinting the scheduler by Thread.yield(). While not very elegant it >>>>>> successfully works around inclusion of unknown number of >>>>>> Thread.sleep()s >>>>>> (they are called in loop). >>>>>> >>>>>> Thanks, >>>>>> >>>>>> -JB- >>>>> >>> > From david.holmes at oracle.com Thu Dec 19 18:27:34 2013 From: david.holmes at oracle.com (David Holmes) Date: Fri, 20 Dec 2013 12:27:34 +1000 Subject: jmx-dev RFR 6309226: TEST: java/lang/management/ThreadMXBean/SynchronizationStatistics.java didn't check Thread.sleep In-Reply-To: <52A5AD7B.1060403@oracle.com> References: <52651646.4050705@oracle.com> <5289E4F5.1080807@oracle.com> <528A8084.5060406@oracle.com> <528B7906.7010708@oracle.com> <528BB41A.4070107@oracle.com> <52A5AD7B.1060403@oracle.com> Message-ID: <52B3AB16.30206@oracle.com> Sorry for the delay - still catching up after vacation (only 435 openjdk emails left :( ). On 9/12/2013 9:46 PM, Jaroslav Bachorik wrote: > On 19.11.2013 19:55, David Holmes wrote: >> On 20/11/2013 12:43 AM, Jaroslav Bachorik wrote: >>> Hi David, >>> >>> On 18.11.2013 22:03, David Holmes wrote: >>>> Hi Jaroslav, >>>> >>>> I think your phaser usage is incorrect: >>>> >>>> 88 public void run() { >>>> 89 p.arriveAndAwaitAdvance(); // phase[1] >>>> 90 synchronized(lock1) { >>>> 91 System.out.println("[LockerThread obtained >>>> Lock1]"); >>>> 92 p.arrive(); // phase[2] >>>> 93 } >>>> 94 p.arriveAndAwaitAdvance(); // phase[3] >>>> 95 } >>>> >>>> Here the current thread can release itself at line 94. You have assumed >>>> that the phase[2] arrive will be a trigger to release the main thread >>>> but it may not have reached its arriveAndAwaitAdvance phase[2] >>>> statement >>>> yet, so the current thread arrives then does arrive-and-wait but the >>>> number of arrivals is 2 so it doesn't wait. >>>> >>>> A CyclicBarrier per phase may be clearer. >>> >>> Yes, thanks for pointing this out. But, wouldn't >>> p.arriveAndAwaitAdvance() instead of p.arrive() suffice? I've tried to >>> replace Phaser with CyclicBarrier but while it seems to work as expected >>> it pollutes code with the necessity to catch various exceptions >>> (InterruptedException, BarrierException etc.). So, if the simple fix >>> with Phaser would do the trick I would better use that. >> >> In this case yes arriveAndAwaitAdvance would fix it. I don't know if >> other tests have the same issue - the concern would be ensuring there's >> no possibility of deadlocks in general. > > I've updated the webrev with the one using p.arriveAndAwaitAdvance() at > line 92 - http://cr.openjdk.java.net/~jbachorik/6309226/webrev.04 > > Are you ok with this change then? I think you have the same problem anywhere that arrive() is used eg: 129 p.arrive(); // phase[2] 130 p.arriveAndAwaitAdvance(); // phase[3] and 185 p.arrive(); // phase[2] 186 } 187 p.arriveAndAwaitAdvance(); // phase[3] David ------ > Thanks, > > -JB- > > >> >> Thanks, >> David >> >>> -JB- >>> >>>> >>>> David >>>> >>>> On 18/11/2013 7:59 PM, Jaroslav Bachorik wrote: >>>>> Hi, >>>>> >>>>> after discussing this with Mandy I've rewritten the test to use the >>>>> j.u.concurrent for synchronization - this also makes it much easier to >>>>> follow the test logic. >>>>> >>>>> The waited time, the blocked time and the waited counts are only >>>>> checked >>>>> for sanity (increasing values) since it is not possible to do the >>>>> reliable checks against hard numbers. >>>>> >>>>> I ran the test in a tight loop for 1500 times using -Xcomp and -Xint >>>>> and >>>>> the test seems to pass constantly. >>>>> >>>>> New webrev: http://cr.openjdk.java.net/~jbachorik/6309226/webrev.03 >>>>> >>>>> Thanks, >>>>> >>>>> -JB- >>>>> >>>>> >>>>> On 21.10.2013 13:55, Jaroslav Bachorik wrote: >>>>>> Please, review this small patch for a test failing due to the updated >>>>>> implementation in JDK6. >>>>>> >>>>>> Issue: https://bugs.openjdk.java.net/browse/JDK-6309226 >>>>>> Webrev: http://cr.openjdk.java.net/~jbachorik/6309226/webrev.00/ >>>>>> >>>>>> The test fails due to the change in mustang where >>>>>> ThreadMXBean.getThreadInfo().getWaitedTime() and >>>>>> ThreadMXBean.getThreadInfo().getWaitedCount() include Thread.sleep() >>>>>> too. Unfortunately, Thread.sleep() is used throughout the test for >>>>>> synchronization purposes and this breaks the test. >>>>>> >>>>>> In the patch I propose to replace Thread.sleep() with busy wait and >>>>>> hinting the scheduler by Thread.yield(). While not very elegant it >>>>>> successfully works around inclusion of unknown number of >>>>>> Thread.sleep()s >>>>>> (they are called in loop). >>>>>> >>>>>> Thanks, >>>>>> >>>>>> -JB- >>>>> >>> > From jaroslav.bachorik at oracle.com Fri Dec 20 00:24:30 2013 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Fri, 20 Dec 2013 09:24:30 +0100 Subject: jmx-dev RFR 6309226: TEST: java/lang/management/ThreadMXBean/SynchronizationStatistics.java didn't check Thread.sleep In-Reply-To: <52B3AB16.30206@oracle.com> References: <52651646.4050705@oracle.com> <5289E4F5.1080807@oracle.com> <528A8084.5060406@oracle.com> <528B7906.7010708@oracle.com> <528BB41A.4070107@oracle.com> <52A5AD7B.1060403@oracle.com> <52B3AB16.30206@oracle.com> Message-ID: <52B3FEBE.40106@oracle.com> On 20.12.2013 03:27, David Holmes wrote: > Sorry for the delay - still catching up after vacation (only 435 openjdk > emails left :( ). NP > > On 9/12/2013 9:46 PM, Jaroslav Bachorik wrote: >> On 19.11.2013 19:55, David Holmes wrote: >>> On 20/11/2013 12:43 AM, Jaroslav Bachorik wrote: >>>> Hi David, >>>> >>>> On 18.11.2013 22:03, David Holmes wrote: >>>>> Hi Jaroslav, >>>>> >>>>> I think your phaser usage is incorrect: >>>>> >>>>> 88 public void run() { >>>>> 89 p.arriveAndAwaitAdvance(); // phase[1] >>>>> 90 synchronized(lock1) { >>>>> 91 System.out.println("[LockerThread obtained >>>>> Lock1]"); >>>>> 92 p.arrive(); // phase[2] >>>>> 93 } >>>>> 94 p.arriveAndAwaitAdvance(); // phase[3] >>>>> 95 } >>>>> >>>>> Here the current thread can release itself at line 94. You have >>>>> assumed >>>>> that the phase[2] arrive will be a trigger to release the main thread >>>>> but it may not have reached its arriveAndAwaitAdvance phase[2] >>>>> statement >>>>> yet, so the current thread arrives then does arrive-and-wait but the >>>>> number of arrivals is 2 so it doesn't wait. >>>>> >>>>> A CyclicBarrier per phase may be clearer. >>>> >>>> Yes, thanks for pointing this out. But, wouldn't >>>> p.arriveAndAwaitAdvance() instead of p.arrive() suffice? I've tried to >>>> replace Phaser with CyclicBarrier but while it seems to work as >>>> expected >>>> it pollutes code with the necessity to catch various exceptions >>>> (InterruptedException, BarrierException etc.). So, if the simple fix >>>> with Phaser would do the trick I would better use that. >>> >>> In this case yes arriveAndAwaitAdvance would fix it. I don't know if >>> other tests have the same issue - the concern would be ensuring there's >>> no possibility of deadlocks in general. >> >> I've updated the webrev with the one using p.arriveAndAwaitAdvance() at >> line 92 - http://cr.openjdk.java.net/~jbachorik/6309226/webrev.04 >> >> Are you ok with this change then? > > I think you have the same problem anywhere that arrive() is used eg: > > 129 p.arrive(); // phase[2] > 130 p.arriveAndAwaitAdvance(); // phase[3] > > and > > 185 p.arrive(); // phase[2] > 186 } > 187 p.arriveAndAwaitAdvance(); // phase[3] Very probable :( Also, when analysing the recurrence of JDK-8029890 I've found out that Phaser.arriveAndAwaitAdvance() actually blocked the thread in a way that it increased the number of contentions reported :( CyclicBarrier seems to wait instead - I will have to use CyclicBarrier when testing the number of reported contentions and Phaser when testing the number of reported waits :/ -JB- > > David > ------ > >> Thanks, >> >> -JB- >> >> >>> >>> Thanks, >>> David >>> >>>> -JB- >>>> >>>>> >>>>> David >>>>> >>>>> On 18/11/2013 7:59 PM, Jaroslav Bachorik wrote: >>>>>> Hi, >>>>>> >>>>>> after discussing this with Mandy I've rewritten the test to use the >>>>>> j.u.concurrent for synchronization - this also makes it much >>>>>> easier to >>>>>> follow the test logic. >>>>>> >>>>>> The waited time, the blocked time and the waited counts are only >>>>>> checked >>>>>> for sanity (increasing values) since it is not possible to do the >>>>>> reliable checks against hard numbers. >>>>>> >>>>>> I ran the test in a tight loop for 1500 times using -Xcomp and -Xint >>>>>> and >>>>>> the test seems to pass constantly. >>>>>> >>>>>> New webrev: http://cr.openjdk.java.net/~jbachorik/6309226/webrev.03 >>>>>> >>>>>> Thanks, >>>>>> >>>>>> -JB- >>>>>> >>>>>> >>>>>> On 21.10.2013 13:55, Jaroslav Bachorik wrote: >>>>>>> Please, review this small patch for a test failing due to the >>>>>>> updated >>>>>>> implementation in JDK6. >>>>>>> >>>>>>> Issue: https://bugs.openjdk.java.net/browse/JDK-6309226 >>>>>>> Webrev: http://cr.openjdk.java.net/~jbachorik/6309226/webrev.00/ >>>>>>> >>>>>>> The test fails due to the change in mustang where >>>>>>> ThreadMXBean.getThreadInfo().getWaitedTime() and >>>>>>> ThreadMXBean.getThreadInfo().getWaitedCount() include Thread.sleep() >>>>>>> too. Unfortunately, Thread.sleep() is used throughout the test for >>>>>>> synchronization purposes and this breaks the test. >>>>>>> >>>>>>> In the patch I propose to replace Thread.sleep() with busy wait and >>>>>>> hinting the scheduler by Thread.yield(). While not very elegant it >>>>>>> successfully works around inclusion of unknown number of >>>>>>> Thread.sleep()s >>>>>>> (they are called in loop). >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> -JB- >>>>>> >>>> >>