From Alan.Bateman at oracle.com Sun Dec 1 00:23:53 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 01 Dec 2013 08:23:53 +0000 Subject: JDK 8 b117 ea test results are now available In-Reply-To: <529492A5.8020405@oracle.com> References: <529492A5.8020405@oracle.com> Message-ID: <529AF219.40907@oracle.com> On 26/11/2013 12:23, Balchandra Vaidya wrote: > > > JDK 8 b117 ea test results are now available at: > http://www.java.net/download/jdk8/testresults/testresults.html > The results page links to a "howto" page [1]. For the jdk tests then giving jtreg a list of directories (cat dir.list). I don't know if you've tried the jtreg groups yet but I think this should be able to mostly replace the hand maintained list of directories. I think for the subset of the tests that you are running the specifying ":jdk_core :jdk_svc :jdk_beans :jdk_imageio" should cover the tests that you are running. -Alan [1] http://download.java.net/jdk8/testresults/docs/howtoruntests.html -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/quality-discuss/attachments/20131201/5ef1887c/attachment.html From balchandra.vaidya at oracle.com Mon Dec 2 02:12:33 2013 From: balchandra.vaidya at oracle.com (Balchandra Vaidya) Date: Mon, 02 Dec 2013 10:12:33 +0000 Subject: JDK 8 b117 ea test results are now available In-Reply-To: <529AF219.40907@oracle.com> References: <529492A5.8020405@oracle.com> <529AF219.40907@oracle.com> Message-ID: <529C5D11.8060406@oracle.com> Hi Alan, On 12/ 1/13 08:23 AM, Alan Bateman wrote: > On 26/11/2013 12:23, Balchandra Vaidya wrote: >> >> >> JDK 8 b117 ea test results are now available at: >> http://www.java.net/download/jdk8/testresults/testresults.html >> > The results page links to a "howto" page [1]. > > For the jdk tests then giving jtreg a list of directories (cat > dir.list). I don't know if you've tried the jtreg groups yet but I > think this should be able to mostly replace the hand maintained list > of directories. I think for the subset of the tests that you are > running the specifying ":jdk_core :jdk_svc :jdk_beans :jdk_imageio" > should cover the tests that you are running. Yes, it is a good idea to replace hand maintained list. I will run the tests using the jtreg groups and update the "howto" page. Thanks Balchandra > > -Alan > > [1] http://download.java.net/jdk8/testresults/docs/howtoruntests.html -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/quality-discuss/attachments/20131202/f92eb3cd/attachment.html From balchandra.vaidya at oracle.com Tue Dec 3 03:38:18 2013 From: balchandra.vaidya at oracle.com (Balchandra Vaidya) Date: Tue, 03 Dec 2013 11:38:18 +0000 Subject: JDK 8 b118 ea test results are now available Message-ID: <529DC2AA.8090506@oracle.com> JDK 8 b118 ea test results are now available at: http://www.java.net/download/jdk8/testresults/testresults.html The jdk and langtools test results contain zero difference when compared with b117 test results. The hotspot test results have 2 differences. 0: /home/jtest/merge/b117/hotspot/JTwork pass: 424; fail: 14; error: 2; not run: 7 1: /home/jtest/merge/b118/hotspot/JTwork pass: 426; fail: 13; error: 2; not run: 7 0 1 Test --- pass compiler/uncommontrap/TestStackBangRbp.java fail pass runtime/ClassFile/OomWhileParsingRepeatedJsr.java 2 differences No new failures present in Nashorn test result: http://download.java.net/jdk8/testresults/archives/b118/emailable-report.html Regards Balchandra -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/quality-discuss/attachments/20131203/51509b85/attachment.html From volker.simonis at gmail.com Mon Dec 9 10:10:21 2013 From: volker.simonis at gmail.com (Volker Simonis) Date: Mon, 9 Dec 2013 19:10:21 +0100 Subject: JDK 8 b118 ea test results are now available In-Reply-To: <529DC2AA.8090506@oracle.com> References: <529DC2AA.8090506@oracle.com> Message-ID: Hi Balchandra, can you please describe in some more detail how you run these tests? I tried to repeat them with a fresh build of both, OpenJDK and JTreg with the same options as described on the testsresult.html page, but I get the following security exception for the test: com/sun/java/swing/plaf/windows/8016551/bug8016551.java ----------System.err:(33/2049)---------- java.lang.reflect.InvocationTargetException at java.awt.EventQueue.invokeAndWait(EventQueue.java:1300) at java.awt.EventQueue.invokeAndWait(EventQueue.java:1275) at javax.swing.SwingUtilities.invokeAndWait(SwingUtilities.java:1350) at bug8016551.main(bug8016551.java:46) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at com.sun.javatest.regtest.MainAction$SameVMRunnable.run(MainAction.java:754) at java.lang.Thread.run(Thread.java:744) Caused by: java.lang.SecurityException: System.exit() forbidden by JT Harness at com.sun.javatest.JavaTestSecurityManager.checkExit(JavaTestSecurityManager.java:117) at javax.swing.JFrame.setDefaultCloseOperation(JFrame.java:395) at bug8016551$1.run(bug8016551.java:57) at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:301) at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:744) at java.awt.EventQueue.access$400(EventQueue.java:97) at java.awt.EventQueue$3.run(EventQueue.java:697) at java.awt.EventQueue$3.run(EventQueue.java:691) at java.security.AccessController.doPrivileged(Native Method) at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:75) at java.awt.EventQueue.dispatchEvent(EventQueue.java:714) at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201) at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116) at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93) at java.awt.EventDispatchThread.run(EventDispatchThread.java:82) In your results this test is listed as PASSED, so there muxt be something you are doing differently. It seems that in your setup, the agentVM is not running with a security manager, but I've no idea how I can achieve that. Here's my complete command line: JT_JAVA= /jtreg/build/images/jtreg/linux/bin/jtreg -dir:/jdk/test -verbose:summary -exclude:/jdk/test/ProblemList.txt -conc:auto -a -ignore:quiet -timeoutFactor:5 -agentvm -testjdk:/images/j2sdk-image -w:/tmp/JTwork -r:/tmp/JTreport com/sun/java/swing/plaf/windows/8016551/bug8016551.java Regards, Volker On Tue, Dec 3, 2013 at 12:38 PM, Balchandra Vaidya wrote: > > > JDK 8 b118 ea test results are now available at: > http://www.java.net/download/jdk8/testresults/testresults.html > > > The jdk and langtools test results contain zero difference when > compared with b117 test results. The hotspot test results > have 2 differences. > > 0: /home/jtest/merge/b117/hotspot/JTwork pass: 424; fail: 14; error: 2; not > run: 7 > 1: /home/jtest/merge/b118/hotspot/JTwork pass: 426; fail: 13; error: 2; not > run: 7 > > 0 1 Test > --- pass compiler/uncommontrap/TestStackBangRbp.java > fail pass runtime/ClassFile/OomWhileParsingRepeatedJsr.java > > 2 differences > > > No new failures present in Nashorn test result: > > http://download.java.net/jdk8/testresults/archives/b118/emailable-report.html > > > > Regards > Balchandra From balchandra.vaidya at oracle.com Mon Dec 9 11:01:46 2013 From: balchandra.vaidya at oracle.com (Balchandra Vaidya) Date: Mon, 09 Dec 2013 19:01:46 +0000 Subject: JDK 8 b118 ea test results are now available In-Reply-To: References: <529DC2AA.8090506@oracle.com> Message-ID: <52A6139A.1020203@oracle.com> Hi Volker, Does the test pass if the -agentvm option is removed? Thanks Balchandra On 12/ 9/13 06:10 PM, Volker Simonis wrote: > Hi Balchandra, > > can you please describe in some more detail how you run these tests? > > I tried to repeat them with a fresh build of both, OpenJDK and JTreg > with the same options as described on the testsresult.html page, but I > get the following security exception for the test: > > com/sun/java/swing/plaf/windows/8016551/bug8016551.java > > ----------System.err:(33/2049)---------- > java.lang.reflect.InvocationTargetException > at java.awt.EventQueue.invokeAndWait(EventQueue.java:1300) > at java.awt.EventQueue.invokeAndWait(EventQueue.java:1275) > at javax.swing.SwingUtilities.invokeAndWait(SwingUtilities.java:1350) > at bug8016551.main(bug8016551.java:46) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:483) > at com.sun.javatest.regtest.MainAction$SameVMRunnable.run(MainAction.java:754) > at java.lang.Thread.run(Thread.java:744) > Caused by: java.lang.SecurityException: System.exit() forbidden by JT Harness > at com.sun.javatest.JavaTestSecurityManager.checkExit(JavaTestSecurityManager.java:117) > at javax.swing.JFrame.setDefaultCloseOperation(JFrame.java:395) > at bug8016551$1.run(bug8016551.java:57) > at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:301) > at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:744) > at java.awt.EventQueue.access$400(EventQueue.java:97) > at java.awt.EventQueue$3.run(EventQueue.java:697) > at java.awt.EventQueue$3.run(EventQueue.java:691) > at java.security.AccessController.doPrivileged(Native Method) > at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:75) > at java.awt.EventQueue.dispatchEvent(EventQueue.java:714) > at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201) > at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116) > at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105) > at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101) > at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93) > at java.awt.EventDispatchThread.run(EventDispatchThread.java:82) > > In your results this test is listed as PASSED, so there muxt be > something you are doing differently. It seems that in your setup, the > agentVM is not running with a security manager, but I've no idea how I > can achieve that. > > Here's my complete command line: > > JT_JAVA= > /jtreg/build/images/jtreg/linux/bin/jtreg > -dir:/jdk/test -verbose:summary > -exclude:/jdk/test/ProblemList.txt -conc:auto -a > -ignore:quiet -timeoutFactor:5 -agentvm > -testjdk:/images/j2sdk-image -w:/tmp/JTwork > -r:/tmp/JTreport > com/sun/java/swing/plaf/windows/8016551/bug8016551.java > > Regards, > Volker > > > > On Tue, Dec 3, 2013 at 12:38 PM, Balchandra Vaidya > wrote: >> >> JDK 8 b118 ea test results are now available at: >> http://www.java.net/download/jdk8/testresults/testresults.html >> >> >> The jdk and langtools test results contain zero difference when >> compared with b117 test results. The hotspot test results >> have 2 differences. >> >> 0: /home/jtest/merge/b117/hotspot/JTwork pass: 424; fail: 14; error: 2; not >> run: 7 >> 1: /home/jtest/merge/b118/hotspot/JTwork pass: 426; fail: 13; error: 2; not >> run: 7 >> >> 0 1 Test >> --- pass compiler/uncommontrap/TestStackBangRbp.java >> fail pass runtime/ClassFile/OomWhileParsingRepeatedJsr.java >> >> 2 differences >> >> >> No new failures present in Nashorn test result: >> >> http://download.java.net/jdk8/testresults/archives/b118/emailable-report.html >> >> >> >> Regards >> Balchandra From volker.simonis at gmail.com Mon Dec 9 11:09:18 2013 From: volker.simonis at gmail.com (Volker Simonis) Date: Mon, 9 Dec 2013 20:09:18 +0100 Subject: JDK 8 b118 ea test results are now available In-Reply-To: <52A6139A.1020203@oracle.com> References: <529DC2AA.8090506@oracle.com> <52A6139A.1020203@oracle.com> Message-ID: Hi Balchandra, I've tried that, but in that case the test fails because of a sun.awt.SunToolkit$OperationTimedOut() in a call to SunToolkit.realSync(). That may be of course a badly written test (see my other mail to http://mail.openjdk.java.net/pipermail/swing-dev/2013-December/003162.html) - who knows? What I'm actually trying to do is running the regression tests on our AIX port, but that's a real PITA if I can't even reproduce the test results from the jdk8 test-results page on Linux/x86_64. And you list this specific test as passed, which I can not understand how you did it (I assume not with the options and setup listed on that page). Thanks and best regards, Volker On Mon, Dec 9, 2013 at 8:01 PM, Balchandra Vaidya wrote: > > Hi Volker, > > Does the test pass if the -agentvm option is removed? > > > Thanks > Balchandra > > > > > On 12/ 9/13 06:10 PM, Volker Simonis wrote: >> >> Hi Balchandra, >> >> can you please describe in some more detail how you run these tests? >> >> I tried to repeat them with a fresh build of both, OpenJDK and JTreg >> with the same options as described on the testsresult.html page, but I >> get the following security exception for the test: >> >> com/sun/java/swing/plaf/windows/8016551/bug8016551.java >> >> ----------System.err:(33/2049)---------- >> java.lang.reflect.InvocationTargetException >> at java.awt.EventQueue.invokeAndWait(EventQueue.java:1300) >> at java.awt.EventQueue.invokeAndWait(EventQueue.java:1275) >> at >> javax.swing.SwingUtilities.invokeAndWait(SwingUtilities.java:1350) >> at bug8016551.main(bug8016551.java:46) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:483) >> at >> com.sun.javatest.regtest.MainAction$SameVMRunnable.run(MainAction.java:754) >> at java.lang.Thread.run(Thread.java:744) >> Caused by: java.lang.SecurityException: System.exit() forbidden by JT >> Harness >> at >> com.sun.javatest.JavaTestSecurityManager.checkExit(JavaTestSecurityManager.java:117) >> at javax.swing.JFrame.setDefaultCloseOperation(JFrame.java:395) >> at bug8016551$1.run(bug8016551.java:57) >> at >> java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:301) >> at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:744) >> at java.awt.EventQueue.access$400(EventQueue.java:97) >> at java.awt.EventQueue$3.run(EventQueue.java:697) >> at java.awt.EventQueue$3.run(EventQueue.java:691) >> at java.security.AccessController.doPrivileged(Native Method) >> at >> java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:75) >> at java.awt.EventQueue.dispatchEvent(EventQueue.java:714) >> at >> java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201) >> at >> java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116) >> at >> java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105) >> at >> java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101) >> at >> java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93) >> at java.awt.EventDispatchThread.run(EventDispatchThread.java:82) >> >> In your results this test is listed as PASSED, so there muxt be >> something you are doing differently. It seems that in your setup, the >> agentVM is not running with a security manager, but I've no idea how I >> can achieve that. >> >> Here's my complete command line: >> >> JT_JAVA= >> /jtreg/build/images/jtreg/linux/bin/jtreg >> -dir:/jdk/test -verbose:summary >> -exclude:/jdk/test/ProblemList.txt -conc:auto -a >> -ignore:quiet -timeoutFactor:5 -agentvm >> -testjdk:/images/j2sdk-image -w:/tmp/JTwork >> -r:/tmp/JTreport >> com/sun/java/swing/plaf/windows/8016551/bug8016551.java >> >> Regards, >> Volker >> >> >> >> On Tue, Dec 3, 2013 at 12:38 PM, Balchandra Vaidya >> wrote: >>> >>> >>> JDK 8 b118 ea test results are now available at: >>> http://www.java.net/download/jdk8/testresults/testresults.html >>> >>> >>> The jdk and langtools test results contain zero difference when >>> compared with b117 test results. The hotspot test results >>> have 2 differences. >>> >>> 0: /home/jtest/merge/b117/hotspot/JTwork pass: 424; fail: 14; error: 2; >>> not >>> run: 7 >>> 1: /home/jtest/merge/b118/hotspot/JTwork pass: 426; fail: 13; error: 2; >>> not >>> run: 7 >>> >>> 0 1 Test >>> --- pass compiler/uncommontrap/TestStackBangRbp.java >>> fail pass runtime/ClassFile/OomWhileParsingRepeatedJsr.java >>> >>> 2 differences >>> >>> >>> No new failures present in Nashorn test result: >>> >>> >>> http://download.java.net/jdk8/testresults/archives/b118/emailable-report.html >>> >>> >>> >>> Regards >>> Balchandra > > From balchandra.vaidya at oracle.com Tue Dec 10 03:20:40 2013 From: balchandra.vaidya at oracle.com (Balchandra Vaidya) Date: Tue, 10 Dec 2013 11:20:40 +0000 Subject: JDK 8 b118 ea test results are now available In-Reply-To: References: <529DC2AA.8090506@oracle.com> <52A6139A.1020203@oracle.com> Message-ID: <52A6F908.3050908@oracle.com> Hi Volker, Thank you for the feedback. The swing-dev was the correct mailing list to discuss the quality of this testcase. It looks like there are two issues. 1) realSync() fails with timeout. It appears, this is a known issue when the testcase was run under Xfce. http://mail.openjdk.java.net/pipermail/swing-dev/2013-December/003165.html 2) Testcase do not run with -agentvm option. It might be an issue in the testcase (or how the testcase was run). I will look into it. I missed this issue because I had changed my scripts to use -othervm sometime back for debugging/analyzing some testcase failure but forgot to change it back. Thank you for pointing it out. Thanks Balchandra On 12/ 9/13 07:09 PM, Volker Simonis wrote: > Hi Balchandra, > > I've tried that, but in that case the test fails because of a > sun.awt.SunToolkit$OperationTimedOut() in a call to > SunToolkit.realSync(). That may be of course a badly written test (see > my other mail to > http://mail.openjdk.java.net/pipermail/swing-dev/2013-December/003162.html) > - who knows? > > What I'm actually trying to do is running the regression tests on our > AIX port, but that's a real PITA if I can't even reproduce the test > results from the jdk8 test-results page on Linux/x86_64. And you list > this specific test as passed, which I can not understand how you did > it (I assume not with the options and setup listed on that page). > > Thanks and best regards, > Volker > > On Mon, Dec 9, 2013 at 8:01 PM, Balchandra Vaidya > wrote: >> Hi Volker, >> >> Does the test pass if the -agentvm option is removed? >> >> >> Thanks >> Balchandra >> >> >> >> >> On 12/ 9/13 06:10 PM, Volker Simonis wrote: >>> Hi Balchandra, >>> >>> can you please describe in some more detail how you run these tests? >>> >>> I tried to repeat them with a fresh build of both, OpenJDK and JTreg >>> with the same options as described on the testsresult.html page, but I >>> get the following security exception for the test: >>> >>> com/sun/java/swing/plaf/windows/8016551/bug8016551.java >>> >>> ----------System.err:(33/2049)---------- >>> java.lang.reflect.InvocationTargetException >>> at java.awt.EventQueue.invokeAndWait(EventQueue.java:1300) >>> at java.awt.EventQueue.invokeAndWait(EventQueue.java:1275) >>> at >>> javax.swing.SwingUtilities.invokeAndWait(SwingUtilities.java:1350) >>> at bug8016551.main(bug8016551.java:46) >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> at java.lang.reflect.Method.invoke(Method.java:483) >>> at >>> com.sun.javatest.regtest.MainAction$SameVMRunnable.run(MainAction.java:754) >>> at java.lang.Thread.run(Thread.java:744) >>> Caused by: java.lang.SecurityException: System.exit() forbidden by JT >>> Harness >>> at >>> com.sun.javatest.JavaTestSecurityManager.checkExit(JavaTestSecurityManager.java:117) >>> at javax.swing.JFrame.setDefaultCloseOperation(JFrame.java:395) >>> at bug8016551$1.run(bug8016551.java:57) >>> at >>> java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:301) >>> at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:744) >>> at java.awt.EventQueue.access$400(EventQueue.java:97) >>> at java.awt.EventQueue$3.run(EventQueue.java:697) >>> at java.awt.EventQueue$3.run(EventQueue.java:691) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at >>> java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:75) >>> at java.awt.EventQueue.dispatchEvent(EventQueue.java:714) >>> at >>> java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201) >>> at >>> java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116) >>> at >>> java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105) >>> at >>> java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101) >>> at >>> java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93) >>> at java.awt.EventDispatchThread.run(EventDispatchThread.java:82) >>> >>> In your results this test is listed as PASSED, so there muxt be >>> something you are doing differently. It seems that in your setup, the >>> agentVM is not running with a security manager, but I've no idea how I >>> can achieve that. >>> >>> Here's my complete command line: >>> >>> JT_JAVA= >>> /jtreg/build/images/jtreg/linux/bin/jtreg >>> -dir:/jdk/test -verbose:summary >>> -exclude:/jdk/test/ProblemList.txt -conc:auto -a >>> -ignore:quiet -timeoutFactor:5 -agentvm >>> -testjdk:/images/j2sdk-image -w:/tmp/JTwork >>> -r:/tmp/JTreport >>> com/sun/java/swing/plaf/windows/8016551/bug8016551.java >>> >>> Regards, >>> Volker >>> >>> >>> >>> On Tue, Dec 3, 2013 at 12:38 PM, Balchandra Vaidya >>> wrote: >>>> >>>> JDK 8 b118 ea test results are now available at: >>>> http://www.java.net/download/jdk8/testresults/testresults.html >>>> >>>> >>>> The jdk and langtools test results contain zero difference when >>>> compared with b117 test results. The hotspot test results >>>> have 2 differences. >>>> >>>> 0: /home/jtest/merge/b117/hotspot/JTwork pass: 424; fail: 14; error: 2; >>>> not >>>> run: 7 >>>> 1: /home/jtest/merge/b118/hotspot/JTwork pass: 426; fail: 13; error: 2; >>>> not >>>> run: 7 >>>> >>>> 0 1 Test >>>> --- pass compiler/uncommontrap/TestStackBangRbp.java >>>> fail pass runtime/ClassFile/OomWhileParsingRepeatedJsr.java >>>> >>>> 2 differences >>>> >>>> >>>> No new failures present in Nashorn test result: >>>> >>>> >>>> http://download.java.net/jdk8/testresults/archives/b118/emailable-report.html >>>> >>>> >>>> >>>> Regards >>>> Balchandra >> From Alan.Bateman at oracle.com Tue Dec 10 07:25:48 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 10 Dec 2013 15:25:48 +0000 Subject: JDK 8 b118 ea test results are now available In-Reply-To: <52A6F908.3050908@oracle.com> References: <529DC2AA.8090506@oracle.com> <52A6139A.1020203@oracle.com> <52A6F908.3050908@oracle.com> Message-ID: <52A7327C.7050202@oracle.com> On 10/12/2013 11:20, Balchandra Vaidya wrote: > : > > 2) Testcase do not run with -agentvm option. It might be an > issue in the testcase (or how the testcase was run). I will look > into it. > > I missed this issue because I had changed my scripts to use -othervm > sometime back for debugging/analyzing some testcase > failure but forgot to change it back. Thank you for pointing it out. For Volker's benefit, othervm is where jtreg spins up a new VM for each test, agentvm is where jtreg re-uses the VM if possible. There are tests that don't or can't clean up and those tests need to ensure that their @run tests have the /othervm option so that the tests runs in its own VM. Alternatively, the TEST.ROOT file has a key that lists directories where all the tests in those directory trees must run in their own VM. Balchandra - one thing that be useful is to look at the client tests to see what can run in agentvm and what can't. For the jdk_core and jdk_svc tests then they can be run in either (although we're still picking off issues as they found in agentvm mode). I'm not aware of any work done yet on getting the tests in the jdk_desktop test group to run in agentvm mode. It may be that TEST.ROOT needs to be updated to see a few additional top-level directories so that all the tests in these areas temporarily run in othervm mode. -Alan From volker.simonis at gmail.com Tue Dec 10 07:37:12 2013 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 10 Dec 2013 16:37:12 +0100 Subject: System.exit() forbidden by JT Harness in agentvm mode Message-ID: Hi, when running the com/sun/java/swing/plaf/windows/8016551/bug8016551.java regression test with the newest version of jtreg (build from source) in agentvm mode I get the following error: ----------System.err:(33/2049)---------- java.lang.reflect.InvocationTargetException at java.awt.EventQueue.invokeAndWait(EventQueue.java:1300) at java.awt.EventQueue.invokeAndWait(EventQueue.java:1275) at javax.swing.SwingUtilities.invokeAndWait(SwingUtilities.java:1350) at bug8016551.main(bug8016551.java:46) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at com.sun.javatest.regtest.MainAction$SameVMRunnable.run(MainAction.java:754) at java.lang.Thread.run(Thread.java:744) Caused by: java.lang.SecurityException: System.exit() forbidden by JT Harness at com.sun.javatest.JavaTestSecurityManager.checkExit(JavaTestSecurityManager.java:117) at javax.swing.JFrame.setDefaultCloseOperation(JFrame.java:395) at bug8016551$1.run(bug8016551.java:57) at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:301) at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:744) at java.awt.EventQueue.access$400(EventQueue.java:97) at java.awt.EventQueue$3.run(EventQueue.java:697) at java.awt.EventQueue$3.run(EventQueue.java:691) at java.security.AccessController.doPrivileged(Native Method) at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:75) at java.awt.EventQueue.dispatchEvent(EventQueue.java:714) at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201) at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116) at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93) at java.awt.EventDispatchThread.run(EventDispatchThread.java:82) If the test is run in othervm mode it succeeds. I looked at this problem a litter loser and found out the following: - there are 99 test which use JFrame.setDefaultCloseOperation(EXIT_ON_CLOSE) - they are in java/awt (15), javax/swing (79), sun/java2d (3), javax/imageio/plugins/gif/GifTransparencyTest.java (setDefaultCloseOperation present but never called) and com/sun/java/swing/plaf/windows/8016551/bug8016551.java - all test in java/awt and sun/java2d are always executed in othervm mode anyway (because of 'othervm.dirs=java/awt ..' in TEST.ROOT) Now I'm not sure if this security-exception is right at this place - i.e. if jtreg/jtharness work as expected here. If yes and if there's no option/workaround to switch this behaviour off, we should either add 'javax/swing' as well to the 'othervm.dirs' list in TEST.ROOT and explicitly flag the remain test to require othervm mode by adding the corresponding '@run main/othervm' tag to the java source file. If adding 'javax/swing' to 'othervm.dirs' is considered to general, we would have to explicitly flag each single test with '@run main/othervm'. What do you think? Volker PS: there are 208 test which call System.exit() directly. I haven't analysed them until now, but the same reasoning applies for them as well. From balchandra.vaidya at oracle.com Tue Dec 10 08:39:46 2013 From: balchandra.vaidya at oracle.com (Balchandra Vaidya) Date: Tue, 10 Dec 2013 16:39:46 +0000 Subject: JDK 8 b118 ea test results are now available In-Reply-To: <52A7327C.7050202@oracle.com> References: <529DC2AA.8090506@oracle.com> <52A6139A.1020203@oracle.com> <52A6F908.3050908@oracle.com> <52A7327C.7050202@oracle.com> Message-ID: <52A743D2.2010104@oracle.com> On 12/10/13 03:25 PM, Alan Bateman wrote: > On 10/12/2013 11:20, Balchandra Vaidya wrote: >> : >> >> 2) Testcase do not run with -agentvm option. It might be an >> issue in the testcase (or how the testcase was run). I will look >> into it. >> >> I missed this issue because I had changed my scripts to use -othervm >> sometime back for debugging/analyzing some testcase >> failure but forgot to change it back. Thank you for pointing it out. > For Volker's benefit, othervm is where jtreg spins up a new VM for > each test, agentvm is where jtreg re-uses the VM if possible. There > are tests that don't or can't clean up and those tests need to ensure > that their @run tests have the /othervm option so that the tests runs > in its own VM. Alternatively, the TEST.ROOT file has a key that lists > directories where all the tests in those directory trees must run in > their own VM. > > Balchandra - one thing that be useful is to look at the client tests > to see what can run in agentvm and what can't. For the jdk_core and > jdk_svc tests then they can be run in either (although we're still > picking off issues as they found in agentvm mode). I'm not aware of > any work done yet on getting the tests in the jdk_desktop test group > to run in agentvm mode. It may be that TEST.ROOT needs to be updated > to see a few additional top-level directories so that all the tests in > these areas temporarily run in othervm mode. In the subset of the tests I have been running, it appears only one test [1] is failing with agentvm option. I will try to run other tests in client groups [Note: I will update the instruction to use jtreg groups with b119 results posting] and see what tests can be run in agentvm mode, but main issue in those remaining client tests are some tests may not consistently produce same results - could stabilizing those tests be a goal for jdk9 repo ? Thanks Balchandra [1] com/sun/java/swing/plaf/windows/8016551/bug8016551.java > > -Alan From jonathan.gibbons at oracle.com Tue Dec 10 08:47:52 2013 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Tue, 10 Dec 2013 08:47:52 -0800 Subject: System.exit() forbidden by JT Harness in agentvm mode In-Reply-To: References: Message-ID: <52A745B8.1070705@oracle.com> Volker, From the history books, initially there was only "samevm" and "othervm" mode and "othervm" mode was the default. "samevm" mode was problematic in the face of bad tests (especially bad tests on Windows) and only the langtools test suite evolved to the point where we recommended the use of "samevm" mode. As a result, "agentvm" mode was added, which is like samevm, but better. It has worked well for most tests, but as you have noticed, does not work well if a test tries to exit the JVM. Those tests need some amount of TLC if they are to leverage "agentvm" mode. You can use othervm.dirs in TEST.ROOT, but that was added as a pragmatic, stop gap measure, to faciitate running most of the tests in the faster "agentvm" mode, while allowing some older parts of the test suite to continue running in the slower "othervm" world. The core-libs team have led an effort to clean up the core-libs tests and to provide the test/Makefile as a standard way of running the tests. It seems like it is time to embark on a similar effort for the client tests. -- Jon On 12/10/2013 07:37 AM, Volker Simonis wrote: > Hi, > > when running the > com/sun/java/swing/plaf/windows/8016551/bug8016551.java regression > test with the newest version of jtreg (build from source) in agentvm > mode I get the following error: > > ----------System.err:(33/2049)---------- > java.lang.reflect.InvocationTargetException > at java.awt.EventQueue.invokeAndWait(EventQueue.java:1300) > at java.awt.EventQueue.invokeAndWait(EventQueue.java:1275) > at javax.swing.SwingUtilities.invokeAndWait(SwingUtilities.java:1350) > at bug8016551.main(bug8016551.java:46) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:483) > at com.sun.javatest.regtest.MainAction$SameVMRunnable.run(MainAction.java:754) > at java.lang.Thread.run(Thread.java:744) > Caused by: java.lang.SecurityException: System.exit() forbidden by JT Harness > at com.sun.javatest.JavaTestSecurityManager.checkExit(JavaTestSecurityManager.java:117) > at javax.swing.JFrame.setDefaultCloseOperation(JFrame.java:395) > at bug8016551$1.run(bug8016551.java:57) > at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:301) > at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:744) > at java.awt.EventQueue.access$400(EventQueue.java:97) > at java.awt.EventQueue$3.run(EventQueue.java:697) > at java.awt.EventQueue$3.run(EventQueue.java:691) > at java.security.AccessController.doPrivileged(Native Method) > at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:75) > at java.awt.EventQueue.dispatchEvent(EventQueue.java:714) > at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201) > at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116) > at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105) > at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101) > at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93) > at java.awt.EventDispatchThread.run(EventDispatchThread.java:82) > > If the test is run in othervm mode it succeeds. > > I looked at this problem a litter loser and found out the following: > > - there are 99 test which use JFrame.setDefaultCloseOperation(EXIT_ON_CLOSE) > - they are in java/awt (15), javax/swing (79), sun/java2d (3), > javax/imageio/plugins/gif/GifTransparencyTest.java > (setDefaultCloseOperation present but never called) and > com/sun/java/swing/plaf/windows/8016551/bug8016551.java > - all test in java/awt and sun/java2d are always executed in othervm > mode anyway (because of 'othervm.dirs=java/awt ..' in TEST.ROOT) > > Now I'm not sure if this security-exception is right at this place - > i.e. if jtreg/jtharness work as expected here. > > If yes and if there's no option/workaround to switch this behaviour > off, we should either add 'javax/swing' as well to the 'othervm.dirs' > list in TEST.ROOT and explicitly flag the remain test to require > othervm mode by adding the corresponding '@run main/othervm' tag to > the java source file. > > If adding 'javax/swing' to 'othervm.dirs' is considered to general, we > would have to explicitly flag each single test with '@run > main/othervm'. > > What do you think? > > Volker > > PS: there are 208 test which call System.exit() directly. I haven't > analysed them until now, but the same reasoning applies for them as > well. From Alan.Bateman at oracle.com Tue Dec 10 08:58:39 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 10 Dec 2013 16:58:39 +0000 Subject: JDK 8 b118 ea test results are now available In-Reply-To: <52A743D2.2010104@oracle.com> References: <529DC2AA.8090506@oracle.com> <52A6139A.1020203@oracle.com> <52A6F908.3050908@oracle.com> <52A7327C.7050202@oracle.com> <52A743D2.2010104@oracle.com> Message-ID: <52A7483F.2080700@oracle.com> On 10/12/2013 16:39, Balchandra Vaidya wrote: > : > > In the subset of the tests I have been running, it appears only one > test [1] > is failing with agentvm option. I will try to run other tests in client > groups [Note: I will update the instruction to use jtreg groups with b119 > results posting] and see what tests can be run in agentvm mode, but > main issue > in those remaining client tests are some tests may not consistently > produce > same results - could stabilizing those tests be a goal for jdk9 repo ? If the tests are unstable then it is pointless running them. Do you know if there are bugs submitted for these issues? If so then it would be good to add the "teststablization" label so that they can be tracked. Also if it doesn't look like the issues can be fixed quickly then excluding the tests (either by @ignore or the ProblemList.txt if they only need to be excluded on some platforms) would be good so that others don't waste time running them. I think it would be great to have a goal to get these tests stable in the jdk9 forest. I don't know how much effort you have to put into it but it would be good to at least get some handle on the extent of the issue. -Alan. From Alan.Bateman at oracle.com Tue Dec 10 09:17:10 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 10 Dec 2013 17:17:10 +0000 Subject: System.exit() forbidden by JT Harness in agentvm mode In-Reply-To: <52A745B8.1070705@oracle.com> References: <52A745B8.1070705@oracle.com> Message-ID: <52A74C96.6020201@oracle.com> On 10/12/2013 16:47, Jonathan Gibbons wrote: > : > > The core-libs team have led an effort to clean up the core-libs tests > and to provide the test/Makefile as a standard way of running the > tests. It seems like it is time to embark on a similar effort for the > client tests. Just to say that the Makefile now translates targets into jtreg groups so do you can do "make test TEST=" then it will run the tests in that group. The client areas tests are currently split into 6 sub-groups: jdk_desktop = \ :jdk_awt \ :jdk_2d \ :jdk_beans \ :jdk_swing \ :jdk_sound \ :jdk_imageio So you can specify TEST=jdk_desktop if you are feeling lucky or run subsets of the tests. The one thing is that the Makefile uses agentvm mode and as we've seen, these client area tests may need work in order to run in this mode. -Alan. From volker.simonis at gmail.com Tue Dec 10 14:02:04 2013 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 10 Dec 2013 23:02:04 +0100 Subject: System.exit() forbidden by JT Harness in agentvm mode In-Reply-To: <52A745B8.1070705@oracle.com> References: <52A745B8.1070705@oracle.com> Message-ID: Hi Jonathan, thanks for you comments. But there's still one thing I don't understand: what's the exact reason that tests in the agentvm mode are running under a special security manager while test running in othervm mode aren't restricted in this way. Is this really necessary? Regards, Volker On Tuesday, December 10, 2013, Jonathan Gibbons wrote: > Volker, > > From the history books, initially there was only "samevm" and "othervm" > mode and "othervm" mode was the default. > > "samevm" mode was problematic in the face of bad tests (especially bad > tests on Windows) and only the langtools test suite evolved to the point > where we recommended the use of "samevm" mode. > > As a result, "agentvm" mode was added, which is like > samevm, but better. It has worked well for most tests, > but as you have noticed, does not work well if a test tries to exit the > JVM. Those tests need some amount of TLC if they are to leverage "agentvm" > mode. > > You can use othervm.dirs in TEST.ROOT, but that was added as a pragmatic, > stop gap measure, to faciitate running most of the tests in the faster > "agentvm" mode, while allowing some older parts of the test suite to > continue running in the slower "othervm" world. > > The core-libs team have led an effort to clean up the core-libs tests and > to provide the test/Makefile as a standard way of running the tests. It > seems like it is time to embark on a similar effort for the client tests. > > -- Jon > > On 12/10/2013 07:37 AM, Volker Simonis wrote: > >> Hi, >> >> when running the >> com/sun/java/swing/plaf/windows/8016551/bug8016551.java regression >> test with the newest version of jtreg (build from source) in agentvm >> mode I get the following error: >> >> ----------System.err:(33/2049)---------- >> java.lang.reflect.InvocationTargetException >> at java.awt.EventQueue.invokeAndWait(EventQueue.java:1300) >> at java.awt.EventQueue.invokeAndWait(EventQueue.java:1275) >> at javax.swing.SwingUtilities.invokeAndWait(SwingUtilities. >> java:1350) >> at bug8016551.main(bug8016551.java:46) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at sun.reflect.NativeMethodAccessorImpl.invoke( >> NativeMethodAccessorImpl.java:62) >> at sun.reflect.DelegatingMethodAccessorImpl.invoke( >> DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:483) >> at com.sun.javatest.regtest.MainAction$SameVMRunnable.run( >> MainAction.java:754) >> at java.lang.Thread.run(Thread.java:744) >> Caused by: java.lang.SecurityException: System.exit() forbidden by JT >> Harness >> at com.sun.javatest.JavaTestSecurityManager.checkExit( >> JavaTestSecurityManager.java:117) >> at javax.swing.JFrame.setDefaultCloseOperation(JFrame.java:395) >> at bug8016551$1.run(bug8016551.java:57) >> at java.awt.event.InvocationEvent.dispatch( >> InvocationEvent.java:301) >> at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:744) >> at java.awt.EventQueue.access$400(EventQueue.java:97) >> at java.awt.EventQueue$3.run(EventQueue.java:697) >> at java.awt.EventQueue$3.run(EventQueue.java:691) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.security.ProtectionDomain$1.doIntersectionPrivilege( >> ProtectionDomain.java:75) >> at java.awt.EventQueue.dispatchEvent(EventQueue.java:714) >> at java.awt.EventDispatchThread.pumpOneEventForFilters( >> EventDispatchThread.java:201) >> at java.awt.EventDispatchThread.pumpEventsForFilter( >> EventDispatchThread.java:116) >> at java.awt.EventDispatchThread.pumpEventsForHierarchy( >> EventDispatchThread.java:105) >> at java.awt.EventDispatchThread.pumpEvents( >> EventDispatchThread.java:101) >> at java.awt.EventDispatchThread.pumpEvents( >> EventDispatchThread.java:93) >> at java.awt.EventDispatchThread.run(EventDispatchThread.java:82) >> >> If the test is run in othervm mode it succeeds. >> >> I looked at this problem a litter loser and found out the following: >> >> - there are 99 test which use JFrame.setDefaultCloseOperation(EXIT_ >> ON_CLOSE) >> - they are in java/awt (15), javax/swing (79), sun/java2d (3), >> javax/imageio/plugins/gif/GifTransparencyTest.java >> (setDefaultCloseOperation present but never called) and >> com/sun/java/swing/plaf/windows/8016551/bug8016551.java >> - all test in java/awt and sun/java2d are always executed in othervm >> mode anyway (because of 'othervm.dirs=java/awt ..' in TEST.ROOT) >> >> Now I'm not sure if this security-exception is right at this place - >> i.e. if jtreg/jtharness work as expected here. >> >> If yes and if there's no option/workaround to switch this behaviour >> off, we should either add 'javax/swing' as well to the 'othervm.dirs' >> list in TEST.ROOT and explicitly flag the remain test to require >> othervm mode by adding the corresponding '@run main/othervm' tag to >> the java source file. >> >> If adding 'javax/swing' to 'othervm.dirs' is considered to general, we >> would have to explicitly flag each single test with '@run >> main/othervm'. >> >> What do you think? >> >> Volker >> >> PS: there are 208 test which call System.exit() directly. I haven't >> analysed them until now, but the same reasoning applies for them as >> well. >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/quality-discuss/attachments/20131210/3ec5e812/attachment.html From jonathan.gibbons at oracle.com Tue Dec 10 14:04:19 2013 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Tue, 10 Dec 2013 14:04:19 -0800 Subject: System.exit() forbidden by JT Harness in agentvm mode In-Reply-To: References: <52A745B8.1070705@oracle.com> Message-ID: <52A78FE3.2000300@oracle.com> In othervm mode, no restrictions are placed on the JVM, and by default, no security manager is installed, so tests can play with all those "dangerous" calls in java.lang.System all they want. We need to be able to write tests that call System.exit or System.setProperties(new Properties()); etc By contrast, in samevm and agentvm modes, there is a presumption that a test will be reasonably well behaved, but just in case, we install a security manager to make sure. -- Jon P.S. I remember when someone wrote a test for System.setProperties that removed the definition of System.lineSeparator, causing jtreg to crash as soon as it tried to write the test results. On 12/10/2013 02:02 PM, Volker Simonis wrote: > > Hi Jonathan, > > thanks for you comments. But there's still one thing I > don't understand: what's the exact reason that tests in the agentvm > mode are running under a special security manager while test running > in othervm mode aren't restricted in this way. Is this really necessary? > > Regards, > Volker From balchandra.vaidya at oracle.com Wed Dec 11 03:58:10 2013 From: balchandra.vaidya at oracle.com (Balchandra Vaidya) Date: Wed, 11 Dec 2013 11:58:10 +0000 Subject: JDK 8 b119 ea test results are now available Message-ID: <52A85352.702@oracle.com> JDK 8 b119 ea test results are now available at: http://www.java.net/download/jdk8/testresults/testresults.html The jdk test results contain 29 differences when compared with b118 test results. 0: /home/jtest/merge/b118/jdk/JTwork pass: 4,633; fail: 13; not run: 904 1: /home/jtest/merge/b119/jdk/JTwork pass: 4,646; fail: 13; not run: 904 0 1 Test --- pass com/sun/corba/se/impl/orb/SetDefaultORBTest.java --- pass com/sun/crypto/provider/Cipher/AES/TestCopySafe.java --- pass com/sun/jdi/ProcessAttachTest.sh --- pass com/sun/net/httpserver/Test9a.java --- pass demo/jvmti/hprof/HeapAllTest.java --- pass demo/jvmti/hprof/HeapBinaryFormatTest.java --- pass demo/jvmti/hprof/HeapDumpTest.java --- pass demo/jvmti/hprof/OptionsTest.java --- pass demo/jvmti/hprof/StackMapTableTest.java --- pass java/beans/XMLDecoder/8028054/TestConstructorFinder.java --- pass java/beans/XMLDecoder/8028054/TestMethodFinder.java --- pass java/io/pathNames/GeneralSolaris.java --- pass java/lang/annotation/typeAnnotations/BadCPIndex.java pass --- java/lang/instrument/PremainClass/NoPremainAgent.sh --- pass java/lang/instrument/PremainClass/NoPremainAgentTest.java --- pass java/lang/instrument/PremainClass/PremainClassTest.java pass --- java/lang/instrument/PremainClass/PremainClassTest.sh pass --- java/lang/instrument/PremainClass/ZeroArgPremainAgent.sh --- pass java/lang/instrument/PremainClass/ZeroArgPremainAgentTest.java pass --- java/lang/reflect/Method/invoke/TestPrivateInterfaceMethodReflect.java --- pass java/net/InetAddress/CheckJNI.java pass --- java/text/Bidi/Bug6665028.java --- pass java/util/logging/XMLFormatterDate.java --- pass javax/xml/jaxp/transform/8004476/XPathExFuncTest.java --- pass javax/xml/jaxp/transform/8004476/XSLTExFuncTest.java pass --- javax/xml/jaxp/transform/jdk8004476/XPathExFuncTest.java pass --- javax/xml/jaxp/transform/jdk8004476/XSLTExFuncTest.java pass --- sun/jvmstat/monitor/MonitoredVm/MonitorVmStartTerminate.sh --- pass sun/security/pkcs11/Signature/TestDSAKeyLength.java 29 differences In addition to the above difference, one testcase [1] fails only with jtreg agentvm mode, but the testcase passes with jtreg othervm mode. There is an open bug [2] for it. The hotspot test results have 8 differences. 0: /home/jtest/merge/b118/hotspot/JTwork pass: 426; fail: 13; error: 2; not run: 7 1: /home/jtest/merge/b119/hotspot/JTwork pass: 420; fail: 11; error: 2; not run: 7 0 1 Test pass --- runtime/6626217/Test6626217.sh pass --- runtime/6929067/Test6929067.sh pass --- runtime/CDSCompressedKPtrs/XShareAuto.java pass --- runtime/InitialThreadOverflow/testme.sh fail --- runtime/LoadClass/LoadClassNegative.java pass --- runtime/XCheckJniJsig/XCheckJSig.java pass --- runtime/jsig/Test8017498.sh fail --- runtime/memory/ReadFromNoaccessArea.java 8 differences The langtools test results have 6 differences. 0: /home/jtest/merge/b118/langtools/JTwork pass: 2,954; not run: 7 1: /home/jtest/merge/b119/langtools/JTwork pass: 2,960; not run: 7 0 1 Test --- pass com/sun/javadoc/testCompletionFailure/TestCompletionFailure.java --- pass tools/javac/T8028504/DontGenerateLVTForGNoneOpTest.java --- pass tools/javac/annotations/AnnotationTypeElementModifiers.java --- pass tools/javac/declaration/method/MethodVoidParameter.java --- pass tools/javac/expression/_super/NonDirectSuper/NonDirectSuper.java --- pass tools/javac/lambda/methodReferenceExecution/MethodReferenceTestMethodHandle.java 6 differences No new failures present in Nashorn test result: http://download.java.net/jdk8/testresults/archives/b119/emailable-report.html Regards Balchandra [1] test/com/sun/java/swing/plaf/windows/8016551/bug8016551.java [2] https://bugs.openjdk.java.net/browse/JDK-8029954 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/quality-discuss/attachments/20131211/ca9dedd0/attachment-0001.html From volker.simonis at gmail.com Wed Dec 11 05:45:52 2013 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 11 Dec 2013 14:45:52 +0100 Subject: JDK 8 b118 ea test results are now available In-Reply-To: <52A743D2.2010104@oracle.com> References: <529DC2AA.8090506@oracle.com> <52A6139A.1020203@oracle.com> <52A6F908.3050908@oracle.com> <52A7327C.7050202@oracle.com> <52A743D2.2010104@oracle.com> Message-ID: Hi Balchandra, what do you think about checking in a little script or Makefile target which produces the test results you report for the ea builds? This would be of great help to produce comparable test results. Regards, Volker On Tue, Dec 10, 2013 at 5:39 PM, Balchandra Vaidya wrote: > > On 12/10/13 03:25 PM, Alan Bateman wrote: >> >> On 10/12/2013 11:20, Balchandra Vaidya wrote: >>> >>> : >>> >>> 2) Testcase do not run with -agentvm option. It might be an >>> issue in the testcase (or how the testcase was run). I will look into >>> it. >>> >>> I missed this issue because I had changed my scripts to use -othervm >>> sometime back for debugging/analyzing some testcase >>> failure but forgot to change it back. Thank you for pointing it out. >> >> For Volker's benefit, othervm is where jtreg spins up a new VM for each >> test, agentvm is where jtreg re-uses the VM if possible. There are tests >> that don't or can't clean up and those tests need to ensure that their @run >> tests have the /othervm option so that the tests runs in its own VM. >> Alternatively, the TEST.ROOT file has a key that lists directories where all >> the tests in those directory trees must run in their own VM. >> >> Balchandra - one thing that be useful is to look at the client tests to >> see what can run in agentvm and what can't. For the jdk_core and jdk_svc >> tests then they can be run in either (although we're still picking off >> issues as they found in agentvm mode). I'm not aware of any work done yet on >> getting the tests in the jdk_desktop test group to run in agentvm mode. It >> may be that TEST.ROOT needs to be updated to see a few additional top-level >> directories so that all the tests in these areas temporarily run in othervm >> mode. > > > In the subset of the tests I have been running, it appears only one test [1] > is failing with agentvm option. I will try to run other tests in client > groups [Note: I will update the instruction to use jtreg groups with b119 > results posting] and see what tests can be run in agentvm mode, but main > issue > in those remaining client tests are some tests may not consistently produce > same results - could stabilizing those tests be a goal for jdk9 repo ? > > > Thanks > Balchandra > > [1] com/sun/java/swing/plaf/windows/8016551/bug8016551.java > >> >> -Alan > > From jonathan.gibbons at oracle.com Wed Dec 11 08:53:55 2013 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Wed, 11 Dec 2013 08:53:55 -0800 Subject: JDK 8 b118 ea test results are now available In-Reply-To: References: <529DC2AA.8090506@oracle.com> <52A6139A.1020203@oracle.com> <52A6F908.3050908@oracle.com> <52A7327C.7050202@oracle.com> <52A743D2.2010104@oracle.com> Message-ID: <52A898A3.5030507@oracle.com> On 12/11/2013 05:45 AM, Volker Simonis wrote: > Hi Balchandra, > > what do you think about checking in a little script or Makefile target > which produces the test results you report for the ea builds? > > This would be of great help to produce comparable test results. > > Regards, > Volker +1 -- Jon From balchandra.vaidya at oracle.com Wed Dec 11 08:58:54 2013 From: balchandra.vaidya at oracle.com (Balchandra Vaidya) Date: Wed, 11 Dec 2013 16:58:54 +0000 Subject: JDK 8 b118 ea test results are now available In-Reply-To: References: <529DC2AA.8090506@oracle.com> <52A6139A.1020203@oracle.com> <52A6F908.3050908@oracle.com> <52A7327C.7050202@oracle.com> <52A743D2.2010104@oracle.com> Message-ID: <52A899CE.6090408@oracle.com> Hi Volker, The custom script I use just wrap the instruction posted (except building jtreg). It just clones the openjdk repository, calls jtreg commands (one each for jdk, langtools and hotspot) with those recommended options, runs diffs, runs nashorn tests, and archives results. Such convenient script may not fit under openjdk repo, but think of adding it in code-tools project in the future. Thanks Balchandra On 11/12/2013 13:45, Volker Simonis wrote: > Hi Balchandra, > > what do you think about checking in a little script or Makefile target > which produces the test results you report for the ea builds? > > This would be of great help to produce comparable test results. > > Regards, > Volker > > On Tue, Dec 10, 2013 at 5:39 PM, Balchandra Vaidya > wrote: >> On 12/10/13 03:25 PM, Alan Bateman wrote: >>> On 10/12/2013 11:20, Balchandra Vaidya wrote: >>>> : >>>> >>>> 2) Testcase do not run with -agentvm option. It might be an >>>> issue in the testcase (or how the testcase was run). I will look into >>>> it. >>>> >>>> I missed this issue because I had changed my scripts to use -othervm >>>> sometime back for debugging/analyzing some testcase >>>> failure but forgot to change it back. Thank you for pointing it out. >>> For Volker's benefit, othervm is where jtreg spins up a new VM for each >>> test, agentvm is where jtreg re-uses the VM if possible. There are tests >>> that don't or can't clean up and those tests need to ensure that their @run >>> tests have the /othervm option so that the tests runs in its own VM. >>> Alternatively, the TEST.ROOT file has a key that lists directories where all >>> the tests in those directory trees must run in their own VM. >>> >>> Balchandra - one thing that be useful is to look at the client tests to >>> see what can run in agentvm and what can't. For the jdk_core and jdk_svc >>> tests then they can be run in either (although we're still picking off >>> issues as they found in agentvm mode). I'm not aware of any work done yet on >>> getting the tests in the jdk_desktop test group to run in agentvm mode. It >>> may be that TEST.ROOT needs to be updated to see a few additional top-level >>> directories so that all the tests in these areas temporarily run in othervm >>> mode. >> >> In the subset of the tests I have been running, it appears only one test [1] >> is failing with agentvm option. I will try to run other tests in client >> groups [Note: I will update the instruction to use jtreg groups with b119 >> results posting] and see what tests can be run in agentvm mode, but main >> issue >> in those remaining client tests are some tests may not consistently produce >> same results - could stabilizing those tests be a goal for jdk9 repo ? >> >> >> Thanks >> Balchandra >> >> [1] com/sun/java/swing/plaf/windows/8016551/bug8016551.java >> >>> -Alan >> From jonathan.gibbons at oracle.com Wed Dec 11 09:55:56 2013 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Wed, 11 Dec 2013 09:55:56 -0800 Subject: JDK 8 b118 ea test results are now available In-Reply-To: <52A899CE.6090408@oracle.com> References: <529DC2AA.8090506@oracle.com> <52A6139A.1020203@oracle.com> <52A6F908.3050908@oracle.com> <52A7327C.7050202@oracle.com> <52A743D2.2010104@oracle.com> <52A899CE.6090408@oracle.com> Message-ID: <52A8A72C.7050300@oracle.com> Balchandra, The important part of that of interest here is the part about "calls jtreg commands (one each for jdk, langtools and hotspot) with those recommended options". Is there a way you could post here the segment of the script that does that? Alternatively, that segment of the script could be a candidate for a target in one or more test/Makefile files. -- Jon On 12/11/2013 08:58 AM, Balchandra Vaidya wrote: > > Hi Volker, > > The custom script I use just wrap the instruction posted (except > building jtreg). It just clones the openjdk repository, calls jtreg > commands > (one each for jdk, langtools and hotspot) with those recommended > options, runs diffs, runs nashorn tests, and archives results. > > Such convenient script may not fit under openjdk repo, but think of > adding it in code-tools project in the future. > > > Thanks > Balchandra > > On 11/12/2013 13:45, Volker Simonis wrote: >> Hi Balchandra, >> >> what do you think about checking in a little script or Makefile target >> which produces the test results you report for the ea builds? >> >> This would be of great help to produce comparable test results. >> >> Regards, >> Volker >> >> On Tue, Dec 10, 2013 at 5:39 PM, Balchandra Vaidya >> wrote: >>> On 12/10/13 03:25 PM, Alan Bateman wrote: >>>> On 10/12/2013 11:20, Balchandra Vaidya wrote: >>>>> : >>>>> >>>>> 2) Testcase do not run with -agentvm option. It might be an >>>>> issue in the testcase (or how the testcase was run). I will >>>>> look into >>>>> it. >>>>> >>>>> I missed this issue because I had changed my scripts to use >>>>> -othervm >>>>> sometime back for debugging/analyzing some testcase >>>>> failure but forgot to change it back. Thank you for pointing >>>>> it out. >>>> For Volker's benefit, othervm is where jtreg spins up a new VM for >>>> each >>>> test, agentvm is where jtreg re-uses the VM if possible. There are >>>> tests >>>> that don't or can't clean up and those tests need to ensure that >>>> their @run >>>> tests have the /othervm option so that the tests runs in its own VM. >>>> Alternatively, the TEST.ROOT file has a key that lists directories >>>> where all >>>> the tests in those directory trees must run in their own VM. >>>> >>>> Balchandra - one thing that be useful is to look at the client >>>> tests to >>>> see what can run in agentvm and what can't. For the jdk_core and >>>> jdk_svc >>>> tests then they can be run in either (although we're still picking off >>>> issues as they found in agentvm mode). I'm not aware of any work >>>> done yet on >>>> getting the tests in the jdk_desktop test group to run in agentvm >>>> mode. It >>>> may be that TEST.ROOT needs to be updated to see a few additional >>>> top-level >>>> directories so that all the tests in these areas temporarily run in >>>> othervm >>>> mode. >>> >>> In the subset of the tests I have been running, it appears only one >>> test [1] >>> is failing with agentvm option. I will try to run other tests in client >>> groups [Note: I will update the instruction to use jtreg groups with >>> b119 >>> results posting] and see what tests can be run in agentvm mode, but >>> main >>> issue >>> in those remaining client tests are some tests may not consistently >>> produce >>> same results - could stabilizing those tests be a goal for jdk9 repo ? >>> >>> >>> Thanks >>> Balchandra >>> >>> [1] com/sun/java/swing/plaf/windows/8016551/bug8016551.java >>> >>>> -Alan >>> > From dalibor.topic at oracle.com Wed Dec 11 10:46:36 2013 From: dalibor.topic at oracle.com (Dalibor Topic) Date: Wed, 11 Dec 2013 19:46:36 +0100 Subject: JDK 8 b118 ea test results are now available In-Reply-To: <52A8A72C.7050300@oracle.com> References: <529DC2AA.8090506@oracle.com> <52A6139A.1020203@oracle.com> <52A6F908.3050908@oracle.com> <52A7327C.7050202@oracle.com> <52A743D2.2010104@oracle.com> <52A899CE.6090408@oracle.com> <52A8A72C.7050300@oracle.com> Message-ID: <52A8B30C.5080506@oracle.com> On 12/11/13 6:55 PM, Jonathan Gibbons wrote: > Alternatively, that segment of the script could be a candidate for a target in > one or more test/Makefile files. That sounds like a very good idea to me. cheers, dalibor topic -- Oracle Dalibor Topic | Principal Product Manager Phone: +494089091214 | Mobile: +491737185961 Oracle Java Platform Group ORACLE Deutschland B.V. & Co. KG | K?hneh?fe 5 | 22761 Hamburg ORACLE Deutschland B.V. & Co. KG Hauptverwaltung: Riesstr. 25, D-80992 M?nchen Registergericht: Amtsgericht M?nchen, HRA 95603 Gesch?ftsf?hrer: J?rgen Kunz Komplement?rin: ORACLE Deutschland Verwaltung B.V. Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 Gesch?ftsf?hrer: Alexander van der Ven, Astrid Kepper, Val Maher Green Oracle Oracle is committed to developing practices and products that help protect the environment From balchandra.vaidya at oracle.com Wed Dec 11 11:26:46 2013 From: balchandra.vaidya at oracle.com (Balchandra Vaidya) Date: Wed, 11 Dec 2013 19:26:46 +0000 Subject: JDK 8 b118 ea test results are now available In-Reply-To: <52A8A72C.7050300@oracle.com> References: <529DC2AA.8090506@oracle.com> <52A6139A.1020203@oracle.com> <52A6F908.3050908@oracle.com> <52A7327C.7050202@oracle.com> <52A743D2.2010104@oracle.com> <52A899CE.6090408@oracle.com> <52A8A72C.7050300@oracle.com> Message-ID: <52A8BC76.60409@oracle.com> On 11/12/2013 17:55, Jonathan Gibbons wrote: > Balchandra, > > The important part of that of interest here is the part about "calls > jtreg commands > (one each for jdk, langtools and hotspot) with those recommended > options". > > Is there a way you could post here the segment of the script that does > that? > Here it is 2.1 Running tests in jdk/test $ jtreg -dir:{openjdk source top directory}/jdk/test -verbose:summary -exclude:{openjdk source top directory}/jdk/test/ProblemList.txt -conc:auto -a -ignore:quiet -timeoutFactor:5 -othervm -testjdk:{location of the test jdk} `cat dir.list ` 2.2 Running tests in langtools/test $ jtreg -dir:{openjdk source top directory}/langtools/test -verbose:summary -conc:auto -a -ignore:quiet -timeoutFactor:5 -agentvm -testjdk:{location of the test jdk} com tools 2.3 Running tests in hotspot/test $ jtreg -dir:{openjdk source top directory}/hotspot/test -verbose:summary -conc:auto -a -ignore:quiet -timeoutFactor:5 -agentvm -testjdk:{location of the test jdk} compiler gc runtime sanity serviceability The instructions is at http://download.java.net/jdk8/testresults/docs/howtoruntests.html and is linked from http://www.java.net/download/jdk8/testresults/testresults.html The caveat in this approach is a human error where one (me) forget to update the instruction or forget to remove any additional flags used in the script temporarily - a mismatch of the results could occur. From the next build (b120), I am going to update "2.1 Running tests in jdk/test" to $ jtreg -dir:{openjdk source top directory}/jdk/test -verbose:summary -exclude:{openjdk source top directory}/jdk/test/ProblemList.txt -conc:auto -a -ignore:quiet -timeoutFactor:5 -agentvm -testjdk:{location of the test jdk} :jdk_core :jdk_svc :jdk_beans :jdk_imageio :jdk_sound :jdk_sctp javax/accessibility com/sun/java/swing javax/print sun/pisces com/sun/awt > Alternatively, that segment of the script could be a candidate for a > target in > one or more test/Makefile files. This is good idea, but my experience with the 'make' is that if one target critically fail, all subsequent targets will not run. I thought it is a restriction of 'make'. Thanks Balchandra > > -- Jon > > > On 12/11/2013 08:58 AM, Balchandra Vaidya wrote: >> >> Hi Volker, >> >> The custom script I use just wrap the instruction posted (except >> building jtreg). It just clones the openjdk repository, calls jtreg >> commands >> (one each for jdk, langtools and hotspot) with those recommended >> options, runs diffs, runs nashorn tests, and archives results. >> >> Such convenient script may not fit under openjdk repo, but think of >> adding it in code-tools project in the future. >> >> >> Thanks >> Balchandra >> >> On 11/12/2013 13:45, Volker Simonis wrote: >>> Hi Balchandra, >>> >>> what do you think about checking in a little script or Makefile target >>> which produces the test results you report for the ea builds? >>> >>> This would be of great help to produce comparable test results. >>> >>> Regards, >>> Volker >>> >>> On Tue, Dec 10, 2013 at 5:39 PM, Balchandra Vaidya >>> wrote: >>>> On 12/10/13 03:25 PM, Alan Bateman wrote: >>>>> On 10/12/2013 11:20, Balchandra Vaidya wrote: >>>>>> : >>>>>> >>>>>> 2) Testcase do not run with -agentvm option. It might be an >>>>>> issue in the testcase (or how the testcase was run). I will >>>>>> look into >>>>>> it. >>>>>> >>>>>> I missed this issue because I had changed my scripts to use >>>>>> -othervm >>>>>> sometime back for debugging/analyzing some testcase >>>>>> failure but forgot to change it back. Thank you for pointing >>>>>> it out. >>>>> For Volker's benefit, othervm is where jtreg spins up a new VM for >>>>> each >>>>> test, agentvm is where jtreg re-uses the VM if possible. There are >>>>> tests >>>>> that don't or can't clean up and those tests need to ensure that >>>>> their @run >>>>> tests have the /othervm option so that the tests runs in its own VM. >>>>> Alternatively, the TEST.ROOT file has a key that lists directories >>>>> where all >>>>> the tests in those directory trees must run in their own VM. >>>>> >>>>> Balchandra - one thing that be useful is to look at the client >>>>> tests to >>>>> see what can run in agentvm and what can't. For the jdk_core and >>>>> jdk_svc >>>>> tests then they can be run in either (although we're still picking >>>>> off >>>>> issues as they found in agentvm mode). I'm not aware of any work >>>>> done yet on >>>>> getting the tests in the jdk_desktop test group to run in agentvm >>>>> mode. It >>>>> may be that TEST.ROOT needs to be updated to see a few additional >>>>> top-level >>>>> directories so that all the tests in these areas temporarily run >>>>> in othervm >>>>> mode. >>>> >>>> In the subset of the tests I have been running, it appears only one >>>> test [1] >>>> is failing with agentvm option. I will try to run other tests in >>>> client >>>> groups [Note: I will update the instruction to use jtreg groups >>>> with b119 >>>> results posting] and see what tests can be run in agentvm mode, but >>>> main >>>> issue >>>> in those remaining client tests are some tests may not consistently >>>> produce >>>> same results - could stabilizing those tests be a goal for jdk9 repo ? >>>> >>>> >>>> Thanks >>>> Balchandra >>>> >>>> [1] com/sun/java/swing/plaf/windows/8016551/bug8016551.java >>>> >>>>> -Alan >>>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/quality-discuss/attachments/20131211/c040d3ce/attachment-0001.html From jonathan.gibbons at oracle.com Wed Dec 11 11:50:32 2013 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Wed, 11 Dec 2013 11:50:32 -0800 Subject: JDK 8 b118 ea test results are now available In-Reply-To: <52A8BC76.60409@oracle.com> References: <529DC2AA.8090506@oracle.com> <52A6139A.1020203@oracle.com> <52A6F908.3050908@oracle.com> <52A7327C.7050202@oracle.com> <52A743D2.2010104@oracle.com> <52A899CE.6090408@oracle.com> <52A8A72C.7050300@oracle.com> <52A8BC76.60409@oracle.com> Message-ID: <52A8C208.3080806@oracle.com> Balchandra, Thanks for posting the commands. That gives us something to reason about. For my part, I have found -conc:auto to be too generous. "auto" gets evaluated to the number of processors on the host system as seen by the Java runtime. In my usage (on langtools) I have found a good rule of thumb to be half that number, so on a 32-way system, I use -conc:16, etc. It is also a function of available memory. I played with the idea of supporting expressions using auto, such as -conc:auto/2 or -conc:auto-1 -- but then I came to my senses. This calculation is much better done in the environment used to run jtreg. I notice in your commands, you do NOT limit the amount of memory for each JVM with -Xmx. That leaves all your JVMs using default ergonomics, which may be unduly optimistic when you have lots of JVMs running. As a general rule for anyone using -conc to run tests concurrently, I strongly recommend that the first few times you do this, use your system activity profiling tool to monitor CPU and memory usage. If all your CPUs are redlining it at 100%, or if you start memory swapping, then your concurrency number is too high. I would recommend that you target an average CPU utilization just below 100%, and no memory swapping. A couple more comments inline. -- Jon P.S. I'll see about converting some of these notes and guidelines into another jtreg page on OpenJDK. -- Jon On 12/11/2013 11:26 AM, Balchandra Vaidya wrote: > > On 11/12/2013 17:55, Jonathan Gibbons wrote: >> Balchandra, >> >> The important part of that of interest here is the part about "calls >> jtreg commands >> (one each for jdk, langtools and hotspot) with those recommended >> options". >> >> Is there a way you could post here the segment of the script that >> does that? >> > Here it is > > 2.1 Running tests in jdk/test > $ jtreg -dir:{openjdk source top directory}/jdk/test -verbose:summary > -exclude:{openjdk source top directory}/jdk/test/ProblemList.txt > -conc:auto -a -ignore:quiet -timeoutFactor:5 -othervm > -testjdk:{location of the test jdk} `cat dir.list > ` > > 2.2 Running tests in langtools/test > $ jtreg -dir:{openjdk source top directory}/langtools/test > -verbose:summary -conc:auto -a -ignore:quiet -timeoutFactor:5 -agentvm > -testjdk:{location of the test jdk} com tools > > 2.3 Running tests in hotspot/test > $ jtreg -dir:{openjdk source top directory}/hotspot/test > -verbose:summary -conc:auto -a -ignore:quiet -timeoutFactor:5 -agentvm > -testjdk:{location of the test jdk} compiler gc runtime sanity > serviceability > > > The instructions is at > http://download.java.net/jdk8/testresults/docs/howtoruntests.html > and is linked from > http://www.java.net/download/jdk8/testresults/testresults.html > > The caveat in this approach is a human error where one (me) forget to > update the instruction > or forget to remove any additional flags used in the script > temporarily - a mismatch > of the results could occur. > > From the next build (b120), I am going to update "2.1 Running tests in > jdk/test" to > > $ jtreg -dir:{openjdk source top directory}/jdk/test -verbose:summary > -exclude:{openjdk source top directory}/jdk/test/ProblemList.txt > -conc:auto -a -ignore:quiet -timeoutFactor:5 -agentvm > -testjdk:{location of the test jdk} :jdk_core :jdk_svc :jdk_beans > :jdk_imageio :jdk_sound :jdk_sctp javax/accessibility > com/sun/java/swing javax/print sun/pisces com/sun/awt Generally, I would say that if you need to put a list of groups on the command line, you need to update the groups file to create a new group that is composed of the individual groups you want to run. I think it would be good to have a group such as :jdk_default or something like that. > > >> Alternatively, that segment of the script could be a candidate for a >> target in >> one or more test/Makefile files. > > This is good idea, but my experience with the 'make' is that if one > target critically fail, all > subsequent targets will not run. I thought it is a restriction of 'make'. Well, ... a) "make -k" keeps on going b) as a user of the Makefile, you should be looking for a single target, such as "make test", and it is up to the implementation of the target to run all the tests, even though some might fail. There are various ways to do this: for example, you can prefix a make rule with "-" to ignore the exit code from a command, or you can do a composite command like jtreg || echo some jtreg tests failed > > > Thanks > Balchandra > >> >> -- Jon -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/quality-discuss/attachments/20131211/eb7d95a1/attachment.html From Alan.Bateman at oracle.com Wed Dec 11 11:54:13 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 11 Dec 2013 19:54:13 +0000 Subject: JDK 8 b118 ea test results are now available In-Reply-To: <52A8BC76.60409@oracle.com> References: <529DC2AA.8090506@oracle.com> <52A6139A.1020203@oracle.com> <52A6F908.3050908@oracle.com> <52A7327C.7050202@oracle.com> <52A743D2.2010104@oracle.com> <52A899CE.6090408@oracle.com> <52A8A72C.7050300@oracle.com> <52A8BC76.60409@oracle.com> Message-ID: <52A8C2E5.7010505@oracle.com> On 11/12/2013 19:26, Balchandra Vaidya wrote: > > : > >> Alternatively, that segment of the script could be a candidate for a >> target in >> one or more test/Makefile files. > > This is good idea, but my experience with the 'make' is that if one > target critically fail, all > subsequent targets will not run. I thought it is a restriction of 'make'. > I think Jon is suggesting that the subset that you run be added to TEST.groups (which will automatically turn it a make targe as way of the jdk_% rule ). On the surface this is a good idea but when I look at the subset of the tests that you are running: :jdk_core :jdk_svc :jdk_beans :jdk_imageio :jdk_sound :jdk_sctp javax/accessibility com/sun/java/swing javax/print sun/pisces com/sun/awt then it's a bit ad hoc. I wouldn't object to adding a special group for this but it really amounts to all jdk tests except for: java/awt javax/swing sun/awt sun/java2d com/apple/eawt If these tests aren't in your runs because of stability issues then we should make sure that there are bugs submitted and that they get some focus. In the interim then unstable tests can be @ignore-d or added to the exclude list. I initially thought that part of the issue was the othervm vs. agentvm discussion but I see in TEST.ROOT that othervm.dirs lists these directories already. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/quality-discuss/attachments/20131211/d6fedf73/attachment.html From Alan.Bateman at oracle.com Wed Dec 11 12:25:50 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 11 Dec 2013 20:25:50 +0000 Subject: JDK 8 b118 ea test results are now available In-Reply-To: <52A8C208.3080806@oracle.com> References: <529DC2AA.8090506@oracle.com> <52A6139A.1020203@oracle.com> <52A6F908.3050908@oracle.com> <52A7327C.7050202@oracle.com> <52A743D2.2010104@oracle.com> <52A899CE.6090408@oracle.com> <52A8A72C.7050300@oracle.com> <52A8BC76.60409@oracle.com> <52A8C208.3080806@oracle.com> Message-ID: <52A8CA4E.6020002@oracle.com> On 11/12/2013 19:50, Jonathan Gibbons wrote: > : > > I notice in your commands, you do NOT limit the amount of memory for > each JVM with -Xmx. That leaves all your JVMs using default > ergonomics, which may be unduly optimistic when you have lots of JVMs > running. Yes, this is very important then running the jdk tests with -concurrency. The :jdk_core and :jdk_svc tests are okay with -vmoption:-Xmx256m, I have not tried anything smaller myself. One thing that some of us don't have a handle on is the maximum number of VMs that are in use during a test run. It's something that Mike and I were chatting about recently [1]. In any case, it's the maximum number of VMs (and thus the memory) that really limits the concurrency. Many of the tests are not compute bound so you can push the concurrency (but only if you have the memory to do it). One other thing to say about concurrency that auto just doesn't work on SPARC, just too many hardware threads. -Alan. [1] http://mail.openjdk.java.net/pipermail/jtreg-use/2013-November/000305.html -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/quality-discuss/attachments/20131211/1959de53/attachment.html From jonathan.gibbons at oracle.com Wed Dec 11 15:10:34 2013 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Wed, 11 Dec 2013 15:10:34 -0800 Subject: JDK 8 b118 ea test results are now available In-Reply-To: <52A8C2E5.7010505@oracle.com> References: <529DC2AA.8090506@oracle.com> <52A6139A.1020203@oracle.com> <52A6F908.3050908@oracle.com> <52A7327C.7050202@oracle.com> <52A743D2.2010104@oracle.com> <52A899CE.6090408@oracle.com> <52A8A72C.7050300@oracle.com> <52A8BC76.60409@oracle.com> <52A8C2E5.7010505@oracle.com> Message-ID: <52A8F0EA.2050008@oracle.com> On 12/11/2013 11:54 AM, Alan Bateman wrote: > On 11/12/2013 19:26, Balchandra Vaidya wrote: >> >> : >> >>> Alternatively, that segment of the script could be a candidate for a >>> target in >>> one or more test/Makefile files. >> >> This is good idea, but my experience with the 'make' is that if one >> target critically fail, all >> subsequent targets will not run. I thought it is a restriction of 'make'. >> > I think Jon is suggesting that the subset that you run be added to > TEST.groups (which will automatically turn it a make targe as way of > the jdk_% rule ). On the surface this is a good idea but when I look > at the subset of the tests that you are running: > > :jdk_core > :jdk_svc > :jdk_beans > :jdk_imageio > :jdk_sound > :jdk_sctp > javax/accessibility > com/sun/java/swing > javax/print > sun/pisces > com/sun/awt > > then it's a bit ad hoc. I wouldn't object to adding a special group > for this but it really amounts to all jdk tests except for: > > java/awt > javax/swing > sun/awt > sun/java2d > com/apple/eawt > > If these tests aren't in your runs because of stability issues then we > should make sure that there are bugs submitted and that they get some > focus. In the interim then unstable tests can be @ignore-d or added to > the exclude list. I initially thought that part of the issue was the > othervm vs. agentvm discussion but I see in TEST.ROOT that > othervm.dirs lists these directories already. > > -Alan > > Yes, the high order point I was trying to make is that something is wrong if you need to specify a long list of tests to run. While we all may take whatever short cuts we choose to get our day to day work done, there should be a standard set of tests[1] that we agree should be run, and which can be run with reasonably concise command line args. -- Jon Ideally, "all" but maybe we're not there yet. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/quality-discuss/attachments/20131211/76b7ec0b/attachment.html From volker.simonis at gmail.com Thu Dec 12 01:51:04 2013 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 12 Dec 2013 10:51:04 +0100 Subject: JDK 8 b118 ea test results are now available In-Reply-To: <52A8F0EA.2050008@oracle.com> References: <529DC2AA.8090506@oracle.com> <52A6139A.1020203@oracle.com> <52A6F908.3050908@oracle.com> <52A7327C.7050202@oracle.com> <52A743D2.2010104@oracle.com> <52A899CE.6090408@oracle.com> <52A8A72C.7050300@oracle.com> <52A8BC76.60409@oracle.com> <52A8C2E5.7010505@oracle.com> <52A8F0EA.2050008@oracle.com> Message-ID: I fully agree! It would be nice to have a single test group and a single make target which could be run by everybody who succeeded to build the OpenJDK. And it should produce the same results like they are posted on http://www.java.net/download/jdk8/testresults/testresults.html . Actually, the tests results posted there should be run with the same make target and test group. It's nice to have a test description an a website but reality shows that it is inevitable that the web page and the real scripts will get out of sync. This can not happen if you have a checked in make-file and group description. Of course it is also time to add test results for the other Oracle supported platforms (Mac, Win and Solaris) to http://www.java.net/download/jdk8/testresults/testresults.html. In my eyes, this is an absolute requirement if we want to get a stable set of tests which can be used to verify something like a port (or some other bigger changes). Concentrating on Linux will not help a lot here. The test group would be a running target of course and everybody who fixes and/or stabilizes a test should update it. Regards, Volker On Thu, Dec 12, 2013 at 12:10 AM, Jonathan Gibbons wrote: > > On 12/11/2013 11:54 AM, Alan Bateman wrote: > > On 11/12/2013 19:26, Balchandra Vaidya wrote: > > > : > > Alternatively, that segment of the script could be a candidate for a target > in > one or more test/Makefile files. > > > This is good idea, but my experience with the 'make' is that if one target > critically fail, all > subsequent targets will not run. I thought it is a restriction of 'make'. > > I think Jon is suggesting that the subset that you run be added to > TEST.groups (which will automatically turn it a make targe as way of the > jdk_% rule ). On the surface this is a good idea but when I look at the > subset of the tests that you are running: > > :jdk_core > :jdk_svc > :jdk_beans > :jdk_imageio > :jdk_sound > :jdk_sctp > javax/accessibility > com/sun/java/swing > javax/print > sun/pisces > com/sun/awt > > then it's a bit ad hoc. I wouldn't object to adding a special group for this > but it really amounts to all jdk tests except for: > > java/awt > javax/swing > sun/awt > sun/java2d > com/apple/eawt > > If these tests aren't in your runs because of stability issues then we > should make sure that there are bugs submitted and that they get some focus. > In the interim then unstable tests can be @ignore-d or added to the exclude > list. I initially thought that part of the issue was the othervm vs. agentvm > discussion but I see in TEST.ROOT that othervm.dirs lists these directories > already. > > -Alan > > > > Yes, the high order point I was trying to make is that something is wrong if > you need to specify a long list of tests to run. While we all may take > whatever short cuts we choose to get our day to day work done, there should > be a standard set of tests[1] that we agree should be run, and which can be > run with reasonably concise command line args. > > -- Jon > > Ideally, "all" but maybe we're not there yet. > > From Alan.Bateman at oracle.com Thu Dec 12 02:49:31 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 12 Dec 2013 10:49:31 +0000 Subject: JDK 8 b118 ea test results are now available In-Reply-To: <52A8F0EA.2050008@oracle.com> References: <529DC2AA.8090506@oracle.com> <52A6139A.1020203@oracle.com> <52A6F908.3050908@oracle.com> <52A7327C.7050202@oracle.com> <52A743D2.2010104@oracle.com> <52A899CE.6090408@oracle.com> <52A8A72C.7050300@oracle.com> <52A8BC76.60409@oracle.com> <52A8C2E5.7010505@oracle.com> <52A8F0EA.2050008@oracle.com> Message-ID: <52A994BB.7020206@oracle.com> On 11/12/2013 23:10, Jonathan Gibbons wrote: > : > > Yes, the high order point I was trying to make is that something is > wrong if you need to specify a long list of tests to run. While we > all may take whatever short cuts we choose to get our day to day work > done, there should be a standard set of tests[1] that we agree should > be run, and which can be run with reasonably concise command line args. > > -- Jon > > Ideally, "all" but maybe we're not there yet. > Suppose we create a jdk_stable (or other name) group in TEST.groups for what "we" consider are the stable tests. Once it is defined in the groups file then it means it can be used by anyone that runs jtreg directly or anyone that uses the make file to run tests ("make test TEST=jdk_stable" for example). Whether it deserves its own make file is another question. So suppose we create such a group then what would be the criteria to be in that group? Clearly the test should be stable in the sense that it should pass when we don't have a bug. It should also clean up after itself. Things that come to mind are: - should be usable with -agentvm? We have /othervm option for @run and we also have othervm.dirs, the main point is that they can be run with either in othervm or agentvm modes and they should just work. I have deliberately not mentioned -samevm here as it's not suitable for the jdk tests. - needs to work with -concurrency, assuming sufficient resources. Is that reasonable to require? We have /exclusive and exclusiveAccess.dirs available for areas that have issues. - needs to run in a reasonable time. I don't think we have guidelines for what is reasonable in the jdk tests but clearly a test that runs for more than a few minutes needs to be looked at. - needs to run headless? Maybe this is controversial but it is somewhat appealing to skip Xvfb or other setup. Also being selfish, I'd like to run tests in a terminal window and not have windows dancing on my desktop. - should not require special configuration? Maybe this is controversial too but there are tests javax/print that fail when there isn't a printer configured. Anything else? If we do something like this then such a group would need to be maintained, at least for period until all tests are stable and fast. -Alan. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/quality-discuss/attachments/20131212/62dccb94/attachment.html From balchandra.vaidya at oracle.com Thu Dec 12 04:11:00 2013 From: balchandra.vaidya at oracle.com (Balchandra Vaidya) Date: Thu, 12 Dec 2013 12:11:00 +0000 Subject: JDK 8 b118 ea test results are now available In-Reply-To: <52A8C208.3080806@oracle.com> References: <529DC2AA.8090506@oracle.com> <52A6139A.1020203@oracle.com> <52A6F908.3050908@oracle.com> <52A7327C.7050202@oracle.com> <52A743D2.2010104@oracle.com> <52A899CE.6090408@oracle.com> <52A8A72C.7050300@oracle.com> <52A8BC76.60409@oracle.com> <52A8C208.3080806@oracle.com> Message-ID: <52A9A7D4.2050905@oracle.com> On 12/11/13 07:50 PM, Jonathan Gibbons wrote: > Balchandra, > > Thanks for posting the commands. That gives us something to reason about. > > For my part, I have found -conc:auto to be too generous. "auto" gets > evaluated to the number of processors on the host system as seen by > the Java runtime. In my usage (on langtools) I have found a good rule > of thumb to be half that number, so on a 32-way system, I use > -conc:16, etc. It is also a function of available memory. > > I played with the idea of supporting expressions using auto, such as > -conc:auto/2 or -conc:auto-1 -- but then I came to my senses. This > calculation is much better done in the environment used to run jtreg. > > I notice in your commands, you do NOT limit the amount of memory for > each JVM with -Xmx. That leaves all your JVMs using default > ergonomics, which may be unduly optimistic when you have lots of JVMs > running. > > As a general rule for anyone using -conc to run tests concurrently, I > strongly recommend that the first few times you do this, use your > system activity profiling tool to monitor CPU and memory usage. If > all your CPUs are redlining it at 100%, or if you start memory > swapping, then your concurrency number is too high. I would recommend > that you target an average CPU utilization just below 100%, and no > memory swapping. > It is a good information. First, I usually run tests on 4 way and 8 way systems. I believe the objective here is to provide repeatable test results. But, the hardware availability, OS versions, kernel, patches, window manager, jtreg options, and many other factors play a role to produce consistent test results. The truth is we may not able to produce results using all combinations, but the results posted can reasonably be used for a comparison. That is, the result posted is not everything/unbreakable. Is it fair to say something like "The test results posted here used -conc:2 -Xmx:512m ..... options. Result could vary if other options were used. However, the community members are encouraged to send email to quality-discuss or file bugs if any issues found". If a testcase pass with -conc:2 but fail with -conc:4 means there might be a bug in the testcase - such failures are good to know. Second point is to make test execution easier. Does it make sense to add default options (concurrency, memory etc.) to jtreg? Or does it make sense to create a separate environment files (similar to what is used in JCK) and read those files in jtreg or other wrapper script? However, I also think that are such micro tuning necessary? Testing different combinations may find bugs which is beneficial to OpenJDK. > A couple more comments inline. > > -- Jon > > P.S. I'll see about converting some of these notes and guidelines > into another jtreg page on OpenJDK. > > -- Jon > > On 12/11/2013 11:26 AM, Balchandra Vaidya wrote: >> >> On 11/12/2013 17:55, Jonathan Gibbons wrote: >>> Balchandra, >>> >>> The important part of that of interest here is the part about "calls >>> jtreg commands >>> (one each for jdk, langtools and hotspot) with those recommended >>> options". >>> >>> Is there a way you could post here the segment of the script that >>> does that? >>> >> Here it is >> >> 2.1 Running tests in jdk/test >> $ jtreg -dir:{openjdk source top directory}/jdk/test -verbose:summary >> -exclude:{openjdk source top directory}/jdk/test/ProblemList.txt >> -conc:auto -a -ignore:quiet -timeoutFactor:5 -othervm >> -testjdk:{location of the test jdk} `cat dir.list >> ` >> >> 2.2 Running tests in langtools/test >> $ jtreg -dir:{openjdk source top directory}/langtools/test >> -verbose:summary -conc:auto -a -ignore:quiet -timeoutFactor:5 >> -agentvm -testjdk:{location of the test jdk} com tools >> >> 2.3 Running tests in hotspot/test >> $ jtreg -dir:{openjdk source top directory}/hotspot/test >> -verbose:summary -conc:auto -a -ignore:quiet -timeoutFactor:5 >> -agentvm -testjdk:{location of the test jdk} compiler gc runtime >> sanity serviceability >> >> >> The instructions is at >> http://download.java.net/jdk8/testresults/docs/howtoruntests.html >> and is linked from >> http://www.java.net/download/jdk8/testresults/testresults.html >> >> The caveat in this approach is a human error where one (me) forget to >> update the instruction >> or forget to remove any additional flags used in the script >> temporarily - a mismatch >> of the results could occur. >> >> From the next build (b120), I am going to update "2.1 Running tests >> in jdk/test" to >> >> $ jtreg -dir:{openjdk source top directory}/jdk/test -verbose:summary >> -exclude:{openjdk source top directory}/jdk/test/ProblemList.txt >> -conc:auto -a -ignore:quiet -timeoutFactor:5 -agentvm >> -testjdk:{location of the test jdk} :jdk_core :jdk_svc :jdk_beans >> :jdk_imageio :jdk_sound :jdk_sctp javax/accessibility >> com/sun/java/swing javax/print sun/pisces com/sun/awt > > Generally, I would say that if you need to put a list of groups on the > command line, you need to update the groups file to create a new group > that is composed of the individual groups you want to run. > > I think it would be good to have a group such as :jdk_default or > something like that. > > Yes, it is a good idea. > >> >> >>> Alternatively, that segment of the script could be a candidate for a >>> target in >>> one or more test/Makefile files. >> >> This is good idea, but my experience with the 'make' is that if one >> target critically fail, all >> subsequent targets will not run. I thought it is a restriction of 'make'. > > Well, ... > a) "make -k" keeps on going > b) as a user of the Makefile, you should be looking for a single > target, such as "make test", and it is up to the implementation of the > target to run all the tests, even though some might fail. There are > various ways to do this: for example, you can prefix a make rule with > "-" to ignore the exit code from a command, or you can do a composite > command like > jtreg || echo some jtreg tests failed I will try again. I remember "make -k" didn't work from start to end. That was an year ago, so I will retry. Is it time to say goodbye to 'make' and say hello to 'ant/gradle' for jdk9 ? Thanks Balchandra > >> >> >> Thanks >> Balchandra >> >>> >>> -- Jon > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/quality-discuss/attachments/20131212/9ace11a2/attachment-0001.html From balchandra.vaidya at oracle.com Thu Dec 12 04:19:44 2013 From: balchandra.vaidya at oracle.com (Balchandra Vaidya) Date: Thu, 12 Dec 2013 12:19:44 +0000 Subject: JDK 8 b118 ea test results are now available In-Reply-To: <52A8C2E5.7010505@oracle.com> References: <529DC2AA.8090506@oracle.com> <52A6139A.1020203@oracle.com> <52A6F908.3050908@oracle.com> <52A7327C.7050202@oracle.com> <52A743D2.2010104@oracle.com> <52A899CE.6090408@oracle.com> <52A8A72C.7050300@oracle.com> <52A8BC76.60409@oracle.com> <52A8C2E5.7010505@oracle.com> Message-ID: <52A9A9E0.9030209@oracle.com> On 12/11/13 07:54 PM, Alan Bateman wrote: > On 11/12/2013 19:26, Balchandra Vaidya wrote: >> >> : >> >>> Alternatively, that segment of the script could be a candidate for a >>> target in >>> one or more test/Makefile files. >> >> This is good idea, but my experience with the 'make' is that if one >> target critically fail, all >> subsequent targets will not run. I thought it is a restriction of 'make'. >> > I think Jon is suggesting that the subset that you run be added to > TEST.groups (which will automatically turn it a make targe as way of > the jdk_% rule ). On the surface this is a good idea but when I look > at the subset of the tests that you are running: > > :jdk_core > :jdk_svc > :jdk_beans > :jdk_imageio > :jdk_sound > :jdk_sctp > javax/accessibility > com/sun/java/swing > javax/print > sun/pisces > com/sun/awt > > then it's a bit ad hoc. I wouldn't object to adding a special group > for this but it really amounts to all jdk tests except for: > > java/awt > javax/swing > sun/awt > sun/java2d > com/apple/eawt > I think it make sense to have an actively managed :jdk_stable group. I know that all tests in e.g. javax/swing are not unstable, but it is a matter of time (and testing on various OS) to figure out the unstable tests that are causing the pain. Thanks Balchandra > If these tests aren't in your runs because of stability issues then we > should make sure that there are bugs submitted and that they get some > focus. In the interim then unstable tests can be @ignore-d or added to > the exclude list. I initially thought that part of the issue was the > othervm vs. agentvm discussion but I see in TEST.ROOT that > othervm.dirs lists these directories already. > > -Alan > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/quality-discuss/attachments/20131212/ca84510e/attachment.html From volker.simonis at gmail.com Thu Dec 12 06:12:39 2013 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 12 Dec 2013 15:12:39 +0100 Subject: JDK 8 b118 ea test results are now available In-Reply-To: <52A994BB.7020206@oracle.com> References: <529DC2AA.8090506@oracle.com> <52A6139A.1020203@oracle.com> <52A6F908.3050908@oracle.com> <52A7327C.7050202@oracle.com> <52A743D2.2010104@oracle.com> <52A899CE.6090408@oracle.com> <52A8A72C.7050300@oracle.com> <52A8BC76.60409@oracle.com> <52A8C2E5.7010505@oracle.com> <52A8F0EA.2050008@oracle.com> <52A994BB.7020206@oracle.com> Message-ID: On Thu, Dec 12, 2013 at 11:49 AM, Alan Bateman wrote: > On 11/12/2013 23:10, Jonathan Gibbons wrote: > > : > > Yes, the high order point I was trying to make is that something is wrong if > you need to specify a long list of tests to run. While we all may take > whatever short cuts we choose to get our day to day work done, there should > be a standard set of tests[1] that we agree should be run, and which can be > run with reasonably concise command line args. > > -- Jon > > Ideally, "all" but maybe we're not there yet. > > Suppose we create a jdk_stable (or other name) group in TEST.groups for what > "we" consider are the stable tests. Once it is defined in the groups file > then it means it can be used by anyone that runs jtreg directly or anyone > that uses the make file to run tests ("make test TEST=jdk_stable" for > example). Whether it deserves its own make file is another question. > > So suppose we create such a group then what would be the criteria to be in > that group? Clearly the test should be stable in the sense that it should > pass when we don't have a bug. It should also clean up after itself. Things > that come to mind are: > > - should be usable with -agentvm? We have /othervm option for @run and we > also have othervm.dirs, the main point is that they can be run with either > in othervm or agentvm modes and they should just work. I have deliberately > not mentioned -samevm here as it's not suitable for the jdk tests. > > - needs to work with -concurrency, assuming sufficient resources. Is that > reasonable to require? We have /exclusive and exclusiveAccess.dirs available > for areas that have issues. > > - needs to run in a reasonable time. I don't think we have guidelines for > what is reasonable in the jdk tests but clearly a test that runs for more > than a few minutes needs to be looked at. > > - needs to run headless? Maybe this is controversial but it is somewhat > appealing to skip Xvfb or other setup. Also being selfish, I'd like to run > tests in a terminal window and not have windows dancing on my desktop. > What about having jdk_stable_headless, jdk_stable_headfull and jdk_stable_all beeing the union of these two? > - should not require special configuration? Maybe this is controversial too > but there are tests javax/print that fail when there isn't a printer > configured. > > Anything else? If we do something like this then such a group would need to > be maintained, at least for period until all tests are stable and fast. > > -Alan. > From Alan.Bateman at oracle.com Thu Dec 12 06:44:13 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 12 Dec 2013 14:44:13 +0000 Subject: JDK 8 b118 ea test results are now available In-Reply-To: References: <529DC2AA.8090506@oracle.com> <52A6139A.1020203@oracle.com> <52A6F908.3050908@oracle.com> <52A7327C.7050202@oracle.com> <52A743D2.2010104@oracle.com> <52A899CE.6090408@oracle.com> <52A8A72C.7050300@oracle.com> <52A8BC76.60409@oracle.com> <52A8C2E5.7010505@oracle.com> <52A8F0EA.2050008@oracle.com> <52A994BB.7020206@oracle.com> Message-ID: <52A9CBBD.6020800@oracle.com> On 12/12/2013 14:12, Volker Simonis wrote: > : > What about having jdk_stable_headless, jdk_stable_headfull and > jdk_stable_all beeing the union of these two? > We could, assuming it isn't too much effort to maintain. There are a few cases where the effort to get the tests to pass when headless shouldn't be too much and I'd have a preference to just doing that rather than listing individual tests in the group definitions. The beans tests are one example when they almost all pass when headless and it looks like there are only 10-12 that would need a small update to work headless. -Alan. From xerxes at zafena.se Fri Dec 13 03:20:35 2013 From: xerxes at zafena.se (=?ISO-8859-1?Q?Xerxes_R=E5nby?=) Date: Fri, 13 Dec 2013 12:20:35 +0100 Subject: Processing 2.1 fails to compile all of its projects using JDK8 EA b119 error: The type java.util.Map$Entry cannot be resolved. Message-ID: <52AAED83.50501@zafena.se> Processing 2.1 fails to compile all of its projects using JDK8 EA b119 error: The type java.util.Map$Entry cannot be resolved. This issue is a reproducer/reduced test case for: https://bugs.openjdk.java.net/browse/JDK-8024935 - compilation succeeds in java7 but fails in java8 I do not have any account at bugs.openjdk.java.net that I can use to update this bug, feel free to attach the information below to the bug. steps to reproduce: #Download processing 2.1 from: https://processing.org/download/ tar zxvf processing-2.1-linux32.tgz cd processing-2.1 #replace the bundled jdk inside processing with JDK8 EA b119 mv java java-bundled mv jdk1.8.0 java #run ./processing Press Sketch->Run without entering any code fails with the following output: Annotation processing got disabled, since it requires a 1.6 compliant JVM /tmp/sketch_131213a3040094527895471164temp/sketch_131213a.java:1: error: The type java.util.Map$Entry cannot be resolved. It is indirectly referenced from required .class files import processing.core.*; ^ 1 problem (1 error) From balchandra.vaidya at oracle.com Fri Dec 13 04:00:10 2013 From: balchandra.vaidya at oracle.com (Balchandra Vaidya) Date: Fri, 13 Dec 2013 12:00:10 +0000 Subject: Processing 2.1 fails to compile all of its projects using JDK8 EA b119 error: The type java.util.Map$Entry cannot be resolved. In-Reply-To: <52AAED83.50501@zafena.se> References: <52AAED83.50501@zafena.se> Message-ID: <52AAF6CA.3090304@oracle.com> Hi Xerxes, Thank you for your feedback. I have updated the bug https://bugs.openjdk.java.net/browse/JDK-8024935 with your instruction to reproduce the issue below. Thanks, Balchandra On 12/13/13 11:20 AM, Xerxes R?nby wrote: > Processing 2.1 fails to compile all of its projects using JDK8 EA b119 > error: The type java.util.Map$Entry cannot be resolved. > > This issue is a reproducer/reduced test case for: > https://bugs.openjdk.java.net/browse/JDK-8024935 - compilation succeeds in java7 but fails in java8 > > I do not have any account at bugs.openjdk.java.net that I can use to update this bug, > feel free to attach the information below to the bug. > > steps to reproduce: > > #Download processing 2.1 from: > https://processing.org/download/ > > tar zxvf processing-2.1-linux32.tgz > cd processing-2.1 > #replace the bundled jdk inside processing with JDK8 EA b119 > mv java java-bundled > mv jdk1.8.0 java > #run > ./processing > > Press > Sketch->Run > without entering any code fails with the following output: > > Annotation processing got disabled, since it requires a 1.6 compliant JVM > /tmp/sketch_131213a3040094527895471164temp/sketch_131213a.java:1: error: The type java.util.Map$Entry cannot be resolved. It is indirectly referenced from required .class files > import processing.core.*; > ^ > 1 problem (1 error) From balchandra.vaidya at oracle.com Fri Dec 13 06:35:34 2013 From: balchandra.vaidya at oracle.com (Balchandra Vaidya) Date: Fri, 13 Dec 2013 14:35:34 +0000 Subject: RFR: 8030035 Create a stable test group in TEST.groups Message-ID: <52AB1B36.7030008@oracle.com> All, I request to review a change for "8030035 - Create a stable test group in TEST.groups" http://cr.openjdk.java.net/~bvaidya/8/8030035/webrev.01/ https://bugs.openjdk.java.net/browse/JDK-8030035 _Background_: The test results [1] for JDK 8 ea build were produced using a long list of test directories [2] as described in the test execution instruction [3]. The above proposed change is taking advantage of jtreg group [4] feature to specify collections of tests and, therefore, eliminates the requirement of maintaining custom test directories [2] for weekly test execution. The above change simplify the jdk/test execution to: jtreg :jdk_stable Thanks Balchandra [1] http://www.java.net/download/jdk8/testresults/testresults.html [2] http://download.java.net/jdk8/testresults/docs/dir.list [3] http://download.java.net/jdk8/testresults/docs/howtoruntests.html [4] http://openjdk.java.net/jtreg/command-help.html -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/quality-discuss/attachments/20131213/09b17fc5/attachment.html From Alan.Bateman at oracle.com Fri Dec 13 07:33:11 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 13 Dec 2013 15:33:11 +0000 Subject: RFR: 8030035 Create a stable test group in TEST.groups In-Reply-To: <52AB1B36.7030008@oracle.com> References: <52AB1B36.7030008@oracle.com> Message-ID: <52AB28B7.6090505@oracle.com> On 13/12/2013 14:35, Balchandra Vaidya wrote: > > All, > > I request to review a change for "8030035 - Create a stable > test group in TEST.groups" > > http://cr.openjdk.java.net/~bvaidya/8/8030035/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8030035 I'm not sure about including javax/print in this list. They might be stable but they only pass if there is a printer attached. I see you have them listed as "known issues" [1] but I think it really reduces the chances that all the tests in the proposed group will pass. I'm also not sure about the SCTP test. We put them into their own group because they are highly sensitive to the kernel configuration and/or the libsctp that is installed. They do pass if SCTP is not in the kernel but they have been troublesome on older kernels that had it configured. I guess I can't really object to including this and we can always remove it if there are complaints. As regards the rest then there are few tests in these areas that fail when running headless. I'm interested in hearing other views on that topic. Volker's suggestion to have a stable and stable-headful is one approach. I guess I just object to tests doing strange polkas on my desktop. Also it is very appealing in server environments so not have to setup a virtual frame buffer. -Alan. [1] http://download.java.net/jdk8/testresults/docs/knownissues.html -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/quality-discuss/attachments/20131213/39f4831a/attachment.html From balchandra.vaidya at oracle.com Fri Dec 13 08:07:36 2013 From: balchandra.vaidya at oracle.com (Balchandra Vaidya) Date: Fri, 13 Dec 2013 16:07:36 +0000 Subject: Processing 2.1 fails to compile all of its projects using JDK8 EA b119 error: The type java.util.Map$Entry cannot be resolved. In-Reply-To: <52AAF6CA.3090304@oracle.com> References: <52AAED83.50501@zafena.se> <52AAF6CA.3090304@oracle.com> Message-ID: <52AB30C8.6070700@oracle.com> [cc'd compiler-dev at openjdk.java.net] On 12/13/13 12:00 PM, Balchandra Vaidya wrote: > > Hi Xerxes, > > Thank you for your feedback. I have updated the bug > https://bugs.openjdk.java.net/browse/JDK-8024935 with > your instruction to reproduce the issue below. > > > Thanks, > Balchandra > > > On 12/13/13 11:20 AM, Xerxes R?nby wrote: >> Processing 2.1 fails to compile all of its projects using JDK8 EA b119 >> error: The type java.util.Map$Entry cannot be resolved. >> >> This issue is a reproducer/reduced test case for: >> https://bugs.openjdk.java.net/browse/JDK-8024935 - compilation >> succeeds in java7 but fails in java8 >> >> I do not have any account at bugs.openjdk.java.net that I can use to >> update this bug, >> feel free to attach the information below to the bug. >> >> steps to reproduce: >> >> #Download processing 2.1 from: >> https://processing.org/download/ >> >> tar zxvf processing-2.1-linux32.tgz >> cd processing-2.1 >> #replace the bundled jdk inside processing with JDK8 EA b119 >> mv java java-bundled >> mv jdk1.8.0 java >> #run >> ./processing >> >> Press >> Sketch->Run >> without entering any code fails with the following output: >> >> Annotation processing got disabled, since it requires a 1.6 compliant >> JVM >> /tmp/sketch_131213a3040094527895471164temp/sketch_131213a.java:1: >> error: The type java.util.Map$Entry cannot be resolved. It is >> indirectly referenced from required .class files >> import processing.core.*; >> ^ >> 1 problem (1 error) > From balchandra.vaidya at oracle.com Mon Dec 16 03:16:17 2013 From: balchandra.vaidya at oracle.com (Balchandra Vaidya) Date: Mon, 16 Dec 2013 11:16:17 +0000 Subject: RFR: 8030035 Create a stable test group in TEST.groups In-Reply-To: <52AB28B7.6090505@oracle.com> References: <52AB1B36.7030008@oracle.com> <52AB28B7.6090505@oracle.com> Message-ID: <52AEE101.60302@oracle.com> On 12/13/13 03:33 PM, Alan Bateman wrote: > On 13/12/2013 14:35, Balchandra Vaidya wrote: >> >> All, >> >> I request to review a change for "8030035 - Create a stable >> test group in TEST.groups" >> >> http://cr.openjdk.java.net/~bvaidya/8/8030035/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8030035 > > I'm not sure about including javax/print in this list. They might be > stable but they only pass if there is a printer attached. I see you > have them listed as "known issues" [1] but I think it really reduces > the chances that all the tests in the proposed group will pass. In the java/print group, 11 tests pass and 5 tests fail when printer is not attached. I can remove it from the jdk_stable group. > > I'm also not sure about the SCTP test. We put them into their own > group because they are highly sensitive to the kernel configuration > and/or the libsctp that is installed. They do pass if SCTP is not in > the kernel but they have been troublesome on older kernels that had it > configured. I guess I can't really object to including this and we can > always remove it if there are complaints. I didn't see any issue with these tests on OEL 6. I am happy to remove it from the jdk_stable group if anyone complain on any other OSes. > > As regards the rest then there are few tests in these areas that fail > when running headless. I'm interested in hearing other views on that > topic. Volker's suggestion to have a stable and stable-headful is one > approach. I guess I just object to tests doing strange polkas on my > desktop. Also it is very appealing in server environments so not have > to setup a virtual frame buffer. I found 13 tests from jdk_stable group that fail in headless mode. In the short term, I can add a note in knownissues.html [1] page. In the long term, hoping to have a solution for those failing tests. Thanks Balchandra > > -Alan. > > [1] http://download.java.net/jdk8/testresults/docs/knownissues.html -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/quality-discuss/attachments/20131216/6863306e/attachment.html From balchandra.vaidya at oracle.com Mon Dec 16 04:22:21 2013 From: balchandra.vaidya at oracle.com (Balchandra Vaidya) Date: Mon, 16 Dec 2013 12:22:21 +0000 Subject: JDK 8 b120 ea test results are now available Message-ID: <52AEF07D.5050008@oracle.com> JDK 8 b120 ea test results are now available at: http://www.java.net/download/jdk8/testresults/testresults.html The jdk test results contain 28 differences when compared with b119 test results. One testcase [1] fails only with jtreg agentvm mode, but the same testcase passes with jtreg othervm mode. There is an open bug [2] for it. One testcase failed with an error because RMI server was not started. The test pass with re-run. Investigating the root-cause. 0: /home/jtest/merge/b119/jdk/JTwork pass: 4,646; fail: 13; not run: 904 1: /home/jtest/merge/b120/jdk/JTwork pass: 4,658; fail: 14; error: 1; not run: 900 0 1 Test pass fail com/sun/java/swing/plaf/windows/8016551/bug8016551.java pass --- com/sun/jmx/snmp/NoInfoLeakTest.java --- pass com/sun/tools/attach/BasicTests.java --- pass com/sun/tools/attach/PermissionTest.java pass --- com/sun/tools/attach/PermissionTests.sh --- pass com/sun/tools/attach/ProviderTest.java pass --- com/sun/tools/attach/ProviderTests.sh --- pass java/lang/management/MemoryMXBean/CollectionUsageThreshold.java pass --- java/lang/management/MemoryMXBean/CollectionUsageThresholdParallelGC.sh pass --- java/lang/management/MemoryMXBean/CollectionUsageThresholdSerialGC.sh --- pass java/rmi/reliability/benchmark/bench/rmi/Main.java --- pass java/rmi/reliability/benchmark/bench/serial/Main.java pass --- java/rmi/reliability/benchmark/runSerialBench.sh --- pass java/util/concurrent/CompletableFuture/ThenComposeAsyncTest.java --- pass java/util/concurrent/ConcurrentHashMap/ConcurrentAssociateTest.java --- pass java/util/concurrent/ConcurrentHashMap/ConcurrentContainsKeyTest.java --- pass java/util/concurrent/ThreadPoolExecutor/CoreThreadTimeOut.java --- pass java/util/logging/TestLogConfigurationDeadLock.java --- pass java/util/logging/TestLoggerBundleSync.java --- pass java/util/stream/TestDoubleSumAverage.java --- pass lib/security/CheckBlacklistedCerts.java --- pass lib/security/java.policy/Ext_AllPolicy.sh --- pass lib/testlibrary/AssertsTest.java --- pass lib/testlibrary/OutputAnalyzerReportingTest.java --- pass lib/testlibrary/OutputAnalyzerTest.java --- pass sun/net/www/protocol/http/RedirectOnPost.java --- pass sun/security/krb5/auto/LoginNoPass.java pass error sun/tools/jstatd/TestJstatdExternalRegistry.java 28 differences The hotspot test results have 2 differences. Investigating the testcase failure. 0: /home/jtest/merge/b119/hotspot/JTwork pass: 420; fail: 11; error: 2; not run: 7 1: /home/jtest/merge/b120/hotspot/JTwork pass: 418; fail: 12; error: 2; not run: 7 0 1 Test pass fail runtime/6925573/SortMethodsTest.java pass --- runtime/8024804/RegisterNatives.java 2 differences The langtools test results have 2 differences. 0: /home/jtest/merge/b119/langtools/JTwork pass: 2,960; not run: 7 1: /home/jtest/merge/b120/langtools/JTwork pass: 2,962; not run: 7 0 1 Test --- pass tools/javac/T8029179/CompileErrorForValidBooleanExpTest.java --- pass tools/javac/annotations/typeAnnotations/failures/CheckForDeclAnnoNPE.java 2 differences No new failures present in Nashorn test result: http://download.java.net/jdk8/testresults/archives/b120/emailable-report.html Regards Balchandra [1] test/com/sun/java/swing/plaf/windows/8016551/bug8016551.java [2] https://bugs.openjdk.java.net/browse/JDK-8029954 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/quality-discuss/attachments/20131216/7b3bdbcd/attachment.html From xerxes at zafena.se Mon Dec 16 05:42:18 2013 From: xerxes at zafena.se (=?ISO-8859-1?Q?Xerxes_R=E5nby?=) Date: Mon, 16 Dec 2013 14:42:18 +0100 Subject: Processing 2.1 fails to compile all of its projects using JDK8 EA b119 error: The type java.util.Map$Entry cannot be resolved. In-Reply-To: <52AB30C8.6070700@oracle.com> References: <52AAED83.50501@zafena.se> <52AAF6CA.3090304@oracle.com> <52AB30C8.6070700@oracle.com> Message-ID: <52AF033A.5020004@zafena.se> Processing internally used the Eclipse ecj compiler: The following code and command can reproduce the issue in combination with JDK 8 b119: $ cat HelloWorld.java import java.util.HashMap; public class HelloWorld { public static void main(String[] args) { System.out.println("Hello World!"); } } #download the eclipse ecj version used internally by processing. wget https://github.com/processing/processing/raw/master/java/mode/ecj.jar tar zxvf jdk-8-ea-bin-b119-linux-i586-05_dec_2013.tar.gz ./jdk1.8.0/bin/java -jar ecj.jar -source 1.6 -classpath . -nowarn HelloWorld.java Annotation processing got disabled, since it requires a 1.6 compliant JVM ---------- 1. ERROR in HelloWorld.java (at line 1) import java.util.HashMap; ^ The type java.util.Map$Entry cannot be resolved. It is indirectly referenced from required .class files ---------- 1 problem (1 error) # The compilation succeed if ecj.jar is invoked without the -source 1.6 argument: ./jdk1.8.0/bin/java -jar ecj.jar -classpath . -nowarn HelloWorld.java ./jdk1.8.0/bin/java HelloWorld Hello World! https://bugs.openjdk.java.net/browse/JDK-8024935 - compilation succeeds in java7 but fails in java8 I do not have any account at bugs.openjdk.java.net that I can use to update this bug, feel free to attach the information above to the bug. Cheers Xerxes 2013-12-13 17:07, Balchandra Vaidya skrev: > [cc'd compiler-dev at openjdk.java.net] > > > On 12/13/13 12:00 PM, Balchandra Vaidya wrote: >> >> Hi Xerxes, >> >> Thank you for your feedback. I have updated the bug >> https://bugs.openjdk.java.net/browse/JDK-8024935 with >> your instruction to reproduce the issue below. >> >> >> Thanks, >> Balchandra >> >> >> On 12/13/13 11:20 AM, Xerxes R?nby wrote: >>> Processing 2.1 fails to compile all of its projects using JDK8 EA b119 >>> error: The type java.util.Map$Entry cannot be resolved. >>> >>> This issue is a reproducer/reduced test case for: >>> https://bugs.openjdk.java.net/browse/JDK-8024935 - compilation succeeds in java7 but fails in java8 >>> >>> I do not have any account at bugs.openjdk.java.net that I can use to update this bug, >>> feel free to attach the information below to the bug. >>> >>> steps to reproduce: >>> >>> #Download processing 2.1 from: >>> https://processing.org/download/ >>> >>> tar zxvf processing-2.1-linux32.tgz >>> cd processing-2.1 >>> #replace the bundled jdk inside processing with JDK8 EA b119 >>> mv java java-bundled >>> mv jdk1.8.0 java >>> #run >>> ./processing >>> >>> Press >>> Sketch->Run >>> without entering any code fails with the following output: >>> >>> Annotation processing got disabled, since it requires a 1.6 compliant JVM >>> /tmp/sketch_131213a3040094527895471164temp/sketch_131213a.java:1: error: The type java.util.Map$Entry cannot be resolved. It is indirectly referenced from required .class files >>> import processing.core.*; >>> ^ >>> 1 problem (1 error) >> > From xerxes at zafena.se Mon Dec 16 06:57:03 2013 From: xerxes at zafena.se (=?ISO-8859-1?Q?Xerxes_R=E5nby?=) Date: Mon, 16 Dec 2013 15:57:03 +0100 Subject: Processing 2.1 fails to compile all of its projects using JDK8 EA b119 error: The type java.util.Map$Entry cannot be resolved. In-Reply-To: <52AF033A.5020004@zafena.se> References: <52AAED83.50501@zafena.se> <52AAF6CA.3090304@oracle.com> <52AB30C8.6070700@oracle.com> <52AF033A.5020004@zafena.se> Message-ID: <52AF14BF.1040302@zafena.se> This have been fixed in eclipse ecj upstrem. Updating java/mode/ecj.jar inside Processing 2.1 to eclipse ecj version 4.3.1 http://www.eclipse.org/downloads/download.php?file=/eclipse/downloads/drops4/R-4.3.1-201309111000/ecj-4.3.1.jar fixes this issue. Cheers Xerxes 2013-12-16 14:42, Xerxes R?nby skrev: > Processing internally used the Eclipse ecj compiler: > The following code and command can reproduce the issue in combination with JDK 8 b119: > > $ cat HelloWorld.java > import java.util.HashMap; > > public class HelloWorld { > public static void main(String[] args) { > System.out.println("Hello World!"); > } > } > > #download the eclipse ecj version used internally by processing. > wget https://github.com/processing/processing/raw/master/java/mode/ecj.jar > > tar zxvf jdk-8-ea-bin-b119-linux-i586-05_dec_2013.tar.gz > > ./jdk1.8.0/bin/java -jar ecj.jar -source 1.6 -classpath . -nowarn HelloWorld.java > Annotation processing got disabled, since it requires a 1.6 compliant JVM > ---------- > 1. ERROR in HelloWorld.java (at line 1) > import java.util.HashMap; > ^ > The type java.util.Map$Entry cannot be resolved. It is indirectly referenced from required .class files > ---------- > 1 problem (1 error) > > > > # The compilation succeed if ecj.jar is invoked without the -source 1.6 argument: > ./jdk1.8.0/bin/java -jar ecj.jar -classpath . -nowarn HelloWorld.java > ./jdk1.8.0/bin/java HelloWorld > Hello World! > > > https://bugs.openjdk.java.net/browse/JDK-8024935 - compilation succeeds in java7 but fails in java8 > I do not have any account at bugs.openjdk.java.net that I can use to update this bug, > feel free to attach the information above to the bug. > > Cheers > Xerxes > > 2013-12-13 17:07, Balchandra Vaidya skrev: >> [cc'd compiler-dev at openjdk.java.net] >> >> >> On 12/13/13 12:00 PM, Balchandra Vaidya wrote: >>> >>> Hi Xerxes, >>> >>> Thank you for your feedback. I have updated the bug >>> https://bugs.openjdk.java.net/browse/JDK-8024935 with >>> your instruction to reproduce the issue below. >>> >>> >>> Thanks, >>> Balchandra >>> >>> >>> On 12/13/13 11:20 AM, Xerxes R?nby wrote: >>>> Processing 2.1 fails to compile all of its projects using JDK8 EA b119 >>>> error: The type java.util.Map$Entry cannot be resolved. >>>> >>>> This issue is a reproducer/reduced test case for: >>>> https://bugs.openjdk.java.net/browse/JDK-8024935 - compilation succeeds in java7 but fails in java8 >>>> >>>> I do not have any account at bugs.openjdk.java.net that I can use to update this bug, >>>> feel free to attach the information below to the bug. >>>> >>>> steps to reproduce: >>>> >>>> #Download processing 2.1 from: >>>> https://processing.org/download/ >>>> >>>> tar zxvf processing-2.1-linux32.tgz >>>> cd processing-2.1 >>>> #replace the bundled jdk inside processing with JDK8 EA b119 >>>> mv java java-bundled >>>> mv jdk1.8.0 java >>>> #run >>>> ./processing >>>> >>>> Press >>>> Sketch->Run >>>> without entering any code fails with the following output: >>>> >>>> Annotation processing got disabled, since it requires a 1.6 compliant JVM >>>> /tmp/sketch_131213a3040094527895471164temp/sketch_131213a.java:1: error: The type java.util.Map$Entry cannot be resolved. It is indirectly referenced from required .class files >>>> import processing.core.*; >>>> ^ >>>> 1 problem (1 error) >>> >> > From balchandra.vaidya at oracle.com Tue Dec 17 05:02:06 2013 From: balchandra.vaidya at oracle.com (Balchandra Vaidya) Date: Tue, 17 Dec 2013 13:02:06 +0000 Subject: RFR: 8030035 Create a stable test group in TEST.groups In-Reply-To: <52AEE101.60302@oracle.com> References: <52AB1B36.7030008@oracle.com> <52AB28B7.6090505@oracle.com> <52AEE101.60302@oracle.com> Message-ID: <52B04B4E.9020406@oracle.com> Here is the updated webrev. I have removed javax/print from the list. http://cr.openjdk.java.net/~bvaidya/8/8030035/webrev.02/ Thanks Balchandra On 12/16/13 11:16 AM, Balchandra Vaidya wrote: > > On 12/13/13 03:33 PM, Alan Bateman wrote: >> On 13/12/2013 14:35, Balchandra Vaidya wrote: >>> >>> All, >>> >>> I request to review a change for "8030035 - Create a stable >>> test group in TEST.groups" >>> >>> http://cr.openjdk.java.net/~bvaidya/8/8030035/webrev.01/ >>> https://bugs.openjdk.java.net/browse/JDK-8030035 >> >> I'm not sure about including javax/print in this list. They might be >> stable but they only pass if there is a printer attached. I see you >> have them listed as "known issues" [1] but I think it really reduces >> the chances that all the tests in the proposed group will pass. > > In the java/print group, 11 tests pass and 5 tests fail when printer is > not attached. I can remove it from the jdk_stable group. > >> >> I'm also not sure about the SCTP test. We put them into their own >> group because they are highly sensitive to the kernel configuration >> and/or the libsctp that is installed. They do pass if SCTP is not in >> the kernel but they have been troublesome on older kernels that had >> it configured. I guess I can't really object to including this and we >> can always remove it if there are complaints. > > I didn't see any issue with these tests on OEL 6. I am happy to remove > it from the jdk_stable group if anyone complain on any other OSes. > >> >> As regards the rest then there are few tests in these areas that fail >> when running headless. I'm interested in hearing other views on that >> topic. Volker's suggestion to have a stable and stable-headful is one >> approach. I guess I just object to tests doing strange polkas on my >> desktop. Also it is very appealing in server environments so not have >> to setup a virtual frame buffer. > > I found 13 tests from jdk_stable group that fail in headless mode. In > the short term, > I can add a note in knownissues.html [1] page. In the long term, > hoping to have a > solution for those failing tests. > > > Thanks > Balchandra > > >> >> -Alan. >> >> [1] http://download.java.net/jdk8/testresults/docs/knownissues.html > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/quality-discuss/attachments/20131217/7570c671/attachment.html From Alan.Bateman at oracle.com Tue Dec 17 07:28:30 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 17 Dec 2013 15:28:30 +0000 Subject: RFR: 8030035 Create a stable test group in TEST.groups In-Reply-To: <52B04B4E.9020406@oracle.com> References: <52AB1B36.7030008@oracle.com> <52AB28B7.6090505@oracle.com> <52AEE101.60302@oracle.com> <52B04B4E.9020406@oracle.com> Message-ID: <52B06D9E.8020708@oracle.com> On 17/12/2013 13:02, Balchandra Vaidya wrote: > > Here is the updated webrev. I have removed javax/print > from the list. > http://cr.openjdk.java.net/~bvaidya/8/8030035/webrev.02/ > I've pushed this to jdk9/dev. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/quality-discuss/attachments/20131217/74fe61c7/attachment.html From volker.simonis at gmail.com Wed Dec 18 08:44:58 2013 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 18 Dec 2013 17:44:58 +0100 Subject: TEST: java/util/logging/LoggerWeakRefLeak.sh Message-ID: Hi, the test 'java/util/logging/LoggerWeakRefLeak.sh' checks for the availability of the '-histo:live' option in jmap which is only available if the SA is available. However I don't think that it should be an error if the option isn't supported because of a missing SA implementation on a certain platfrom. I suggest the following fix: --- a/test/java/util/logging/LoggerWeakRefLeak.sh Tue Dec 17 19:01:21 2013 +0100 +++ b/test/java/util/logging/LoggerWeakRefLeak.sh Wed Dec 18 17:40:46 2013 +0100 @@ -83,9 +83,9 @@ fi if [ "$status" != 0 ]; then - echo "ERROR: 'jmap $jmap_option' is not supported so this test" - echo "ERROR: cannot work reliably. Aborting!" - exit 2 + echo "WARNING: 'jmap $jmap_option' is not supported on this platform" + echo "WARNING: so this test cannot work reliably. Aborting!" + exit 0 fi fi I want to put this fix into the patch for "8028537: PPC64: Updated jdk/test scripts to understand the AIX os and environment" so I don't think we need a special BugID for it. What do you think? Regards, Volker From volker.simonis at gmail.com Wed Dec 18 08:48:03 2013 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 18 Dec 2013 17:48:03 +0100 Subject: TEST: java/util/logging/LoggerWeakRefLeak.sh In-Reply-To: References: Message-ID: Sorry, I forgot to mention that the same problem holds true for the test java/util/logging/AnonLoggerWeakRefLeak.sh as well. And it can be fixed in the same way as proposed for LoggerWeakRefLeak.sh. Volker On Wed, Dec 18, 2013 at 5:44 PM, Volker Simonis wrote: > Hi, > > the test 'java/util/logging/LoggerWeakRefLeak.sh' checks for the > availability of the '-histo:live' option in jmap which is only > available if the SA is available. > > However I don't think that it should be an error if the option isn't > supported because of a missing SA implementation on a certain > platfrom. > > I suggest the following fix: > > --- a/test/java/util/logging/LoggerWeakRefLeak.sh Tue Dec 17 > 19:01:21 2013 +0100 > +++ b/test/java/util/logging/LoggerWeakRefLeak.sh Wed Dec 18 > 17:40:46 2013 +0100 > @@ -83,9 +83,9 @@ > fi > > if [ "$status" != 0 ]; then > - echo "ERROR: 'jmap $jmap_option' is not supported so this test" > - echo "ERROR: cannot work reliably. Aborting!" > - exit 2 > + echo "WARNING: 'jmap $jmap_option' is not supported on this platform" > + echo "WARNING: so this test cannot work reliably. Aborting!" > + exit 0 > fi > fi > > I want to put this fix into the patch for "8028537: PPC64: Updated > jdk/test scripts to understand the AIX os and environment" so I don't > think we need a special BugID for it. > > What do you think? > > Regards, > Volker From Alan.Bateman at oracle.com Wed Dec 18 08:58:53 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 18 Dec 2013 16:58:53 +0000 Subject: TEST: java/util/logging/LoggerWeakRefLeak.sh In-Reply-To: References: Message-ID: <52B1D44D.8060801@oracle.com> On 18/12/2013 16:44, Volker Simonis wrote: > Hi, > > the test 'java/util/logging/LoggerWeakRefLeak.sh' checks for the > availability of the '-histo:live' option in jmap which is only > available if the SA is available. > > However I don't think that it should be an error if the option isn't > supported because of a missing SA implementation on a certain > platfrom. The tests have a lot of implementation dependencies and I'm sure you'll run into others too. > > I suggest the following fix: > > --- a/test/java/util/logging/LoggerWeakRefLeak.sh Tue Dec 17 > 19:01:21 2013 +0100 > +++ b/test/java/util/logging/LoggerWeakRefLeak.sh Wed Dec 18 > 17:40:46 2013 +0100 > @@ -83,9 +83,9 @@ > fi > > if [ "$status" != 0 ]; then > - echo "ERROR: 'jmap $jmap_option' is not supported so this test" > - echo "ERROR: cannot work reliably. Aborting!" > - exit 2 > + echo "WARNING: 'jmap $jmap_option' is not supported on this platform" > + echo "WARNING: so this test cannot work reliably. Aborting!" > + exit 0 > fi > fi > > I want to put this fix into the patch for "8028537: PPC64: Updated > jdk/test scripts to understand the AIX os and environment" so I don't > think we need a special BugID for it. > > What do you think? > I'd prefer to see this one (and AnonLoggerWeakRefLeak) changed to not use jmap to detect leaks but that is beyond the scope of what you are doing. What you have is fine and shouldn't have any impact on other platforms. -Alan From volker.simonis at gmail.com Wed Dec 18 10:35:15 2013 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 18 Dec 2013 19:35:15 +0100 Subject: TEST: java/util/logging/LoggerWeakRefLeak.sh In-Reply-To: <52B1D44D.8060801@oracle.com> References: <52B1D44D.8060801@oracle.com> Message-ID: On Wed, Dec 18, 2013 at 5:58 PM, Alan Bateman wrote: > On 18/12/2013 16:44, Volker Simonis wrote: >> >> Hi, >> >> the test 'java/util/logging/LoggerWeakRefLeak.sh' checks for the >> availability of the '-histo:live' option in jmap which is only >> available if the SA is available. >> >> However I don't think that it should be an error if the option isn't >> supported because of a missing SA implementation on a certain >> platfrom. > > The tests have a lot of implementation dependencies and I'm sure you'll run > into others too. > Yes, but the situation is starting to improve slowly:) With the following two additional small changes: diff -r 1baadb741a33 test/sun/tools/jinfo/Basic.sh --- a/test/sun/tools/jinfo/Basic.sh Tue Dec 17 19:01:21 2013 +0100 +++ b/test/sun/tools/jinfo/Basic.sh Wed Dec 18 19:23:21 2013 +0100 @@ -45,7 +45,7 @@ runSA=true -if [ $isMacos = true ]; then +if [ $isMacos = true -o $isAIX = true -o `uname -m` = ppc64 ]; then runSA=false fi diff -r 1baadb741a33 test/tools/launcher/Settings.java --- a/test/tools/launcher/Settings.java Tue Dec 17 19:01:21 2013 +0100 +++ b/test/tools/launcher/Settings.java Wed Dec 18 19:23:21 2013 +0100 @@ -73,16 +73,20 @@ } static void runTestOptionDefault() throws IOException { + String stackSize = "256"; // in kb + if (getArch().equals("ppc64")) { + stackSize = "800"; + } TestResult tr = null; tr = doExec(javaCmd, "-Xms64m", "-Xmx512m", - "-Xss256k", "-XshowSettings", "-jar", testJar.getAbsolutePath()); + "-Xss" + stackSize + "k", "-XshowSettings", "-jar", testJar.getAbsolutePath()); containsAllOptions(tr); if (!tr.isOK()) { System.out.println(tr.status); throw new RuntimeException("test fails"); } tr = doExec(javaCmd, "-Xms65536k", "-Xmx712m", - "-Xss256000", "-XshowSettings", "-jar", testJar.getAbsolutePath()); + "-Xss" + stackSize + "000", "-XshowSettings", "-jar", testJar.getAbsolutePath()); containsAllOptions(tr); if (!tr.isOK()) { System.out.println(tr.status); I get the same pass rate on Linux/PPC64 like on Linux/x86_64 (4672 passed / 1 Failed / 0 Errors / 900 Not run). The one that fails also fails in jdk8b120 so it's nothing I'm going to worry about in the next days or so:) Admittedly, Linux/PPC64 was the easy part and for AIX I still have about 100 failures and 50 errors, but I hope to cut that drastically down with a few changes until the end of the week. > >> >> I suggest the following fix: >> >> --- a/test/java/util/logging/LoggerWeakRefLeak.sh Tue Dec 17 >> 19:01:21 2013 +0100 >> +++ b/test/java/util/logging/LoggerWeakRefLeak.sh Wed Dec 18 >> 17:40:46 2013 +0100 >> @@ -83,9 +83,9 @@ >> fi >> >> if [ "$status" != 0 ]; then >> - echo "ERROR: 'jmap $jmap_option' is not supported so this test" >> - echo "ERROR: cannot work reliably. Aborting!" >> - exit 2 >> + echo "WARNING: 'jmap $jmap_option' is not supported on this >> platform" >> + echo "WARNING: so this test cannot work reliably. Aborting!" >> + exit 0 >> fi >> fi >> >> I want to put this fix into the patch for "8028537: PPC64: Updated >> jdk/test scripts to understand the AIX os and environment" so I don't >> think we need a special BugID for it. >> >> What do you think? >> > I'd prefer to see this one (and AnonLoggerWeakRefLeak) changed to not use > jmap to detect leaks but that is beyond the scope of what you are doing. > What you have is fine and shouldn't have any impact on other platforms. > Thanks! > -Alan From balchandra.vaidya at oracle.com Fri Dec 27 01:46:11 2013 From: balchandra.vaidya at oracle.com (Balchandra Vaidya) Date: Fri, 27 Dec 2013 09:46:11 +0000 Subject: JDK 8 b121 ea test results are now available Message-ID: <52BD4C63.6070509@oracle.com> JDK 8 b121 ea test results are now available at: http://www.java.net/download/jdk8/testresults/testresults.html The jdk test results contain 1 difference. 0: /home/jtest/merge/b120/jdk/JTwork pass: 4,658; fail: 14; error: 1; not run: 900 1: /home/jtest/merge/b121/jdk/JTwork pass: 4,659; fail: 14; not run: 900 0 1 Test error pass sun/tools/jstatd/TestJstatdExternalRegistry.java 1 differences The hotspot test results contain 5 differences. 0: /home/jtest/merge/b120/hotspot/JTwork pass: 418; fail: 12; error: 2; not run: 7 1: /home/jtest/merge/b121/hotspot/JTwork pass: 423; fail: 10; error: 1; not run: 7 0 1 Test fail pass compiler/7141637/SpreadNullArg.java --- pass compiler/reflection/ArrayNewInstanceOfVoid.java error pass compiler/regalloc/C1ObjectSpillInLogicOp.java fail pass runtime/6925573/SortMethodsTest.java --- pass runtime/8024804/RegisterNatives.java 5 differences No differences found in langtools test results. No new failures present in Nashorn test result: http://download.java.net/jdk8/testresults/archives/b121/emailable-report.html Regards Balchandra -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/quality-discuss/attachments/20131227/e61ed6d1/attachment.html