From jonathan.gibbons at oracle.com Fri Jun 21 07:51:48 2013 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Fri, 21 Jun 2013 07:51:48 -0700 Subject: JTreg Build/Documentation Feedback In-Reply-To: References: Message-ID: <51C46884.1080505@oracle.com> Richard, Thank you for your feedback. I will work to address the issues you have raised. -- Jon On 06/21/2013 06:48 AM, Richard Warburton wrote: > Hi, > > Just a bit of feedback on building jtreg myself. Compilation issues are > with respect to building the hg head. > > *Java.net Projects* > Documentation and links for jtharness and sigtest both point to the > java.netsites. It looks like the code is now hosted in openjdk, in > code tools. Is > there any reason for the documentation to be somewhere totally different > rather than standardised on the openjdk wiki? > > *Java 7* > By default jtreg doesn't seem to compile with a java 7 JDK, because even > though target is set to 5, source isn't set and the combination of > source=7, target=5 isn't allowed. Proposal: when target is set to 5, set > source to 5 as well or tell people to use a Java 6 jvm. > > *Optional Dependencies* > Both junit and testng are listed as optional dependencies, but the default > build (make -C make) fails without them available. It would be more > friendly to have a better error message around this or not compile > junit/testng specific code if the dependency is missing. > > *Directories* > The make script seems to assume that the non-standard > directory /java/devtools/share/ exists and puts junit and testng from that > path on the classpath. I needed to create this directory and symbol /opt > to it, even though junit and testng were in the places referred to by > make/build.properties. Again this is using 'make -C make' > > regards, > > Dr. Richard Warburton > > http://insightfullogic.com > @RichardWarburto From roger.riggs at oracle.com Mon Jun 10 12:55:40 2013 From: roger.riggs at oracle.com (roger riggs) Date: Mon, 10 Jun 2013 15:55:40 -0400 Subject: jtreg: how to set maxOutputSize? Message-ID: <51B62F3C.6020204@oracle.com> jtreg /jtharness is limiting the output size for some test runs. I see a documented setting for javatest.maxOutputSize but it does not seem to change the behavior. Invoked as: $JTREG_HOME/linux/bin/jtreg -Djavatest.maxOutputSize=1000000 -jdk:$JAVA_HOME -dir:$WS/jdk/test/ $* Any suggestions? Thanks, Roger From roger.riggs at oracle.com Mon Jun 24 12:20:51 2013 From: roger.riggs at oracle.com (roger riggs) Date: Mon, 24 Jun 2013 15:20:51 -0400 Subject: Request for Review of jtreg reporting improvements Message-ID: <51C89C13.2040801@oracle.com> Please review changes to address these jtreg reporting output issues: CODETOOLS-7900130 Test summary does not show TestNg dataProvider arguments CODETOOLS-7900165 Exception in TestNG DataSource does not result in a reported test failure CODETOOLS-7900039 TestNG Expected exception stack traces in reports are misleading CODETOOLS-7900112 Don't print unnecessary stack traces (dup of 7900039) http://cr.openjdk.java.net/~rriggs/webrev-jtreg-testng-reporting-7900130/ What is sufficient testing for jtreg changes? Roger From jonathan.gibbons at oracle.com Mon Jun 24 13:24:34 2013 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Mon, 24 Jun 2013 13:24:34 -0700 Subject: Request for Review of jtreg reporting improvements In-Reply-To: <51C89C13.2040801@oracle.com> References: <51C89C13.2040801@oracle.com> Message-ID: <51C8AB02.9010900@oracle.com> On 06/24/2013 12:20 PM, roger riggs wrote: > Please review changes to address these jtreg reporting output issues: > > CODETOOLS-7900130 > Test summary > does not show TestNg dataProvider arguments > CODETOOLS-7900165 > Exception in > TestNG DataSource does not result in a reported test failure > CODETOOLS-7900039 > TestNG Expected > exception stack traces in reports are misleading > CODETOOLS-7900112 > Don't print > unnecessary stack traces (dup of 7900039) > > http://cr.openjdk.java.net/~rriggs/webrev-jtreg-testng-reporting-7900130/ > > What is sufficient testing for jtreg changes? > > Roger > > I try and create a test case in make/tests, perhaps using a toy test suite in src/share/test/javatest/regtest. Use jtreg to run the test(s) in the test suite and grep the output for what you are (not) looking for. Failing that, a thorough eyeball test works too. -- Jon From Roger.Riggs at Oracle.com Tue Jun 25 13:18:51 2013 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 25 Jun 2013 16:18:51 -0400 Subject: Request for Review of jtreg reporting improvements In-Reply-To: <51C8AB02.9010900@oracle.com> References: <51C89C13.2040801@oracle.com> <51C8AB02.9010900@oracle.com> Message-ID: <51C9FB2B.2040700@Oracle.com> Hi, The webrev has been updated with tests for each of the changes: http://cr.openjdk.java.net/~rriggs/webrev-jtreg-testng-reporting-7900130 Thanks for any comments, Roger On 6/24/13 4:24 PM, Jonathan Gibbons wrote: > On 06/24/2013 12:20 PM, roger riggs wrote: >> Please review changes to address these jtreg reporting output issues: >> >> CODETOOLS-7900130 >> Test summary >> does not show TestNg dataProvider arguments >> CODETOOLS-7900165 >> Exception in >> TestNG DataSource does not result in a reported test failure >> CODETOOLS-7900039 >> TestNG >> Expected exception stack traces in reports are misleading >> CODETOOLS-7900112 >> Don't print >> unnecessary stack traces (dup of 7900039) >> >> http://cr.openjdk.java.net/~rriggs/webrev-jtreg-testng-reporting-7900130/ >> >> >> What is sufficient testing for jtreg changes? >> >> Roger >> >> > > I try and create a test case in make/tests, perhaps using a toy test > suite in src/share/test/javatest/regtest. Use jtreg to run the test(s) > in the test suite and grep the output for what you are (not) looking for. > > Failing that, a thorough eyeball test works too. > > -- Jon