From alexandre.iline at oracle.com Thu Apr 12 18:38:51 2018 From: alexandre.iline at oracle.com (Alexandre (Shura) Iline) Date: Thu, 12 Apr 2018 11:38:51 -0700 Subject: Tool suggestion: JTReg test module dependencies verifier. In-Reply-To: References: <1BE70BC9-B15C-498E-B3C2-7E01C36AFC65@oracle.com> Message-ID: Jon, I do not think I have ever replied to this in writing, even though we have discussed this in person. There are indeed a few checks on test description which could be useful, in addition to what you mentioned. I am specifically calling that out as non-requirement because my thinking was that a tool to verify JTResg test description could be a next step. Such tool would then be using the "JTReg test module dependencies verifier? as a module of a sort. Shura > On Mar 29, 2018, at 3:51 PM, Jonathan Gibbons wrote: > > Shura, > > I think the tool you are proposing could be the first in a series to help check for issues in jtreg test descriptions that for whatever reason (typically performance) we do not want to check every time we use jtreg to run tests. > > Other (future) checks could be, using @compile where @build would be more appropriate, checking the set of @build directives, and so on. > > -- Jon > > On 3/23/18 5:54 PM, Alexandre (Shura) Iline wrote: >> Hi. >> >> This is an invitation to discuss requirements for a ?JTReg test module dependencies verifier? (for not having a better name). >> >> Background: test module dependencies are declared with @modules JTReg tag and modules entry in test suite configuration files. Please consult JTReg documentation for more information. >> >> Problem statement: At the current state there is no effective way for a test developer to verify correctness of module declarations for a test or a group of tests. To verify that information, a test developer needs to manually limit module availability to those declared as module dependencies. This could be achieved either through VM options or by building a custom JDK image. That operation has to be repeated for every subset of tests having unique set of module dependencies. This is a tedious, error prone manual process. >> >> Intent of the tool is to assess correctness of declared module dependencies of JTReg tests. >> >> I. Functional requirements >> 1. The tool must be able to detect tests which, for execution, depend on presence of more modules than declared. >> 2. There must be a way to specify a subset of tests for analysis in a way that is compatible with JTReg. >> 3. Results of the module dependencies assessment should be stored in a machine parse-able format. >> 3. Results of the module dependencies assessment should be provided in a human readable format. >> 4. The tool must allow to specify additional test execution parameters which influence test behavior. >> 5. The tool may provide some analysis of test artifacts in order to identify missing module dependencies. >> 6. The tool may provide some analysis of test artifacts in order to identify extra, not needed module dependencies. >> 7. The tool must provide a command-line interface. >> >> II. Non-functional requirements >> 1. The tool must use JTReg tool or JTReg API to perform test selection and test execution. >> >> III. Non-requirement >> 1. The tool does not have to be a part of JTReg source repository. >> 2. The tool does not have to be a part of binary JTReg distribution. >> 3. It is not a requirement for the tool to verify other JTReg test tags or any other aspects of test suite consistency. >> >> >> Shura > From jonathan.gibbons at oracle.com Fri Apr 13 22:19:09 2018 From: jonathan.gibbons at oracle.com (jonathan.gibbons at oracle.com) Date: Fri, 13 Apr 2018 22:19:09 +0000 Subject: hg: code-tools/jtreg: Updates to facilitate testing via the build-all script Message-ID: <201804132219.w3DMJ9X3002860@aojmv0008.oracle.com> Changeset: d4af698e1681 Author: jjg Date: 2018-04-13 15:18 -0700 URL: http://hg.openjdk.java.net/code-tools/jtreg/rev/d4af698e1681 Updates to facilitate testing via the build-all script ! README ! make/Makefile ! make/build-all.sh ! test/basic/Basic.gmk ! test/bootclasspath/BootClassPathTest.gmk ! test/ignoresymbolfile/IgnoreSymbolFileTest.gmk ! test/keywords/testKeywords.gmk ! test/maxOutputSize/MaxOutputSize.gmk ! test/rerun/RerunTest.gmk ! test/testng-junit/TestNGJUnitTest.gmk ! test/timeouthandlers/TimeoutHandlerTest.gmk From jonathan.gibbons at oracle.com Fri Apr 13 23:52:52 2018 From: jonathan.gibbons at oracle.com (jonathan.gibbons at oracle.com) Date: Fri, 13 Apr 2018 23:52:52 +0000 Subject: hg: code-tools/jtreg: set default for MAKE_ARGS parameter Message-ID: <201804132352.w3DNqqpL018142@aojmv0008.oracle.com> Changeset: 972f31b5aa10 Author: jjg Date: 2018-04-13 16:52 -0700 URL: http://hg.openjdk.java.net/code-tools/jtreg/rev/972f31b5aa10 set default for MAKE_ARGS parameter ! make/build-all.sh From jonathan.gibbons at oracle.com Fri Apr 13 23:57:23 2018 From: jonathan.gibbons at oracle.com (jonathan.gibbons at oracle.com) Date: Fri, 13 Apr 2018 23:57:23 +0000 Subject: hg: code-tools/jtreg: Allow to skip downloading external components Message-ID: <201804132357.w3DNvNKG020604@aojmv0008.oracle.com> Changeset: 26b22708cd1b Author: jjg Date: 2018-04-13 16:55 -0700 URL: http://hg.openjdk.java.net/code-tools/jtreg/rev/26b22708cd1b Allow to skip downloading external components ! make/build-all.sh From sophiag at ca.ibm.com Tue Apr 17 18:31:37 2018 From: sophiag at ca.ibm.com (Sophia Guo) Date: Tue, 17 Apr 2018 14:31:37 -0400 Subject: Error: Unexpected exception occurred! java.lang.IllegalStateException with default -w and -r Message-ID: Hi all, I'm running jtreg tests with default -w and -r get "Error: Unexpected exception occurred! java.lang.IllegalStateException" jtreg version details: jtreg, version 4.2 dev b12 jdk version: jdk8 ( no issue to jdk9 and jdk10) command: ./jtreg/bin/jtreg -verbose:all -jdk:./jdk8u172-b03/ ./openjdk-jdk8u/jdk/test/java/math/BigDecimal/AddTests.java result: TEST: java/math/BigDecimal/AddTests.java TEST JDK: /Users/sophiag/adoptjdktest/openjdk8binary/jdk8u172-b03 ACTION: build -- Passed. All files up to date REASON: Named class compiled on demand TIME: 0.001 seconds messages: command: build AddTests reason: Named class compiled on demand elapsed time (seconds): 0.001 ... ... TEST RESULT: Passed. Execution successful Test results: passed: 1 Error: Unexpected exception occurred! java.lang.IllegalStateException java.lang.IllegalStateException at com.sun.javatest.TestResult.setTestDescription(TestResult.java:1919) at com.sun.javatest.TestResultTable.getCachedResult (TestResultTable.java:2193) at com.sun.javatest.TRT_TreeNode.processFile(TRT_TreeNode.java:1115) at com.sun.javatest.TRT_TreeNode.scanIfNeeded(TRT_TreeNode.java:651) at com.sun.javatest.TRT_TreeNode.getChildCount(TRT_TreeNode.java:125) at com.sun.javatest.TRT_Iterator$PseudoFrame.nextIndex (TRT_Iterator.java:989) at com.sun.javatest.TRT_Iterator.findNext(TRT_Iterator.java:616) at com.sun.javatest.TRT_Iterator.nextElement(TRT_Iterator.java:186) at com.sun.javatest.TRT_Iterator.next(TRT_Iterator.java:208) at com.sun.javatest.report.ReportSettings.setupSortedResults (ReportSettings.java:483) at com.sun.javatest.report.Report.writeReports(Report.java:111) at com.sun.javatest.report.Report.writeReport(Report.java:90) at com.sun.javatest.regtest.report.RegressionReporter.report (RegressionReporter.java:102) at com.sun.javatest.regtest.tool.Tool.batchHarness(Tool.java:1868) at com.sun.javatest.regtest.tool.Tool.run(Tool.java:1242) at com.sun.javatest.regtest.tool.Tool.run(Tool.java:967) at com.sun.javatest.regtest.tool.Tool.main(Tool.java:139) at com.sun.javatest.regtest.Main.main(Main.java:58) Regards, Sophia Guo Software Developer, IBM Runtime Technologies IBM Ottawa Software Lab 3755 Riverside Drive, Ottawa, Ontario, Canada K1G 4K9 Phone: (613) 356-5686, tie-line: 315-5686 From adam.farley at uk.ibm.com Wed Apr 18 12:47:22 2018 From: adam.farley at uk.ibm.com (Adam Farley8) Date: Wed, 18 Apr 2018 13:47:22 +0100 Subject: Bug Report: ConstantPool vs constantPool Message-ID: Hi All, Windows is having trouble with the constantPool and ConstantPool directories because they use the same name. Parent folder: test/hotspot/jtreg/runtime Is it a trivial matter to solve this? E.g. get the files from one copied into the other, followed by the deletion of the redundant constantPool? Best Regards Adam Farley Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From jonathan.gibbons at oracle.com Mon Apr 23 23:46:49 2018 From: jonathan.gibbons at oracle.com (jonathan.gibbons at oracle.com) Date: Mon, 23 Apr 2018 23:46:49 +0000 Subject: hg: code-tools/jtreg: 2 new changesets Message-ID: <201804232346.w3NNknpZ019782@aojmv0008.oracle.com> Changeset: 33f1ab385957 Author: jjg Date: 2018-04-23 16:40 -0700 URL: http://hg.openjdk.java.net/code-tools/jtreg/rev/33f1ab385957 update spec version ! src/share/doc/javatest/regtest/tag-spec.html Changeset: 7b1496d2790e Author: jjg Date: 2018-04-23 16:46 -0700 URL: http://hg.openjdk.java.net/code-tools/jtreg/rev/7b1496d2790e update to use JT Harness 6.0 b01 ! make/Defs.gmk ! make/build-all.sh From jonathan.gibbons at oracle.com Tue Apr 24 15:13:45 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Tue, 24 Apr 2018 08:13:45 -0700 Subject: Bug Report: ConstantPool vs constantPool In-Reply-To: References: Message-ID: <796afbd5-1832-0493-b4c5-0da8c12194ef@oracle.com> Adam, While it would be good for jtreg to detect this,? the real fix is that the directories need to be renamed, which is a matter for the hotspot team. (cc:-ed) -- Jon On 4/18/18 5:47 AM, Adam Farley8 wrote: > Hi All, > > Windows is having trouble with the constantPool and ConstantPool > directories because they use the same name. > > Parent folder: test/hotspot/jtreg/runtime > > Is it a trivial matter to solve this? > > E.g. get the files from one copied into the other, followed by the > deletion of the redundant constantPool? > > Best Regards > > Adam Farley > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU