From david.holmes at oracle.com Thu May 1 05:45:49 2014 From: david.holmes at oracle.com (David Holmes) Date: Thu, 01 May 2014 15:45:49 +1000 Subject: Proposal: jtreg tests with native components In-Reply-To: <6D79B1FC-EEDA-497B-B622-D791D89E0DD3@oracle.com> References: <5360C4EB.80201@oracle.com> <6D79B1FC-EEDA-497B-B622-D791D89E0DD3@oracle.com> Message-ID: <5361DF8D.5010800@oracle.com> On 30/04/2014 9:39 PM, Staffan Larsen wrote: > > On 30 apr 2014, at 11:39, David Holmes wrote: > >> Hi Staffan, >> >> On 25/04/2014 10:02 PM, Staffan Larsen wrote: >>> There are a couple of jtreg tests today that depend on native components (either JNI libraries or executables). These are handled in one of two ways: >>> >>> 1) The binaries are pre-compiled and checked into the repository (often inside jar files). >>> 2) The test will try to invoke a compiler (gcc, cl, ?) when the test is being run. >>> >>> Neither of these are very good solutions. #1 makes it hard to run the setup the test for all platforms and requires binaries in the source control system. #2 is hit-and-miss: the correct compiler may or may not be installed on the test machine, and the approach requires platform specific logic to be maintained. >> >> #2 is far from perfect but ... >> >>> I would like to propose that these native components are instead compiled when the product is built by the same makefile logic as the product. At product build time we know we have access to the (correct) compilers and we have excellent support in the makefiles for building on all platforms. >>> >>> If we build the native test components together with the product, we also have to take care of distributing the result together with the product when we do testing across a larger number of machines. We will also need a way to tell the jtreg tests where these pre-built binaries are located. >> >> don't under estimate the complexity involved in building then "distributing" the test binaries. > > I don?t. It will be complicated, but I?m sure we can do it. The question is whether it is worth it relative to the size of the problem. >> >> You will still need to maintain platform specific logic as you won't necessarily be able to use the CFLAGS etc that the main build process uses. > > Can you explain more? Why can?t I use CFLAGS as it is? You _may_ be able to, you may not. I know we already had issues where the CFLAGS as being used for the JDK sources also got applied to building the code-generator utility programs and that didn't work correctly. Here's sample CFLAGS from a JDK build CFLAGS_JDKLIB:= -W -Wall -Wno-unused -Wno-parentheses -pipe -D_GNU_SOURCE -D_REENTRANT -D_LARGEFILE64_SOURCE -fno-omit-frame-pointer -D_LITTLE_ENDIAN -DLINUX -DNDEBUG -DARCH='"i586"' -Di586 -DRELEASE='"$(RELEASE)"' -I/export/users/dh198349/ejdk8u-dev/build/b13/linux-i586-ea/jdk/include -I/export/users/dh198349/ejdk8u-dev/build/b13/linux-i586-ea/jdk/include/linux -I/export/users/dh198349/jdk8u-dev/jdk/src/share/javavm/export -I/export/users/dh198349/jdk8u-dev/jdk/src/solaris/javavm/export -I/export/users/dh198349/jdk8u-dev/jdk/src/share/native/common -I/export/users/dh198349/jdk8u-dev/jdk/src/solaris/native/common -m32 -fno-strict-aliasing -fPIC Does that make sense for compiling a test? Does it depend on whether we are building a native library or a native executable? >> >> Also talk to SQE as I'm pretty sure there is an existing project to look at how to better handle this, at least for the internal test suites. > > I have talked to SQE. I don?t know of any other projects to handle this. :) It wasn't SQE, it was your project as referenced in a few bug reports last August/September. David > /Staffan > > >> >> David >> ----- >> >>> I suggest that at the end of a distributed build run, the pre-built test binaries are packaged in a zip or tar file (just like the product bits) and stored next to the product bundles. When we run distributed tests, we need to pick up the product bundle and the test bundle before the testing is started. >>> >>> To tell the tests where the native code is, I would like to add a flag to jtreg to point out the path to the binaries. This should cause jtreg to set java.library.path before invoking a test and also set a test.* property which can be used by test to find it?s native components. >>> >>> This kind of setup would make it easier to add and maintain tests that have a native component. I think this will be especially important as more tests are written using jtreg in the hotspot repository. >>> >>> Thoughts on this? Is the general approach ok? There are lots of details to be figured out, but at this stage I would like to hear feedback on the idea as such. >>> >>> Thanks, >>> /Staffan >>> > From dmitry.samersoff at oracle.com Thu May 1 16:12:30 2014 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Thu, 01 May 2014 20:12:30 +0400 Subject: Proposal: jtreg tests with native components In-Reply-To: References: Message-ID: <5362726E.9030102@oracle.com> Staffan, Couple of tests require both Java and native code, so compiling it's all into the single directory might be a problem. Some test used to access additional data in TESTSRC so we have to copy it as well. One of possible solution is to allow the tests to support two independent steps - "build" and "run". on build side - run all tests with "build" parameter, tests that don't have build step will just do nothing and continue work as today. Then bundle entire test tree. on test side - run all tests with the "run" parameters. Test that have build step have to copy all required files from build output directory to work directory. JDK configuration options could be provided through include file for make/shell or property file. It allows the test to achieve whatever build and test behavior it requires and allows us to move to right direction, without breaking existing things. -Dmitry On 2014-04-25 16:02, Staffan Larsen wrote: > There are a couple of jtreg tests today that depend on native components (either JNI libraries or executables). These are handled in one of two ways: > > 1) The binaries are pre-compiled and checked into the repository (often inside jar files). > 2) The test will try to invoke a compiler (gcc, cl, ?) when the test is being run. > > Neither of these are very good solutions. #1 makes it hard to run the setup the test for all platforms and requires binaries in the source control system. #2 is hit-and-miss: the correct compiler may or may not be installed on the test machine, and the approach requires platform specific logic to be maintained. > > I would like to propose that these native components are instead compiled when the product is built by the same makefile logic as the product. At product build time we know we have access to the (correct) compilers and we have excellent support in the makefiles for building on all platforms. > > If we build the native test components together with the product, we also have to take care of distributing the result together with the product when we do testing across a larger number of machines. We will also need a way to tell the jtreg tests where these pre-built binaries are located. > > I suggest that at the end of a distributed build run, the pre-built test binaries are packaged in a zip or tar file (just like the product bits) and stored next to the product bundles. When we run distributed tests, we need to pick up the product bundle and the test bundle before the testing is started. > > To tell the tests where the native code is, I would like to add a flag to jtreg to point out the path to the binaries. This should cause jtreg to set java.library.path before invoking a test and also set a test.* property which can be used by test to find it?s native components. > > This kind of setup would make it easier to add and maintain tests that have a native component. I think this will be especially important as more tests are written using jtreg in the hotspot repository. > > Thoughts on this? Is the general approach ok? There are lots of details to be figured out, but at this stage I would like to hear feedback on the idea as such. > > Thanks, > /Staffan > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From jonathan.gibbons at oracle.com Thu May 1 16:24:14 2014 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 01 May 2014 09:24:14 -0700 Subject: Proposal: jtreg tests with native components In-Reply-To: <5362726E.9030102@oracle.com> References: <5362726E.9030102@oracle.com> Message-ID: <5362752E.204@oracle.com> Dmitry, Beyond Staffan's proposal for jtreg to provide tests with a pointer to the location of precompiled native code, there are no plans to change the way jtreg execute tests, processes action tags like @build, @run, etc. -- Jon On 05/01/2014 09:12 AM, Dmitry Samersoff wrote: > Staffan, > > Couple of tests require both Java and native code, so compiling it's all > into the single directory might be a problem. Some test used to access > additional data in TESTSRC so we have to copy it as well. > > One of possible solution is to allow the tests to support two > independent steps - "build" and "run". > > on build side - run all tests with "build" parameter, tests that don't > have build step will just do nothing and continue work as today. Then > bundle entire test tree. > > on test side - run all tests with the "run" parameters. Test that have > build step have to copy all required files from build output directory > to work directory. > > JDK configuration options could be provided through include file for > make/shell or property file. > > It allows the test to achieve whatever build and test behavior it > requires and allows us to move to right direction, without breaking > existing things. > > -Dmitry > > On 2014-04-25 16:02, Staffan Larsen wrote: >> There are a couple of jtreg tests today that depend on native components (either JNI libraries or executables). These are handled in one of two ways: >> >> 1) The binaries are pre-compiled and checked into the repository (often inside jar files). >> 2) The test will try to invoke a compiler (gcc, cl, ?) when the test is being run. >> >> Neither of these are very good solutions. #1 makes it hard to run the setup the test for all platforms and requires binaries in the source control system. #2 is hit-and-miss: the correct compiler may or may not be installed on the test machine, and the approach requires platform specific logic to be maintained. >> >> I would like to propose that these native components are instead compiled when the product is built by the same makefile logic as the product. At product build time we know we have access to the (correct) compilers and we have excellent support in the makefiles for building on all platforms. >> >> If we build the native test components together with the product, we also have to take care of distributing the result together with the product when we do testing across a larger number of machines. We will also need a way to tell the jtreg tests where these pre-built binaries are located. >> >> I suggest that at the end of a distributed build run, the pre-built test binaries are packaged in a zip or tar file (just like the product bits) and stored next to the product bundles. When we run distributed tests, we need to pick up the product bundle and the test bundle before the testing is started. >> >> To tell the tests where the native code is, I would like to add a flag to jtreg to point out the path to the binaries. This should cause jtreg to set java.library.path before invoking a test and also set a test.* property which can be used by test to find it?s native components. >> >> This kind of setup would make it easier to add and maintain tests that have a native component. I think this will be especially important as more tests are written using jtreg in the hotspot repository. >> >> Thoughts on this? Is the general approach ok? There are lots of details to be figured out, but at this stage I would like to hear feedback on the idea as such. >> >> Thanks, >> /Staffan >> > From staffan.larsen at oracle.com Fri May 2 06:53:26 2014 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Fri, 2 May 2014 08:53:26 +0200 Subject: Proposal: jtreg tests with native components In-Reply-To: <5361DF8D.5010800@oracle.com> References: <5360C4EB.80201@oracle.com> <6D79B1FC-EEDA-497B-B622-D791D89E0DD3@oracle.com> <5361DF8D.5010800@oracle.com> Message-ID: <69D2F327-937E-4C4F-B8C7-8D75059D89E6@oracle.com> On 1 maj 2014, at 07:45, David Holmes wrote: > On 30/04/2014 9:39 PM, Staffan Larsen wrote: >> >> On 30 apr 2014, at 11:39, David Holmes wrote: >> >>> Hi Staffan, >>> >>> On 25/04/2014 10:02 PM, Staffan Larsen wrote: >>>> There are a couple of jtreg tests today that depend on native components (either JNI libraries or executables). These are handled in one of two ways: >>>> >>>> 1) The binaries are pre-compiled and checked into the repository (often inside jar files). >>>> 2) The test will try to invoke a compiler (gcc, cl, ?) when the test is being run. >>>> >>>> Neither of these are very good solutions. #1 makes it hard to run the setup the test for all platforms and requires binaries in the source control system. #2 is hit-and-miss: the correct compiler may or may not be installed on the test machine, and the approach requires platform specific logic to be maintained. >>> >>> #2 is far from perfect but ... >>> >>>> I would like to propose that these native components are instead compiled when the product is built by the same makefile logic as the product. At product build time we know we have access to the (correct) compilers and we have excellent support in the makefiles for building on all platforms. >>>> >>>> If we build the native test components together with the product, we also have to take care of distributing the result together with the product when we do testing across a larger number of machines. We will also need a way to tell the jtreg tests where these pre-built binaries are located. >>> >>> don't under estimate the complexity involved in building then "distributing" the test binaries. >> >> I don?t. It will be complicated, but I?m sure we can do it. > > The question is whether it is worth it relative to the size of the problem. I think we will see a large influx of these kinds of tests, especially in the hotspot repo. > >>> >>> You will still need to maintain platform specific logic as you won't necessarily be able to use the CFLAGS etc that the main build process uses. >> >> Can you explain more? Why can?t I use CFLAGS as it is? > > You _may_ be able to, you may not. I know we already had issues where the CFLAGS as being used for the JDK sources also got applied to building the code-generator utility programs and that didn't work correctly. Here's sample CFLAGS from a JDK build > > CFLAGS_JDKLIB:= -W -Wall -Wno-unused -Wno-parentheses -pipe -D_GNU_SOURCE -D_REENTRANT -D_LARGEFILE64_SOURCE -fno-omit-frame-pointer -D_LITTLE_ENDIAN -DLINUX -DNDEBUG -DARCH='"i586"' -Di586 -DRELEASE='"$(RELEASE)"' -I/export/users/dh198349/ejdk8u-dev/build/b13/linux-i586-ea/jdk/include -I/export/users/dh198349/ejdk8u-dev/build/b13/linux-i586-ea/jdk/include/linux -I/export/users/dh198349/jdk8u-dev/jdk/src/share/javavm/export -I/export/users/dh198349/jdk8u-dev/jdk/src/solaris/javavm/export -I/export/users/dh198349/jdk8u-dev/jdk/src/share/native/common -I/export/users/dh198349/jdk8u-dev/jdk/src/solaris/native/common -m32 -fno-strict-aliasing -fPIC > > Does that make sense for compiling a test? Does it depend on whether we are building a native library or a native executable? I think they make sense, at least initially. If not, we can tune them, but that "tuning? will be in one central location and not spread out in a lot of shell scripts. I also plan to allow individual tests to override the flags if necessary (for example to link with X11). For executables there is CFLAGS_JDKEXE. Thanks, /Staffan > >>> >>> Also talk to SQE as I'm pretty sure there is an existing project to look at how to better handle this, at least for the internal test suites. >> >> I have talked to SQE. I don?t know of any other projects to handle this. > > :) It wasn't SQE, it was your project as referenced in a few bug reports last August/September. > > David > > >> /Staffan >> >> >>> >>> David >>> ----- >>> >>>> I suggest that at the end of a distributed build run, the pre-built test binaries are packaged in a zip or tar file (just like the product bits) and stored next to the product bundles. When we run distributed tests, we need to pick up the product bundle and the test bundle before the testing is started. >>>> >>>> To tell the tests where the native code is, I would like to add a flag to jtreg to point out the path to the binaries. This should cause jtreg to set java.library.path before invoking a test and also set a test.* property which can be used by test to find it?s native components. >>>> >>>> This kind of setup would make it easier to add and maintain tests that have a native component. I think this will be especially important as more tests are written using jtreg in the hotspot repository. >>>> >>>> Thoughts on this? Is the general approach ok? There are lots of details to be figured out, but at this stage I would like to hear feedback on the idea as such. >>>> >>>> Thanks, >>>> /Staffan >>>> >> From jonathan.gibbons at oracle.com Tue May 27 23:28:26 2014 From: jonathan.gibbons at oracle.com (jonathan.gibbons at oracle.com) Date: Tue, 27 May 2014 23:28:26 +0000 Subject: hg: code-tools/jtreg: Added tag jtreg4.1-b09 for changeset c54815dea64c Message-ID: <201405272328.s4RNSQ8H003327@aojmv0008> Changeset: 80d20010e8eb Author: jjg Date: 2014-05-27 16:27 -0700 URL: http://hg.openjdk.java.net/code-tools/jtreg/rev/80d20010e8eb Added tag jtreg4.1-b09 for changeset c54815dea64c ! .hgtags From jonathan.gibbons at oracle.com Wed May 28 22:37:46 2014 From: jonathan.gibbons at oracle.com (jonathan.gibbons at oracle.com) Date: Wed, 28 May 2014 22:37:46 +0000 Subject: hg: code-tools/jtreg: 7900892: Add -nativepath support Message-ID: <201405282237.s4SMbkf5008013@aojmv0008> Changeset: 65366b9b757b Author: sla Date: 2014-05-28 15:32 -0700 URL: http://hg.openjdk.java.net/code-tools/jtreg/rev/65366b9b757b 7900892: Add -nativepath support Reviewed-by: jjg + make/tests/TestNativePath.gmk ! src/share/classes/com/sun/javatest/regtest/Action.java ! src/share/classes/com/sun/javatest/regtest/Main.java ! src/share/classes/com/sun/javatest/regtest/MainAction.java ! src/share/classes/com/sun/javatest/regtest/RegressionParameters.java ! src/share/classes/com/sun/javatest/regtest/RegressionScript.java ! src/share/classes/com/sun/javatest/regtest/RegressionTestFinder.java ! src/share/classes/com/sun/javatest/regtest/i18n.properties + src/share/test/javatest/regtest/data/nativepath/NativesEmpty.java + src/share/test/javatest/regtest/data/nativepath/NativesOK.java + src/share/test/javatest/regtest/data/nativepath/TEST.ROOT From jonathan.gibbons at oracle.com Thu May 29 01:44:05 2014 From: jonathan.gibbons at oracle.com (jonathan.gibbons at oracle.com) Date: Thu, 29 May 2014 01:44:05 +0000 Subject: hg: code-tools/jtreg: 7900895: Problem compiling lots of tests together on Windows Message-ID: <201405290144.s4T1i5bT005498@aojmv0008> Changeset: 4f7c0100451e Author: jjg Date: 2014-05-28 18:43 -0700 URL: http://hg.openjdk.java.net/code-tools/jtreg/rev/4f7c0100451e 7900895: Problem compiling lots of tests together on Windows + make/tests/CompileArgFileTest.gmk ! src/share/classes/com/sun/javatest/regtest/Action.java ! src/share/classes/com/sun/javatest/regtest/BuildAction.java ! src/share/classes/com/sun/javatest/regtest/CompileAction.java ! src/share/classes/com/sun/javatest/regtest/MainAction.java ! src/share/classes/com/sun/javatest/regtest/RegressionScript.java + src/share/test/javatest/regtest/GenCompileTests.java From jonathan.gibbons at oracle.com Thu May 29 23:50:03 2014 From: jonathan.gibbons at oracle.com (jonathan.gibbons at oracle.com) Date: Thu, 29 May 2014 23:50:03 +0000 Subject: hg: code-tools/jtreg: 7900935: remove bad .PHONY tag Message-ID: <201405292350.s4TNo3hi013874@aojmv0008> Changeset: b4ed94c36892 Author: jjg Date: 2014-05-29 16:49 -0700 URL: http://hg.openjdk.java.net/code-tools/jtreg/rev/b4ed94c36892 7900935: remove bad .PHONY tag ! make/tests/TestNativePath.gmk From jonathan.gibbons at oracle.com Thu May 29 23:52:29 2014 From: jonathan.gibbons at oracle.com (jonathan.gibbons at oracle.com) Date: Thu, 29 May 2014 23:52:29 +0000 Subject: hg: code-tools/jtreg: 7900913: new @requires tag Message-ID: <201405292352.s4TNqT4t014372@aojmv0008> Changeset: 31003a1c46d9 Author: jjg Date: 2014-05-29 16:51 -0700 URL: http://hg.openjdk.java.net/code-tools/jtreg/rev/31003a1c46d9 7900913: new @requires tag + make/tests/RequiresTest.gmk ! src/share/classes/com/sun/javatest/regtest/CachingTestFilter.java + src/share/classes/com/sun/javatest/regtest/Expr.java ! src/share/classes/com/sun/javatest/regtest/OS.java + src/share/classes/com/sun/javatest/regtest/RegressionContext.java ! src/share/classes/com/sun/javatest/regtest/RegressionParameters.java ! src/share/classes/com/sun/javatest/regtest/RegressionScript.java ! src/share/classes/com/sun/javatest/regtest/RegressionTestFinder.java ! src/share/classes/com/sun/javatest/regtest/RegressionTestSuite.java ! src/share/classes/com/sun/javatest/regtest/TestManager.java + src/share/test/javatest/regtest/data/requires/EmptyTest.java + src/share/test/javatest/regtest/data/requires/EvalTest.java + src/share/test/javatest/regtest/data/requires/ExprTest.java + src/share/test/javatest/regtest/data/requires/FlavorTest.java + src/share/test/javatest/regtest/data/requires/MultipleTest.java + src/share/test/javatest/regtest/data/requires/SyntaxTest.java + src/share/test/javatest/regtest/data/requires/TEST.ROOT From erik.joelsson at oracle.com Mon May 5 09:23:40 2014 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 05 May 2014 09:23:40 -0000 Subject: Proposal: jtreg tests with native components In-Reply-To: <69D2F327-937E-4C4F-B8C7-8D75059D89E6@oracle.com> References: <5360C4EB.80201@oracle.com> <6D79B1FC-EEDA-497B-B622-D791D89E0DD3@oracle.com> <5361DF8D.5010800@oracle.com> <69D2F327-937E-4C4F-B8C7-8D75059D89E6@oracle.com> Message-ID: <53675897.7040205@oracle.com> On 2014-05-02 08:53, Staffan Larsen wrote: > On 1 maj 2014, at 07:45, David Holmes wrote: > >> On 30/04/2014 9:39 PM, Staffan Larsen wrote: >> The question is whether it is worth it relative to the size of the >> problem. > I think we will see a large influx of these kinds of tests, especially in the hotspot repo. I don't doubt it will be worth it. >>>> You will still need to maintain platform specific logic as you won't necessarily be able to use the CFLAGS etc that the main build process uses. >>> Can you explain more? Why can?t I use CFLAGS as it is? >> You _may_ be able to, you may not. I know we already had issues where the CFLAGS as being used for the JDK sources also got applied to building the code-generator utility programs and that didn't work correctly. Here's sample CFLAGS from a JDK build >> >> CFLAGS_JDKLIB:= -W -Wall -Wno-unused -Wno-parentheses -pipe -D_GNU_SOURCE -D_REENTRANT -D_LARGEFILE64_SOURCE -fno-omit-frame-pointer -D_LITTLE_ENDIAN -DLINUX -DNDEBUG -DARCH='"i586"' -Di586 -DRELEASE='"$(RELEASE)"' -I/export/users/dh198349/ejdk8u-dev/build/b13/linux-i586-ea/jdk/include -I/export/users/dh198349/ejdk8u-dev/build/b13/linux-i586-ea/jdk/include/linux -I/export/users/dh198349/jdk8u-dev/jdk/src/share/javavm/export -I/export/users/dh198349/jdk8u-dev/jdk/src/solaris/javavm/export -I/export/users/dh198349/jdk8u-dev/jdk/src/share/native/common -I/export/users/dh198349/jdk8u-dev/jdk/src/solaris/native/common -m32 -fno-strict-aliasing -fPIC >> >> Does that make sense for compiling a test? Does it depend on whether we are building a native library or a native executable? > I think they make sense, at least initially. If not, we can tune them, but that "tuning? will be in one central location and not spread out in a lot of shell scripts. I also plan to allow individual tests to override the flags if necessary (for example to link with X11). > > For executables there is CFLAGS_JDKEXE. For the initial proof of concept, just using the existing CFLAGS_JDKLIB and CFLAGS_JDKEXE will be fine. Going forward, I would like to see CFLAGS_TESTLIB and CFLAGS_TESTEXE being provided by configure. There will certainly be a lot of common things for both product and test libs and as pointed out above, some differences too. We should leverage the logic already present in our configure script to provide suitable defaults. /Erik > Thanks, > /Staffan > >>>> Also talk to SQE as I'm pretty sure there is an existing project to look at how to better handle this, at least for the internal test suites. >>> I have talked to SQE. I don?t know of any other projects to handle this. >> :) It wasn't SQE, it was your project as referenced in a few bug reports last August/September. >> >> David >> >> >>> /Staffan >>> >>> >>>> David >>>> ----- >>>> >>>>> I suggest that at the end of a distributed build run, the pre-built test binaries are packaged in a zip or tar file (just like the product bits) and stored next to the product bundles. When we run distributed tests, we need to pick up the product bundle and the test bundle before the testing is started. >>>>> >>>>> To tell the tests where the native code is, I would like to add a flag to jtreg to point out the path to the binaries. This should cause jtreg to set java.library.path before invoking a test and also set a test.* property which can be used by test to find it?s native components. >>>>> >>>>> This kind of setup would make it easier to add and maintain tests that have a native component. I think this will be especially important as more tests are written using jtreg in the hotspot repository. >>>>> >>>>> Thoughts on this? Is the general approach ok? There are lots of details to be figured out, but at this stage I would like to hear feedback on the idea as such. >>>>> >>>>> Thanks, >>>>> /Staffan >>>>>