From volker.simonis at gmail.com Fri Dec 1 16:29:44 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 1 Dec 2017 17:29:44 +0100 Subject: RFR(XS): CODETOOLS-7902075: jtreg doesn't build if jh.jar doesn't contain signatures Message-ID: Hi, can I please have a sponsor for the following simple fix of the jtreg build: http://cr.openjdk.java.net/~simonis/webrevs/2017/CODETOOLS-7902075/ https://bugs.openjdk.java.net/browse/CODETOOLS-7902075 Change "7901920: jh.jar is signed with a weak algorithm and its certificate is expired" introduced a logic to remove outdated signatures from jh.jar. However, there's no default location for downloading jh.jar and probably most available versions (e.g. the one from Maven: http://central.maven.org/maven2/javax/help/javahelp/2.0.05/javahelp-2.0.05.jar) if not all of them don't contain signatures at all. This breaks the build because removing the signatures from the jar file with zip will fail with the following error: zip warning: name not matched: META-INF/*.SF zip warning: name not matched: META-INF/*.RSA zip warning: name not matched: META-INF/*.DSA zip error: Nothing to do! (/net/usr.work/openjdk/tools/jtreg/jtreg/build/images/jtreg/lib/jh.jar It would be great if the sponsor could also tag this fix with jtreg4.2-b11 after pushing it because the last tagged version (i.e. jtreg4.2-b10) doesn't build out of the box without JCov (which should be optional and which was fixed by "7902072: Add support for imported license files"). Thank you and best regards, Volker From jonathan.gibbons at oracle.com Fri Dec 1 18:52:58 2017 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Fri, 01 Dec 2017 10:52:58 -0800 Subject: RFR(XS): CODETOOLS-7902075: jtreg doesn't build if jh.jar doesn't contain signatures In-Reply-To: References: Message-ID: <5A21A50A.7010509@oracle.com> Volker, I can push the fix for you, but it may already be redundant. If you use JT Harness 5, then you don't need JavaHelp at all. Just leave JAVAHELP_JAR unset. I've been pushing changes to jtreg this week to make it easier to build jtreg, and although I haven't yet flipped the switch on making the default be to use JT Harness 5, I have verified that you can build and run jtreg, using JT Harness 5, without JavaHelp and without JCov. -- Jon On 12/01/2017 08:29 AM, Volker Simonis wrote: > Hi, > > can I please have a sponsor for the following simple fix of the jtreg build: > > http://cr.openjdk.java.net/~simonis/webrevs/2017/CODETOOLS-7902075/ > https://bugs.openjdk.java.net/browse/CODETOOLS-7902075 > > Change "7901920: jh.jar is signed with a weak algorithm and its > certificate is expired" introduced a logic to remove outdated > signatures from jh.jar. > > However, there's no default location for downloading jh.jar and > probably most available versions (e.g. the one from Maven: > http://central.maven.org/maven2/javax/help/javahelp/2.0.05/javahelp-2.0.05.jar) > if not all of them don't contain signatures at all. This breaks the > build because removing the signatures from the jar file with zip will > fail with the following error: > > zip warning: name not matched: META-INF/*.SF > zip warning: name not matched: META-INF/*.RSA > zip warning: name not matched: META-INF/*.DSA > zip error: Nothing to do! > (/net/usr.work/openjdk/tools/jtreg/jtreg/build/images/jtreg/lib/jh.jar > > It would be great if the sponsor could also tag this fix with > jtreg4.2-b11 after pushing it because the last tagged version (i.e. > jtreg4.2-b10) doesn't build out of the box without JCov (which should > be optional and which was fixed by "7902072: Add support for imported > license files"). > > Thank you and best regards, > Volker From sadhak001 at gmail.com Sun Dec 3 17:02:46 2017 From: sadhak001 at gmail.com (Mani Sarkar) Date: Sun, 03 Dec 2017 17:02:46 +0000 Subject: code-tools-dev Digest, Vol 32, Issue 1 In-Reply-To: References: Message-ID: Thanks Jon for the jtreg enhancements, look forward to it's effect on the adoptopenjdk code tools build farm. On Sat, 2 Dec 2017 11:56 , wrote: > Send code-tools-dev mailing list submissions to > code-tools-dev at openjdk.java.net > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.openjdk.java.net/mailman/listinfo/code-tools-dev > or, via email, send a message with subject or body 'help' to > code-tools-dev-request at openjdk.java.net > > You can reach the person managing the list at > code-tools-dev-owner at openjdk.java.net > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of code-tools-dev digest..." > > > Today's Topics: > > 1. RFR(XS): CODETOOLS-7902075: jtreg doesn't build if jh.jar > doesn't contain signatures (Volker Simonis) > 2. Re: RFR(XS): CODETOOLS-7902075: jtreg doesn't build if jh.jar > doesn't contain signatures (Jonathan Gibbons) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 1 Dec 2017 17:29:44 +0100 > From: Volker Simonis > To: code-tools-dev > Subject: RFR(XS): CODETOOLS-7902075: jtreg doesn't build if jh.jar > doesn't contain signatures > Message-ID: > < > CA+3eh12X020bnrXe0Y0ngFnPqZC6xAwZSn4B_BiOtjSqGZqsWw at mail.gmail.com> > Content-Type: text/plain; charset="UTF-8" > > Hi, > > can I please have a sponsor for the following simple fix of the jtreg > build: > > http://cr.openjdk.java.net/~simonis/webrevs/2017/CODETOOLS-7902075/ > https://bugs.openjdk.java.net/browse/CODETOOLS-7902075 > > Change "7901920: jh.jar is signed with a weak algorithm and its > certificate is expired" introduced a logic to remove outdated > signatures from jh.jar. > > However, there's no default location for downloading jh.jar and > probably most available versions (e.g. the one from Maven: > > http://central.maven.org/maven2/javax/help/javahelp/2.0.05/javahelp-2.0.05.jar > ) > if not all of them don't contain signatures at all. This breaks the > build because removing the signatures from the jar file with zip will > fail with the following error: > > zip warning: name not matched: META-INF/*.SF > zip warning: name not matched: META-INF/*.RSA > zip warning: name not matched: META-INF/*.DSA > zip error: Nothing to do! > (/net/usr.work/openjdk/tools/jtreg/jtreg/build/images/jtreg/lib/jh.jar > > It would be great if the sponsor could also tag this fix with > jtreg4.2-b11 after pushing it because the last tagged version (i.e. > jtreg4.2-b10) doesn't build out of the box without JCov (which should > be optional and which was fixed by "7902072: Add support for imported > license files"). > > Thank you and best regards, > Volker > > > ------------------------------ > > Message: 2 > Date: Fri, 01 Dec 2017 10:52:58 -0800 > From: Jonathan Gibbons > To: Volker Simonis , code-tools-dev > > Subject: Re: RFR(XS): CODETOOLS-7902075: jtreg doesn't build if jh.jar > doesn't contain signatures > Message-ID: <5A21A50A.7010509 at oracle.com> > Content-Type: text/plain; charset=utf-8; format=flowed > > Volker, > > I can push the fix for you, but it may already be redundant. > > If you use JT Harness 5, then you don't need JavaHelp at all. > Just leave JAVAHELP_JAR unset. > > I've been pushing changes to jtreg this week to make it easier > to build jtreg, and although I haven't yet flipped the switch on making > the default be to use JT Harness 5, I have verified that you can > build and run jtreg, using JT Harness 5, without JavaHelp and without > JCov. > > -- Jon > > On 12/01/2017 08:29 AM, Volker Simonis wrote: > > Hi, > > > > can I please have a sponsor for the following simple fix of the jtreg > build: > > > > http://cr.openjdk.java.net/~simonis/webrevs/2017/CODETOOLS-7902075/ > > https://bugs.openjdk.java.net/browse/CODETOOLS-7902075 > > > > Change "7901920: jh.jar is signed with a weak algorithm and its > > certificate is expired" introduced a logic to remove outdated > > signatures from jh.jar. > > > > However, there's no default location for downloading jh.jar and > > probably most available versions (e.g. the one from Maven: > > > http://central.maven.org/maven2/javax/help/javahelp/2.0.05/javahelp-2.0.05.jar > ) > > if not all of them don't contain signatures at all. This breaks the > > build because removing the signatures from the jar file with zip will > > fail with the following error: > > > > zip warning: name not matched: META-INF/*.SF > > zip warning: name not matched: META-INF/*.RSA > > zip warning: name not matched: META-INF/*.DSA > > zip error: Nothing to do! > > (/net/usr.work/openjdk/tools/jtreg/jtreg/build/images/jtreg/lib/jh.jar > > > > It would be great if the sponsor could also tag this fix with > > jtreg4.2-b11 after pushing it because the last tagged version (i.e. > > jtreg4.2-b10) doesn't build out of the box without JCov (which should > > be optional and which was fixed by "7902072: Add support for imported > > license files"). > > > > Thank you and best regards, > > Volker > > > > End of code-tools-dev Digest, Vol 32, Issue 1 > ********************************************* > -- @theNeomatrix369 | Blog | @adoptopenjdk | Dev communities Meet-a-Project - MutabilityDetector | Github | Slideshare | LinkedIn Come to Devoxx UK 2018: http://www.devoxx.co.uk/ Don't chase success, rather aim for "Excellence", and success will come chasing after you! From volker.simonis at gmail.com Mon Dec 4 09:38:53 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Mon, 4 Dec 2017 10:38:53 +0100 Subject: RFR(XS): CODETOOLS-7902075: jtreg doesn't build if jh.jar doesn't contain signatures In-Reply-To: <5A21A50A.7010509@oracle.com> References: <5A21A50A.7010509@oracle.com> Message-ID: Hi Jonathan, thanks a lot for the quick reaction. So what is currently the recommended version of JTHanress, 4 or 5 ? And what is Oracle using for running the JTreg tests (e.g. in the HotSpot per-integration testing formerly known as JPRT, now probably Mach5) ? We want to be as close as possible to the Oracle setup, but the JTreg build instructions page [1] still mention JTHarness 4.6. A site which mentions the default test setups (equivalent to the "Supported Build Platforms" [2] page) would be extremely useful. Thank you and best regards, Volker [1] http://openjdk.java.net/jtreg/build.html [2] https://wiki.openjdk.java.net/display/Build/Supported+Build+Platforms On Fri, Dec 1, 2017 at 7:52 PM, Jonathan Gibbons wrote: > Volker, > > I can push the fix for you, but it may already be redundant. > > If you use JT Harness 5, then you don't need JavaHelp at all. > Just leave JAVAHELP_JAR unset. > > I've been pushing changes to jtreg this week to make it easier > to build jtreg, and although I haven't yet flipped the switch on making > the default be to use JT Harness 5, I have verified that you can > build and run jtreg, using JT Harness 5, without JavaHelp and without > JCov. > > -- Jon > > > On 12/01/2017 08:29 AM, Volker Simonis wrote: >> >> Hi, >> >> can I please have a sponsor for the following simple fix of the jtreg >> build: >> >> http://cr.openjdk.java.net/~simonis/webrevs/2017/CODETOOLS-7902075/ >> https://bugs.openjdk.java.net/browse/CODETOOLS-7902075 >> >> Change "7901920: jh.jar is signed with a weak algorithm and its >> certificate is expired" introduced a logic to remove outdated >> signatures from jh.jar. >> >> However, there's no default location for downloading jh.jar and >> probably most available versions (e.g. the one from Maven: >> >> http://central.maven.org/maven2/javax/help/javahelp/2.0.05/javahelp-2.0.05.jar) >> if not all of them don't contain signatures at all. This breaks the >> build because removing the signatures from the jar file with zip will >> fail with the following error: >> >> zip warning: name not matched: META-INF/*.SF >> zip warning: name not matched: META-INF/*.RSA >> zip warning: name not matched: META-INF/*.DSA >> zip error: Nothing to do! >> (/net/usr.work/openjdk/tools/jtreg/jtreg/build/images/jtreg/lib/jh.jar >> >> It would be great if the sponsor could also tag this fix with >> jtreg4.2-b11 after pushing it because the last tagged version (i.e. >> jtreg4.2-b10) doesn't build out of the box without JCov (which should >> be optional and which was fixed by "7902072: Add support for imported >> license files"). >> >> Thank you and best regards, >> Volker > > From jonathan.gibbons at oracle.com Mon Dec 4 23:19:53 2017 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Mon, 04 Dec 2017 15:19:53 -0800 Subject: RFR(XS): CODETOOLS-7902075: jtreg doesn't build if jh.jar doesn't contain signatures In-Reply-To: References: <5A21A50A.7010509@oracle.com> Message-ID: <5A25D819.7090407@oracle.com> Hi Volker, We're in transition. I've done all the work to enable JT Harness 5 to be the default. Oracle is still using 4.6, but we're going through some internal administrative trivia to change that. I'm hoping that that will happen this week, and once it does, I'll then change the default and add an appropriate new tag. Internally within Oracle, most folk use a jtreg build of the most recently tagged version, although we do have some CI systems using jtreg tip, for the purpose of testing jtreg itself. My expectation is that once JT Harness 5 is the default, we will remove support for JT Harness 4.6, and in particular, remove the need/ability to include JavaHelp. But I don't expect that to happen until sometime next year. Once the default has been changed, I'll updated the jtreg build page [1]. -- Jon [1] http://openjdk.java.net/jtreg/build.html On 12/04/2017 01:38 AM, Volker Simonis wrote: > Hi Jonathan, > > thanks a lot for the quick reaction. > > So what is currently the recommended version of JTHanress, 4 or 5 ? > And what is Oracle using for running the JTreg tests (e.g. in the > HotSpot per-integration testing formerly known as JPRT, now probably > Mach5) ? > > We want to be as close as possible to the Oracle setup, but the JTreg > build instructions page [1] still mention JTHarness 4.6. A site which > mentions the default test setups (equivalent to the "Supported Build > Platforms" [2] page) would be extremely useful. > > Thank you and best regards, > Volker > > [1] http://openjdk.java.net/jtreg/build.html > [2] https://wiki.openjdk.java.net/display/Build/Supported+Build+Platforms > > On Fri, Dec 1, 2017 at 7:52 PM, Jonathan Gibbons > wrote: >> Volker, >> >> I can push the fix for you, but it may already be redundant. >> >> If you use JT Harness 5, then you don't need JavaHelp at all. >> Just leave JAVAHELP_JAR unset. >> >> I've been pushing changes to jtreg this week to make it easier >> to build jtreg, and although I haven't yet flipped the switch on making >> the default be to use JT Harness 5, I have verified that you can >> build and run jtreg, using JT Harness 5, without JavaHelp and without >> JCov. >> >> -- Jon >> >> >> On 12/01/2017 08:29 AM, Volker Simonis wrote: >>> Hi, >>> >>> can I please have a sponsor for the following simple fix of the jtreg >>> build: >>> >>> http://cr.openjdk.java.net/~simonis/webrevs/2017/CODETOOLS-7902075/ >>> https://bugs.openjdk.java.net/browse/CODETOOLS-7902075 >>> >>> Change "7901920: jh.jar is signed with a weak algorithm and its >>> certificate is expired" introduced a logic to remove outdated >>> signatures from jh.jar. >>> >>> However, there's no default location for downloading jh.jar and >>> probably most available versions (e.g. the one from Maven: >>> >>> http://central.maven.org/maven2/javax/help/javahelp/2.0.05/javahelp-2.0.05.jar) >>> if not all of them don't contain signatures at all. This breaks the >>> build because removing the signatures from the jar file with zip will >>> fail with the following error: >>> >>> zip warning: name not matched: META-INF/*.SF >>> zip warning: name not matched: META-INF/*.RSA >>> zip warning: name not matched: META-INF/*.DSA >>> zip error: Nothing to do! >>> (/net/usr.work/openjdk/tools/jtreg/jtreg/build/images/jtreg/lib/jh.jar >>> >>> It would be great if the sponsor could also tag this fix with >>> jtreg4.2-b11 after pushing it because the last tagged version (i.e. >>> jtreg4.2-b10) doesn't build out of the box without JCov (which should >>> be optional and which was fixed by "7902072: Add support for imported >>> license files"). >>> >>> Thank you and best regards, >>> Volker >> From volker.simonis at gmail.com Tue Dec 5 08:19:15 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 5 Dec 2017 09:19:15 +0100 Subject: RFR(XS): CODETOOLS-7902075: jtreg doesn't build if jh.jar doesn't contain signatures In-Reply-To: <5A25D819.7090407@oracle.com> References: <5A21A50A.7010509@oracle.com> <5A25D819.7090407@oracle.com> Message-ID: Hi Jon, thanks a lot for the detailed information. That all sound very reasonable, so we will start to use JT Harness 5 as well. Regards, Volker On Tue, Dec 5, 2017 at 12:19 AM, Jonathan Gibbons wrote: > Hi Volker, > > We're in transition. > > I've done all the work to enable JT Harness 5 to be the default. > > Oracle is still using 4.6, but we're going through some internal > administrative trivia to change that. I'm hoping that that will > happen this week, and once it does, I'll then change the default > and add an appropriate new tag. > > Internally within Oracle, most folk use a jtreg build of the > most recently tagged version, although we do have some CI > systems using jtreg tip, for the purpose of testing jtreg itself. > > My expectation is that once JT Harness 5 is the default, > we will remove support for JT Harness 4.6, and in particular, > remove the need/ability to include JavaHelp. But I don't expect > that to happen until sometime next year. > > Once the default has been changed, I'll updated the jtreg > build page [1]. > > -- Jon > > [1] http://openjdk.java.net/jtreg/build.html > > > > On 12/04/2017 01:38 AM, Volker Simonis wrote: >> >> Hi Jonathan, >> >> thanks a lot for the quick reaction. >> >> So what is currently the recommended version of JTHanress, 4 or 5 ? >> And what is Oracle using for running the JTreg tests (e.g. in the >> HotSpot per-integration testing formerly known as JPRT, now probably >> Mach5) ? >> >> We want to be as close as possible to the Oracle setup, but the JTreg >> build instructions page [1] still mention JTHarness 4.6. A site which >> mentions the default test setups (equivalent to the "Supported Build >> Platforms" [2] page) would be extremely useful. >> >> Thank you and best regards, >> Volker >> >> [1] http://openjdk.java.net/jtreg/build.html >> [2] https://wiki.openjdk.java.net/display/Build/Supported+Build+Platforms >> >> On Fri, Dec 1, 2017 at 7:52 PM, Jonathan Gibbons >> wrote: >>> >>> Volker, >>> >>> I can push the fix for you, but it may already be redundant. >>> >>> If you use JT Harness 5, then you don't need JavaHelp at all. >>> Just leave JAVAHELP_JAR unset. >>> >>> I've been pushing changes to jtreg this week to make it easier >>> to build jtreg, and although I haven't yet flipped the switch on making >>> the default be to use JT Harness 5, I have verified that you can >>> build and run jtreg, using JT Harness 5, without JavaHelp and without >>> JCov. >>> >>> -- Jon >>> >>> >>> On 12/01/2017 08:29 AM, Volker Simonis wrote: >>>> >>>> Hi, >>>> >>>> can I please have a sponsor for the following simple fix of the jtreg >>>> build: >>>> >>>> http://cr.openjdk.java.net/~simonis/webrevs/2017/CODETOOLS-7902075/ >>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7902075 >>>> >>>> Change "7901920: jh.jar is signed with a weak algorithm and its >>>> certificate is expired" introduced a logic to remove outdated >>>> signatures from jh.jar. >>>> >>>> However, there's no default location for downloading jh.jar and >>>> probably most available versions (e.g. the one from Maven: >>>> >>>> >>>> http://central.maven.org/maven2/javax/help/javahelp/2.0.05/javahelp-2.0.05.jar) >>>> if not all of them don't contain signatures at all. This breaks the >>>> build because removing the signatures from the jar file with zip will >>>> fail with the following error: >>>> >>>> zip warning: name not matched: META-INF/*.SF >>>> zip warning: name not matched: META-INF/*.RSA >>>> zip warning: name not matched: META-INF/*.DSA >>>> zip error: Nothing to do! >>>> (/net/usr.work/openjdk/tools/jtreg/jtreg/build/images/jtreg/lib/jh.jar >>>> >>>> It would be great if the sponsor could also tag this fix with >>>> jtreg4.2-b11 after pushing it because the last tagged version (i.e. >>>> jtreg4.2-b10) doesn't build out of the box without JCov (which should >>>> be optional and which was fixed by "7902072: Add support for imported >>>> license files"). >>>> >>>> Thank you and best regards, >>>> Volker >>> >>> > From jonathan.gibbons at oracle.com Thu Dec 14 00:32:31 2017 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Wed, 13 Dec 2017 16:32:31 -0800 Subject: CODETOOLS-7902083: Simplify building jtreg Message-ID: <5A31C69F.4090800@oracle.com> This is for folk who are interested in building jtreg from source. As some of you have (rightfully) commented over the past years, jtreg has not been an easy tool to build from source. And, as some of you may have noticed, there has been some amount of activity over the past weeks and months to address this issue. This work has been led by Erik Helin (thanks, Erik!) and we're now getting to the point where we can show what we have been working towards. The core of the work to build jtreg is still the Makefiles as before, although as was recently noted, we've been simplifying the specification of the dependencies. Separately, Erik has helped provide updates to the way that some of the Code Tools dependencies can be built. Building on all that work, we can now get to the next stage, to provide a script that will download binaries for some components (JUnit, TestNG) and will download and build source for other components (AsmTools, JCov, JTHarness), for which there are no official binaries. To run the script, you just need to have Ant and a suitable "java" on your path, and to specify the location of an install of JDK 1.8 as an argument to the script. wget is used to download files, which honors proxy settings for those that need to use them. The script is deliberately fairly simple, and suitable for use in a CI system. You can see a webrev for the script at http://cr.openjdk.java.net/~jjg/7902083/webrev.00/ Example of use: $ which ant /opt/ant/1.9.4/bin/ant $ which java /opt/jdk/1.8.0/bin/java $ sh make/build-all.sh /opt/jdk/1.8.0 ... build output ... $ ls build/images/jtreg bin COPYRIGHT doc legal lib LICENSE README release $ Once this settles down a bit, I'll update the public docs on the jtreg web pages. -- Jon From stuart.monteith at linaro.org Thu Dec 14 13:19:42 2017 From: stuart.monteith at linaro.org (Stuart Monteith) Date: Thu, 14 Dec 2017 13:19:42 +0000 Subject: CODETOOLS-7902083: Simplify building jtreg In-Reply-To: <5A31C69F.4090800@oracle.com> References: <5A31C69F.4090800@oracle.com> Message-ID: Thanks for this Jonathan, This will simplify my automation greatly. The script looks fine to me and tests fine on my Ubuntu 17.10 x86 system and Ubuntu 14.04 aarch64 system. BR, Stuart On 14 December 2017 at 00:32, Jonathan Gibbons wrote: > This is for folk who are interested in building jtreg from source. > > As some of you have (rightfully) commented over the past years, jtreg has > not been an easy tool to build from source. > > And, as some of you may have noticed, there has been some amount of activity > over the past weeks and months to address this issue. This work has been led > by Erik Helin (thanks, Erik!) and we're now getting to the point where we > can show what we have been working towards. > > The core of the work to build jtreg is still the Makefiles as before, > although as was recently noted, we've been simplifying the specification of > the dependencies. > > Separately, Erik has helped provide updates to the way that some of the Code > Tools dependencies can be built. > > Building on all that work, we can now get to the next stage, to provide a > script that will download binaries for some components (JUnit, TestNG) and > will download and build source for other components (AsmTools, JCov, > JTHarness), for which there are no official binaries. > > To run the script, you just need to have Ant and a suitable "java" on your > path, and to specify the location of an install of JDK 1.8 as an argument to > the script. wget is used to download files, which honors proxy settings for > those that need to use them. The script is deliberately fairly simple, and > suitable for use in a CI system. > > You can see a webrev for the script at > http://cr.openjdk.java.net/~jjg/7902083/webrev.00/ > > Example of use: > > $ which ant > /opt/ant/1.9.4/bin/ant > $ which java > /opt/jdk/1.8.0/bin/java > $ sh make/build-all.sh /opt/jdk/1.8.0 > ... build output ... > $ ls build/images/jtreg > bin COPYRIGHT doc legal lib LICENSE README release > $ > > > Once this settles down a bit, I'll update the public docs on the jtreg web > pages. > > -- Jon > > From jonathan.gibbons at oracle.com Thu Dec 14 17:03:15 2017 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 14 Dec 2017 09:03:15 -0800 Subject: CODETOOLS-7902083: Simplify building jtreg In-Reply-To: References: <5A31C69F.4090800@oracle.com> Message-ID: To be clear, the majority of the credit for this goes to Erik. :-) But thanks to you for checking out the script. I'll push it to the repo soon, and we can tweak/evolve it there as needed. -- Jon On 12/14/17 5:19 AM, Stuart Monteith wrote: > Thanks for this Jonathan, > This will simplify my automation greatly. The script looks fine to > me and tests fine on my Ubuntu 17.10 x86 system and Ubuntu 14.04 > aarch64 system. > > BR, > Stuart > > On 14 December 2017 at 00:32, Jonathan Gibbons > wrote: >> This is for folk who are interested in building jtreg from source. >> >> As some of you have (rightfully) commented over the past years, jtreg has >> not been an easy tool to build from source. >> >> And, as some of you may have noticed, there has been some amount of activity >> over the past weeks and months to address this issue. This work has been led >> by Erik Helin (thanks, Erik!) and we're now getting to the point where we >> can show what we have been working towards. >> >> The core of the work to build jtreg is still the Makefiles as before, >> although as was recently noted, we've been simplifying the specification of >> the dependencies. >> >> Separately, Erik has helped provide updates to the way that some of the Code >> Tools dependencies can be built. >> >> Building on all that work, we can now get to the next stage, to provide a >> script that will download binaries for some components (JUnit, TestNG) and >> will download and build source for other components (AsmTools, JCov, >> JTHarness), for which there are no official binaries. >> >> To run the script, you just need to have Ant and a suitable "java" on your >> path, and to specify the location of an install of JDK 1.8 as an argument to >> the script. wget is used to download files, which honors proxy settings for >> those that need to use them. The script is deliberately fairly simple, and >> suitable for use in a CI system. >> >> You can see a webrev for the script at >> http://cr.openjdk.java.net/~jjg/7902083/webrev.00/ >> >> Example of use: >> >> $ which ant >> /opt/ant/1.9.4/bin/ant >> $ which java >> /opt/jdk/1.8.0/bin/java >> $ sh make/build-all.sh /opt/jdk/1.8.0 >> ... build output ... >> $ ls build/images/jtreg >> bin COPYRIGHT doc legal lib LICENSE README release >> $ >> >> >> Once this settles down a bit, I'll update the public docs on the jtreg web >> pages. >> >> -- Jon >> >> From jonathan.gibbons at oracle.com Fri Dec 15 22:34:26 2017 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Fri, 15 Dec 2017 14:34:26 -0800 Subject: CODETOOLS-7902083: Simplify building jtreg In-Reply-To: <5A31C69F.4090800@oracle.com> References: <5A31C69F.4090800@oracle.com> Message-ID: <5A344DF2.7060103@oracle.com> I've pushed the changeset containing this new script. There are two changes since I posted the webrev: 1. Change to use "shasum -a 1" instead of "sha1sum" 2. Update script to download and use jcommander.jar in conjunction with TestNG. With the second change in particular, all the jtreg self-tests pass, when the image is built with the build-all.sh script. -- Jon On 12/13/2017 04:32 PM, Jonathan Gibbons wrote: > This is for folk who are interested in building jtreg from source. > > As some of you have (rightfully) commented over the past years, jtreg > has not been an easy tool to build from source. > > And, as some of you may have noticed, there has been some amount of > activity over the past weeks and months to address this issue. This > work has been led by Erik Helin (thanks, Erik!) and we're now getting > to the point where we can show what we have been working towards. > > The core of the work to build jtreg is still the Makefiles as before, > although as was recently noted, we've been simplifying the > specification of the dependencies. > > Separately, Erik has helped provide updates to the way that some of > the Code Tools dependencies can be built. > > Building on all that work, we can now get to the next stage, to > provide a script that will download binaries for some components > (JUnit, TestNG) and will download and build source for other > components (AsmTools, JCov, JTHarness), for which there are no > official binaries. > > To run the script, you just need to have Ant and a suitable "java" on > your path, and to specify the location of an install of JDK 1.8 as an > argument to the script. wget is used to download files, which honors > proxy settings for those that need to use them. The script is > deliberately fairly simple, and suitable for use in a CI system. > > You can see a webrev for the script at > http://cr.openjdk.java.net/~jjg/7902083/webrev.00/ > > Example of use: > > $ which ant > /opt/ant/1.9.4/bin/ant > $ which java > /opt/jdk/1.8.0/bin/java > $ sh make/build-all.sh /opt/jdk/1.8.0 > ... build output ... > $ ls build/images/jtreg > bin COPYRIGHT doc legal lib LICENSE README release > $ > > > Once this settles down a bit, I'll update the public docs on the jtreg > web pages. > > -- Jon > > From sadhak001 at gmail.com Sat Dec 16 02:32:02 2017 From: sadhak001 at gmail.com (Mani Sarkar) Date: Sat, 16 Dec 2017 02:32:02 +0000 Subject: CODETOOLS-7902083: Simplify building jtreg In-Reply-To: <5A344DF2.7060103@oracle.com> References: <5A31C69F.4090800@oracle.com> <5A344DF2.7060103@oracle.com> Message-ID: Great, thanks! Will incorporate it with our build scripts on the adopt openjdk build farm. On Fri, 15 Dec 2017 22:34 Jonathan Gibbons, wrote: > I've pushed the changeset containing this new script. > > There are two changes since I posted the webrev: > > 1. Change to use "shasum -a 1" instead of "sha1sum" > 2. Update script to download and use jcommander.jar in conjunction with > TestNG. > > With the second change in particular, all the jtreg self-tests pass, > when the image is built with the build-all.sh script. > > -- Jon > > On 12/13/2017 04:32 PM, Jonathan Gibbons wrote: > > This is for folk who are interested in building jtreg from source. > > > > As some of you have (rightfully) commented over the past years, jtreg > > has not been an easy tool to build from source. > > > > And, as some of you may have noticed, there has been some amount of > > activity over the past weeks and months to address this issue. This > > work has been led by Erik Helin (thanks, Erik!) and we're now getting > > to the point where we can show what we have been working towards. > > > > The core of the work to build jtreg is still the Makefiles as before, > > although as was recently noted, we've been simplifying the > > specification of the dependencies. > > > > Separately, Erik has helped provide updates to the way that some of > > the Code Tools dependencies can be built. > > > > Building on all that work, we can now get to the next stage, to > > provide a script that will download binaries for some components > > (JUnit, TestNG) and will download and build source for other > > components (AsmTools, JCov, JTHarness), for which there are no > > official binaries. > > > > To run the script, you just need to have Ant and a suitable "java" on > > your path, and to specify the location of an install of JDK 1.8 as an > > argument to the script. wget is used to download files, which honors > > proxy settings for those that need to use them. The script is > > deliberately fairly simple, and suitable for use in a CI system. > > > > You can see a webrev for the script at > > http://cr.openjdk.java.net/~jjg/7902083/webrev.00/ > > > > Example of use: > > > > $ which ant > > /opt/ant/1.9.4/bin/ant > > $ which java > > /opt/jdk/1.8.0/bin/java > > $ sh make/build-all.sh /opt/jdk/1.8.0 > > ... build output ... > > $ ls build/images/jtreg > > bin COPYRIGHT doc legal lib LICENSE README release > > $ > > > > > > Once this settles down a bit, I'll update the public docs on the jtreg > > web pages. > > > > -- Jon > > > > > > -- @theNeomatrix369 | Blog | @adoptopenjdk | Dev communities Meet-a-Project - MutabilityDetector | Github | Slideshare | LinkedIn Come to Devoxx UK 2018: http://www.devoxx.co.uk/ Don't chase success, rather aim for "Excellence", and success will come chasing after you! From martinrb at google.com Sat Dec 16 18:51:48 2017 From: martinrb at google.com (Martin Buchholz) Date: Sat, 16 Dec 2017 10:51:48 -0800 Subject: CODETOOLS-7902083: Simplify building jtreg In-Reply-To: References: <5A31C69F.4090800@oracle.com> <5A344DF2.7060103@oracle.com> Message-ID: Thanks for working on this. I've written similar scripts to gather up all dependencies and build. I suggest using wget's -q flag. I suggest creating little shell subroutines to check the checksums (which print the actual checksum in case of mismatch) and to download maven jars. Here's some random shell code I've written that could get reused: def_make_var() { typeset -n varref="$1" varref="$2" MAKE_FLAGS+=("$1=$2") } def_make_file() { [[ -f "$2" ]] || die "%s: no such file" "$2" def_make_var "$@" } def_make_dir() { [[ -d "$2" ]] || die "%s: no such directory" "$2" def_make_var "$@" } download_maven_jar() { local -r groupId="$1" artifactId="$2" version="$3" local -r jarfile="${artifactId}-${version}.jar" wget -qO"/tmp/$jarfile" \ " http://search.maven.org/remotecontent?filepath=${groupId}/${artifactId}/${version}/${jarfile} " def_make_file "${artifactId^^}_JAR" "/tmp/$jarfile" DOWNLOADED_MAVEN_JARS+=("/tmp/$jarfile") } On Sat, Dec 16, 2017 at 9:55 AM, Martin Buchholz wrote: > testng-6.9.5 was packaged poorly, with jdk8 classfiles. > testng-6.9.8 had different packaging problems. > testng-6.9.9 is golden, with jdk7 class files. > (of course, there are also more recent testng versions) > > --- a/make/build-all.sh > +++ b/make/build-all.sh > @@ -172,12 +172,12 @@ > TESTNG_DEPS_DIR=${JTREG_DEPS_DIR}/testng > mkdir ${TESTNG_DEPS_DIR} > > -TESTNG_JAR=${TESTNG_DEPS_DIR}/testng-6.9.5.jar > -wget ${MAVEN_REPO_URL}/org/testng/testng/6.9.5/testng-6.9.5.jar -O > ${TESTNG_JAR} > -printf "5d12ea207fc47c3f341a3f8ecc88a3eac396a777 ${TESTNG_JAR}" | > shasum -a 1 --check - > +TESTNG_JAR=${TESTNG_DEPS_DIR}/testng-6.9.9.jar > +wget ${MAVEN_REPO_URL}/org/testng/testng/6.9.9/testng-6.9.9.jar -O > ${TESTNG_JAR} > +printf "8b211c4c5bed389e169a2173da32d07f93ee2fa8 ${TESTNG_JAR}" | > shasum -a 1 --check - > > TESTNG_LICENSE=${TESTNG_DEPS_DIR}/LICENSE.txt > -wget https://raw.githubusercontent.com/cbeust/testng/testng-6.9. > 5/LICENSE.txt -O ${TESTNG_LICENSE} > +wget https://raw.githubusercontent.com/cbeust/testng/6.9.9/LICENSE.txt > -O ${TESTNG_LICENSE} > > JCOMMANDER_JAR=${TESTNG_DEPS_DIR}/jcommander-1.72.jar > wget ${MAVEN_REPO_URL}/com/beust/jcommander/1.72/jcommander-1.72.jar -O > ${JCOMMANDER_JAR} > > > On Fri, Dec 15, 2017 at 2:34 PM, Jonathan Gibbons < > jonathan.gibbons at oracle.com> wrote: > >> I've pushed the changeset containing this new script. >> >> There are two changes since I posted the webrev: >> >> 1. Change to use "shasum -a 1" instead of "sha1sum" >> 2. Update script to download and use jcommander.jar in conjunction with >> TestNG. >> >> With the second change in particular, all the jtreg self-tests pass, when >> the image is built with the build-all.sh script. >> >> -- Jon >> >> >> On 12/13/2017 04:32 PM, Jonathan Gibbons wrote: >> >>> This is for folk who are interested in building jtreg from source. >>> >>> As some of you have (rightfully) commented over the past years, jtreg >>> has not been an easy tool to build from source. >>> >>> And, as some of you may have noticed, there has been some amount of >>> activity over the past weeks and months to address this issue. This work >>> has been led by Erik Helin (thanks, Erik!) and we're now getting to the >>> point where we can show what we have been working towards. >>> >>> The core of the work to build jtreg is still the Makefiles as before, >>> although as was recently noted, we've been simplifying the specification of >>> the dependencies. >>> >>> Separately, Erik has helped provide updates to the way that some of the >>> Code Tools dependencies can be built. >>> >>> Building on all that work, we can now get to the next stage, to provide >>> a script that will download binaries for some components (JUnit, TestNG) >>> and will download and build source for other components (AsmTools, JCov, >>> JTHarness), for which there are no official binaries. >>> >>> To run the script, you just need to have Ant and a suitable "java" on >>> your path, and to specify the location of an install of JDK 1.8 as an >>> argument to the script. wget is used to download files, which honors proxy >>> settings for those that need to use them. The script is deliberately fairly >>> simple, and suitable for use in a CI system. >>> >>> You can see a webrev for the script at >>> http://cr.openjdk.java.net/~jjg/7902083/webrev.00/ >>> >>> Example of use: >>> >>> $ which ant >>> /opt/ant/1.9.4/bin/ant >>> $ which java >>> /opt/jdk/1.8.0/bin/java >>> $ sh make/build-all.sh /opt/jdk/1.8.0 >>> ... build output ... >>> $ ls build/images/jtreg >>> bin COPYRIGHT doc legal lib LICENSE README release >>> $ >>> >>> >>> Once this settles down a bit, I'll update the public docs on the jtreg >>> web pages. >>> >>> -- Jon >>> >>> >>> >> > From jonathan.gibbons at oracle.com Sun Dec 17 20:17:13 2017 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Sun, 17 Dec 2017 12:17:13 -0800 Subject: CODETOOLS-7902083: Simplify building jtreg In-Reply-To: References: <5A31C69F.4090800@oracle.com> <5A344DF2.7060103@oracle.com> Message-ID: <3595a6a0-073c-90aa-c8ba-f37e24364e7c@oracle.com> Martin, Thanks for the feedback and suggestions. I'll look at incorporating the scripting changes, and will separately investigate moving jtreg onto newer versions of external components. -- Jon On 12/16/17 10:51 AM, Martin Buchholz wrote: > Thanks for working on this.? I've written similar scripts to gather up > all dependencies and build. > > I suggest using wget's -q flag. > > I suggest creating little shell subroutines to check the checksums > (which print the actual checksum in case of mismatch) and to download > maven jars. > > Here's some random shell code I've written that could get reused: > > def_make_var() { > ? typeset -n varref="$1" > ? varref="$2" > ? MAKE_FLAGS+=("$1=$2") > } > > def_make_file() { > ? [[ -f "$2" ]] || die "%s: no such file" "$2" > ? def_make_var "$@" > } > > def_make_dir() { > ? [[ -d "$2" ]] || die "%s: no such directory" "$2" > ? def_make_var "$@" > } > > download_maven_jar() { > ? local -r groupId="$1" artifactId="$2" version="$3" > ? local -r jarfile="${artifactId}-${version}.jar" > ? wget -qO"/tmp/$jarfile" \ > ? ? > "http://search.maven.org/remotecontent?filepath=${groupId}/${artifactId}/${version}/${jarfile} > " > ? def_make_file "${artifactId^^}_JAR" "/tmp/$jarfile" > ? DOWNLOADED_MAVEN_JARS+=("/tmp/$jarfile") > } > > > > On Sat, Dec 16, 2017 at 9:55 AM, Martin Buchholz > wrote: > > testng-6.9.5 was packaged poorly, with jdk8 classfiles. > testng-6.9.8 had different packaging problems. > testng-6.9.9 is golden, with jdk7 class files. > (of course, there are also more recent testng versions) > > --- a/make/build-all.sh > +++ b/make/build-all.sh > @@ -172,12 +172,12 @@ > ?TESTNG_DEPS_DIR=${JTREG_DEPS_DIR}/testng > ?mkdir ${TESTNG_DEPS_DIR} > -TESTNG_JAR=${TESTNG_DEPS_DIR}/testng-6.9.5.jar > -wget ${MAVEN_REPO_URL}/org/testng/testng/6.9.5/testng-6.9.5.jar > -O ${TESTNG_JAR} > -printf "5d12ea207fc47c3f341a3f8ecc88a3eac396a777 ${TESTNG_JAR}" | > shasum -a 1 --check - > +TESTNG_JAR=${TESTNG_DEPS_DIR}/testng-6.9.9.jar > +wget ${MAVEN_REPO_URL}/org/testng/testng/6.9.9/testng-6.9.9.jar > -O ${TESTNG_JAR} > +printf "8b211c4c5bed389e169a2173da32d07f93ee2fa8 ${TESTNG_JAR}" | > shasum -a 1 --check - > ?TESTNG_LICENSE=${TESTNG_DEPS_DIR}/LICENSE.txt > -wget > https://raw.githubusercontent.com/cbeust/testng/testng-6.9.5/LICENSE.txt > > -O ${TESTNG_LICENSE} > +wget > https://raw.githubusercontent.com/cbeust/testng/6.9.9/LICENSE.txt > > -O ${TESTNG_LICENSE} > ?JCOMMANDER_JAR=${TESTNG_DEPS_DIR}/jcommander-1.72.jar > ?wget > ${MAVEN_REPO_URL}/com/beust/jcommander/1.72/jcommander-1.72.jar -O > ${JCOMMANDER_JAR} > > > On Fri, Dec 15, 2017 at 2:34 PM, Jonathan Gibbons > > > wrote: > > I've pushed the changeset containing this new script. > > There are two changes since I posted the webrev: > > 1. Change to use "shasum -a 1" instead of "sha1sum" > 2. Update script to download and use jcommander.jar in > conjunction with TestNG. > > With the second change in particular, all the jtreg self-tests > pass, when the image is built with the build-all.sh script. > > -- Jon > > > On 12/13/2017 04:32 PM, Jonathan Gibbons wrote: > > This is for folk who are interested in building jtreg from > source. > > As some of you have (rightfully) commented over the past > years, jtreg has not been an easy tool to build from source. > > And, as some of you may have noticed, there has been some > amount of activity over the past weeks and months to > address this issue. This work has been led by Erik Helin > (thanks, Erik!) and we're now getting to the point where > we can show what we have been working towards. > > The core of the work to build jtreg is still the Makefiles > as before, although as was recently noted, we've been > simplifying the specification of the dependencies. > > Separately, Erik has helped provide updates to the way > that some of the Code Tools dependencies can be built. > > Building on all that work, we can now get to the next > stage, to provide a script that will download binaries for > some components (JUnit, TestNG) and will download and > build source for other components (AsmTools, JCov, > JTHarness), for which there are no official binaries. > > To run the script, you just need to have Ant and a > suitable "java" on your path, and to specify the location > of an install of JDK 1.8 as an argument to the script. > wget is used to download files, which honors proxy > settings for those that need to use them. The script is > deliberately fairly simple, and suitable for use in a CI > system. > > You can see a webrev for the script at > http://cr.openjdk.java.net/~jjg/7902083/webrev.00/ > > > Example of use: > > $ which ant > /opt/ant/1.9.4/bin/ant > $ which java > /opt/jdk/1.8.0/bin/java > $ sh make/build-all.sh /opt/jdk/1.8.0 > ... build output ... > $ ls build/images/jtreg > bin? COPYRIGHT? doc? legal? lib? LICENSE README? release > $ > > > Once this settles down a bit, I'll update the public docs > on the jtreg web pages. > > -- Jon > > > > > From sadhak001 at gmail.com Sun Dec 17 21:46:38 2017 From: sadhak001 at gmail.com (Mani Sarkar) Date: Sun, 17 Dec 2017 21:46:38 +0000 Subject: CODETOOLS-7902083: Simplify building jtreg In-Reply-To: <7b8de36f-dbb2-9ddc-5f68-049c60bce314@oracle.com> References: <5A31C69F.4090800@oracle.com> <7b8de36f-dbb2-9ddc-5f68-049c60bce314@oracle.com> Message-ID: Hi Jon, I remember that the versions before this release also included asmtools during the build process, but we found lately that the final artifact did not actually contain the asmtools commands inside the final jtreg artifact. Has this been resolved in the current version, I will soon be building it, and would like to be able to use jtreg features that make use of asmtools. Thanks. Cheers, Mani On Sun, 17 Dec 2017 21:04 Maurizio Cimadamore, < maurizio.cimadamore at oracle.com> wrote: > Looks great - thanks for doing this. > > If there's interest, I could also put some effort in order to integrate > the plugins build into the system. In principle, it should be doable, by > adding a bunch of env variables (to point at the IDEA runtime jar). Of > course that would be an optional part of the build. > > Cheers > Maurizio > > > On 14/12/17 00:32, Jonathan Gibbons wrote: > > This is for folk who are interested in building jtreg from source. > > > > As some of you have (rightfully) commented over the past years, jtreg > > has not been an easy tool to build from source. > > > > And, as some of you may have noticed, there has been some amount of > > activity over the past weeks and months to address this issue. This > > work has been led by Erik Helin (thanks, Erik!) and we're now getting > > to the point where we can show what we have been working towards. > > > > The core of the work to build jtreg is still the Makefiles as before, > > although as was recently noted, we've been simplifying the > > specification of the dependencies. > > > > Separately, Erik has helped provide updates to the way that some of > > the Code Tools dependencies can be built. > > > > Building on all that work, we can now get to the next stage, to > > provide a script that will download binaries for some components > > (JUnit, TestNG) and will download and build source for other > > components (AsmTools, JCov, JTHarness), for which there are no > > official binaries. > > > > To run the script, you just need to have Ant and a suitable "java" on > > your path, and to specify the location of an install of JDK 1.8 as an > > argument to the script. wget is used to download files, which honors > > proxy settings for those that need to use them. The script is > > deliberately fairly simple, and suitable for use in a CI system. > > > > You can see a webrev for the script at > > http://cr.openjdk.java.net/~jjg/7902083/webrev.00/ > > > > Example of use: > > > > $ which ant > > /opt/ant/1.9.4/bin/ant > > $ which java > > /opt/jdk/1.8.0/bin/java > > $ sh make/build-all.sh /opt/jdk/1.8.0 > > ... build output ... > > $ ls build/images/jtreg > > bin COPYRIGHT doc legal lib LICENSE README release > > $ > > > > > > Once this settles down a bit, I'll update the public docs on the jtreg > > web pages. > > > > -- Jon > > > > > > -- @theNeomatrix369 | Blog | @adoptopenjdk | Dev communities Meet-a-Project - MutabilityDetector | Github | Slideshare | LinkedIn Come to Devoxx UK 2018: http://www.devoxx.co.uk/ Don't chase success, rather aim for "Excellence", and success will come chasing after you! From jonathan.gibbons at oracle.com Mon Dec 18 01:29:11 2017 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Sun, 17 Dec 2017 17:29:11 -0800 Subject: CODETOOLS-7902083: Simplify building jtreg In-Reply-To: References: <5A31C69F.4090800@oracle.com> <7b8de36f-dbb2-9ddc-5f68-049c60bce314@oracle.com> Message-ID: <28292475-3d84-09ca-bcc7-9644b0b8ac68@oracle.com> Mani, I know of no issues with respect to asmtools.? When I build jtreg with this script, all the self-tests pass.? If you encounter issues with asmtools, please let me know. -- Jon On 12/17/17 1:46 PM, Mani Sarkar wrote: > > Hi Jon, > > I remember that the versions before this release also included > asmtools during the build process, but we found lately that the final > artifact did not actually contain the asmtools commands inside the > final jtreg artifact. > > Has this been resolved in the current version, I will soon be building > it, and would like to be able to use jtreg features that make use of? > asmtools. > > Thanks. > > Cheers, > Mani > > > On Sun, 17 Dec 2017 21:04 Maurizio Cimadamore, > > wrote: > > Looks great - thanks for doing this. > > If there's interest, I could also put some effort in order to > integrate > the plugins build into the system. In principle, it should be > doable, by > adding a bunch of env variables (to point at the IDEA runtime jar). Of > course that would be an optional part of the build. > > Cheers > Maurizio > > > On 14/12/17 00:32, Jonathan Gibbons wrote: > > This is for folk who are interested in building jtreg from source. > > > > As some of you have (rightfully) commented over the past years, > jtreg > > has not been an easy tool to build from source. > > > > And, as some of you may have noticed, there has been some amount of > > activity over the past weeks and months to address this issue. This > > work has been led by Erik Helin (thanks, Erik!) and we're now > getting > > to the point where we can show what we have been working towards. > > > > The core of the work to build jtreg is still the Makefiles as > before, > > although as was recently noted, we've been simplifying the > > specification of the dependencies. > > > > Separately, Erik has helped provide updates to the way that some of > > the Code Tools dependencies can be built. > > > > Building on all that work, we can now get to the next stage, to > > provide a script that will download binaries for some components > > (JUnit, TestNG) and will download and build source for other > > components (AsmTools, JCov, JTHarness), for which there are no > > official binaries. > > > > To run the script, you just need to have Ant and a suitable > "java" on > > your path, and to specify the location of an install of JDK 1.8 > as an > > argument to the script. wget is used to download files, which honors > > proxy settings for those that need to use them. The script is > > deliberately fairly simple, and suitable for use in a CI system. > > > > You can see a webrev for the script at > > http://cr.openjdk.java.net/~jjg/7902083/webrev.00/ > > > > > Example of use: > > > > $ which ant > > /opt/ant/1.9.4/bin/ant > > $ which java > > /opt/jdk/1.8.0/bin/java > > $ sh make/build-all.sh /opt/jdk/1.8.0 > > ... build output ... > > $ ls build/images/jtreg > > bin? COPYRIGHT? doc? legal? lib? LICENSE? README? release > > $ > > > > > > Once this settles down a bit, I'll update the public docs on the > jtreg > > web pages. > > > > -- Jon > > > > > > -- > > @theNeomatrix369 ?| Blog > ?| @adoptopenjdk | Dev communities > > Meet-a-Project - MutabilityDetector > ?| Github > ?| Slideshare > ?|LinkedIn > > > Come to Devoxx UK 2018:http://www.devoxx.co.uk/ > > > Don't chase success, rather aim for "Excellence", and success will > come chasing after you! > From jonathan.gibbons at oracle.com Mon Dec 18 01:32:22 2017 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Sun, 17 Dec 2017 17:32:22 -0800 Subject: CODETOOLS-7902083: Simplify building jtreg In-Reply-To: <7b8de36f-dbb2-9ddc-5f68-049c60bce314@oracle.com> References: <5A31C69F.4090800@oracle.com> <7b8de36f-dbb2-9ddc-5f68-049c60bce314@oracle.com> Message-ID: Maurizio, If you can do this in a way that makes it optional, that would be great.?? Having put effort into pruning the set of dependencies, I've no desire to see the set creep up again unnecessarily, although I admit the usefulness of the plugin. -- Jon On 12/17/17 1:04 PM, Maurizio Cimadamore wrote: > Looks great - thanks for doing this. > > If there's interest, I could also put some effort in order to > integrate the plugins build into the system. In principle, it should > be doable, by adding a bunch of env variables (to point at the IDEA > runtime jar). Of course that would be an optional part of the build. > > Cheers > Maurizio > > > On 14/12/17 00:32, Jonathan Gibbons wrote: >> This is for folk who are interested in building jtreg from source. >> >> As some of you have (rightfully) commented over the past years, jtreg >> has not been an easy tool to build from source. >> >> And, as some of you may have noticed, there has been some amount of >> activity over the past weeks and months to address this issue. This >> work has been led by Erik Helin (thanks, Erik!) and we're now getting >> to the point where we can show what we have been working towards. >> >> The core of the work to build jtreg is still the Makefiles as before, >> although as was recently noted, we've been simplifying the >> specification of the dependencies. >> >> Separately, Erik has helped provide updates to the way that some of >> the Code Tools dependencies can be built. >> >> Building on all that work, we can now get to the next stage, to >> provide a script that will download binaries for some components >> (JUnit, TestNG) and will download and build source for other >> components (AsmTools, JCov, JTHarness), for which there are no >> official binaries. >> >> To run the script, you just need to have Ant and a suitable "java" on >> your path, and to specify the location of an install of JDK 1.8 as an >> argument to the script. wget is used to download files, which honors >> proxy settings for those that need to use them. The script is >> deliberately fairly simple, and suitable for use in a CI system. >> >> You can see a webrev for the script at >> http://cr.openjdk.java.net/~jjg/7902083/webrev.00/ >> >> Example of use: >> >> $ which ant >> /opt/ant/1.9.4/bin/ant >> $ which java >> /opt/jdk/1.8.0/bin/java >> $ sh make/build-all.sh /opt/jdk/1.8.0 >> ... build output ... >> $ ls build/images/jtreg >> bin? COPYRIGHT? doc? legal? lib? LICENSE? README? release >> $ >> >> >> Once this settles down a bit, I'll update the public docs on the >> jtreg web pages. >> >> -- Jon >> >> > From sadhak001 at gmail.com Tue Dec 19 23:33:17 2017 From: sadhak001 at gmail.com (Mani Sarkar) Date: Tue, 19 Dec 2017 23:33:17 +0000 Subject: CODETOOLS-7902083: Simplify building jtreg In-Reply-To: <43aac4d2-b252-06ac-b900-bd6746013d47@oracle.com> References: <5A31C69F.4090800@oracle.com> <7b8de36f-dbb2-9ddc-5f68-049c60bce314@oracle.com> <43aac4d2-b252-06ac-b900-bd6746013d47@oracle.com> Message-ID: Hi Jon, Amended the scripts to use the latest build-all.sh from the tip, and got this - https://ci.adoptopenjdk.net/view/Dependencies/job/jtreg/. Still using our old script to build from the last tag. Both the artifacts do contain asmtools.jar - let me know if it passes your sanity check. Thanks again. Cheers, Mani On Mon, 18 Dec 2017 at 10:18 Maurizio Cimadamore < maurizio.cimadamore at oracle.com> wrote: > > > On 18/12/17 01:32, Jonathan Gibbons wrote: > > Maurizio, > > > > If you can do this in a way that makes it optional, that would be > > great. Having put effort into pruning the set of dependencies, I've > > no desire to see the set creep up again unnecessarily, although I > > admit the usefulness of the plugin. > Message understood :-) > > Btw, I'm bringing this up because, currently, the only way to build the > plugin is through the IDE itself, which, while not unreasonable (after > all you have to have an IDE if you want to use the plugin :-)), it > require some configuration and I've seen people getting stuck quite > frequently - a build command would probably mitigate some of those > concerns. > > Maurizio > > > > -- Jon > > > > > > On 12/17/17 1:04 PM, Maurizio Cimadamore wrote: > >> Looks great - thanks for doing this. > >> > >> If there's interest, I could also put some effort in order to > >> integrate the plugins build into the system. In principle, it should > >> be doable, by adding a bunch of env variables (to point at the IDEA > >> runtime jar). Of course that would be an optional part of the build. > >> > >> Cheers > >> Maurizio > >> > >> > >> On 14/12/17 00:32, Jonathan Gibbons wrote: > >>> This is for folk who are interested in building jtreg from source. > >>> > >>> As some of you have (rightfully) commented over the past years, > >>> jtreg has not been an easy tool to build from source. > >>> > >>> And, as some of you may have noticed, there has been some amount of > >>> activity over the past weeks and months to address this issue. This > >>> work has been led by Erik Helin (thanks, Erik!) and we're now > >>> getting to the point where we can show what we have been working > >>> towards. > >>> > >>> The core of the work to build jtreg is still the Makefiles as > >>> before, although as was recently noted, we've been simplifying the > >>> specification of the dependencies. > >>> > >>> Separately, Erik has helped provide updates to the way that some of > >>> the Code Tools dependencies can be built. > >>> > >>> Building on all that work, we can now get to the next stage, to > >>> provide a script that will download binaries for some components > >>> (JUnit, TestNG) and will download and build source for other > >>> components (AsmTools, JCov, JTHarness), for which there are no > >>> official binaries. > >>> > >>> To run the script, you just need to have Ant and a suitable "java" > >>> on your path, and to specify the location of an install of JDK 1.8 > >>> as an argument to the script. wget is used to download files, which > >>> honors proxy settings for those that need to use them. The script is > >>> deliberately fairly simple, and suitable for use in a CI system. > >>> > >>> You can see a webrev for the script at > >>> http://cr.openjdk.java.net/~jjg/7902083/webrev.00/ > >>> > >>> Example of use: > >>> > >>> $ which ant > >>> /opt/ant/1.9.4/bin/ant > >>> $ which java > >>> /opt/jdk/1.8.0/bin/java > >>> $ sh make/build-all.sh /opt/jdk/1.8.0 > >>> ... build output ... > >>> $ ls build/images/jtreg > >>> bin COPYRIGHT doc legal lib LICENSE README release > >>> $ > >>> > >>> > >>> Once this settles down a bit, I'll update the public docs on the > >>> jtreg web pages. > >>> > >>> -- Jon > >>> > >>> > >> > > > > -- @theNeomatrix369 | Blog | @adoptopenjdk | Dev communities Meet-a-Project - MutabilityDetector | Github | Slideshare | LinkedIn Come to Devoxx UK 2018: http://www.devoxx.co.uk/ Don't chase success, rather aim for "Excellence", and success will come chasing after you! From jonathan.gibbons at oracle.com Tue Dec 19 23:57:09 2017 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Tue, 19 Dec 2017 15:57:09 -0800 Subject: CODETOOLS-7902083: Simplify building jtreg In-Reply-To: References: <5A31C69F.4090800@oracle.com> <7b8de36f-dbb2-9ddc-5f68-049c60bce314@oracle.com> <43aac4d2-b252-06ac-b900-bd6746013d47@oracle.com> Message-ID: <44c82008-98df-44d0-b065-c15ea9558bdc@oracle.com> Sounds good;? I expect to promote (and tag) a new version tomorrow, getting ready for JDK 11. -- Jon On 12/19/17 3:33 PM, Mani Sarkar wrote: > Hi Jon, > > Amended the scripts to use the latest build-all.sh from the tip, and > got this - https://ci.adoptopenjdk.net/view/Dependencies/job/jtreg/. > > Still using our old script to build from the last tag. > > Both the?artifacts do contain asmtools.jar - let me know if it passes > your sanity check. > > Thanks again. > > Cheers, > Mani > > On Mon, 18 Dec 2017 at 10:18 Maurizio Cimadamore > > wrote: > > > > On 18/12/17 01:32, Jonathan Gibbons wrote: > > Maurizio, > > > > If you can do this in a way that makes it optional, that would be > > great.?? Having put effort into pruning the set of dependencies, > I've > > no desire to see the set creep up again unnecessarily, although I > > admit the usefulness of the plugin. > Message understood :-) > > Btw, I'm bringing this up because, currently, the only way to > build the > plugin is through the IDE itself, which, while not unreasonable (after > all you have to have an IDE if you want to use the plugin :-)), it > require some configuration and I've seen people getting stuck quite > frequently - a build command would probably mitigate some of those > concerns. > > Maurizio > > > > -- Jon > > > > > > On 12/17/17 1:04 PM, Maurizio Cimadamore wrote: > >> Looks great - thanks for doing this. > >> > >> If there's interest, I could also put some effort in order to > >> integrate the plugins build into the system. In principle, it > should > >> be doable, by adding a bunch of env variables (to point at the IDEA > >> runtime jar). Of course that would be an optional part of the > build. > >> > >> Cheers > >> Maurizio > >> > >> > >> On 14/12/17 00:32, Jonathan Gibbons wrote: > >>> This is for folk who are interested in building jtreg from source. > >>> > >>> As some of you have (rightfully) commented over the past years, > >>> jtreg has not been an easy tool to build from source. > >>> > >>> And, as some of you may have noticed, there has been some > amount of > >>> activity over the past weeks and months to address this issue. > This > >>> work has been led by Erik Helin (thanks, Erik!) and we're now > >>> getting to the point where we can show what we have been working > >>> towards. > >>> > >>> The core of the work to build jtreg is still the Makefiles as > >>> before, although as was recently noted, we've been simplifying the > >>> specification of the dependencies. > >>> > >>> Separately, Erik has helped provide updates to the way that > some of > >>> the Code Tools dependencies can be built. > >>> > >>> Building on all that work, we can now get to the next stage, to > >>> provide a script that will download binaries for some components > >>> (JUnit, TestNG) and will download and build source for other > >>> components (AsmTools, JCov, JTHarness), for which there are no > >>> official binaries. > >>> > >>> To run the script, you just need to have Ant and a suitable "java" > >>> on your path, and to specify the location of an install of JDK 1.8 > >>> as an argument to the script. wget is used to download files, > which > >>> honors proxy settings for those that need to use them. The > script is > >>> deliberately fairly simple, and suitable for use in a CI system. > >>> > >>> You can see a webrev for the script at > >>> http://cr.openjdk.java.net/~jjg/7902083/webrev.00/ > > >>> > >>> Example of use: > >>> > >>> $ which ant > >>> /opt/ant/1.9.4/bin/ant > >>> $ which java > >>> /opt/jdk/1.8.0/bin/java > >>> $ sh make/build-all.sh /opt/jdk/1.8.0 > >>> ... build output ... > >>> $ ls build/images/jtreg > >>> bin? COPYRIGHT? doc? legal? lib? LICENSE? README release > >>> $ > >>> > >>> > >>> Once this settles down a bit, I'll update the public docs on the > >>> jtreg web pages. > >>> > >>> -- Jon > >>> > >>> > >> > > > > -- > > @theNeomatrix369 ?| Blog > ?| @adoptopenjdk | Dev communities > > Meet-a-Project - MutabilityDetector > ?| Github > ?| Slideshare > ?|LinkedIn > > > Come to Devoxx UK 2018:http://www.devoxx.co.uk/ > > > Don't chase success, rather aim for "Excellence", and success will > come chasing after you! > From jonathan.gibbons at oracle.com Thu Dec 21 01:09:12 2017 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Wed, 20 Dec 2017 17:09:12 -0800 Subject: CODETOOLS-7902083: Simplify building jtreg In-Reply-To: References: <5A31C69F.4090800@oracle.com> <7b8de36f-dbb2-9ddc-5f68-049c60bce314@oracle.com> <43aac4d2-b252-06ac-b900-bd6746013d47@oracle.com> Message-ID: <5A3B09B8.5050905@oracle.com> I've posted a new tag (jtreg4.2-b11) on the jtreg repo, so I would expect to see a new build on your CI system soon. If you're using the new make/build-all.sh script, then everything should build as intended. If you're not using build-all.sh, note that you need to build jtreg with variables like TESTNG_HOME, JUNIT_HOME, ASMTOOLS_HOME etc defined at the time you run "make". It is not enough to copy the corresponding jar files into the lib directory; they must also be put in the Class-Path entry of the jtreg.jar MANIFEST.MF file (and the makefiles will take care of doing that.) It occurs to me you might still be using the (old) Ant build.xml file. If so, I recommend you convert to using the new build-all.sh script. -- Jon On 12/19/2017 03:33 PM, Mani Sarkar wrote: > Hi Jon, > > Amended the scripts to use the latest build-all.sh from the tip, and > got this - https://ci.adoptopenjdk.net/view/Dependencies/job/jtreg/. > > Still using our old script to build from the last tag. > > Both the artifacts do contain asmtools.jar - let me know if it passes > your sanity check. > > Thanks again. > > Cheers, > Mani > > On Mon, 18 Dec 2017 at 10:18 Maurizio Cimadamore > > wrote: > > > > On 18/12/17 01:32, Jonathan Gibbons wrote: > > Maurizio, > > > > If you can do this in a way that makes it optional, that would be > > great. Having put effort into pruning the set of dependencies, > I've > > no desire to see the set creep up again unnecessarily, although I > > admit the usefulness of the plugin. > Message understood :-) > > Btw, I'm bringing this up because, currently, the only way to > build the > plugin is through the IDE itself, which, while not unreasonable (after > all you have to have an IDE if you want to use the plugin :-)), it > require some configuration and I've seen people getting stuck quite > frequently - a build command would probably mitigate some of those > concerns. > > Maurizio > > > > -- Jon > > > > > > On 12/17/17 1:04 PM, Maurizio Cimadamore wrote: > >> Looks great - thanks for doing this. > >> > >> If there's interest, I could also put some effort in order to > >> integrate the plugins build into the system. In principle, it > should > >> be doable, by adding a bunch of env variables (to point at the IDEA > >> runtime jar). Of course that would be an optional part of the > build. > >> > >> Cheers > >> Maurizio > >> > >> > >> On 14/12/17 00:32, Jonathan Gibbons wrote: > >>> This is for folk who are interested in building jtreg from source. > >>> > >>> As some of you have (rightfully) commented over the past years, > >>> jtreg has not been an easy tool to build from source. > >>> > >>> And, as some of you may have noticed, there has been some > amount of > >>> activity over the past weeks and months to address this issue. > This > >>> work has been led by Erik Helin (thanks, Erik!) and we're now > >>> getting to the point where we can show what we have been working > >>> towards. > >>> > >>> The core of the work to build jtreg is still the Makefiles as > >>> before, although as was recently noted, we've been simplifying the > >>> specification of the dependencies. > >>> > >>> Separately, Erik has helped provide updates to the way that > some of > >>> the Code Tools dependencies can be built. > >>> > >>> Building on all that work, we can now get to the next stage, to > >>> provide a script that will download binaries for some components > >>> (JUnit, TestNG) and will download and build source for other > >>> components (AsmTools, JCov, JTHarness), for which there are no > >>> official binaries. > >>> > >>> To run the script, you just need to have Ant and a suitable "java" > >>> on your path, and to specify the location of an install of JDK 1.8 > >>> as an argument to the script. wget is used to download files, > which > >>> honors proxy settings for those that need to use them. The > script is > >>> deliberately fairly simple, and suitable for use in a CI system. > >>> > >>> You can see a webrev for the script at > >>> http://cr.openjdk.java.net/~jjg/7902083/webrev.00/ > > >>> > >>> Example of use: > >>> > >>> $ which ant > >>> /opt/ant/1.9.4/bin/ant > >>> $ which java > >>> /opt/jdk/1.8.0/bin/java > >>> $ sh make/build-all.sh /opt/jdk/1.8.0 > >>> ... build output ... > >>> $ ls build/images/jtreg > >>> bin COPYRIGHT doc legal lib LICENSE README release > >>> $ > >>> > >>> > >>> Once this settles down a bit, I'll update the public docs on the > >>> jtreg web pages. > >>> > >>> -- Jon > >>> > >>> > >> > > > > -- > > @theNeomatrix369 | Blog > | @adoptopenjdk | Dev communities > > Meet-a-Project - MutabilityDetector > | Github > | Slideshare > |LinkedIn > > > Come to Devoxx UK 2018:http://www.devoxx.co.uk/ > > > Don't chase success, rather aim for "Excellence", and success will > come chasing after you! > From jonathan.gibbons at oracle.com Thu Dec 21 01:12:29 2017 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Wed, 20 Dec 2017 17:12:29 -0800 Subject: CODETOOLS-7902083: Simplify building jtreg In-Reply-To: <5A3B09B8.5050905@oracle.com> References: <5A31C69F.4090800@oracle.com> <7b8de36f-dbb2-9ddc-5f68-049c60bce314@oracle.com> <43aac4d2-b252-06ac-b900-bd6746013d47@oracle.com> <5A3B09B8.5050905@oracle.com> Message-ID: <5A3B0A7D.4030607@oracle.com> My previous message was intended for Mani. I'm sorry I did not make that clearer. -- Jon On 12/20/2017 05:09 PM, Jonathan Gibbons wrote: > I've posted a new tag (jtreg4.2-b11) on the jtreg repo, so I would > expect to see a new build on your CI system soon. > > If you're using the new make/build-all.sh script, then everything > should build as intended. > > If you're not using build-all.sh, note that you need to build jtreg > with variables like TESTNG_HOME, JUNIT_HOME, ASMTOOLS_HOME etc defined > at the time you run "make". It is not enough to copy the corresponding > jar files into the lib directory; they must also be put in the > Class-Path entry of the jtreg.jar MANIFEST.MF file (and the makefiles > will take care of doing that.) > > It occurs to me you might still be using the (old) Ant build.xml > file. If so, I recommend you convert to using the new build-all.sh > script. > > -- Jon > > On 12/19/2017 03:33 PM, Mani Sarkar wrote: >> Hi Jon, >> >> Amended the scripts to use the latest build-all.sh from the tip, and >> got this - https://ci.adoptopenjdk.net/view/Dependencies/job/jtreg/. >> >> Still using our old script to build from the last tag. >> >> Both the artifacts do contain asmtools.jar - let me know if it passes >> your sanity check. >> >> Thanks again. >> >> Cheers, >> Mani >> >> On Mon, 18 Dec 2017 at 10:18 Maurizio Cimadamore >> > > wrote: >> >> >> >> On 18/12/17 01:32, Jonathan Gibbons wrote: >> > Maurizio, >> > >> > If you can do this in a way that makes it optional, that would be >> > great. Having put effort into pruning the set of dependencies, >> I've >> > no desire to see the set creep up again unnecessarily, although I >> > admit the usefulness of the plugin. >> Message understood :-) >> >> Btw, I'm bringing this up because, currently, the only way to >> build the >> plugin is through the IDE itself, which, while not unreasonable >> (after >> all you have to have an IDE if you want to use the plugin :-)), it >> require some configuration and I've seen people getting stuck quite >> frequently - a build command would probably mitigate some of those >> concerns. >> >> Maurizio >> > >> > -- Jon >> > >> > >> > On 12/17/17 1:04 PM, Maurizio Cimadamore wrote: >> >> Looks great - thanks for doing this. >> >> >> >> If there's interest, I could also put some effort in order to >> >> integrate the plugins build into the system. In principle, it >> should >> >> be doable, by adding a bunch of env variables (to point at the >> IDEA >> >> runtime jar). Of course that would be an optional part of the >> build. >> >> >> >> Cheers >> >> Maurizio >> >> >> >> >> >> On 14/12/17 00:32, Jonathan Gibbons wrote: >> >>> This is for folk who are interested in building jtreg from >> source. >> >>> >> >>> As some of you have (rightfully) commented over the past years, >> >>> jtreg has not been an easy tool to build from source. >> >>> >> >>> And, as some of you may have noticed, there has been some >> amount of >> >>> activity over the past weeks and months to address this issue. >> This >> >>> work has been led by Erik Helin (thanks, Erik!) and we're now >> >>> getting to the point where we can show what we have been working >> >>> towards. >> >>> >> >>> The core of the work to build jtreg is still the Makefiles as >> >>> before, although as was recently noted, we've been >> simplifying the >> >>> specification of the dependencies. >> >>> >> >>> Separately, Erik has helped provide updates to the way that >> some of >> >>> the Code Tools dependencies can be built. >> >>> >> >>> Building on all that work, we can now get to the next stage, to >> >>> provide a script that will download binaries for some components >> >>> (JUnit, TestNG) and will download and build source for other >> >>> components (AsmTools, JCov, JTHarness), for which there are no >> >>> official binaries. >> >>> >> >>> To run the script, you just need to have Ant and a suitable >> "java" >> >>> on your path, and to specify the location of an install of >> JDK 1.8 >> >>> as an argument to the script. wget is used to download files, >> which >> >>> honors proxy settings for those that need to use them. The >> script is >> >>> deliberately fairly simple, and suitable for use in a CI system. >> >>> >> >>> You can see a webrev for the script at >> >>> http://cr.openjdk.java.net/~jjg/7902083/webrev.00/ >> >> >>> >> >>> Example of use: >> >>> >> >>> $ which ant >> >>> /opt/ant/1.9.4/bin/ant >> >>> $ which java >> >>> /opt/jdk/1.8.0/bin/java >> >>> $ sh make/build-all.sh /opt/jdk/1.8.0 >> >>> ... build output ... >> >>> $ ls build/images/jtreg >> >>> bin COPYRIGHT doc legal lib LICENSE README release >> >>> $ >> >>> >> >>> >> >>> Once this settles down a bit, I'll update the public docs on the >> >>> jtreg web pages. >> >>> >> >>> -- Jon >> >>> >> >>> >> >> >> > >> >> -- >> >> @theNeomatrix369 | Blog >> | @adoptopenjdk | Dev communities >> >> Meet-a-Project - MutabilityDetector >> | Github >> | Slideshare >> |LinkedIn >> >> >> Come to Devoxx UK 2018:http://www.devoxx.co.uk/ >> >> >> Don't chase success, rather aim for "Excellence", and success will >> come chasing after you! >> > From volker.simonis at gmail.com Fri Dec 22 15:17:42 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 22 Dec 2017 16:17:42 +0100 Subject: CODETOOLS-7902083: Simplify building jtreg In-Reply-To: <5A3B0A7D.4030607@oracle.com> References: <5A31C69F.4090800@oracle.com> <7b8de36f-dbb2-9ddc-5f68-049c60bce314@oracle.com> <43aac4d2-b252-06ac-b900-bd6746013d47@oracle.com> <5A3B09B8.5050905@oracle.com> <5A3B0A7D.4030607@oracle.com> Message-ID: Hi Jonathan, thanks a lot for improving the build. This is a huge step ahead and it works great. I've just tried it and the build succeeded out of the box! There's however one issue I still see. The default build will set the version and build number to '4-2 dev b00'. Notice that such a build is pretty useless, because such versions of JTreg will be rejected when doing OpenJDK regression tests because the OpenJDK test have minimal JTreg build requirements in their TEST.ROOT file like for example: # Minimum jtreg version requiredVersion=4.2 b08 Before the introduction of build-all.sh it was possible to pass BUILD_NUMBER through the environment to make. When using build-all.sh this is not possible any more, because build-all.sh calls make without honoring if BUILD_NUMBER was set in the environment. I've opened "CODETOOLS-7902089: build-all.sh should honor BUILD_NUMBER from environment (and use a reasonable default)" (https://bugs.openjdk.java.net/browse/CODETOOLS-7902089) and came up with the following fix: http://cr.openjdk.java.net/~simonis/webrevs/2017/CODETOOLS-7902089 As an extension I propose to set the build number by default to the latest available tag automatically, even if we're building tip because I think that's what users actually want (and need) if they build JTreg for executing the OpenJDK tests. This feature is already realized in my webrev. Finally, we could set the build number to something like "b11+" if "b11" was the latest tagged build, but we're several changes ahead already (this would resemble the "hg id" output which prints a "+" after the actual change hash if the working directory contains additional local changes). I consider this to be the best solution, but in order to work we would also have to update JTReg's build number parsing routine in com.sun.javatest.regtest.tool.Version.getBuild() which currently only accepts a plain build number (i.e. "b[0-9]+"). This feature is currently not in my webrev but I'd be happy to add it if you'd consider it useful. Can you please review and sponsor my change? Thank you and merry Christmas:) Volker On Thu, Dec 21, 2017 at 2:12 AM, Jonathan Gibbons wrote: > My previous message was intended for Mani. I'm sorry I did not make that > clearer. > > -- Jon > > On 12/20/2017 05:09 PM, Jonathan Gibbons wrote: >> >> I've posted a new tag (jtreg4.2-b11) on the jtreg repo, so I would expect >> to see a new build on your CI system soon. >> >> If you're using the new make/build-all.sh script, then everything should >> build as intended. >> >> If you're not using build-all.sh, note that you need to build jtreg with >> variables like TESTNG_HOME, JUNIT_HOME, ASMTOOLS_HOME etc defined at the >> time you run "make". It is not enough to copy the corresponding jar files >> into the lib directory; they must also be put in the Class-Path entry of the >> jtreg.jar MANIFEST.MF file (and the makefiles will take care of doing that.) >> >> It occurs to me you might still be using the (old) Ant build.xml file. If >> so, I recommend you convert to using the new build-all.sh script. >> >> -- Jon >> >> On 12/19/2017 03:33 PM, Mani Sarkar wrote: >>> >>> Hi Jon, >>> >>> Amended the scripts to use the latest build-all.sh from the tip, and got >>> this - https://ci.adoptopenjdk.net/view/Dependencies/job/jtreg/. >>> >>> Still using our old script to build from the last tag. >>> >>> Both the artifacts do contain asmtools.jar - let me know if it passes >>> your sanity check. >>> >>> Thanks again. >>> >>> Cheers, >>> Mani >>> >>> On Mon, 18 Dec 2017 at 10:18 Maurizio Cimadamore >>> > >>> wrote: >>> >>> >>> >>> On 18/12/17 01:32, Jonathan Gibbons wrote: >>> > Maurizio, >>> > >>> > If you can do this in a way that makes it optional, that would be >>> > great. Having put effort into pruning the set of dependencies, >>> I've >>> > no desire to see the set creep up again unnecessarily, although I >>> > admit the usefulness of the plugin. >>> Message understood :-) >>> >>> Btw, I'm bringing this up because, currently, the only way to >>> build the >>> plugin is through the IDE itself, which, while not unreasonable >>> (after >>> all you have to have an IDE if you want to use the plugin :-)), it >>> require some configuration and I've seen people getting stuck quite >>> frequently - a build command would probably mitigate some of those >>> concerns. >>> >>> Maurizio >>> > >>> > -- Jon >>> > >>> > >>> > On 12/17/17 1:04 PM, Maurizio Cimadamore wrote: >>> >> Looks great - thanks for doing this. >>> >> >>> >> If there's interest, I could also put some effort in order to >>> >> integrate the plugins build into the system. In principle, it >>> should >>> >> be doable, by adding a bunch of env variables (to point at the >>> IDEA >>> >> runtime jar). Of course that would be an optional part of the >>> build. >>> >> >>> >> Cheers >>> >> Maurizio >>> >> >>> >> >>> >> On 14/12/17 00:32, Jonathan Gibbons wrote: >>> >>> This is for folk who are interested in building jtreg from >>> source. >>> >>> >>> >>> As some of you have (rightfully) commented over the past years, >>> >>> jtreg has not been an easy tool to build from source. >>> >>> >>> >>> And, as some of you may have noticed, there has been some >>> amount of >>> >>> activity over the past weeks and months to address this issue. >>> This >>> >>> work has been led by Erik Helin (thanks, Erik!) and we're now >>> >>> getting to the point where we can show what we have been working >>> >>> towards. >>> >>> >>> >>> The core of the work to build jtreg is still the Makefiles as >>> >>> before, although as was recently noted, we've been simplifying >>> the >>> >>> specification of the dependencies. >>> >>> >>> >>> Separately, Erik has helped provide updates to the way that >>> some of >>> >>> the Code Tools dependencies can be built. >>> >>> >>> >>> Building on all that work, we can now get to the next stage, to >>> >>> provide a script that will download binaries for some components >>> >>> (JUnit, TestNG) and will download and build source for other >>> >>> components (AsmTools, JCov, JTHarness), for which there are no >>> >>> official binaries. >>> >>> >>> >>> To run the script, you just need to have Ant and a suitable >>> "java" >>> >>> on your path, and to specify the location of an install of JDK >>> 1.8 >>> >>> as an argument to the script. wget is used to download files, >>> which >>> >>> honors proxy settings for those that need to use them. The >>> script is >>> >>> deliberately fairly simple, and suitable for use in a CI system. >>> >>> >>> >>> You can see a webrev for the script at >>> >>> http://cr.openjdk.java.net/~jjg/7902083/webrev.00/ >>> >>> >>> >>> >>> Example of use: >>> >>> >>> >>> $ which ant >>> >>> /opt/ant/1.9.4/bin/ant >>> >>> $ which java >>> >>> /opt/jdk/1.8.0/bin/java >>> >>> $ sh make/build-all.sh /opt/jdk/1.8.0 >>> >>> ... build output ... >>> >>> $ ls build/images/jtreg >>> >>> bin COPYRIGHT doc legal lib LICENSE README release >>> >>> $ >>> >>> >>> >>> >>> >>> Once this settles down a bit, I'll update the public docs on the >>> >>> jtreg web pages. >>> >>> >>> >>> -- Jon >>> >>> >>> >>> >>> >> >>> > >>> >>> -- >>> >>> @theNeomatrix369 | Blog >>> | @adoptopenjdk | Dev communities >>> >>> Meet-a-Project - MutabilityDetector >>> | Github >>> | Slideshare >>> |LinkedIn >>> >>> >>> Come to Devoxx UK 2018:http://www.devoxx.co.uk/ >>> >>> >>> Don't chase success, rather aim for "Excellence", and success will come >>> chasing after you! >>> >> > From jonathan.gibbons at oracle.com Fri Dec 22 15:49:25 2017 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Fri, 22 Dec 2017 07:49:25 -0800 Subject: CODETOOLS-7902083: Simplify building jtreg In-Reply-To: References: <5A31C69F.4090800@oracle.com> <7b8de36f-dbb2-9ddc-5f68-049c60bce314@oracle.com> <43aac4d2-b252-06ac-b900-bd6746013d47@oracle.com> <5A3B09B8.5050905@oracle.com> <5A3B0A7D.4030607@oracle.com> Message-ID: <17b33335-30d0-8316-8200-6b0cebddfafe@oracle.com> Volker, Thanks for the feedback. I'll take a look at your webrev. -- Jon On 12/22/17 7:17 AM, Volker Simonis wrote: > Hi Jonathan, > > thanks a lot for improving the build. This is a huge step ahead and it > works great. I've just tried it and the build succeeded out of the > box! > > There's however one issue I still see. The default build will set the > version and build number to '4-2 dev b00'. Notice that such a build is > pretty useless, because such versions of JTreg will be rejected when > doing OpenJDK regression tests because the OpenJDK test have minimal > JTreg build requirements in their TEST.ROOT file like for example: > > # Minimum jtreg version > requiredVersion=4.2 b08 > > Before the introduction of build-all.sh it was possible to pass > BUILD_NUMBER through the environment to make. > > When using build-all.sh this is not possible any more, because > build-all.sh calls make without honoring if BUILD_NUMBER was set in > the environment. > > I've opened "CODETOOLS-7902089: build-all.sh should honor BUILD_NUMBER > from environment (and use a reasonable default)" > (https://bugs.openjdk.java.net/browse/CODETOOLS-7902089) and came up > with the following fix: > > http://cr.openjdk.java.net/~simonis/webrevs/2017/CODETOOLS-7902089 > > As an extension I propose to set the build number by default to the > latest available tag automatically, even if we're building tip because > I think that's what users actually want (and need) if they build JTreg > for executing the OpenJDK tests. This feature is already realized in > my webrev. > > Finally, we could set the build number to something like "b11+" if > "b11" was the latest tagged build, but we're several changes ahead > already (this would resemble the "hg id" output which prints a "+" > after the actual change hash if the working directory contains > additional local changes). > > I consider this to be the best solution, but in order to work we would > also have to update JTReg's build number parsing routine in > com.sun.javatest.regtest.tool.Version.getBuild() which currently only > accepts a plain build number (i.e. "b[0-9]+"). This feature is > currently not in my webrev but I'd be happy to add it if you'd > consider it useful. > > Can you please review and sponsor my change? > > Thank you and merry Christmas:) > Volker > > > > On Thu, Dec 21, 2017 at 2:12 AM, Jonathan Gibbons > wrote: >> My previous message was intended for Mani. I'm sorry I did not make that >> clearer. >> >> -- Jon >> >> On 12/20/2017 05:09 PM, Jonathan Gibbons wrote: >>> I've posted a new tag (jtreg4.2-b11) on the jtreg repo, so I would expect >>> to see a new build on your CI system soon. >>> >>> If you're using the new make/build-all.sh script, then everything should >>> build as intended. >>> >>> If you're not using build-all.sh, note that you need to build jtreg with >>> variables like TESTNG_HOME, JUNIT_HOME, ASMTOOLS_HOME etc defined at the >>> time you run "make". It is not enough to copy the corresponding jar files >>> into the lib directory; they must also be put in the Class-Path entry of the >>> jtreg.jar MANIFEST.MF file (and the makefiles will take care of doing that.) >>> >>> It occurs to me you might still be using the (old) Ant build.xml file. If >>> so, I recommend you convert to using the new build-all.sh script. >>> >>> -- Jon >>> >>> On 12/19/2017 03:33 PM, Mani Sarkar wrote: >>>> Hi Jon, >>>> >>>> Amended the scripts to use the latest build-all.sh from the tip, and got >>>> this - https://ci.adoptopenjdk.net/view/Dependencies/job/jtreg/. >>>> >>>> Still using our old script to build from the last tag. >>>> >>>> Both the artifacts do contain asmtools.jar - let me know if it passes >>>> your sanity check. >>>> >>>> Thanks again. >>>> >>>> Cheers, >>>> Mani >>>> >>>> On Mon, 18 Dec 2017 at 10:18 Maurizio Cimadamore >>>> > >>>> wrote: >>>> >>>> >>>> >>>> On 18/12/17 01:32, Jonathan Gibbons wrote: >>>> > Maurizio, >>>> > >>>> > If you can do this in a way that makes it optional, that would be >>>> > great. Having put effort into pruning the set of dependencies, >>>> I've >>>> > no desire to see the set creep up again unnecessarily, although I >>>> > admit the usefulness of the plugin. >>>> Message understood :-) >>>> >>>> Btw, I'm bringing this up because, currently, the only way to >>>> build the >>>> plugin is through the IDE itself, which, while not unreasonable >>>> (after >>>> all you have to have an IDE if you want to use the plugin :-)), it >>>> require some configuration and I've seen people getting stuck quite >>>> frequently - a build command would probably mitigate some of those >>>> concerns. >>>> >>>> Maurizio >>>> > >>>> > -- Jon >>>> > >>>> > >>>> > On 12/17/17 1:04 PM, Maurizio Cimadamore wrote: >>>> >> Looks great - thanks for doing this. >>>> >> >>>> >> If there's interest, I could also put some effort in order to >>>> >> integrate the plugins build into the system. In principle, it >>>> should >>>> >> be doable, by adding a bunch of env variables (to point at the >>>> IDEA >>>> >> runtime jar). Of course that would be an optional part of the >>>> build. >>>> >> >>>> >> Cheers >>>> >> Maurizio >>>> >> >>>> >> >>>> >> On 14/12/17 00:32, Jonathan Gibbons wrote: >>>> >>> This is for folk who are interested in building jtreg from >>>> source. >>>> >>> >>>> >>> As some of you have (rightfully) commented over the past years, >>>> >>> jtreg has not been an easy tool to build from source. >>>> >>> >>>> >>> And, as some of you may have noticed, there has been some >>>> amount of >>>> >>> activity over the past weeks and months to address this issue. >>>> This >>>> >>> work has been led by Erik Helin (thanks, Erik!) and we're now >>>> >>> getting to the point where we can show what we have been working >>>> >>> towards. >>>> >>> >>>> >>> The core of the work to build jtreg is still the Makefiles as >>>> >>> before, although as was recently noted, we've been simplifying >>>> the >>>> >>> specification of the dependencies. >>>> >>> >>>> >>> Separately, Erik has helped provide updates to the way that >>>> some of >>>> >>> the Code Tools dependencies can be built. >>>> >>> >>>> >>> Building on all that work, we can now get to the next stage, to >>>> >>> provide a script that will download binaries for some components >>>> >>> (JUnit, TestNG) and will download and build source for other >>>> >>> components (AsmTools, JCov, JTHarness), for which there are no >>>> >>> official binaries. >>>> >>> >>>> >>> To run the script, you just need to have Ant and a suitable >>>> "java" >>>> >>> on your path, and to specify the location of an install of JDK >>>> 1.8 >>>> >>> as an argument to the script. wget is used to download files, >>>> which >>>> >>> honors proxy settings for those that need to use them. The >>>> script is >>>> >>> deliberately fairly simple, and suitable for use in a CI system. >>>> >>> >>>> >>> You can see a webrev for the script at >>>> >>> http://cr.openjdk.java.net/~jjg/7902083/webrev.00/ >>>> >>>> >>> >>>> >>> Example of use: >>>> >>> >>>> >>> $ which ant >>>> >>> /opt/ant/1.9.4/bin/ant >>>> >>> $ which java >>>> >>> /opt/jdk/1.8.0/bin/java >>>> >>> $ sh make/build-all.sh /opt/jdk/1.8.0 >>>> >>> ... build output ... >>>> >>> $ ls build/images/jtreg >>>> >>> bin COPYRIGHT doc legal lib LICENSE README release >>>> >>> $ >>>> >>> >>>> >>> >>>> >>> Once this settles down a bit, I'll update the public docs on the >>>> >>> jtreg web pages. >>>> >>> >>>> >>> -- Jon >>>> >>> >>>> >>> >>>> >> >>>> > >>>> >>>> -- >>>> >>>> @theNeomatrix369 | Blog >>>> | @adoptopenjdk | Dev communities >>>> >>>> Meet-a-Project - MutabilityDetector >>>> | Github >>>> | Slideshare >>>> |LinkedIn >>>> >>>> >>>> Come to Devoxx UK 2018:http://www.devoxx.co.uk/ >>>> >>>> >>>> Don't chase success, rather aim for "Excellence", and success will come >>>> chasing after you! >>>> From martinrb at google.com Fri Dec 22 17:58:34 2017 From: martinrb at google.com (Martin Buchholz) Date: Fri, 22 Dec 2017 09:58:34 -0800 Subject: CODETOOLS-7902083: Simplify building jtreg In-Reply-To: References: <5A31C69F.4090800@oracle.com> <7b8de36f-dbb2-9ddc-5f68-049c60bce314@oracle.com> <43aac4d2-b252-06ac-b900-bd6746013d47@oracle.com> <5A3B09B8.5050905@oracle.com> <5A3B0A7D.4030607@oracle.com> Message-ID: I agree having to fiddle with the build number is a pain point, but it's shared with openjdk proper. You can't rely on mercurial tags; code can get moved out of mercurial (hg archive) or into a different source code system. I would prefer BUILD_NUMBER checked in, and write a little script that modified the source and hg tagged it at the same time. And then of course check in the script! On Fri, Dec 22, 2017 at 7:17 AM, Volker Simonis wrote: > Hi Jonathan, > > thanks a lot for improving the build. This is a huge step ahead and it > works great. I've just tried it and the build succeeded out of the > box! > > There's however one issue I still see. The default build will set the > version and build number to '4-2 dev b00'. Notice that such a build is > pretty useless, because such versions of JTreg will be rejected when > doing OpenJDK regression tests because the OpenJDK test have minimal > JTreg build requirements in their TEST.ROOT file like for example: > > # Minimum jtreg version > requiredVersion=4.2 b08 > > Before the introduction of build-all.sh it was possible to pass > BUILD_NUMBER through the environment to make. > > When using build-all.sh this is not possible any more, because > build-all.sh calls make without honoring if BUILD_NUMBER was set in > the environment. > > I've opened "CODETOOLS-7902089: build-all.sh should honor BUILD_NUMBER > from environment (and use a reasonable default)" > (https://bugs.openjdk.java.net/browse/CODETOOLS-7902089) and came up > with the following fix: > > http://cr.openjdk.java.net/~simonis/webrevs/2017/CODETOOLS-7902089 > > As an extension I propose to set the build number by default to the > latest available tag automatically, even if we're building tip because > I think that's what users actually want (and need) if they build JTreg > for executing the OpenJDK tests. This feature is already realized in > my webrev. > > Finally, we could set the build number to something like "b11+" if > "b11" was the latest tagged build, but we're several changes ahead > already (this would resemble the "hg id" output which prints a "+" > after the actual change hash if the working directory contains > additional local changes). > > I consider this to be the best solution, but in order to work we would > also have to update JTReg's build number parsing routine in > com.sun.javatest.regtest.tool.Version.getBuild() which currently only > accepts a plain build number (i.e. "b[0-9]+"). This feature is > currently not in my webrev but I'd be happy to add it if you'd > consider it useful. > > Can you please review and sponsor my change? > > Thank you and merry Christmas:) > Volker > > > > On Thu, Dec 21, 2017 at 2:12 AM, Jonathan Gibbons > wrote: > > My previous message was intended for Mani. I'm sorry I did not make that > > clearer. > > > > -- Jon > > > > On 12/20/2017 05:09 PM, Jonathan Gibbons wrote: > >> > >> I've posted a new tag (jtreg4.2-b11) on the jtreg repo, so I would > expect > >> to see a new build on your CI system soon. > >> > >> If you're using the new make/build-all.sh script, then everything should > >> build as intended. > >> > >> If you're not using build-all.sh, note that you need to build jtreg with > >> variables like TESTNG_HOME, JUNIT_HOME, ASMTOOLS_HOME etc defined at the > >> time you run "make". It is not enough to copy the corresponding jar > files > >> into the lib directory; they must also be put in the Class-Path entry > of the > >> jtreg.jar MANIFEST.MF file (and the makefiles will take care of doing > that.) > >> > >> It occurs to me you might still be using the (old) Ant build.xml file. > If > >> so, I recommend you convert to using the new build-all.sh script. > >> > >> -- Jon > >> > >> On 12/19/2017 03:33 PM, Mani Sarkar wrote: > >>> > >>> Hi Jon, > >>> > >>> Amended the scripts to use the latest build-all.sh from the tip, and > got > >>> this - https://ci.adoptopenjdk.net/view/Dependencies/job/jtreg/. > >>> > >>> Still using our old script to build from the last tag. > >>> > >>> Both the artifacts do contain asmtools.jar - let me know if it passes > >>> your sanity check. > >>> > >>> Thanks again. > >>> > >>> Cheers, > >>> Mani > >>> > >>> On Mon, 18 Dec 2017 at 10:18 Maurizio Cimadamore > >>> >> > >>> wrote: > >>> > >>> > >>> > >>> On 18/12/17 01:32, Jonathan Gibbons wrote: > >>> > Maurizio, > >>> > > >>> > If you can do this in a way that makes it optional, that would be > >>> > great. Having put effort into pruning the set of dependencies, > >>> I've > >>> > no desire to see the set creep up again unnecessarily, although I > >>> > admit the usefulness of the plugin. > >>> Message understood :-) > >>> > >>> Btw, I'm bringing this up because, currently, the only way to > >>> build the > >>> plugin is through the IDE itself, which, while not unreasonable > >>> (after > >>> all you have to have an IDE if you want to use the plugin :-)), it > >>> require some configuration and I've seen people getting stuck quite > >>> frequently - a build command would probably mitigate some of those > >>> concerns. > >>> > >>> Maurizio > >>> > > >>> > -- Jon > >>> > > >>> > > >>> > On 12/17/17 1:04 PM, Maurizio Cimadamore wrote: > >>> >> Looks great - thanks for doing this. > >>> >> > >>> >> If there's interest, I could also put some effort in order to > >>> >> integrate the plugins build into the system. In principle, it > >>> should > >>> >> be doable, by adding a bunch of env variables (to point at the > >>> IDEA > >>> >> runtime jar). Of course that would be an optional part of the > >>> build. > >>> >> > >>> >> Cheers > >>> >> Maurizio > >>> >> > >>> >> > >>> >> On 14/12/17 00:32, Jonathan Gibbons wrote: > >>> >>> This is for folk who are interested in building jtreg from > >>> source. > >>> >>> > >>> >>> As some of you have (rightfully) commented over the past years, > >>> >>> jtreg has not been an easy tool to build from source. > >>> >>> > >>> >>> And, as some of you may have noticed, there has been some > >>> amount of > >>> >>> activity over the past weeks and months to address this issue. > >>> This > >>> >>> work has been led by Erik Helin (thanks, Erik!) and we're now > >>> >>> getting to the point where we can show what we have been > working > >>> >>> towards. > >>> >>> > >>> >>> The core of the work to build jtreg is still the Makefiles as > >>> >>> before, although as was recently noted, we've been simplifying > >>> the > >>> >>> specification of the dependencies. > >>> >>> > >>> >>> Separately, Erik has helped provide updates to the way that > >>> some of > >>> >>> the Code Tools dependencies can be built. > >>> >>> > >>> >>> Building on all that work, we can now get to the next stage, to > >>> >>> provide a script that will download binaries for some > components > >>> >>> (JUnit, TestNG) and will download and build source for other > >>> >>> components (AsmTools, JCov, JTHarness), for which there are no > >>> >>> official binaries. > >>> >>> > >>> >>> To run the script, you just need to have Ant and a suitable > >>> "java" > >>> >>> on your path, and to specify the location of an install of JDK > >>> 1.8 > >>> >>> as an argument to the script. wget is used to download files, > >>> which > >>> >>> honors proxy settings for those that need to use them. The > >>> script is > >>> >>> deliberately fairly simple, and suitable for use in a CI > system. > >>> >>> > >>> >>> You can see a webrev for the script at > >>> >>> http://cr.openjdk.java.net/~jjg/7902083/webrev.00/ > >>> > >>> >>> > >>> >>> Example of use: > >>> >>> > >>> >>> $ which ant > >>> >>> /opt/ant/1.9.4/bin/ant > >>> >>> $ which java > >>> >>> /opt/jdk/1.8.0/bin/java > >>> >>> $ sh make/build-all.sh /opt/jdk/1.8.0 > >>> >>> ... build output ... > >>> >>> $ ls build/images/jtreg > >>> >>> bin COPYRIGHT doc legal lib LICENSE README release > >>> >>> $ > >>> >>> > >>> >>> > >>> >>> Once this settles down a bit, I'll update the public docs on > the > >>> >>> jtreg web pages. > >>> >>> > >>> >>> -- Jon > >>> >>> > >>> >>> > >>> >> > >>> > > >>> > >>> -- > >>> > >>> @theNeomatrix369 | Blog > >>> | @adoptopenjdk | Dev communities > >>> > >>> Meet-a-Project - MutabilityDetector > >>> | Github > >>> | Slideshare > >>> |LinkedIn > >>> > >>> > >>> Come to Devoxx UK 2018:http://www.devoxx.co.uk/ > >>> > >>> > >>> Don't chase success, rather aim for "Excellence", and success will come > >>> chasing after you! > >>> > >> > > > From jonathan.gibbons at oracle.com Fri Dec 22 19:40:14 2017 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Fri, 22 Dec 2017 11:40:14 -0800 Subject: CODETOOLS-7902083: Simplify building jtreg In-Reply-To: References: <5A31C69F.4090800@oracle.com> <7b8de36f-dbb2-9ddc-5f68-049c60bce314@oracle.com> <43aac4d2-b252-06ac-b900-bd6746013d47@oracle.com> <5A3B09B8.5050905@oracle.com> <5A3B0A7D.4030607@oracle.com> Message-ID: <5A3D5F9E.6080905@oracle.com> Volker, Thank you for noting the BUILD_NUMBER problem. I've pushed your fix (with a minor tweak to the syntax for consistency). I think there may be more changes to come, but your patch at least addresses the ability to pass through the BUILD_* info. I don't think that patching the code for the build number is the way to go. I think the "milestone" field is a better field to indicate deviations from a standard build. We might also want to look at aligning the syntax of tags and version strings with OpenJDK. Various folk have suggested some other minor improvements to the script in general, and I'll look at incorporating those changes as well. -- Jon On 12/22/2017 07:17 AM, Volker Simonis wrote: > Hi Jonathan, > > thanks a lot for improving the build. This is a huge step ahead and it > works great. I've just tried it and the build succeeded out of the > box! > > There's however one issue I still see. The default build will set the > version and build number to '4-2 dev b00'. Notice that such a build is > pretty useless, because such versions of JTreg will be rejected when > doing OpenJDK regression tests because the OpenJDK test have minimal > JTreg build requirements in their TEST.ROOT file like for example: > > # Minimum jtreg version > requiredVersion=4.2 b08 > > Before the introduction of build-all.sh it was possible to pass > BUILD_NUMBER through the environment to make. > > When using build-all.sh this is not possible any more, because > build-all.sh calls make without honoring if BUILD_NUMBER was set in > the environment. > > I've opened "CODETOOLS-7902089: build-all.sh should honor BUILD_NUMBER > from environment (and use a reasonable default)" > (https://bugs.openjdk.java.net/browse/CODETOOLS-7902089) and came up > with the following fix: > > http://cr.openjdk.java.net/~simonis/webrevs/2017/CODETOOLS-7902089 > > As an extension I propose to set the build number by default to the > latest available tag automatically, even if we're building tip because > I think that's what users actually want (and need) if they build JTreg > for executing the OpenJDK tests. This feature is already realized in > my webrev. > > Finally, we could set the build number to something like "b11+" if > "b11" was the latest tagged build, but we're several changes ahead > already (this would resemble the "hg id" output which prints a "+" > after the actual change hash if the working directory contains > additional local changes). > > I consider this to be the best solution, but in order to work we would > also have to update JTReg's build number parsing routine in > com.sun.javatest.regtest.tool.Version.getBuild() which currently only > accepts a plain build number (i.e. "b[0-9]+"). This feature is > currently not in my webrev but I'd be happy to add it if you'd > consider it useful. > > Can you please review and sponsor my change? > > Thank you and merry Christmas:) > Volker > > > > On Thu, Dec 21, 2017 at 2:12 AM, Jonathan Gibbons > wrote: >> My previous message was intended for Mani. I'm sorry I did not make that >> clearer. >> >> -- Jon >> >> On 12/20/2017 05:09 PM, Jonathan Gibbons wrote: >>> I've posted a new tag (jtreg4.2-b11) on the jtreg repo, so I would expect >>> to see a new build on your CI system soon. >>> >>> If you're using the new make/build-all.sh script, then everything should >>> build as intended. >>> >>> If you're not using build-all.sh, note that you need to build jtreg with >>> variables like TESTNG_HOME, JUNIT_HOME, ASMTOOLS_HOME etc defined at the >>> time you run "make". It is not enough to copy the corresponding jar files >>> into the lib directory; they must also be put in the Class-Path entry of the >>> jtreg.jar MANIFEST.MF file (and the makefiles will take care of doing that.) >>> >>> It occurs to me you might still be using the (old) Ant build.xml file. If >>> so, I recommend you convert to using the new build-all.sh script. >>> >>> -- Jon >>> >>> On 12/19/2017 03:33 PM, Mani Sarkar wrote: >>>> Hi Jon, >>>> >>>> Amended the scripts to use the latest build-all.sh from the tip, and got >>>> this - https://ci.adoptopenjdk.net/view/Dependencies/job/jtreg/. >>>> >>>> Still using our old script to build from the last tag. >>>> >>>> Both the artifacts do contain asmtools.jar - let me know if it passes >>>> your sanity check. >>>> >>>> Thanks again. >>>> >>>> Cheers, >>>> Mani >>>> >>>> On Mon, 18 Dec 2017 at 10:18 Maurizio Cimadamore >>>> > >>>> wrote: >>>> >>>> >>>> >>>> On 18/12/17 01:32, Jonathan Gibbons wrote: >>>> > Maurizio, >>>> > >>>> > If you can do this in a way that makes it optional, that would be >>>> > great. Having put effort into pruning the set of dependencies, >>>> I've >>>> > no desire to see the set creep up again unnecessarily, although I >>>> > admit the usefulness of the plugin. >>>> Message understood :-) >>>> >>>> Btw, I'm bringing this up because, currently, the only way to >>>> build the >>>> plugin is through the IDE itself, which, while not unreasonable >>>> (after >>>> all you have to have an IDE if you want to use the plugin :-)), it >>>> require some configuration and I've seen people getting stuck quite >>>> frequently - a build command would probably mitigate some of those >>>> concerns. >>>> >>>> Maurizio >>>> > >>>> > -- Jon >>>> > >>>> > >>>> > On 12/17/17 1:04 PM, Maurizio Cimadamore wrote: >>>> >> Looks great - thanks for doing this. >>>> >> >>>> >> If there's interest, I could also put some effort in order to >>>> >> integrate the plugins build into the system. In principle, it >>>> should >>>> >> be doable, by adding a bunch of env variables (to point at the >>>> IDEA >>>> >> runtime jar). Of course that would be an optional part of the >>>> build. >>>> >> >>>> >> Cheers >>>> >> Maurizio >>>> >> >>>> >> >>>> >> On 14/12/17 00:32, Jonathan Gibbons wrote: >>>> >>> This is for folk who are interested in building jtreg from >>>> source. >>>> >>> >>>> >>> As some of you have (rightfully) commented over the past years, >>>> >>> jtreg has not been an easy tool to build from source. >>>> >>> >>>> >>> And, as some of you may have noticed, there has been some >>>> amount of >>>> >>> activity over the past weeks and months to address this issue. >>>> This >>>> >>> work has been led by Erik Helin (thanks, Erik!) and we're now >>>> >>> getting to the point where we can show what we have been working >>>> >>> towards. >>>> >>> >>>> >>> The core of the work to build jtreg is still the Makefiles as >>>> >>> before, although as was recently noted, we've been simplifying >>>> the >>>> >>> specification of the dependencies. >>>> >>> >>>> >>> Separately, Erik has helped provide updates to the way that >>>> some of >>>> >>> the Code Tools dependencies can be built. >>>> >>> >>>> >>> Building on all that work, we can now get to the next stage, to >>>> >>> provide a script that will download binaries for some components >>>> >>> (JUnit, TestNG) and will download and build source for other >>>> >>> components (AsmTools, JCov, JTHarness), for which there are no >>>> >>> official binaries. >>>> >>> >>>> >>> To run the script, you just need to have Ant and a suitable >>>> "java" >>>> >>> on your path, and to specify the location of an install of JDK >>>> 1.8 >>>> >>> as an argument to the script. wget is used to download files, >>>> which >>>> >>> honors proxy settings for those that need to use them. The >>>> script is >>>> >>> deliberately fairly simple, and suitable for use in a CI system. >>>> >>> >>>> >>> You can see a webrev for the script at >>>> >>> http://cr.openjdk.java.net/~jjg/7902083/webrev.00/ >>>> >>>> >>> >>>> >>> Example of use: >>>> >>> >>>> >>> $ which ant >>>> >>> /opt/ant/1.9.4/bin/ant >>>> >>> $ which java >>>> >>> /opt/jdk/1.8.0/bin/java >>>> >>> $ sh make/build-all.sh /opt/jdk/1.8.0 >>>> >>> ... build output ... >>>> >>> $ ls build/images/jtreg >>>> >>> bin COPYRIGHT doc legal lib LICENSE README release >>>> >>> $ >>>> >>> >>>> >>> >>>> >>> Once this settles down a bit, I'll update the public docs on the >>>> >>> jtreg web pages. >>>> >>> >>>> >>> -- Jon >>>> >>> >>>> >>> >>>> >> >>>> > >>>> >>>> -- >>>> >>>> @theNeomatrix369 | Blog >>>> | @adoptopenjdk | Dev communities >>>> >>>> Meet-a-Project - MutabilityDetector >>>> | Github >>>> | Slideshare >>>> |LinkedIn >>>> >>>> >>>> Come to Devoxx UK 2018:http://www.devoxx.co.uk/ >>>> >>>> >>>> Don't chase success, rather aim for "Excellence", and success will come >>>> chasing after you! >>>> From volker.simonis at gmail.com Sat Dec 23 17:22:11 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Sat, 23 Dec 2017 17:22:11 +0000 Subject: CODETOOLS-7902083: Simplify building jtreg In-Reply-To: <5A3D5F9E.6080905@oracle.com> References: <5A31C69F.4090800@oracle.com> <7b8de36f-dbb2-9ddc-5f68-049c60bce314@oracle.com> <43aac4d2-b252-06ac-b900-bd6746013d47@oracle.com> <5A3B09B8.5050905@oracle.com> <5A3B0A7D.4030607@oracle.com> <5A3D5F9E.6080905@oracle.com> Message-ID: Thanks for pushing! Jonathan Gibbons schrieb am Fr. 22. Dez. 2017 um 20:44: > Volker, > > Thank you for noting the BUILD_NUMBER problem. > I've pushed your fix (with a minor tweak to the syntax for consistency). > I think there may be more changes to come, but your patch at least > addresses the ability to pass through the BUILD_* info. > > I don't think that patching the code for the build number is the way > to go. I think the "milestone" field is a better field to indicate > deviations from a standard build. We might also want to look at > aligning the syntax of tags and version strings with OpenJDK. > > Various folk have suggested some other minor improvements to the > script in general, and I'll look at incorporating those changes as well. > > -- Jon > > On 12/22/2017 07:17 AM, Volker Simonis wrote: > > Hi Jonathan, > > > > thanks a lot for improving the build. This is a huge step ahead and it > > works great. I've just tried it and the build succeeded out of the > > box! > > > > There's however one issue I still see. The default build will set the > > version and build number to '4-2 dev b00'. Notice that such a build is > > pretty useless, because such versions of JTreg will be rejected when > > doing OpenJDK regression tests because the OpenJDK test have minimal > > JTreg build requirements in their TEST.ROOT file like for example: > > > > # Minimum jtreg version > > requiredVersion=4.2 b08 > > > > Before the introduction of build-all.sh it was possible to pass > > BUILD_NUMBER through the environment to make. > > > > When using build-all.sh this is not possible any more, because > > build-all.sh calls make without honoring if BUILD_NUMBER was set in > > the environment. > > > > I've opened "CODETOOLS-7902089: build-all.sh should honor BUILD_NUMBER > > from environment (and use a reasonable default)" > > (https://bugs.openjdk.java.net/browse/CODETOOLS-7902089) and came up > > with the following fix: > > > > http://cr.openjdk.java.net/~simonis/webrevs/2017/CODETOOLS-7902089 > > > > As an extension I propose to set the build number by default to the > > latest available tag automatically, even if we're building tip because > > I think that's what users actually want (and need) if they build JTreg > > for executing the OpenJDK tests. This feature is already realized in > > my webrev. > > > > Finally, we could set the build number to something like "b11+" if > > "b11" was the latest tagged build, but we're several changes ahead > > already (this would resemble the "hg id" output which prints a "+" > > after the actual change hash if the working directory contains > > additional local changes). > > > > I consider this to be the best solution, but in order to work we would > > also have to update JTReg's build number parsing routine in > > com.sun.javatest.regtest.tool.Version.getBuild() which currently only > > accepts a plain build number (i.e. "b[0-9]+"). This feature is > > currently not in my webrev but I'd be happy to add it if you'd > > consider it useful. > > > > Can you please review and sponsor my change? > > > > Thank you and merry Christmas:) > > Volker > > > > > > > > On Thu, Dec 21, 2017 at 2:12 AM, Jonathan Gibbons > > wrote: > >> My previous message was intended for Mani. I'm sorry I did not make that > >> clearer. > >> > >> -- Jon > >> > >> On 12/20/2017 05:09 PM, Jonathan Gibbons wrote: > >>> I've posted a new tag (jtreg4.2-b11) on the jtreg repo, so I would > expect > >>> to see a new build on your CI system soon. > >>> > >>> If you're using the new make/build-all.sh script, then everything > should > >>> build as intended. > >>> > >>> If you're not using build-all.sh, note that you need to build jtreg > with > >>> variables like TESTNG_HOME, JUNIT_HOME, ASMTOOLS_HOME etc defined at > the > >>> time you run "make". It is not enough to copy the corresponding jar > files > >>> into the lib directory; they must also be put in the Class-Path entry > of the > >>> jtreg.jar MANIFEST.MF file (and the makefiles will take care of doing > that.) > >>> > >>> It occurs to me you might still be using the (old) Ant build.xml > file. If > >>> so, I recommend you convert to using the new build-all.sh script. > >>> > >>> -- Jon > >>> > >>> On 12/19/2017 03:33 PM, Mani Sarkar wrote: > >>>> Hi Jon, > >>>> > >>>> Amended the scripts to use the latest build-all.sh from the tip, and > got > >>>> this - https://ci.adoptopenjdk.net/view/Dependencies/job/jtreg/. > >>>> > >>>> Still using our old script to build from the last tag. > >>>> > >>>> Both the artifacts do contain asmtools.jar - let me know if it passes > >>>> your sanity check. > >>>> > >>>> Thanks again. > >>>> > >>>> Cheers, > >>>> Mani > >>>> > >>>> On Mon, 18 Dec 2017 at 10:18 Maurizio Cimadamore > >>>> maurizio.cimadamore at oracle.com>> > >>>> wrote: > >>>> > >>>> > >>>> > >>>> On 18/12/17 01:32, Jonathan Gibbons wrote: > >>>> > Maurizio, > >>>> > > >>>> > If you can do this in a way that makes it optional, that would > be > >>>> > great. Having put effort into pruning the set of > dependencies, > >>>> I've > >>>> > no desire to see the set creep up again unnecessarily, > although I > >>>> > admit the usefulness of the plugin. > >>>> Message understood :-) > >>>> > >>>> Btw, I'm bringing this up because, currently, the only way to > >>>> build the > >>>> plugin is through the IDE itself, which, while not unreasonable > >>>> (after > >>>> all you have to have an IDE if you want to use the plugin :-)), > it > >>>> require some configuration and I've seen people getting stuck > quite > >>>> frequently - a build command would probably mitigate some of > those > >>>> concerns. > >>>> > >>>> Maurizio > >>>> > > >>>> > -- Jon > >>>> > > >>>> > > >>>> > On 12/17/17 1:04 PM, Maurizio Cimadamore wrote: > >>>> >> Looks great - thanks for doing this. > >>>> >> > >>>> >> If there's interest, I could also put some effort in order to > >>>> >> integrate the plugins build into the system. In principle, it > >>>> should > >>>> >> be doable, by adding a bunch of env variables (to point at the > >>>> IDEA > >>>> >> runtime jar). Of course that would be an optional part of the > >>>> build. > >>>> >> > >>>> >> Cheers > >>>> >> Maurizio > >>>> >> > >>>> >> > >>>> >> On 14/12/17 00:32, Jonathan Gibbons wrote: > >>>> >>> This is for folk who are interested in building jtreg from > >>>> source. > >>>> >>> > >>>> >>> As some of you have (rightfully) commented over the past > years, > >>>> >>> jtreg has not been an easy tool to build from source. > >>>> >>> > >>>> >>> And, as some of you may have noticed, there has been some > >>>> amount of > >>>> >>> activity over the past weeks and months to address this > issue. > >>>> This > >>>> >>> work has been led by Erik Helin (thanks, Erik!) and we're now > >>>> >>> getting to the point where we can show what we have been > working > >>>> >>> towards. > >>>> >>> > >>>> >>> The core of the work to build jtreg is still the Makefiles as > >>>> >>> before, although as was recently noted, we've been > simplifying > >>>> the > >>>> >>> specification of the dependencies. > >>>> >>> > >>>> >>> Separately, Erik has helped provide updates to the way that > >>>> some of > >>>> >>> the Code Tools dependencies can be built. > >>>> >>> > >>>> >>> Building on all that work, we can now get to the next stage, > to > >>>> >>> provide a script that will download binaries for some > components > >>>> >>> (JUnit, TestNG) and will download and build source for other > >>>> >>> components (AsmTools, JCov, JTHarness), for which there are > no > >>>> >>> official binaries. > >>>> >>> > >>>> >>> To run the script, you just need to have Ant and a suitable > >>>> "java" > >>>> >>> on your path, and to specify the location of an install of > JDK > >>>> 1.8 > >>>> >>> as an argument to the script. wget is used to download files, > >>>> which > >>>> >>> honors proxy settings for those that need to use them. The > >>>> script is > >>>> >>> deliberately fairly simple, and suitable for use in a CI > system. > >>>> >>> > >>>> >>> You can see a webrev for the script at > >>>> >>> http://cr.openjdk.java.net/~jjg/7902083/webrev.00/ > >>>> > >>>> >>> > >>>> >>> Example of use: > >>>> >>> > >>>> >>> $ which ant > >>>> >>> /opt/ant/1.9.4/bin/ant > >>>> >>> $ which java > >>>> >>> /opt/jdk/1.8.0/bin/java > >>>> >>> $ sh make/build-all.sh /opt/jdk/1.8.0 > >>>> >>> ... build output ... > >>>> >>> $ ls build/images/jtreg > >>>> >>> bin COPYRIGHT doc legal lib LICENSE README release > >>>> >>> $ > >>>> >>> > >>>> >>> > >>>> >>> Once this settles down a bit, I'll update the public docs on > the > >>>> >>> jtreg web pages. > >>>> >>> > >>>> >>> -- Jon > >>>> >>> > >>>> >>> > >>>> >> > >>>> > > >>>> > >>>> -- > >>>> > >>>> @theNeomatrix369 | Blog > >>>> | @adoptopenjdk | Dev > communities > >>>> > >>>> Meet-a-Project - MutabilityDetector > >>>> | Github > >>>> | Slideshare > >>>> |LinkedIn > >>>> > >>>> > >>>> Come to Devoxx UK 2018:http://www.devoxx.co.uk/ > >>>> > >>>> > >>>> Don't chase success, rather aim for "Excellence", and success will > come > >>>> chasing after you! > >>>> > > From volker.simonis at gmail.com Sat Dec 23 17:27:45 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Sat, 23 Dec 2017 17:27:45 +0000 Subject: CODETOOLS-7902083: Simplify building jtreg In-Reply-To: References: <5A31C69F.4090800@oracle.com> <7b8de36f-dbb2-9ddc-5f68-049c60bce314@oracle.com> <43aac4d2-b252-06ac-b900-bd6746013d47@oracle.com> <5A3B09B8.5050905@oracle.com> <5A3B0A7D.4030607@oracle.com> Message-ID: Martin Buchholz schrieb am Fr. 22. Dez. 2017 um 18:58: > I agree having to fiddle with the build number is a pain point, but it's > shared with openjdk proper. > > You can't rely on mercurial tags; code can get moved out of mercurial (hg > archive) or into a different source code system. > I agree, but this change should at least help the average user who starts by cloning the Mercurial repo. That doesn?t mean I?m not open for further improvements ;) > I would prefer BUILD_NUMBER checked in, and write a little script that > modified the source and hg tagged it at the same time. And then of course > check in the script! > > On Fri, Dec 22, 2017 at 7:17 AM, Volker Simonis > wrote: > >> Hi Jonathan, >> >> thanks a lot for improving the build. This is a huge step ahead and it >> works great. I've just tried it and the build succeeded out of the >> box! >> >> There's however one issue I still see. The default build will set the >> version and build number to '4-2 dev b00'. Notice that such a build is >> pretty useless, because such versions of JTreg will be rejected when >> doing OpenJDK regression tests because the OpenJDK test have minimal >> JTreg build requirements in their TEST.ROOT file like for example: >> >> # Minimum jtreg version >> requiredVersion=4.2 b08 >> >> Before the introduction of build-all.sh it was possible to pass >> BUILD_NUMBER through the environment to make. >> >> When using build-all.sh this is not possible any more, because >> build-all.sh calls make without honoring if BUILD_NUMBER was set in >> the environment. >> >> I've opened "CODETOOLS-7902089: build-all.sh should honor BUILD_NUMBER >> from environment (and use a reasonable default)" >> (https://bugs.openjdk.java.net/browse/CODETOOLS-7902089) and came up >> with the following fix: >> >> http://cr.openjdk.java.net/~simonis/webrevs/2017/CODETOOLS-7902089 >> >> As an extension I propose to set the build number by default to the >> latest available tag automatically, even if we're building tip because >> I think that's what users actually want (and need) if they build JTreg >> for executing the OpenJDK tests. This feature is already realized in >> my webrev. >> >> Finally, we could set the build number to something like "b11+" if >> "b11" was the latest tagged build, but we're several changes ahead >> already (this would resemble the "hg id" output which prints a "+" >> after the actual change hash if the working directory contains >> additional local changes). >> >> I consider this to be the best solution, but in order to work we would >> also have to update JTReg's build number parsing routine in >> com.sun.javatest.regtest.tool.Version.getBuild() which currently only >> accepts a plain build number (i.e. "b[0-9]+"). This feature is >> currently not in my webrev but I'd be happy to add it if you'd >> consider it useful. >> >> Can you please review and sponsor my change? >> >> Thank you and merry Christmas:) >> Volker >> >> >> >> On Thu, Dec 21, 2017 at 2:12 AM, Jonathan Gibbons >> wrote: >> > My previous message was intended for Mani. I'm sorry I did not make that >> > clearer. >> > >> > -- Jon >> > >> > On 12/20/2017 05:09 PM, Jonathan Gibbons wrote: >> >> >> >> I've posted a new tag (jtreg4.2-b11) on the jtreg repo, so I would >> expect >> >> to see a new build on your CI system soon. >> >> >> >> If you're using the new make/build-all.sh script, then everything >> should >> >> build as intended. >> >> >> >> If you're not using build-all.sh, note that you need to build jtreg >> with >> >> variables like TESTNG_HOME, JUNIT_HOME, ASMTOOLS_HOME etc defined at >> the >> >> time you run "make". It is not enough to copy the corresponding jar >> files >> >> into the lib directory; they must also be put in the Class-Path entry >> of the >> >> jtreg.jar MANIFEST.MF file (and the makefiles will take care of doing >> that.) >> >> >> >> It occurs to me you might still be using the (old) Ant build.xml >> file. If >> >> so, I recommend you convert to using the new build-all.sh script. >> >> >> >> -- Jon >> >> >> >> On 12/19/2017 03:33 PM, Mani Sarkar wrote: >> >>> >> >>> Hi Jon, >> >>> >> >>> Amended the scripts to use the latest build-all.sh from the tip, and >> got >> >>> this - https://ci.adoptopenjdk.net/view/Dependencies/job/jtreg/. >> >>> >> >>> Still using our old script to build from the last tag. >> >>> >> >>> Both the artifacts do contain asmtools.jar - let me know if it passes >> >>> your sanity check. >> >>> >> >>> Thanks again. >> >>> >> >>> Cheers, >> >>> Mani >> >>> >> >>> On Mon, 18 Dec 2017 at 10:18 Maurizio Cimadamore >> >>> > maurizio.cimadamore at oracle.com>> >> >>> wrote: >> >>> >> >>> >> >>> >> >>> On 18/12/17 01:32, Jonathan Gibbons wrote: >> >>> > Maurizio, >> >>> > >> >>> > If you can do this in a way that makes it optional, that would >> be >> >>> > great. Having put effort into pruning the set of dependencies, >> >>> I've >> >>> > no desire to see the set creep up again unnecessarily, although >> I >> >>> > admit the usefulness of the plugin. >> >>> Message understood :-) >> >>> >> >>> Btw, I'm bringing this up because, currently, the only way to >> >>> build the >> >>> plugin is through the IDE itself, which, while not unreasonable >> >>> (after >> >>> all you have to have an IDE if you want to use the plugin :-)), it >> >>> require some configuration and I've seen people getting stuck >> quite >> >>> frequently - a build command would probably mitigate some of those >> >>> concerns. >> >>> >> >>> Maurizio >> >>> > >> >>> > -- Jon >> >>> > >> >>> > >> >>> > On 12/17/17 1:04 PM, Maurizio Cimadamore wrote: >> >>> >> Looks great - thanks for doing this. >> >>> >> >> >>> >> If there's interest, I could also put some effort in order to >> >>> >> integrate the plugins build into the system. In principle, it >> >>> should >> >>> >> be doable, by adding a bunch of env variables (to point at the >> >>> IDEA >> >>> >> runtime jar). Of course that would be an optional part of the >> >>> build. >> >>> >> >> >>> >> Cheers >> >>> >> Maurizio >> >>> >> >> >>> >> >> >>> >> On 14/12/17 00:32, Jonathan Gibbons wrote: >> >>> >>> This is for folk who are interested in building jtreg from >> >>> source. >> >>> >>> >> >>> >>> As some of you have (rightfully) commented over the past >> years, >> >>> >>> jtreg has not been an easy tool to build from source. >> >>> >>> >> >>> >>> And, as some of you may have noticed, there has been some >> >>> amount of >> >>> >>> activity over the past weeks and months to address this issue. >> >>> This >> >>> >>> work has been led by Erik Helin (thanks, Erik!) and we're now >> >>> >>> getting to the point where we can show what we have been >> working >> >>> >>> towards. >> >>> >>> >> >>> >>> The core of the work to build jtreg is still the Makefiles as >> >>> >>> before, although as was recently noted, we've been simplifying >> >>> the >> >>> >>> specification of the dependencies. >> >>> >>> >> >>> >>> Separately, Erik has helped provide updates to the way that >> >>> some of >> >>> >>> the Code Tools dependencies can be built. >> >>> >>> >> >>> >>> Building on all that work, we can now get to the next stage, >> to >> >>> >>> provide a script that will download binaries for some >> components >> >>> >>> (JUnit, TestNG) and will download and build source for other >> >>> >>> components (AsmTools, JCov, JTHarness), for which there are no >> >>> >>> official binaries. >> >>> >>> >> >>> >>> To run the script, you just need to have Ant and a suitable >> >>> "java" >> >>> >>> on your path, and to specify the location of an install of JDK >> >>> 1.8 >> >>> >>> as an argument to the script. wget is used to download files, >> >>> which >> >>> >>> honors proxy settings for those that need to use them. The >> >>> script is >> >>> >>> deliberately fairly simple, and suitable for use in a CI >> system. >> >>> >>> >> >>> >>> You can see a webrev for the script at >> >>> >>> http://cr.openjdk.java.net/~jjg/7902083/webrev.00/ >> >>> >> >>> >>> >> >>> >>> Example of use: >> >>> >>> >> >>> >>> $ which ant >> >>> >>> /opt/ant/1.9.4/bin/ant >> >>> >>> $ which java >> >>> >>> /opt/jdk/1.8.0/bin/java >> >>> >>> $ sh make/build-all.sh /opt/jdk/1.8.0 >> >>> >>> ... build output ... >> >>> >>> $ ls build/images/jtreg >> >>> >>> bin COPYRIGHT doc legal lib LICENSE README release >> >>> >>> $ >> >>> >>> >> >>> >>> >> >>> >>> Once this settles down a bit, I'll update the public docs on >> the >> >>> >>> jtreg web pages. >> >>> >>> >> >>> >>> -- Jon >> >>> >>> >> >>> >>> >> >>> >> >> >>> > >> >>> >> >>> -- >> >>> >> >>> @theNeomatrix369 | Blog >> >>> | @adoptopenjdk | Dev >> communities >> >>> >> >>> Meet-a-Project - MutabilityDetector >> >>> | Github >> >>> | Slideshare >> >>> |LinkedIn >> >>> >> >>> >> >>> Come to Devoxx UK 2018:http://www.devoxx.co.uk/ >> >>> >> >>> >> >>> Don't chase success, rather aim for "Excellence", and success will >> come >> >>> chasing after you! >> >>> >> >> >> > >> > > From martinrb at google.com Wed Dec 27 23:17:13 2017 From: martinrb at google.com (Martin Buchholz) Date: Wed, 27 Dec 2017 15:17:13 -0800 Subject: CODETOOLS-7902083: Simplify building jtreg In-Reply-To: <5A3D5F9E.6080905@oracle.com> References: <5A31C69F.4090800@oracle.com> <7b8de36f-dbb2-9ddc-5f68-049c60bce314@oracle.com> <43aac4d2-b252-06ac-b900-bd6746013d47@oracle.com> <5A3B09B8.5050905@oracle.com> <5A3B0A7D.4030607@oracle.com> <5A3D5F9E.6080905@oracle.com> Message-ID: Because TEST.ROOT files contain lines like: requiredVersion=4.2 b11 the BUILD_NUMBER is effectively a public API that cannot be changed. The architected way might be to bump the "4.2" number every time there's a feature release, but it's probably too late for that. So I would still put BUILD_NUMBER = 11 into the Makefile every time it's tagged. Extra work for jjg, but only for jjg. On Fri, Dec 22, 2017 at 11:40 AM, Jonathan Gibbons < jonathan.gibbons at oracle.com> wrote: > Volker, > > Thank you for noting the BUILD_NUMBER problem. > I've pushed your fix (with a minor tweak to the syntax for consistency). > I think there may be more changes to come, but your patch at least > addresses the ability to pass through the BUILD_* info. > > I don't think that patching the code for the build number is the way > to go. I think the "milestone" field is a better field to indicate > deviations from a standard build. We might also want to look at > aligning the syntax of tags and version strings with OpenJDK. > > Various folk have suggested some other minor improvements to the > script in general, and I'll look at incorporating those changes as well. > > -- Jon > > > On 12/22/2017 07:17 AM, Volker Simonis wrote: > >> Hi Jonathan, >> >> thanks a lot for improving the build. This is a huge step ahead and it >> works great. I've just tried it and the build succeeded out of the >> box! >> >> There's however one issue I still see. The default build will set the >> version and build number to '4-2 dev b00'. Notice that such a build is >> pretty useless, because such versions of JTreg will be rejected when >> doing OpenJDK regression tests because the OpenJDK test have minimal >> JTreg build requirements in their TEST.ROOT file like for example: >> >> # Minimum jtreg version >> requiredVersion=4.2 b08 >> >> Before the introduction of build-all.sh it was possible to pass >> BUILD_NUMBER through the environment to make. >> >> When using build-all.sh this is not possible any more, because >> build-all.sh calls make without honoring if BUILD_NUMBER was set in >> the environment. >> >> I've opened "CODETOOLS-7902089: build-all.sh should honor BUILD_NUMBER >> from environment (and use a reasonable default)" >> (https://bugs.openjdk.java.net/browse/CODETOOLS-7902089) and came up >> with the following fix: >> >> http://cr.openjdk.java.net/~simonis/webrevs/2017/CODETOOLS-7902089 >> >> As an extension I propose to set the build number by default to the >> latest available tag automatically, even if we're building tip because >> I think that's what users actually want (and need) if they build JTreg >> for executing the OpenJDK tests. This feature is already realized in >> my webrev. >> >> Finally, we could set the build number to something like "b11+" if >> "b11" was the latest tagged build, but we're several changes ahead >> already (this would resemble the "hg id" output which prints a "+" >> after the actual change hash if the working directory contains >> additional local changes). >> >> I consider this to be the best solution, but in order to work we would >> also have to update JTReg's build number parsing routine in >> com.sun.javatest.regtest.tool.Version.getBuild() which currently only >> accepts a plain build number (i.e. "b[0-9]+"). This feature is >> currently not in my webrev but I'd be happy to add it if you'd >> consider it useful. >> >> Can you please review and sponsor my change? >> >> Thank you and merry Christmas:) >> Volker >> >> >> >> On Thu, Dec 21, 2017 at 2:12 AM, Jonathan Gibbons >> wrote: >> >>> My previous message was intended for Mani. I'm sorry I did not make that >>> clearer. >>> >>> -- Jon >>> >>> On 12/20/2017 05:09 PM, Jonathan Gibbons wrote: >>> >>>> I've posted a new tag (jtreg4.2-b11) on the jtreg repo, so I would >>>> expect >>>> to see a new build on your CI system soon. >>>> >>>> If you're using the new make/build-all.sh script, then everything should >>>> build as intended. >>>> >>>> If you're not using build-all.sh, note that you need to build jtreg with >>>> variables like TESTNG_HOME, JUNIT_HOME, ASMTOOLS_HOME etc defined at the >>>> time you run "make". It is not enough to copy the corresponding jar >>>> files >>>> into the lib directory; they must also be put in the Class-Path entry >>>> of the >>>> jtreg.jar MANIFEST.MF file (and the makefiles will take care of doing >>>> that.) >>>> >>>> It occurs to me you might still be using the (old) Ant build.xml file. >>>> If >>>> so, I recommend you convert to using the new build-all.sh script. >>>> >>>> -- Jon >>>> >>>> On 12/19/2017 03:33 PM, Mani Sarkar wrote: >>>> >>>>> Hi Jon, >>>>> >>>>> Amended the scripts to use the latest build-all.sh from the tip, and >>>>> got >>>>> this - https://ci.adoptopenjdk.net/view/Dependencies/job/jtreg/. >>>>> >>>>> Still using our old script to build from the last tag. >>>>> >>>>> Both the artifacts do contain asmtools.jar - let me know if it passes >>>>> your sanity check. >>>>> >>>>> Thanks again. >>>>> >>>>> Cheers, >>>>> Mani >>>>> >>>>> On Mon, 18 Dec 2017 at 10:18 Maurizio Cimadamore >>>>> >>>> >> >>>>> wrote: >>>>> >>>>> >>>>> >>>>> On 18/12/17 01:32, Jonathan Gibbons wrote: >>>>> > Maurizio, >>>>> > >>>>> > If you can do this in a way that makes it optional, that would >>>>> be >>>>> > great. Having put effort into pruning the set of dependencies, >>>>> I've >>>>> > no desire to see the set creep up again unnecessarily, although >>>>> I >>>>> > admit the usefulness of the plugin. >>>>> Message understood :-) >>>>> >>>>> Btw, I'm bringing this up because, currently, the only way to >>>>> build the >>>>> plugin is through the IDE itself, which, while not unreasonable >>>>> (after >>>>> all you have to have an IDE if you want to use the plugin :-)), it >>>>> require some configuration and I've seen people getting stuck >>>>> quite >>>>> frequently - a build command would probably mitigate some of those >>>>> concerns. >>>>> >>>>> Maurizio >>>>> > >>>>> > -- Jon >>>>> > >>>>> > >>>>> > On 12/17/17 1:04 PM, Maurizio Cimadamore wrote: >>>>> >> Looks great - thanks for doing this. >>>>> >> >>>>> >> If there's interest, I could also put some effort in order to >>>>> >> integrate the plugins build into the system. In principle, it >>>>> should >>>>> >> be doable, by adding a bunch of env variables (to point at the >>>>> IDEA >>>>> >> runtime jar). Of course that would be an optional part of the >>>>> build. >>>>> >> >>>>> >> Cheers >>>>> >> Maurizio >>>>> >> >>>>> >> >>>>> >> On 14/12/17 00:32, Jonathan Gibbons wrote: >>>>> >>> This is for folk who are interested in building jtreg from >>>>> source. >>>>> >>> >>>>> >>> As some of you have (rightfully) commented over the past >>>>> years, >>>>> >>> jtreg has not been an easy tool to build from source. >>>>> >>> >>>>> >>> And, as some of you may have noticed, there has been some >>>>> amount of >>>>> >>> activity over the past weeks and months to address this issue. >>>>> This >>>>> >>> work has been led by Erik Helin (thanks, Erik!) and we're now >>>>> >>> getting to the point where we can show what we have been >>>>> working >>>>> >>> towards. >>>>> >>> >>>>> >>> The core of the work to build jtreg is still the Makefiles as >>>>> >>> before, although as was recently noted, we've been simplifying >>>>> the >>>>> >>> specification of the dependencies. >>>>> >>> >>>>> >>> Separately, Erik has helped provide updates to the way that >>>>> some of >>>>> >>> the Code Tools dependencies can be built. >>>>> >>> >>>>> >>> Building on all that work, we can now get to the next stage, >>>>> to >>>>> >>> provide a script that will download binaries for some >>>>> components >>>>> >>> (JUnit, TestNG) and will download and build source for other >>>>> >>> components (AsmTools, JCov, JTHarness), for which there are no >>>>> >>> official binaries. >>>>> >>> >>>>> >>> To run the script, you just need to have Ant and a suitable >>>>> "java" >>>>> >>> on your path, and to specify the location of an install of JDK >>>>> 1.8 >>>>> >>> as an argument to the script. wget is used to download files, >>>>> which >>>>> >>> honors proxy settings for those that need to use them. The >>>>> script is >>>>> >>> deliberately fairly simple, and suitable for use in a CI >>>>> system. >>>>> >>> >>>>> >>> You can see a webrev for the script at >>>>> >>> http://cr.openjdk.java.net/~jjg/7902083/webrev.00/ >>>>> >>>>> >>> >>>>> >>> Example of use: >>>>> >>> >>>>> >>> $ which ant >>>>> >>> /opt/ant/1.9.4/bin/ant >>>>> >>> $ which java >>>>> >>> /opt/jdk/1.8.0/bin/java >>>>> >>> $ sh make/build-all.sh /opt/jdk/1.8.0 >>>>> >>> ... build output ... >>>>> >>> $ ls build/images/jtreg >>>>> >>> bin COPYRIGHT doc legal lib LICENSE README release >>>>> >>> $ >>>>> >>> >>>>> >>> >>>>> >>> Once this settles down a bit, I'll update the public docs on >>>>> the >>>>> >>> jtreg web pages. >>>>> >>> >>>>> >>> -- Jon >>>>> >>> >>>>> >>> >>>>> >> >>>>> > >>>>> >>>>> -- >>>>> >>>>> @theNeomatrix369 | Blog >>>>> | @adoptopenjdk | Dev communities >>>>> >>>>> Meet-a-Project - MutabilityDetector >>>>> | Github >>>>> | Slideshare >>>>> |LinkedIn >>>>> >>>>> >>>>> Come to Devoxx UK 2018:http://www.devoxx.co.uk/ >>>>> >>>>> >>>>> Don't chase success, rather aim for "Excellence", and success will come >>>>> chasing after you! >>>>> >>>>> >