CODETOOLS-7902083: Simplify building jtreg

Sun Dec 17 20:17:13 UTC 2017

Martin,

Thanks for the feedback and suggestions.

I'll look at incorporating the scripting changes, and will separately 
investigate moving jtreg onto newer versions of external components.

-- Jon

On 12/16/17 10:51 AM, Martin Buchholz wrote:
> Thanks for working on this.  I've written similar scripts to gather up 
> all dependencies and build.
>
> I suggest using wget's -q flag.
>
> I suggest creating little shell subroutines to check the checksums 
> (which print the actual checksum in case of mismatch) and to download 
> maven jars.
>
> Here's some random shell code I've written that could get reused:
>
> def_make_var() {
>   typeset -n varref="$1"
>   varref="$2"
>   MAKE_FLAGS+=("$1=$2")
> }
>
> def_make_file() {
>   [[ -f "$2" ]] || die "%s: no such file" "$2"
>   def_make_var "$@"
> }
>
> def_make_dir() {
>   [[ -d "$2" ]] || die "%s: no such directory" "$2"
>   def_make_var "$@"
> }
>
> download_maven_jar() {
>   local -r groupId="$1" artifactId="$2" version="$3"
>   local -r jarfile="${artifactId}-${version}.jar"
>   wget -qO"/tmp/$jarfile" \
>     
> "http://search.maven.org/remotecontent?filepath=${groupId}/${artifactId}/${version}/${jarfile} 
> <http://search.maven.org/remotecontent?filepath=$%7BgroupId%7D/$%7BartifactId%7D/$%7Bversion%7D/$%7Bjarfile%7D>"
>   def_make_file "${artifactId^^}_JAR" "/tmp/$jarfile"
>   DOWNLOADED_MAVEN_JARS+=("/tmp/$jarfile")
> }
>
>
>
> On Sat, Dec 16, 2017 at 9:55 AM, Martin Buchholz <martinrb at google.com 
> <mailto:martinrb at google.com>> wrote:
>
>     testng-6.9.5 was packaged poorly, with jdk8 classfiles.
>     testng-6.9.8 had different packaging problems.
>     testng-6.9.9 is golden, with jdk7 class files.
>     (of course, there are also more recent testng versions)
>
>     --- a/make/build-all.sh
>     +++ b/make/build-all.sh
>     @@ -172,12 +172,12 @@
>      TESTNG_DEPS_DIR=${JTREG_DEPS_DIR}/testng
>      mkdir ${TESTNG_DEPS_DIR}
>     -TESTNG_JAR=${TESTNG_DEPS_DIR}/testng-6.9.5.jar
>     -wget ${MAVEN_REPO_URL}/org/testng/testng/6.9.5/testng-6.9.5.jar
>     -O ${TESTNG_JAR}
>     -printf "5d12ea207fc47c3f341a3f8ecc88a3eac396a777 ${TESTNG_JAR}" |
>     shasum -a 1 --check -
>     +TESTNG_JAR=${TESTNG_DEPS_DIR}/testng-6.9.9.jar
>     +wget ${MAVEN_REPO_URL}/org/testng/testng/6.9.9/testng-6.9.9.jar
>     -O ${TESTNG_JAR}
>     +printf "8b211c4c5bed389e169a2173da32d07f93ee2fa8 ${TESTNG_JAR}" |
>     shasum -a 1 --check -
>      TESTNG_LICENSE=${TESTNG_DEPS_DIR}/LICENSE.txt
>     -wget
>     https://raw.githubusercontent.com/cbeust/testng/testng-6.9.5/LICENSE.txt
>     <https://raw.githubusercontent.com/cbeust/testng/testng-6.9.5/LICENSE.txt>
>     -O ${TESTNG_LICENSE}
>     +wget
>     https://raw.githubusercontent.com/cbeust/testng/6.9.9/LICENSE.txt
>     <https://raw.githubusercontent.com/cbeust/testng/6.9.9/LICENSE.txt>
>     -O ${TESTNG_LICENSE}
>      JCOMMANDER_JAR=${TESTNG_DEPS_DIR}/jcommander-1.72.jar
>      wget
>     ${MAVEN_REPO_URL}/com/beust/jcommander/1.72/jcommander-1.72.jar -O
>     ${JCOMMANDER_JAR}
>
>
>     On Fri, Dec 15, 2017 at 2:34 PM, Jonathan Gibbons
>     <jonathan.gibbons at oracle.com <mailto:jonathan.gibbons at oracle.com>>
>     wrote:
>
>         I've pushed the changeset containing this new script.
>
>         There are two changes since I posted the webrev:
>
>         1. Change to use "shasum -a 1" instead of "sha1sum"
>         2. Update script to download and use jcommander.jar in
>         conjunction with TestNG.
>
>         With the second change in particular, all the jtreg self-tests
>         pass, when the image is built with the build-all.sh script.
>
>         -- Jon
>
>
>         On 12/13/2017 04:32 PM, Jonathan Gibbons wrote:
>
>             This is for folk who are interested in building jtreg from
>             source.
>
>             As some of you have (rightfully) commented over the past
>             years, jtreg has not been an easy tool to build from source.
>
>             And, as some of you may have noticed, there has been some
>             amount of activity over the past weeks and months to
>             address this issue. This work has been led by Erik Helin
>             (thanks, Erik!) and we're now getting to the point where
>             we can show what we have been working towards.
>
>             The core of the work to build jtreg is still the Makefiles
>             as before, although as was recently noted, we've been
>             simplifying the specification of the dependencies.
>
>             Separately, Erik has helped provide updates to the way
>             that some of the Code Tools dependencies can be built.
>
>             Building on all that work, we can now get to the next
>             stage, to provide a script that will download binaries for
>             some components (JUnit, TestNG) and will download and
>             build source for other components (AsmTools, JCov,
>             JTHarness), for which there are no official binaries.
>
>             To run the script, you just need to have Ant and a
>             suitable "java" on your path, and to specify the location
>             of an install of JDK 1.8 as an argument to the script.
>             wget is used to download files, which honors proxy
>             settings for those that need to use them. The script is
>             deliberately fairly simple, and suitable for use in a CI
>             system.
>
>             You can see a webrev for the script at
>             http://cr.openjdk.java.net/~jjg/7902083/webrev.00/
>             <http://cr.openjdk.java.net/%7Ejjg/7902083/webrev.00/>
>
>             Example of use:
>
>             $ which ant
>             /opt/ant/1.9.4/bin/ant
>             $ which java
>             /opt/jdk/1.8.0/bin/java
>             $ sh make/build-all.sh /opt/jdk/1.8.0
>             ... build output ...
>             $ ls build/images/jtreg
>             bin  COPYRIGHT  doc  legal  lib  LICENSE README  release
>             $
>
>
>             Once this settles down a bit, I'll update the public docs
>             on the jtreg web pages.
>
>             -- Jon
>
>
>
>
>