Coming Soon: jtreg 6

[Jonathan Gibbons]

JDK folk,

This is a general heads-up that jtreg 6 is ready for use and that we 
should soon update JDK to use it.  There are not many changes to the 
jtreg tool itself; the most significant change in jtreg 6 is the updated 
versions of the TestNG and JUnit libraries, which in turn have some new 
dependencies of their own.

There is a new, improved script to drive the Makefile, for those that 
want to build jtreg themselves. The new build script (make/build.sh) 
separates the specification of the dependencies from the code used to 
download the dependencies. The script can also be configured to download 
from non-default locations, which may be useful for those behind 
corporate firewalls. And, as always, for those that want to control the 
downloads themselves, you can still drive the Makefile directly. In its 
simplest form, you can use the new script as follows:
     sh make/build.sh --jdk /path/to/JDK8

jtreg is also adopting the JDK version-numbering scheme, as defined in 
"JEP 223: New Version-String Scheme"[1], so the days of using the build 
number as a significant part of the version string are finally over.  
This version of jtreg is "6"; subsequent versions will be "6.1", "6.2", 
... "7", ... etc. As before, we will tag the jtreg repo to mark the 
commit used for each version.

jtreg uses TestNG 7.3.0 and JUnit 4.13.2, as defined in 
make/build-support/version-numbers. This version of TestNG depends on 
Google Guice, and JUnit depends on Hamcrest. Suitable version numbers 
for those libraries are also defined in that file.

TestNG 7.3.0 has a number of incompatibilities compared to the version 
we were previously using, including:

 1. The module name is now set explicitly to `org.testng` instead of
    allowing the default name `testng` to be used. This affects modular
    tests that explicitly `require` the TestNG jar as an automatic
    module and/or export packages to that module. This will require a
    small number of tests to be updated when we switch to the new version.

 2. It is no longer possible to declare static methods as test methods
    with the `@Test` annotation.

 3. The behavior of `@BeforeGroups` has changed and should generally be
    avoided in jtreg tests.

 4. TestNG checks a number of new system properties. This needs to be
    taken into account in tests that need to grant permission to access
    those properties.

All JDK tests affected by #2, #3, #4 have already been updated. Tests 
affected by #1 will be updated as part of the Pull Request to switch to 
using jtreg 6. We have done internal test runs of most test tiers and do 
not believe there are any more test failures related to the upgrade.

These incompatibilities mean that it will not be possible to use jtreg 6 
for older/update releases unless/until the changes to the affected tests 
are also backported. That decision is left to those managing those releases.

NOTE: Users building jtreg should NOT use TestNG 7.4.0 to run JDK tests. 
There is a bug[2] in TestNG 7.4.0 that breaks a number of JDK tests.

Finally, thanks to everyone who has helped thus far, including Mikael 
Vidstedt for initiating work on the new build script and other related 
improvements, and to the developers who have helped analyze and fix the 
issues cause by the TestNG upgrade.

-- Jon

[1] https://openjdk.java.net/jeps/223
[2] https://github.com/cbeust/testng/issues/2527
     https://github.com/cbeust/testng/issues/2540