JTreg upwards compatibility?

[Jonathan Gibbons]

On 05/12/2016 08:10 AM, Volker Simonis wrote:
> Hi,
>
> I wanted to ask if JTreg is supposed to be upwards compatible.
>
> I.e. should it be possible to run OpenJDK 7 and 8 regression tests
> with the newest JTreg version?
>
> If not, is there a source which describes the required JTreg version
> for successfully running the regression tests of the various OpenJDK
> releases? Which JTreg versions is Oracle using for running its OpenJDK
> 7 and 8 regression tests?
>
> Thank you and best regards,
> Volker

Generally, the answer to your question is "yes".

This is one of the reasons why we abandoned "samevm" mode
a year or more ago, in favor of "agentvm" mode".

The general policy statement is as follows:

1. jtreg can run tests all the way back to JDK 1.1 in othervm mode.
     (Although this is not routinely tested, these days.)

2. jtreg can run tests all the way back to JDK 5 in agentvm mode,
     although some tests use TestNG and JUnit, and the versions
     of these libraries recommended for use with JDK 9 tests require
     the use of JDK 7 or greater.  However, I do not believe any of
     the standard JDK tests for OpenJDK 6 use TestNG or JUnit, so
     this should not be an issue.  If it is, there are ways to override
     the versions of these libraries that are bundled with a jtreg build.

There is a potential issue that can reasonably be addressed.  If you
run jtreg through the <repo>/test/Makefile mechanisms, the standard
invocation is to use the same build/version of JDK to run jtreg as the
one being tested. This was expedient at the time, although not the
best practice.  The current recommendation is to use a recent stable
build of JDK to *run* jtreg, that is distinct from the version of JDK
being tested.   If you use <repo>/test/Makefile for JDK 6, this may
require some changes to the makefiles.

There is discussion about abandoning support for running tests on very
old versions of JDK, such as for JDK 5 and earlier, and at the same time,
raising the bar for the version used to *run* jtreg up to the latest
supported version (currently JDK 8.)  This would not affect the ability
to be able to run tests in othervm mode or agentvm mode on all
supported releases. One of the primary concerns is whether the latest
supported version (i.e. JDK 8) can run on all the hardware/OS
combinations used to test the oldest supported versions. However,
this is becoming less of an issue these days, since JDK 8 does appear
to be available on all relevant platforms.

There is a general desire within the JDK team to use a single version of
jtreg to run tests on all supported versions of JDK, and not to have to 
freeze
or fork different versions of jtreg to run tests on different releases.

-- Jon