testing a openjdk8u212-b03 on Solaris SPARC 10

[Erik Joelsson]

Hello Matthias,

On 2019-05-24 05:14, Matthias Apitz wrote:
> El día viernes, mayo 24, 2019 a las 11:55:09a. m. +0200, Matthias Apitz escribió:
>
>> I tried to use the JT regression test tool jtreg-4.2-b14.tar.gz the
>> following way (as explained in its docs:
>> http://openjdk.java.net/jtreg/runtests.html
>>
>>
>>      $ export JT_HOME=/export/home/sisis/guru/jtreg
>>      $ export JT_JAVA=/usr/local/sisis-pap/jdk1.8.0_31/bin/java
>>      $ export PRODUCT_HOME=/export/home/sisis/guru/jdk8u212-b03/build/solaris-sparcv9-normal-server-release/images/j2sdk-image
>>
>> (btw: The script has a bug: the shebang must be modified to '/usr/bin/bash')
>>
>>      $ cd test
>>      $ nohup /usr/sfw/bin/gmake all
> I followed the hint in http://openjdk.java.net/jtreg/runtests.html  to
> use the new built JDK for the testing itself. I set:
>
>     $ export JT_JAVA=/export/home/sisis/guru/jdk8u212-b03/build/solaris-sparcv9-normal-server-release/images/j2sdk-image/bin/java
For JT_JAVA, you generally want the JDK you trust the most, which is 
usually the boot jdk.
> Then I looked in the test/Makefile and started not 'all', but 'default',
> i.e.
>
>     $ nohup /usr/sfw/bin/gmake all
Running the 'all' target is certainly not expected to pass. Test 
stability in general is something that we put a lot of effort into in 
JDK 9 and up, but even so, I would not expect all tests to pass in 
current mainline either. We have sub selections of tests that are 
considered more or less stable.
> 'default' means as target: jdk_core langtools_jtreg
> and these test until now are all passed with 'Execution successful'
> Is this good enough as a test?

Building from source, using a different compiler version and a different 
environment than has been thoroughly tested is always a risk. We can of 
course not say if anything is good enough for you, that is ultimately 
your responsibility to figure out.

That said, I don't know what a good selection would be. I do think you 
should include something from hotspot as well as that does not seem to 
be included in the default target. Severin is putting together some 
basic set of tests for jdk8u here [1] so perhaps you could look into his 
choices there. There is unfortunately no baseline test results available 
that I know of, so there really is no way to know if any test failure 
you see is significant.

Ultimately, what should matter is what you need to run, so I would 
recommend testing the software you intend to use this JDK with.

[1] http://mail.openjdk.java.net/pipermail/build-dev/2019-May/025585.html

/Erik