[aarch64-port-dev ] RFR: Backports from jdk9 to jdk8

[Edward Nevill]

On Thu, 2015-07-02 at 13:40 +0100, Andrew Haley wrote:
> On 07/02/2015 01:04 PM, Edward Nevill wrote:
> 
> before
> 
> > JDK: Test results: passed: 5,526; failed: 224; error: 11
> 
> after
> 
> > JDK: Test results: passed: 5,520; failed: 224; error: 16
> 
> ?

The number of failures/errors on jdk varies with every run. For example, I just ran the above two tests again and I got

before

Test results: passed: 5,519; failed: 229; error: 12

after

Test results: passed: 5,516; failed: 230; error: 14

Even when the number of failing tests is the same, the actual set of failing tests is different. For example, in the case above where the no of failures is reported as 224, if I sort and diff the log files I get the output below which shows the set of failing tests is completely different.

http://people.linaro.org/~edward.nevill/sorted_log_diffs

This rather limits the usefulness of the jdk test suite. The only thing I use it for is as statistical evidence that it isn't completely broken.

All the best,
Ed.