Early Access Build Test Results

[Joe Darcy]

On 01/09/2013 02:03 PM, Jonathan Gibbons wrote:
> Rory,
>
> It is good to see that we are now able to publish Early Access Build 
> Test Results.
>
> What is being done to address the test failures that you report? 
> Ideally, test failures should have corresponding bugs filed on 
> JBS/bugs.sun.com.
>
> It would also be good to see the complete list of tests that did not 
> pass for a build. Right now, the numbers under Failed and Error do not 
> match your list of "known issues".  How about automatically publishing 
> tests listed in JTreport/text/summary.txt that are reported as 
> "Failed." or "Error."?
>
> -- Jon
>
>

Hello,

I agree it is very welcome to see the regression test results for builds.

I also agree with Jon that it would be very helpful to see the full 
summary.txt output files for the test runs.  Such files would allow 
developers to compare the test results of their own builds to the recent 
promoted builds.  As a point for comparison, when I was release manager 
of OpenJDK 6, I published the summary.txt files as well as the jtdiff 
output; for a few examples see:

https://blogs.oracle.com/darcy/entry/openjdk_6_b22_regression_test
https://blogs.oracle.com/darcy/entry/openjdk_6_b21_regression_test
https://blogs.oracle.com/darcy/entry/openjdk_6_b20_regression_test
https://blogs.oracle.com/darcy/entry/openjdk_6_b19_regression_test

It would would be useful to have persistent per-build pages to serve as 
an archive to test results over time.

Finally, how do the jtreg options used to generate the reported results 
compare to the jtreg options used in the "make test" target?

Thanks,

-Joe