Early Access Build Test Results

[Rory O'Donnell Oracle, Dublin Ireland]

Hi Jon, Joe

Thank you both for your feedback.

We will look to post summary.txt in the coming weeks. Secondly, we will 
look at archiving a summary.txt per build.
Balchandra will look at the jtreg options and come back to you on that.

Will update you on our progress.

Rgds, Rory


On 10/01/2013 00:21, Joe Darcy wrote:
> On 01/09/2013 02:03 PM, Jonathan Gibbons wrote:
>> Rory,
>>
>> It is good to see that we are now able to publish Early Access Build 
>> Test Results.
>>
>> What is being done to address the test failures that you report? 
>> Ideally, test failures should have corresponding bugs filed on 
>> JBS/bugs.sun.com.
>>
>> It would also be good to see the complete list of tests that did not 
>> pass for a build. Right now, the numbers under Failed and Error do 
>> not match your list of "known issues".  How about automatically 
>> publishing tests listed in JTreport/text/summary.txt that are 
>> reported as "Failed." or "Error."?
>>
>> -- Jon
>>
>>
>
> Hello,
>
> I agree it is very welcome to see the regression test results for builds.
>
> I also agree with Jon that it would be very helpful to see the full 
> summary.txt output files for the test runs.  Such files would allow 
> developers to compare the test results of their own builds to the 
> recent promoted builds.  As a point for comparison, when I was release 
> manager of OpenJDK 6, I published the summary.txt files as well as the 
> jtdiff output; for a few examples see:
>
> https://blogs.oracle.com/darcy/entry/openjdk_6_b22_regression_test
> https://blogs.oracle.com/darcy/entry/openjdk_6_b21_regression_test
> https://blogs.oracle.com/darcy/entry/openjdk_6_b20_regression_test
> https://blogs.oracle.com/darcy/entry/openjdk_6_b19_regression_test
>
> It would would be useful to have persistent per-build pages to serve 
> as an archive to test results over time.
>
> Finally, how do the jtreg options used to generate the reported 
> results compare to the jtreg options used in the "make test" target?
>
> Thanks,
>
> -Joe
>

-- 
Rgds,
Rory O'Donnell

Senior Quality Engineering Manager
Java Development Group
Oracle EMEA, Block P5,
East Point Business Park, Dublin 3
Phone: +353 (0)1 8033887