jtreg testing integrated

[Martin Buchholz]

On Tue, May 20, 2008 at 2:32 AM, Mark Wielaard <mark at klomp.org> wrote:
>> I like a policy of "Read my lips; no new test failures" but OpenJDK
>> is not quite there; we get test failure creep when changes in
>> one component break another component's tests.
>
> Yes, that would be idea. At least for openjdk6/icedtea we seem to be
> pretty close actually. It will be more challenging for openjdk7. I
> haven't quite figured out all the dynamics around "workspace
> integration". But I assume we can get the master tree to zero fail and
> then demand that any integration cycle doesn't introduce regressions.

There are too many tests to require team integrators to run
them all on each integration cycle.  For a few years I've advocated
adding another level to the tree of workspaces.  My model is to
rename the current MASTER workspace to PURGATORY, and
add a "golden MASTER".
The idea is that once a week or so all tests are run exhaustively,
and when it is confirmed that there are no new test failures,
the tested code from PURGATORY is promoted to MASTER.

> That brings up the question how to export a JTreport/JTwork environment.
> I only posted the text results http://icedtea.classpath.org/~mjw/jtreg/
> since the JTreport and JTwork files have all kinds of hard coded
> absolute path references. It would be nice to be able to export it all
> so I can upload it to some public site for others to look at and compare
> with.

My (unavailable) diff-javatest script had to contend with absolute
paths in the html in the report directory as well.  It made paths
relative by removing root dirs.  It would be good if javatest's output
was made more "portable" in this sense.  It's hard, because you
really do want direct pointers to failing tests and .jtr files, and
their location relative to the report directory cannot in general
be relativized.

Martin