JDK 8 b118 ea test results are now available

[Volker Simonis]

I fully agree! It would be nice to have a single test group and a
single make target which could be run by everybody who succeeded to
build the OpenJDK. And it should produce the same results like they
are posted on http://www.java.net/download/jdk8/testresults/testresults.html
. Actually, the tests results posted there should be run with the same
make target and test group.

It's nice to have a test description an a website but reality shows
that it is inevitable that the web page and the real scripts will get
out of sync. This can not happen if you have a checked in make-file
and group description.

Of course it is also time to add test results for the other Oracle
supported platforms (Mac, Win and Solaris) to
http://www.java.net/download/jdk8/testresults/testresults.html. In my
eyes, this is an absolute requirement if we want to get a stable set
of tests which can be used to verify something like a port (or some
other bigger changes). Concentrating on Linux will not help a lot
here.

The test group would be a running target of course and everybody who
fixes and/or stabilizes a test should update it.

Regards,
Volker

On Thu, Dec 12, 2013 at 12:10 AM, Jonathan Gibbons
<jonathan.gibbons at oracle.com> wrote:
>
> On 12/11/2013 11:54 AM, Alan Bateman wrote:
>
> On 11/12/2013 19:26, Balchandra Vaidya wrote:
>
>
> :
>
> Alternatively, that segment of the script could be a candidate for a target
> in
> one or more test/Makefile files.
>
>
> This is good idea, but my experience with the 'make' is that if one target
> critically fail, all
> subsequent targets will not run. I thought it is a restriction of 'make'.
>
> I think Jon is suggesting that the subset that you run be added to
> TEST.groups (which will automatically turn it a make targe as way of the
> jdk_% rule ). On the surface this is a good idea but when I look at the
> subset of the tests that you are running:
>
> :jdk_core
> :jdk_svc
> :jdk_beans
> :jdk_imageio
> :jdk_sound
> :jdk_sctp
> javax/accessibility
> com/sun/java/swing
> javax/print
> sun/pisces
> com/sun/awt
>
> then it's a bit ad hoc. I wouldn't object to adding a special group for this
> but it really amounts to all jdk tests except for:
>
> java/awt
> javax/swing
> sun/awt
> sun/java2d
> com/apple/eawt
>
> If these tests aren't in your runs because of stability issues then we
> should make sure that there are bugs submitted and that they get some focus.
> In the interim then unstable tests can be @ignore-d or added to the exclude
> list. I initially thought that part of the issue was the othervm vs. agentvm
> discussion but I see in TEST.ROOT that othervm.dirs lists these directories
> already.
>
> -Alan
>
>
>
> Yes, the high order point I was trying to make is that something is wrong if
> you need to specify a long list of tests to run.   While we all may take
> whatever short cuts we choose to get our day to day work done, there should
> be a standard set of tests[1] that we agree should be run, and which can be
> run with reasonably concise command line args.
>
> -- Jon
>
> Ideally, "all" but maybe we're not there yet.
>
>