First pass jtreg test results

[Volker Simonis]

Hi Steve,

sorry for the delay. I've finally found some time to look at your test
results.

First of all I think the presentation of the results is great!

Could you please explain in some more detail how you ran the tests?

I tried my self today on Linux/PPC64 with the same setup as suggested on
http://download.java.net/jdk8/testresults/docs/howtoruntests.html (which
isn't online now but which should be available again real soon).

One thing I found out is that you shouldn't use '-conc:auto' because
Linux/PPC boxes tend to report quite a significant number of CPUS (32 on my
box) and '-conc:auto' means that the harness will start <nr. of CPUs>
agents which can easily lead to the "out of memory" and "can not create
native thread" errors you see in your tests. Setting '-conc:4' for example
should give you much better results. I got the following result for the
test set listed below:

*Test results: passed: 3,255; failed: 415; error: 26*

I'll post my results for Oracle and OpenJDK on x86 tomorrow,
Regards,
Volker

PS: this is the command line I'm currently using:

/priv/openjdk/OpenJDK/jtreg/linux/bin/jtreg
-dir:/priv/openjdk/OpenJDK/jdk7u/jdk/test -verbose:summary
-exclude:/priv/openjdk/OpenJDK/jdk7u/jdk/test/ProblemList.txt -conc:8 -a
-ignore:quiet -timeoutFactor:5 -agentvm
-testjdk:/priv/openjdk/OpenJDK/j2sdk-image `cat
/priv/openjdk/OpenJDK/dir.list`

and the file dir.list contains the following tests:

sun/security sun/misc sun/tools sun/pisces sun/util sun/nio sun/invoke
sun/rmi sun/text sun/management sun/jvmstat sun/net java/security java/io
java/util java/nio java/lang java/beans java/rmi java/text java/math
java/net tools com/sun com/oracle sun/misc sun/tools sun/pisces sun/util
sun/nio sun/invoke sun/rmi sun/text sun/management sun/jvmstat sun/net demo
sample vm javax/security javax/naming javax/smartcardio javax/accessibility
javax/print javax/crypto javax/xml javax/imageio javax/sound javax/rmi
javax/management javax/script




On Sun, Jan 20, 2013 at 4:24 PM, Steve Poole <spoole at linux.vnet.ibm.com>wrote:

> hi - all .
>
> After a lot of effort spinning wheels and dealing with some hardware
> issues I can finally announce the public appearance of ppcaix test results
>
> The data is hosted off my dropbox account starting here ->
> https://dl.dropbox.com/u/7639904/openjdk/ppcaix/testruns/index.html
>
>
> Now for the caveats..
>
> 1 - This is "hot off the press" so expect errors / issues / omissions etc
>
> 2 - The tests themselves are the result of running the jtreg tests in the
> jck repo.  I've done very little triaging of the results so the high number
> of failures will probably quickly descend with a little attention.
>
> 3 - I'm missing the results for the linux intel 64 bit runs I have (server
> is down - hope to get them published sometime next week)
>
>
> And the questions..
>
> A - Are these results ok in terms of available data?  I could publish the
> junit versions of the data too but I'm not sure that its worthwhile
>
> B - I'm keen to make it very easy to compare results across the various
> runs.  I'd like to be able to pick a testcase and compare the detailed
> results side by side.  Will that be useful or is what we have enough for
> now?
>
> C - How often should we refresh this data?    Once a day or once a week?
>
> D -  What about running  / publishing  jtreg tests for the hotspot repo?
>
> Cheers
>
> Steve
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/attachments/20130123/109ecc84/attachment.html