First pass jtreg test results

[Steve Poole]

Thanks Volker - I was running with 8 not 4 so I've turned that down and am running the tests again

On 23 Jan 2013, at 17:44, Volker Simonis <volker.simonis at gmail.com> wrote:

> Hi Steve,
> 
> sorry for the delay. I've finally found some time to look at your test results.
> 
> First of all I think the presentation of the results is great!
> 
> Could you please explain in some more detail how you ran the tests?
> 
> I tried my self today on Linux/PPC64 with the same setup as suggested on http://download.java.net/jdk8/testresults/docs/howtoruntests.html (which isn't online now but which should be available again real soon).
> 
> One thing I found out is that you shouldn't use '-conc:auto' because Linux/PPC boxes tend to report quite a significant number of CPUS (32 on my box) and '-conc:auto' means that the harness will start <nr. of CPUs> agents which can easily lead to the "out of memory" and "can not create native thread" errors you see in your tests. Setting '-conc:4' for example should give you much better results. I got the following result for the test set listed below:
> 
> Test results: passed: 3,255; failed: 415; error: 26
> 
> I'll post my results for Oracle and OpenJDK on x86 tomorrow,
> Regards,
> Volker
> 
> PS: this is the command line I'm currently using:
> 
> /priv/openjdk/OpenJDK/jtreg/linux/bin/jtreg -dir:/priv/openjdk/OpenJDK/jdk7u/jdk/test -verbose:summary -exclude:/priv/openjdk/OpenJDK/jdk7u/jdk/test/ProblemList.txt -conc:8 -a -ignore:quiet -timeoutFactor:5 -agentvm -testjdk:/priv/openjdk/OpenJDK/j2sdk-image `cat /priv/openjdk/OpenJDK/dir.list`
> 
> and the file dir.list contains the following tests:
> 
> sun/security sun/misc sun/tools sun/pisces sun/util sun/nio sun/invoke sun/rmi sun/text sun/management sun/jvmstat sun/net java/security java/io java/util java/nio java/lang java/beans java/rmi java/text java/math java/net tools com/sun com/oracle sun/misc sun/tools sun/pisces sun/util sun/nio sun/invoke sun/rmi sun/text sun/management sun/jvmstat sun/net demo sample vm javax/security javax/naming javax/smartcardio javax/accessibility javax/print javax/crypto javax/xml javax/imageio javax/sound javax/rmi javax/management javax/script
> 
> 
> 
> 
> On Sun, Jan 20, 2013 at 4:24 PM, Steve Poole <spoole at linux.vnet.ibm.com> wrote:
> hi - all .
> 
> After a lot of effort spinning wheels and dealing with some hardware issues I can finally announce the public appearance of ppcaix test results
> 
> The data is hosted off my dropbox account starting here ->  https://dl.dropbox.com/u/7639904/openjdk/ppcaix/testruns/index.html
> 
> 
> Now for the caveats..
> 
> 1 - This is "hot off the press" so expect errors / issues / omissions etc
> 
> 2 - The tests themselves are the result of running the jtreg tests in the jck repo.  I've done very little triaging of the results so the high number of failures will probably quickly descend with a little attention.
> 
> 3 - I'm missing the results for the linux intel 64 bit runs I have (server is down - hope to get them published sometime next week)
> 
> 
> And the questions..
> 
> A - Are these results ok in terms of available data?  I could publish the junit versions of the data too but I'm not sure that its worthwhile
> 
> B - I'm keen to make it very easy to compare results across the various runs.  I'd like to be able to pick a testcase and compare the detailed results side by side.  Will that be useful or is what we have enough for now?
> 
> C - How often should we refresh this data?    Once a day or once a week?
> 
> D -  What about running  / publishing  jtreg tests for the hotspot repo?
> 
> Cheers
> 
> Steve
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/attachments/20130124/8873ad56/attachment.html