MacOS X 10.7 Lion & Ubuntu 12.04 - JTReg tests failing!
Mani Sarkar
sadhak001 at gmail.com
Fri Mar 15 18:50:56 PDT 2013
Hi,
I ran JTReg tests on my Ubuntu 12.04 and MacOS X 10.7 Lion systems, via the
*~/sources/jdk8_tl/test* folder after installing *jtreg* and setting up the
environment with paths settings for jt_home and product_home. I got the
below results, although the issues were similar in both environments there
are differences between the two across more than one test component.
For completeness sake I have combined the results of the two into one email
and not omitted any common bits.
*MacOS X 10.7 Lion (System config: Intel Dual Core 2 Duo 2.53Ghz, 2GB,
250GB)*
*
*
*jck7compiler (logs: http://bit.ly/13NkHTG)*
*Error: Unable to access jarfile
/java/re/jck/7/promoted/latest/binaries/JCK-compiler-7/lib/jtjck.jar*
*Missing file:
/Users/openjdk/sources/jdk8_tl/jdk/test/../build/macosx-i586/testoutput/jck7compiler/JTreport/text/summary.txt
*
*EXIT CODE: 1*
*make[2]: *** [_generic_jck_tests] Error 1*
*make[1]: *** [jck7compiler] Error 2*
*make: *** [jck7compiler] Error 2*
*jck7devtools **(logs: http://bit.ly/Y4P4zI)*
*Error: Unable to access jarfile
/java/re/jck/7/promoted/latest/binaries/JCK-devtools-7/lib/jtjck.jar*
*Missing file: /Users/**openjdk**
/sources/jdk8_tl/jdk/test/../build/macosx-i586/testoutput/jck7devtools/JTreport/text/summary.txt
*
*EXIT CODE: 1*
*make[2]: *** [_generic_jck_tests] Error 1*
*make[1]: *** [jck7devtools] Error 2*
*make: *** [jck7devtools] Error 2*
*jck7runtime (logs: http://bit.ly/WAFCa5)*
*Error: Unable to access jarfile
/java/re/jck/7/promoted/latest/binaries/JCK-runtime-7/lib/jtjck.jar*
*Missing file: /Users/**openjdk**
/sources/jdk8_tl/jdk/test/../build/macosx-i586/testoutput/jck7runtime/JTreport/text/summary.txt
*
*EXIT CODE: 1*
*make[2]: *** [_generic_jck_tests] Error 1*
*make[1]: *** [jck7runtime] Error 2*
*make: *** [jck7runtime] Error 2*
*
*
(Do the above require additional configuration settings (missing
environment variable) to be applied to make them run?)
*jdk_awt* *(logs: http://bit.ly/10mkVfq)*
(TEST STATS: name=jdk_awt run=354 pass=313 fail=41 excluded=0)
*Failure reasons: Various RuntimeExceptions - these could differ from
machine-to-machine, how to determine if they are not false negatives?*
*jdk_beans1* *(logs: http://bit.ly/Y27Aqu)*
(TEST STATS: name=jdk_beans1 run=72 pass=70 fail=2 excluded=0)
*Failure reasons: **Tests failed due to AssertionError and <xxxx>Type
errors!*
*Same reason as under Ubuntu - ** test is expected to fail as per *
http://download.java.net/jdk8/testresults/archives/b80/jdk-b80-summary.txt?
*
*
*jdk_jfr (logs: **http://bit.ly/Y0lHQb)*
*Directory "/Users/**openjdk**/sources/jdk8_tl/jdk/test/../build/macosx-i586/testoutput/jdk_jfr/JTreport"
not found: creating*
*Directory "/Users/**openjdk**/sources/jdk8_tl/jdk/test/../build/macosx-i586/testoutput/jdk_jfr/JTwork"
not found: creating*
*Directory "/Users/**openjdk**/sources/jdk8_tl/jdk/test/../build/macosx-i586/testoutput/jdk_jfr/JTwork/scratch"
not found: creating*
*Missing file: /Users/**openjdk**
/sources/jdk8_tl/jdk/test/../build/macosx-i586/testoutput/jdk_jfr/JTreport/text/summary.txt
*
*EXIT CODE: 0*
*EXIT CODE: 0*
*Failure reasons: (make file issues?)*
*jdk_lang* *(logs: http://bit.ly/Z97COE)*
(TEST STATS: name=jdk_lang run=426 pass=415 fail=11 excluded=5)
*Failure reasons: Tests failed without specifying any reason, ??? returned!*
FAILED: java/lang/invoke/AccessControlTest.java
FAILED: java/lang/invoke/BigArityTest.java
FAILED: java/lang/invoke/ClassValueTest.java
FAILED: java/lang/invoke/InvokeGenericTest.java
FAILED: java/lang/invoke/JavaDocExamplesTest.java
FAILED: java/lang/invoke/MethodHandlesTest.java
FAILED: java/lang/invoke/MethodTypeTest.java
FAILED: java/lang/invoke/PermuteArgsTest.java
FAILED: java/lang/invoke/PrivateInvokeTest.java
FAILED: java/lang/invoke/RicochetTest.java
FAILED: java/lang/invoke/ThrowExceptionsTest.java
*
These above tests should have passed as per the test results archive!
*
*jdk_management* *(logs: http://bit.ly/Yjn6gQ)*
(Test results: passed: 45; failed: 1)
*Failure reasons: ACTION: main -- Failed. Execution failed: `main' threw
exception: java.lang.RuntimeException: Committed virtual memory size
illegal value: 3272564736 bytes (MIN = 1; MAX = 2415919104) - is that
normal?*
*
*
*This test shouldn't have failed as per the test results archive!*
*jdk_other (logs: **http://bit.ly/WcLYdi)*
(TEST STATS: name=jdk_other run=88 pass=87 fail=1 excluded=2)
*Failure reasons: ACTION: main -- Failed. Execution failed: `main' threw
exception: java.lang.RuntimeException: Test failed - exit return code
non-zero (exitStatus==1)
REASON: User specified action: run main TraceJFrame JFrameCreateTime - this
test is expected to fail as per the test results archive!
*
*
*
*jdk_swing* *(logs: http://bit.ly/WcM5FX)*
(TEST STATS: name=jdk_swing run=289 pass=251 fail=38 excluded=0)
*Failure reasons: various errors and warnings*
*Same situation here, as per the test results archive, none of these failed
tests were run - so shouldn't it have done the same here or did I have to
run it with a parameter. *
*
*
*jdk_time* *(logs: http://bit.ly/10mmN7Z)*
(TEST STATS: name=jdk_time run=120 pass=118 fail=2 excluded=0)
*Failure reasons: parse error and runtime error*
*java/time/tck/java/time/format/TCKDateTimeFormatters.java failure - should
haven't failed as per the test results archive but I have heard of
date/time and formatter tests to fail, whats the real reason here?*
*
*
*langtools_jtreg* *(logs: http://bit.ly/VTLASj)*
*Failure reasons: **(fails to run, make issue ?)*
I have compared them with the reference at
http://download.java.net/jdk8/testresults/testresults.html?q=download/jdk8/testresults/testresults.html,
which expect a number of tests to fail or not run.
-----------------------------
*Ubuntu 12.04 (System config: Intel i586 2.53Ghz, 6GB, 500GB)*
*jck7compiler*
*Error: Unable to access jarfile
/java/re/jck/7/promoted/latest/binaries/JCK-compiler-7/lib/jtjck.jar*
*Missing file:
/home/openjdk/sources/jdk8_tl/jdk/test/../build/linux-i586/testoutput/jck7compiler/JTreport/text/summary.txt
*
*EXIT CODE: 1*
*make[2]: *** [_generic_jck_tests] Error 1*
*make[2]: Leaving directory `/home/openjdk/sources/jdk8_tl/jdk/test'*
*make[1]: *** [jck7compiler] Error 2*
*make[1]: Leaving directory `/home/openjdk/sources/jdk8_tl/jdk/test'*
*make: *** [jck7compiler] Error 2*
*jck7devtools*
*Error: Unable to access jarfile
/java/re/jck/7/promoted/latest/binaries/JCK-devtools-7/lib/jtjck.jar*
*Missing file:
/home/openjdk/sources/jdk8_tl/jdk/test/../build/linux-i586/testoutput/jck7devtools/JTreport/text/summary.txt
*
*EXIT CODE: 1*
*make[2]: *** [_generic_jck_tests] Error 1*
*make[2]: Leaving directory `/home/openjdk/sources/jdk8_tl/jdk/test'*
*make[1]: *** [jck7devtools] Error 2*
*make[1]: Leaving directory `/home/openjdk/sources/jdk8_tl/jdk/test'*
*make: *** [jck7devtools] Error 2*
*jck7runtime*
Error: Unable to access jarfile
/java/re/jck/7/promoted/latest/binaries/JCK-runtime-7/lib/jtjck.jar
Missing file:
/home/openjdk/sources/jdk8_tl/jdk/test/../build/linux-i586/testoutput/jck7runtime/JTreport/text/summary.txt
EXIT CODE: 1
make[2]: *** [_generic_jck_tests] Error 1
make[2]: Leaving directory `/home/openjdk/sources/jdk8_tl/jdk/test'
make[1]: *** [jck7runtime] Error 2
make[1]: Leaving directory `/home/openjdk/sources/jdk8_tl/jdk/test'
make: *** [jck7runtime] Error 2
(Do the above require additional configuration settings (missing
environment variable) to be applied to make them run?)
*jdk_awt **(logs: **http://bit.ly/15EDdf4* <http://bit.ly/15EDdf4>*)*
*STATUS:Failed.`main' threw exception: java.lang.RuntimeException: Test
failed*
*
*
*TEST RESULT: Failed. Execution failed: `main' threw exception:
java.lang.RuntimeException: Test failed*
*--------------------------------------------------*
*Missing file:
/home/openjdk/sources/jdk8_tl/jdk/test/../build/linux-i586/testoutput/jdk_awt/JTreport/text/summary.txt
*
*EXIT CODE: 143*
*EXIT CODE: 143*
*make[2]: *** [jtreg_tests] Error 143*
*make[2]: Leaving directory `/home/openjdk/sources/jdk8_tl/jdk/test'*
*make[1]: *** [jdk_awt] Error 2*
*make[1]: Leaving directory `/home/openjdk/sources/jdk8_tl/jdk/test'*
*make: *** [jdk_awt] Error 2*
*Failure reasons: One RuntimeExceptions and test execution exits out?*
*jdk_beans1 **(logs: **http://bit.ly/WB2BBR** )*
(TEST STATS: name=jdk_beans1 run=72 pass=71 fail=1 excluded=0)
*Failure reasons: **Tests failed due to <xxxx>Type errors! - is it correct
this rest is expected to fail as per *
http://download.java.net/jdk8/testresults/archives/b80/jdk-b80-summary.txt?
*jdk_jfr **(logs: **http://bit.ly/12Dwmoe**)*
*Directory
"/home/openjdk/sources/jdk8_tl/jdk/test/../build/linux-i586/testoutput/jdk_jfr/JTreport"
not found: creating*
*Directory
"/home/openjdk/sources/jdk8_tl/jdk/test/../build/linux-i586/testoutput/jdk_jfr/JTwork"
not found: creating*
*Directory
"/home/openjdk/sources/jdk8_tl/jdk/test/../build/linux-i586/testoutput/jdk_jfr/JTwork/scratch"
not found: creating*
*Missing file:
/home/openjdk/sources/jdk8_tl/jdk/test/../build/linux-i586/testoutput/jdk_jfr/JTreport/text/summary.txt
*
*EXIT CODE: 0*
*EXIT CODE: 0*
*make[2]: Leaving directory `/home/openjdk/sources/jdk8_tl/jdk/test'*
*make[1]: Leaving directory `/home/openjdk/sources/jdk8_tl/jdk/test'*
*
*
*Failure reasons: (make file issues?)*
*jdk_jmx **(logs: **http://bit.ly/X9FgHZ**)*
(TEST STATS: name=jdk_jmx run=218 pass=217 fail=1 excluded=0)
*Failure reasons: **TEST FAILS: addresses differ - is this a system
specific issue?*
*jdk_lang **(logs: **http://bit.ly/Y0PIPJ**)*
(TEST STATS: name=jdk_lang run=427 pass=415 fail=12 excluded=5)
*Failure reasons: Tests failed without specifying any reason, ??? returned!*
*
*
*
FAILED: java/lang/invoke/AccessControlTest.java
FAILED: java/lang/invoke/BigArityTest.java
FAILED: java/lang/invoke/ClassValueTest.java
FAILED: java/lang/invoke/InvokeGenericTest.java
FAILED: java/lang/invoke/JavaDocExamplesTest.java
FAILED: java/lang/invoke/MethodHandlesTest.java
FAILED: java/lang/invoke/MethodTypeTest.java
FAILED: java/lang/invoke/PermuteArgsTest.java
FAILED: java/lang/invoke/PrivateInvokeTest.java
FAILED: java/lang/invoke/RicochetTest.java
FAILED: java/lang/invoke/ThrowExceptionsTest.java
These should have passed as per the test results archive!
*
*
*
*jdk_nio **(logs: **http://bit.ly/13MkyPU**)*
(TEST STATS: name=jdk_nio run=320 pass=317 fail=3 excluded=0)
*Failure reasons:
java.nio.channels.ClosedByInterruptException, java.lang.InterruptedException
exceptions & java.net.SocketTimeoutException*
*(these tests should have passed, can't find any failures in the test
results archive - possible unpredictable and different on different
machines?)*
*jdk_security3 **(logs: **http://bit.ly/ZsdFQq**)*
(TEST STATS: name=jdk_security3 run=530 pass=528 fail=2 excluded=5)
*Failure reasons: RuntimeExceptions on both instances*
*jdk_swing **(logs: **http://bit.ly/10vVnjT**)*
(TEST STATS: name=jdk_swing run=289 pass=226 fail=63 excluded=0)
*Failure reasons: Mostly RuntimeExceptions*
*As per the test results archive, none of these failed tests were run - so
shouldn't it have done the same here or did I have to run it with a
parameter. *
*jdk_time **(logs: **http://bit.ly/10vVrAj**)*
(TEST STATS: name=jdk_time run=120 pass=119 fail=1 excluded=0)
*Failure reasons: RuntimeException due to format errors (possible
unpredictable and different on different machines)*
*java/time/test/java/util/TestFormatter.java - this test has know to fail
according to the test results archive.*
*
*
*jdk_tools **(logs: **http://bit.ly/10vVsUP**)*
(TEST STATS: name=jdk_tools run=113 pass=112 fail=1 excluded=5)
*Failure reasons: ###TestError###: string <.*Client VM.*> not found <<===
VM not loading in client mode*
FAILED: tools/launcher/ExecutionEnvironment.java - *passed according to the
test archive.*
*langtools_jtreg* *(logs: **http://bit.ly/Y0PXKK**)*
*Failure reasons: **(fails to run, make issue ?)*
*
*
I have compared them with the reference at
http://download.java.net/jdk8/testresults/testresults.html?q=download/jdk8/testresults/testresults.html,
which expect a number of tests to fail or not run and made notes above.
-----------------------------
There are more packages failing tests on Ubuntu than MacOS, more tests in
total have failed and under each component. Should I report these to other
report mailing lists as well?
Rory, Balchandra - am I running these tests differently than you are since
your test results (test results archive) differs from mine.
I run the tests using the below syntax:
*$ make TEST=<test component name> *
Just to isolate the test failures I ran each test component rather than
what people usually do jdk_default, jdk_core and jdk_all. Could you also
make the make command failure error messages a bit more verbose and
human-friendly so we can figure out whats to start looking at. If I have
missed out on config I would love to know.
Thanks.
Cheers,
mani
--
*Twitter:* @theNeomatrix369
*Blog:* http://neomatrix369.wordpress.com
*JUG activity:* LJC Advocate (@adoptopenjdk & @adoptajsr programs)
*Meet-a-Project:* https://github.com/MutabilityDetector
*Come to Devoxx UK 2013:* http://www.devoxx.com/display/UK13/Home
*Don't chase success, rather aim for "Excellence", and success will come
chasing after you!*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/quality-discuss/attachments/20130316/2f9813e6/attachment-0001.html
More information about the quality-discuss
mailing list