Error: Unable to access jarfile /java/re/jck/7/promoted/latest/binaries/JCK-compiler-7/lib/jtjck.jar

Missing file: /Users/openjdk/sources/jdk8_tl/jdk/test/../build/macosx-i586/testoutput/jck7compiler/JTreport/text/summary.txt

EXIT CODE: 1

make[2]: *** [_generic_jck_tests] Error 1

make[1]: *** [jck7compiler] Error 2

make: *** [jck7compiler] Error 2

jck7devtools (logs: http://bit.ly/Y4P4zI)

Error: Unable to access jarfile /java/re/jck/7/promoted/latest/binaries/JCK-devtools-7/lib/jtjck.jar

Missing file: /Users/openjdk/sources/jdk8_tl/jdk/test/../build/macosx-i586/testoutput/jck7devtools/JTreport/text/summary.txt

EXIT CODE: 1

make[2]: *** [_generic_jck_tests] Error 1

make[1]: *** [jck7devtools] Error 2

make: *** [jck7devtools] Error 2

jck7runtime (logs: http://bit.ly/WAFCa5)

Error: Unable to access jarfile /java/re/jck/7/promoted/latest/binaries/JCK-runtime-7/lib/jtjck.jar

Missing file: /Users/openjdk/sources/jdk8_tl/jdk/test/../build/macosx-i586/testoutput/jck7runtime/JTreport/text/summary.txt

EXIT CODE: 1

make[2]: *** [_generic_jck_tests] Error 1

make[1]: *** [jck7runtime] Error 2

make: *** [jck7runtime] Error 2

(Do the above require additional configuration settings (missing environment variable) to be applied to make them run?)

jdk_awt (logs: http://bit.ly/10mkVfq)

(TEST STATS: name=jdk_awt run=354 pass=313 fail=41 excluded=0)

Failure reasons: Various RuntimeExceptions - these could differ from machine-to-machine, how to determine if they are not false negatives?

jdk_beans1 (logs: http://bit.ly/Y27Aqu)

(TEST STATS: name=jdk_beans1 run=72 pass=70 fail=2 excluded=0)

Failure reasons: Tests failed due to AssertionError and <xxxx>Type errors!

Same reason as under Ubuntu - test is expected to fail as per http://download.java.net/jdk8/testresults/archives/b80/jdk-b80-summary.txt?

jdk_jfr (logs: http://bit.ly/Y0lHQb)

Directory "/Users/openjdk/sources/jdk8_tl/jdk/test/../build/macosx-i586/testoutput/jdk_jfr/JTreport" not found: creating

Directory "/Users/openjdk/sources/jdk8_tl/jdk/test/../build/macosx-i586/testoutput/jdk_jfr/JTwork" not found: creating

Directory "/Users/openjdk/sources/jdk8_tl/jdk/test/../build/macosx-i586/testoutput/jdk_jfr/JTwork/scratch" not found: creating

Missing file: /Users/openjdk/sources/jdk8_tl/jdk/test/../build/macosx-i586/testoutput/jdk_jfr/JTreport/text/summary.txt

EXIT CODE: 0

Failure reasons: (make file issues?)

jdk_lang (logs: http://bit.ly/Z97COE)

(TEST STATS: name=jdk_lang run=426 pass=415 fail=11 excluded=5)

Failure reasons: Tests failed without specifying any reason, ??? returned!

FAILED: java/lang/invoke/AccessControlTest.java

FAILED: java/lang/invoke/BigArityTest.java

FAILED: java/lang/invoke/ClassValueTest.java

FAILED: java/lang/invoke/InvokeGenericTest.java

FAILED: java/lang/invoke/JavaDocExamplesTest.java

FAILED: java/lang/invoke/MethodHandlesTest.java

FAILED: java/lang/invoke/MethodTypeTest.java

FAILED: java/lang/invoke/PermuteArgsTest.java

FAILED: java/lang/invoke/PrivateInvokeTest.java

FAILED: java/lang/invoke/RicochetTest.java

FAILED: java/lang/invoke/ThrowExceptionsTest.java

These above tests should have passed as per the test results archive!

jdk_management (logs: http://bit.ly/Yjn6gQ)

(Test results: passed: 45; failed: 1)

Failure reasons: ACTION: main -- Failed. Execution failed: `main' threw exception: java.lang.RuntimeException: Committed virtual memory size illegal value: 3272564736 bytes (MIN = 1; MAX = 2415919104) - is that normal?

This test shouldn't have failed as per the test results archive!

jdk_other (logs: http://bit.ly/WcLYdi)

(TEST STATS: name=jdk_other run=88 pass=87 fail=1 excluded=2)

Failure reasons: ACTION: main -- Failed. Execution failed: `main' threw exception: java.lang.RuntimeException: Test failed - exit return code non-zero (exitStatus==1)
REASON: User specified action: run main TraceJFrame JFrameCreateTime - this test is expected to fail as per the test results archive!

jdk_swing (logs: http://bit.ly/WcM5FX)

(TEST STATS: name=jdk_swing run=289 pass=251 fail=38 excluded=0)

Failure reasons: various errors and warnings

Same situation here, as per the test results archive, none of these failed tests were run - so shouldn't it have done the same here or did I have to run it with a parameter.

jdk_time (logs: http://bit.ly/10mmN7Z)

(TEST STATS: name=jdk_time run=120 pass=118 fail=2 excluded=0)

Failure reasons: parse error and runtime error

java/time/tck/java/time/format/TCKDateTimeFormatters.java failure - should haven't failed as per the test results archive but I have heard of date/time and formatter tests to fail, whats the real reason here?

langtools_jtreg (logs: http://bit.ly/VTLASj)

Failure reasons: (fails to run, make issue ?)

I have compared them with the reference at http://download.java.net/jdk8/testresults/testresults.html?q=download/jdk8/testresults/testresults.html, which expect a number of tests to fail or not run.

-----------------------------

Ubuntu 12.04 (System config: Intel i586 2.53Ghz, 6GB, 500GB)

jck7compiler

Error: Unable to access jarfile /java/re/jck/7/promoted/latest/binaries/JCK-compiler-7/lib/jtjck.jar

Missing file: /home/openjdk/sources/jdk8_tl/jdk/test/../build/linux-i586/testoutput/jck7compiler/JTreport/text/summary.txt

EXIT CODE: 1

make[2]: *** [_generic_jck_tests] Error 1

make[2]: Leaving directory `/home/openjdk/sources/jdk8_tl/jdk/test'

make[1]: *** [jck7compiler] Error 2

make[1]: Leaving directory `/home/openjdk/sources/jdk8_tl/jdk/test'

make: *** [jck7compiler] Error 2

jck7devtools

Error: Unable to access jarfile /java/re/jck/7/promoted/latest/binaries/JCK-devtools-7/lib/jtjck.jar

Missing file: /home/openjdk/sources/jdk8_tl/jdk/test/../build/linux-i586/testoutput/jck7devtools/JTreport/text/summary.txt

EXIT CODE: 1

make[2]: *** [_generic_jck_tests] Error 1

make[2]: Leaving directory `/home/openjdk/sources/jdk8_tl/jdk/test'

make[1]: *** [jck7devtools] Error 2

make[1]: Leaving directory `/home/openjdk/sources/jdk8_tl/jdk/test'

make: *** [jck7devtools] Error 2

jck7runtime

Error: Unable to access jarfile /java/re/jck/7/promoted/latest/binaries/JCK-runtime-7/lib/jtjck.jar

Missing file: /home/openjdk/sources/jdk8_tl/jdk/test/../build/linux-i586/testoutput/jck7runtime/JTreport/text/summary.txt

EXIT CODE: 1

make[2]: *** [_generic_jck_tests] Error 1

make[2]: Leaving directory `/home/openjdk/sources/jdk8_tl/jdk/test'

make[1]: *** [jck7runtime] Error 2

make[1]: Leaving directory `/home/openjdk/sources/jdk8_tl/jdk/test'

make: *** [jck7runtime] Error 2

(Do the above require additional configuration settings (missing environment variable) to be applied to make them run?)

jdk_awt (logs: http://bit.ly/15EDdf4)

STATUS:Failed.`main' threw exception: java.lang.RuntimeException: Test failed

TEST RESULT: Failed. Execution failed: `main' threw exception: java.lang.RuntimeException: Test failed

--------------------------------------------------

Missing file: /home/openjdk/sources/jdk8_tl/jdk/test/../build/linux-i586/testoutput/jdk_awt/JTreport/text/summary.txt

EXIT CODE: 143

make[2]: *** [jtreg_tests] Error 143

make[2]: Leaving directory `/home/openjdk/sources/jdk8_tl/jdk/test'

make[1]: *** [jdk_awt] Error 2

make[1]: Leaving directory `/home/openjdk/sources/jdk8_tl/jdk/test'

make: *** [jdk_awt] Error 2

Failure reasons: One RuntimeExceptions and test execution exits out?

jdk_beans1 (logs: http://bit.ly/WB2BBR )

(TEST STATS: name=jdk_beans1 run=72 pass=71 fail=1 excluded=0)

Failure reasons: Tests failed due to <xxxx>Type errors! - is it correct this rest is expected to fail as per http://download.java.net/jdk8/testresults/archives/b80/jdk-b80-summary.txt?

jdk_jfr (logs: http://bit.ly/12Dwmoe)

Directory "/home/openjdk/sources/jdk8_tl/jdk/test/../build/linux-i586/testoutput/jdk_jfr/JTreport" not found: creating

Directory "/home/openjdk/sources/jdk8_tl/jdk/test/../build/linux-i586/testoutput/jdk_jfr/JTwork" not found: creating

Directory "/home/openjdk/sources/jdk8_tl/jdk/test/../build/linux-i586/testoutput/jdk_jfr/JTwork/scratch" not found: creating

Missing file: /home/openjdk/sources/jdk8_tl/jdk/test/../build/linux-i586/testoutput/jdk_jfr/JTreport/text/summary.txt

EXIT CODE: 0

make[2]: Leaving directory `/home/openjdk/sources/jdk8_tl/jdk/test'

make[1]: Leaving directory `/home/openjdk/sources/jdk8_tl/jdk/test'

Failure reasons: (make file issues?)

jdk_jmx (logs: http://bit.ly/X9FgHZ)

(TEST STATS: name=jdk_jmx run=218 pass=217 fail=1 excluded=0)

Failure reasons: TEST FAILS: addresses differ - is this a system specific issue?

jdk_lang (logs: http://bit.ly/Y0PIPJ)

(TEST STATS: name=jdk_lang run=427 pass=415 fail=12 excluded=5)

Failure reasons: Tests failed without specifying any reason, ??? returned!

FAILED: java/lang/invoke/AccessControlTest.java
FAILED: java/lang/invoke/BigArityTest.java

FAILED: java/lang/invoke/ClassValueTest.java
FAILED: java/lang/invoke/InvokeGenericTest.java
FAILED: java/lang/invoke/JavaDocExamplesTest.java

FAILED: java/lang/invoke/MethodHandlesTest.java
FAILED: java/lang/invoke/MethodTypeTest.java
FAILED: java/lang/invoke/PermuteArgsTest.java

FAILED: java/lang/invoke/PrivateInvokeTest.java
FAILED: java/lang/invoke/RicochetTest.java
FAILED: java/lang/invoke/ThrowExceptionsTest.java

These should have passed as per the test results archive!

jdk_nio (logs: http://bit.ly/13MkyPU)

(TEST STATS: name=jdk_nio run=320 pass=317 fail=3 excluded=0)

Failure reasons: java.nio.channels.ClosedByInterruptException, java.lang.InterruptedException exceptions & java.net.SocketTimeoutException

(these tests should have passed, can't find any failures in the test results archive - possible unpredictable and different on different machines?)

jdk_security3 (logs: http://bit.ly/ZsdFQq)

(TEST STATS: name=jdk_security3 run=530 pass=528 fail=2 excluded=5)

Failure reasons: RuntimeExceptions on both instances

jdk_swing (logs: http://bit.ly/10vVnjT)

(TEST STATS: name=jdk_swing run=289 pass=226 fail=63 excluded=0)

Failure reasons: Mostly RuntimeExceptions

As per the test results archive, none of these failed tests were run - so shouldn't it have done the same here or did I have to run it with a parameter.

jdk_time (logs: http://bit.ly/10vVrAj)

(TEST STATS: name=jdk_time run=120 pass=119 fail=1 excluded=0)

Failure reasons: RuntimeException due to format errors (possible unpredictable and different on different machines)

java/time/test/java/util/TestFormatter.java - this test has know to fail according to the test results archive.

jdk_tools (logs: http://bit.ly/10vVsUP)

(TEST STATS: name=jdk_tools run=113 pass=112 fail=1 excluded=5)

Failure reasons: ###TestError###: string <.*Client VM.*> not found <<=== VM not loading in client mode

FAILED: tools/launcher/ExecutionEnvironment.java - passed according to the test archive.

langtools_jtreg (logs: http://bit.ly/Y0PXKK)

Failure reasons: (fails to run, make issue ?)

I have compared them with the reference at http://download.java.net/jdk8/testresults/testresults.html?q=download/jdk8/testresults/testresults.html, which expect a number of tests to fail or not run and made notes above.

-----------------------------

There are more packages failing tests on Ubuntu than MacOS, more tests in total have failed and under each component. Should I report these to other report mailing lists as well?

Rory, Balchandra - am I running these tests differently than you are since your test results (test results archive) differs from mine.

I run the tests using the below syntax:

$ make TEST=<test component name>

Just to isolate the test failures I ran each test component rather than what people usually do jdk_default, jdk_core and jdk_all. Could you also make the make command failure error messages a bit more verbose and human-friendly so we can figure out whats to start looking at. If I have missed out on config I would love to know.

Thanks.