Try to run core libs tests -- All IO tests failing

Wed Jul 31 14:56:03 UTC 2013

Nick,

A *workaround* for this problem would be to locally edit the names of 
the problem tests into jdk/test/ProblemList.txt so that they are not run.

There is general work underway to improve/simplify the Makefile support 
for jtreg, that should address this problem.

-- Jon

On 07/31/2013 07:49 AM, Nick Williams wrote:
> That's how I'm running it.
>
> $ pwd
> /foo/bar/jdk8/jdk8
> $ cd test
> $ make PRODUCT_HOME=`pwd`/../build/macosx-x86_64-normal-server-release/images/j2sdk-image/ all
>
> If one test fails in a batch, none of the following batches run. So if one test fails in java.io, java.lang won't run. Makes it difficult to run all of the tests, especial since there are two tests in java.lang (ThreadLocalSupplierTests due to java.lang.VerifyError: Bad local variable type, and DefaultMethodModeling due to java.lang.NullPointerException at java.util.Objects.requireNonNull when checking class B) that fail consistently (on Mac OS X at least) that I haven't been able to solve by identifying issues with the testing environment.
>
> Nick
>
> On Jul 31, 2013, at 9:44 AM, Chris Hegarty wrote:
>
>> I believe if you run the tests from the top-level test/Makefile, not jdk/test/Makefile, all individual test batches ( invocations of jtreg ) get run.
>>
>> -Chris.
>>
>> On 30/07/2013 21:46, Jonathan Gibbons wrote:
>>> jtreg itself does not "stop on error". If you're driving the tests
>>> through the makefiles, the makefiles may partition the work into
>>> separate jtreg runs on separate parts of the test suite.
>>>
>>> -- Jon
>>>
>>>
>>> On 07/30/2013 01:13 PM, Nick Williams wrote:
>>>> Okay, this is indeed very interesting. After two hours it was only
>>>> about half-way through the java.io tests and all of them had failed so
>>>> far. On a sheer hunch and nothing more, I unplugged my ethernet cable,
>>>> thus disconnecting me from any/all networks and the Internet. BOOM.
>>>> The rest of the java.io tests finished (and passed) in a few seconds,
>>>> leaving 137 that had failed. I then re-ran the tests while still
>>>> disconnected from the Internet and 312 of the java.io tests passed
>>>> this time, leaving only 1 failure (java/io/BufferedReader/Lines.java,
>>>> output below).
>>>>
>>>> I plugged my Ethernet back in and ran the tests again and java.io
>>>> started failing every test again, timing out after 60 seconds each.
>>>> Curiously extending my hunch I remained connected over Ethernet and
>>>> connected to a remote network via OpenVPN. Now all of the java.io test
>>>> pass again (except that same one, with the same output).
>>>>
>>>> What on Earth??? (If it helps, I'm on Mac OS X 10.7.5.)
>>>>
>>>> Unfortunately, the one failed java.io test prevents jtreg from
>>>> continuing on to the next set of tests (java.lang). Anyone know how to
>>>> tell jtreg to continue running on error?
>>>>
>>>> Nick
>>>>
>>>> #Test Results (version 2)
>>>> #Tue Jul 30 14:53:42 CDT 2013
>>>> #-----testdescription-----
>>>> $file=/Users/Nicholas/Documents/OpenJDK/Projects/jdk8/jdk8/jdk/test/java/io/BufferedReader/Lines.java
>>>>
>>>> $root=/Users/Nicholas/Documents/OpenJDK/Projects/jdk8/jdk8/jdk/test
>>>> keywords=bug8003258
>>>> run=USER_SPECIFIED testng Lines\n
>>>> source=Lines.java
>>>> title=\
>>>>
>>>> #-----environment-----
>>>>
>>>> #-----testresult-----
>>>> end=Tue Jul 30 14\:53\:42 CDT 2013
>>>> execStatus=Error. Unexpected exception caught from test
>>>> java/io/BufferedReader/Lines.java\: java.lang.NullPointerException
>>>> sections=script_messages Details
>>>>
>>>> #section:script_messages
>>>> ----------messages:(0/0)----------
>>>>
>>>> #section:Details
>>>> ----------messages:(0/0)----------
>>>> ----------Stack trace:(10/672)----------
>>>> java.lang.NullPointerException
>>>> at
>>>> com.sun.javatest.regtest.TestNGReporter.getMatcher(TestNGReporter.java:97)
>>>>
>>>> at com.sun.javatest.regtest.TestNGReporter.add(TestNGReporter.java:80)
>>>> at com.sun.javatest.regtest.TestNGAction.endAction(TestNGAction.java:131)
>>>> at com.sun.javatest.regtest.MainAction.run(MainAction.java:260)
>>>> at
>>>> com.sun.javatest.regtest.RegressionScript.run(RegressionScript.java:149)
>>>> at com.sun.javatest.Script.run(Script.java:228)
>>>> at com.sun.javatest.DefaultTestRunner.runTest(DefaultTestRunner.java:174)
>>>> at
>>>> com.sun.javatest.DefaultTestRunner.access$100(DefaultTestRunner.java:43)
>>>> at com.sun.javatest.DefaultTestRunner$1.run(DefaultTestRunner.java:66)
>>>> result: Not run. Test running...
>>>>
>>>>
>>>> test result: Error. Unexpected exception caught from test
>>>> java/io/BufferedReader/Lines.java: java.lang.NullPointerException
>>>>
>>>>
>>>> On Jul 30, 2013, at 1:39 PM, Nick Williams wrote:
>>>>
>>>>> On Jul 30, 2013, at 1:16 PM, Nick Williams wrote:
>>>>>
>>>>>> On Jul 30, 2013, at 1:10 PM, Alan Bateman wrote:
>>>>>>
>>>>>>> On 30/07/2013 11:02, Nick Williams wrote:
>>>>>>>> I'm running the core libs tests locally (for the first time), and
>>>>>>>> every java.io test is failing. They're all failing for the same
>>>>>>>> reason (output below), and I suspect it's something local and not
>>>>>>>> an actual problem. But the test output is not very helpful. Note
>>>>>>>> that all of the java.beans tests passed with flying colors. Once
>>>>>>>> it got to java.io, everything started failing. Worse, each test is
>>>>>>>> taking 60 seconds to fail. I could be here a while...
>>>>>>>>
>>>>>>>> Thoughts?
>>>>>>> What command are you using? From the output it suggests that the
>>>>>>> connection between jtreg and the agent VM cannot be established.
>>>>>>>
>>>>>>> -Alan.
>>>>>> Command, just like README-builds.html#testing says:
>>>>>>
>>>>>> cd test && make
>>>>>> PRODUCT_HOME=`pwd`/../build/macosx-x86_64-normal-server-release/images/j2sdk-image/
>>>>>> all
>>>>>>
>>>>>> I assume this is specifically an java.io-related problem because all
>>>>>> the java.beans tests passed without exception and so far every
>>>>>> java.io test through java.io.Externalizable (that's as far as it has
>>>>>> gotten) has failed without exception. Wouldn't a problem with jtreg
>>>>>> communicating with the agent VM have shown up in java.beans as well?
>>>>>> I could be wrong of course.
>>>>> I see now that java.beans tests run in "othervm mode" while java.io
>>>>> tests run in "agentvm mode." Didn't realize they were being run
>>>>> differently. Your explanation makes much more sense now, but I still
>>>>> don't know what's wrong...
>>>>>
>>>>>>>> Here's the output from one of the tests. It looks like all the
>>>>>>>> other test outputs:
>>>>>>>>
>>>>>>>> #Test Results (version 2)
>>>>>>>> #Tue Jul 30 12:38:15 CDT 2013
>>>>>>>> #-----testdescription-----
>>>>>>>> $file=/Users/Nicholas/Documents/OpenJDK/Projects/jdk8/jdk8/jdk/test/java/io/BufferedInputStream/ReadAfterClose.java
>>>>>>>>
>>>>>>>> $root=/Users/Nicholas/Documents/OpenJDK/Projects/jdk8/jdk8/jdk/test
>>>>>>>> keywords=bug4143651
>>>>>>>> run=ASSUMED_ACTION main ReadAfterClose\n
>>>>>>>> source=ReadAfterClose.java
>>>>>>>> title=Test if I/O methods will check if the stream has been closed.
>>>>>>>>
>>>>>>>> #-----environment-----
>>>>>>>>
>>>>>>>> #-----testresult-----
>>>>>>>> description=file\:/Users/Nicholas/Documents/OpenJDK/Projects/jdk8/jdk8/jdk/test/java/io/BufferedInputStream/ReadAfterClose.java
>>>>>>>>
>>>>>>>> elapsed=60007 0\:01\:00.007
>>>>>>>> end=Tue Jul 30 12\:38\:15 CDT 2013
>>>>>>>> environment=regtest
>>>>>>>> execStatus=Error. Cannot get VM for test\:
>>>>>>>> java.net.SocketTimeoutException\: Accept timed out
>>>>>>>> hostname=unknown
>>>>>>>> javatestOS=Mac OS X 10.7.5 (x86_64)
>>>>>>>> javatestVersion=4.4.1
>>>>>>>> jtregVersion=jtreg 4.1 dev b00
>>>>>>>> script=com.sun.javatest.regtest.RegressionScript
>>>>>>>> sections=script_messages build compile
>>>>>>>> start=Tue Jul 30 12\:37\:15 CDT 2013
>>>>>>>> test=java/io/BufferedInputStream/ReadAfterClose.java
>>>>>>>> user.name=Nicholas
>>>>>>>> work=/Users/Nicholas/Documents/OpenJDK/Projects/jdk8/jdk8/jdk/build/macosx-amd64/testoutput/jdk_io/JTwork/java/io/BufferedInputStream
>>>>>>>>
>>>>>>>>
>>>>>>>> #section:script_messages
>>>>>>>> ----------messages:(5/308)----------
>>>>>>>> JDK under test:
>>>>>>>> (/Users/Nicholas/Documents/OpenJDK/Projects/jdk8/jdk8/test/../build/macosx-x86_64-normal-server-release/images/j2sdk-image)
>>>>>>>>
>>>>>>>> openjdk version "1.8.0-internal"
>>>>>>>> OpenJDK Runtime Environment (build
>>>>>>>> 1.8.0-internal-icholas_2013_07_29_21_14-b00)
>>>>>>>> OpenJDK 64-Bit Server VM (build 25.0-b42, mixed mode)
>>>>>>>>
>>>>>>>>
>>>>>>>> #section:build
>>>>>>>> ----------messages:(3/100)----------
>>>>>>>> command: build ReadAfterClose
>>>>>>>> reason: Named class compiled on demand
>>>>>>>> elapsed time (seconds): 60.005
>>>>>>>> result: Error. Cannot get VM for test:
>>>>>>>> java.net.SocketTimeoutException: Accept timed out
>>>>>>>>
>>>>>>>> #section:compile
>>>>>>>> ----------messages:(3/235)----------
>>>>>>>> command: compile -XDignore.symbol.file=true
>>>>>>>> /Users/Nicholas/Documents/OpenJDK/Projects/jdk8/jdk8/jdk/test/java/io/BufferedInputStream/ReadAfterClose.java
>>>>>>>>
>>>>>>>> reason: .class file out of date or does not exist
>>>>>>>> elapsed time (seconds): 60.005
>>>>>>>> result: Error. Cannot get VM for test:
>>>>>>>> java.net.SocketTimeoutException: Accept timed out
>>>>>>>>
>>>>>>>>
>>>>>>>> test result: Error. Cannot get VM for test:
>>>>>>>> java.net.SocketTimeoutException: Accept timed out
>>>>>>>>