RFR 8215729: Enhance makefiles to allow collecting code coverage with JCov

David Holmes david.holmes at oracle.com
Wed Jan 9 21:31:37 UTC 2019 

On 10/01/2019 5:46 am, Alexandre (Shura) Iline wrote:
> David,
> 
> At least these tests appear to be failing because of extra lines in the error output (showing test names with a line or two from jtr file):

Thanks. I think the difference here compared to 8196741 is that these 
tests fail due to extra output on stderr, while the VM warnings go to 
stdout.

David

> java/io/File/DeleteOnExitNPE.java
> 	java.lang.RuntimeException: Failed: No output should have been received from the process
> java/lang/ProcessBuilder/SkipTest.java
> 	java.lang.AssertionError: header char wrong, expected: 65, actual: 80
> java/lang/Runtime/exec/ArgWithSpaceAndFinalBackslash.java
> 	java.lang.Exception: Test failed: Got "Picked up JAVA_TOOL_OPTIONS: -Xms64m -Xmx4g
> 	Picked up _JAVA_OPTIONS: -Xms64m -Xmx4g
> 	foo bar\\baz\\", expected "foo bar\\baz\\"
> java/lang/Runtime/exec/ExitValue.java
> 	java.lang.Exception: Unexpected stderr: Picked up JAVA_TOOL_OPTIONS: -Xms64m -Xmx4g
> 	Picked up _JAVA_OPTIONS: -Xms64m -Xmx4g
> java/lang/Thread/UncaughtExceptions.sh
> 	FAIL: Expected "Exception in thread "Thread-0".*Seppuku", got Picked up JAVA_TOOL_OPTIONS: -Xms64m -Xmx4g
> java/net/InetAddress/ptr/Lookup.java
> 	IPv4 lookup results: [192.0.43.7:icann.orgPicked up JAVA_TOOL_OPTIONS: -Xms64m -Xmx4g
> 	Picked up _JAVA_OPTIONS: -Xms64m -Xmx4g
> 	]
> java/net/URLConnection/ContentHandlers/ContentHandlersTest.java
> 	java.lang.RuntimeException: Expected a different kind of failure: Picked up JAVA_TOOL_OPTIONS: -Xms64m -Xmx4g
> 	Picked up _JAVA_OPTIONS: -Xms64m -Xmx4g
> java/net/spi/URLStreamHandlerProvider/Basic.java
> 	java.lang.RuntimeException: Picked up JAVA_TOOL_OPTIONS: -Xms64m -Xmx4g
> 	Picked up _JAVA_OPTIONS: -Xms64m -Xmx4g
> java/nio/channels/spi/SelectorProvider/inheritedChannel/InheritedChannelTest.java
> 	 stderr: [Picked up JAVA_TOOL_OPTIONS: -Xms64m -Xmx4g
> 	Picked up _JAVA_OPTIONS: -Xms64m -Xmx4g
> 	Bad response or premature EOF, bytes read:
> 	java.lang.RuntimeException: Bad response or premature EOF from service
> java/security/AccessController/DoPrivAccompliceTest.java
> 	java.lang.RuntimeException: stderr was not empty
> java/util/logging/LoggingDeadlock2.java
> 	Picked up JAVA_TOOL_OPTIONS: -Xms64m -Xmx4g
> 	Picked up _JAVA_OPTIONS: -Xms64m -Xmx4g
> 	 not equal to
> java/util/zip/EntryCount64k.java
> 	java.lang.RuntimeException: '\A\Z' missing from stderr
> javax/naming/spi/providers/InitialContextTest.java
> 	java.lang.RuntimeException: Expected a different kind of failure: Picked up JAVA_TOOL_OPTIONS: -Xms64m -Xmx4g
> 	Picked up _JAVA_OPTIONS: -Xms64m -Xmx4g
> sun/rmi/runtime/Log/6409194/NoConsoleOutput.java
> 	java.lang.Error: TEST FAILED: unexpected subprocess output
> sun/tools/jrunscript/jrunscript-eTest.sh
> sun/tools/jrunscript/jrunscript-fTest.sh
> sun/tools/jrunscript/jrunscriptTest.sh
> 	Output of jrunscript -e differ from expected output. Failed.
> tools/jar/LeadingGarbage.java
> 	test LeadingGarbage.test_canListWithLeadingGarbage(): failure
> 	java.lang.RuntimeException: '\\A\\Z' missing from stderr
> tools/jar/modularJar/Basic.java
> 	java.lang.AssertionError: Unknown value Picked up JAVA_TOOL_OPTIONS: -Xms64m -Xmx4g
> tools/jar/multiRelease/Basic.java
> 	java.lang.RuntimeException: stderr was not empty
> tools/jjs/jjs-fileTest.sh
> 	Output of jjs file.js differ from expected output. Failed.	
> 
> 
> 
> 
> Other failures are:
> 
> java/io/CharArrayReader/OverflowInRead.java
> java/lang/ProcessBuilder/InheritIO/InheritIO.sh
> java/lang/StackTraceElement/ModuleFrames.java
> java/lang/StackTraceElement/SerialTest.java
> java/lang/StackWalker/VerifyStackTrace.java
> java/lang/invoke/lambda/LogGeneratedClassesTest.java
> java/net/MulticastSocket/PromiscuousIPv6.java
> java/net/URL/JarHandlerPkgPrefix/JarHandlerPkgPrefix.java
> java/nio/Buffer/LimitDirectMemory.java
> java/nio/channels/DatagramChannel/PromiscuousIPv6.java
> java/util/concurrent/locks/Lock/TimedAcquireLeak.java
> jdk/internal/misc/VM/RuntimeArguments.java
> jdk/modules/etc/JdkQualifiedExportTest.java
> tools/jar/multiRelease/ApiValidatorTest.java
> 
> 
> Again, a thorough analysis will need to be done - here I just quickly scanned the jtr files.
> 
> Shura
> 
>> On Jan 8, 2019, at 6:15 PM, David Holmes <David.Holmes at oracle.com> wrote:
>>
>> Hi Shura,
>>
>> Sorry not a review just a comment on one thing ...
>>
>> On 9/01/2019 4:35 am, Alexandre (Shura) Iline wrote:
>>> Hi,
>>> Could you please take a look on a change which allows to run tests while collecting code coverage with JCov. This is a continuation of work done in
>>> JDK-8214309: Enhance makefiles to allow generating JCov instrumented build.
>>> This adds make targets jcov-test and others like jcov-test-tier1 etc. After running the tests, one is left with a coverage report and also a data file containing coverage, for further analysis.
>>> Part of the changes are related to increasing maximum and initial heap size, which is needed to be done because instrumented classes are bigger in size. The way it is done for JTReg tests is by setting both _JAVA_OPTIONS and JAVA_TOOL_OPTIONS environment variables.
>>> As it stands now, not all tests pass while running with code coverage, which pass normally. This particular enhancement is not addressing failing tests. The tests will need to be fixed separately if ever. For example, there are 35 tests which fail in open/test/jdk:jdk_core. A quick look tells that the majority of the failures are in tests which verify error output and discover an unexpected line "Picked up _JAVA_OPTIONS: -Xmx4g”.
>>
>> We've been bitten in the past with VM warnings causing tests to fail due to the unexpected output and made a conscious effort to fix all such tests. Unfortunately a few still remain. Do your 35 failures match up with the tests in the blocking issues for this bug:
>>
>> https://bugs.openjdk.java.net/browse/JDK-8196741
>>
>> ? 35 seems a lot. I hope we haven't regressed in this area.
>>
>> Thanks,
>> David
>>
>>> Enhancement: https://bugs.openjdk.java.net/browse/JDK-8215729
>>> Webrev: http://cr.openjdk.java.net/~shurailine/8215729/webrev.01/
>>> Shura
>

More information about the build-dev mailing list