Questions about JCK test failures on x86_64 Ubuntu 16.04 with openjdk8

Fri Jan 19 12:58:18 UTC 2018

Hi Simon,

In your given list java_awt/Window/OpacityPixelChecking_Cross looks like 
a typo and there is no such testcase, instead I will consider
api/java_awt/Window/indexTGF.html\#ShapePixelChecking_Cross

1. Is x86_64 with Ubuntu 16.04 a platform we would expect to run the JCK 
successfully with openjdk8?
Yes, Even I tried today with Oracle JDK8u172b03 as well as Non 
commercial build (i.e openjdk), we usually prefer Physical display and 
desktop mode to run these tests.
all below 13 tests are passing on multi jvm mode.
api/java_awt/Window/indexTGF.html\#OpacityPixelChecking
api/java_awt/Window/indexTGF.html\#ShapePixelChecking
api/java_awt/Window/indexTGF.html\#ShapePixelChecking_Cross
api/java_awt/SplashScreen/CreateGraphicsTestsTsp.html\#CreateGraphicsTestsTsp
api/java_awt/SplashScreen/StaticContentTestsArgGIF.html\#StaticContentTestsArgGIF
api/java_awt/SplashScreen/StaticContentTestsArgJPG.html\#StaticContentTestsArgJPG
api/java_awt/SplashScreen/StaticContentTestsArgPNG.html\#StaticContentTestsArgPNG
api/java_awt/SplashScreen/StaticContentTestsMnfGIF.html\#StaticContentTestsMnfGIF
api/java_awt/SplashScreen/StaticContentTestsMnfJPG.html\#StaticContentTestsMnfJPG
api/java_awt/SplashScreen/StaticContentTestsMnfPNG.html\#StaticContentTestsMnfPNG
api/java_awt/SplashScreen/index.html\#PrecedenceTest
api/java_awt/Frame/indexTGF.html\#GetSetExtendedState
api/java_awt/awt_focus_subsystem/focusable_windows/index.html\#FocusableWindow3

I agree, these are unstable tests on Ubuntu, and it’s ok if you execute 
first 3 tests in group mode (usually what we used to certify), it will 
likely to be skipped in your execution.

I would suggest you to use the increased timeoutfactor while executing 
these tests,  for example -timeoutFactor 4.0

Also, while executing these tests, please try to have white background, 
possibly you can put about:blank in your browser window which can show 
white background, on top these background tests will render colours.

2. Is x86_64 a platform with Ubuntu 16.04 a platform which has been 
tested already?
Yes, we tested already.
- If so were those runs successful?
Yes, But you need to execute some tests multiple times to make it pass 
as it is computing pixels.
- If the runs were successful what windows manager was used in the testing?
Ubuntu 16.04 - window manager - Compiz

3. On which other x86 linux operating systems have the awt tests been 
executed successfully, and which windows manager setup was used?
OEL 6 (Oracle Linux - 6 and redhat compatible is 6.6)
Window manager - matacity.

Please let me know if you need jtr's for your perusal.

Thanks,
Govindan

>
> -------- Original Message --------
> Subject: 	Questions about JCK test failures on x86_64 Ubuntu 16.04 
> with openjdk8
> Date: 	Wed, 10 Jan 2018 12:38:16 +0000
> From: 	Simon Rushton <rushton at uk.ibm.com>
> To: 	conformance-discuss at openjdk.java.net
>
>
>
> JCK8b is being run at the AdoptOpenJDK project with a view to marking 
> builds created at the AdoptOpenJDK site as JCK compliant.
>
> The test machine is linux_x86_64 running Ubuntu 16.04.  We are unable 
> to get a clean run of the JCK because a number of the JCK tests in the 
> java_awt test suite seem to be sensitive to the window manager and 
> display setup.  The test results are consistent across the 
> AdoptOpenJDK builds and Oracle builds.
>
> Specifically:
>
> 1. If the standard Ubuntu desktop is used (DISPLAY=:0) the following 
> tests fail:
> java_awt/Window/OpacityPixelChecking
> java_awt/Window/ShapePixelChecking
> java_awt/Window/OpacityPixelChecking_Cross
> The tests pass with Windows 7.
>
> The OpacityPixelChecking test will pass if a delay is inserted between 
> two lines in the test case:
>
> fgWindow.setOpacity(1.0f);
> <insert delay here>
> fgWindow.setBackground(fgWindowColor);
>
> So perhaps the test case assumes that some asynchronous activity has 
> occurred before is actually has happened?
>
> 2. If a plain Xvfb X server is used without a window manager running, 
> two java_awt tests fail which do not fail with the 'real' display:
> a. 
> api/java_awt/awt_focus_subsystem/focusable_windows/index_FocusableWindow3.
> - There was an issue about this test six years ago which showed on 
> OSX: 
> http://macosx-port-dev.openjdk.java.narkive.com/jcQB6tyL/7u4-review-request-for-7145827-macosx-jck-failure-in-b11-focusablewindow3- 
> it was supposedly fixed by jdk8b115.
> b. api/java_awt/Frame/indexTGF_GetSetExtendedState
> - see also https://bugs.openjdk.java.net/browse/JDK-7145827
> - This test passes when metacity is running on the Xvfb display (but 
> see below).
>
> 3. If the metacity window manager is running on the Xvfb display, the 
> api/java_awt/Frame/indexTGF_GetSetExtendedState test passes (2b. 
> above), but these tests fail instead:
>
> SplashScreen/CreateGraphicsTestsTsp_CreateGraphicsTestsTsp
> SplashScreen/StaticContentTestsArgGIF_StaticContentTestsArgGIF
> SplashScreen/StaticContentTestsArgJPG_StaticContentTestsArgJPG
> SplashScreen/StaticContentTestsArgPNG_StaticContentTestsArgPNG
> SplashScreen/StaticContentTestsMnfGIF_StaticContentTestsMnfGIF
> SplashScreen/StaticContentTestsMnfJPG_StaticContentTestsMnfJPG
> SplashScreen/StaticContentTestsMnfPNG_StaticContentTestsMnfPNG
> SplashScreen/index_PrecedenceTest
>
>
> So some questions:
> 1. Is x86_64 with Ubuntu 16.04 a platform we would expect to run the 
> JCK successfully with openjdk8?
> 2. Is x86_64 a platform with Ubuntu 16.04 a platform which has been 
> tested already?
> - If so were those runs successful?
> - If the runs were successful what windows manager was used in the 
> testing?
> 3. On which other x86 linux operating systems have the awt tests been 
> executed successfully, and which windows manager setup was used?
>
>
>
> ------------------------------------------------------------------------
>
> *Simon M Rushton* 	 Hursley Park 	
> Java Test Architect 	 Hursley, SO21 2JN
> 430821 	 United Kingdom
> IBM Software Group, Application and Integration Middleware Software 		
> Phone: 	+44-1962-818298 		
> e-mail: 	rushton at uk.ibm.com 		
>
>
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with 
> number 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/conformance-discuss/attachments/20180119/35b73ce4/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 360 bytes
Desc: not available
URL: <http://mail.openjdk.java.net/pipermail/conformance-discuss/attachments/20180119/35b73ce4/attachment.gif>