<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p>Hi Vitaly,</p>
A little bit of background, the purpose of the posting of EA build
results is to provide consistent, reproducible<br>
results that people can generate and compare against in their own
environment.<br>
<br>
To this end, we do not run the awt, swing and 2d tests, as there are
so many variables and the results <br>
can be inconsistent.<br>
<br>
<div class="moz-cite-prefix">On 04/10/2016 03:35, Vitaly Provodin
wrote:<br>
</div>
<blockquote
cite="mid:A372D87B-421F-4C25-99EE-7C9B75FA0066@jetbrains.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<p class="gmail-p1"><span class="gmail-s1">Hello,</span></p>
<p class="gmail-p2">My question relates to failures of the
Regression tests (jdk/test) on OpenJDK8 and how to identify
whether a failure is expected or not.</p>
<p class="gmail-p2">Running java/awt and javax/swing tests against
OpenJDK8 on various platforms I got some failures. Trying to
find any explanations for these failures I looked through the
JDK Bug System and could not find any bugs explaining them, for
example, for the following two tests</p>
</blockquote>
As I mentioned above, we don't run these tests as part of the
regular testing see NOTE 2 <br>
<a class="moz-txt-link-freetext" href="http://download.java.net/openjdk/testresults/9/testresults.html">http://download.java.net/openjdk/testresults/9/testresults.html</a><br>
2) ** Results do not includes test results for awt, swing and 2d.
<blockquote
cite="mid:A372D87B-421F-4C25-99EE-7C9B75FA0066@jetbrains.com"
type="cite">
<p class="gmail-p3"><i class="">-java/awt/Frame/UnfocusableMaximizedFrameResizablity/UnfocusableMaximizedFrameResizablity.java</i><br
class="">
</p>
<blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex;
border-left-width: 1px; border-left-style: solid;
border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><span
class="gmail-s1">Failed.<br class="">
</span><span class="gmail-s1">Program
`/Users/builduser/buildAgent/work/13ab0d4c51b0de0b/jbsdk/jdk/Contents/Home/bin/java'
timed out (timeout set to 240000ms, elapsed time was
240786ms).</span></blockquote>
</blockquote>
<blockquote
cite="mid:A372D87B-421F-4C25-99EE-7C9B75FA0066@jetbrains.com"
type="cite">
<p class="gmail-p3"><span class="gmail-s1"></span></p>
<p class="gmail-p5"><i class="">-
javax/swing/JPopupMenu/6800513/bug6800513.java</i><br class="">
</p>
<div class=""><span class="gmail-s1"></span><br
class="webkit-block-placeholder">
</div>
<blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex;
border-left-width: 1px; border-left-style: solid;
border-left-color: rgb(204, 204, 204); padding-left: 1ex;">Failed.<br
class="">
<span class="gmail-s1">Execution failed: `main' threw exception:
java.lang.Exception: popup class is:
javax.swing.PopupFactory$HeavyWeightPopup, expected:
javax.swing.PopupFactory$LightWeightPopup</span></blockquote>
</blockquote>
<blockquote
cite="mid:A372D87B-421F-4C25-99EE-7C9B75FA0066@jetbrains.com"
type="cite">
<div class=""><span class="gmail-s1"></span><br
class="webkit-block-placeholder">
</div>
<div class="">
<p class="gmail-p5">I found out that these tests were excluded
from runs against JDK 8 Update Releases Early Access Build
(EAB) - see Test Results page at <a moz-do-not-send="true"
href="http://download.java.net/openjdk/testresults/8"
class="">http://download.java.net/openjdk/testresults/8</a>.
Please note, I do not know the status of these results,
whether they are actual for OpenJDK8 final release or not.
Also, I paid attention to that most of the failures I am
talking about are not included into jdk/test/ProblemList.txt
as for now.</p>
</div>
</blockquote>
As I mentioned above, we don't run these tests as part of the
regular testing see NOTE 2 <br>
<a class="moz-txt-link-freetext" href="http://download.java.net/openjdk/testresults/9/testresults.html">http://download.java.net/openjdk/testresults/9/testresults.html</a><br>
2) ** Results do not includes test results for awt, swing and 2d.<br>
<br>
<blockquote
cite="mid:A372D87B-421F-4C25-99EE-7C9B75FA0066@jetbrains.com"
type="cite">
<div class="">
<p class="gmail-p5">Supposing once these tests were excluded
from runs, there were some causes for that e.g. some known
product or test issues which were not intended to be fixed for
this particular release. My question is how can I get to know
these causes in order to <span style="font-size: 13px;
white-space: nowrap;" class="">differentiate expected and </span>unexpected
failures - any of these tests, which were excluded from EAB
runs because of one issue, may fail because of another.</p>
</div>
</blockquote>
With the exception of awt, swing and 2d, the test results or any
differences from one build to another are highlighted in the diff
column.<br>
<br>
<blockquote
cite="mid:A372D87B-421F-4C25-99EE-7C9B75FA0066@jetbrains.com"
type="cite">
<div class="">
<p class="gmail-p5">I tried to follow the link at Summary column
placed on the page <a moz-do-not-send="true"
href="http://download.java.net/openjdk/testresults/9"
class="">http://download.java.net/openjdk/testresults/9</a> and
could not get information about statuses of the tests (similar
to OpenJDK8). </p>
</div>
</blockquote>
This link works for me, can you retry ?
<blockquote
cite="mid:A372D87B-421F-4C25-99EE-7C9B75FA0066@jetbrains.com"
type="cite">
<div class="">
<p class="gmail-p5">So another question, how can I get the list
of regression tests (if any) which expectedly fail for some
particular OpenJDK release?</p>
</div>
</blockquote>
In the test results page - for example
<a class="moz-txt-link-freetext" href="http://download.java.net/openjdk/testresults/9/testresults.html">http://download.java.net/openjdk/testresults/9/testresults.html</a><br>
<br>
The summary link provides a link to the tests that ran successfully.
The diff column provides a link to the failures that are different
to the previous build by component - jdk, hotspot & langtools<br>
<br>
I hope that helps.<br>
<br>
Rgds,Rory<br>
<blockquote
cite="mid:A372D87B-421F-4C25-99EE-7C9B75FA0066@jetbrains.com"
type="cite">
<div class="">
<p class="gmail-p5">Thanks,<br class="">
</p>
<p class="gmail-p5">Vitaly </p>
</div>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA, Dublin,Ireland</pre>
</body>
</html>