<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hi Phil,<br>
<br>
<div class="moz-cite-prefix">On 8/20/2015 11:14 AM, Phil Race wrote:<br>
</div>
<blockquote cite="mid:55D618F9.3020404@oracle.com" type="cite">Joe,
<br>
<br>
How is this keyword interpreted and used ?
<br>
</blockquote>
<br>
From the TEST.ROOT file in the jdk/test directory:<br>
<br>
# The "randomness" keyword marks tests using randomness with test<br>
# cases differing from run to run. (A test using a fixed random seed<br>
# would not count as "randomness" by this definition.) Extra care<br>
# should be taken to handle test failures of intermittent or<br>
# randomness tests.<br>
<br>
<blockquote cite="mid:55D618F9.3020404@oracle.com" type="cite">How
did you select the tests below to be so marked ?
<br>
</blockquote>
<br>
Running the command<br>
<br>
find java/awt java/beans/ javax/swing/ javax/imageio/
javax/sound -type f | xargs grep -l -i random<br>
<br>
and looking for cases were randomness was used such as extracting
values from a Random or SecureRandom object. In particular, test
just using random-access-file and the like did get tagged with the
keyword.<br>
<br>
<blockquote cite="mid:55D618F9.3020404@oracle.com" type="cite">I
might ask more once I understand the answers to these but looking
<br>
at just one of these - MTGraphicsAccessTest.java - there is no
such
<br>
thing a spurious failure of this test. If you ever see a failure
that is a real
<br>
bug. Perhaps all the passes are spurious if there is in fact a bug
<br>
somewhere that would cause a crash but the test is not catching
<br>
it but that does not seem like a reason to exclude the test -
assuming
<br>
that is what this keyword will be used for.
<br>
</blockquote>
<br>
We've seen cases elsewhere where using a random number generator has
obscured the cause of a intermittent test failure or a true product
bug. If a test using randomness fails intermittently, then its
random number generator should be switched over to a utility random
number generator which always prints out the seed and allows the
seed to be set. We've had a number of fixes in core libs from
following this policy, including JDK-8022224 and JDK-6854417
<meta name="ProgId" content="PowerPoint.Slide">
<meta name="Generator" content="Microsoft PowerPoint 12">
<br>
<br>
HTH,<br>
<br>
-Joe<br>
<br>
<blockquote cite="mid:55D618F9.3020404@oracle.com" type="cite">
<br>
-phil.
<br>
<br>
On 08/20/2015 10:47 AM, joe darcy wrote:
<br>
<blockquote type="cite">Hello,
<br>
<br>
As part of implementing tiered testing [1], client library tests
which use randomness should be marked with the corresponding
keyword. The analogous changes have been made in core libs, see
JDK-8078334: Mark regression tests using randomness.
<br>
<br>
Webrev at
<br>
<br>
JDK-8134084 : Mark client libs regression tests using
randomness
<br>
<a class="moz-txt-link-freetext" href="http://cr.openjdk.java.net/~darcy/8134084.0/">http://cr.openjdk.java.net/~darcy/8134084.0/</a>
<br>
<br>
and patch below.
<br>
<br>
Thanks,
<br>
<br>
-Joe
<br>
<br>
[1]
<a class="moz-txt-link-freetext" href="http://mail.openjdk.java.net/pipermail/jdk9-dev/2015-March/001991.html">http://mail.openjdk.java.net/pipermail/jdk9-dev/2015-March/001991.html</a>
<br>
<br>
---
old/test/java/awt/EventDispatchThread/LoopRobustness/LoopRobustness.java
2015-08-20 10:39:36.425041467 -0700
<br>
+++
new/test/java/awt/EventDispatchThread/LoopRobustness/LoopRobustness.java
2015-08-20 10:39:36.273041463 -0700
<br>
@@ -30,6 +30,7 @@
<br>
* @library ../../regtesthelpers
<br>
* @build Util
<br>
* @run main LoopRobustness
<br>
+ * @key randomness
<br>
*/
<br>
<br>
import java.awt.*;
<br>
---
old/test/java/awt/FullScreen/AltTabCrashTest/AltTabCrashTest.java
2015-08-20 10:39:36.805041477 -0700
<br>
+++
new/test/java/awt/FullScreen/AltTabCrashTest/AltTabCrashTest.java
2015-08-20 10:39:36.653041473 -0700
<br>
@@ -31,6 +31,7 @@
<br>
@run main/othervm/timeout=100 -Dsun.java2d.d3d=True
AltTabCrashTest -auto -changedm
<br>
@run main/othervm/timeout=100 -Dsun.java2d.d3d=True
AltTabCrashTest -auto -usebs -changedm
<br>
@run main/othervm/timeout=100 -Dsun.java2d.opengl=True
AltTabCrashTest -auto
<br>
+ @key randomness
<br>
*/
<br>
<br>
import java.awt.AWTException;
<br>
---
old/test/java/awt/FullScreen/DisplayChangeVITest/DisplayChangeVITest.java
2015-08-20 10:39:37.193041487 -0700
<br>
+++
new/test/java/awt/FullScreen/DisplayChangeVITest/DisplayChangeVITest.java
2015-08-20 10:39:37.041041483 -0700
<br>
@@ -29,6 +29,7 @@
<br>
* @run main/othervm/timeout=200 DisplayChangeVITest
<br>
* @run main/othervm/timeout=200 -Dsun.java2d.d3d=false
DisplayChangeVITest
<br>
* @run main/othervm/timeout=200 -Dsun.java2d.opengl=true
DisplayChangeVITest
<br>
+ * @key randomness
<br>
*/
<br>
<br>
import java.awt.Color;
<br>
---
old/test/java/awt/FullScreen/MultimonFullscreenTest/MultimonFullscreenTest.java
2015-08-20 10:39:37.577041497 -0700
<br>
+++
new/test/java/awt/FullScreen/MultimonFullscreenTest/MultimonFullscreenTest.java
2015-08-20 10:39:37.425041493 -0700
<br>
@@ -50,6 +50,7 @@
<br>
* @run main/manual/othervm -Dsun.java2d.d3d=True
MultimonFullscreenTest
<br>
* @run main/manual/othervm -Dsun.java2d.noddraw=true
MultimonFullscreenTest
<br>
* @run main/manual/othervm -Dsun.java2d.opengl=True
MultimonFullscreenTest
<br>
+ * @key randomness
<br>
*/
<br>
<br>
import java.awt.Button;
<br>
---
old/test/java/awt/Graphics2D/MTGraphicsAccessTest/MTGraphicsAccessTest.java
2015-08-20 10:39:37.957041506 -0700
<br>
+++
new/test/java/awt/Graphics2D/MTGraphicsAccessTest/MTGraphicsAccessTest.java
2015-08-20 10:39:37.797041502 -0700
<br>
@@ -29,6 +29,7 @@
<br>
<br>
@author <a class="moz-txt-link-abbreviated" href="mailto:Dmitri.Trembovetski@sun.com">Dmitri.Trembovetski@sun.com</a> area=Graphics
<br>
@run main MTGraphicsAccessTest
<br>
+ @key randomness
<br>
*/
<br>
<br>
import java.awt.*;
<br>
---
old/test/java/awt/Graphics2D/RenderClipTest/RenderClipTest.java
2015-08-20 10:39:38.333041516 -0700
<br>
+++
new/test/java/awt/Graphics2D/RenderClipTest/RenderClipTest.java
2015-08-20 10:39:38.185041512 -0700
<br>
@@ -27,6 +27,7 @@
<br>
* @summary Tests clipping invariance for AA rectangle and line
primitives
<br>
* @run main RenderClipTest -strict -readfile 6766342.tests
<br>
* @run main RenderClipTest -rectsuite -count 10
<br>
+ * @key randomness
<br>
*/
<br>
<br>
import java.awt.*;
<br>
---
old/test/java/awt/GridLayout/ComponentPreferredSize/ComponentPreferredSize.java
2015-08-20 10:39:38.765041527 -0700
<br>
+++
new/test/java/awt/GridLayout/ComponentPreferredSize/ComponentPreferredSize.java
2015-08-20 10:39:38.569041522 -0700
<br>
@@ -36,6 +36,7 @@
<br>
* @build ExtendedRobot
<br>
* @run main ComponentPreferredSize
<br>
* @run main ComponentPreferredSize -hg 20 -vg 20
<br>
+ * @key randomness
<br>
*/
<br>
<br>
public class ComponentPreferredSize {
<br>
---
old/test/java/awt/Window/ShapedAndTranslucentWindows/Shaped.java
2015-08-20 10:39:39.169041537 -0700
<br>
+++
new/test/java/awt/Window/ShapedAndTranslucentWindows/Shaped.java
2015-08-20 10:39:39.017041534 -0700
<br>
@@ -46,6 +46,7 @@
<br>
* @library ../../../../lib/testlibrary
<br>
* @build Common ExtendedRobot
<br>
* @run main Shaped
<br>
+ * @key randomness
<br>
*/
<br>
public class Shaped extends Common{
<br>
<br>
---
old/test/java/awt/Window/ShapedAndTranslucentWindows/ShapedByAPI.java
2015-08-20 10:39:39.549041547 -0700
<br>
+++
new/test/java/awt/Window/ShapedAndTranslucentWindows/ShapedByAPI.java
2015-08-20 10:39:39.397041543 -0700
<br>
@@ -44,6 +44,7 @@
<br>
* @author Dmitriy Ermashov (<a class="moz-txt-link-abbreviated" href="mailto:dmitriy.ermashov@oracle.com">dmitriy.ermashov@oracle.com</a>)
<br>
* @library ../../../../lib/testlibrary
<br>
* @run main ShapedByAPI
<br>
+ * @key randomness
<br>
*/
<br>
public class ShapedByAPI extends Common {
<br>
<br>
---
old/test/java/awt/Window/ShapedAndTranslucentWindows/ShapedTranslucent.java
2015-08-20 10:39:39.933041557 -0700
<br>
+++
new/test/java/awt/Window/ShapedAndTranslucentWindows/ShapedTranslucent.java
2015-08-20 10:39:39.777041553 -0700
<br>
@@ -45,6 +45,7 @@
<br>
* @library ../../../../lib/testlibrary
<br>
* @build Common ExtendedRobot
<br>
* @run main ShapedTranslucent
<br>
+ * @key randomness
<br>
*/
<br>
public class ShapedTranslucent extends Common {
<br>
<br>
---
old/test/java/awt/Window/ShapedAndTranslucentWindows/StaticallyShaped.java
2015-08-20 10:39:40.313041567 -0700
<br>
+++
new/test/java/awt/Window/ShapedAndTranslucentWindows/StaticallyShaped.java
2015-08-20 10:39:40.161041563 -0700
<br>
@@ -45,6 +45,7 @@
<br>
* @library ../../../../lib/testlibrary
<br>
* @build Common ExtendedRobot
<br>
* @run main StaticallyShaped
<br>
+ * @key randomness
<br>
*/
<br>
<br>
public class StaticallyShaped extends Common {
<br>
---
old/test/java/awt/Window/ShapedAndTranslucentWindows/Translucent.java
2015-08-20 10:39:40.697041577 -0700
<br>
+++
new/test/java/awt/Window/ShapedAndTranslucentWindows/Translucent.java
2015-08-20 10:39:40.545041573 -0700
<br>
@@ -43,6 +43,7 @@
<br>
* @library ../../../../lib/testlibrary
<br>
* @build Common ExtendedRobot
<br>
* @run main Translucent
<br>
+ * @key randomness
<br>
*/
<br>
public class Translucent extends Common {
<br>
<br>
---
old/test/java/awt/Window/setLocRelativeTo/SetLocationRelativeToTest.java
2015-08-20 10:39:41.077041586 -0700
<br>
+++
new/test/java/awt/Window/setLocRelativeTo/SetLocationRelativeToTest.java
2015-08-20 10:39:40.921041582 -0700
<br>
@@ -34,6 +34,7 @@
<br>
@library ../../../../lib/testlibrary
<br>
@build ExtendedRobot
<br>
@run main/timeout=1200 SetLocationRelativeToTest
<br>
+@key randomness
<br>
*/
<br>
<br>
public class SetLocationRelativeToTest {
<br>
--- old/test/java/awt/font/LineBreakMeasurer/FRCTest.java
2015-08-20 10:39:41.461041596 -0700
<br>
+++ new/test/java/awt/font/LineBreakMeasurer/FRCTest.java
2015-08-20 10:39:41.309041592 -0700
<br>
@@ -26,6 +26,7 @@
<br>
* @bug 6448405 6519513 6745225
<br>
* @summary static HashMap cache in LineBreakMeasurer can grow
wihout bounds
<br>
* @run main/othervm/timeout=600 -client -Xms16m -Xmx16m
FRCTest
<br>
+ * @key randomness
<br>
*/
<br>
import java.awt.*;
<br>
import java.awt.image.*;
<br>
---
old/test/java/awt/geom/AffineTransform/GetTypeOptimization.java
2015-08-20 10:39:41.845041606 -0700
<br>
+++
new/test/java/awt/geom/AffineTransform/GetTypeOptimization.java
2015-08-20 10:39:41.689041602 -0700
<br>
@@ -29,6 +29,7 @@
<br>
* This test also confirms that isIdentity() returns
the
<br>
* optimal value under all histories of modification.
<br>
* @run main GetTypeOptimization
<br>
+ * @key randomness
<br>
*/
<br>
<br>
import java.awt.geom.AffineTransform;
<br>
---
old/test/java/awt/geom/AffineTransform/TestRotateMethods.java
2015-08-20 10:39:42.221041616 -0700
<br>
+++
new/test/java/awt/geom/AffineTransform/TestRotateMethods.java
2015-08-20 10:39:42.073041612 -0700
<br>
@@ -36,6 +36,7 @@
<br>
*
<br>
* @author flar
<br>
* @run main TestRotateMethods
<br>
+ * @key randomness
<br>
*/
<br>
<br>
import java.awt.geom.AffineTransform;
<br>
--- old/test/java/awt/image/BufferedImage/TinyScale.java
2015-08-20 10:39:42.593041625 -0700
<br>
+++ new/test/java/awt/image/BufferedImage/TinyScale.java
2015-08-20 10:39:42.445041622 -0700
<br>
@@ -22,9 +22,10 @@
<br>
*/
<br>
<br>
/*
<br>
- * @test %W% %E%
<br>
+ * @test
<br>
* @bug 7016495
<br>
* @summary Test tiny scales of BufferedImage
<br>
+ * @key randomness
<br>
*/
<br>
<br>
import java.awt.*;
<br>
--- old/test/javax/imageio/plugins/jpeg/JpegWriterLeakTest.java
2015-08-20 10:39:42.973041635 -0700
<br>
+++ new/test/javax/imageio/plugins/jpeg/JpegWriterLeakTest.java
2015-08-20 10:39:42.817041631 -0700
<br>
@@ -28,6 +28,7 @@
<br>
* even if destroy() or reset() methods is not
invoked.
<br>
*
<br>
* @run main JpegWriterLeakTest
<br>
+ * @key randomness
<br>
*/
<br>
<br>
import java.awt.Color;
<br>
--- old/test/javax/imageio/plugins/png/ShortHistogramTest.java
2015-08-20 10:39:43.345041645 -0700
<br>
+++ new/test/javax/imageio/plugins/png/ShortHistogramTest.java
2015-08-20 10:39:43.193041641 -0700
<br>
@@ -28,6 +28,7 @@
<br>
* hIST chunk if length of image palette in not power
of two.
<br>
*
<br>
* @run main ShortHistogramTest 15
<br>
+ * @key randomness
<br>
*/
<br>
<br>
import java.awt.Color;
<br>
---
old/test/javax/sound/midi/Gervill/SoftFilter/TestProcessAudio.java
2015-08-20 10:39:43.717041654 -0700
<br>
+++
new/test/javax/sound/midi/Gervill/SoftFilter/TestProcessAudio.java
2015-08-20 10:39:43.569041650 -0700
<br>
@@ -24,6 +24,7 @@
<br>
/* @test
<br>
@summary Test SoftFilter processAudio method
<br>
@modules java.desktop/com.sun.media.sound
<br>
+ @key randomness
<br>
*/
<br>
<br>
import java.io.File;
<br>
--- old/test/javax/sound/sampled/FileWriter/AlawEncoderSync.java
2015-08-20 10:39:44.093041664 -0700
<br>
+++ new/test/javax/sound/sampled/FileWriter/AlawEncoderSync.java
2015-08-20 10:39:43.941041660 -0700
<br>
@@ -27,6 +27,7 @@
<br>
* @bug 7058852
<br>
* @summary Tests that Alaw encoder works properly in
multithreaded environment
<br>
* @author Alex Menkov
<br>
+ * @key randomness
<br>
*/
<br>
<br>
import java.io.ByteArrayInputStream;
<br>
--- old/test/javax/swing/JColorChooser/Test4165217.java
2015-08-20 10:39:44.461041673 -0700
<br>
+++ new/test/javax/swing/JColorChooser/Test4165217.java
2015-08-20 10:39:44.309041669 -0700
<br>
@@ -26,6 +26,7 @@
<br>
* @bug 4165217
<br>
* @summary Tests JColorChooser serialization
<br>
* @author Ilya Boyandin
<br>
+ * @key randomness
<br>
*/
<br>
<br>
import java.awt.Color;
<br>
--- old/test/javax/swing/JFileChooser/6868611/bug6868611.java
2015-08-20 10:39:44.833041683 -0700
<br>
+++ new/test/javax/swing/JFileChooser/6868611/bug6868611.java
2015-08-20 10:39:44.681041679 -0700
<br>
@@ -26,6 +26,7 @@
<br>
@summary FileSystemView throws NullPointerException
<br>
@author Pavel Porvatov
<br>
@run main bug6868611
<br>
+ @key randomness
<br>
*/
<br>
<br>
import javax.swing.*;
<br>
--- old/test/javax/swing/JFrame/4962534/bug4962534.java
2015-08-20 10:39:45.205041692 -0700
<br>
+++ new/test/javax/swing/JFrame/4962534/bug4962534.java
2015-08-20 10:39:45.053041688 -0700
<br>
@@ -27,6 +27,7 @@
<br>
@summary JFrame dances very badly
<br>
@author <a class="moz-txt-link-abbreviated" href="mailto:dav@sparc.spb.su">dav@sparc.spb.su</a> area=
<br>
@run applet bug4962534.html
<br>
+ @key randomness
<br>
*/
<br>
import java.applet.Applet;
<br>
import java.awt.*;
<br>
--- old/test/javax/swing/system/6799345/TestShutdown.java
2015-08-20 10:39:45.577041702 -0700
<br>
+++ new/test/javax/swing/system/6799345/TestShutdown.java
2015-08-20 10:39:45.433041698 -0700
<br>
@@ -28,6 +28,7 @@
<br>
@author art
<br>
@modules java.desktop/sun.awt
<br>
@run main TestShutdown
<br>
+ @key randomness
<br>
*/
<br>
<br>
import java.awt.*;
<br>
<br>
</blockquote>
<br>
</blockquote>
<br>
</body>
</html>