JMH 1.0 SNAPSHOT Failed ResultFormatTest

[Bernd Eckenfels]

Am Fri, 15 Aug 2014 23:32:16 +0400
schrieb Aleksey Shipilev <aleksey.shipilev at oracle.com>:

> This must be specific to Windows, since CI builds on Linux are fine.
> 
> > Failed tests:
> >   ResultFormatTest.jsonTest_Stream:137->compare:109 Mismatch
> > expected:<...         "rawData" :[ []> but was:<...
> > "rawData" :[]> ResultFormatTest.jsonTest:123->compare:109 Mismatch
> > expected:<...         "rawData" :[ []> but was:<...
> > "rawData" :[]>
> 
> Looks like a trailing whitespace to me... CRLF? Will check on some
> Windows machines around.

Hm no, does not look line line endings, you use lines with the
readLine. When I change the golden file to contain CRLF instead of LF
(which it gets from mercurial) then the test fails as well.

What I noticed is, the golden file contains different spacing for
rawData array:


L37         "scoreUnit" : "ops/ms",
            "rawData" : [
                [
                    942.0,
                    384.0
                ],

...
L89             "scoreUnit" : "ops/ms",
                "rawData" :[
                    [
                        390.0,
                        787.0
                    ],

And when I add code to print out the data you can see the file starts
to differ at line 38 with different formatting (. means actual line):

 0037             "scoreUnit" : "ops/ms",
.0037             "scoreUnit" : "ops/ms",
 0038             "rawData" :[
.0038             "rawData" :
 0039                 [
.0039  [
 0040                     942.0,
.0040                 [
 0041                     384.0
.0041                     942.0,
 0042                 ],
.0042                     384.0

Maybe different JSON toolkit versions?

Gruss
Bernd

diff -r d5526434fd4a
jmh-core/src/test/java/org/openjdk/jmh/results/format/ResultFormatTest.java
---
a/jmh-core/src/test/java/org/openjdk/jmh/results/format/ResultFormatTest.java
Fri Aug 15 18:38:54 2014 +0400 +++
b/jmh-core/src/test/java/org/openjdk/jmh/results/format/ResultFormatTest.java
Fri Aug 15 23:07:24 2014 +0200 @@ -101,12 +101,17 @@ }

     private void compare(String actualFile, String goldenFile) throws
     IOException {
+        System.out.printf("%s vs. %s%n", goldenFile, actualFile);
         BufferedReader actualReader = new BufferedReader(new
     FileReader(actualFile)); BufferedReader goldenReader = new
     BufferedReader(new
     InputStreamReader(ResultFormatTest.class.getResourceAsStream("/org/openjdk/jmh/results/format/"
     + goldenFile)));
+        int line=0;
         while (true) {
             String goldenLine = goldenReader.readLine();
             String actualLine = actualReader.readLine();
-            Assert.assertEquals("Mismatch", goldenLine, actualLine);
+            line++;
+            System.out.printf(" %04d %s%n", line, goldenLine);
+            System.out.printf(".%04d %s%n", line, actualLine);
+            //Assert.assertEquals("Mismatch L" + line, goldenLine,
     actualLine); if (goldenLine == null && actualLine == null) break;
         }
     }

C