RFR: 8316814: NMT: A separate script or Java program is needed to analyze and make useful reports ouf of JMH benchmarks outputs.

[Afshin Zafari]

On Tue, 10 Oct 2023 18:09:39 GMT, Gerard Ziemski <gziemski at openjdk.org> wrote:

> Can we add error values to the report please?

The error values are added to the report now.

> Would a layout like this be easier to parse (everything on one line)
> 
> ```
>                               Method Threads   Off    Summary Diff     Detail Diff
>               mixAallocateFreeMemory     0     40.41    50.44 24.83%   87.01  115.34%
>               mixAallocateFreeMemory     4      2.30     2.19 -4.65%   2.47   7.57%
>          mixAllocateReallocateMemory     0    158.70   170.41 7.38%    225.05 41.81%
>          mixAllocateReallocateMemory     4      2.28     2.34 2.72%    2.25   -1.53%
>                   onlyAllocateMemory     0    113.95   117.48 3.10%    147.35 29.31%
>                   onlyAllocateMemory     4      2.16     2.37 9.84%    2.29   6.03%
> ```
> 
> I'm not sure myself yet which one I prefer, they both look a bit busy, but this one requires less vertical scanning, but more horizontal scanning...

Fixed.
New layout is: 

                                  Method Threads             Off                  Summary                          Detail
                  mixAallocateFreeMemory     0     40.405 (±  8.115)     50.436 (±  4.576)  24.826%    87.010 (± 10.630) 115.345%
                  mixAallocateFreeMemory     4      2.299 (±  0.658)      2.192 (±  0.737)  -4.654%     2.473 (±  0.728)   7.569%
             mixAllocateReallocateMemory     0    158.700 (± 10.750)    170.408 (±  4.556)   7.377%   225.054 (±  4.381)  41.811%
             mixAllocateReallocateMemory     4      2.282 (±  0.723)      2.344 (±  0.645)   2.717%     2.247 (±  0.599)  -1.534%
                      onlyAllocateMemory     0    113.949 (± 21.257)    117.480 (±  8.846)   3.099%   147.346 (±  8.226)  29.309%
                      onlyAllocateMemory     4      2.155 (±  0.682)      2.367 (±  0.694)   9.838%     2.285 (±  0.611)   6.032%

> I think we might need 3 decimal places to show to the user (like the original output).
> 
> When I try the numbers as shown say here:
> 
> ```
>               mixAallocateFreeMemory     4      2.30   2.19    2.47
>                                                       -4.65%   7.57%
> ```
> 
> 20.19 - 2.30 == -0.11 --> -0.11 / 2.30 --> -0.047826086956522 * 100 --> -4.78%
> 
> I get quite a different number --> -4.78% vs -4.65%

Excellent catch! Numbers shown with 3 decimal digits.

> Just noticed a small typo in mixAallocateFreeMemory, it should be mixAllocateFreeMemory I think.

Unfortunately the typo is in the original JMH test. Since that change is not yet merged, the typo cannot be fixed in the test now.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/16070#issuecomment-1759144141
PR Comment: https://git.openjdk.org/jdk/pull/16070#issuecomment-1759145162
PR Comment: https://git.openjdk.org/jdk/pull/16070#issuecomment-1759146635
PR Comment: https://git.openjdk.org/jdk/pull/16070#issuecomment-1761750396