[intrinsics]: performance before after (String::format)

[Vicente Romero]

Hi,

To complete the picture please find attached the performance results for 
Objects.hash for a number of experiments. In general they don't look as 
good as the ones for String::format. In general it seems like there is 
no much gain unless the number of parameters is large and all the 
parameters are constants. This is understandable because the compiler 
generates an LDC of the result. In all other cases the performance is 
just a bit better or a lot worst.

Thanks,
Vicente

On 2/22/19 12:33 PM, Vicente Romero wrote:
> Hi,
>
> I have executed some performance tests on the intrinsics code to 
> compare the before and after. Please find the benchmark results and 
> the JMH based benchmark attached. This benchmark is based on a 
> previous one written by Hannes. The benchmark compares the execution 
> between the JDK built from [1], referred here as JDK13, and [2] which 
> is the amber repo, branch `intrinsics-project`.
>
> Some conclusions from the benchmark results:
>
>   * the intrinsified code is faster in all cases, for which
>     intrinsified code is produced, compared to the legit (JDK13
>     vanilla) code
>   * there are wide variations though
>
> For example for the test: `testStringFormatBoxedArray` which is 
> basically benchmarking the performance of: `String.format("%s: %d ", 
> args);` where args is: `static final Object[] args = { "Bob", i23 };`, 
> there is basically no visible gain as in this case the 
> intrinsification is bailing out and producing same code as vanilla 
> JDK13. This result is expected. The next test with not so much gain 
> is: `testStringFormat1ConstantFloat` which is testing:
>
>     `String.format("%g", 1.0)`
>
> the execution is ~2.5 times faster in the intrinsified version but 
> nothing compared to: `testStringFormat1ConstantStr` which is ~40 times 
> faster. Another interesting conclusion is that the improvement fades 
> out with the number of parameters for some cases but keeps constant 
> for others. For example it is as fast to concatenate 1 or 100 strings 
> but formating one primitive int is ~45 times faster vs a 3.5 
> improvement when formating a hundred.
>
> I have also attached the table I used to play with the numbers.
>
> Thanks,
>
> Vicente
>
> [1] http://hg.openjdk.java.net/jdk/jdk
>
> [2] http://hg.openjdk.java.net/amber/amber
>

-------------- next part --------------
JDK13

Benchmark                                       Mode  Cnt      Score      Error   Units
FormatterBenchmark.testHash100DoubleConstants  thrpt    5     23.385 ±    2.846  ops/ms
FormatterBenchmark.testHash100IntConstants     thrpt    5     21.588 ±    4.107  ops/ms
FormatterBenchmark.testHash100IntVariables     thrpt    5     26.592 ±    3.031  ops/ms
FormatterBenchmark.testHash100References       thrpt    5    430.670 ±   21.266  ops/ms
FormatterBenchmark.testHash1DoubleConstant     thrpt    5  21187.706 ±  449.976  ops/ms
FormatterBenchmark.testHash1IntConstant        thrpt    5  41139.332 ± 3865.956  ops/ms
FormatterBenchmark.testHash1IntVariable        thrpt    5  42798.895 ± 2254.518  ops/ms
FormatterBenchmark.testHash1Reference          thrpt    5  36007.401 ± 3389.955  ops/ms
FormatterBenchmark.testHash2DoubleConstants    thrpt    5   4502.750 ±  266.133  ops/ms
FormatterBenchmark.testHash2IntConstants       thrpt    5   9027.443 ±  333.040  ops/ms
FormatterBenchmark.testHash2IntVariables       thrpt    5   9018.512 ±  237.860  ops/ms
FormatterBenchmark.testHash2References         thrpt    5  12513.845 ±  675.880  ops/ms


JDK13+intrinsics
Benchmark                                       Mode  Cnt      Score       Error   Units
FormatterBenchmark.testHash100DoubleConstants  thrpt    5  42191.208 ±  3341.833  ops/ms
FormatterBenchmark.testHash100IntConstants     thrpt    5  42120.514 ±  6430.702  ops/ms
FormatterBenchmark.testHash100IntVariables     thrpt    5      3.993 ±     1.070  ops/ms
FormatterBenchmark.testHash100References       thrpt    5      3.425 ±     1.386  ops/ms
FormatterBenchmark.testHash1DoubleConstant     thrpt    5  41651.197 ±  6179.802  ops/ms
FormatterBenchmark.testHash1IntConstant        thrpt    5  38128.380 ± 16815.052  ops/ms
FormatterBenchmark.testHash1IntVariable        thrpt    5  42564.255 ±  1612.840  ops/ms
FormatterBenchmark.testHash1Reference          thrpt    5  30692.986 ±  6473.312  ops/ms
FormatterBenchmark.testHash2DoubleConstants    thrpt    5  36643.845 ±  9326.509  ops/ms
FormatterBenchmark.testHash2IntConstants       thrpt    5  42661.835 ±  1466.292  ops/ms
FormatterBenchmark.testHash2IntVariables       thrpt    5  41573.286 ±  3085.629  ops/ms
FormatterBenchmark.testHash2References         thrpt    5  35126.523 ±   848.846  ops/ms