[intrinsics]: performance before after (String::format)

[Vicente Romero]

Hi,

I have executed some performance tests on the intrinsics code to compare 
the before and after. Please find the benchmark results and the JMH 
based benchmark attached. This benchmark is based on a previous one 
written by Hannes. The benchmark compares the execution between the JDK 
built from [1], referred here as JDK13, and [2] which is the amber repo, 
branch `intrinsics-project`.

Some conclusions from the benchmark results:

  * the intrinsified code is faster in all cases, for which intrinsified
    code is produced, compared to the legit (JDK13 vanilla) code
  * there are wide variations though

For example for the test: `testStringFormatBoxedArray` which is 
basically benchmarking the performance of: `String.format("%s: %d ", 
args);` where args is: `static final Object[] args = { "Bob", i23 };`, 
there is basically no visible gain as in this case the intrinsification 
is bailing out and producing same code as vanilla JDK13. This result is 
expected. The next test with not so much gain is: 
`testStringFormat1ConstantFloat` which is testing:

     `String.format("%g", 1.0)`

the execution is ~2.5 times faster in the intrinsified version but 
nothing compared to: `testStringFormat1ConstantStr` which is ~40 times 
faster. Another interesting conclusion is that the improvement fades out 
with the number of parameters for some cases but keeps constant for 
others. For example it is as fast to concatenate 1 or 100 strings but 
formating one primitive int is ~45 times faster vs a 3.5 improvement 
when formating a hundred.

I have also attached the table I used to play with the numbers.

Thanks,

Vicente

[1] http://hg.openjdk.java.net/jdk/jdk

[2] http://hg.openjdk.java.net/amber/amber

-------------- next part --------------
JDK13
Benchmark                                                   Mode  Cnt    Score    Error   Units
FormatterBenchmark.testStringFormat100ConstantInts         thrpt    5    2.864 ±  0.340  ops/ms
FormatterBenchmark.testStringFormat100ConstantStrs         thrpt    5    3.228 ±  0.666  ops/ms
FormatterBenchmark.testStringFormat100VariableInts         thrpt    5    2.816 ±  0.341  ops/ms
FormatterBenchmark.testStringFormat100VariableStrs         thrpt    5    3.211 ±  0.575  ops/ms
FormatterBenchmark.testStringFormat1ConstantBoolean        thrpt    5  315.551 ±  8.292  ops/ms
FormatterBenchmark.testStringFormat1ConstantChar           thrpt    5  279.724 ± 23.779  ops/ms
FormatterBenchmark.testStringFormat1ConstantFloat          thrpt    5  182.302 ± 59.609  ops/ms
FormatterBenchmark.testStringFormat1ConstantHashCode       thrpt    5  254.617 ± 66.590  ops/ms
FormatterBenchmark.testStringFormat1ConstantHashCodeUpper  thrpt    5  239.205 ± 47.271  ops/ms
FormatterBenchmark.testStringFormat1ConstantHex            thrpt    5  251.704 ± 29.072  ops/ms
FormatterBenchmark.testStringFormat1ConstantInt            thrpt    5  224.860 ± 40.024  ops/ms
FormatterBenchmark.testStringFormat1ConstantStr            thrpt    5  221.417 ± 12.912  ops/ms
FormatterBenchmark.testStringFormat1VariableInt            thrpt    5  194.054 ± 51.083  ops/ms
FormatterBenchmark.testStringFormat1VariableStr            thrpt    5  239.140 ±  8.922  ops/ms
FormatterBenchmark.testStringFormat2ConstantInt            thrpt    5  125.693 ± 18.497  ops/ms
FormatterBenchmark.testStringFormat2ConstantStrs           thrpt    5  113.263 ± 12.821  ops/ms
FormatterBenchmark.testStringFormat2VariableInts           thrpt    5  113.932 ±  3.371  ops/ms
FormatterBenchmark.testStringFormat2VariableStrs           thrpt    5   87.530 ± 17.973  ops/ms
FormatterBenchmark.testStringFormatBoxedArray              thrpt    5   95.182 ± 16.235  ops/ms
FormatterBenchmark.testStringFormatConstantStrInt          thrpt    5  122.026 ± 37.231  ops/ms
FormatterBenchmark.testStringFormatConstantStrInteger      thrpt    5  122.967 ± 18.317  ops/ms


JDK13_intrinsics
Benchmark                                                   Mode  Cnt      Score      Error   Units
FormatterBenchmark.testStringFormat100ConstantInts         thrpt    5     10.190 ±    0.537  ops/ms
FormatterBenchmark.testStringFormat100ConstantStrs         thrpt    5    170.628 ±  407.232  ops/ms
FormatterBenchmark.testStringFormat100VariableInts         thrpt    5      9.793 ±    1.247  ops/ms
FormatterBenchmark.testStringFormat100VariableStrs         thrpt    5    148.536 ±  423.740  ops/ms
FormatterBenchmark.testStringFormat1ConstantBoolean        thrpt    5  10362.809 ±  652.923  ops/ms
FormatterBenchmark.testStringFormat1ConstantChar           thrpt    5  10927.519 ±  372.063  ops/ms
FormatterBenchmark.testStringFormat1ConstantFloat          thrpt    5    450.377 ±   89.512  ops/ms
FormatterBenchmark.testStringFormat1ConstantHashCode       thrpt    5   3982.346 ±  919.287  ops/ms
FormatterBenchmark.testStringFormat1ConstantHashCodeUpper  thrpt    5   2959.208 ±   44.042  ops/ms
FormatterBenchmark.testStringFormat1ConstantHex            thrpt    5   6786.572 ± 1434.287  ops/ms
FormatterBenchmark.testStringFormat1ConstantInt            thrpt    5  10023.419 ±  513.649  ops/ms
FormatterBenchmark.testStringFormat1ConstantStr            thrpt    5   8985.132 ± 1269.345  ops/ms
FormatterBenchmark.testStringFormat1VariableInt            thrpt    5  10037.494 ±  445.459  ops/ms
FormatterBenchmark.testStringFormat1VariableStr            thrpt    5  10442.803 ±  460.030  ops/ms
FormatterBenchmark.testStringFormat2ConstantInt            thrpt    5   9636.416 ± 1662.162  ops/ms
FormatterBenchmark.testStringFormat2ConstantStrs           thrpt    5   9654.380 ±  237.301  ops/ms
FormatterBenchmark.testStringFormat2VariableInts           thrpt    5   9664.939 ±  422.009  ops/ms
FormatterBenchmark.testStringFormat2VariableStrs           thrpt    5   9497.214 ± 1245.206  ops/ms
FormatterBenchmark.testStringFormatBoxedArray              thrpt    5    126.647 ±    9.318  ops/ms
FormatterBenchmark.testStringFormatConstantStrInt          thrpt    5   8800.224 ± 1269.453  ops/ms
FormatterBenchmark.testStringFormatConstantStrInteger      thrpt    5   6584.981 ± 1049.194  ops/ms