Vector API: passing vectors as arguments

[Piotr Rżysko]

Hi,

I use the Vector API in a JSON parser (simdjson-java [1]) that I’ve been
developing. Recently, I’ve noticed that sometimes when I have a loop that
performs operations on vectors, extracting helper methods from the loop and
passing vectors to them causes a significant drop in performance.

To illustrate the problem, I’ve prepared several implementations of the
same algorithm. The algorithm has been extracted from the parser:

   -

   OriginalStructuralIndexer [2]: An initial implementation in which, to
   reuse some pieces of code, I extracted multiple helper methods that perform
   operations on the vectors loaded by the main loop.
   -

   LoadingInStepStructuralIndexer [3]: A modified version of the
   OriginalStructuralIndexer in which vector loading is done in the method
   called from the loop; performance is significantly better.
   -

   InlinedStepStructuralIndexer [4]: More operations on vectors are
   manually inlined; performance is slightly better compared to the
   LoadingInStepStructuralIndexer.
   -

   InlinedIndexStructuralIndexer [5]: All operations on vectors are
   manually inlined in the main loop; performance is the best out of all the
   implementations.

Please take a look at the comments at the top of each class. They include
the results I obtained from running benchmarks [6] of the implementations
on my desktop (256-bit registers, Temurin-21.0.1). If you would like to run
the benchmarks, please follow the instructions in the README [7].

Overall, the most surprising to me was the poor performance of the
OriginalStructuralIndexer, which I assume was caused by vector boxing and
the JIT’s inability to inline the helper methods. I have two questions
regarding this:

   1. As a general rule, to achieve the best possible performance, should
   all vector operations in hot methods/loops always be manually inlined? If
   that’s the case, is there still a possibility that even in fully inlined
   code vector boxing occurs?
   2.

   Are vectors declared as static final fields guaranteed to be inlined
   properly in methods/loops using them, or is it safer to always explicitly
   create them within methods/loops?

Best regards,
Piotr

[1] https://github.com/simdjson/simdjson-java
[2]
https://github.com/piotrrzysko/vector-api-benchmarks/blob/main/src/main/java/io/github/piotrrzysko/simdjson/OriginalStructuralIndexer.java
[3]
https://github.com/piotrrzysko/vector-api-benchmarks/blob/main/src/main/java/io/github/piotrrzysko/simdjson/LoadingInStepStructuralIndexer.java
[4]
https://github.com/piotrrzysko/vector-api-benchmarks/blob/main/src/main/java/io/github/piotrrzysko/simdjson/InlinedStepStructuralIndexer.java
[5]
https://github.com/piotrrzysko/vector-api-benchmarks/blob/main/src/main/java/io/github/piotrrzysko/simdjson/InlinedIndexStructuralIndexer.java
[6]
https://github.com/piotrrzysko/vector-api-benchmarks/blob/main/src/jmh/java/io/github/piotrrzysko/StructuralIndexerBenchmark.java
[7] https://github.com/piotrrzysko/vector-api-benchmarks/blob/main/README.md
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/panama-dev/attachments/20240531/1b1d9b06/attachment-0001.htm>