Sharing experience with FFM API and jextract

[Zoran Sevarac]

Hi all,

I want to share our experience and some benchmark results using FFM API and
jextract in Deep Netts.

We've used jextract to generate bindings for CUDA libraries to create a
replacement for JCUDA  that was previously used, and here are some
observations (note that we didn't generate entire API just a subset that we
needed)

What we liked:
1. Generating bindings using jextract is pretty straightforward.  It
requires a bit of experimentation and practice to figure out  available
options  and the best configuration.
2. We actually didn't have to write any binding code, everything was
generated, and it will be easy to regenerate for a new version when
required.

Performance and benchmarking
We benchmarked 100 iterations of image classification using VGGNet
convolutional neural network:
1. We got the same performance as with JNI (used by JCuda).
2. Nearly same performance as PyTorch (Python and C)
3. Java 25 performed significantly better  and more stable than  Java 22,
while running the benchmark.
Java 25, required only one warmup iteration after which results were
stable, and had significantly lower variation overall, while Java 22
required up to 40 warmup iterations to stabilize, and had large variations
(up to 8x).
So difference in stability between Java 22 and 25 is huge (at least for our
use case)

Few observations
1. Enums are mapped to functions. It would make more sense if they could be
Java enums.
2. Code for dependent libraries was duplicated  for each library if it was
in a different package. The solution for us was to put all generated
bindings/classes in a single package.
3. Binding for C types are duplicated for each .h file/generated class.
Would be cleaner if these are generated on only one shared place
4. Generated API will always require a layer on top of it. It strictly
corresponds to native API style, for C with pointers/MemorySegments and
status code returns.
5. Jextract is available only in version 22 (and based on JDK22), which can
add confusion which version to use for which version of Java
https://jdk.java.net/jextract/
It would be great if it follows the releases of JDK

I hope this helps and keep doing great work.

Best regards
Zoran Sevarac

.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/panama-dev/attachments/20251017/c2003b0d/attachment.htm>