Sharing experience with FFM API and jextract

[Maurizio Cimadamore]

Hi Zoran, thanks for the feedback! Some comments inline below

On 17/10/2025 18:14, Zoran Sevarac wrote:
> Hi all,
>
> I want to share our experience and some benchmark results using FFM 
> API and jextract in Deep Netts.
>
> We've used jextract to generate bindings for CUDA libraries to create 
> a replacement for JCUDA  that was previously used, and here are some 
> observations (note that we didn't generate entire API just a subset 
> that we needed)
>
> What we liked:
> 1. Generating bindings using jextract is pretty straightforward.  It 
> requires a bit of experimentation and practice to figure out available 
> options  and the best configuration.
> 2. We actually didn't have to write any binding code, everything was 
> generated, and it will be easy to regenerate for a new version when 
> required.
>
> Performance and benchmarking
> We benchmarked 100 iterations of image classification using VGGNet 
> convolutional neural network:
> 1. We got the same performance as with JNI (used by JCuda).
> 2. Nearly same performance as PyTorch (Python and C)
> 3. Java 25 performed significantly better and more stable than Java 
> 22, while running the benchmark.
> Java 25, required only one warmup iteration after which results were 
> stable, and had significantly lower variation overall, while Java 22 
> required up to 40 warmup iterations to stabilize, and had large 
> variations  (up to 8x).
> So difference in stability between Java 22 and 25 is huge (at least 
> for our use case)
I'm glad that you got reasonable performance out of it, and that you 
noticed the jump from 22 to 25. A lot of polishing has happened behind 
the curtain to make warmup faster (and more is to come :-) ).
>
> Few observations
> 1. Enums are mapped to functions. It would make more sense if they 
> could be Java enums.

See:

https://git.openjdk.org/jextract/pull/284

It is a tricky topic, hopefully that PR summarizes the main reasons as 
to why that isn't the slamdunk it seems it is...

> 2. Code for dependent libraries was duplicated  for each library if it 
> was in a different package. The solution for us was to put all 
> generated bindings/classes in a single package.

Yep. With jextract filtering you can at least generate different 
libraries in different packages though. Have you tried that?

https://github.com/openjdk/jextract/blob/master/doc/GUIDE.md#filtering

> 3. Binding for C types are duplicated for each .h file/generated 
> class. Would be cleaner if these are generated on only one shared place
Yep -- this is an open issue that we'd like to address. It is also 
problematic for JDK adoption. Likely at some point it will be possible 
to include/exclude these as well.
> 4. Generated API will always require a layer on top of it. It strictly 
> corresponds to native API style, for C with pointers/MemorySegments 
> and status code returns.
This is sort of by design. Jextract is intended to give you usable 
bindings that do not require direct use of method handles/var handles, 
but that introduce the least amount possible (none virtually) of 
performance overhead. The task of making bindings hgher-level is a lot 
more creative, and requires good taste in applying and coupling language 
features with API design so as to achieve optimal balance for users. A 
tool can generate _one_ opinionated high-level API. But there are as 
many opinionated APIs as there are users (see your comments on enums :-) 
), so the only principled choice here is sticking to the C API.
> 5. Jextract is available only in version 22 (and based on JDK22), 
> which can add confusion which version to use for which version of Java 
> https://jdk.java.net/jextract/

This is confusing I agree. We plan to ship 25 binaries soon, and we'll 
try to make sure the versions don't diverge too much in the future

Thanks

Maurizio