OpenSSL and panama-foreign

[Rémy Maucherat]

Hi,

Apache Tomcat currently uses a JNI wrapper, called tomcat-native, to
be able to use OpenSSL. It has been a long term goal to replace that
JNI wrapper with "something else", as maintenance is annoying, support
and debugging is annoying, adding new features is annoying, etc etc.

After a failed attempt three years ago with Panama, the recent JEP 412
in Java 17 made it seem things were getting more mature and it looked
feature complete enough for our use case.

The good news is I was able to produce a functional equivalent of the
OpenSSL JNI based code using the new APIs:
Original: https://github.com/apache/tomcat/tree/10.1.0-M7/java/org/apache/tomcat/util/net/openssl
and portions of
https://github.com/apache/tomcat-native/tree/1.2.31/native
Java 17: https://github.com/apache/tomcat/tree/10.1.0-M7/modules/openssl-java17
panama-foreign: https://github.com/rmaucher/openssl-panama-foreign

Overall, this is very good, the needed functionality is there and the
performance seems equivalent to JNI.

However, I am running into cores (apparently caused by memory
corruption) with the panama-foreign branch, while the Java 17 version
seems solid.

Example of a core: https://pastebin.com/Acq4tFgR
Basically, some memory corruption causes crashes on some cleanup or
other operations. It only happens under load with concurrency (I use
something like "ab -n 10000 -c 10 https://127.0.0.1:8443/tomcat.svg"
to generate a burst of parallel handshaking activity). The Java 17
version passes the same test. Things work fine if there's no
significant load or if all operations are serialized using sync.

I am using shared scopes for the SSL context and SSL engine. Each
engine has its own scope, so allocated memory would be released once
done with the associated connection.

Any ideas of what could explain the problems with panama-foreign ?

Thanks,
Rémy