Loading native libraries from the classpath

[Maurizio Cimadamore]

On 16/01/2024 01:43, Sebastian Stenzel wrote:
> Hi,
>
> I basically agree with this being an application packaging problem, however the jlink mechanism you mentioned only works with jmods, not with jar files or am I mistaken?
>
> The problem with jmods is that afaik you can't publish them to Maven Central, ruling out this option for library developers. Unless you want to force downstream users to download your library manually like we did in medieval times. 😉 Maybe this needs to be solved by the Maven team, making jmods easier to distribute, but they are very careful when it comes to allowing deployment of further file formats on Central, as can be seen with Sigstore. [1]

Yeah, unfortunately that is the conundrum - which is why I didn't just 
say "jmods for the winner" :-)

This seems a general area where modules and custom JDK images could add 
a lot of value, simplifying testing and launcher configurations. 
Unfortunately none of it works out of the box, not with IDEs, not with 
existing build tools (in jextract we had to do a couple of hacks to be 
able to integrate jlink in our workflow), which leads to the current 
chicken-and-egg problem.

That said, I'm starting to see a push towards trying to ship monolithic 
application images [1] where application and supporting libraries 
(native and not) are provided in a single executable launcher. If 
efforts like these prove successful, they might indicate a way forward 
from where we are now.

> So in my opinion the extract-from-jar approach is merely a workaround for a missing proper solution.

Yes!

Maurizio

[1] - 
https://mail.openjdk.org/pipermail/leyden-dev/2023-February/000106.html

>
> Cheers,
> Sebastian
>
> [1]https://central.sonatype.org/news/20220310_sigstore/
>
>> Am 15.01.2024 um 19:10 schrieb Maurizio Cimadamore<maurizio.cimadamore at oracle.com>:
>>
>> Hi,
>> extracting libraries from jars into some temp folder, and loading from there is indeed a common use case.
>>
>> That said, the logic to support this is rather fragile, it is not uncommon to run code in machines which don't have write access to /tmp - in which case such an approach would fail.
>>
>> For this reasons, at the time being we have not provided explicit support for this mode of operation - we tend to view this more as a packaging problem, rather than an API one. For instance, when linking modules into a custom JDK image (using jlink) it is possible to bring in custom native libraries as well, which greatly simplifies running/testing an application (we use this technique for jextract itself [1]). But I understand that much of the toolchain (gradle, maven et. alias) are mostly written with jars in mind, so this might not be an option.
>>
>> Maurizio
>>
>> [1] -https://github.com/openjdk/jextract/blob/master/build.gradle#L97
>>
>>
>> On 15/01/2024 16:51, Jeffrey Yemin wrote:
>>> Hi,
>>>
>>> It seems fairly common for modules published to Maven Central to embed native libraries in the .jar file itself, in order to avoid the complications of applications having to install the native libraries themselves.
>>>
>>> Some libraries (including one,https://github.com/mongodb/libmongocrypt/tree/master/bindings/java/mongocrypt, that I maintain for MongoDB) delegate the extraction of the appropriate native library to JNA, while others that use JNI directly (likehttps://github.com/xerial/snappy-java) implement the extraction via custom code (seehttps://github.com/xerial/snappy-java/blob/v1.1.10.5/src/main/java/org/xerial/snappy/SnappyLoader.java#L53-L56).
>>>
>>> As the community starts to move away from JNI and JNA to the new Foreign Function API defined in JEP 454, are there any plans to support the automatic loading of native libraries from the classpath in the Java platform itself?
>>>
>>>
>>> Thanks,
>>> Jeff Yemin
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/panama-dev/attachments/20240116/98a95358/attachment.htm>