bundling native libs in jars or modules

[Johan Vos]

I fully agree it should somehow somewhere sometime be standardized.

But it is extremely unlikely it will be standardized before the Java 11
release, and we need a solution by the time Java 11 is released (and
preferably sooner, as in "tomorrow") hence we need to have this or a
similar workaround in openjfx.

For the real solution, project panama or jigsaw is probably the right place
(and I think a JEP might be required?)

- Johan

On Fri, May 18, 2018 at 3:51 AM Samuel Audet <samuel at skymind.io> wrote:

> Yes, SWT caches in ~/.swt/lib/ by default. This is the kind of thing that
> should be standardized somewhere, but I'm not getting much feedback from
> Oracle:
> http://mail.openjdk.java.net/pipermail/panama-dev/2018-May/001669.html
> We've been needing a mechanism like that with JNLP as well, so this isn't
> anything new. I'd really like to know why we're not getting any feedback
> about this...
>
> Samuel
>
>
> On Fri, May 18, 2018 at 4:59 AM, Tom Schindl <tom.schindl at bestsolution.at>
> wrote:
>
>> To answer my own question the trick swt uses is to encode the build-id
>> into the dll/so so it never needs to delete a file
>>
>> See
>> http://git.eclipse.org/c/platform/eclipse.platform.swt.git/tree/bundles/org.eclipse.swt/Eclipse%20SWT%20PI/common/org/eclipse/swt/internal/Library.java#n144
>>
>> Tom
>>
>> Von meinem iPhone gesendet
>>
>> Am 17.05.2018 um 21:50 schrieb Tom Schindl <tom.schindl at bestsolution.at>:
>>
>> Did you check how SWT does addresses the problem? If you run it outside
>> of OSGi it needs to deal with this problem as well.
>>
>> Tom
>>
>> Von meinem iPhone gesendet
>>
>> Am 17.05.2018 um 21:25 schrieb Johan Vos <johan.vos at gluonhq.com>:
>>
>>
>> Note: This is different from what is discussed in
>>
>> http://mail.openjdk.java.net/pipermail/openjfx-dev/2018-April/021762.html
>>
>>
>> The following text is about bundling native code within a jar, and doesn't
>>
>> talk about how to deal with platform-specific libraries in generic jar
>>
>> files.
>>
>>
>> At this moment, the standalone SDK for JavaFX 11-ea works fine, and javafx
>>
>> apps can be executed since the javafx jars will load the required native
>>
>> code from the libs that are part of the SDK.
>>
>>
>> When we want to download the javafx modules as modules (or jars with
>>
>> module-info), we don't have a local SDK with native libraries. The native
>>
>> libraries need to come with the downloaded jar. There are a number of
>>
>> options to do this, and a number of ongoing discussions (e.g. see
>>
>> http://mail.openjdk.java.net/pipermail/panama-dev/2018-April/001541.html
>> ).
>>
>> In summary, jmods already have support for native code but they are not
>>
>> scoped at runtime, and jars don't have a standard way for supporting
>> native
>>
>> code.
>>
>>
>> I hope there will be a long-term solution for this in the VM in the
>> future,
>>
>> but clearly we can't rely on this for the 11 release.
>>
>> Hence, we need to provide some JavaFX-specific fixes.
>>
>> Just to try the mechanism, I created a quick (and dirty) workaround in the
>>
>> NativeLibLoader. I committed it to a nativelibs branch of my openjfx fork:
>>
>>
>> https://github.com/johanvos/openjdk-jfx/commit/f2c4c57a6197634e9c3bc1298ea14700057155a3
>>
>>
>> The good thing is that all JavaFX native lib loading is already delegated
>>
>> to the NativeLibLoader, so I only had to patch one file.
>>
>>
>> I modified the generated javafx.graphics.jar to include all libs that are
>>
>> generated (something that can be automated in a gradle task), and could
>>
>> successfully run javafx apps using
>>
>> java -p /path/to/my/jars --add-modules javafx.controls
>>
>> without specifying java.library.path or native libs or so.
>>
>>
>> This works for jars that I put myself on the classpath, hence it should
>>
>> also work for jars downloaded from maven central etc.
>>
>>
>> In case there will be a standard for bundling native libs with jar files
>> in
>>
>> a future JVM, we can remove this hack. But even in the meantime, there are
>>
>> better solutions.
>>
>> One of the best options I think we should consider is using JavaCPP (
>>
>> https://github.com/bytedeco/javacpp)  . The main author of JavaCPP,
>> Samuel
>>
>> Audet (in cc) describes here (
>>
>>
>> https://github.com/tensorflow/tensorflow/issues/18397#issuecomment-381371636
>> )
>>
>> why my hack will fail on Windows.
>>
>>
>> Another advantage of JavaCPP is that it is proven to work on ios and
>>
>> Android as well. Actually, that would allow us to get rid of the
>>
>> ios-specific code that is currently in NativeLibLoader.java.
>>
>>
>> Again, this is only tackling the issue of bunding native code with jars
>> and
>>
>> not dealing with platform-specific resolution (although JavaCPP can help
>>
>> there as well).
>>
>>
>> - Johan
>>
>>
>>
>