<div dir="ltr">Thanks! Yes creating JMODs is one way to do it, but if you were to distribute jextract as a library and not just an application you'd get user complaints because it'd take more work to consume:<div><ul><li>JMODs are platform specific but build tools don't make it easy to select the right artifacts based on platforms. You can do it but it often requires custom Maven/Gradle plugins and such.<br><br></li><li>You can't put JMODs on the module path so users would now need to run jlink to get a JDK they could use, but build tools don't invoke jlink during the normal development cycle and don't easily support switching which JDK they use half way through based on the output of build tasks (maybe they should but last time I tried, they don't). You could write extra plugins to teach them to do that maybe, but then you'd hit performance problems - jlinking is a fairly heavy and slow operation, and doesn't cache anything. All this complexity is why some vendor JDKs pre-jlink JavaFX.<br><br></li><li>It requires libraries to be modular and jlinkable. Sadly this is often quite challenging :( e.g. out of the box a vanilla Spring Boot Web app can't be linked because the module graph is invalid. The Spring guys know and don't plan to fix it. See <a href="https://github.com/spring-projects/spring-boot/issues/33942">https://github.com/spring-projects/spring-boot/issues/33942</a></li></ul><div>So this runs into a fundamental question that Jigsaw never really resolved and maybe Leyden needs to: is jlink meant to be a last step optimization process you run before distribution (today, yes) or is it meant to be an integrated part of the compile-and-test cycle? If it's the former you can get bugs that only appear at distribution time. If it's the latter then it needs a different performance and compatibility model. GraalVM Native Image faces the same problem: it's slow to compile and introduces new bugs, so it's tough to integrate it into the standard development and testing process. Ordinary HotSpot with a classpath/module path is so great for development because changing things about your app is just so darn fast, users are loath to lose that. Hence the proliferation of libraries that extract code on the fly. It's inelegant and creates other problems but it preserves the ultra-fast build loops that people love so much, and doesn't require special support from build systems.</div></div><div><br></div><div>Sponsored Link: maybe jextract should be distributed with Conveyor ;) It'd be convenient to have installs that can keep themselves up to date, added to the path automatically etc.</div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, 24 Feb 2023 at 16:09, Maurizio Cimadamore <<a href="mailto:maurizio.cimadamore@oracle.com">maurizio.cimadamore@oracle.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p><br>
</p>
<div>On 24/02/2023 14:45, Mike Hearn wrote:<br>
</div>
<blockquote type="cite">
<div>Especially once Panama ships the Java ecosystem could
strongly benefit from a standardization of how native components
are bundled into and loaded from libraries, as current build
systems and the JVM tooling don't have much to say on the topic.
The result is a lot of wheel reinvention across the ecosystem in
the form of NativeLibraryLoader.java classes, always unique per
project, and a bunch of bugs / developer friction that doesn't
need to happen.</div>
</blockquote>
<p>I tend to agree with the overall assessment. Shipping native
libraries with Java projects is a known pain point, and it would
be nice to have some solution for that. That being said, while I'm
aware that the best way to make things work in today's world is by
shipping native libraries in a jar, and then extract them
_somewhere_, so that they can be loaded with
`System::loadLibrary`, I'm not sure how much that can be viewed as
a full solution, rather than a workaround. I can imagine cases
where extracting libraries into a custom folder is not feasible
(e.g. because of missing permissions). My general feeling is that,
with jars and native libraries it's like trying to fit a round peg
in a square hole: surely you can devise some pragmatic solution
which makes things sort of work, but what you get is always a
little brittle.</p>
<p>If you look at what we did for jextract [1], the approach we used
was different: jextract is written entirely in Java, but has a
dependency (via Foreign Function & Memory API) on libclang.
When we build jextract, we create a jmod [2] for jextract, with
the native library for libclang in the right place. We then create
a JDK image which contains jdk.compiler, java.base and the newly
created jextract module. The resulting JDK will have the libraries
in the right place. This means that we can provide a launcher
simply by calling jextract's entry point using the custom JDK
image. You can run jextract and run all jextract tests against
this custom image, which then requires zero extra custom arguments
passed on the command line (because the native libraries are added
in the right place already).</p>
<p>An approach such as this seems more promising than doing heroics
with jarfiles, at least for applications, and one that could be
more amenable to things like code signing (in jextract we don't do
this, but I don't see why that could not be added).<br>
</p>
<p>Maurizio</p>
<p>[1] - <a href="https://jdk.java.net/jextract/" target="_blank">https://jdk.java.net/jextract/</a><br>
[2] - <a href="https://github.com/openjdk/jextract/blob/master/build.gradle" target="_blank">https://github.com/openjdk/jextract/blob/master/build.gradle</a><br>
</p>
<blockquote type="cite">
<div><br>
</div>
<div>The good news is that all this would be very cheap to
improve. All that's needed is:</div>
<div>
<ul>
<li>A defined layout for JARs (or JMODs) that
standardizes where to place native libraries given an OS and
CPU architecture. <br>
<br>
</li>
<li>Tooling that extracts native code to a user-specified
directory that's then appended to the java.library.path at
runtime (e.g. a flag to the java launcher?), so that once
build systems learn to pass this flag or do the extraction
themselves library authors can just deprecate and eventually
remove all their custom loader code (which is large,
complex, copy/pasted between projects and inconsistent).<br>
<br>
</li>
<li>Support for that mechanism in jlink.</li>
</ul>
</div>
</blockquote>
</div>
</blockquote></div>