<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<p><br>
</p>
<div class="moz-cite-prefix">On 24/02/2023 16:59, Mike Hearn wrote:<br>
</div>
<blockquote type="cite" cite="mid:CAGv+2bPpwEAoeKaDKRmwwxmfZojvHU=Gjy2fPucEbmb23zrzpg@mail.gmail.com">
<div dir="ltr">Thanks! Yes creating JMODs is one way to do it, but
if you were to distribute jextract as a library and not just an
application you'd get user complaints because it'd take more
work to consume:</div>
</blockquote>
Yep - I agree libraries are a different beast (in my email I
specifically mentioned "application").<br>
<blockquote type="cite" cite="mid:CAGv+2bPpwEAoeKaDKRmwwxmfZojvHU=Gjy2fPucEbmb23zrzpg@mail.gmail.com">
<div dir="ltr">
<div>
<ul>
<li>JMODs are platform specific but build tools don't make
it easy to select the right artifacts based on platforms.
You can do it but it often requires custom Maven/Gradle
plugins and such.<br>
<br>
</li>
<li>You can't put JMODs on the module path so users would
now need to run jlink to get a JDK they could use, but
build tools don't invoke jlink during the normal
development cycle and don't easily support switching which
JDK they use half way through based on the output of build
tasks (maybe they should but last time I tried, they
don't). You could write extra plugins to teach them to do
that maybe, but then you'd hit performance problems -
jlinking is a fairly heavy and slow operation, and doesn't
cache anything. All this complexity is why some vendor
JDKs pre-jlink JavaFX.<br>
<br>
</li>
<li>It requires libraries to be modular and jlinkable. Sadly
this is often quite challenging :( e.g. out of the box a
vanilla Spring Boot Web app can't be linked because the
module graph is invalid. The Spring guys know and don't
plan to fix it. See <a href="https://urldefense.com/v3/__https://github.com/spring-projects/spring-boot/issues/33942__;!!ACWV5N9M2RV99hQ!PGVRhNK33FPB-AdBjX-73zOMCjfYscyXY5nxOYkjDfozfiCxl7ro2vkGZ-9Q-OtGSg4utZDaY6MGrIz5rbEh4UyAIKtl$" moz-do-not-send="true">https://github.com/spring-projects/spring-boot/issues/33942</a></li>
</ul>
</div>
</div>
</blockquote>
It's true that jmod are mostly designed to be inputs to jlink. That
said, my general feeling is that tools such as jlink are being
under-used and, when used correctly they can simplify processes
quite a bit. So perhaps investing in that direction might provide
better dividends.<br>
<blockquote type="cite" cite="mid:CAGv+2bPpwEAoeKaDKRmwwxmfZojvHU=Gjy2fPucEbmb23zrzpg@mail.gmail.com">
<div dir="ltr">
<div>
<div>So this runs into a fundamental question that Jigsaw
never really resolved and maybe Leyden needs to: is jlink
meant to be a last step optimization process you run before
distribution (today, yes) or is it meant to be an integrated
part of the compile-and-test cycle? </div>
</div>
</div>
</blockquote>
<p>I personally think that if jlink was used as part of
compile-and-test cycle it would be for the betterment of mankind
:-) I can't count the number of times where I was staring at a
maven/gradle build file in confusion, trying to understand why a
certain dependency wasn't being pulled in, or why the IDE had a
"view" of the world that didn't match the one provided by
Maven/Gradle. While IDEs have become quite good at this, there is
still a lot of friction which, IMHO completely goes away if the
artifact of a build is not just a bunch of classes, but a full JDK
image. But, perhaps, we're getting away from the scope of this
mailing list :-)</p>
<p>Maurizio<br>
</p>
<blockquote type="cite" cite="mid:CAGv+2bPpwEAoeKaDKRmwwxmfZojvHU=Gjy2fPucEbmb23zrzpg@mail.gmail.com">
<div dir="ltr">
<div>
<div>If it's the former you can get bugs that only appear at
distribution time. If it's the latter then it needs a
different performance and compatibility model. GraalVM
Native Image faces the same problem: it's slow to compile
and introduces new bugs, so it's tough to integrate it into
the standard development and testing process. Ordinary
HotSpot with a classpath/module path is so great for
development because changing things about your app is just
so darn fast, users are loath to lose that. Hence the
proliferation of libraries that extract code on the fly.
It's inelegant and creates other problems but it preserves
the ultra-fast build loops that people love so much, and
doesn't require special support from build systems.</div>
</div>
</div>
</blockquote>
<p>I have jextract setup with jlink for my daily work and honestly I
can't complain about it being "slow". I can change code and run
all tests and it feels quite instant and natural (and with very
very little fiddling). The main issue is that all the build tools
we know and love (Maven, Gradle) are _not_ designed around the
idea of linked images - which makes it very difficult to do what
we did (and I understand why somebody might look elsewhere rather
than picking a fight with gradle to try and run jlink, and then
use the generated image to run tests).</p>
<blockquote type="cite" cite="mid:CAGv+2bPpwEAoeKaDKRmwwxmfZojvHU=Gjy2fPucEbmb23zrzpg@mail.gmail.com">
<div dir="ltr">
<div><br>
</div>
<div>Sponsored Link: maybe jextract should be distributed with
Conveyor ;) It'd be convenient to have installs that can keep
themselves up to date, added to the path automatically etc.</div>
</div>
</blockquote>
<p>Hehe</p>
<p>Maurizio<br>
</p>
<blockquote type="cite" cite="mid:CAGv+2bPpwEAoeKaDKRmwwxmfZojvHU=Gjy2fPucEbmb23zrzpg@mail.gmail.com">
<div dir="ltr">
<div><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Fri, 24 Feb 2023 at 16:09,
Maurizio Cimadamore <<a href="mailto:maurizio.cimadamore@oracle.com" moz-do-not-send="true" class="moz-txt-link-freetext">maurizio.cimadamore@oracle.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p><br>
</p>
<div>On 24/02/2023 14:45, Mike Hearn wrote:<br>
</div>
<blockquote type="cite">
<div>Especially once Panama ships the Java ecosystem could
strongly benefit from a standardization of how native
components are bundled into and loaded from libraries,
as current build systems and the JVM tooling don't have
much to say on the topic. The result is a lot of wheel
reinvention across the ecosystem in the form of
NativeLibraryLoader.java classes, always unique per
project, and a bunch of bugs / developer friction that
doesn't need to happen.</div>
</blockquote>
<p>I tend to agree with the overall assessment. Shipping
native libraries with Java projects is a known pain point,
and it would be nice to have some solution for that. That
being said, while I'm aware that the best way to make
things work in today's world is by shipping native
libraries in a jar, and then extract them _somewhere_, so
that they can be loaded with `System::loadLibrary`, I'm
not sure how much that can be viewed as a full solution,
rather than a workaround. I can imagine cases where
extracting libraries into a custom folder is not feasible
(e.g. because of missing permissions). My general feeling
is that, with jars and native libraries it's like trying
to fit a round peg in a square hole: surely you can devise
some pragmatic solution which makes things sort of work,
but what you get is always a little brittle.</p>
<p>If you look at what we did for jextract [1], the approach
we used was different: jextract is written entirely in
Java, but has a dependency (via Foreign Function &
Memory API) on libclang. When we build jextract, we create
a jmod [2] for jextract, with the native library for
libclang in the right place. We then create a JDK image
which contains jdk.compiler, java.base and the newly
created jextract module. The resulting JDK will have the
libraries in the right place. This means that we can
provide a launcher simply by calling jextract's entry
point using the custom JDK image. You can run jextract and
run all jextract tests against this custom image, which
then requires zero extra custom arguments passed on the
command line (because the native libraries are added in
the right place already).</p>
<p>An approach such as this seems more promising than doing
heroics with jarfiles, at least for applications, and one
that could be more amenable to things like code signing
(in jextract we don't do this, but I don't see why that
could not be added).<br>
</p>
<p>Maurizio</p>
<p>[1] - <a href="https://urldefense.com/v3/__https://jdk.java.net/jextract/__;!!ACWV5N9M2RV99hQ!PGVRhNK33FPB-AdBjX-73zOMCjfYscyXY5nxOYkjDfozfiCxl7ro2vkGZ-9Q-OtGSg4utZDaY6MGrIz5rbEh4WVwpbl5$" target="_blank" moz-do-not-send="true">https://jdk.java.net/jextract/</a><br>
[2] - <a href="https://urldefense.com/v3/__https://github.com/openjdk/jextract/blob/master/build.gradle__;!!ACWV5N9M2RV99hQ!PGVRhNK33FPB-AdBjX-73zOMCjfYscyXY5nxOYkjDfozfiCxl7ro2vkGZ-9Q-OtGSg4utZDaY6MGrIz5rbEh4cHXAD9_$" target="_blank" moz-do-not-send="true">https://github.com/openjdk/jextract/blob/master/build.gradle</a><br>
</p>
<blockquote type="cite">
<div><br>
</div>
<div>The good news is that all this would be very cheap to
improve. All that's needed is:</div>
<div>
<ul>
<li>A defined layout for JARs (or JMODs) that
standardizes where to place native libraries given
an OS and CPU architecture. <br>
<br>
</li>
<li>Tooling that extracts native code to a
user-specified directory that's then appended to the
java.library.path at runtime (e.g. a flag to the
java launcher?), so that once build systems learn to
pass this flag or do the extraction themselves
library authors can just deprecate and eventually
remove all their custom loader code (which is large,
complex, copy/pasted between projects and
inconsistent).<br>
<br>
</li>
<li>Support for that mechanism in jlink.</li>
</ul>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</body>
</html>