<!DOCTYPE html><html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<p>Jiangli,</p>
<p>First of all: I tried building the leyden branch
"hermetic-java-runtime" but failed. :-( I tried some various
attempts to work around the failures, but the javastatic file I
ended up with were unable to start. <br>
</p>
<p>What kind of environment are you using to build this branch? (OS,
compiler, libc version, etc) Do you have any special configure
flags that are required?<br>
</p>
<p>I would very much like to test it out myself to check this static
java launcher that you are creating.<br>
</p>
<p>On 2024-04-16 03:01, Jiangli Zhou wrote:</p>
<blockquote type="cite" cite="mid:CALrW1jyLoDOtXPqfrgeqt17KyQpnSL+Z5ofLvjngKzSOwhDjDw@mail.gmail.com"><span style="white-space: pre-wrap">
</span>
<pre class="moz-quote-pre" wrap="">The main purpose of StaticLink.gmk is to support the static-java-image
make target, which can be used to perform the actual static linking
step using libjvm.a and JDK static libraries. That currently doesn't
exist in the JDK mainline. Creating a "fully" statically linked Java
launcher is the first step (out of many) towards supporting
static/hermetic Java.</pre>
</blockquote>
<p>I am still not sure it is the best design to have this in a
separate file. You are in some way "just" creating a new launcher.</p>
<blockquote type="cite" cite="mid:CALrW1jyLoDOtXPqfrgeqt17KyQpnSL+Z5ofLvjngKzSOwhDjDw@mail.gmail.com">
<pre class="moz-quote-pre" wrap="">As part of cleaning/refactoring/integrating for the static linking
step, we want to agree and decide/accept on the following:
- Support the "fully" statically linked java launcher for testing and
demoing the capability of static JDK support, e.g.
- Support running jtreg testing using the "fully" statically linked
Java launcher</pre>
</blockquote>
<p>So how do you want to test native jtreg tests? By compiling the
jtreg tests to native .a files and statically link with those as
well, or to make this a hybrid mode where a statically linked
launcher loads the jtreg tests dynamically?</p>
<blockquote type="cite" cite="mid:CALrW1jyLoDOtXPqfrgeqt17KyQpnSL+Z5ofLvjngKzSOwhDjDw@mail.gmail.com">
<pre class="moz-quote-pre" wrap=""> - Set up tests in github workflow to help detect any breaking
changes for static support, e.g. new symbol issues introduced by any
changes. There were some earlier discussions on this with Ron and Alan
during the zoom meetings.</pre>
</blockquote>
<p>If we do this correct, we should not have to worry about symbol
conflicts. (Local symbols should all be hidden before creation of
the static libraries.) That said, creating static libraries needs
to be regularly tested or it will break.<br>
</p>
<blockquote type="cite" cite="mid:CALrW1jyLoDOtXPqfrgeqt17KyQpnSL+Z5ofLvjngKzSOwhDjDw@mail.gmail.com">
<pre class="moz-quote-pre" wrap="">- Which JDK native libraries to be statically linked with the new
launcher target? E.g. StaticLink.gmk currently excludes libjsound.a,
libawt_xawt.a, etc from statically linked with the launcher.</pre>
</blockquote>
That sounds rather arbitrarily. I would assume that a statically
linked launcher should include native libraries from all included
modules. If you are not interested in java.desktop native libraries,
then you'd need to exclude java.desktop from the build. But my
assumption here is that we need to build a solution that works with
all modules included in the JDK, and that is what must be tested in
GHA.<br>
<blockquote type="cite" cite="mid:CALrW1jyLoDOtXPqfrgeqt17KyQpnSL+Z5ofLvjngKzSOwhDjDw@mail.gmail.com">
<pre class="moz-quote-pre" wrap="">- Do we want more than one statically linked launcher target, based on
the set of linked native libraries?</pre>
</blockquote>
<p>Do you mean that you want like "static-java-java.base-only",
"static-java-eveything-but-java.desktop" etc? That does not sound
like something that belongs in the makefiles. If you want to build
a JDK that can only support a certain subset of modules, you need
to specify those module in your build scripts as input to
configure.</p>
<blockquote type="cite" cite="mid:CALrW1jyLoDOtXPqfrgeqt17KyQpnSL+Z5ofLvjngKzSOwhDjDw@mail.gmail.com">
<pre class="moz-quote-pre" wrap="">(II) What missing:
- Static linking step as mentioned above
(III) What needs to be improved (require cleanups and refactoring, and
you mentioned some of those in your response as well):
- Support building both the static libraries and dynamic libraries
using the same set of .o files, instead of separately compiled .o
files. That helps improve build speed and reduce memory overhead for
building JDK. Your current refactoring work aims to help that.
- Clean up the usages of STATIC_BUILD macro. Most of the usages are in
test code.
- Other runtime fixes/enhancements in the leyden
<a class="moz-txt-link-freetext" href="https://github.com/openjdk/leyden/tree/hermetic-java-runtime">https://github.com/openjdk/leyden/tree/hermetic-java-runtime</a> branch
I think most work mentioned in III has dependencies on II. We need a
workable base to be able to build the "fully" statically linked
launcher for building and testing the work mentioned in III, when
integrating any of those to the JDK mainline. The makefile refactoring
work can be done in parallel but does not need to be completed before
we add the static linking step in JDK mainline.</pre>
</blockquote>
<p>I'm not sure I understand why you consider (II) to be a
prerequisite for the (III) issues. From my point of view, they are
all mutually independent changes that needs to be done, and if
anything, I think getting in code to create a statically linked
launcher will probably be easier if we get further along on the
other issues.</p>
<p>In particular, I'd like to see changes to the STATIC_BUILD macro
come in first. Ideally, we should not send in such a macro at all.
The JNI_OnLoad stuff is bugging me mostly. <br>
</p>
<blockquote type="cite" cite="mid:CALrW1jyLoDOtXPqfrgeqt17KyQpnSL+Z5ofLvjngKzSOwhDjDw@mail.gmail.com"><span style="white-space: pre-wrap">
</span><span style="white-space: pre-wrap">
</span>
<pre class="moz-quote-pre" wrap="">As of today, the leyden
<a class="moz-txt-link-freetext" href="https://github.com/openjdk/leyden/tree/hermetic-java-runtime">https://github.com/openjdk/leyden/tree/hermetic-java-runtime</a> branch
can build a "fully" statically linked Java launcher. The issue of
compiling the dynamic and static libraries .o files separately is not
a blocker. It's good to have it resolved at some point of time.</pre>
</blockquote>
<p>Weeeell, yes and no... It is not *technically* a blocker, but
unless we can reuse the .o files, including static builds in the
standard testing on GHA will effectively double the build time,
and that is definitely not acceptable. And the alternative is to
bring in static builds without having it regularly tested, which I
don't think is acceptable either. </p>
<p>So the conclusion is that getting static and dynamic libraries
built from the same .o files is in effect a blocker for Hermetic
Java.<span style="white-space: pre-wrap">
</span></p>
<blockquote type="cite" cite="mid:CALrW1jyLoDOtXPqfrgeqt17KyQpnSL+Z5ofLvjngKzSOwhDjDw@mail.gmail.com">
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">This, in turn, require several changes:
1) The linking code needs to be cleaned up, and all technical debt needs to be resolved. This is what I have been doing since I started working on static builds for Hermetic Java. JDK-8329704 (which was integrated yesterday) was the first major milestone of this cleanup. Now, the path were to find a library created by the JDK (static or dynamic) is encapsulated in ResolveLibPath. This is currently a monster, but at least all knowledge is collected in a single location, instead of spread over the code base. Getting this simplified is the next step.
2) We need to stop passing the STATIC_BUILD define when compiling. This is partially addressed in your PR, where you have replaced #ifdef STATIC_BUILD with a dynamic lookup. But there is also the problem of JNI/JVMTI entry points. I have been pondering how we can compile the code in a way so we support both dynamic and static name resolution, and I think I have a solution.
This is unfortunately quite complex, and I have started a discussion with Alan if it is possible to update the JNI spec so that both static and dynamic entry points can have the form "JNI_OnLoad_<library-name>". Ideally, I'd like to see us push for this with as much effort as possible. If we got this in place, static builds would be much easier, and the changes required for Hermetic Java even smaller.
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">
Thumbs up! That seems to be a good direction. Currently in the leyden
branch, it first looks up the unique
JNI_OnLoad<_lib_name>|Agent_OnLoad<_lib_name> etc for built-in
libraries, then search for the dynamic libraries using the
conventional naming when necessary. e.g.:
<a class="moz-txt-link-freetext" href="https://github.com/openjdk/leyden/commit/a5c886d2e85a0ff0c3712a5488ae61d8c9d7ba1a">https://github.com/openjdk/leyden/commit/a5c886d2e85a0ff0c3712a5488ae61d8c9d7ba1a</a>
<a class="moz-txt-link-freetext" href="https://github.com/openjdk/leyden/commit/1da8e3240e0bd27366d19f2e7dde386e46015135">https://github.com/openjdk/leyden/commit/1da8e3240e0bd27366d19f2e7dde386e46015135</a>
When spec supports JNI_OnLoad_<library-name> and etc. for dynamic
libraries, we may still need to support the conventional naming
without the <_lib_name> part for existing libraries out there.</pre>
</blockquote>
<p>That looks interesting. Basically, this is very similar to what I
imagine needs to be done in the mainline. However, I don't think
we can just change this code like that without a CSR? Or are we
allowed to treat JDK-internal libraries different than the
specification states?</p>
<p>I would say getting this part into mainline should be the main
focus right now, not the StaticLink.gmk file. And that includes
getting any CSR approvals etc.<br>
</p>
<blockquote type="cite" cite="mid:CALrW1jyLoDOtXPqfrgeqt17KyQpnSL+Z5ofLvjngKzSOwhDjDw@mail.gmail.com">
<pre class="moz-quote-pre" wrap="">Thank you for taking this on! Potentially we could consider taking the
objcopy to localizing hotspot symbols on unix-like platforms, based on
<a class="moz-txt-link-freetext" href="https://github.com/openjdk/jdk/pull/17456">https://github.com/openjdk/jdk/pull/17456</a> discussions. Additional
testing is still needed to verify the solution.</pre>
</blockquote>
We don't need any additional testing on that; proof of concept shows
it works. It just needs to be implemented. With JDK-8330261 (pushed
today), the ground is now prepared to get it done. It is next on my
todo-list.<br>
<blockquote type="cite" cite="mid:CALrW1jyLoDOtXPqfrgeqt17KyQpnSL+Z5ofLvjngKzSOwhDjDw@mail.gmail.com">
<pre class="moz-quote-pre" wrap="">It might be a good idea to follow up on the static linking discussion
in tomorrow's zoom meeting (hope you'll be able to join tomorrow).</pre>
</blockquote>
<p>I'll join but might be somewhat late due to family commitments. <br>
</p>
<p>/Magnus<br>
</p>
</body>
</html>