Hermetic Java project meeting

Tue May 21 05:17:45 UTC 2024

On Tue, May 7, 2024 at 5:26 AM Magnus Ihse Bursie <
magnus.ihse.bursie at oracle.com> wrote:

> On 2024-05-07 06:04, Jiangli Zhou wrote:
>
> On Tue, Apr 30, 2024 at 5:42 AM Magnus Ihse Bursie<magnus.ihse.bursie at oracle.com> <magnus.ihse.bursie at oracle.com> wrote:
>
> I am not sure why clang insisted on picking up ld and not lld. I remeber
> trying with -fuse-ld=lld, and that it did not work either.
> Unfortunately, I don't remember exactly what the problems were.
>
> I started reinstalling my Linux workstation yesterday, but something
> went wrong, and it failed so hard that it got semi-bricked by the new
> installation, so I need to redo everything from scratch. :-( After that
> is done, I'll re-test. Hopefully this was just my old installation that
> was too broken.
>
> I decided to spend the time to reinstall my machine. Now linking with
> clang works. Kind of. For some reason, it still picks up binutils ld and
> not lld, and then -l:libc++.a does not work, but when I replaced it with
> -l:libstdc++.a it worked just fine. I guess we need to either forcefully
> add -fuse-ld=lld to our clang compilation lines, or figure out if clang is
> going to call the binutils or llvm ld, and select the right option.
>

https://lld.llvm.org/#using-lld has some information on using lld instead
of the default linker.

> I still find the logic for how clang and gcc locates the default linker to
> be mostly magic. I guess I need to make a deep dive in understanding this
> to be able to resolve this properly.
>
> The JDK and VM code has pre-existing assumptions about the JDK
> directories and dynamic linking (e.g. the .so).
> JLI_IsStaticJDK|JLI_SetStaticJDK|JVM_IsStaticJDK|JVM_SetStaticJDK is
> needed for static JDK support to handle those cases correctly.
> CreateExecutionEnvironment that I mentioned earlier is one of the
> examples.
>
> I'm quite certain the issue that you are running into is due to the
> incorrect static check/handling in CreateExecutionEnvironment.
>
> I'll have a look at that, thanks for the pointer.
>
> In my branch, I am only using compile-time #ifdef checks for static vs
> dynamic. In the long run, the runtime checks that you have done are a
> good thing, but at the moment they are just adding intrusive changes
> without providing any benefit -- if we can't reuse .o files between
> dynamic and static compilation, there is no point in introducing a
> runtime check when we already have a working compile-time check.
>
> I haven't seen your branch/code. I'd suggest not going with the #ifdef
> checks as that's the opposite direction of what we want to achieve. It
> doesn't seem to be worth your effort to add more #ifdef checks in
> order to do static linking build work, even those are for temporary
> testing reasons.
>
> Okaaaaay... My understanding was that you wanted to push for the quickest
> possible integration of building a static java launcher into mainline.
>
That's correct. Please see more details below.

> To do that as fast as possible, we need to use the existing framework for
> separating statically and dynamically linked libraries, which means doing
> compile time checks using #ifdefs.
>
Using #ifdefs is not the most efficient path for us to get static Java
launcher support in mainline. That's because most of the runtime changes
for static Java support in hermetic-java-runtime branch are already done
using `JLI_IsStaticJDK|JVM_IsStaticJDK` checks. We should not convert those
to use #ifdefs then later convert the #ifdef back to runtime checks again
during the integration work.

As suggested and discussed earlier we can aim to get the static Java
related changes into mainline incrementally. Following is a path that I
think would work effectively and "fast" by limiting potentially wasted
efforts:

Step 1 - Get the makefile changes for linking `javastatic` without any of
the runtime changes; Don't enable any build and testing for `javastatic` in
this step yet
Step 2 - Incrementally get the runtime changes reviewed and integrated into
mainline;
              Enable building for `javastatic` as a test in github workflow
when we can run HelloWorld using static launcher in mainline;
              Enable testing tier 1 for `javastatic` in workflow when we
can run jtreg tests with the static launcher - could be done in a later
step;
Step 3 - Remove all STATIC_BUILD macros in JDK runtime code; Also cleanup
the macros in tests (can be done later)
              CSR and JNI specification work to support
JNI_OnLoad_<lib_name> and friends for JNI dynamic library and builtin
library
Step 4 - Build (makefile) changes to support linking .a and .so libraries
using the same set of .o objects, to avoid compiling the .c/.c++ source
twice

Those lay the foundation for the hermetic Java work in mainline.

> Are you saying now that the priorities has changed, and that you want to
> start by introducing your framework for the runtime lookup if we are static
> or dynamic?
>
By "runtime lookup", I think you were referring to the JNI native library
lookup. We can handle them as part of the step 2 above.

I think for any of the runtime changes, we need to be able to build in the
mainline (although initially not included in the github workflow).

>
> To be honest, I think your prototype is rather hacky in how you implement
> this, and I reckon that it will require quite a lot of work to be accepted
> into mainline. I also think you need a CSR for changing the Hotspot/JDK
> behavior wrt this, which further adds to the process.
>

For CSR work, we can do that as part of step #3. Actually, for the
builtin/dynamic library lookup support, I think the enhancements in
hermetic-java-runtime are already close to the proper shape (not hacky).

> If you want to go that route instead, then I'll put my work on hold until
> you have gotten a working solution for the runtime lookup in mainline. I
> gather this means that there is no real stress for me anymore.
>
Ron and Alan mentioned Tuesday morning PT may not work the best for you.
Would you be open for a separate time to discuss the details on moving
forward?

Best,
Jiangli

>
> /Magnus
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/leyden-dev/attachments/20240520/4deb22e0/attachment.htm>