Hermetic Java (static image packaging/formatting) investigation and proposal

[Magnus Ihse Bursie]

Hi Jiangli,

On 2023-02-08 03:08, Jiangli Zhou wrote:
> Hi Brian,
>
> Here are the main buckets of the changes discovered in JDK/VM to
> support the proposed hermetic image:
>
> 1) Resolve symbol conflicts to fully support JDK static builds. Those
> are mainly caused by duplicated symbols defined in different native
> libraries or VM code.
>
> 2) Complete the built-in native library support in JDK. For easier and
> more reliable testing/release/deployment, we wanted to support JDK
> dynamic and static builds with the same set of object files (.o).
> We've changed to use unique names for
> JNI_OnLoad|JNI_OnUnload|Agent_OnLoad|Agent_OnUnload|Agent_OnAttach in
> different JDK JNI libraries by default. For both dynamic linked and
> static linked JDK builds, we use unique symbols for JNI_OnLoad
> function and friends. However, non-builtin application JNI libraries
> can still have the default JNI_OnLoad|... naming. We still properly
> support application JNI libraries using the default JNI_OnLoad (and
> friends) naming.
>
> As we wanted to produce dynamic and static builds from the same set of
> object files, we've moved away from using the STATIC_BUILD macro.
>
> We've also done some makefile work to build both dynamic shared
> libraries (DSOs) and static libraries, within one JDK build.

This sounds like interesting work indeed. However, I am inclined to 
agree with Andrew and wonder how much it relates to Project Leyden. It 
might be that Leyden will need some kind of packaging story, and that 
this can have a role to play in that. But it is not immediately clear 
that it does fit in, and indeed, I think this is not one of Leyden main 
problem areas at the time.

But your code sounds very much interesting from a pure build 
perspective! For at least this part of the code, I think you should 
ignore Leyden for now, and just see if the static build changes you have 
made could be fit for inclusion in OpenJDK.

The static build part of the build system has been sadly neglected due 
to resource limitations, for a long time. :( The rudimentary system 
(actually, more like two separate systems) we have was put in place 
mostly due to external requirements from Project Mobile and the Graal 
integration, and was tacked on mostly as an after-thought. It is not 
regularly tested, and I'd frankly be surprised if it actually works 
right now. So I fully understand if you have been staying away from 
STATIC_BUILD. :)

It sounds like you have created a more dynamic system to be able to 
select per library, if it should be compiled statically or dynamically. 
Do I understand you correctly? If done correctly, it can probably help 
bring a better abstraction to the build process.

If you are willing to contribute your work to OpenJDK, I would 
definitely be interested in studying it in detail. As you might be 
aware, contributions to OpenJDK must be done on the OpenJDK 
infrastructure. One way to do this is to create a branch in the sandbox 
repo[1], and push your changes there.

If it turns out to be of use for Project Leyden, all the better if it is 
already in place. And if it turns out that this is orthogonal to Project 
Leyden, I still think a cleanup in this area might be beneficial for all 
of the JDK.

/Magnus

[1] https://github.com/openjdk/jdk-sandbox