Hermetic Java (static image packaging/formatting) investigation and proposal

[Jiangli Zhou]

Hi Florian,

Thanks for looking into this!

On Mon, Feb 13, 2023 at 1:58 AM Florian Weimer <fweimer at redhat.com> wrote:

> * Jiangli Zhou:
>
> > 1. The executable image (see slide #10 ofa [1]) consists of three
> > sections: the ELF executable section (see slide #14), the JDK runtime
> > section (see slide #20, #21) and the JAR section (see slide #22).
>
> These sections are not ELF sections, right?
>

Just to clarify that the ELF sections you referred to in the above are the
sections within an ELF file (e.g.
https://manpages.debian.org/stretch/manpages/elf.5.en.html), if my
understanding is correct. The sections described for a hermetic Java image
are indeed not ELF sections in that sense. An ELF file as a whole is
encapsulated in the hermetic image at the beginning. We can use a better
name (if ELF section may cause any confusions), e.g. executable section, to
describe the section containing the ELF file.


>
> > 2. The Java launcher executable is statically linked with Hotspot/JDK
> > natives and application JNI natives (see slide #15 - #18).
>
> Doesn't this make the resulting executable non-distributable in almost
> all cases, for licensing reasons?  (Hotspot is under the GPL, version 2,
> most open-source Java libraries use the Apache license, version 2.0, and
> those two are generally considered incompatible.)  At least for those
> developers who do not have special licensing arrangements with Oracle
> and thus receive OpenJDK under the default terms (including the assembly
> exception, but it doesn't seem to help here because it only extends to
> OpenJDK components).
>

Those are good questions. I have to admit that I'm not the appropriate one
equipped with the right set of knowledge to answer these questions. This
mailing list may not be the proper forum for the topic either. It's
probably a good idea to seek the right set of experts and channels for
discussions and possible solutions.


>
> [from further down the thread]
>
> > Yeah, the loadLibrary and friends need to be able look up built-in
> > libraries in the executable (within the image ELF section). The
> > existing JDK code is already able to handle built-in libraries
> > (partially). Please see more details for built-in native support in
> > earlier comments.
>
> I believe that will require a custom glibc patch that has not been
> upstreamed.  I think Google's approach is here:
>
>   <
> https://sourceware.org/git/?p=glibc.git;a=blob;f=dlfcn/dlopen.c;h=faf7a48ea71df186b3581235bffcc7a3692f1325;hb=refs/heads/google/grte/v5-2.27/master#l95
> >
>

The loadLibrary work for built-in native libraries (with JDK static
support) does not require the glibc patch for dlopen_with_offset.

During the investigation, we evaluated different approaches and I
prototyped the alternative approach (slide #18 in
http://cr.openjdk.java.net/~jiangli/hermetic_java.pdf) using
dlopen_with_offset, before looking into the JDK static support.
dlopen_with_offset would be needed if the DSOs (not statically linked with
the executable) were embedded within an image. We decided not to go with
that due to several considerations. The main ones include:

- dlopen_with_offset has not landed in glibc, and there is no plan to do so
AFAIK
- When a shared library is embedded in an executable image (as illustrated
in slide #18 in http://cr.openjdk.java.net/~jiangli/hermetic_java.pdf),
there are issues for the existing standard tools (e.g. perf) to identify
the build ID for mapping the correct symbol file associated with the DSO.
This's the bigger reason why we did not choose file embedded DSOs. Happy to
provide additional details from the investigation on this, if you are
interested.


> The offset argument is then threaded through the entire code, e.g.:
>
>   <
> https://sourceware.org/git/?p=glibc.git;a=blob;f=elf/dl-open.c;h=178270ca1554c1a81db794fb110a962693839887;hb=refs/heads/google/grte/v5-2.27/master#l539
> >
>
> There is a different proposed glibc interface for this:
>
>   [PATCH v2 2/2] dlfcn,elf: implement dlmem() function [BZ #11767]
>   <https://sourceware.org/pipermail/libc-alpha/2023-February/145476.html>
>
> Maybe you could check if that interface could serve your needs as well?
>

Thanks for pointing to dlmem(). So it looks like we could read the DSO into
a buffer then load with dlmem(). Cool! The build ID and symbol file mapping
could still be problematic?

Best,
Jiangli


>
> Thanks,
> Florian
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/leyden-dev/attachments/20230213/2d282c21/attachment.htm>