Hermetic Java (static image packaging/formatting) investigation and proposal

[Jiangli Zhou]

On Tue, Feb 14, 2023 at 1:03 AM Florian Weimer <fweimer at redhat.com> wrote:

> * Jiangli Zhou:
>
> > On Mon, Feb 13, 2023 at 1:58 AM Florian Weimer <fweimer at redhat.com>
> wrote:
> >
> >  * Jiangli Zhou:
> >
> >  > 1. The executable image (see slide #10 ofa [1]) consists of three
> >  > sections: the ELF executable section (see slide #14), the JDK runtime
> >  > section (see slide #20, #21) and the JAR section (see slide #22).
> >
> >  These sections are not ELF sections, right?
> >
> > Just to clarify that the ELF sections you referred to in the above are
> > the sections within an ELF file
> > (e.g. https://manpages.debian.org/stretch/manpages/elf.5.en.html), if
> > my understanding is correct.  The sections described for a hermetic
> > Java image are indeed not ELF sections in that sense. An ELF file as a
> > whole is encapsulated in the hermetic image at the beginning. We can
> > use a better name (if ELF section may cause any confusions),
> > e.g. executable section, to describe the section containing the ELF
> > file.
>
> I wouldn't use “section” or “segment” in this context if it's not about
> the ELF constructs.  ELF doesn't use “part” or “appendix” as
> terminology, so that would work.
>

Thanks. Alternatively, 'component' may also be used without potential
confusion?


>
> >  [from further down the thread]
> >
> >  > Yeah, the loadLibrary and friends need to be able look up built-in
> >  > libraries in the executable (within the image ELF section). The
> >  > existing JDK code is already able to handle built-in libraries
> >  > (partially). Please see more details for built-in native support in
> >  > earlier comments.
> >
> >  I believe that will require a custom glibc patch that has not been
> >  upstreamed.  I think Google's approach is here:
> >
> >
> >  <
> https://sourceware.org/git/?p=glibc.git;a=blob;f=dlfcn/dlopen.c;h=faf7a48ea71df186b3581235bffcc7a3692f1325;hb=refs/heads/google/grte/v5-2.27/master#l95
> >
> >
> >
> > The loadLibrary work for built-in native libraries (with JDK static
> > support) does not require the glibc patch for dlopen_with_offset.
>
> Okay, but that means that Hotspot can't be re-linked or unbundled.
>

That's an interesting use case. Any specific requirements/benefits for
re-linking an existing hermetic image with different hotspot code/version?

>From our observations of production usages, change in JDK libraries or the
VM would be deployed via new releases (including both JDK release and the
application release). Those include different testing cycles/stages before
the new binary reaches production. The process includes rebuilding the
hermetic image.


>
> >  There is a different proposed glibc interface for this:
> >
> >    [PATCH v2 2/2] dlfcn,elf: implement dlmem() function [BZ #11767]
> >    <
> https://sourceware.org/pipermail/libc-alpha/2023-February/145476.html>
> >
> >  Maybe you could check if that interface could serve your needs as well?
> >
> > Thanks for pointing to dlmem(). So it looks like we could read the DSO
> > into a buffer then load with dlmem(). Cool! The build ID and symbol
> > file mapping could still be problematic?
>
> I don't think so, it's linked into the glibc dynamic loader data
> structures like any other mapping.  It probably depends on the tooling.
> If it uses DT_DEBUG/_r_debug, it should be able to identify these
> mappings even though they are not file-based.  If the tooling uses
> /proc/PID/maps and not _r_debug, that's of course a different matter.


If I understand correctly from past (internal) discussions with the folks
from the tooling side, the existing tools mostly have the assumption that
the ELF header starts at the beginning of an ELF file (the hermetic Java
image can be treated as an ELF by tools since that's at the beginning of
the image). If we are going with supporting file embedded DSOs, it would
require propagating the support in various tools (perf, gdb, lldb, ...).


>
> Note that dlmem is just a proposed new interface for now.
>

Thanks for that data point.

Best,
Jiangli


>
> Thanks,
> Florian
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/leyden-dev/attachments/20230214/6c103cc6/attachment.htm>