Hermetic Java (static image packaging/formatting) investigation and proposal
Brian Goetz
brian.goetz at oracle.com
Tue Feb 7 21:36:36 UTC 2023
I have a few more questions about the scope of the changes here. As far
as I can tell, there are three main areas where you've modified the JDK:
- the boot classloader, which will look for classes in the ELF before
looking on the regular class path;
- System::loadLibrary and friends, which will similarly look for
shared libraries in the ELF before looking on the regular library path;
- various code paths that look for things in the home directory.
Is that a roughly complete list of the modification areas? If not, what
did I miss? If so, what is the scale and intrusiveness of these
modifications? (For the third item above, for example, I could imagine
these changes happening in a lot of places; I'd guess less so for the
other two.)
Thanks,
-Brian
On 2/2/2023 8:08 PM, Jiangli Zhou wrote:
> Hi Brian,
>
> Thanks for the response!
>
> On Thu, Feb 2, 2023 at 9:16 AM Brian Goetz <brian.goetz at oracle.com> wrote:
>
> Thanks for sharing this information!
>
> As far as I understand it, this is a _packaging mechanism_ to take
> a combination of JDK, application classfiles and resources, and
> native libraries, and combine them into a single executable, and
> the benefits of this include "one file to distribute" and "can't
> run the app on the wrong JDK". Further, the system classloader
> has been modified to load classes out of the image rather than
> from the file system.
>
> My question is about your relationship to the "closed world"
> assumption (and the optimizations that can derive from it); I am
> unsure of whether you make any closed-world assumptions in your
> current approach? Is there any reason why classes cannot be
> loaded dynamically, `invokedynamic` sites can't be linked
> dynamically, etc? Secondarily, does your current implementation
> perform any optimizations related to faster startup and warmup?
>
> My assumption is that the answer is "no restrictions on dynamism,
> no specific optimizations for startup/warmup, it's purely a
> packaging mechanism", combined with a reminder that the sorts of
> optimizations that have been explored elsewhere (Native Image,
> CraC, SnapStart) could equally well be combined with your
> packaging approach.
>
> Do I have it right?
>
>
> You are right in the above. :-) In our current investigation/work, we
> don't apply any restrictions on loading classes and JNI native
> libraries. When desired, it can still dynamically load classes and
> native libraries from outside the image. With a hermetic Java image,
> our usages (at least all the cases that we have experimented with so
> far) do not require such flexibility. In those cases, closed world
> assumptions could be applied. We haven't done any optimization related
> work yet, beyond the packaging/formatting itself. We hope the hermetic
> Java image solution can be a vessel to faciality those sorts of
> optimizations.
>
> Best regards,
> Jiangli
>
>
> Cheers,
> -Brian
>
>
>
> On 2/2/2023 11:13 AM, Jiangli Zhou wrote:
>> (Resending in plain text formatting)
>>
>> Hi,
>>
>> During the last one and a half years, Google has done some
>> extensive research on linux-x64 with Java static image, as
>> project Hermetic Java [1]. We would like to share our
>> experiences/results with the community and present our approach
>> for discussion under the Leyden project. We hope to contribute
>> the work to OpenJDK through project Leyden, via the JEP [2]
>> process as needed.
>>
>> With Hermetic Java, our main goal is to create a single
>> executable image including the Java runtime environment, Java
>> application and the dependencies. This addresses some real-world
>> Java deployment issues and challenges that we have encountered
>> over the years. We believe it fits very well with the overall
>> goal of project Leyden in the following aspects:
>>
>> - Provide a build-time created static image derived from an
>> application and JDK; Image executes as a standalone program.
>> - Satisfy closed-world constraints.
>> - Is built on top of OpenJDK and can utilize existing OpenJDK
>> components including the Hotspot VM, runtime JIT compiler (C1,
>> C2), CDS, etc.
>>
>> Our focus has been on the image packaging and formatting part.
>> This works roughly as follows:
>>
>> 1. The executable image (see slide #10 of [1]) consists of three
>> sections: the ELF executable section (see slide #14), the JDK
>> runtime section (see slide #20, #21) and the JAR section (see
>> slide #22).
>>
>> The ELF section is at the beginning of the image and contains the
>> Java launcher executable, which allows the image to work as a
>> native executable. The JDK runtime section contains the JDK
>> lib/modules image starting at a page-aligned file offset. This
>> section can include other data that requires special alignment,
>> such as the CDS archive. The JAR section holds the Java
>> application classes, dependent library classes, and resources.
>> JDK runtime resource files, such as java.security and java.policy
>> are also packaged within the JAR section.
>>
>> 2. The Java launcher executable is statically linked with
>> Hotspot/JDK natives and application JNI natives (see slide #15 -
>> #18).
>>
>> For static native library support, we enhance and complete
>> existing OpenJDK work [3, 4, 5]. It provides a flexible solution
>> for loading built-in (static) native libraries while still
>> allowing dynamically loading shared JNI libraries (if desired).
>>
>> 3. With a single executable image, we define the image file path
>> as the java.home (see slide #23). A JavaHome class is used to
>> provide uniform APIs for accessing JDK resources in traditional
>> and Hermetic Java (single image) execution modes.
>>
>> Hermetic Java is an accumulation of wisdom that Google obtained
>> from real-world production deployments over many years (years
>> before the current project research/experiments). We would love
>> to gather feedback from community members. Any input and feedback
>> are welcome and appreciated!
>>
>> We are happy to provide additional information and answer
>> questions (open to discussions in any form).
>>
>> [1] http://cr.openjdk.java.net/~jiangli/hermetic_java.pdf
>>
>> [2] http://cr.openjdk.java.net/~mr/jep/jep-2.0-02.html
>>
>> [3] https://bugs.openjdk.org/browse/JDK-8005716
>>
>> [4] https://bugs.openjdk.org/browse/JDK-8136556
>>
>> [5] https://bugs.openjdk.org/browse/JDK-8232748
>>
>> Best regards,
>>
>> Jiangli
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/leyden-dev/attachments/20230207/e9ba2d9d/attachment.htm>
More information about the leyden-dev
mailing list