May 2022 - leyden-dev

Re: Experimentation with build time and runtime class initialization in qbicc
by David P Grove 06 Jun '22

06 Jun '22

Hi, I’ve appended the contents of the referenced wiki page in this email. Apologies in advance if the formatting doesn’t come through as intended. There is a full implementation of this (GPLv2 + Classpath exception) as part of the qbicc project on GitHub. There is also a GitHub discussion in the qbicc project that links to various GitHub issues that capture the history that led to the current design. I will not hyperlink to those here so that if people have any IP concerns, they can avoid seeing them. They are easily findable. Regards, --dave ## Overview One of the goals of the qbicc project is to explore technical approaches for adapting Java's specification of class initialization to fully support native image compilation. Enabling build-time evaluation of complex class initialization logic is essential for obtaining much of the benefits of native image compilation: reduced memory footprint and fast startup. However, both the core JDK and many frameworks will not be primarily be used in native image scenarios. Therefore, it is essential that the approach taken for build-time initialization enables both the existing runtime class initialization and the new build-time class initialization logic to co-exist. Furthermore, for as many cases as possible, the class initialization code should be shared between the two usage scenarios and have non-surprising semantics in both. ## Build-time Initialization In qbicc, all classes are initialized at build-time. Class initialization at build time is performed according to the existing semantics of Java class initialization driven by build-time execution of the `<clinit>` methods of reachable classes. The set of reachable classes is determined iteratively, starting with the program entrypoints and adding the methods and classes they utilize until no further reachable classes are discovered (a fixed point is reached). After build-time initialization has completed, a build-time heap has been constructed that contains the objects that were created during the build-time execution of the `<clinit>` methods. Using the reachable static fields of the reachable program as roots, this build-time heap is serialized into the native image. This set of objects will form the initial runtime heap of the program when it is executed. ## Runtime Initializers There are cases where one or more initialization actions of a class **must** be executed at program runtime. Most typically these involve the creation of native resources (open files, threads, etc) that cannot be successfully serialized into the build time heap. Qbicc supports runtime initialization by allowing static fields of a classes to be declared as runtime initialized. These fields will be initialized lazily, at first access, by executing a runtime initializer (`<rtinit>`) associated with the accessed field. Runtime initialization is localized: accessing a particular static field will cause its runtime initializer to be executed but has no implications for other runtime initializers defined either in the field's defining class or any superclass or implemented interface of the field's defining class. When serialized from the build-time heap to the runtime heap, all runtime-initialized fields will be serialized with the zero (uninitialized) value appropriate for their type. Qbicc allows related static fields in the same class to share a common `<rtinit>` method. The first access to any of the fields will cause the execution of the associated `<rtinit>` method and the initialization of all the fields. ## Adjusting Heap Serialization For some objects it is necessary to initialize them during build-time initialization, but "reset" them before they are used at runtime. Qbicc supports this by allowing fields to be annotated to be serialized as the type-appropriate zero value or as a primitive constant value. This value replacement happens as the build time heap is serialized. One common scenario is to invalidate objects that are wrapping native resources. For example, when a `FileDescriptor` is serialized its `fd` and `handle` instance fields are serialized as `-1` and its `closed` field is serialize as `true`. Thus, any attempt to use the build-time FileDescriptor at runtime will raise the appropriate exception. ## Patching: Migration for Existing Classes The runtime initialization mechanisms described above are currently enabled via a set of annotations. This allows qbicc to implement the desired semantics without requiring any changes to the Java compiler, class file format, or language specification. In the long term, we believe small modifications to the Java specification, for example defining a `rtinit { ... }` similar to the existing `static { ... }` construct could enable a simpler specification. The primary annotation for runtime initialization is `RuntimeAspect`. This annotation is defined on a class and is interpreted as meaning that the `<clinit>` method of the class should be interpreted as an `<rtinit>` method. This method will not be executed during build-time initialization and instead will be deferred until the first access of one of the static fields defined in the class. To allow us to "externally" modify JDK core classes for qbicc, we have developed an annotation-driven patcher infrastructure. The patcher allows the declaration of patch classes that add, remove, and modify the methods and fields of an existing class. This modification includes the replacement of the `<clinit>` method and the declaration of multiple `RuntimeAspect` patch classes. The best way to explore what is possible with the patcher is to examine the java.base/src directory in the qbicc-class-library project. It makes extensive use of the patcher annotations to adapt the core JDK classes to qbicc while still allowing us to consume the upstream OpenJDK code base via an unmodified git submodule. ## Design Alternatives A number of alternatives were considered before arriving at the final design documented here. The technical discussions and options considered can be explored starting in qbicc discussion #764 on GitHub. From: Brian Goetz <brian.goetz(a)oracle.com> Date: Thursday, May 26, 2022 at 2:21 PM To: David P Grove <groved(a)us.ibm.com>, "leyden-dev(a)openjdk.java.net" <leyden-dev(a)openjdk.java.net> Subject: [EXTERNAL] Re: Experimentation with build time and runtime class initialization in qbicc Hi David; Would like to understand more about this, but first, from an IP-hygiene perspective, documents linked from this list should be under the OpenJDK terms and conditions. Can you post the contents of that document here, so there are no ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. ZjQcmQRYFpfptBannerEnd Hi David; Would like to understand more about this, but first, from an IP-hygiene perspective, documents linked from this list should be under the OpenJDK terms and conditions. Can you post the contents of that document here, so there are no issues there? Thanks, -Brian On 5/26/2022 12:35 PM, David P Grove wrote: Hi, In the qbicc project, we’ve been exploring options for adapting Java’s class initialization semantics for native images. In particular, we are trying to arrive at a non-surprising semantics that in a native-image scenarios allows most initialization to happen at build-time while still enabling runtime initialization of selected static fields. Our current design and experience is captured here: https://github.com/qbicc/qbicc/wiki/Class-Initialization-in-qbicc<https://github.com/qbicc/qbicc/wiki/Class-Initialization-in-qbicc>. In a nutshell, the idea is to initialize classes via build-time execution of existing <clinit> methods as per normal Java semantics while adding per-static-field <rtinit> methods to provide a capability for runtime-reinitialization of a field before its first access. --dave

6 13

Re: Project Leyden: Beginnings
by Anton Kozlov 01 Jun '22

01 Jun '22

Great to see Leyden started! I also think that Leyden and CRaC are working toward the same goal from different sides. Assuming a program to be a series of actions, A B C D E F In CRaC, we don't need to specify dependencies and correctness of all permutations, the only difference from the usual Java programming model is that between each subsequent pair of actions checkpoint and restore may happen. Those are rather transparent, except they may invalidate some assumptions about the program environment. So a non-trivial task for the CRaC Project is how to find these assumptions implemented in the program state. If an explicit language-level mechanism will be developed in Leyden to mark executions for build time, the same mechanism will probably be useful to mark now-wrong assumptions after restore for CRaC. So I'm excited and curious about what's going to be developed in Leyden. I also hope CRaC Project to be useful for Leyden. Not sure about pieces of the program or VM state, like a standalone JIT compilations, that would be orthogonal tasks for CRaC. But why not a closed world start image that is restored into an open world Java instance, to achieve a great start-up, warm-up, and footprint of Leyden, without sacrificing the dynamic Java features in the runtime. Thanks, Anton

2 2

Can Ahead of Time code benefit regular Java applications too?
by Julian Waters 01 Jun '22

01 Jun '22

Hi all, Since Leyden's goal has shifted from originally exploring only binaries compiled directly to native code, to "address the long-term pain points of Java’s slow startup time, slow time to peak performance, and large footprint", would there be any merit in looking at allowing native code to be embedded within jars to bypass the Interpreter at runtime? (Maybe have Ahead of Time code that replaces the Interpreter be compiled by C1, and treat it as part of the C1 pipeline so it can be profiled while being run) Ideally it'd be similar to the now defunct jaotc, but more compact (within the jar itself or perhaps the classfiles somehow) instead of compiling the Ahead of Time code into an entirely separate file which then needs to be explicitly passed to the JVM at runtime. This may or may not be a good starting point before advancing to entirely standalone Java binaries, but I digress. Perhaps the experience of the CraC team would be of some help in this area? best regards, Julian

3 4

Improve determinism in the Java language
by Ioi Lam 01 Jun '22

01 Jun '22

A lot of the recent Leyden discussion has been around "what optimizations can be done ahead of time" (e.g., static field initialization). However, I think we also need to look at a lower level. One reason that Java has been difficult to optimize ahead-of-time is the tremendous dynamism in the language. Here are a few things that I think we can do to make Java programs more deterministic so that ahead of time optimizations can be applied: 1 Deterministic Program Code A Java program can essentially rewrite itself and even the libraries it uses. Here's an example: class App { static { if (...) { MethodHandles.lookup() .defineClass(.. hacked App$Bar ...); } } static final Bar bar = new Bar(); static class Bar { .... } } - We can't effectively AOT-compile the program code because the native code may not match the runtime generated bytecodes. - We can't pre-initialize the App.bar field because its shape may be different. One idea is to disallow such code patching when Leyden is enabled. For example, we can require that to use Leyden, an application must be "prelinked", which means that as soon as the application is loaded, the classes App and App$Bar are already loaded. The defineClass() call will fail with a LinkageError (duplicated class definition). 2 Decouple class namespaces from dynamic bytecode generation This is a corollary of the above item. Java uses ClassLoader.defineClass() for BOTH namespace and dynamic bytecode generation. I would stipulate that most users of Leyden want to do the former and not the latter. We should have a new API to load a fixed set of classes into a namespace. 3 <clinit> order Java allows <clinit>s that recursively depend on each other. The result depends on the reference order of these classes. class A { static final int a = B.b++; } class B { static final int b = A.a++; } We could have a problem if the application assumes that A is always initialized before B, but the Leyden optimizer initializes them in the opposite order. We could: - Refuse to optimize classes that have mutually recursive <clinit>, or - Change the language spec to give the JVM more freedom to decide the initialization order.

2 1

Project Leyden: Beginnings
by mark.reinhold＠oracle.com 31 May '22

31 May '22

The ultimate goal of this Project, as stated in the Call for Discussion [1], is to address the long-term pain points of Java’s slow startup time, slow time to peak performance, and large footprint. In the Call for Discussion I proposed that we address these pain points by introducing a concept of _static run-time images_ to the Java Platform, and to the JDK. - A static image is a standalone program, derived from an application and a JDK, which runs that application -- and no other. - A static image is a _closed world_ with respect to the classes that it can load: At run time it cannot load classes from outside the image, nor can it create classes dynamically. The closed-world constraint imposes strict limits on Java’s natural dynamism, particularly on the run-time reflection and class-loading features upon which so many existing Java libraries and frameworks depend. Not all applications are well suited to this constraint, and not all developers are willing to live with it. So rather than adopt the closed-world constraint at the start, I propose that we instead pursue a gradual, incremental approach. We will explore a spectrum of constraints, weaker than the closed-world constraint, and discover what optimizations they enable. The resulting optimizations will almost certainly be weaker than those enabled by the closed-world constraint. Because the constraints are weaker, however, the optimizations will likely be applicable to a broader range of existing code -- thus they will be more useful to more developers. We will work incrementally along this spectrum of constraints, starting small and simple so that we can develop a firm understanding of the changes required to the Java Platform Specification. Along the way we will strive, of course, to preserve Java’s core values of readability, compatibility, and generality. We will lean heavily on existing components of the JDK including the HotSpot JVM, the C2 compiler, application class-data sharing (CDS), and the `jlink` linking tool. In the long run we will likely embrace the full closed-world constraint in order to produce fully-static images. Between now and then, however, we will develop and deliver incremental improvements which developers can use sooner rather than later. Let us begin! - Mark [1] https://mail.openjdk.java.net/pipermail/discuss/2020-April/005429.html // https://openjdk.java.net/projects/leyden/notes/01-beginnings

6 6

Re: Experimentation with build time and runtime class initialization in qbicc
by David P Grove 27 May '22

27 May '22

From: Brian Goetz <brian.goetz(a)oracle.com> > From reading your notes, it seems that at build time, you start with the root class(es), execute their <clinit>, > which will cause loading of more classes, more <clinits>, and you iterate until there are no new classes to initialize. > You then treat the statics as roots, and serialize those objects to the initial heap image. But before doing that, > you exclude (zero out) any which are marked as "reinitialize at runtime." This is correct. In addition, as qbicc serializes each object, it also looks for annotations on instance fields that indicate that instead of serializing the build-time value of the instance field it should substitute a different value (FileDescriptor is a motivating example...we want to serialize a closed FileDescriptor to ensure any runtime reads/writes through it will result in the proper exception being raised). > ... what happens in cases like this: > > class Aliased { > @RuntimeInitialized private static final Socket s = ...; > private static final Socket copy = s; >} First, I'll say what this code snippet would do with qbicc, then I'll say what the program should be to get the semantics the programmer probably intended. At build time, qbicc will execute the <clinit> of Aliased, presumably a Socket object will be allocated by ... and references to that Socket object will be stored in s and copy. Any build-time usage of either s or copy via a build-time executed getfield will get a reference to that Socket object in the build-time heap. The @RuntimeInitialized has no impact on the build-time execution of code. At the end of compilation, when we serialize the static fields for Aliased, we will write null for s and a reference to the serialized Socket object for copy. In the generated code, all getfields to s will be preceded by checks to ensure that the <rtinit> method for s has been executed (similar to how a clinit check would be generated in a JVM). Since copy does not have a <rtinit>, getfields to copy in the generated code will not be preceded by any checks. The first time s is accessed at runtime, the ... code will be executed by the <rtinit> method and a new Socket object will be created and stored in s. The fields s and copy will now point to distinct Socket objects. Usages of the Socket object reachable from copy would likely result in an exception because the backing FileDescriptor for the Socket object referenced from copy would have been modified during the serialization process so that its instance fields have values as if the FileDescriptor had been closed. Using the syntax above, one would need to write this code to get the intended aliasing at both build-time and runtime. class Aliased { @RuntimeInitilalized private static final Socket s = ...; @RuntimeInitilalized private static final Socket copy = s; } The way we would actually write this pattern in qbicc today is a little more indirect because we (1) we don't want to change javac and (2) we don’t want to directly edit OpenJDK source code (to make it easier to consume updates). Therefore, we define a "patch class" with a @RuntimeAspect annotation that qbicc combines with the unmodified bytecodes of the Aliased class to get what we need. I've added a third field just to emphasize that we need to allow the <rtinit> of a class to be a subset of its <clinit>. class Aliased { private static final Socket s = ...; private static final Socket copy = s; private static final Object anotherField = ... } @RuntimeAspect(Aliased.class) class Aliased_RT { private static final Socket s = ...; private static final Socket copy = s; } The only part of the Aliased_RT class we are interested in is the <clinit> method that javac generated for it. The qbicc compiler takes Aliased_RT's <clinit> and uses it as the <rtinit> method for the fields s and copy of the Aliased class. The rest of the Aliased_RT class is ignored. If one was able to change javac, then the simpler @RuntimeInitialized syntax you had used would be better. From a single class definition, javac could generate both a <clinit> method that initialized s, copy, and anotherField and an <rtinit> method that initialized s and copy. Finally, qbicc does not attempt to recognize when an object that is directly referred to by a @RuntimeInitialized static field is also reachable in some other (perhaps deeply nested) way. As a result, it is certainly possible to write programs where build-time and runtime-time identity (==) of two access paths is different. So far, this hasn't been an issue for us, but it is one of the ways in which one could detect at runtime that something non-standard has happened. Hope this explains more clearly without being tediously long, --dave

2 1

Experimentation with build time and runtime class initialization in qbicc
by David P Grove 26 May '22

26 May '22

Hi, In the qbicc project, we’ve been exploring options for adapting Java’s class initialization semantics for native images. In particular, we are trying to arrive at a non-surprising semantics that in a native-image scenarios allows most initialization to happen at build-time while still enabling runtime initialization of selected static fields. Our current design and experience is captured here: https://github.com/qbicc/qbicc/wiki/Class-Initialization-in-qbicc. In a nutshell, the idea is to initialize classes via build-time execution of existing <clinit> methods as per normal Java semantics while adding per-static-field <rtinit> methods to provide a capability for runtime-reinitialization of a field before its first access. --dave

2 1

Re: Project Leyden: Beginnings
by Volker Simonis 25 May '22

25 May '22

Hi Mark, Thanks for your all your efforts to finally get Leyden started! > The ultimate goal of this Project, as stated in the Call for Discussion > [1], is to address the long-term pain points of Java’s slow startup > time, slow time to peak performance, and large footprint. > You're probably aware of the CRaC project [1] which as well addresses the first two of these pain points (slow startup & slow time to peak performance) by leveraging checkpointing (aka. snapshotting) and restoring (aka. resuming) of fully warmed up JVM instances. [1] https://openjdk.java.net/projects/crac/ > In the Call for Discussion I proposed that we address these pain points > by introducing a concept of _static run-time images_ to the Java > Platform, and to the JDK. > > - A static image is a standalone program, derived from an application > and a JDK, which runs that application -- and no other. > > - A static image is a _closed world_ with respect to the classes that > it can load: At run time it cannot load classes from outside the > image, nor can it create classes dynamically. > > The closed-world constraint imposes strict limits on Java’s natural > dynamism, particularly on the run-time reflection and class-loading > features upon which so many existing Java libraries and frameworks > depend. Not all applications are well suited to this constraint, and > not all developers are willing to live with it. > > So rather than adopt the closed-world constraint at the start, I propose > that we instead pursue a gradual, incremental approach. > Now that the goal of exploring "static images" is not the main goal of Leyden anymore (you should probably update the Leyden project page [2] to reflect this), the goals of CRaC and Leyden seem to match even more. CRaC's new execution model doesn't impose any constraints on "Java’s natural dynamism" so it should naturally support most server-side applications out of the box. Instead, CRaC imposes a new constraint for Java applications which we call "snapsafety". A snapsafe application can operate correctly and securely after it has been restored from a previously checkpointed (and possibly cloned) state. The main challenge for CRaC is to first make the JVM and the core libraries snapsafe before it exposes hooks to libraries and application to give them a chance to become snapsafe as well. [2] https://openjdk.java.net/projects/leyden/ > We will explore a spectrum of constraints, weaker than the closed-world > constraint, and discover what optimizations they enable. The resulting > optimizations will almost certainly be weaker than those enabled by the > closed-world constraint. Because the constraints are weaker, however, > the optimizations will likely be applicable to a broader range of > existing code -- thus they will be more useful to more developers. > > We will work incrementally along this spectrum of constraints, starting > small and simple so that we can develop a firm understanding of the > changes required to the Java Platform Specification. Along the way we > will strive, of course, to preserve Java’s core values of readability, > compatibility, and generality. > It seems to me that "snapsafety" could be such a constraint and I hope for a fruitful and successful cooperation between the two projects. Thank you and best regards, Volker > We will lean heavily on existing components of the JDK including the > HotSpot JVM, the C2 compiler, application class-data sharing (CDS), and > the `jlink` linking tool. > > In the long run we will likely embrace the full closed-world constraint > in order to produce fully-static images. Between now and then, however, > we will develop and deliver incremental improvements which developers > can use sooner rather than later. > > Let us begin! > > - Mark > > > [1] https://mail.openjdk.java.net/pipermail/discuss/2020-April/005429.html > > // https://openjdk.java.net/projects/leyden/notes/01-beginnings

4 5

Please add leyden-dev to the OpenJDK mailinglist info page
by Volker Simonis 24 May '22

24 May '22

Can you please add leyden-dev to the list of OpenJDK mailing lists at https://mail.openjdk.java.net/mailman/listinfo? Thanks, Volker

1 0

Useful information learnt from Graal
by Julian Waters 24 May '22

24 May '22

Hi all, I'd like to link a document written by Andrew Dinn on information learnt from Graal here, since it may prove to be a useful starting point: https://github.com/adinn/leydendoc/blob/main/LeydenLessonsfromGraalNative.p… I'm excited that Leyden is finally starting! best regards, Julian

1 0