GraalVM Capabilities Important to Native Java & Leyden

[Jason Greene]

Quarkus has been utilizing GraalVM in real-world native Java deployments for
quite some time. Based on that experience, we recommend carrying over
(adapting as required) specific capabilities essential to this use case. We
feel all of these goals are consistent with the original Leyden announcement
but wanted to affirm them, along with some supporting information we think is
relevant. Thank you for considering our feedback.


Native Java Goals
=================

Minimal Footprint Comparable to Container Friendly Languages
------------------------------------------------------------

In a cloud environment, the base overhead of a JVM runtime is multiplied
across many more process instances, introduced by the combination of the
architectural decoupling required to embrace cloud orchestration (i.e.,
building a system using multiple microservices on Kubernetes vs. a single JVM
app), as well as the need for horizontal elastic scalability, and the multiple
instances per service required to respond to load and provide fault tolerance.

Due to a large number of smaller sized processes and a lack of shareability in
a container-isolated system, the base overhead introduced by executable code
(both application and JVM components), metaspace, and other non-heap
structures have a significant impact on the combined memory usage of the
overall system, and the resulting container density achievable with a given
set of resources.

While other languages, such as Golang, have demonstrated reduced overhead
ideal for this usage pattern, Java has shown mass appeal: a record of
improving developer teams’ productivity, a vibrant ecosystem of frameworks,
and millions of developers with years of experience using the language.
Ideally, those gains are not sacrificed to achieve the desired performance of
a cloud-native runtime. Therefore, a native Java runtime should alter the
chosen trade-offs (including API and potentially JLS changes) to fully realize
an environment that combines the core Java language and a minimalist
footprint.

Instant Boot
------------

Also important in an elastic cloud environment, and even more so in a
serverless functions environment, is the ability to cold start immediately
on-demand, processing and returning data with minimal latency from a
non-existent process to a fully operational process that has produced a reply.
Such processes need to support a variance in life-span, potentially lasting
for only a single request, to persisting for long periods of time processing
many requests.

This need encompasses two aspects. The first is the JVM startup itself is as
minimal as possible, limiting the amount of JDK initialization code required
to be executed. The second is that the application can start from some form of
precomputed state, minimizing the amount of startup activity required to run.


GraalVM Benefits to Preserve (and Enhance)
==========================================

Closed-world Optimization
-------------------------

GraalVM’s architectural decision to fully embrace closed-world optimization is
essential to achieving the footprint goals above. Since microservice-style
applications only use a subset of the JDK and the libraries they depend on,
significant gains can be had by fine-grained elimination of unreachable dead
code and fields. For that to be a reality, changes to the expectations of a
Java runtime are required. Notably, dynamic code/class-loading offers limited
benefits in an immutable container-based application yet prevents the
significant gains that are ideal to the environment. Therefore, dynamic class
definition should be disallowed, as is the case with GraalVM native image.

Secondly, Java reflection targets should be explicitly declared in an opt-in
manner to support minimal and partial inclusions of public and
package-protected members. To do otherwise forces entire APIs to be included,
which contradicts the desired objective to include only what is necessary for
a small service. In a purpose-driven scenario, it’s not uncommon for a small
percentage of an API surface to be used by an application.

Build-time Static Initialization
--------------------------------

Java’s facility of per-class initialization is heavily utilized in virtually
all applications, resulting in the execution of hundreds to thousands of
initializers, even in very simple scenarios. For example, the simple case of a
plain Java “Hello World” involves over 100 initializers. In addition to the
added CPU cost executing a chain of per-class initializers, additional memory
is required to retain one-time code (including dependencies), the associated
metadata, and transient heap structures. GraalVM’s approach eliminates these
costs by generating an optimal initial heap using the output of prerun static
initializers. With build-time static initializers, compiled code output can
exclude large swaths of code, including entire dependency trees. For example,
suppose an application using Jackson pre-parses all JSON configuration into a
standard Java collection style structure. In that case, the entire
muli-megabyte Jackson dependency tree can be completely excluded from the
resulting code output. The resulting initial content contains only the needed
collection data and is stored in the binary image executable. On process
launch, this initial heap content is mapped directly from the image into the
process address space creating an instant working state with minimal
initialization cost.

Service implementations that capitalize on this technique have the ability to
launch and respond to requests immediately with a pre-constructed heap, taking
only a few milliseconds to complete a full cold start. Such an ability becomes
important with elastic scalability since the ability to respond to new load is
limited by the time to spin up functional services. The most extreme case
being serverless scenarios where scaling occurs from zero, and cold start time
is effectively added to the response time of a request.

Of course, there are also scenarios where a static initializer or some portion
of the initializer should be executed at runtime. GraalVM addresses this
through configuration, although with Leyden, there is an opportunity to look
at other approaches such as extending the language via a keyword or annotation
to allow the developer to indicate the appropriate lifecycle phase for the
initializer (build or runtime) or through an alternative mechanism such as
dynamic constants (with appropriate language support) which could be similarly
configured for an appropriate lifecycle phase.

API to Influence Compilation Output
-----------------------------------
Another important ability for native images is a build-time API to influence
compilation output, allowing applications and runtime frameworks to
dynamically assess and register the needs of the application. This is
important since a framework is often in the best position to determine which
application methods, fields, and types should be accessible through
reflection. As an example, data serialization frameworks typically employ
mapping annotations to mark which types are to be serializable and which
methods or fields will be used later on during marshaling. Using this
information, the framework can inform the native image compiler to register
those members, avoiding the need to doubly declare type information, as well
as the consistency problems it would entail. Another example is an agent-style
monitoring framework. Under such a framework, a configured policy separate
from source code might indicate which members of which types will be probed
and should be retained (as opposed to eliminated or inlined)

GraalVM provides such a mechanism through a rich Feature API, which covers
these use-cases and others, and has been successfully used by Quarkus and
other GraalVM supporting projects.


Adaptation/Bridging Facilities and Tools
----------------------------------------

A challenge with using native Java is that developers will want to reuse
existing Java libraries before they are updated to handle the new runtime
target. Many of these libraries incorporate code with assumptions that the JVM
is a fully dynamic environment supporting open-ended discovery and arbitrary
code definition. Additionally, while discouraged, it’s not uncommon to see the
usage of JDK internal APIs, some of which may not be present or may not
support the same interactions in a native setting.

To address this need, GraalVM includes a bytecode substitution feature, which
allows Java developers the ability to adjust certain behavior of a library.
This capability can be used to correct or replace usages of internal API
(e.g., Unsafe), adjust discovery logic to be native friendly, correct
OS-specific patterns, prune code with broken linkage, adjust JNI usage, and
many other usage scenarios.

While the ability to retrofit existing Java libraries effectively enables
usage, ideally, the Leyden contracts would obviate the need for post-Leyden
Java code to require it. This is desirable since built-in language semantics
would offer improved readability, reliability, and robustness. That said,
until such time that the Java community can react and fully update the
ecosystem, there will be a need for some form of code adaptation facility,
similar to GraalVM substitutions. However, such a facility does not have to be
part of the JDK distribution. It could exist as an external community project
and simply integrate with an OpenJDK SPI or act as an external compilation
preprocessing step.