Inconsistency with service loading by layer or by class loader

[David Lloyd]

On Mon, Dec 16, 2024 at 4:21 AM Alan Bateman <alan.bateman at oracle.com>
wrote:

> On 14/12/2024 17:10, David Lloyd wrote:
>
> :
>
> The reason to lazily load and link modules is basically analogous to the
> reason that we lazily load and link classes. In fact there is very little
> difference. While requiring all classes to be preloaded may have some
> benefits in certain circumstances (take GraalVM for example), it also
> significantly restricts flexibility in a few important ways, and of course
> has a heavy performance cost, and so is not generally considered to be a
> good strategy for application runtimes or even the Java launcher itself.
>
> For modules, we face the same issue. I can, for example, create a single
> application layer with a thousand modules in it. Before the application can
> start, every JAR has to be opened and every module has to be loaded, which
> entails parsing or dynamically creating descriptors (generally a
> combination of these things), each with dozens of support objects for
> things like dependencies, exports, and services, and then their internal
> graphs have to be resolved and wired and checked for consistency. So there
> is a significant performance cost there.
>
>
> Startup with a large number of modules on the application module path is
> an interesting topic. Reliable configuration means the modules required by
> the initial module must be recursively enumerated to create the module
> graph, and this does mean the module path must be scanned (once) to find
> the modules.
>
> Efforts to date have been focused on shifting the work to link-time with
> jlink. When the module system was introduced we included a jlink plugin to
> generate the module graph (the Configuration) at link time, then a fast
> reconstitute of the Configutation at run-time. In JDK 12 it went a step
> further with CDS support for object archiving and enabling CDS by default.
> It improved again in JDK 16 with the archiving of the boot layer. The
> overall effect of all these efforts is all code to initialize the module
> system and the boot layer disappears from startup.
>
> I think it would be interesting to explore shifting the generation of the
> Configuration for application / child layers to either build time or link
> time. There is a lots of interesting directions to explore. We had many
> ideas in this area during JDK 9 but had to focus on the core system. I
> suspect this could be a lot more fruitful direction that wouldn't forsake
> reliable configuration.
>

While that is a very interesting topic, it is in a sense the opposite of
what I'm focused on right now, which is the application container case. It
is true that there is a great deal of interest in moving things towards
build time for a fairly broad class of application. However, in this case
I'm starting from the other end of the spectrum, where we have containers
which can dynamically load and unload applications. This use case, while
historical, is still wide

Your point about reliable configuration is an interesting one. We have had
a concept of modules that predates Java 9 by several years. This modular
concept was focused on a more relative integrity. That is to say, for a
given module that is loaded, we ensure that each of its immediate
dependencies is loadable, and provide an isolated visibility between
modules. Within this system, we don't have to verify the integrity of the
entire graph and we therefore don't actually need a concept of "layers" as
defined by the JPMS (rather, layering is a natural consequence of the
design). In fact we've always considered the requirement to validate the
entire graph as an anti-requirement, because it doesn't actually give the
user much practical benefit, while also being very costly, and it might be
open-ended or even downloaded on demand, and also because the applications
within a container may number between one, to dozens, to literally
thousands at once. The libraries which support these applications form a
complex graph of many hundreds to thousands of JARs. Some of these JARs may
represent multiple versions of the same library. This is allowed and
supported as an inherent property of a module system which provides
isolation and only validates integrity on a local, per-module basis.

Many features in OpenJDK historically exist at least in part because of use
cases like ours: for example, better class unloading support, GCs which
support giant heaps, etc. The inflexibility of the JPMS in this regard is
the primary reason why we (and presumably others) have not yet integrated
with it. I am (and have been) trying to change that, but to do so, I
believe we will need some kind of concession around service loading since
that's the missing piece right now. It's clear that static
(build-time-oriented) Java has a bright future, but classic dynamic
(run-time-oriented) Java isn't going anywhere any time soon. Excluding one
for the sake of the other is not a wise course.

-- 
- DML • he/him
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/jigsaw-dev/attachments/20241216/6b43f8f4/attachment-0001.htm>