Inconsistency with service loading by layer or by class loader

[David Lloyd]

On Sun, Dec 15, 2024 at 8:55 AM Ron Pressler <ron.pressler at oracle.com>
wrote:

>
>
> > On 14 Dec 2024, at 17:10, David Lloyd <david.lloyd at redhat.com> wrote:
> >
> >
> > For modules, we face the same issue. I can, for example, create a single
> application layer with a thousand modules in it. Before the application can
> start, every JAR has to be opened and every module has to be loaded, which
> entails parsing or dynamically creating descriptors (generally a
> combination of these things), each with dozens of support objects for
> things like dependencies, exports, and services, and then their internal
> graphs have to be resolved and wired and checked for consistency. So there
> is a significant performance cost there. But, that is not the only problem.
> A user application is often a combination of many modules in addition to
> our own basic modules, plus a significant amount of generated code. The
> resultant module graph might even have internal consistency problems (for
> example, having multiple versions of a module, or cyclical dependencies)
> that can't realistically be resolved socially because many of these modules
> are going to be from third parties that we may or may not have any control
> over, and might even be dependencies brought in by the user which we know
> nothing about. These graphs could be the result of very complex resolution
> of Maven artifacts for example.
>
> I’m having a hard time understanding the scenario described here,
> especially when it comes to who loads what and how, and what goes into
> named or unnamed modules (as it’s not an all-or-nothing proposition).
>
> Is it the case that by “application” you mean JBoss, hereinafter called
> “the container”, and by “user application” you mean some web application
> deployed intto the container?


No, the "single application layer" is for illustrative purposes to explain
the cost of having such a large layer.


> Does the container load the user application as a service provider?


No.

Does the user application also use its own ServiceLoader with classloader
> isolation in addition to that employed by the container itself?


It might. But to illustrate the point, it is sufficient to narrow the focus
to just ServiceLoader usage within a third-party library.


> If a user application is a single unit constructed by Maven, how can there
> be version conflicts *within that unit*, and if there are any, how is the
> author of that unit — the user application — expected to deal with them?
>

Because, among other reasons, a user application being built for deployment
will typically include "provided" APIs which in turn have implementations
and dependency graphs which are not expressed within their individual
project build. The application has a local view which ends at the
encapsulation boundary of these libraries.

In the (common) case where the user application isn’t itself modular (and
> possibly contains circular dependencies within its JARs), can’t the
> container load the user application into the unnamed module of some module
> layer that the container creates? Why would the user application itself
> need to be broken up into layers?
>

By our historical (pre-Java 9) definition, every application is modular. We
have never had a problem loading application JARs into isolated class
loaders that are linked together, even if there are circular dependencies.
The problem arises when we want the JDK to recognize these modules as being
JPMS modules.

Historically, I agree that there were not many reasons to do so. Doing so
amounted to two key benefits: encapsulation (essentially, blocking
reflection or hiding packages at module boundaries), and fancy stack
traces. We achieved parity for stack traces using named class loaders.
Blocking reflection has never been a priority for us, and we can adequately
hide packages by preventing them from being accessed at a class loader
boundary.

But things are changing. Native access is now being authorized by Module,
Unsafe is being deprecated, and it's not hard to imagine a world where
modular libraries begin to have other privileges not afforded to the
classical class path world. So it is becoming more important to find ways
to bridge the gap for third-party libraries and for classical containers.

You are right that we don't really have to bother with this. We could in
fact use what influence we have to push users and library authors away from
modularity, citing its limited benefit and these and other practical
concerns (which are far more significant to most users than any subjective
principles), instead of doing the bulk of the heavy lifting in terms of
trying to bring them on board as we are today. The resultant continued
bifurcation of the ecosystem would surely cause pressure that could hinder
progress for things like jlink and other future module-oriented
technologies. Module-only APIs would continue to be unused by popular
libraries and frameworks. But, I'd still get a paycheck, and our users will
be happy enough with the results; we can after all resort to such measures
as `--add-opens` for `java.base` in our launcher and essentially do
whatever we need to in that way.

But instead, we're trying to find ways to modernize the ecosystem and draw
within the lines. Andrew rightly described this as a "rear guard action"
rather than a vote of confidence. The JPMS is not what we ever wanted it to
be. Our concerns with this system and its principles have been largely
dismissed since its inception. But with a couple accommodations, it could
be close enough for our purposes and we can find a way forward.

I hope we can close this meta-debate to get back to the problem at hand.
-- 
- DML • he/him
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/jigsaw-dev/attachments/20241216/ee9bb4a9/attachment-0001.htm>