[External] : Re: Inconsistency with service loading by layer or by class loader

[Ron Pressler]

> On 16 Dec 2024, at 19:38, David Lloyd <david.lloyd at redhat.com> wrote:
> 
> 
> If you look back at the old jigsaw-dev archives, you'll see my name, and those of my colleagues, quite frequently. I am well aware of the intent of this system and how it was implemented. We do not (and you should not) consider these specifications and their implementations to be holy writ. They are in fact no different from those of any other project, specifically that they can and should be changed as needed. I am expressing such a need.

Yes, and now I think I understand what it is :) Now that we understand the problem, we could contemplate solutions.


> 
> I'm asking for the OpenJDK team to instead consider a different design for service loading in the Java platform itself, in the absence of an alternative solution which can use what exists today. Unfortunately, the constraints around the JPMS are such that I do not believe there is another design that will generally work for us.

I understand what solution you’re proposing, but as with any feature design, the severity of the problem and its root causes need to be understood before settling on a specific solution.

The issue you’re facing arises when making a particular design choice: a container that loads every module in a separate layer and with multiple parents. We need to ask ourselves, how common is this design, why do people pick it (yes, you explained why you picked it, but we’ll need to study alternatives), and are we expecting the number of such use cases to grow or shrink over time?


> I do believe that there are possible workarounds which belie the conceit that the encapsulation of the module system is as strong as you present it; the existence of `ModuleLayer.Controller` itself shows that it is nowhere near absolute. This level of encapsulation is much more superficial than, say, member access control, which is to say that it can be worked around in various ways without "breaking the rules" (for example, by having a class loader define generated helper classes in every package of a module, I can gain access to their original Lookup objects, breaking *any* form modular of encapsulation without breaking any of the rules of the platform specification). If you would recoil at such a thing, consider again the difference between "intent" and "specification".

I would be interested to hear more about this, because our assumption is that the encapsulation — once integrity by default is done, i.e. there is no hidden use of Unsafe or JNI etc. — is, in fact, absolute (modulo bugs), to the point that finally the runtime will be able to actually trust Java’s invariants. Both the security of the platform and the correctness of the compilers depend on it, so if you think we’re wrong — this is important. You mention ModuleLayer.Controller, but I don’t see how it undermines anything (in particular notice how the method that enables a capability with potentially global impact — native access — is caller-sensitive). As an example, how would library use it to mutate a string (something that was trivial before strong encapsulation and the JNI restriction)?

> 
> I can tell you that several design choices of the JPMS had nothing to do with encapsulation principles and everything to do with the philosophy of those who created the constraints. There's nothing inherent in the system that makes it necessary to prevent circularity, *or* to eagerly load all modules in a layer, *or* to eagerly resolve services.

True, but strong encapsulation is not the only benefit this feature provides and not every aspect of it serves strong encapsulation.

> there are real implications to these choices that mean that many kinds of Java application containers cannot be satisfactorily migrated to use JPMS modules. I hope you recognize that I am trying to change that, but that doing so may require some form of compromise from the platform itself.

I presume you’re trying to use the feature because you want to enjoy the benefits it offers now and in the future. Obviously, we want all our features to be as easy to use to maximise the value users get from them, but we also need to balance the cost of additional complexity and the utility it adds to the ecosystem as a whole. So we can consider that, and you can consider if there really is no acceptable design for a container other than loading every module in its own layer and constructing a complex layer graph. Maybe it turns out that *both* of these things are true, i.e. it would be a good idea to change ServiceLoader and also your design.

> 
> Yes. Quarkus currently does not use JPMS modules for the user application at build time or in any deployment mode, both of which instead rely on an arrangement of specialized class loaders, with all classes ending up in flat unnamed modules. My current experiments revolve around trying to create more modular-oriented and encapsulated packaging options.

So you want Quarkus to also load one module per layer?

— Ron