Coupling modules and class loaders

[mark.reinhold at oracle.com]

2015/12/15 9:50 -0800, david.lloyd at redhat.com:
> On 12/14/2015 01:44 PM, mark.reinhold at oracle.com wrote:
>> ...
>> 
>> What I'm thinking of is an existing application with its own non-trivial
>> class-loader architecture which, due to compatibility constraints, cannot
>> be changed.  If at least one of its class loaders loads classes from
>> logically-distinct components that are later converted into corresponding
>> modules then upgrading the application to load those components as
>> modules would, under your approach, require replacing that one class
>> loader with many class loaders, which would break compatibility.
>> 
>> Imagine, e.g., a Java EE application server that loads implementations of
>> all the standard EE components using one class loader.  Imagine further
>> that existing users (for good or for ill) depend upon that.  If a future
>> version of Java EE defines those components as modules then the only way
>> to arrange for this app server to load them as such would be to change
>> its class-loader architecture in a way that would break existing uses.
>> 
>> (This is, roughly, the situation with Oracle's WebLogic app server, as
>> I've mentioned previously.)
> 
> With a multi-class-loader arrangement, it should be possible to emulate 
> any single-class-loader arrangement, in two ways and with one caveat 
> that I know of:
> 
> * You can create modules which provide "cut down" views of another 
> module, such that the module appears as multiple modules but with one 
> initiating class loader for all classes actually loaded from them;

I think you mean one defining loader, not one initiating loader.  If
loader M actually loads a module, and loaders X and Y define "cut down"
views of M by delegating to M, then all three loaders can be initiating
loaders of classes in M but only M will be the defining loader of such
classes.

> * You can create a module which aggregates multiple modules into one, as 
> long as no code relies on the the specific relationship between the 
> initiating class loaders of classes within the module (as the initiating 
> class loader will reflect the module in which the class was defined; 
> this is the caveat I was referring to above).

Again, I think you mean defining loader.  If loaders M1 and M2 load
modules, and loader X aggregates those into a single apparent module,
then M1 and M2 will be defining loaders and X will be an initiating
loader.

> In the JDK itself, there's the special case of the null return of 
> Class.getClassLoader() being used for legacy security checks, for code 
> that depends on this, though there are multiple ways that I know of in 
> which this could potentially be mitigated, if this is a problem that you 
> are thinking of and you want to discuss it.
> 
> What do you think about these options in terms of WebLogic?  Are there 
> additional cases that cannot be covered by fixes in these two areas? 
> Are there other compatibility cases beyond these that you have in mind 
> where using multiple class loaders for either platform components or for 
> Java EE specification modules could break existing behaviors, or does 
> the combination of the change of initiating class loader and behavior of 
> Class.getClassLoader() summarize the potential issues that you see?

To make an approach along these lines work for the JDK, for WebLogic,
and for any other application with a non-trivial, non-loader-per-module
class-loader architecture, I think you're going to have to fundamentally
redefine the meaning of the Class::getClassLoader method.  It would no
longer return the defining loader of a class but, rather, an initiating,
aggregating loader as you describe above.  Given the roles of class
loaders in both VM-level access checking and SecurityManager-level
permission checking, this is far from being a simple change.

To take just one case, suppose a class C is loaded and defined by a
module class loader M1, which itself is aggregated together with some
other module class loader M2 by an aggregating loader X, so if I have
C's Class object then its getClassLoader method will return X.  Suppose
further that X is an instance of an end-user-supplied subclass of a
ClassLoader subclass provided by the application, long ago, as part of
its supported API.  When one of the defineClass methods of X is invoked
to define some new class D, what will be the defining loader of D?

If the defining loader is not X, then X must somehow delegate class
definition to either M1 or M2, but which one?  If D's package is already
defined to just one of those loaders then maybe defining it with that
loader would (often) be the right thing.  What would X do if D's package
is split across M1 and M2, or would you disallow that?  What would X do
if D's package is not yet defined in either M1 or M2, just define D with
X?  That might work in some cases but not if some code later on depends
upon D having been defined by M1 specifically.

If the defining loader is X, rather than M1 or M2, then D would have a
different defining loader relative to classes now in M1 or M2 than it did
when run on pre-9 releases.  If D is meant to have package-private access
to some class E in the same package, whose defining loader was X but is
now M1 due to modularization, then it will fail, since D's run-time
package will be different from that of E, since its defining loader is
different.

(You could consider an even deeper change and say that X is the defining
 loader, not just an initiating loader, of all classes in both M1 and M2.
 Module-private types in those modules would then, however, no longer be
 strongly encapsulated.)

So, that's just one case of one family of methods, defineClass, in the
existing ClassLoader API.  To change the meaning of Class::getClassLoader
would require a deep analysis of all the methods in ClassLoader, not just
this one.  Maybe there's some super-clever way to make them all work in a
manner that's both sensible and compatible, but I don't know what that is
and I can't believe it'd be straightforward.  This kind of complexity is
exactly why we abandoned the module-per-loader approach implemented in
the first Jigsaw prototype.

- Mark