Coupling modules and class loaders

[David M. Lloyd]

On 12/14/2015 01:44 PM, mark.reinhold at oracle.com wrote:
> 2015/12/10 7:39 -0800, david.lloyd at redhat.com:
>> On 12/09/2015 07:27 PM, mark.reinhold at oracle.com wrote:
>>> 2015/10/21 3:53 -0700, david.lloyd at redhat.com:
>>>> ...
>>>>
>>>> I propose that modules are best defined as a specialization of class
>>>> loaders.  Class loaders already have many of the properties that define
>>>> a module: encapsulation, isolation, unique identity for packages.  In
>>>> fact OSGi already uses this unit.  The only thing they are really
>>>> missing is the ability to use a graph-like dependency structure, hence
>>>> the specialization.
>>>
>>> Class loaders do not provide strong encapsulation.  How do you propose to
>>> extend them to do so?
>>
>> Sure they do - here are some ways:
>>
>> * Package-private and protected members in one class loader are
>> inaccessible to packages in another class loader, even if the package
>> has the same name.
>
> (Yes, that's strong encapsulation, but not strong encapsulation of modules
>   or of module-like entities, which is what I meant.)

Ah, well I was looking at it from the other perspective: with the 
exception that this restriction *also* applies by package name, this is 
exactly per-class-loader encapsulation.  But that's all the other thread 
I guess.

>> * Run-time linkage is strongly encapsulated by class loader boundary; I
>> can declare a public type that nevertheless cannot be linked against by
>> anyone, if the class loader denies such access.  There are several use
>> cases for this.
>
> That is encapsulation of a sort, but it's not strong since it works
> differently with respect to reflection (which, yes, is the approach that
> you advocate).

I guess there's a definition of "encapsulation" and "strong 
encapsulation" implicit here.  I am thinking of strength as relative 
though: we can take the existing level of encapsulation, and build upon 
it in various ways, but in the end we need to agree what our final 
compatibility and security requirements are.

I think that the answer to both is that we should provide a means of 
encapsulation which provides the user with the ability to secure things 
at a module level, while also maintaining existing behaviors to a 
maximal degree.

>>> The primary motivation is, as Rémi noted, to make it easy for existing
>>> applications that use class loaders in sophisticated (and sometimes very
>>> strange) ways to adopt modules.  If the maintainers of such applications
>>> must make deep changes to their class-loading architectures in order to
>>> support modules then some of them simply won't bother.
>>
>> I don't think they would though.  Even in the complete compatibility
>> case, the "flat classpath" module could and would still just be one
>> module with the whole class path on it, which would be not unlike Ye
>> Olde Application Class Loader, while still allowing the forward
>> migration path of adding real module dependencies to the ugly legacy
>> class path.  ...  If people are doing weird/nasty/ugly stuff with class
>> loaders, they would certainly be free to continue to do so if they want;
>> the per-class loader idea does not necessarily require that the module
>> class loader class be final, nor does it forbid it from being used as a
>> parent or delegate, unless you're thinking of something else?
>
> What I'm thinking of is an existing application with its own non-trivial
> class-loader architecture which, due to compatibility constraints, cannot
> be changed.  If at least one of its class loaders loads classes from
> logically-distinct components that are later converted into corresponding
> modules then upgrading the application to load those components as
> modules would, under your approach, require replacing that one class
> loader with many class loaders, which would break compatibility.
>
> Imagine, e.g., a Java EE application server that loads implementations of
> all the standard EE components using one class loader.  Imagine further
> that existing users (for good or for ill) depend upon that.  If a future
> version of Java EE defines those components as modules then the only way
> to arrange for this app server to load them as such would be to change
> its class-loader architecture in a way that would break existing uses.
>
> (This is, roughly, the situation with Oracle's WebLogic app server, as
>   I've mentioned previously.)

With a multi-class-loader arrangement, it should be possible to emulate 
any single-class-loader arrangement, in two ways and with one caveat 
that I know of:

* You can create modules which provide "cut down" views of another 
module, such that the module appears as multiple modules but with one 
initiating class loader for all classes actually loaded from them;
* You can create a module which aggregates multiple modules into one, as 
long as no code relies on the the specific relationship between the 
initiating class loaders of classes within the module (as the initiating 
class loader will reflect the module in which the class was defined; 
this is the caveat I was referring to above).

(While these capabilities are not unlike those provided by Jigsaw, I 
think it is useful to enumerate and reconsider them freshly in the 
context of class-loader-oriented modules, if you're wondering why I 
might be saying something already well-understood.)

In the JDK itself, there's the special case of the null return of 
Class.getClassLoader() being used for legacy security checks, for code 
that depends on this, though there are multiple ways that I know of in 
which this could potentially be mitigated, if this is a problem that you 
are thinking of and you want to discuss it.

What do you think about these options in terms of WebLogic?  Are there 
additional cases that cannot be covered by fixes in these two areas? 
Are there other compatibility cases beyond these that you have in mind 
where using multiple class loaders for either platform components or for 
Java EE specification modules could break existing behaviors, or does 
the combination of the change of initiating class loader and behavior of 
Class.getClassLoader() summarize the potential issues that you see?

We made this exact change several years ago, though at the time our 
decision was that in this case, the internal class loader arrangement of 
the deployment, application server, and platform classes was not a 
primary compatibility factor (particularly given the language in the 
Java EE specification which specifically addresses reliance on the types 
or arrangements of class loaders), as long as certain of our spec- and 
non-spec-derived existing behaviors remained the same (e.g. reliance on 
TCCL to load classes and resources).  This admittedly made the process 
much simpler.

>>> The fact that OSGi and JBoss have successfully used class loaders to
>>> implement module systems that are capable of loading a wide variety of
>>> existing artifacts is interesting.  It does not, however, imply that all
>>> existing applications can be converted to use the class-loader-per-module
>>> approach, nor that they should be forced to do so.  The proposed design
>>> therefore neither mandates nor forbids that approach.
>>
>> I think we probably would not be able to put bounds on this without
>> enumerating specific use cases/behaviors (especially those that would
>> fail in this situation but not in the other), as I haven't encountered
>> nor been able to imagine a situation that would fail solely due to the
>> introduction of class loader based modules.
>
> See above.

-- 
- DML