Interoperation (Was: Re: Module-system requirements)

[David M. Lloyd]

On 02/15/2015 07:24 PM, mark.reinhold at oracle.com wrote:
> 2015/2/11 11:56 -0800, david.lloyd at redhat.com:
>> I'm concerned that the interoperation point doesn't quite go far enough.
>>    It says "It must be possible for another module system, such as OSGi,
>> to locate Java modules and resolve them using its own resolver, except
>> possibly for core system modules."
>>
>> I agree with this requirement, such as it is, but I would also assert
>> that it should be possible to retrofit not only OSGi but Java EE, Maven,
>> and even legacy "modules" like extensions and so on as modules.  The
>> reason I make this assertion is that we have done all of these things
>> with success and (I dare say) relative ease, even to the extent of
>> achieving both EE and OSGi certification with such a retrofit.
>
> Can you explain what you mean, exactly, by "retrofit"?

We have implemented both a certified OSGi implementation as well as a 
certified Java EE implementation which maps these specifications' module 
concepts onto our own concept of a module.  Also we have multiple 
projects which consume Maven dependency graphs in various ways to map 
these graphs onto our module concept, which works for the most part 
(though Maven dependency graphs are often overly inclusive, since the 
build system does not really do much to discourage erring on the side of 
pulling in everything you can "just in case").

> Are you saying that the JBoss module system can consume Java EE WAR
> files, OSGi bundles, and Maven projects, as they stand and without
> change?

Yes*.  This is because of implementation choices we've made - each 
module is a class loader, and we can carefully control which classes 
*and* resources are exported from one to another.  That said, I do think 
it's still worth exploring a different approach than ours regarding 
conceptually combining resource directories and packages, which is what 
we do today and is not without disadvantage.

* Occasionally there is a library that makes deep or odd assumptions 
about the class path or class loader behavior, *but* in practice these 
are typically older libraries that already don't work in EE and OSGi 
environments.  Overall we found that the vast majority of libraries work 
just fine in our modular environment.

>> ...
>>
>> To achieve this additional requirement does require a few very specific
>> implementation considerations but I do not believe that any of these
>> considerations contraindicate any of the other requirements stated here.
>
> Can you be more specific as to what those considerations are?  Can they
> be expressed as module-system requirements?

In the case of both Java EE and OSGi, I think it's safe to say that you 
need to apply a strict 1:1 mapping between modules and class loaders for 
this to work.  Also you need an API to allow run-time systems to create 
their own configurations (we call them "module loaders", akin in purpose 
and function to class loaders) and assemble modules from resources 
(JARs) and dependency specifications, and such APIs have to be able to 
relink modules or deregister modules to allow them to be GC'd (though 
the boot module loader in our case does not expose relink or unload 
functionality for obvious reasons).

OSGi specifically requires that bundle class loaders implement an OSGi 
spec interface, implying that the module class loader class must be 
extensible (as I said a couple of years back though, we make most of our 
methods final on the module class loader to disallow any really bizarre 
behavior, and this works well in practice).  We also had to implement a 
pre- and/or post-class-define hook per module to allow some OSGi magic 
to happen.  Finally there is a per-module fallback resource loader that 
our OSGi implementation used for one particular part of the spec.

Java EE's requirements are much simpler and are satisfied by 
module-granularity dependencies and (in certain cases) resource importing.

The APIs to do these things are all pretty simple though.

>> Thus I feel this requirement should be amended to something along the
>> lines of:
>>
>> Interoperation - It must be possible for another module system, such as
>> OSGi, to locate Java modules and resolve them using its own resolver,
>> except possibly for core system modules.  It must be possible for other
>> module systems to reuse the framework provided by this specification in
>> order to facilitate a standard, well-defined mechanism by which
>> different present and future module systems may interoperate.
>
> Unless I've misunderstood something, this sounds more like a research
> project -- a "meta" module system, in fact -- which is well outside the
> scope of this JSR.

Maybe, but I contend that a lot (enough, insofar as any amount is ever 
enough) of research has been done, and that in any event there will have 
to be an API which implements the functionality of this JSR anyway, and 
it will be used as such no matter our intent so IMO it's best to use 
what has been learned to solve the problem in the best manner possible.

-- 
- DML