#LayerPrimitives

[Andrew Dinn]

I'm not really sure this discussion is appropriate to this list rather
than the EG lists. However, I'm going to reply here anyway because i)
the subject has now been raised here and ii) I think this addresses the
adequacy of the current implementation as much as it concerns the
definition of the spec.

On 17/03/17 16:12, Stephen Colebourne wrote:
> I note the discussions on various threads about JPMS primitives.
> 
> My first expectation is that OSGI and JBoss modules can run in Java 9
> in classpath mode.
> 
> I believe this to be true today.

Indeed, so do I modulo, perhaps, a few wrinkles.

> My second expectation is that there are sufficient primitive
> operations within JPMS to allow a _new_ module system (not OSGI or
> JBoss Modules) to be built that permits some but not all of the
> features seen in OSGI and JBoss Modules. Thes should include:
> - dynamic loading/reloading of modules
> - lifecycle callbacks
> - multiple versions of a class loaded at the same time
> - handling of clashing packages
> - some form handling for module cycles

Arguably, that is answered in the positive by the first point. JBoss
Modules and OSGi can already do that /in classpath mode/. The room for
argument lies in the questions (highlighted in the above qualification)
as to

  whether code which avails itself of the capabilities offered by an
alternative module system should also be able to avail itself of the
functionality offered by JPMS

  whether and to what degree independent deployment units which,
respectively, employ the capabilities of some alternative module system
and JPMS should be able to interoperate

Regarding the first point, I have to ask why someone using an
alternative module system to modularize their code would also want to
use JPMS to modularize that same code. Of course, saying no to that that
use case shouldn't rule out the second option of integrating independent
deployment units that have been independently modularized (possibly by
different developers, possibly by the one developer), However,
interoperability is no reason to assume that the two module systems need
to operate completely hand in glove.

So, let me pursue that 'arguably' I dropped into an earlier paragraph
and argue now, up front, that /my/ perception of requests to support
more 'dynamic' capabilities in JPMS is that they are based on a
misguided desire to allow alternative module systems, specifically JBoss
Modulss and OSGi, to operate hand in glove with JPMS rather than
interoperate in a coherent and useful manner.

For example, concerns have been expressed on this list regarding the
ability to implement inversion of control in EE deployments that are
themselves JPMS modules. There is an assumption behind this 'problem'
that it makes sense to 'import' EE functionality into a JPMS-modularized
deployment. Is that level of mixing and matching actually appropriate?
Why *should* a module system designed (for perfectly good reasons) to
implement static and restrictive access be expected also to support the
dynamic and open-ended access needed to implement EE functionality that
depends on, e.g. inversion of control? Why so, *especially*, when there
are already two available module systems that can be used (on top of
JPMS) to achieve the required level of isolation, dynamic discovery etc.

Note that JPMS does not itself directly address the issue of what makes
sense when it comes to deploying modules in EE containers. It defines
what happens in SE when you deploy via either the MODULEPATH or the
CLASSPATH. It also provides a basic mechanism (the current Layer API)
that allows this module/jar split to be extended hierarchically but
defines no policy as to how it must be used. It is the responsibility of
EE implementors (and, in good time, EE spec committees) to *define* a
coherent notion of what it means to integrate a modular deployment into
EE, in particular into existing EE modularization mechanisms. So far, I
believe that question has been addressed by assuming that every possible
mode of interoperation up to and including concurrent usage needs to be
thrown in i.e. kitchen sink and all, irrespective of whether that makes
sense or is useful.

Let me explain that a tad further. If you ignore the Jigsaw Layer API
then modular jars function as modules only if you place them in the
MODULEPATH. If you put a modular jar into the CLASSPATH then its modular
behaviour is ignored. The Layer API implements a mechanism for
replicating this split between modular and non-modular deployment units
hierarchically. Each Layer provides a new MODULEPATH and CLASSPATH above
its parent Layer's MODULEPATH and CLASSPATH. Layers can see everything
below but not what lies above.

This support for hierarchical layering provides enough flexibility to
satisfy one of the key requirements for an EE container -- supporting
alternative versions of specific libraries. EE deployment managers can
launch different deployments in their own Layer sitting above the JDK
base Layer. That means they can configure different versions of a
JPMS-modular dependency in the MODULEPATH of each such higher Layer
instead of having to choose a single version to deploy in the MODULEPATH
of the base Layer. n.b. This is a *very* important use case because it
is very likely that library jars developed to run in SE will want to
adopt the use of JPMS. Hence, it is very important that they can be used
in EE deployed in different versions.

Now, the current attempts to integrate OSGi and JBoss Modules into this
model assume that it is necessary for EE functionality to be available
in any such Layer in *both* MODULEPATH and CLASSPATH deployments.
However, it is precisely the desire to deploy EE components in the
MODULEPATH that ties everything up in knots. EE deployments need to be
lazily linked. They need to be openly accessible to the underlying EE
implementation -- components like the Transaction Manager, JPA
implementation etc. That really doesn't fit with the way JPMS works. But
that is no reason to demand that JPMS must, in consequence, change to
accommodate these characteristics. IF i) it doesn't do that job and ii)
nothing else does that job either THEN the demand might seem reasonable.
However, as I noted above, conjunct ii of that 'IF' is false twice over:
false because we have JBoss Modules perfectly able to work in the
CLASSPATH; and false because we have OSGi perfectly able to work in the
CLASSPATH.

It seems quite reasonable to me to adopt the position that the
generalization of CLASSPATH deployment provided by EE is only available
via the CLASSPATH i.e. that EE deployments will not operate with EE
capabilities if deployed in the MODULEPATH, whether in the base layer or
in layers managed by an EE container. That's completely consistent with
the current converse status quo that JPMS deployments will not operate
as JPMS modules if they are deployed in the CLASSPATH. In particular, if
you want a deployment unit to use the CLASSPATH-based capabilities of EE
you cannot mix into it the modular capabilities provided by JPMS.

> It is clear to me that JPMS does contain an API with some flexibility
> here, but it is hard to judge whether my second expectation is met.
> This is because the discussion is ledf by OSGI and JBoss Modules
> migration concerns, rather than by consideration of what a new module
> system would need. Perhaps these concerns are the same, perhaps not. I
> would like to see a discussion from the EG about what blocks a
> theoretical new extended module system built on top of JPMS.

Indeed, I believe the question of what EE actually needs to do to
support and integrate JPMS really needs to be addressed explicitly
rather than proceed on the current unjustified presumptions. See above
for a start to that discussion.

> My third expectation is that it should be possible for bytecode/class
> generation tools to generate code inside a module, including in a
> package that is not known by the module graph.

That's only true if if you assume a specific mode of use for those
modular deployments i.e. that they need to be subject to the sort of
code transformations and access extensions that is appropriate when
using EE capabilities (as opposed to underpinning code using those
capabilities). In other words it's only true if you feel compelled to
(mis-)use Jigsaw for something it was consciously and deliberately not
designed to do.

> Note that while I understand the desire for OSGI and JBoss Modules to
> fit into JPMS and get the benefits, I do not consider that necessary
> to complete the JSR - it is more important to close out Java 9 and
> move on. These existing systems work today, and provide benefits
> today, something that won't change. That they won't get the enhanced
> security benefits of JPMS modules is a secondary concern IMO.

I completely agree with this. Irrespective of whether the current
attempts to integrate JPMS with OSGi and JBoss Modules (or EE in
general) are the right way to do it, they are both a long way from
approaching a proof of concept,let alone a complete and coherent
implementation. Indeed, I think they both suffer from an inadequate
specification of what is being attempted and why it is needed. That
means both are a long way from ever providing a complete definition of
what might or might not be adequate as regards the necessary supporting
functionality offered by JPMS. To me it is still an open question
whether EE actually needs the 'enhanced security benefits of JPMS
modules'. Contrariwise, it is *critical* for many other reasons that we
close out Java 9 and move on.

regards,


Andrew Dinn
-----------
Senior Principal Software Engineer
Red Hat UK Ltd
Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander