#LayerPrimitives

[Neil Bartlett]

I think there is some confusion about the purpose and scope of Tom Watson’s JPMS-OSGi interoperability experiment. He has talked about this publicly, but I would like to make some comments from the point of view of the wider OSGi community.

Tom works for IBM. They have many products built with OSGi, notably WebSphere Application Server and Liberty Profile, but also things like Lotus Notes and Domino. In the future it is likely that application server customers will want/need to deploy JPMS modules onto those servers. Thus it is a very real commercial question for IBM whether JPMS modules can be deployed onto an OSGi-based application server.

Tom’s experiment was intended to answer that question, which he did affirmatively, but there are some wide-reaching consequences. In order to deploy JPMS modules with properly declared dependencies into an application server, it was necessary to create an unbroken “chain” of JPMS modules from the boot layer through to the application module layer. This required wrapping OSGi bundles as JPMS modules and dynamically constructing many intermediate layers… this was necessary to support permitted features of OSGi that are absent in JPMS, such as cyclic dependencies, split packages, multiple module versions, multiple exports of packages, etc. It was an impressive engineering feat, but very complex. It’s also important to note that he did NOT show full bidirectional interoperability of JPMS and OSGi: it is not possible to arbitrarily mix JPMS modules and OSGi bundles. Instead, there is a strictly layered sandwich: JPMS's boot layer at the bottom, a “wrapped OSGI” filling, and JPMS modules on the top.

The vast majority of OSGi users do not have these concerns. I personally regard application servers as a dying product category. Nevertheless we need OSGi to work on Java 9 (it does, in classpath mode), and beyond that there might be some useful new features that we can leverage. Pretending that OSGi bundles are JPMS modules, or vice versa, is NOT of interest to many people.

What are those new features? JPMS is much more limited than OSGi in almost every respect, but one feature we may want to use is the supposedly enhanced security. That is, it would be useful if OSGi bundle authors could choose to lock down some packages so that they cannot be accessed from outside the bundle even reflectively. Unfortunately the implementation of this feature is tightly coupled to the entire of the rest of JPMS (ironically, JPMS is not very modular). We cannot simply map types to modules and elect to restrict access based on rules or callbacks that we define. To get this security feature we have to use the full JPMS structure of modules, module descriptors, layers, etc. It’s for this reason that we’re trying to hammer OSGi’s square peg into JPMS’s round hole.

As for Stephen’s question about building future module systems on top of JPMS, I believe that any such effort will be just as severely constrained by the choices being made in JPMS as we in OSGi and JBoss Modules are today.

Regards,
Neil


> On 20 Mar 2017, at 12:25, Stephen Colebourne <scolebourne at joda.org> wrote:
> 
> On 17 March 2017 at 16:39, David M. Lloyd <david.lloyd at redhat.com> wrote:
>>> It is clear to me that JPMS does contain an API with some flexibility
>>> here, but it is hard to judge whether my second expectation is met.
>>> This is because the discussion is ledf by OSGI and JBoss Modules
>>> migration concerns, rather than by consideration of what a new module
>>> system would need. Perhaps these concerns are the same, perhaps not. I
>>> would like to see a discussion from the EG about what blocks a
>>> theoretical new extended module system built on top of JPMS.
>> 
>> In the green field, all things are possible.
>> 
>> You could create a variety of module systems that behave in a variety of
>> ways and yet still adhere to the current JPMS restrictions.  If you're
>> willing to discard, rewrite, or retool all existing software, and change
>> best practices (even if the current practices are not particularly harmful
>> or problematic, or even useful), and derive your acceptable use cases from
>> the implementation restrictions (instead of the other way around), you can
>> conform to anything and fit in any situation.
> 
> I think the evidence is that the JPMS approaches the problem space in
> a different way to OSGI/JBossModules, so it is no surprise to find a
> clash. These existing module systems still work in classpath mode. It
> seems entirely reasonable that they don't work in module mode, since
> they represent a clashing view of modularization. Thus, I'm suggesting
> that it is right to consider the whole problem space again in terms of
> JPMS, not just try to adapt existing projects.
> 
>>> it is more important to close out Java 9 and move on.
>> Is it though?
> 
> Yes. The JPMS can be enhanced in future if necessary. The core team
> has spent copious amounts of time on modules, and it is time we got
> back to adding features that will benefit productivity, an area where
> Java remains weak. Whether modules will be widely adopted remains open
> to question, but modularizing the JDK itself will be a step forward,
> even if that is the only benefit.
> 
>> Will they care that the community process was
>> shunned in favor of expedience?
> 
> The JCP has tended to work OK for Java EE, but has never been a
> suitable mechanism for driving SE. I view it as an artificial box
> ticking process for SE - a zombie:
> http://blog.joda.org/2011/06/java-se-7-passes-in-zombie-jcp_1590.html
> 
> IMO, the issues to resolve are module naming (where all the advice is
> that short names will fail the wider community) and automatic modules
> (where guessing names is not viable).
> 
> Stephen