Concerns with JPMS spec and Jigsaw implementation

Tue May 2 21:14:58 UTC 2017

Apologies that this has turned into rather a long note.  I wanted to 
capture in a single message the details that led me to suggest a "no"
vote for proceeding at this time.

I've tried to structure the note to outline the community issues that
we should aim to resolve in order to reach closer consensus, and the
outstanding  technical issues I think remain important to draw to a
conclusion in time for Proposed Final Draft.

== Community Issues ==

(1) Importantly, several expert group (EG) members have said we are not 
ready for moving to Proposed Final Draft.

While I understand the desire to keep the process moving along, I believe 
there is value in listening to the collective wisdom of the EG and 
respecting their opinion.  It should not come as a shock that executive 
committee representatives talk to the EG members and support their 
position as to whether the specification is ready to proceed.

We should only move to Proposed Final Draft when the EG reaches broad 
consensus that we are ready to do so, and that decision should be based 
upon the merit of the specification.  Multiple EG members have said that 
we are not ready yet.

e.g.
David's note
http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000626.html
Robert's note
http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000631.html
Neil's note
http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000633.html
Tim's note
http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000644.html

Mark's rebuttal
http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000635.html

Mark's decision to proceed despite lack of consensus
http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000650.html

(2) The EG have offered their insight on decisions that JPMS may regret.

While I understand that JPMS is not intended to be a replacement for the
OSGi and JBoss module systems in place today, there is an onus on the EG
to ensure that JPMS  doesn't cause any 'harm' to the platform now, and in
the foreseeable future.

There are examples where EG members have tried to offer experience and 
suggestions that have not been accepted, and there are examples where EG
members have arguably overreached the goals of JPMS.

A number of the JSR 376 design decisions have been made in the face of 
experienced module system designers participating in the expert group. 
The EG are not just there to provide credibility for the process.

e.g. 
Brian's call for naming conventions
"The build tools can help this migration from the old world to the new
one, but you must let us help."
http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-April/000677.html

Tom's review of implementation effectiveness
http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000623.html

David's suggestion to use class loaders for separation/isolation
http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000608.html

David's objection to prohibiting cycles
http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-February/000600.html

(3) Reluctance to accept changes that support alternative implementations.

The JPMS design was originally exclusively for the modularization of the 
platform itself.  Subsequently APIs emerged that form part of the SE 
platform to allow applications to interact with the JPMS. 

While it was generally agreed that the JPMS APIs would not provide a 
meta-module definition for different module system implementations, 
requests for enhancements that allow for better support of existing, 
successful module systems have been refused without reasonable 
justification.

e.g.
Remi's note on adding APIs (even if not to support existing systems)
http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000655.html

(4) Closing out issues despite EG objections. 

While not all objections will be fully resolved to everyone's 
satisfaction, 
I'd expect to see consensus on how to move forward, e.g. by agreement that
ideas are out of scope, should be deferred, are too complex to implement,
etc. especially where the EG objection is that such decisions are likely
to be detrimental to the Java community.

e.g.
David's objection to decisions on issue resolution
http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-April/000679.html

Remi's call for finding "common ground"
http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-April/000668.html

== Specific technical issues with the spec ==

I'll reiterate that I'm not expecting this JSR to design a replacement
for existing module systems that already do a fine job addressing their
goals.

The technical issues I call out below come down to ensuring that the JPMS 
design 'does no harm', i.e. it should not cause undue work, surprise, 
problems, and confusion to developers; and ultimately potential issues
for our customers.

While they have all been discussed on these lists, I don't think we have
reached broad consensus on their resolution, and we should try harder to
find a mutual understanding.

*Significant*

  Insufficient isolation - non-exported (private) packages in a module 
should be strictly an implementation detail.  The fact that JPMS defaults 
to a single class loader per layer means private packages can cause 
surprising conflicts when being loaded at the same time.

  Automatic module names - modules' dependencies are expressed as module 
names, so implementers providing a standard API must all use the same 
module name.  However, automatic module names are based upon jar file
names which are implementation details. We need to decide if module names
are shorthand for a set of APIs, or a specific implementation.

  Access restrictions break libraries doing reflection - a module 
must know a priori if it will be open to reflection, and know the
reflector's module name. This will break dependency injection etc.

*Lesser technical issues / Nice to haves*

Just for the record, these items which have been discussed by the EG 
could have been addressed by JPMS, but I think we can proceed knowing
these limitations.  I'll just outline them for the record.

 - Lack of support for runtime cycles

 - Lack of support for module versioning

 - Non-textual module descriptor

 - Transitive requirements are explicitly named. That is, where a module 
requires another module transitively, it forms part of module definition 
and downstream changes to the transitively required module can cause 
issues. Addressing insufficient isolation can ameliorate this problem.

 - API not isomorphic with module description - you cannot do things with 
the JPMS API that can be done via binary module descriptor. e.g. the
module name, package name, version string can be different in the binary 
description.

 - Lack of support for split packages

 - JPMS module resolution is not pluggable by alternative module system 
implementations.  This results in awkward support to implement 
alternative module system rules.  Layers are created and resolved
according to strict JPMS rules.  A layer Controller is available to 
create dynamic connections between runtime modules which may break JPMS 
rules in order to support other module system rules.

In my opinion, the EC voting "yes" to proceed to Proposed Final Draft 
would be appropriate when the EG are ready to call for the public review 
because an understanding (not necessarily universal agreement) has been 
reached across the team.

Thanks for reading!
Tim

Remi Forax <forax at univ-mlv.fr> wrote on 01/05/2017 23:12:20:
> while i'm also disappointed by the outcome of some discussions.
> I would like to know precisely what are the issues you would like us
> to have a closer consensus. 
> 
> I do not like the blog post of Scott Stark mostly because a lot of 
> references are from 2015 so a lot things are outdated in that post.
> So apart from the automatic modules, what are points you want to 
> discuss more ?
> 
> regards,
> Rémi
> 
> ----- Mail original -----
> > De: "Tim Ellison" <Tim_Ellison at uk.ibm.com>
> > À: "jpms-spec-experts" <jpms-spec-experts at openjdk.java.net>
> > Cc: jpms-spec-comments at openjdk.java.net
> > Envoyé: Vendredi 28 Avril 2017 17:48:43
> > Objet: Re: Concerns with JPMS spec and Jigsaw implementation
> 
> > mark.reinhold at oracle.com wrote on 27/04/2017 21:02:09:
> >> 2017/4/14 20:18:03 -0700, Scott Stark <sstark at redhat.com>:
> >> > Via our participation in JSR 376/JPMS we have raised a number of
> > concerns
> >> > regarding the implementation decision in Jigsaw as well as the 
scope
> > and
> >> > consensus of the expert group efforts. We, along with other EC
> > members, and
> >> > EG members have compiled a document that details these concerns. 
These
> >> > concerns are outlined in the following blog:
> >> > https://developer.jboss.org/blogs/scott.stark/2017/04/14/critical-
> >> 
> > 
deficiencies-in-jigsawjsr-376-java-platform-module-system-ec-member-concerns
> >> 
> >> This document does not raise any substantive technical issues that 
have
> > not
> >> already been discussed in the JSR 376 EG.
> > 
> > I believe that document demonstrates there is still work required to 
bring
> > the
> > community closer to an agreement on the proposed standard.
> > 
> > Even where the technical issues have already been discussed in the EG, 
it
> > is
> > incumbent upon this group to make best efforts to understand and 
resolve
> > any
> > differences.  If people feel that their argument has not been 
considered,
> > then
> > it is not surprising that frustration ensues.
> > 
> > I am personally disappointed that the move to call the Public Review
> > ballot
> > was made despite explicit objection from a number of EG members.
> > 
> >> > As it stands, Red Hat will not vote for the approval of this public
> > review
> >> > draft of JPMS as it is not in the best interest of the Java 
community.
> >> 
> >> Acknowledged.
> > 
> > IBM is also voting "no" which reflects our position that the JSR is 
not
> > ready at
> > this time to move beyond the Public Review stage and proceed to 
Proposed
> > Final
> > Draft.
> > 
> > The JSR 376 Expert Group and the public have raised a number of 
reasonable
> > issues
> > and concerns with the current public review draft of the specification
> > that
> > warrant further discussion and resolution.  We advocate work 
continuing
> > amongst
> > all members of the Expert Group to address the issues documented on 
the
> > mailing
> > lists.
> > 
> > IBM would like to see closer consensus amongst the entire Expert Group
> > before
> > this specification proceeds to the next step.
> > 
> > Regards,
> > Tim
> > 
> > 
> > Unless stated otherwise above:
> > IBM United Kingdom Limited - Registered in England and Wales with 
number
> > 741598.
> > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 
3AU
> 

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU