From forax at univ-mlv.fr  Mon May  1 22:12:20 2017
From: forax at univ-mlv.fr (Remi Forax)
Date: Tue, 2 May 2017 00:12:20 +0200 (CEST)
Subject: Concerns with JPMS spec and Jigsaw implementation
In-Reply-To: <OFC36630D7.60EAE30C-ON80258110.00530A8C-80258110.0056DB61@notes.na.collabserv.com>
References: <1622960697.16469710.1492226246622.JavaMail.zimbra@redhat.com>
 <1031408094.16469711.1492226283945.JavaMail.zimbra@redhat.com>
 <20170427200209.55053B6A18@eggemoggin.niobe.net>
 <OFC36630D7.60EAE30C-ON80258110.00530A8C-80258110.0056DB61@notes.na.collabserv.com>
Message-ID: <1158582525.322786.1493676740729.JavaMail.zimbra@u-pem.fr>

Hi Tim,
while i'm also disappointed by the outcome of some discussions.
I would like to know precisely what are the issues you would like us to have a closer consensus.   

I do not like the blog post of Scott Stark mostly because a lot of references are from 2015 so a lot things are outdated in that post.
So apart from the automatic modules, what are points you want to discuss more ?

regards,
R?mi

----- Mail original -----
> De: "Tim Ellison" <Tim_Ellison at uk.ibm.com>
> ?: "jpms-spec-experts" <jpms-spec-experts at openjdk.java.net>
> Cc: jpms-spec-comments at openjdk.java.net
> Envoy?: Vendredi 28 Avril 2017 17:48:43
> Objet: Re: Concerns with JPMS spec and Jigsaw implementation

> mark.reinhold at oracle.com wrote on 27/04/2017 21:02:09:
>> 2017/4/14 20:18:03 -0700, Scott Stark <sstark at redhat.com>:
>> > Via our participation in JSR 376/JPMS we have raised a number of
> concerns
>> > regarding the implementation decision in Jigsaw as well as the scope
> and
>> > consensus of the expert group efforts. We, along with other EC
> members, and
>> > EG members have compiled a document that details these concerns. These
>> > concerns are outlined in the following blog:
>> > https://developer.jboss.org/blogs/scott.stark/2017/04/14/critical-
>> 
> deficiencies-in-jigsawjsr-376-java-platform-module-system-ec-member-concerns
>> 
>> This document does not raise any substantive technical issues that have
> not
>> already been discussed in the JSR 376 EG.
> 
> I believe that document demonstrates there is still work required to bring
> the
> community closer to an agreement on the proposed standard.
> 
> Even where the technical issues have already been discussed in the EG, it
> is
> incumbent upon this group to make best efforts to understand and resolve
> any
> differences.  If people feel that their argument has not been considered,
> then
> it is not surprising that frustration ensues.
> 
> I am personally disappointed that the move to call the Public Review
> ballot
> was made despite explicit objection from a number of EG members.
> 
>> > As it stands, Red Hat will not vote for the approval of this public
> review
>> > draft of JPMS as it is not in the best interest of the Java community.
>> 
>> Acknowledged.
> 
> IBM is also voting "no" which reflects our position that the JSR is not
> ready at
> this time to move beyond the Public Review stage and proceed to Proposed
> Final
> Draft.
> 
> The JSR 376 Expert Group and the public have raised a number of reasonable
> issues
> and concerns with the current public review draft of the specification
> that
> warrant further discussion and resolution.  We advocate work continuing
> amongst
> all members of the Expert Group to address the issues documented on the
> mailing
> lists.
> 
> IBM would like to see closer consensus amongst the entire Expert Group
> before
> this specification proceeds to the next step.
> 
> Regards,
> Tim
> 
> 
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

From Tim_Ellison at uk.ibm.com  Tue May  2 21:14:58 2017
From: Tim_Ellison at uk.ibm.com (Tim Ellison)
Date: Tue, 2 May 2017 22:14:58 +0100
Subject: Concerns with JPMS spec and Jigsaw implementation
In-Reply-To: <1158582525.322786.1493676740729.JavaMail.zimbra@u-pem.fr>
References: <1622960697.16469710.1492226246622.JavaMail.zimbra@redhat.com>
 <1031408094.16469711.1492226283945.JavaMail.zimbra@redhat.com>
 <20170427200209.55053B6A18@eggemoggin.niobe.net>
 <OFC36630D7.60EAE30C-ON80258110.00530A8C-80258110.0056DB61@notes.na.collabserv.com>
 <1158582525.322786.1493676740729.JavaMail.zimbra@u-pem.fr>
Message-ID: <OF6E9D7DB6.BA5F120E-ON80258114.00682955-80258114.0074B9C1@notes.na.collabserv.com>

Apologies that this has turned into rather a long note.  I wanted to 
capture in a single message the details that led me to suggest a "no"
vote for proceeding at this time.

I've tried to structure the note to outline the community issues that
we should aim to resolve in order to reach closer consensus, and the
outstanding  technical issues I think remain important to draw to a
conclusion in time for Proposed Final Draft.

== Community Issues ==

(1) Importantly, several expert group (EG) members have said we are not 
ready for moving to Proposed Final Draft.

While I understand the desire to keep the process moving along, I believe 
there is value in listening to the collective wisdom of the EG and 
respecting their opinion.  It should not come as a shock that executive 
committee representatives talk to the EG members and support their 
position as to whether the specification is ready to proceed.

We should only move to Proposed Final Draft when the EG reaches broad 
consensus that we are ready to do so, and that decision should be based 
upon the merit of the specification.  Multiple EG members have said that 
we are not ready yet.

e.g.
David's note
http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000626.html
Robert's note
http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000631.html
Neil's note
http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000633.html
Tim's note
http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000644.html

Mark's rebuttal
http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000635.html

Mark's decision to proceed despite lack of consensus
http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000650.html


(2) The EG have offered their insight on decisions that JPMS may regret.

While I understand that JPMS is not intended to be a replacement for the
OSGi and JBoss module systems in place today, there is an onus on the EG
to ensure that JPMS  doesn't cause any 'harm' to the platform now, and in
the foreseeable future.

There are examples where EG members have tried to offer experience and 
suggestions that have not been accepted, and there are examples where EG
members have arguably overreached the goals of JPMS.

A number of the JSR 376 design decisions have been made in the face of 
experienced module system designers participating in the expert group. 
The EG are not just there to provide credibility for the process.

e.g. 
Brian's call for naming conventions
"The build tools can help this migration from the old world to the new
one, but you must let us help."
http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-April/000677.html

Tom's review of implementation effectiveness
http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000623.html

David's suggestion to use class loaders for separation/isolation
http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000608.html

David's objection to prohibiting cycles
http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-February/000600.html


(3) Reluctance to accept changes that support alternative implementations.

The JPMS design was originally exclusively for the modularization of the 
platform itself.  Subsequently APIs emerged that form part of the SE 
platform to allow applications to interact with the JPMS. 

While it was generally agreed that the JPMS APIs would not provide a 
meta-module definition for different module system implementations, 
requests for enhancements that allow for better support of existing, 
successful module systems have been refused without reasonable 
justification.

e.g.
Remi's note on adding APIs (even if not to support existing systems)
http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000655.html


(4) Closing out issues despite EG objections. 

While not all objections will be fully resolved to everyone's 
satisfaction, 
I'd expect to see consensus on how to move forward, e.g. by agreement that
ideas are out of scope, should be deferred, are too complex to implement,
etc. especially where the EG objection is that such decisions are likely
to be detrimental to the Java community.

e.g.
David's objection to decisions on issue resolution
http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-April/000679.html

Remi's call for finding "common ground"
http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-April/000668.html



== Specific technical issues with the spec ==

I'll reiterate that I'm not expecting this JSR to design a replacement
for existing module systems that already do a fine job addressing their
goals.

The technical issues I call out below come down to ensuring that the JPMS 
design 'does no harm', i.e. it should not cause undue work, surprise, 
problems, and confusion to developers; and ultimately potential issues
for our customers.

While they have all been discussed on these lists, I don't think we have
reached broad consensus on their resolution, and we should try harder to
find a mutual understanding.

*Significant*

  Insufficient isolation - non-exported (private) packages in a module 
should be strictly an implementation detail.  The fact that JPMS defaults 
to a single class loader per layer means private packages can cause 
surprising conflicts when being loaded at the same time.

  Automatic module names - modules' dependencies are expressed as module 
names, so implementers providing a standard API must all use the same 
module name.  However, automatic module names are based upon jar file
names which are implementation details. We need to decide if module names
are shorthand for a set of APIs, or a specific implementation.

  Access restrictions break libraries doing reflection - a module 
must know a priori if it will be open to reflection, and know the
reflector's module name. This will break dependency injection etc.


*Lesser technical issues / Nice to haves*

Just for the record, these items which have been discussed by the EG 
could have been addressed by JPMS, but I think we can proceed knowing
these limitations.  I'll just outline them for the record.

 - Lack of support for runtime cycles

 - Lack of support for module versioning

 - Non-textual module descriptor

 - Transitive requirements are explicitly named. That is, where a module 
requires another module transitively, it forms part of module definition 
and downstream changes to the transitively required module can cause 
issues. Addressing insufficient isolation can ameliorate this problem.

 - API not isomorphic with module description - you cannot do things with 
the JPMS API that can be done via binary module descriptor. e.g. the
module name, package name, version string can be different in the binary 
description.

 - Lack of support for split packages

 - JPMS module resolution is not pluggable by alternative module system 
implementations.  This results in awkward support to implement 
alternative module system rules.  Layers are created and resolved
according to strict JPMS rules.  A layer Controller is available to 
create dynamic connections between runtime modules which may break JPMS 
rules in order to support other module system rules.



In my opinion, the EC voting "yes" to proceed to Proposed Final Draft 
would be appropriate when the EG are ready to call for the public review 
because an understanding (not necessarily universal agreement) has been 
reached across the team.

Thanks for reading!
Tim




Remi Forax <forax at univ-mlv.fr> wrote on 01/05/2017 23:12:20:
> while i'm also disappointed by the outcome of some discussions.
> I would like to know precisely what are the issues you would like us
> to have a closer consensus. 
> 
> I do not like the blog post of Scott Stark mostly because a lot of 
> references are from 2015 so a lot things are outdated in that post.
> So apart from the automatic modules, what are points you want to 
> discuss more ?
> 
> regards,
> R?mi
> 
> ----- Mail original -----
> > De: "Tim Ellison" <Tim_Ellison at uk.ibm.com>
> > ?: "jpms-spec-experts" <jpms-spec-experts at openjdk.java.net>
> > Cc: jpms-spec-comments at openjdk.java.net
> > Envoy?: Vendredi 28 Avril 2017 17:48:43
> > Objet: Re: Concerns with JPMS spec and Jigsaw implementation
> 
> > mark.reinhold at oracle.com wrote on 27/04/2017 21:02:09:
> >> 2017/4/14 20:18:03 -0700, Scott Stark <sstark at redhat.com>:
> >> > Via our participation in JSR 376/JPMS we have raised a number of
> > concerns
> >> > regarding the implementation decision in Jigsaw as well as the 
scope
> > and
> >> > consensus of the expert group efforts. We, along with other EC
> > members, and
> >> > EG members have compiled a document that details these concerns. 
These
> >> > concerns are outlined in the following blog:
> >> > https://developer.jboss.org/blogs/scott.stark/2017/04/14/critical-
> >> 
> > 
deficiencies-in-jigsawjsr-376-java-platform-module-system-ec-member-concerns
> >> 
> >> This document does not raise any substantive technical issues that 
have
> > not
> >> already been discussed in the JSR 376 EG.
> > 
> > I believe that document demonstrates there is still work required to 
bring
> > the
> > community closer to an agreement on the proposed standard.
> > 
> > Even where the technical issues have already been discussed in the EG, 
it
> > is
> > incumbent upon this group to make best efforts to understand and 
resolve
> > any
> > differences.  If people feel that their argument has not been 
considered,
> > then
> > it is not surprising that frustration ensues.
> > 
> > I am personally disappointed that the move to call the Public Review
> > ballot
> > was made despite explicit objection from a number of EG members.
> > 
> >> > As it stands, Red Hat will not vote for the approval of this public
> > review
> >> > draft of JPMS as it is not in the best interest of the Java 
community.
> >> 
> >> Acknowledged.
> > 
> > IBM is also voting "no" which reflects our position that the JSR is 
not
> > ready at
> > this time to move beyond the Public Review stage and proceed to 
Proposed
> > Final
> > Draft.
> > 
> > The JSR 376 Expert Group and the public have raised a number of 
reasonable
> > issues
> > and concerns with the current public review draft of the specification
> > that
> > warrant further discussion and resolution.  We advocate work 
continuing
> > amongst
> > all members of the Expert Group to address the issues documented on 
the
> > mailing
> > lists.
> > 
> > IBM would like to see closer consensus amongst the entire Expert Group
> > before
> > this specification proceeds to the next step.
> > 
> > Regards,
> > Tim
> > 
> > 
> > Unless stated otherwise above:
> > IBM United Kingdom Limited - Registered in England and Wales with 
number
> > 741598.
> > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 
3AU
> 

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

From mark.reinhold at oracle.com  Thu May  4 17:38:32 2017
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Thu,  4 May 2017 10:38:32 -0700 (PDT)
Subject: Proposal: #AutomaticModuleNames (revised)
Message-ID: <20170504173833.07D4CC3DAF@eggemoggin.niobe.net>

Issue summary
-------------

  #AutomaticModuleNames -- Revise the algorithm that computes the names
  of automatic modules to include the Maven group identifier, when
  available in a JAR file's `pom.properties` file, so that module names
  are less likely to collide, or else drop the automatic-modules feature
  entirely, since it may be more trouble than it's worth.  [1]

Proposal
--------

  - Do not drop the automatic-modules feature.  It's a critical part of
    the overall migration story: It allows application developers to
    get started modularizing their own code without having to wait for
    all of the libraries and frameworks that they use to be modularized.
    Popular libraries and frameworks that are actively maintained will
    hopefully be modularized fairly soon, but less-active projects will
    take longer -- perhaps years or even decades -- and many inactive or
    abandoned projects will never be modularized.  This feature has been
    well-received by a wide variety of developers despite the fact that
    some expert users are not comfortable with it.

  - Define a JAR-file manifest attribute, `Automatic-Module-Name`, whose
    value is used as the name of the automatic module defined by that JAR
    file when it is placed on the module path, as previously suggested
    [2][3].  If a JAR file on the module path does not have such a
    manifest attribute then its automatic-module name is computed using
    the existing filename-based algorithm.

    The name of this attribute is now `Automatic-Module-Name` rather than
    `Module-Name`.  The former makes it clear that if a JAR file on the
    module path contains both an `Automatic-Module-Name` manifest
    attribute and an explicit `module-info.class` file then the manifest
    attribute is irrelevant, because the module defined by the JAR file
    is explicit rather than automatic.  `Module-Name` would also suggest
    that there are `Module-Exports` and `Module-Requires` attributes,
    which there are not.

    With this mechanism a library maintainer can claim a stable module
    name with very little effort; in Maven, e.g., a manifest attribute
    can be added in just a few lines of a project's `pom.xml` file.  A
    library maintainer can claim a stable name early, before all of the
    library's dependencies are modularized, which will allow libraries
    and applications that depend upon it to be modularized immediately.
    A user of a library can suggest a stable name to the library's
    maintainer, and easily submit a patch that implements it.  This will
    enable the uncoordinated modularization of libraries across the
    entire ecosystem.

  - Strongly recommend that all modules be named according to the reverse
    Internet domain-name convention.  A module's name should correspond
    to the name of its principal exported API package, which should also
    follow that convention.  If a module does not have such a package, or
    if for legacy reasons it must have a name that does not correspond to
    one of its exported packages, then its name should at least start
    with the reversed form of an Internet domain with which the author is
    associated.

  - Make sure that developers are aware of the risks of publishing, for
    broad use, explicit modules that require automatic modules.  An
    automatic module is unreliable, since it can depend on types on the
    class path, and its name and exported packages could change if and
    when it's converted into an explicit module.  It's fine to declare
    and use explicit modules that require automatic modules in limited
    settings.  If you publish such a module to, e.g., Maven Central, then
    you must be prepared to revise both the declaration and content of
    your module as its dependencies are modularized over time.  This may
    have an adverse impact on the users of your module, especially if
    your module `requires transitive` an automatic module.

The following three points are essentially unchanged from the first
proposal [4]:

  - Do not revise the algorithm that computes the names of automatic
    modules.  The suggestion to use Maven coordinates when available [5]
    would help less than half of the most popular projects in use today,
    as determined by three different surveys.  It would result in module
    names that are often annoyingly verbose and known to be objectionable
    to some module authors, and it would raise non-trivial issues with
    regard to standardization and convention.  The names generated by the
    current algorithm do not follow the reverse-DNS convention, but they
    are at least easily derivable from the name of the original JAR
    files.  [6]

  - To increase awareness of when automatic modules are used, and of the
    consequences of their use, suggest that Java language compilers issue
    two new types of warnings, and implement these warnings in the RI:

      - An opt-in (i.e., off-by-default) warning on a reference to
        an automatic module in a `requires` directive in a module
        declaration, and

      - An opt-out (i.e., on-by-default) warning on a reference to
        an automatic module in a `requires transitive` directive in
        a module declaration.

    The first warning allows developers to be maximally hygienic if they
    so choose.  The second helps make the author of an explicit module
    aware that they're putting the users of that module at risk by
    establishing implied readability to an automatic module.

  - Encourage Java run-time systems to make it easy to identify via,
    e.g., command-line options, which observable modules are automatic,
    which observable modules would establish implied readability to
    automatic modules if resolved, and which automatic observable modules
    are actually resolved.  Implement these capabilities in the RI.

Notes
-----

This revised proposal is informed by a vigorous discussion on the OpenJDK
jigsaw-dev mailing list [7] of the first proposal [4].  Two things became
clear in that discussion, and in related discussions triggered by it.

First, after twenty years the reverse-DNS naming convention is deeply
ingrained in the minds of most Java developers.  Stephen Colebourne made
a clear and passionate case for it [8] and many developers voiced support
for his argument, both on Twitter and elsewhere.  Some people may prefer
shorter, project-oriented names, and those can be fine to use in limited
projects that will never, ever see the light of day outside of a single
organization.  If you create a module that has even a small chance of
being open-sourced in the future, however, then the safest course is to
choose a reverse-DNS name for it at the start.

Second, the fact that the name and effective API of an automatic module
are inherently unstable is less of a problem than I previously thought
[9][a].  Developers who use Maven and related tools, such as Gradle --
which is most developers -- are used to coping with breakage when they
upgrade the dependencies of a project to a newer version [b][c].  If the
name or effective API of an automatic module change, either when its
maintainer adds an `Automatic-Module-Name` attribute or writes an
explicit `module-info.java` file, then the maintainers of projects that
use that module will just fix any problems that they encounter when they
upgrade.

After thinking about the first observation I'm now convinced that if
tools insert `Automatic-Module-Name` manifest attributes then they will
only do so when the module names that they define follow the reverse-DNS
convention, since to do otherwise would go against what most developers
strongly prefer and expect.  After thinking about that and the second
observation I hereby retract [d] my previous advice [9] that automatic
modules should not be given stable names.

                                  * * *

In related threads on jigsaw-dev both Stephen Colebourne [e] and Robert
Scholte [f] suggest a new kind of explicit module, variously called
"partial" or "incomplete" or "soft", which can read all unnamed modules
so that dependencies that have not yet been modularized can be left on
the class path.  This is an attractive notion on the surface, and in fact
one that we partly implemented in the prototype code base over two years
ago, using the term "loose".  We didn't pursue it then, however, and I
don't think it's worth pursuing now, for two reasons:

  - If "partial" (or whatever) modules were to replace automatic modules
    then application developers would have to wait until all of their
    dependencies have been at least "partially" modularized before they
    can even begin to experiment with modularizing their own code.

  - We could solve that by keeping automatic modules and adding partial
    modules, but then we'd have a more complex module system and a more
    complex story to tell.  To consider just one possible question,
    should developers go ahead and use their dependencies as automatic
    modules, or wait for them to become named automatic modules, or wait
    for them to become partial modules?

The `Automatic-Module-Name` manifest attribute is, from a technical
standpoint, aesthetically unpleasant since it defines a second way
to name modules.  Taken alone, however, it does enable uncoordinated
modularization using stable module names with little effort, and so
it appears to be the least-bad option on the table.


[1] http://openjdk.java.net/projects/jigsaw/spec/issues/#AutomaticModuleNames
[2] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-February/000582.html
[3] http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleNameInManifest
[4] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-April/000667.html
[5] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-January/000537.html
[6] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-April/000666.html
[7] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/thread.html#11955
[8] http://blog.joda.org/2017/04/java-se-9-jpms-module-naming.html
[9] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-April/000682.html
[a] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012304.html
[b] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012327.html
[c] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012335.html
[d] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012367.html
[e] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012370.html
[f] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012399.html

From forax at univ-mlv.fr  Thu May  4 22:09:20 2017
From: forax at univ-mlv.fr (Remi Forax)
Date: Thu, 04 May 2017 22:09:20 +0000
Subject: Proposal: #AutomaticModuleNames (revised)
In-Reply-To: <20170504173833.07D4CC3DAF@eggemoggin.niobe.net>
References: <20170504173833.07D4CC3DAF@eggemoggin.niobe.net>
Message-ID: <38B2B19F-8B07-4D9C-B70F-56FF15674D5E@univ-mlv.fr>

Being able to provide stable names as fast as possible for jars will ease the transition toward a modular world.

I'm very pleased by this proposal. 

thanks Mark,
R?mi


On May 4, 2017 7:38:32 PM GMT+02:00, mark.reinhold at oracle.com wrote:
>Issue summary
>-------------
>
>  #AutomaticModuleNames -- Revise the algorithm that computes the names
>  of automatic modules to include the Maven group identifier, when
>  available in a JAR file's `pom.properties` file, so that module names
> are less likely to collide, or else drop the automatic-modules feature
>  entirely, since it may be more trouble than it's worth.  [1]
>
>Proposal
>--------
>
>  - Do not drop the automatic-modules feature.  It's a critical part of
>    the overall migration story: It allows application developers to
>    get started modularizing their own code without having to wait for
>   all of the libraries and frameworks that they use to be modularized.
>    Popular libraries and frameworks that are actively maintained will
>    hopefully be modularized fairly soon, but less-active projects will
>   take longer -- perhaps years or even decades -- and many inactive or
>   abandoned projects will never be modularized.  This feature has been
>    well-received by a wide variety of developers despite the fact that
>    some expert users are not comfortable with it.
>
> - Define a JAR-file manifest attribute, `Automatic-Module-Name`, whose
>  value is used as the name of the automatic module defined by that JAR
>    file when it is placed on the module path, as previously suggested
>    [2][3].  If a JAR file on the module path does not have such a
>    manifest attribute then its automatic-module name is computed using
>    the existing filename-based algorithm.
>
>  The name of this attribute is now `Automatic-Module-Name` rather than
>    `Module-Name`.  The former makes it clear that if a JAR file on the
>    module path contains both an `Automatic-Module-Name` manifest
>   attribute and an explicit `module-info.class` file then the manifest
>    attribute is irrelevant, because the module defined by the JAR file
>   is explicit rather than automatic.  `Module-Name` would also suggest
>    that there are `Module-Exports` and `Module-Requires` attributes,
>    which there are not.
>
>    With this mechanism a library maintainer can claim a stable module
>    name with very little effort; in Maven, e.g., a manifest attribute
>    can be added in just a few lines of a project's `pom.xml` file.  A
>    library maintainer can claim a stable name early, before all of the
>    library's dependencies are modularized, which will allow libraries
>    and applications that depend upon it to be modularized immediately.
>    A user of a library can suggest a stable name to the library's
>   maintainer, and easily submit a patch that implements it.  This will
>    enable the uncoordinated modularization of libraries across the
>    entire ecosystem.
>
>- Strongly recommend that all modules be named according to the reverse
>    Internet domain-name convention.  A module's name should correspond
>   to the name of its principal exported API package, which should also
>  follow that convention.  If a module does not have such a package, or
>  if for legacy reasons it must have a name that does not correspond to
>    one of its exported packages, then its name should at least start
>  with the reversed form of an Internet domain with which the author is
>    associated.
>
>  - Make sure that developers are aware of the risks of publishing, for
>    broad use, explicit modules that require automatic modules.  An
>    automatic module is unreliable, since it can depend on types on the
>    class path, and its name and exported packages could change if and
>    when it's converted into an explicit module.  It's fine to declare
>    and use explicit modules that require automatic modules in limited
>  settings.  If you publish such a module to, e.g., Maven Central, then
>    you must be prepared to revise both the declaration and content of
>   your module as its dependencies are modularized over time.  This may
>    have an adverse impact on the users of your module, especially if
>    your module `requires transitive` an automatic module.
>
>The following three points are essentially unchanged from the first
>proposal [4]:
>
>  - Do not revise the algorithm that computes the names of automatic
>   modules.  The suggestion to use Maven coordinates when available [5]
>   would help less than half of the most popular projects in use today,
>   as determined by three different surveys.  It would result in module
>  names that are often annoyingly verbose and known to be objectionable
>    to some module authors, and it would raise non-trivial issues with
>  regard to standardization and convention.  The names generated by the
>   current algorithm do not follow the reverse-DNS convention, but they
>    are at least easily derivable from the name of the original JAR
>    files.  [6]
>
> - To increase awareness of when automatic modules are used, and of the
>  consequences of their use, suggest that Java language compilers issue
>    two new types of warnings, and implement these warnings in the RI:
>
>      - An opt-in (i.e., off-by-default) warning on a reference to
>        an automatic module in a `requires` directive in a module
>        declaration, and
>
>      - An opt-out (i.e., on-by-default) warning on a reference to
>        an automatic module in a `requires transitive` directive in
>        a module declaration.
>
>   The first warning allows developers to be maximally hygienic if they
>    so choose.  The second helps make the author of an explicit module
>    aware that they're putting the users of that module at risk by
>    establishing implied readability to an automatic module.
>
>  - Encourage Java run-time systems to make it easy to identify via,
>    e.g., command-line options, which observable modules are automatic,
>    which observable modules would establish implied readability to
>  automatic modules if resolved, and which automatic observable modules
>    are actually resolved.  Implement these capabilities in the RI.
>
>Notes
>-----
>
>This revised proposal is informed by a vigorous discussion on the
>OpenJDK
>jigsaw-dev mailing list [7] of the first proposal [4].  Two things
>became
>clear in that discussion, and in related discussions triggered by it.
>
>First, after twenty years the reverse-DNS naming convention is deeply
>ingrained in the minds of most Java developers.  Stephen Colebourne
>made
>a clear and passionate case for it [8] and many developers voiced
>support
>for his argument, both on Twitter and elsewhere.  Some people may
>prefer
>shorter, project-oriented names, and those can be fine to use in
>limited
>projects that will never, ever see the light of day outside of a single
>organization.  If you create a module that has even a small chance of
>being open-sourced in the future, however, then the safest course is to
>choose a reverse-DNS name for it at the start.
>
>Second, the fact that the name and effective API of an automatic module
>are inherently unstable is less of a problem than I previously thought
>[9][a].  Developers who use Maven and related tools, such as Gradle --
>which is most developers -- are used to coping with breakage when they
>upgrade the dependencies of a project to a newer version [b][c].  If
>the
>name or effective API of an automatic module change, either when its
>maintainer adds an `Automatic-Module-Name` attribute or writes an
>explicit `module-info.java` file, then the maintainers of projects that
>use that module will just fix any problems that they encounter when
>they
>upgrade.
>
>After thinking about the first observation I'm now convinced that if
>tools insert `Automatic-Module-Name` manifest attributes then they will
>only do so when the module names that they define follow the
>reverse-DNS
>convention, since to do otherwise would go against what most developers
>strongly prefer and expect.  After thinking about that and the second
>observation I hereby retract [d] my previous advice [9] that automatic
>modules should not be given stable names.
>
>                                  * * *
>
>In related threads on jigsaw-dev both Stephen Colebourne [e] and Robert
>Scholte [f] suggest a new kind of explicit module, variously called
>"partial" or "incomplete" or "soft", which can read all unnamed modules
>so that dependencies that have not yet been modularized can be left on
>the class path.  This is an attractive notion on the surface, and in
>fact
>one that we partly implemented in the prototype code base over two
>years
>ago, using the term "loose".  We didn't pursue it then, however, and I
>don't think it's worth pursuing now, for two reasons:
>
> - If "partial" (or whatever) modules were to replace automatic modules
>    then application developers would have to wait until all of their
>    dependencies have been at least "partially" modularized before they
>    can even begin to experiment with modularizing their own code.
>
>  - We could solve that by keeping automatic modules and adding partial
>    modules, but then we'd have a more complex module system and a more
>    complex story to tell.  To consider just one possible question,
>    should developers go ahead and use their dependencies as automatic
>   modules, or wait for them to become named automatic modules, or wait
>    for them to become partial modules?
>
>The `Automatic-Module-Name` manifest attribute is, from a technical
>standpoint, aesthetically unpleasant since it defines a second way
>to name modules.  Taken alone, however, it does enable uncoordinated
>modularization using stable module names with little effort, and so
>it appears to be the least-bad option on the table.
>
>
>[1]
>http://openjdk.java.net/projects/jigsaw/spec/issues/#AutomaticModuleNames
>[2]
>http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-February/000582.html
>[3]
>http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleNameInManifest
>[4]
>http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-April/000667.html
>[5]
>http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-January/000537.html
>[6]
>http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-April/000666.html
>[7]
>http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/thread.html#11955
>[8] http://blog.joda.org/2017/04/java-se-9-jpms-module-naming.html
>[9]
>http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-April/000682.html
>[a]
>http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012304.html
>[b]
>http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012327.html
>[c]
>http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012335.html
>[d]
>http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012367.html
>[e]
>http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012370.html
>[f]
>http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012399.html

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

From brianf at infinity.nu  Fri May  5 00:36:10 2017
From: brianf at infinity.nu (Brian Fox)
Date: Thu, 4 May 2017 17:36:10 -0700
Subject: Proposal: #AutomaticModuleNames (revised)
In-Reply-To: <38B2B19F-8B07-4D9C-B70F-56FF15674D5E@univ-mlv.fr>
References: <20170504173833.07D4CC3DAF@eggemoggin.niobe.net>
 <38B2B19F-8B07-4D9C-B70F-56FF15674D5E@univ-mlv.fr>
Message-ID: <CAMk_BehZKkMFo6X5kD=nf8S3OyazcXnqox_s6B3B+nfJwsbcug@mail.gmail.com>

On Thu, May 4, 2017 at 3:09 PM, Remi Forax <forax at univ-mlv.fr> wrote:

> Being able to provide stable names as fast as possible for jars will ease
> the transition toward a modular world.
>
> I'm very pleased by this proposal.
>
>
Yes, this is a very good thing. Thank you Mark.

--Brian



> thanks Mark,
> R?mi
>
>
> On May 4, 2017 7:38:32 PM GMT+02:00, mark.reinhold at oracle.com wrote:
> >Issue summary
> >-------------
> >
> >  #AutomaticModuleNames -- Revise the algorithm that computes the names
> >  of automatic modules to include the Maven group identifier, when
> >  available in a JAR file's `pom.properties` file, so that module names
> > are less likely to collide, or else drop the automatic-modules feature
> >  entirely, since it may be more trouble than it's worth.  [1]
> >
> >Proposal
> >--------
> >
> >  - Do not drop the automatic-modules feature.  It's a critical part of
> >    the overall migration story: It allows application developers to
> >    get started modularizing their own code without having to wait for
> >   all of the libraries and frameworks that they use to be modularized.
> >    Popular libraries and frameworks that are actively maintained will
> >    hopefully be modularized fairly soon, but less-active projects will
> >   take longer -- perhaps years or even decades -- and many inactive or
> >   abandoned projects will never be modularized.  This feature has been
> >    well-received by a wide variety of developers despite the fact that
> >    some expert users are not comfortable with it.
> >
> > - Define a JAR-file manifest attribute, `Automatic-Module-Name`, whose
> >  value is used as the name of the automatic module defined by that JAR
> >    file when it is placed on the module path, as previously suggested
> >    [2][3].  If a JAR file on the module path does not have such a
> >    manifest attribute then its automatic-module name is computed using
> >    the existing filename-based algorithm.
> >
> >  The name of this attribute is now `Automatic-Module-Name` rather than
> >    `Module-Name`.  The former makes it clear that if a JAR file on the
> >    module path contains both an `Automatic-Module-Name` manifest
> >   attribute and an explicit `module-info.class` file then the manifest
> >    attribute is irrelevant, because the module defined by the JAR file
> >   is explicit rather than automatic.  `Module-Name` would also suggest
> >    that there are `Module-Exports` and `Module-Requires` attributes,
> >    which there are not.
> >
> >    With this mechanism a library maintainer can claim a stable module
> >    name with very little effort; in Maven, e.g., a manifest attribute
> >    can be added in just a few lines of a project's `pom.xml` file.  A
> >    library maintainer can claim a stable name early, before all of the
> >    library's dependencies are modularized, which will allow libraries
> >    and applications that depend upon it to be modularized immediately.
> >    A user of a library can suggest a stable name to the library's
> >   maintainer, and easily submit a patch that implements it.  This will
> >    enable the uncoordinated modularization of libraries across the
> >    entire ecosystem.
> >
> >- Strongly recommend that all modules be named according to the reverse
> >    Internet domain-name convention.  A module's name should correspond
> >   to the name of its principal exported API package, which should also
> >  follow that convention.  If a module does not have such a package, or
> >  if for legacy reasons it must have a name that does not correspond to
> >    one of its exported packages, then its name should at least start
> >  with the reversed form of an Internet domain with which the author is
> >    associated.
> >
> >  - Make sure that developers are aware of the risks of publishing, for
> >    broad use, explicit modules that require automatic modules.  An
> >    automatic module is unreliable, since it can depend on types on the
> >    class path, and its name and exported packages could change if and
> >    when it's converted into an explicit module.  It's fine to declare
> >    and use explicit modules that require automatic modules in limited
> >  settings.  If you publish such a module to, e.g., Maven Central, then
> >    you must be prepared to revise both the declaration and content of
> >   your module as its dependencies are modularized over time.  This may
> >    have an adverse impact on the users of your module, especially if
> >    your module `requires transitive` an automatic module.
> >
> >The following three points are essentially unchanged from the first
> >proposal [4]:
> >
> >  - Do not revise the algorithm that computes the names of automatic
> >   modules.  The suggestion to use Maven coordinates when available [5]
> >   would help less than half of the most popular projects in use today,
> >   as determined by three different surveys.  It would result in module
> >  names that are often annoyingly verbose and known to be objectionable
> >    to some module authors, and it would raise non-trivial issues with
> >  regard to standardization and convention.  The names generated by the
> >   current algorithm do not follow the reverse-DNS convention, but they
> >    are at least easily derivable from the name of the original JAR
> >    files.  [6]
> >
> > - To increase awareness of when automatic modules are used, and of the
> >  consequences of their use, suggest that Java language compilers issue
> >    two new types of warnings, and implement these warnings in the RI:
> >
> >      - An opt-in (i.e., off-by-default) warning on a reference to
> >        an automatic module in a `requires` directive in a module
> >        declaration, and
> >
> >      - An opt-out (i.e., on-by-default) warning on a reference to
> >        an automatic module in a `requires transitive` directive in
> >        a module declaration.
> >
> >   The first warning allows developers to be maximally hygienic if they
> >    so choose.  The second helps make the author of an explicit module
> >    aware that they're putting the users of that module at risk by
> >    establishing implied readability to an automatic module.
> >
> >  - Encourage Java run-time systems to make it easy to identify via,
> >    e.g., command-line options, which observable modules are automatic,
> >    which observable modules would establish implied readability to
> >  automatic modules if resolved, and which automatic observable modules
> >    are actually resolved.  Implement these capabilities in the RI.
> >
> >Notes
> >-----
> >
> >This revised proposal is informed by a vigorous discussion on the
> >OpenJDK
> >jigsaw-dev mailing list [7] of the first proposal [4].  Two things
> >became
> >clear in that discussion, and in related discussions triggered by it.
> >
> >First, after twenty years the reverse-DNS naming convention is deeply
> >ingrained in the minds of most Java developers.  Stephen Colebourne
> >made
> >a clear and passionate case for it [8] and many developers voiced
> >support
> >for his argument, both on Twitter and elsewhere.  Some people may
> >prefer
> >shorter, project-oriented names, and those can be fine to use in
> >limited
> >projects that will never, ever see the light of day outside of a single
> >organization.  If you create a module that has even a small chance of
> >being open-sourced in the future, however, then the safest course is to
> >choose a reverse-DNS name for it at the start.
> >
> >Second, the fact that the name and effective API of an automatic module
> >are inherently unstable is less of a problem than I previously thought
> >[9][a].  Developers who use Maven and related tools, such as Gradle --
> >which is most developers -- are used to coping with breakage when they
> >upgrade the dependencies of a project to a newer version [b][c].  If
> >the
> >name or effective API of an automatic module change, either when its
> >maintainer adds an `Automatic-Module-Name` attribute or writes an
> >explicit `module-info.java` file, then the maintainers of projects that
> >use that module will just fix any problems that they encounter when
> >they
> >upgrade.
> >
> >After thinking about the first observation I'm now convinced that if
> >tools insert `Automatic-Module-Name` manifest attributes then they will
> >only do so when the module names that they define follow the
> >reverse-DNS
> >convention, since to do otherwise would go against what most developers
> >strongly prefer and expect.  After thinking about that and the second
> >observation I hereby retract [d] my previous advice [9] that automatic
> >modules should not be given stable names.
> >
> >                                  * * *
> >
> >In related threads on jigsaw-dev both Stephen Colebourne [e] and Robert
> >Scholte [f] suggest a new kind of explicit module, variously called
> >"partial" or "incomplete" or "soft", which can read all unnamed modules
> >so that dependencies that have not yet been modularized can be left on
> >the class path.  This is an attractive notion on the surface, and in
> >fact
> >one that we partly implemented in the prototype code base over two
> >years
> >ago, using the term "loose".  We didn't pursue it then, however, and I
> >don't think it's worth pursuing now, for two reasons:
> >
> > - If "partial" (or whatever) modules were to replace automatic modules
> >    then application developers would have to wait until all of their
> >    dependencies have been at least "partially" modularized before they
> >    can even begin to experiment with modularizing their own code.
> >
> >  - We could solve that by keeping automatic modules and adding partial
> >    modules, but then we'd have a more complex module system and a more
> >    complex story to tell.  To consider just one possible question,
> >    should developers go ahead and use their dependencies as automatic
> >   modules, or wait for them to become named automatic modules, or wait
> >    for them to become partial modules?
> >
> >The `Automatic-Module-Name` manifest attribute is, from a technical
> >standpoint, aesthetically unpleasant since it defines a second way
> >to name modules.  Taken alone, however, it does enable uncoordinated
> >modularization using stable module names with little effort, and so
> >it appears to be the least-bad option on the table.
> >
> >
> >[1]
> >http://openjdk.java.net/projects/jigsaw/spec/issues/#AutomaticModuleNames
> >[2]
> >http://mail.openjdk.java.net/pipermail/jpms-spec-experts/
> 2017-February/000582.html
> >[3]
> >http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleNameInManifest
> >[4]
> >http://mail.openjdk.java.net/pipermail/jpms-spec-experts/
> 2017-April/000667.html
> >[5]
> >http://mail.openjdk.java.net/pipermail/jpms-spec-experts/
> 2017-January/000537.html
> >[6]
> >http://mail.openjdk.java.net/pipermail/jpms-spec-experts/
> 2017-April/000666.html
> >[7]
> >http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-
> April/thread.html#11955
> >[8] http://blog.joda.org/2017/04/java-se-9-jpms-module-naming.html
> >[9]
> >http://mail.openjdk.java.net/pipermail/jpms-spec-experts/
> 2017-April/000682.html
> >[a]
> >http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012304.html
> >[b]
> >http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012327.html
> >[c]
> >http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012335.html
> >[d]
> >http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012367.html
> >[e]
> >http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012370.html
> >[f]
> >http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012399.html
>
> --
> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>

From david.lloyd at redhat.com  Fri May  5 18:15:47 2017
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Fri, 5 May 2017 13:15:47 -0500
Subject: The critical missing pieces and a path forward
Message-ID: <3b52b3b6-d95d-dacf-06ce-8fceaa1a8cf5@redhat.com>

Fellow experts,

We and other EC members and community members have, on several 
occasions, communicated the various deficiencies we see in the 
specification.  This is an update to make sure that it is very clear 
which few technical, objective criteria are being missed by the current 
specification that actually pushes it over the line to unacceptability 
to Red Hat in its current form.  I can't speak for other EG or EC 
members, but I believe this accurately summarizes our position.

The first criterion is to allow cycles to exist among modules at run 
time.  Our experience with deploying graphs which include third-party 
modules leads me to believe that this is going to be real problem that 
people encounter, which will be surprising and (in some cases) very 
difficult to fix or work around.  On the other hand, it should be quite 
easy to allow this within the module resolver code, especially given the 
self-contained and all-at-once nature of its algorithm.

The second criterion is to re-evaluate (piecewise if necessary) the 
(still very small) module primitives patch.  This will be a huge boon to 
container developers and framework authors (yes, this includes us, as 
well as others).  This is a very small effort even if the entire patch 
is taken (which is something that, as I've said before, we're open to 
discussion on, and there is much we can discuss and compromise on, on a 
point-by-point basis, if only such discussion could be allowed).  This 
bridge may open the path to allow the *entire* Java ecosystem to begin 
to converge on Jigsaw in ways beyond the class path, and indeed might 
mean the difference between unification and fragmentation.  Mark himself 
has admitted that the specification isn't perfect, which (in my opinion) 
also strongly argues in favor of the sensible strategy of providing a 
hook for extensibility.  This concession could provide relief for _many_ 
of the other problems that we and others have identified.

The third criterion is to provide package namespace isolation among 
module path modules.  I maintain that the easiest way to do this is to 
isolate each module path module to its own class loader, which also 
should greatly ease the transition for existing Java code that uses the 
existing ClassLoader API.  It is clear to myself and others that package 
namespace isolation is a fundamental expectation of a module system; the 
few existing widely-deployed module runtimes have this behavior.  Our 
own experience tells me that this is going to be a real problem for many 
nontrivial applications.  Not having this type of isolation dramatically 
undermines the value of the system.

Our concerns with automatic modules are largely mitigated by the fact 
that nobody is *forced* to use them; while I and others continue to 
harbor opinions on them, my feeling is that if the current parties of 
the discussion can reach a consensus (and it seems close), then that is 
satisfactory from our perspective.  If the EG reaches a consensus in 
this regard then that would be ideal.

We also have a concern that the changes to reflection may still be too 
drastic for the community to cope with, and it's possible that this 
might still be a deal-breaker for other EC and EG members.  I think that 
it is a good idea to ensure that there is consensus among the rest of 
the EG before we move forward with this as-is.

That's it.  After very detailed consideration and discussion of all the 
problems we see in the specification - both within Red Hat and with 
other individuals outside of Red Hat - these are what we genuinely 
believe will hurt the community of Java users the most, and what we can 
_realistically_ solve in a short time frame by working together.  Most, 
if not all, of the numerous other issues that have been identified 
should be largely mitigable with the help of these three primary changes.

Mark, please don't consider this to be self-serving.  Don't assume that 
we or others could only possibly vote "no" out of self-interest, after 
we have capitulated on so many other things.

I hope that you can see that we aren't asking for something unreasonable 
or impossible.  We aren't looking for a "meta" module system.  There are 
many other things I and others wish could be better about this 
specification, but we're not trying to achieve perfection.  We're 
looking for practical solutions to a few critical problems that push 
this over the minimum line for us, and may also solve critical concerns 
shared by other EG and EC members.  This isn't the end of the road; 
let's actually discuss these three things and move forward.

-- 
- DML

From rfscholte at apache.org  Sun May  7 18:50:05 2017
From: rfscholte at apache.org (Robert Scholte)
Date: Sun, 07 May 2017 20:50:05 +0200
Subject: Proposal: #AutomaticModuleNames (revised)
In-Reply-To: <20170504173833.07D4CC3DAF@eggemoggin.niobe.net>
References: <20170504173833.07D4CC3DAF@eggemoggin.niobe.net>
Message-ID: <op.yzvs9r00kdkhrr@desktop-2khsk44>

With the support of the Automatic-Module-Name attribute in the MANIFEST  
file all library developers can help their users to provide the intended  
module name.
However, for library developers depending on jars that will never become a  
module it is very frustrating that they can never distribute their jar as  
a modular jar, unless they shade+relocate those classes or switch to other  
libraries. Especially for library developers it is a matter of "all or  
nothing", and the attribute will help with the active projects if they are  
willing to provide this attribute. But if you depend on just 1 automodule  
WITHOUT the attribute, you're forced (adviced) to not add a module  
descriptor. Those are the consequences of these decisions.

I still think that the loose/soft modules is a better fit for the  
community and maybe it is better to not support both automatic modules and  
loose/soft modules. Knowing that the decision to have automatic modules is  
not part of any discussion anymore, the Automatic-Module-Name attribute is  
a minimum requirement to help migrating.

I agree on the recommendation to use reverse DNS for modules names.

thanks,
Robert

From forax at univ-mlv.fr  Sun May  7 19:12:15 2017
From: forax at univ-mlv.fr (Remi Forax)
Date: Sun, 7 May 2017 21:12:15 +0200 (CEST)
Subject: Proposal: #AutomaticModuleNames (revised)
In-Reply-To: <op.yzvs9r00kdkhrr@desktop-2khsk44>
References: <20170504173833.07D4CC3DAF@eggemoggin.niobe.net>
 <op.yzvs9r00kdkhrr@desktop-2khsk44>
Message-ID: <2065727702.3526537.1494184335844.JavaMail.zimbra@u-pem.fr>

----- Mail original -----
> De: "Robert Scholte" <rfscholte at apache.org>
> ?: jpms-spec-experts at openjdk.java.net
> Envoy?: Dimanche 7 Mai 2017 20:50:05
> Objet: Re: Proposal: #AutomaticModuleNames (revised)

> With the support of the Automatic-Module-Name attribute in the MANIFEST
> file all library developers can help their users to provide the intended
> module name.
> However, for library developers depending on jars that will never become a
> module it is very frustrating that they can never distribute their jar as
> a modular jar, unless they shade+relocate those classes or switch to other
> libraries. Especially for library developers it is a matter of "all or
> nothing", and the attribute will help with the active projects if they are
> willing to provide this attribute. But if you depend on just 1 automodule
> WITHOUT the attribute, you're forced (adviced) to not add a module
> descriptor. Those are the consequences of these decisions.

you can rename the jar or use a better library (a maintained one !).

> 
> I still think that the loose/soft modules is a better fit for the
> community and maybe it is better to not support both automatic modules and
> loose/soft modules. Knowing that the decision to have automatic modules is
> not part of any discussion anymore, the Automatic-Module-Name attribute is
> a minimum requirement to help migrating.

no, because with loose modules, everyone will start to name its dependencies with the name he wants and it will rapidly become a mess,
imagine jboss and spring (or any other frameworks) using the same library with two different names. 

> 
> I agree on the recommendation to use reverse DNS for modules names.

so am i.

> 
> thanks,
> Robert

regards,
R?mi

From mark.reinhold at oracle.com  Sun May  7 22:52:46 2017
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Sun, 07 May 2017 15:52:46 -0700
Subject: Proposal: #AutomaticModuleNames (revised)
In-Reply-To: <op.yzvs9r00kdkhrr@desktop-2khsk44>
References: <20170504173833.07D4CC3DAF@eggemoggin.niobe.net>
 <op.yzvs9r00kdkhrr@desktop-2khsk44>
Message-ID: <20170507155246.738941635@eggemoggin.niobe.net>

2017/5/7 11:50:05 -0700, Robert Scholte <rfscholte at apache.org>:
> With the support of the Automatic-Module-Name attribute in the MANIFEST  
> file all library developers can help their users to provide the intended  
> module name.
> However, for library developers depending on jars that will never become a  
> module it is very frustrating that they can never distribute their jar as  
> a modular jar, unless they shade+relocate those classes or switch to other  
> libraries. Especially for library developers it is a matter of "all or  
> nothing", and the attribute will help with the active projects if they are  
> willing to provide this attribute. But if you depend on just 1 automodule  
> WITHOUT the attribute, you're forced (adviced) to not add a module  
> descriptor. Those are the consequences of these decisions.

Understood and agreed.

The advice to avoid depending upon automatic modules in a published
explicit module should perhaps have an exception to say that it's okay
for such an explicit module to depend upon an abandoned library as an
automatic module.

> I still think that the loose/soft modules is a better fit for the  
> community and maybe it is better to not support both automatic modules and  
> loose/soft modules.

I tried to explain my view on loose/soft/partial modules in the revised
proposal.  I elaborated further over on the jigsaw-dev list, so for the
EG record I'll repeat those thoughts here:

It's true that application developers could get started by writing their
own partial modules, but I don't think that's the best way to encourage
everyone to approach modularization.

A useful aspect of the present design is that an explicit module is a
complete module.  If you've written a `module-info.java` file for your
component then you're done.  If explicit modules can be either partial
or complete then I worry that many people will partially modularize a
component but then never come back to completely modularize it.

I also think it's essential for a developer new to all of this to be
able to experiment immediately with both strong encapsulation
(`exports`) and reliable configuration (`requires`).  A story that says
"write a partial module now, come back and make it complete later on
after all of its dependencies have been (perhaps partially) modularized"
is a story in which you won't be able to write and debug your `requires`
directives until all of your dependencies have been (at least partially)
modularized.  That could be a pretty long story.

>                     Knowing that the decision to have automatic modules is  
> not part of any discussion anymore, the Automatic-Module-Name attribute is  
> a minimum requirement to help migrating.
> 
> I agree on the recommendation to use reverse DNS for modules names.

Given that you appear satisfied with this proposal, I'll mark it as
closed in the issue list.  Thanks for all your help with this topic.

- Mark


[1] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-May/000687.html

From mark.reinhold at oracle.com  Sun May  7 23:22:41 2017
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Sun, 07 May 2017 16:22:41 -0700
Subject: Closing out three open issues: #AutomaticModuleNames,
 #ModuleNameInManifest, and #ModuleNameSyntax
Message-ID: <20170507162241.666570053@eggemoggin.niobe.net>

We've finally resolved #AutomaticModuleNames and two closely-related
issues, #ModuleNameInManifest and #ModuleNameSyntax, so I've marked
them as such in the issue list:

  http://openjdk.java.net/projects/jigsaw/spec/issues/

- Mark

From mark.reinhold at oracle.com  Mon May  8 14:57:36 2017
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Mon,  8 May 2017 07:57:36 -0700 (PDT)
Subject: The critical missing pieces and a path forward
In-Reply-To: <3b52b3b6-d95d-dacf-06ce-8fceaa1a8cf5@redhat.com>
References: <3b52b3b6-d95d-dacf-06ce-8fceaa1a8cf5@redhat.com>
Message-ID: <20170508145736.531AFC4422@eggemoggin.niobe.net>

2017/5/5 11:15:47 -0700, david.lloyd at redhat.com:
> Fellow experts,
> 
> We and other EC members and community members have, on several 
> occasions, communicated the various deficiencies we see in the 
> specification.  This is an update to make sure that it is very clear 
> which few technical, objective criteria are being missed by the current 
> specification that actually pushes it over the line to unacceptability 
> to Red Hat in its current form.  I can't speak for other EG or EC 
> members, but I believe this accurately summarizes our position.

Thank you for the summary.  We've discussed all of these issues before.
Since then I've seen no new information that persuades me to change my
views on them, but I'll summarize those views here for the sake of the
public record.

First, however, a broader point.

Part of our collective role here is to act as responsible stewards of
the Java Platform.  If we are to take that at all seriously then we must
think as carefully as we can about every change or addition that we make.
Every change to existing functionality carries compatibility risk.  Every
addition places hard constraints upon how the platform can evolve in the
future.  That a change or addition appears useful to some is just the
start of what must be a thorough analysis of its wider and long-term
consequences.  That a change or addition is "quite easy" to implement
is completely irrelevant.

History bears witness to this.  In the early years of Java we sometimes
chose to expose low-level functionality, or violate fundamental language
invariants, or leave APIs open for extension without having designed them
for that purpose.  In each case it was convenient and expedient and, yes,
"quite easy", but twenty years later we're still paying for many of those
decisions.

To cite just one example, consider the fact that `final` does not really
mean "final", which I've mentioned before [1].  The core reflection API
introduced in Java 1.1 brought the ability to change the value of `final`
fields.  This seemed useful, and implementing it was "quite easy" at the
time, and it did not have any obvious impact on the rest of the system.
In the long run, however, it has greatly complicated if not prevented JIT
compiler optimizations based upon or enabled by constant folding -- which
is an awful lot of them.

In the context of a library or an application the consequences of any
particular design choice are often apparent, and it's often possible to
correct a bad choice over time.  We're working here, however, on the Java
Platform itself.  Millions of developers and users and customers depend
upon it, every day.  In that context the consequences of any particular
design choice are often difficult to predict -- and bad choices are
sometimes impossible to correct.

If responsible stewardship means anything then it means that we must
reject any proposal that merely appears useful to some and is "quite
easy" to implement.

To act otherwise would be the height of irresponsibility.

Now, on to your criteria.

> The first criterion is to allow cycles to exist among modules at run 
> time.  Our experience with deploying graphs which include third-party 
> modules leads me to believe that this is going to be real problem that 
> people encounter, which will be surprising and (in some cases) very 
> difficult to fix or work around.

>From previous conversations my understanding is that your experience is
based upon deploying graphs of existing unchanged non-modular JAR files,
with all of their messy flaws, in a Java EE application server.  From a
purely technical perspective, it's impressive that you can do that.  If
we were aiming in this JSR only to describe and standardize that practice
then I would agree that cycles should be allowed.

A primary goal of this JSR is, however, to create a module system that's
approachable by all developers -- not just developers of application
servers -- for use in the construction, maintenance, and deployment of
libraries and large applications.  This demands that we think more
broadly about how to help all developers write programs "in the large".
One result of that thinking is that this module system imposes some
constraints that are intended to make working systems easier to reason
about, and to guide developers toward better practices.  Adapting to
these constraints will sometimes require changes to existing code, but
we've assumed from the start that modularization is an active process
undertaken by the maintainer of a component rather than something done
after-the-fact by someone else who cannot modify the component.

Forbidding cyclic module relationships is one of these new constraints.
That such relationships are generally a bad idea is not just my personal
view; see, e.g., Kirk Knoernschild's compelling arguments against them
in his fine book, _Java Application Architecture_ [2], specifically in
??4.4, "Cyclic Dependencies -- The Death Knell".

As I wrote previously [3], if real experience with this module system as
used directly by developers reveals that cycles should be allowed then in
a future release we can revise the specification to allow them.  If we do
so now, however, then we can never go back on that choice.

(For those not familiar with the details of the design: Cycles are not
 allowed when modules are resolved but it is possible to create them
 post-resolution via the API, if needed [4][5].  Cycles are also set up
 amongst all automatic modules, after resolution, to make it easier to
 treat existing JAR files as modules without change [6].)

>                                   On the other hand, it should be quite 
> easy to allow this within the module resolver code, especially given the 
> self-contained and all-at-once nature of its algorithm.

The ease with which a change can be implemented has no relevance at all
to whether it is the right change for the Java Platform for all time.
(See above.)

> The second criterion is to re-evaluate (piecewise if necessary) the 
> (still very small) module primitives patch.  This will be a huge boon to 
> container developers and framework authors (yes, this includes us, as 
> well as others).  This is a very small effort even if the entire patch 
> is taken

The ease with which a change can be implemented has no relevance at all
to whether it is the right change for the Java Platform for all time.
(See above.)

>          (which is something that, as I've said before, we're open to 
> discussion on, and there is much we can discuss and compromise on, on a 
> point-by-point basis, if only such discussion could be allowed).  This 
> bridge may open the path to allow the *entire* Java ecosystem to begin 
> to converge on Jigsaw in ways beyond the class path, and indeed might 
> mean the difference between unification and fragmentation.  Mark himself 
> has admitted that the specification isn't perfect, which (in my opinion) 
> also strongly argues in favor of the sensible strategy of providing a 
> hook for extensibility.  This concession could provide relief for _many_ 
> of the other problems that we and others have identified.

As I wrote earlier [7], the primitives that you propose to expose were
not designed for use outside of the reference implementation itself.  To
expose them would constrain the future evolution of this module system,
and the rest of the platform, with a commitment to support them for the
long term.  They were not designed for that purpose.

To design a proper set of such primitives, i.e., a "meta" module system,
is not a goal of this JSR.  If that had been a goal then we would have
started by studying the internals of many existing module systems in
order to identify a common set of primitives to which we would be willing
to commit for the long term.  That's not what we've done, however, and
without having done that I can't be confident that what we have today is
the right set of primitives, or even a good one.  To commit to this set
of primitives now would be irresponsible.

> The third criterion is to provide package namespace isolation among 
> module path modules.  I maintain that the easiest way to do this is to 
> isolate each module path module to its own class loader, which also 
> should greatly ease the transition for existing Java code that uses the 
> existing ClassLoader API.  It is clear to myself and others that package 
> namespace isolation is a fundamental expectation of a module system; the 
> few existing widely-deployed module runtimes have this behavior.  Our 
> own experience tells me that this is going to be a real problem for many 
> nontrivial applications.  Not having this type of isolation dramatically 
> undermines the value of the system.

I understand the concern here, but the impact of not isolating package
namespaces by default is ameliorated by the fact that most developers use
the broadly-accepted reverse-DNS convention for package names.  If an
application works properly on the class path today then it likely does
not have conflicting packages to start with, since conflicting packages
on the class path so often lead to trouble.

This module system loads application modules into a single class loader
by default so that developers have one less behavioral difference to
worry about as they upgrade their components to explicit modules.  An
early prototype did load every module into its own loader by default.
With that we found that a fair bit of existing code -- not just code in
the JDK itself -- is written assuming that the grandparent of its loader
is the boot loader, that its loader (or perhaps its loader's parent) is
the thread-context loader, that code in nearby modules is in the same
loader, and so on.  In a future release we can explore loading every
application module into its own loader, and maybe even make that the
default behavior over time, but for now a more conservative approach is
preferable.

(If do you need to load a set of modules with each module in its own
 class loader then you can arrange for that in just a few lines of code
 via the API [8].)

- Mark


[1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/011983.html
[2] http://www.kirkk.com/modularity/
[3] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000646.html
[4] http://cr.openjdk.java.net/~mr/jigsaw/spec/api/java/lang/Module.html#addReads-java.lang.Module-
[5] http://cr.openjdk.java.net/~mr/jigsaw/spec/api/java/lang/ModuleLayer.Controller.html#addReads-java.lang.Module-java.lang.Module-
[6] http://openjdk.java.net/projects/jigsaw/spec/sotms/#automatic-modules
[7] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-February/000585.html
[8] http://cr.openjdk.java.net/~mr/jigsaw/spec/api/java/lang/ModuleLayer.html#defineModulesWithManyLoaders-java.lang.module.Configuration-java.lang.ClassLoader-

From david.lloyd at redhat.com  Mon May  8 16:38:07 2017
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Mon, 8 May 2017 11:38:07 -0500
Subject: The critical missing pieces and a path forward
In-Reply-To: <20170508145736.531AFC4422@eggemoggin.niobe.net>
References: <3b52b3b6-d95d-dacf-06ce-8fceaa1a8cf5@redhat.com>
 <20170508145736.531AFC4422@eggemoggin.niobe.net>
Message-ID: <46dfdb09-07bd-d1f2-6f0c-0a793c89540b@redhat.com>

On 05/08/2017 09:57 AM, mark.reinhold at oracle.com wrote:
 > [...]
> This module system loads application modules into a single class loader
> by default so that developers have one less behavioral difference to
> worry about as they upgrade their components to explicit modules.  An
> early prototype did load every module into its own loader by default.
> With that we found that a fair bit of existing code -- not just code in
> the JDK itself -- is written assuming that the grandparent of its loader
> is the boot loader, that its loader (or perhaps its loader's parent) is
> the thread-context loader, that code in nearby modules is in the same
> loader, and so on.

In addition to being an unusual time to suddenly start worrying about 
compatibility, this is very surprising to me as we have not seen this, 
including at a time when our main middleware product completely dropped 
hierarchical class loading back a number of years ago during Java 6 days 
(which is when I would have expected to see these problems crop up).  Do 
you have any public examples you can cite?

-- 
- DML

From forax at univ-mlv.fr  Mon May  8 17:40:45 2017
From: forax at univ-mlv.fr (Remi Forax)
Date: Mon, 8 May 2017 19:40:45 +0200 (CEST)
Subject: The critical missing pieces and a path forward
In-Reply-To: <20170508145736.531AFC4422@eggemoggin.niobe.net>
References: <3b52b3b6-d95d-dacf-06ce-8fceaa1a8cf5@redhat.com>
 <20170508145736.531AFC4422@eggemoggin.niobe.net>
Message-ID: <687100980.366344.1494265245695.JavaMail.zimbra@u-pem.fr>


----- Mail original -----
> De: "mark reinhold" <mark.reinhold at oracle.com>
> ?: "David M. Lloyd" <david.lloyd at redhat.com>
> Cc: jpms-spec-experts at openjdk.java.net
> Envoy?: Lundi 8 Mai 2017 16:57:36
> Objet: Re: The critical missing pieces and a path forward

[...]

> 
> This module system loads application modules into a single class loader
> by default so that developers have one less behavioral difference to
> worry about as they upgrade their components to explicit modules.  An
> early prototype did load every module into its own loader by default.
> With that we found that a fair bit of existing code -- not just code in
> the JDK itself -- is written assuming that the grandparent of its loader
> is the boot loader, that its loader (or perhaps its loader's parent) is
> the thread-context loader, that code in nearby modules is in the same
> loader, and so on.  In a future release we can explore loading every
> application module into its own loader, and maybe even make that the
> default behavior over time, but for now a more conservative approach is
> preferable.
> 
> (If do you need to load a set of modules with each module in its own
> class loader then you can arrange for that in just a few lines of code
> via the API [8].)

I disagree, respectfully, but you shall not touch my classloaders ...
ClassLoaders are a user-defined construct, not something the platform should touch.

The VM can be changed to do not use a dictionnary by classloader but uses a data structure that map a class name to a Class depending on the current resolved module,
i.e. instead of ClassLoader -> String -> Class, ClassLoader -> Module -> String -> Class.
It's not simple and calls like Unsafe.defineClass will not work anymore (or becomes @CallerSensitive) because they lack the current module context.  

> 
> - Mark
> 
> 

R?mi

> http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-February/000585.html
> [8]
> http://cr.openjdk.java.net/~mr/jigsaw/spec/api/java/lang/ModuleLayer.html#defineModulesWithManyLoaders-java.lang.module.Configuration-java.lang.ClassLoader-

From forax at univ-mlv.fr  Mon May  8 17:41:26 2017
From: forax at univ-mlv.fr (Remi Forax)
Date: Mon, 8 May 2017 19:41:26 +0200 (CEST)
Subject: The critical missing pieces and a path forward
In-Reply-To: <46dfdb09-07bd-d1f2-6f0c-0a793c89540b@redhat.com>
References: <3b52b3b6-d95d-dacf-06ce-8fceaa1a8cf5@redhat.com>
 <20170508145736.531AFC4422@eggemoggin.niobe.net>
 <46dfdb09-07bd-d1f2-6f0c-0a793c89540b@redhat.com>
Message-ID: <1258803247.366550.1494265286269.JavaMail.zimbra@u-pem.fr>

David,
people are very creative with classloaders, do not using the delegation scheme as recommended is one example, that the reason why the module system can not use classloaders for itself, it will broke all applications created by creative people we're talking about.

As Mark said, there was a prototype of jigsaw that tries try to use classloaders for defining modules, i've tried to port one of my application on that prototype, it was an utter failure. I do not even imagine OSGI or JBoss Module working with that setup.

R?mi

----- Mail original -----
> De: "David M. Lloyd" <david.lloyd at redhat.com>
> ?: "mark reinhold" <mark.reinhold at oracle.com>
> Cc: jpms-spec-experts at openjdk.java.net
> Envoy?: Lundi 8 Mai 2017 18:38:07
> Objet: Re: The critical missing pieces and a path forward

> On 05/08/2017 09:57 AM, mark.reinhold at oracle.com wrote:
> > [...]
>> This module system loads application modules into a single class loader
>> by default so that developers have one less behavioral difference to
>> worry about as they upgrade their components to explicit modules.  An
>> early prototype did load every module into its own loader by default.
>> With that we found that a fair bit of existing code -- not just code in
>> the JDK itself -- is written assuming that the grandparent of its loader
>> is the boot loader, that its loader (or perhaps its loader's parent) is
>> the thread-context loader, that code in nearby modules is in the same
>> loader, and so on.
> 
> In addition to being an unusual time to suddenly start worrying about
> compatibility, this is very surprising to me as we have not seen this,
> including at a time when our main middleware product completely dropped
> hierarchical class loading back a number of years ago during Java 6 days
> (which is when I would have expected to see these problems crop up).  Do
> you have any public examples you can cite?
> 
> --
> - DML

From mark.reinhold at oracle.com  Mon May  8 19:25:05 2017
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Mon, 08 May 2017 12:25:05 -0700
Subject: The critical missing pieces and a path forward
In-Reply-To: <687100980.366344.1494265245695.JavaMail.zimbra@u-pem.fr>
References: <3b52b3b6-d95d-dacf-06ce-8fceaa1a8cf5@redhat.com>
 <20170508145736.531AFC4422@eggemoggin.niobe.net>
 <687100980.366344.1494265245695.JavaMail.zimbra@u-pem.fr>
Message-ID: <20170508122505.995529696@eggemoggin.niobe.net>

2017/5/8 10:40:45 -0700, Remi Forax <forax at univ-mlv.fr>:
> 2017/5/8 7:57:36 -0700, mark.reinhold at oracle.com:
>> ...
>> 
>> (If do you need to load a set of modules with each module in its own
>> class loader then you can arrange for that in just a few lines of code
>> via the API [8].)
> 
> I disagree, respectfully, but you shall not touch my classloaders ...
> ClassLoaders are a user-defined construct, not something the platform
> should touch.

I wouldn't dream of touching your class loaders!

I was just pointing out that there's a convenient API by which you can
arrange for per-module class loaders, in which case the loaders are
provided by the system.  If you want to provide your own loaders then
you can do that too, with a different API to which you pass a function
from module names to loaders [1].  It's up to you.

> The VM can be changed to do not use a dictionnary by classloader but
> uses a data structure that map a class name to a Class depending on
> the current resolved module, i.e. instead of ClassLoader -> String ->
> Class, ClassLoader -> Module -> String -> Class.  It's not simple and
> calls like Unsafe.defineClass will not work anymore (or becomes
> @CallerSensitive) because they lack the current module context.

I'm not sure why any of this would be necessray.

- Mark


[1] http://cr.openjdk.java.net/~mr/jigsaw/spec/api/java/lang/ModuleLayer.html#defineModules-java.lang.module.Configuration-java.util.function.Function-

From forax at univ-mlv.fr  Mon May  8 21:39:31 2017
From: forax at univ-mlv.fr (forax at univ-mlv.fr)
Date: Mon, 8 May 2017 23:39:31 +0200 (CEST)
Subject: The critical missing pieces and a path forward
In-Reply-To: <20170508122505.995529696@eggemoggin.niobe.net>
References: <3b52b3b6-d95d-dacf-06ce-8fceaa1a8cf5@redhat.com>
 <20170508145736.531AFC4422@eggemoggin.niobe.net>
 <687100980.366344.1494265245695.JavaMail.zimbra@u-pem.fr>
 <20170508122505.995529696@eggemoggin.niobe.net>
Message-ID: <960407572.383882.1494279571957.JavaMail.zimbra@u-pem.fr>

----- Mail original -----
> De: "mark reinhold" <mark.reinhold at oracle.com>
> ?: "Remi Forax" <forax at univ-mlv.fr>
> Cc: jpms-spec-experts at openjdk.java.net
> Envoy?: Lundi 8 Mai 2017 21:25:05
> Objet: Re: The critical missing pieces and a path forward

> 2017/5/8 10:40:45 -0700, Remi Forax <forax at univ-mlv.fr>:
>> 2017/5/8 7:57:36 -0700, mark.reinhold at oracle.com:
>>> ...
>>> 
>>> (If do you need to load a set of modules with each module in its own
>>> class loader then you can arrange for that in just a few lines of code
>>> via the API [8].)
>> 
>> I disagree, respectfully, but you shall not touch my classloaders ...
>> ClassLoaders are a user-defined construct, not something the platform
>> should touch.
> 
> I wouldn't dream of touching your class loaders!
> 
> I was just pointing out that there's a convenient API by which you can
> arrange for per-module class loaders, in which case the loaders are
> provided by the system.  If you want to provide your own loaders then
> you can do that too, with a different API to which you pass a function
> from module names to loaders [1].  It's up to you.
> 
>> The VM can be changed to do not use a dictionnary by classloader but
>> uses a data structure that map a class name to a Class depending on
>> the current resolved module, i.e. instead of ClassLoader -> String ->
>> Class, ClassLoader -> Module -> String -> Class.  It's not simple and
>> calls like Unsafe.defineClass will not work anymore (or becomes
>> @CallerSensitive) because they lack the current module context.
> 
> I'm not sure why any of this would be necessary.

allow non exported packages with the same name in different modules, allow several modules with different versions if not in the same transitive dependency set.  

> 
> - Mark
>

R?mi

From david.lloyd at redhat.com  Tue May  9 12:49:14 2017
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Tue, 9 May 2017 07:49:14 -0500
Subject: The critical missing pieces and a path forward
In-Reply-To: <20170508145736.531AFC4422@eggemoggin.niobe.net>
References: <3b52b3b6-d95d-dacf-06ce-8fceaa1a8cf5@redhat.com>
 <20170508145736.531AFC4422@eggemoggin.niobe.net>
Message-ID: <530d03fa-cf66-896e-479e-90c18cf317bf@redhat.com>

On 05/08/2017 09:57 AM, mark.reinhold at oracle.com wrote:
> This module system loads application modules into a single class loader
> by default so that developers have one less behavioral difference to
> worry about as they upgrade their components to explicit modules.  An
> early prototype did load every module into its own loader by default.

What about a command line option to either opt in to or opt out of 
having separated class loaders?  The class loader of each module in this 
case should be able to resolve only the same set of types and resources 
that its corresponding module does.  This way, users can use the module 
path without custom code, while still being able to take advantage of 
more advanced use cases.

Ideally jlink would also support such an option.

-- 
- DML

From david.lloyd at redhat.com  Tue May  9 13:19:20 2017
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Tue, 9 May 2017 08:19:20 -0500
Subject: The critical missing pieces and a path forward
In-Reply-To: <20170508145736.531AFC4422@eggemoggin.niobe.net>
References: <3b52b3b6-d95d-dacf-06ce-8fceaa1a8cf5@redhat.com>
 <20170508145736.531AFC4422@eggemoggin.niobe.net>
Message-ID: <d08d3738-410a-52e0-c769-eb2b26357b4f@redhat.com>

On 05/08/2017 09:57 AM, mark.reinhold at oracle.com wrote:
> 2017/5/5 11:15:47 -0700, david.lloyd at redhat.com:
>> Fellow experts,
>>
>> We and other EC members and community members have, on several
>> occasions, communicated the various deficiencies we see in the
>> specification.  This is an update to make sure that it is very clear
>> which few technical, objective criteria are being missed by the current
>> specification that actually pushes it over the line to unacceptability
>> to Red Hat in its current form.  I can't speak for other EG or EC
>> members, but I believe this accurately summarizes our position.
> 
> Thank you for the summary.  We've discussed all of these issues before.
> Since then I've seen no new information that persuades me to change my
> views on them, but I'll summarize those views here for the sake of the
> public record.
> 
> First, however, a broader point.
> 
> Part of our collective role here is to act as responsible stewards of
> the Java Platform.  If we are to take that at all seriously then we must
> think as carefully as we can about every change or addition that we make.
> Every change to existing functionality carries compatibility risk.  Every
> addition places hard constraints upon how the platform can evolve in the
> future.  That a change or addition appears useful to some is just the
> start of what must be a thorough analysis of its wider and long-term
> consequences.  That a change or addition is "quite easy" to implement
> is completely irrelevant.
> 
> History bears witness to this.  In the early years of Java we sometimes
> chose to expose low-level functionality, or violate fundamental language
> invariants, or leave APIs open for extension without having designed them
> for that purpose.  In each case it was convenient and expedient and, yes,
> "quite easy", but twenty years later we're still paying for many of those
> decisions.
> 
> To cite just one example, consider the fact that `final` does not really
> mean "final", which I've mentioned before [1].  The core reflection API
> introduced in Java 1.1 brought the ability to change the value of `final`
> fields.  This seemed useful, and implementing it was "quite easy" at the
> time, and it did not have any obvious impact on the rest of the system.
> In the long run, however, it has greatly complicated if not prevented JIT
> compiler optimizations based upon or enabled by constant folding -- which
> is an awful lot of them.
> 
> In the context of a library or an application the consequences of any
> particular design choice are often apparent, and it's often possible to
> correct a bad choice over time.  We're working here, however, on the Java
> Platform itself.  Millions of developers and users and customers depend
> upon it, every day.  In that context the consequences of any particular
> design choice are often difficult to predict -- and bad choices are
> sometimes impossible to correct.
> 
> If responsible stewardship means anything then it means that we must
> reject any proposal that merely appears useful to some and is "quite
> easy" to implement.
> 
> To act otherwise would be the height of irresponsibility.
> 
> Now, on to your criteria.
> 
>> The first criterion is to allow cycles to exist among modules at run
>> time.  Our experience with deploying graphs which include third-party
>> modules leads me to believe that this is going to be real problem that
>> people encounter, which will be surprising and (in some cases) very
>> difficult to fix or work around.
> 
>  From previous conversations my understanding is that your experience is
> based upon deploying graphs of existing unchanged non-modular JAR files,
> with all of their messy flaws, in a Java EE application server.  From a
> purely technical perspective, it's impressive that you can do that.  If
> we were aiming in this JSR only to describe and standardize that practice
> then I would agree that cycles should be allowed.
> 
> A primary goal of this JSR is, however, to create a module system that's
> approachable by all developers -- not just developers of application
> servers -- for use in the construction, maintenance, and deployment of
> libraries and large applications.  This demands that we think more
> broadly about how to help all developers write programs "in the large".
> One result of that thinking is that this module system imposes some
> constraints that are intended to make working systems easier to reason
> about, and to guide developers toward better practices.  Adapting to
> these constraints will sometimes require changes to existing code, but
> we've assumed from the start that modularization is an active process
> undertaken by the maintainer of a component rather than something done
> after-the-fact by someone else who cannot modify the component.
> 
> Forbidding cyclic module relationships is one of these new constraints.
> That such relationships are generally a bad idea is not just my personal
> view; see, e.g., Kirk Knoernschild's compelling arguments against them
> in his fine book, _Java Application Architecture_ [2], specifically in
> ?4.4, "Cyclic Dependencies -- The Death Knell".

I think this argument is deliberately missing the point.  Today, it is 
relatively difficult to successfully build a piece of Java code which 
has cyclic build dependencies.  But when you're talking about a 
potentially large graph, a cyclic dependency is not always a design 
choice: sometimes, it's an accident of assembly.  A harmless accident, 
which should not be punished.

> As I wrote previously [3], if real experience with this module system as
> used directly by developers reveals that cycles should be allowed then in
> a future release we can revise the specification to allow them.  If we do
> so now, however, then we can never go back on that choice.

Conceptually speaking, why would we ever need back out on the choice? 
The only possible presentation of this problem is that all module builds 
had succeeded, but a run-time cycle is unexpectedly introduced when the 
final application is assembled.  There is no design principle being 
violated in this case; it's a far more obscure happening.  To enforce at 
this level doesn't really help users in any way.  If they hit this error 
at run time - maybe even during final or near-final testing - now our 
users must potentially go back to the design stage to completely 
refactor one or more modules.

Think of, say, an enterprise environment where different components are 
developed by different teams, and they later decide over time that each 
must use APIs from one another, for various reasons.  They should not 
have to fight over which group must refactor their module to split API 
and implementation.  Sometimes, particularly in an enterprise 
environment, such a split just doesn't make any sense.

If a module has no cycles at build, the run time should always accept it 
without error.  I think this makes far more sense as an ideology than 
the idea that modules must have no cycles at build time (where design 
has an impact) and also at run time (where design takes a back seat to 
assembly and deployment).

I think we have concrete reasons to allow cycles, and abstract reasons 
to disallow them.  And maybe I'm wrong and this kind of thing just won't 
happen with Java 9 modules; but if it does, it could potentially cause 
an unreasonably disproportionate amount of pain in comparison to the 
(AFAICT zero) benefit of enforcing what can only be described as an 
arbitrary restriction.

-- 
- DML

From misterm at gmail.com  Tue May  9 13:34:19 2017
From: misterm at gmail.com (Michael Nascimento)
Date: Tue, 9 May 2017 10:34:19 -0300
Subject: The critical missing pieces and a path forward
In-Reply-To: <d08d3738-410a-52e0-c769-eb2b26357b4f@redhat.com>
References: <3b52b3b6-d95d-dacf-06ce-8fceaa1a8cf5@redhat.com>
 <20170508145736.531AFC4422@eggemoggin.niobe.net>
 <d08d3738-410a-52e0-c769-eb2b26357b4f@redhat.com>
Message-ID: <CAAWA2oWfKj=enbyHV=_cqfZbMQPQVwk2O0h_og_-ZoSCwEC2Vg@mail.gmail.com>

On Tue, May 9, 2017 at 10:19 AM, David M. Lloyd <david.lloyd at redhat.com>
wrote:

> Think of, say, an enterprise environment where different components are
> developed by different teams, and they later decide over time that each
> must use APIs from one another, for various reasons.  They should not have
> to fight over which group must refactor their module to split API and
> implementation.  Sometimes, particularly in an enterprise environment, such
> a split just doesn't make any sense.
>

Or such a split is not realistically possible in such an environment due to
corporate politics. As ridiculous as it sounds, multiple versions and
cycles must be allowed because when you have multiple open-source projects
and multiple groups/divisions in an company working to assemble one
application, while it would be theoretically possible to polish modules and
dependencies to abide to JPMS current rules, it is not politically possible
to do so. And as painful as it sounds, these corporate users is what makes
the Java ecosystem financially valuable.

So, please, don't disregard these points. If these big corporate users
cannot adopt JPMS, at some point they will adopt a different language that
works for them without so many rules.

Regards,
Michael

From Tim_Ellison at uk.ibm.com  Wed May 10 19:43:30 2017
From: Tim_Ellison at uk.ibm.com (Tim Ellison)
Date: Wed, 10 May 2017 19:43:30 +0000
Subject: Meeting to reach a conclusion (resend)
Message-ID: <OF82FCC473.234FA4B7-ON0025811C.006C5A69-1494445410725@notes.na.collabserv.com>

I don't see my first attempt to send this email in the list archive, so re-sending...


   me --- Meeting to reach a conclusion --- 
    From:"Tim Ellison" <Tim_Ellison at uk.ibm.com>To:jpms-spec-experts at openjdk.java.netDate:Wed, 10 May 2017 15:54Subject:Meeting to reach a conclusion
  
   
     I'm mindful that the clock is now running under the JCP process [1] for us to submit a revised specification for reconsideration.
   
      
   
     Mark has said there is no business pressure to hit the JDK9 schedule dates, however, we should aim to square away remaining issues as quickly as possible and submit a reconsideration.  We don't have to use the full 30-days afforded, and should work to ensure the spec delivery stays on track.
   
      
   
     I think it would be useful to increase the bandwidth of EG communication during this period.  The mailing list conversation to date has been "bursty", and at times open to misinterpretation.  So I propose we try and find some time within the next two weeks where the EG can gather around a whiteboard and talk.  Ideally this would be face-to-face if we can find a suitable time and place, because I think we'd make much more progress in one or two days of focused conversation.  If not a physical meeting we can fall back to conference call, video call, etc.
   
      
   
     Of course, all decisions and conclusions in such a meeting would be brought back to this mailing list for openness and posterity, with a suitable period for observers to comment before re-submission.
   
      
   
     Are people open to such a suggestion?
   
      
   
     [1] https://www.jcp.org/en/procedures/jcp2#3.4.5
   
      
   
     Regards,
   
     TimUnless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


From david.lloyd at redhat.com  Wed May 10 20:04:35 2017
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Wed, 10 May 2017 15:04:35 -0500
Subject: Meeting to reach a conclusion (resend)
In-Reply-To: <OF82FCC473.234FA4B7-ON0025811C.006C5A69-1494445410725@notes.na.collabserv.com>
References: <OF82FCC473.234FA4B7-ON0025811C.006C5A69-1494445410725@notes.na.collabserv.com>
Message-ID: <16d9e5fe-ffc7-39d1-0959-b662d2602c33@redhat.com>

I'm open to this idea.  I would like to continue to publish proposals on 
this list until such a meeting can take place though.  It might serve as 
a good foundation for discussion at the very least.

Some of the EG might be at DevoxxUK tomorrow and/or Wednesday (I'm not, 
but other Red Hat folks are/will be).  That might be a good 
(preliminary?) opportunity to meet face to face, at least for some 
subset of the EG.

On 05/10/2017 02:43 PM, Tim Ellison wrote:
> I don't see my first attempt to send this email in the list archive, so re-sending...
> 
> 
>     me --- Meeting to reach a conclusion ---
>      From:"Tim Ellison" <Tim_Ellison at uk.ibm.com>To:jpms-spec-experts at openjdk.java.netDate:Wed, 10 May 2017 15:54Subject:Meeting to reach a conclusion
>    
>     
>       I'm mindful that the clock is now running under the JCP process [1] for us to submit a revised specification for reconsideration.
>     
>        
>     
>       Mark has said there is no business pressure to hit the JDK9 schedule dates, however, we should aim to square away remaining issues as quickly as possible and submit a reconsideration.  We don't have to use the full 30-days afforded, and should work to ensure the spec delivery stays on track.
>     
>        
>     
>       I think it would be useful to increase the bandwidth of EG communication during this period.  The mailing list conversation to date has been "bursty", and at times open to misinterpretation.  So I propose we try and find some time within the next two weeks where the EG can gather around a whiteboard and talk.  Ideally this would be face-to-face if we can find a suitable time and place, because I think we'd make much more progress in one or two days of focused conversation.  If not a physical meeting we can fall back to conference call, video call, etc.
>     
>        
>     
>       Of course, all decisions and conclusions in such a meeting would be brought back to this mailing list for openness and posterity, with a suitable period for observers to comment before re-submission.
>     
>        
>     
>       Are people open to such a suggestion?
>     
>        
>     
>       [1] https://www.jcp.org/en/procedures/jcp2#3.4.5
>     
>        
>     
>       Regards,
>     
>       TimUnless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
> 


-- 
- DML

From david.lloyd at redhat.com  Wed May 10 20:09:20 2017
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Wed, 10 May 2017 15:09:20 -0500
Subject: Layer Primitives (1/3): addExports()
Message-ID: <b1d9fdbf-1fc4-9b9e-3af2-cedcff1ca53c@redhat.com>

I'm now breaking up each layer primitive in order to try to get some 
kind of focused/non-blanket bidirectional discussion on each one, with a 
view towards forward progress.  I'm targeting what I view as a 
minimum/compromise set.

I'd like to re-propose adding the following method to the layer 
controller specification and implementation [1].

Justification:

? No new functionality is introduced to the specification or 
implementation by this method, other than to simplify access for layer 
controllers.
? Module#addExports() exists and does the exact same thing.
? It is not reasonable to require containers to generate bytecode within 
their contained modules in order to call this method.
? Since Module#addExports() exists, there is no scenario in which adding 
this primitive to the layer controller could possibly cause any form of 
liability or technical debt that does not already exist.
? ModuleLayer.Controller#addOpens() exists, which grants even wider 
access than addExports() would, thus there is no security risk that does 
not already exist in a greater form.
? The controller is only available to user-defined Layers, so there is 
no risk to the platform, jlink, etc.
? The patch itself is very small and simple, and should be very low-risk.

Drawbacks:

None that I am aware of.

Patch information:

The content and format of the JavaDoc are drawn from neighboring 
methods.  I relinquish any copyright claim to the patch in this mail. 
Please do not get bogged down by any formatting problems introduced by 
the mailing list; the purpose of directly including the patch is to give 
a clear, unambiguous subject for discussion.  I can provide a proper 
webrev (or whatever other form is requested) if needed.

[1]:

diff --git a/jdk/src/java.base/share/classes/java/lang/ModuleLayer.java 
b/jdk/src/java.base/share/classes/java/lang/ModuleLayer.java
index 549662f..20e3021 100644
--- a/jdk/src/java.base/share/classes/java/lang/ModuleLayer.java
+++ b/jdk/src/java.base/share/classes/java/lang/ModuleLayer.java
@@ -269,6 +269,34 @@ public final class ModuleLayer {
              source.implAddOpens(pn, target);
              return this;
          }
+
+        /**
+         * Updates module {@code source} in the layer to export a 
package to
+         * module {@code target}. This method is a no-op if {@code source}
+         * already exports the package to at least {@code target}.
+         *
+         * @param  source
+         *         The source module
+         * @param  pn
+         *         The package name
+         * @param  target
+         *         The target module to read
+         *
+         * @return This controller
+         *
+         * @throws IllegalArgumentException
+         *         If {@code source} is not in the layer or the package 
is not
+         *         in the source module
+         *
+         * @see Module#addExports
+         */
+        public Controller addExports(Module source, String pn, Module 
target) {
+            Objects.requireNonNull(source);
+            Objects.requireNonNull(target);
+            ensureInLayer(source);
+            source.implAddExports(pn, target);
+            return this;
+        }
      }



-- 
- DML

From david.lloyd at redhat.com  Wed May 10 20:31:46 2017
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Wed, 10 May 2017 15:31:46 -0500
Subject: Layer Primitives (1/3): addExports()
In-Reply-To: <b1d9fdbf-1fc4-9b9e-3af2-cedcff1ca53c@redhat.com>
References: <b1d9fdbf-1fc4-9b9e-3af2-cedcff1ca53c@redhat.com>
Message-ID: <d853eeb8-15d6-064b-a50a-bd402f507df8@redhat.com>

I apologize, the patch given corresponds to an older version.  The 
updated patch is as follows:

diff --git a/jdk/src/java.base/share/classes/java/lang/ModuleLayer.java 
b/jdk/src/java.base/share/classes/java/lang/ModuleLayer.java
index 549662f..58ffbd2 100644
--- a/jdk/src/java.base/share/classes/java/lang/ModuleLayer.java
+++ b/jdk/src/java.base/share/classes/java/lang/ModuleLayer.java
@@ -48,6 +48,7 @@ import jdk.internal.loader.ClassLoaderValue;
  import jdk.internal.loader.Loader;
  import jdk.internal.loader.LoaderPool;
  import jdk.internal.module.ServicesCatalog;
+import jdk.internal.reflect.Reflection;
  import sun.security.util.SecurityConstants;


@@ -269,6 +270,39 @@ public final class ModuleLayer {
              source.implAddOpens(pn, target);
              return this;
          }
+
+        /**
+         * Updates module {@code source} in the layer to export a 
package to
+         * module {@code target}. This method is a no-op if {@code source}
+         * already exports the package to at least {@code target}.
+         *
+         * @param  source
+         *         The source module
+         * @param  pn
+         *         The package name
+         * @param  other
+         *         The module to export to
+         *
+         * @return This controller
+         *
+         * @throws IllegalArgumentException
+         *         If {@code pn} is {@code null}, or {@code source} is 
not in the layer
+         *         or the package is not in the source module
+         *
+         * @see Module#addExports
+         */
+        public Controller addExports(Module source, String pn, Module 
other) {
+            if (pn == null)
+                throw new IllegalArgumentException("package is null");
+            Objects.requireNonNull(other);
+
+            if (source.isNamed()) {
+                ensureInLayer(source);
+                source.implAddExports(pn, other);
+            }
+
+            return this;
+        }
      }





On 05/10/2017 03:09 PM, David M. Lloyd wrote:
> I'm now breaking up each layer primitive in order to try to get some 
> kind of focused/non-blanket bidirectional discussion on each one, with a 
> view towards forward progress.  I'm targeting what I view as a 
> minimum/compromise set.
> 
> I'd like to re-propose adding the following method to the layer 
> controller specification and implementation [1].
> 
> Justification:
> 
> ? No new functionality is introduced to the specification or 
> implementation by this method, other than to simplify access for layer 
> controllers.
> ? Module#addExports() exists and does the exact same thing.
> ? It is not reasonable to require containers to generate bytecode within 
> their contained modules in order to call this method.
> ? Since Module#addExports() exists, there is no scenario in which adding 
> this primitive to the layer controller could possibly cause any form of 
> liability or technical debt that does not already exist.
> ? ModuleLayer.Controller#addOpens() exists, which grants even wider 
> access than addExports() would, thus there is no security risk that does 
> not already exist in a greater form.
> ? The controller is only available to user-defined Layers, so there is 
> no risk to the platform, jlink, etc.
> ? The patch itself is very small and simple, and should be very low-risk.
> 
> Drawbacks:
> 
> None that I am aware of.
> 
> Patch information:
> 
> The content and format of the JavaDoc are drawn from neighboring 
> methods.  I relinquish any copyright claim to the patch in this mail. 
> Please do not get bogged down by any formatting problems introduced by 
> the mailing list; the purpose of directly including the patch is to give 
> a clear, unambiguous subject for discussion.  I can provide a proper 
> webrev (or whatever other form is requested) if needed.
> 
> [1]:
> 
> diff --git a/jdk/src/java.base/share/classes/java/lang/ModuleLayer.java 
> b/jdk/src/java.base/share/classes/java/lang/ModuleLayer.java
> index 549662f..20e3021 100644
> --- a/jdk/src/java.base/share/classes/java/lang/ModuleLayer.java
> +++ b/jdk/src/java.base/share/classes/java/lang/ModuleLayer.java
> @@ -269,6 +269,34 @@ public final class ModuleLayer {
>               source.implAddOpens(pn, target);
>               return this;
>           }
> +
> +        /**
> +         * Updates module {@code source} in the layer to export a 
> package to
> +         * module {@code target}. This method is a no-op if {@code source}
> +         * already exports the package to at least {@code target}.
> +         *
> +         * @param  source
> +         *         The source module
> +         * @param  pn
> +         *         The package name
> +         * @param  target
> +         *         The target module to read
> +         *
> +         * @return This controller
> +         *
> +         * @throws IllegalArgumentException
> +         *         If {@code source} is not in the layer or the package 
> is not
> +         *         in the source module
> +         *
> +         * @see Module#addExports
> +         */
> +        public Controller addExports(Module source, String pn, Module 
> target) {
> +            Objects.requireNonNull(source);
> +            Objects.requireNonNull(target);
> +            ensureInLayer(source);
> +            source.implAddExports(pn, target);
> +            return this;
> +        }
>       }
> 
> 
> 


-- 
- DML

From david.lloyd at redhat.com  Wed May 10 20:48:08 2017
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Wed, 10 May 2017 15:48:08 -0500
Subject: Layer Primitives (2/3): addUses()
Message-ID: <a631e20d-2ab7-d841-7523-9db1ef34c8e5@redhat.com>

I'd like to re-propose adding the following method to the layer 
controller specification and implementation [1].

Justification:

? No new functionality is introduced to the specification or 
implementation by this method, other than to simplify access for layer 
controllers.
? Module#addUses() exists and does the exact same thing.
? It is not reasonable to require containers to generate bytecode within 
their contained modules in order to call this method.
? Since Module#addUses() exists, there is no scenario in which adding 
this primitive to the layer controller could possibly cause any form of 
liability or technical debt that does not already exist.
? The controller is only available to user-defined Layers, so there is 
no risk to the platform, jlink, etc.
? The patch itself is very small and simple, and should be very low-risk.

Drawbacks:

None that I am aware of.

Patch information:

The content and format of the JavaDoc are drawn from neighboring 
methods.  I relinquish any copyright claim to the patch in this mail. 
Please do not get bogged down by any formatting problems introduced by 
the mailing list; the purpose of directly including the patch is to give 
a clear, unambiguous subject for discussion.  I can provide a proper 
webrev (or whatever other form is requested) if needed.  This patch 
applies atop the previous (1/3) patch, but can trivially be applied 
without it as well.

[1]:

diff --git a/jdk/src/java.base/share/classes/java/lang/ModuleLayer.java 
b/jdk/src/java.base/share/classes/java/lang/ModuleLayer.java
index e224b4a..7f52b7c 100644
--- a/jdk/src/java.base/share/classes/java/lang/ModuleLayer.java
+++ b/jdk/src/java.base/share/classes/java/lang/ModuleLayer.java
@@ -303,6 +303,33 @@ public final class ModuleLayer {

              return this;
          }
+
+        /**
+         * Updates module {@code source} in the layer to add a
+         * service dependence on the given service type. This method is
+         * a no-op when invoked on an unnamed module or an automatic 
module.
+         *
+         * @param  service
+         *         The service type
+         *
+         * @return this module
+         *
+         * @throws IllegalStateException
+         *         If this is a named module and the caller is not this 
module
+         *
+         * @see Module#addUses(Class)
+         * @see ModuleDescriptor#uses()
+         */
+        public Controller addUses(Module source, Class<?> service) {
+            Objects.requireNonNull(service);
+
+            if (source.isNamed() && ! 
source.getDescriptor().isAutomatic()) {
+                ensureInLayer(source);
+                source.implAddUses(service);
+            }
+
+            return this;
+        }
      }


-- 
- DML

From martijnverburg at gmail.com  Wed May 10 21:31:49 2017
From: martijnverburg at gmail.com (Martijn Verburg)
Date: Wed, 10 May 2017 21:31:49 +0000
Subject: Meeting to reach a conclusion (resend)
In-Reply-To: <16d9e5fe-ffc7-39d1-0959-b662d2602c33@redhat.com>
References: <OF82FCC473.234FA4B7-ON0025811C.006C5A69-1494445410725@notes.na.collabserv.com>
 <16d9e5fe-ffc7-39d1-0959-b662d2602c33@redhat.com>
Message-ID: <CAP7YuARnBu-GYn1ANZYLay=jcWVLuQBMBAv=LipYAXEGtVsATg@mail.gmail.com>

I'm not an EG member - but happy to facilitate.  The LJC is also planning a
hack day this Saturday to try out some of the latest ideas.

Cheers
Martijn

On Wed, 10 May 2017 at 21:05, David M. Lloyd <david.lloyd at redhat.com> wrote:

> I'm open to this idea.  I would like to continue to publish proposals on
> this list until such a meeting can take place though.  It might serve as
> a good foundation for discussion at the very least.
>
> Some of the EG might be at DevoxxUK tomorrow and/or Wednesday (I'm not,
> but other Red Hat folks are/will be).  That might be a good
> (preliminary?) opportunity to meet face to face, at least for some
> subset of the EG.
>
> On 05/10/2017 02:43 PM, Tim Ellison wrote:
> > I don't see my first attempt to send this email in the list archive, so
> re-sending...
> >
> >
> >     me --- Meeting to reach a conclusion ---
> >      From:"Tim Ellison" <Tim_Ellison at uk.ibm.com
> >To:jpms-spec-experts at openjdk.java.netDate:Wed, 10 May 2017
> 15:54Subject:Meeting to reach a conclusion
> >
> >
> >       I'm mindful that the clock is now running under the JCP process
> [1] for us to submit a revised specification for reconsideration.
> >
> >
> >
> >       Mark has said there is no business pressure to hit the JDK9
> schedule dates, however, we should aim to square away remaining issues as
> quickly as possible and submit a reconsideration.  We don't have to use the
> full 30-days afforded, and should work to ensure the spec delivery stays on
> track.
> >
> >
> >
> >       I think it would be useful to increase the bandwidth of EG
> communication during this period.  The mailing list conversation to date
> has been "bursty", and at times open to misinterpretation.  So I propose we
> try and find some time within the next two weeks where the EG can gather
> around a whiteboard and talk.  Ideally this would be face-to-face if we can
> find a suitable time and place, because I think we'd make much more
> progress in one or two days of focused conversation.  If not a physical
> meeting we can fall back to conference call, video call, etc.
> >
> >
> >
> >       Of course, all decisions and conclusions in such a meeting would
> be brought back to this mailing list for openness and posterity, with a
> suitable period for observers to comment before re-submission.
> >
> >
> >
> >       Are people open to such a suggestion?
> >
> >
> >
> >       [1] https://www.jcp.org/en/procedures/jcp2#3.4.5
> >
> >
> >
> >       Regards,
> >
> >       TimUnless stated otherwise above:
> > IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
> 3AU
> >
>
>
> --
> - DML
>
-- 
Cheers, Martijn (Sent from Gmail Mobile)

From david.lloyd at redhat.com  Wed May 10 22:08:04 2017
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Wed, 10 May 2017 17:08:04 -0500
Subject: Layer Primitives (3/3): addPackage()
Message-ID: <a7dc1e9d-586b-582c-9f3f-9a82bc11d21c@redhat.com>

Saving the controversial one for last: I'd like to re-propose adding the 
following method to the layer controller specification and 
implementation [1].

Justification:

? Existing containers and frameworks from multiple vendors and 
developers have the ability to add static and dynamic content to 
applications at run time for a variety of reasons and with a variety of 
security requirements.
? A very large number of Java users are using Java by way of such 
containers and frameworks; this functionality is generally useful.
? Containers and frameworks _will_ be using JPMS modules dynamically in 
order (at minimum) to take advantage of the security benefits provided 
thereby; the ModuleLayer.Controller API exists in recognition of this fact.
? While it is possible, via various techniques, to provide the illusion 
of adding modular content at run time by other means, these other 
techniques introduce various user-visible problems that can lead to 
undue confusion and difficulty; additionally, it would result in a 
fairly substantial increase in downstream complexity for containers, 
whereas the upstream complexity should be considerably less and can be 
utilized by all.
? The controller is only available to user-defined Layers, so there is 
no risk or exposure to the platform, jlink, etc.
? The proposed behavior is very consistent with the existing behavior of 
the module system and its implementation, and as such can be implemented 
in terms of functionality already necessarily existent in the current 
JDK codebase.
? The patch itself is very small.

Drawbacks:

? The platform would have to be specified to have the capability to 
dynamically register a package-to-module mapping for any user-managed 
Layer, which would be used for subsequent diagnostic and security 
calculations.  This is significantly mitigated by the fact that the 
reference implementation already has the required capability.  It is 
also mitigated by the fact that the implementation would not need to 
provide or support this ability for JDK-managed layer(s).
? A determination would have to be made as to whether the supplemental 
package would figure into resolver calculations; however, this should be 
mitigated by the fact that it is already possible to dynamically add 
exports.  Exporting a freshly minted package should be no different from 
exporting a previously hidden but existent package as far as it is 
concerned; the implementation seems to square with this idea.  This can 
be discussed further if some new information comes to light.
? Adding code dynamically may theoretically introduce various 
deoptimizations in the compiler (JIT) implementation.  However, adding a 
package alone should not introduce any such cases that do not already 
exist by adding classes or packages to the unnamed module, or by 
defining classes into existent packages within a module.  It is 
understood to be unlikely that the compiler would have to make any 
special consideration for this case.  This can be discussed further if 
some new information comes to light.
? The complexity and maintenance cost seems be quite low, but is not 
zero.  However, the gain to container developers, and indirectly the 
value to users thereby, should outweigh the complexity and maintenance 
cost; dynamic and self-modifying code has been a foundation capability 
of Java since the 1990s.  Enabling dynamic containers in what could be a 
substantial way, especially at such a low cost, should be considered a 
worthwhile goal that will benefit a large portion of the community. 
This can be discussed further if it is shown that the complexity or 
maintenance cost is likely to be more substantial than expected or if 
new information comes to light.


Patch information:

The content and format of the JavaDoc are drawn from neighboring 
methods.  I relinquish any copyright claim to the patch in this mail. 
Please do not get bogged down by any formatting problems introduced by 
the mailing list; the purpose of directly including the patch is to give 
a clear, unambiguous subject for discussion.  I can provide a proper 
webrev (or whatever other form is requested) if needed.  This patch 
applies atop the previous (2/3) patch, but can trivially be applied 
without it as well.

[1]:

diff --git a/jdk/src/java.base/share/classes/java/lang/Module.java 
b/jdk/src/java.base/share/classes/java/lang/Module.java
index f6ab49b..789d02f 100644
--- a/jdk/src/java.base/share/classes/java/lang/Module.java
+++ b/jdk/src/java.base/share/classes/java/lang/Module.java
@@ -1016,7 +1016,7 @@ public final class Module implements 
AnnotatedElement {
       * @throws IllegalArgumentException if the package name is not legal
       * @throws IllegalStateException if the package is defined to 
another module
       */
-    private void implAddPackage(String pn, boolean syncVM) {
+    void implAddPackage(String pn, boolean syncVM) {
          // no-op if unnamed module
          if (!isNamed())
              return;
diff --git a/jdk/src/java.base/share/classes/java/lang/ModuleLayer.java 
b/jdk/src/java.base/share/classes/java/lang/ModuleLayer.java
index 541be5f..7cb6377 100644
--- a/jdk/src/java.base/share/classes/java/lang/ModuleLayer.java
+++ b/jdk/src/java.base/share/classes/java/lang/ModuleLayer.java
@@ -332,6 +332,31 @@ public final class ModuleLayer {

              return this;
          }
+
+        /**
+         * Updates module {@code source} in the layer to contain a new 
package.
+         * This method is a no-op if {@code source} already contains 
package {@code pn}.
+         *
+         * @param  source
+         *         The source module
+         * @param  pn
+         *         The package name to add
+         *
+         * @return This controller
+         *
+         * @throws IllegalArgumentException
+         *         If {@code pn} is {@code null}, or {@code source} is 
not in the layer
+         */
+        public Controller addPackage(Module source, String pn) {
+            Objects.requireNonNull(source);
+            if (pn == null)
+                throw new IllegalArgumentException("package is null");
+            if (source.isNamed()) {
+                ensureInLayer(source);
+                source.implAddPackage(pn, true);
+            }
+            return this;
+        }
      }



-- 
- DML

From david.lloyd at redhat.com  Wed May 10 22:24:46 2017
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Wed, 10 May 2017 17:24:46 -0500
Subject: The critical missing pieces and a path forward
In-Reply-To: <530d03fa-cf66-896e-479e-90c18cf317bf@redhat.com>
References: <3b52b3b6-d95d-dacf-06ce-8fceaa1a8cf5@redhat.com>
 <20170508145736.531AFC4422@eggemoggin.niobe.net>
 <530d03fa-cf66-896e-479e-90c18cf317bf@redhat.com>
Message-ID: <129f302a-e846-2e3d-6114-7bea721b93b9@redhat.com>

On 05/09/2017 07:49 AM, David M. Lloyd wrote:
> On 05/08/2017 09:57 AM, mark.reinhold at oracle.com wrote:
>> This module system loads application modules into a single class loader
>> by default so that developers have one less behavioral difference to
>> worry about as they upgrade their components to explicit modules.  An
>> early prototype did load every module into its own loader by default.
> 
> What about a command line option to either opt in to or opt out of 
> having separated class loaders?  The class loader of each module in this 
> case should be able to resolve only the same set of types and resources 
> that its corresponding module does.  This way, users can use the module 
> path without custom code, while still being able to take advantage of 
> more advanced use cases.
> 
> Ideally jlink would also support such an option.

As time passes since the vote, it is becoming clear that at least some 
of the "no" votes were cast in whole or in part because isolating 
package namespaces within modules is indeed considered a fundamental 
expectation of a module system's capabilities.  Please give this idea 
due consideration.


-- 
- DML

From volker.berlin at goebel-clan.de  Thu May 11 20:03:10 2017
From: volker.berlin at goebel-clan.de (Volker Berlin)
Date: Thu, 11 May 2017 22:03:10 +0200
Subject: The critical missing pieces and a path forward
In-Reply-To: <20170508145736.531AFC4422@eggemoggin.niobe.net>
References: <3b52b3b6-d95d-dacf-06ce-8fceaa1a8cf5@redhat.com>
 <20170508145736.531AFC4422@eggemoggin.niobe.net>
Message-ID: <f503147c-01c0-45b1-583e-3b7a4d3df206@goebel-clan.de>


Am 08.05.2017 um 16:57 schrieb mark.reinhold at oracle.com:
> Forbidding cyclic module relationships is one of these new constraints.
> That such relationships are generally a bad idea is not just my personal
> view; see, e.g., Kirk Knoernschild's compelling arguments against them
> in his fine book,_Java Application Architecture_  [2], specifically in
> ??4.4, "Cyclic Dependencies -- The Death Knell".
I does not understand the discussion here. It is inpossible to write a 
large complex application without cyclic dependencies. The classic cycle 
is "Logging" and "Configuration". The configuration define the type of 
logging like level, file, format, etc. If there any errors in the 
configuration then it write it to the logging module.
Another frequently cycle is between "Login/Permission" and "GUI". The 
login needs a GUI. And the dialog elements of the GUI needs a permission 
check.

Because we have understand this and the hierarchically classloader can 
not handle cycles that we have develop for years a classloader with cycles.
The alterative to cycles are large monolithic modulus/components. Such 
modules are very bad to maintain.

> (For those not familiar with the details of the design: Cycles are not
>   allowed when modules are resolved but it is possible to create them
>   post-resolution via the API, if needed [4][5].  Cycles are also set up
>   amongst all automatic modules, after resolution, to make it easier to
>   treat existing JAR files as modules without change [6].)
A light point but what are the advantages of modules in this case. I see 
no advantages across from classloaders. Because classloaders will never 
removed there is no need for a switch.

Volker
i-net software

From alex.buckley at oracle.com  Thu May 11 21:47:06 2017
From: alex.buckley at oracle.com (Alex Buckley)
Date: Thu, 11 May 2017 14:47:06 -0700
Subject: The critical missing pieces and a path forward
In-Reply-To: <f503147c-01c0-45b1-583e-3b7a4d3df206@goebel-clan.de>
References: <3b52b3b6-d95d-dacf-06ce-8fceaa1a8cf5@redhat.com>
 <20170508145736.531AFC4422@eggemoggin.niobe.net>
 <f503147c-01c0-45b1-583e-3b7a4d3df206@goebel-clan.de>
Message-ID: <5914DBDA.7000300@oracle.com>

On 5/11/2017 1:03 PM, Volker Berlin wrote:
> Am 08.05.2017 um 16:57 schrieb mark.reinhold at oracle.com:
>> Forbidding cyclic module relationships is one of these new constraints.
>> That such relationships are generally a bad idea is not just my personal
>> view; see, e.g., Kirk Knoernschild's compelling arguments against them
>> in his fine book,_Java Application Architecture_  [2], specifically in
>> ??4.4, "Cyclic Dependencies -- The Death Knell".
> I does not understand the discussion here. It is inpossible to write a
> large complex application without cyclic dependencies. The classic cycle
> is "Logging" and "Configuration". The configuration define the type of
> logging like level, file, format, etc. If there any errors in the
> configuration then it write it to the logging module.

It sounds like:

- The implementation of "Logging" accesses the API of "Configuration", 
to determine how much logging has been configured; and

- The implementation of "Configuration" accesses the API of "Logging", 
to record errors in config files.

The cycle can be broken by refactoring either "Configuration" or 
"Logging". In the spirit of the recent jigsaw-dev discussion about Log4j 
[1], let's refactor "Logging" into "LoggingIntf" and "LoggingImpl":

- "LoggingIntf" 'exports' an API for general use, so "Configuration" 
'requires' "LoggingIntf".

- "LoggingIntf" also 'exports' an SPI.

- "LoggingImpl" 'requires' "LoggingIntf" in order to gain access to the 
SPI, and 'provides' an implementation of the SPI.

- "LoggingIntf" 'uses' its own SPI and discovers providers via 
ServiceLoader.

- It may be that "LoggingImpl" 'requires' many other modules, and they 
in turn have their own 'requires', eventually leading back to 
"LoggingIntf", and that's fine. The dependencies of "LoggingImpl" may 
even have their own 'uses' and 'provides' directives, all of which will 
be bound in the same fashion as "LoggingImpl"-to-"LoggingIntf".

Alex

[1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012562.html

From david.lloyd at redhat.com  Thu May 11 23:17:45 2017
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Thu, 11 May 2017 18:17:45 -0500
Subject: A compromise approach on module isolation
Message-ID: <e57dc552-d365-156b-1307-245dff80e656@redhat.com>

In the wake of the PR vote, one of the most oft-cited unmet expectations 
was the lack of package namespace isolation among modules.  I'd like to 
propose a patch [1] which would allow this type of module isolation to 
be used by the run time on an opt-in or opt-out basis.

Justification:

? Large applications are more likely to encounter concealed and 
non-concealed package conflicts.
? By making the change opt-in or opt-out, any application which expects 
all of its modules to be loaded by a single class loader can still 
function, while allowing more complex applications (which already would 
necessarily have required at least some separated class loaders) to 
continue to work.
? The patch seems to have a small complexity and maintenance footprint.

Drawbacks:

None that I am aware of.

Patch information:

The approach of this patch simply looks for a system property and, 
depending on the value of this property, establishes isolated module 
loaders.  I relinquish any copyright claim to the patch in this mail. 
Please do not get bogged down by any formatting problems introduced by 
the mailing list; the purpose of directly including the patch is to give 
a clear, unambiguous subject for discussion.  I can provide a proper 
webrev (or whatever other form is requested) if needed.

Other than the system property (which is just one possible approach), no 
API or specification changes should be needed.  This version uses the 
property "jdk.module.isolated" defaulting to "false".

[1]:

diff --git 
a/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java 
b/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
index d976aab..f06adfb 100644
--- a/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
+++ b/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
@@ -112,6 +112,15 @@ public static ClassLoader appClassLoader() {
      }

      /**
+     * Returns a new isolated application module class loader.
+     *
+     * @return the new application module class loader (not {@code null})
+     */
+    public static ClassLoader appModuleClassLoader() {
+        return new AppModuleClassLoader(APP_LOADER);
+    }
+
+    /**
       * The class loader that is used to find resources in modules 
defined to
       * the boot class loader. It is not used for class loading.
       */
@@ -220,6 +229,21 @@ protected Package defineOrCheckPackage(String pn, 
Manifest man, URL url) {
      }

      /**
+     * An isolated application module class loader that is a {@code 
BuiltinClassLoader} with
+     * the application class loader as parent.
+     */
+    private static class AppModuleClassLoader extends BuiltinClassLoader {
+        static {
+            if (!ClassLoader.registerAsParallelCapable())
+                throw new InternalError();
+        }
+
+        AppModuleClassLoader(AppClassLoader parent) {
+            super("app", parent, null);
+        }
+    }
+
+    /**
       * Returns a {@code URLClassPath} of file URLs to each of the 
elements in
       * the given class path.
       */
diff --git 
a/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java 
b/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java
index b0f465d..cc2bdb6 100644
--- 
a/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java
+++ 
b/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java
@@ -85,6 +85,8 @@ public ClassLoader apply(String name) {
          return platformModules;
      }

+    private static final boolean ISOLATED_MODULE_PATH = 
Boolean.parseBoolean(System.getProperty("jdk.module.isolated", "false"));
+
      /**
       * Returns the function to map modules in the given configuration 
to the
       * built-in class loaders.
@@ -102,6 +104,8 @@ public ClassLoader apply(String name) {
              if (!bootModules.contains(mn)) {
                  if (platformModules.contains(mn)) {
                      map.put(mn, platformClassLoader);
+                } else if (ISOLATED_MODULE_PATH) {
+                    map.put(mn, ClassLoaders.appModuleClassLoader());
                  } else {
                      map.put(mn, appClassLoader);
                  }


-- 
- DML

From david.lloyd at redhat.com  Fri May 12 00:18:26 2017
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Thu, 11 May 2017 19:18:26 -0500
Subject: A compromise approach on circularity checking
Message-ID: <27cf8102-27d5-92c1-f7fd-1c472df96ad4@redhat.com>

I'd like to propose a patch [1] which allows run-time circularity 
checking to be disabled on an opt-out basis.  This allows an approach 
wherein data can be collected as to the usefulness of circularity 
checking, without locking users into it (at least in the near term).

Justification:

? In many environments, including large enterprises, or large open- and 
closed-source projects which may consume or assemble many dependencies, 
it is possible that a module which builds without cycles may fail due to 
a cycle at assembly time.
? By making this change opt-out, most users need not worry about it; if 
assembly problems occur at the end of the development or test process, 
however, users have an "escape hatch" at their disposal.
? If it becomes clear that allowing cycles at run time is indeed a valid 
requirement, then cycle checking at run time can be disabled permanently 
in Java 10 or a later release.

Drawbacks:

None that I am aware of.

Patch information:

The approach of this patch simply looks for a system property and, 
depending on the value of this property, enables or disables cycle 
checking.  I relinquish any copyright claim to the patch in this mail. 
Please do not get bogged down by any formatting problems introduced by 
the mailing list; the purpose of directly including the patch is to give 
a clear, unambiguous subject for discussion.  I can provide a proper 
webrev (or whatever other form is requested) if needed.

Other than the system property (which is just one possible approach), no 
API or specification changes should be needed.  This version uses the 
property "jdk.module.detect-cycles" defaulting to "true".

[1]:

diff --git 
a/jdk/src/java.base/share/classes/java/lang/module/Resolver.java 
b/jdk/src/java.base/share/classes/java/lang/module/Resolver.java
index a723e63..2451358 100644
--- a/jdk/src/java.base/share/classes/java/lang/module/Resolver.java
+++ b/jdk/src/java.base/share/classes/java/lang/module/Resolver.java
@@ -351,6 +351,8 @@ final class Resolver {
          }
      }

+    private static final boolean DETECT_CYCLES = 
Boolean.parseBoolean(System.getProperty("jdk.module.detect-cycles", 
"true"));
+
      /**
       * Execute post-resolution checks and returns the module graph of 
resolved
       * modules as {@code Map}. The resolved modules will be in the given
@@ -362,7 +364,7 @@ final class Resolver {
                                                      boolean check)
      {
          if (check) {
-            detectCycles();
+            if (DETECT_CYCLES) detectCycles();
              checkHashes();
          }



-- 
- DML

From forax at univ-mlv.fr  Fri May 12 06:17:35 2017
From: forax at univ-mlv.fr (Remi Forax)
Date: Fri, 12 May 2017 08:17:35 +0200 (CEST)
Subject: An alternative to "restricted keywords"
In-Reply-To: <2bdf5cbb-b3a1-dc16-702f-af7fc9b4c645@berlin.de>
References: <2bdf5cbb-b3a1-dc16-702f-af7fc9b4c645@berlin.de>
Message-ID: <641041596.2374187.1494569855008.JavaMail.zimbra@u-pem.fr>

[CC JPMS expert mailing list because, it's an important issue IMO]

I've a counter proposition.

I do not like your proposal because from the user point of view, '^' looks like a hack, it's not used anywhere else in the grammar. 
I agree that restricted keywords are not properly specified in JLS. Reading your mail, i've discovered that what i was calling restricted keywords is not what javac implements :(
I agree that restricted keywords should be only enabled when parsing module-info.java
I agree that doing error recovery on the way the grammar for module-info is currently implemented in javac leads to less than ideal error messages.

In my opinion, both
   module m { requires transitive transitive; }
   module m { requires transitive; }
should be rejected because what javac implements something more close to the javascript ASI rules than restricted keywords as currently specified by Alex.

For me, a restricted keyword is a keyword which is activated if you are at a position in the grammar where it can be recognized and because it's a keyword, it tooks over an identifier.
by example for 
  module m { 
if the next token is 'requires', it should be recognized as a keyword because you can parse a directive 'required ...' so there is a production that will starts with the 'required' keyword.

so 
  module m { requires transitive; }
should be rejected because transitive should be recognized as a keyword after requires and the compiler should report a missing module name.
 
and
  module m { requires transitive transitive; }
should be rejected because the grammar that parse the modifiers is defined as "a loop" so from the grammar point of view it's like
  module m { requires Modifier Modifier; }
so the the front end of the compiler should report a missing module name and a later phase should report that there is twice the same modifier 'transitive'.

I believe that with this definition of 'restricted keyword', compiler can recover error more easily and offers meaningful error message and the module-info part of the grammar is LR(1).

regards,
R?mi

----- Mail original -----
> De: "Stephan Herrmann" <stephan.herrmann at berlin.de>
> ?: jigsaw-dev at openjdk.java.net
> Envoy?: Mardi 9 Mai 2017 16:56:11
> Objet: An alternative to "restricted keywords"

> (1) I understand the need for avoiding that new module-related
> keywords conflict with existing code, where these words may be used
> as identifiers. Moreover, it must be possible for a module declaration
> to refer to packages or types thusly named.
> 
> However,
> 
> (2) The currently proposed "restricted keywords" are not appropriately
> specified in JLS.
> 
> (3) The currently proposed "restricted keywords" pose difficulties to
> the implementation of all tools that need to parse a module declaration.
> 
> (4) A simple alternative to "restricted keywords" exists, which has not
> received the attention it deserves.
> 
> Details:
> 
> (2) The current specification implicitly violates the assumption that
> parsing can be performed on the basis of a token stream produced by
> a scanner (aka lexer). From discussion on this list we learned that
> the following examples are intended to be syntactically legal:
>    module m { requires transitive transitive; }
>    module m { requires transitive; }
> (Please for the moment disregard heuristic solutions, while we are
>  investigating whether generally "restricted keywords" is a well-defined
>  concept, or not.)
> Of the three occurrences of "transitive", #1 is a keyword, the others
> are identifiers. At the point when the parser has consumed "requires"
> and now asks about classification of the word "transitive", the scanner
> cannot possible answer this classification. It can only answer for sure,
> after the *parser* has accepted the full declaration. Put differently,
> the parser must consume more tokens than have been classified by the
> Scanner. Put differently, to faithfully parse arbitrary grammars using
> a concept of "restricted keywords", scanners must provide speculative
> answers, which may later need to be revised by backtracking or similar
> exhaustive exploration of the space of possible interpretations.
> 
> The specification is totally silent about this fundamental change.
> 
> 
> (3) "restricted keywords" pose three problems to tool implementations:
> 
> (3.a) Any known practical approach to implement a parser with
> "restricted keywords" requires to leverage heuristics, which are based
> on the exact set of rules defined in the grammar. Such heuristics
> reduce the look-ahead that needs to be performed by the scanner,
> in order to avoid the full exhaustive exploration mentioned above.
> A set of such heuristic is extremely fragile and can easily break when
> later more rules are added to the grammar. This means small future
> language changes can easily break any chosen strategy.
> 
> (3.b) If parsing works for error-free input, this doesn't imply that
> a parser will be able to give any useful answer for input with syntax
> errors. As a worst-case example consider an arbitrary input sequence
> consisting of just the two words "requires" and "transitive" in random
> order and with no punctuation.
> A parser will not be able to detect any structure in this sequence.
> By comparison, normal keywords serve as a baseline, where parsing
> typically can resume regardless of any leading garbage.
> While this is not relevant for normal compilation, it is paramount
> for assistive functions, which most of the time operate on incomplete
> text, likely to contain even syntax errors.
> Strictly speaking, any "module declaration" with syntax errors is
> not a ModuleDeclaration, and thus none of the "restrictive keywords"
> can be interpreted as keywords (which per JLS can only happen inside
> a ModuleDeclaration).
> All this means, that functionality like code completion is
> systematically broken in a language using "restricted keywords".
> 
> (3.c) Other IDE functionality assumes that small fragments of the
> input text can be scanned out of context. The classical example here
> is syntax highlighting but there are more examples.
> Any such functionality has to be re-implemented, replacing the
> highly efficient local scanning with full parsing of the input text.
> For functionality that is implicitly invoked per keystroke, or on
> mouse hover etc, this difference in efficiency negatively affects
> the overall user experience of an IDE.
> 
> 
> (4) The following proposal avoids all difficulties described above:
> 
> * open, module, requires, transitive, exports, opens, to, uses,
>    provides, and with are "module words", to which the following
>    interpretation is applied:
>    * within any ordinary compilation unit, a module word is a normal
>      identifier.
>    * within a modular compilation unit, all module words are
>      (unconditional) keywords.
> * We introduce three new auxiliary non-terminals:
>      LegacyPackageName:
>          LegacyIdentifier
>          LegacyPackageName . LegacyIdentifier
>      LegacyTypeName:
>          LegacyIdentifier
>          LegacyTypeName . LegacyIdentifier
>      LegacyIdentifier:
>          Identifier
>          ^open
>          ^module
>          ...
>          ^with
> * We modify all productions in 7.7, replacing PackageName with
>   LegacyPackageName and replacing TypeName with LegacyTypeName.
> * After parsing, each of the words '^open', '^module' etc.
>   is interpreted by removing the leading '^' (escape character).
> 
> Here, '^' is chosen as the escape character following the precedent
> of Xtext. Plenty of other options for this purpose are possible, too.
> 
> 
> 
> This proposal completely satisfies the requirements (1), and avoids
> all of the problems (2) and (3). There's an obvious price to pay:
> users will have to add the escape character when referring to code
> that uses a module word as a package name or type name.
> 
> Not only is this a very low price compared to the benefits; one can
> even argue that it also helps the human reader of a module declaration,
> because it clearly marks which occurrences of a module word are indeed
> identifiers.
> 
> An IDE can easily help in interactively adding escapes where necessary.
> 
> Finally, in this trade-off it is relevant to consider the expected
> frequencies: legacy names (needing escape) will surely be the exception
> - by magnitudes. So, the little price needing to be paid, will only
> affect a comparatively small number of locations.
> 
> 
> Stephan

From forax at univ-mlv.fr  Fri May 12 06:24:33 2017
From: forax at univ-mlv.fr (Remi Forax)
Date: Fri, 12 May 2017 08:24:33 +0200 (CEST)
Subject: A compromise approach on circularity checking
In-Reply-To: <27cf8102-27d5-92c1-f7fd-1c472df96ad4@redhat.com>
References: <27cf8102-27d5-92c1-f7fd-1c472df96ad4@redhat.com>
Message-ID: <743970072.2375906.1494570273644.JavaMail.zimbra@u-pem.fr>

Hi Davidn
thanks for keep us focus on what are the issues we should discuss.


I agree with this proposal, detecting cycle by default will raise awareness that there is a mismatch beetween the configuration at compile time and the configuration at runtime so it should be enabled by default and i agree that from a pragmatic point of view, you can disable it because this is the current behavior and it will help people to transitioning to JPMS.

R?mi

----- Mail original -----
> De: "David M. Lloyd" <david.lloyd at redhat.com>
> ?: jpms-spec-experts at openjdk.java.net
> Envoy?: Vendredi 12 Mai 2017 02:18:26
> Objet: A compromise approach on circularity checking

> I'd like to propose a patch [1] which allows run-time circularity
> checking to be disabled on an opt-out basis.  This allows an approach
> wherein data can be collected as to the usefulness of circularity
> checking, without locking users into it (at least in the near term).
> 
> Justification:
> 
> ? In many environments, including large enterprises, or large open- and
> closed-source projects which may consume or assemble many dependencies,
> it is possible that a module which builds without cycles may fail due to
> a cycle at assembly time.
> ? By making this change opt-out, most users need not worry about it; if
> assembly problems occur at the end of the development or test process,
> however, users have an "escape hatch" at their disposal.
> ? If it becomes clear that allowing cycles at run time is indeed a valid
> requirement, then cycle checking at run time can be disabled permanently
> in Java 10 or a later release.
> 
> Drawbacks:
> 
> None that I am aware of.
> 
> Patch information:
> 
> The approach of this patch simply looks for a system property and,
> depending on the value of this property, enables or disables cycle
> checking.  I relinquish any copyright claim to the patch in this mail.
> Please do not get bogged down by any formatting problems introduced by
> the mailing list; the purpose of directly including the patch is to give
> a clear, unambiguous subject for discussion.  I can provide a proper
> webrev (or whatever other form is requested) if needed.
> 
> Other than the system property (which is just one possible approach), no
> API or specification changes should be needed.  This version uses the
> property "jdk.module.detect-cycles" defaulting to "true".
> 
> [1]:
> 
> diff --git
> a/jdk/src/java.base/share/classes/java/lang/module/Resolver.java
> b/jdk/src/java.base/share/classes/java/lang/module/Resolver.java
> index a723e63..2451358 100644
> --- a/jdk/src/java.base/share/classes/java/lang/module/Resolver.java
> +++ b/jdk/src/java.base/share/classes/java/lang/module/Resolver.java
> @@ -351,6 +351,8 @@ final class Resolver {
>          }
>      }
> 
> +    private static final boolean DETECT_CYCLES =
> Boolean.parseBoolean(System.getProperty("jdk.module.detect-cycles",
> "true"));
> +
>      /**
>       * Execute post-resolution checks and returns the module graph of
> resolved
>       * modules as {@code Map}. The resolved modules will be in the given
> @@ -362,7 +364,7 @@ final class Resolver {
>                                                      boolean check)
>      {
>          if (check) {
> -            detectCycles();
> +            if (DETECT_CYCLES) detectCycles();
>              checkHashes();
>          }
> 
> 
> 
> --
> - DML

From forax at univ-mlv.fr  Fri May 12 08:28:08 2017
From: forax at univ-mlv.fr (Remi Forax)
Date: Fri, 12 May 2017 10:28:08 +0200 (CEST)
Subject: A compromise approach on module isolation
In-Reply-To: <e57dc552-d365-156b-1307-245dff80e656@redhat.com>
References: <e57dc552-d365-156b-1307-245dff80e656@redhat.com>
Message-ID: <142082553.2488054.1494577688036.JavaMail.zimbra@u-pem.fr>

Hi David,
with this patch, there is an issue if people (like me :) ) are using jlink.

With jlink, application modules becomes system modules so depending if the application is delivered as a bunch of jars or as a custom jlink image, so behavior will be different because the application modules deploy with jlink will not be sandboxed.

Maybe it means that jlink has to be changed, i do not know, but i think i would prefer that the module itself should decide using the module-info if it should run its own classloader or not, at least jlink can refuse to work if a module ask for being deployed in its own classloader

And i'm still not sure that this kind of code has to be in the jdk, it can be easily written outside without putting the burden of having to maintain it in the JDK.  

regards,
R?mi


----- Mail original -----
> De: "David M. Lloyd" <david.lloyd at redhat.com>
> ?: jpms-spec-experts at openjdk.java.net
> Envoy?: Vendredi 12 Mai 2017 01:17:45
> Objet: A compromise approach on module isolation

> In the wake of the PR vote, one of the most oft-cited unmet expectations
> was the lack of package namespace isolation among modules.  I'd like to
> propose a patch [1] which would allow this type of module isolation to
> be used by the run time on an opt-in or opt-out basis.
> 
> Justification:
> 
> ? Large applications are more likely to encounter concealed and
> non-concealed package conflicts.
> ? By making the change opt-in or opt-out, any application which expects
> all of its modules to be loaded by a single class loader can still
> function, while allowing more complex applications (which already would
> necessarily have required at least some separated class loaders) to
> continue to work.
> ? The patch seems to have a small complexity and maintenance footprint.
> 
> Drawbacks:
> 
> None that I am aware of.
> 
> Patch information:
> 
> The approach of this patch simply looks for a system property and,
> depending on the value of this property, establishes isolated module
> loaders.  I relinquish any copyright claim to the patch in this mail.
> Please do not get bogged down by any formatting problems introduced by
> the mailing list; the purpose of directly including the patch is to give
> a clear, unambiguous subject for discussion.  I can provide a proper
> webrev (or whatever other form is requested) if needed.
> 
> Other than the system property (which is just one possible approach), no
> API or specification changes should be needed.  This version uses the
> property "jdk.module.isolated" defaulting to "false".
> 
> [1]:
> 
> diff --git
> a/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
> b/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
> index d976aab..f06adfb 100644
> --- a/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
> +++ b/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
> @@ -112,6 +112,15 @@ public static ClassLoader appClassLoader() {
>      }
> 
>      /**
> +     * Returns a new isolated application module class loader.
> +     *
> +     * @return the new application module class loader (not {@code null})
> +     */
> +    public static ClassLoader appModuleClassLoader() {
> +        return new AppModuleClassLoader(APP_LOADER);
> +    }
> +
> +    /**
>       * The class loader that is used to find resources in modules
> defined to
>       * the boot class loader. It is not used for class loading.
>       */
> @@ -220,6 +229,21 @@ protected Package defineOrCheckPackage(String pn,
> Manifest man, URL url) {
>      }
> 
>      /**
> +     * An isolated application module class loader that is a {@code
> BuiltinClassLoader} with
> +     * the application class loader as parent.
> +     */
> +    private static class AppModuleClassLoader extends BuiltinClassLoader {
> +        static {
> +            if (!ClassLoader.registerAsParallelCapable())
> +                throw new InternalError();
> +        }
> +
> +        AppModuleClassLoader(AppClassLoader parent) {
> +            super("app", parent, null);
> +        }
> +    }
> +
> +    /**
>       * Returns a {@code URLClassPath} of file URLs to each of the
> elements in
>       * the given class path.
>       */
> diff --git
> a/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java
> b/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java
> index b0f465d..cc2bdb6 100644
> ---
> a/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java
> +++
> b/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java
> @@ -85,6 +85,8 @@ public ClassLoader apply(String name) {
>          return platformModules;
>      }
> 
> +    private static final boolean ISOLATED_MODULE_PATH =
> Boolean.parseBoolean(System.getProperty("jdk.module.isolated", "false"));
> +
>      /**
>       * Returns the function to map modules in the given configuration
> to the
>       * built-in class loaders.
> @@ -102,6 +104,8 @@ public ClassLoader apply(String name) {
>              if (!bootModules.contains(mn)) {
>                  if (platformModules.contains(mn)) {
>                      map.put(mn, platformClassLoader);
> +                } else if (ISOLATED_MODULE_PATH) {
> +                    map.put(mn, ClassLoaders.appModuleClassLoader());
>                  } else {
>                      map.put(mn, appClassLoader);
>                  }
> 
> 
> --
> - DML

From david.lloyd at redhat.com  Fri May 12 13:45:03 2017
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Fri, 12 May 2017 08:45:03 -0500
Subject: A compromise approach on module isolation
In-Reply-To: <142082553.2488054.1494577688036.JavaMail.zimbra@u-pem.fr>
References: <e57dc552-d365-156b-1307-245dff80e656@redhat.com>
 <142082553.2488054.1494577688036.JavaMail.zimbra@u-pem.fr>
Message-ID: <35f5ab53-43ff-0645-09f6-dab09c2a2c86@redhat.com>

Hi R?mi, thanks for the feedback.

At the moment you can fix this by editing the launcher scripts and 
adding "-Djdk.module.isolated=true" to the end of the 
"JLINK_VM_OPTIONS=" line (and similarly for Windows).

I think the tool can do this automatically with only minimal changes. 
I'll have a look at it.

On 05/12/2017 03:28 AM, Remi Forax wrote:
> Hi David,
> with this patch, there is an issue if people (like me :) ) are using jlink.
> 
> With jlink, application modules becomes system modules so depending if the application is delivered as a bunch of jars or as a custom jlink image, so behavior will be different because the application modules deploy with jlink will not be sandboxed.
> 
> Maybe it means that jlink has to be changed, i do not know, but i think i would prefer that the module itself should decide using the module-info if it should run its own classloader or not, at least jlink can refuse to work if a module ask for being deployed in its own classloader
> 
> And i'm still not sure that this kind of code has to be in the jdk, it can be easily written outside without putting the burden of having to maintain it in the JDK.
> 
> regards,
> R?mi
> 
> 
> ----- Mail original -----
>> De: "David M. Lloyd" <david.lloyd at redhat.com>
>> ?: jpms-spec-experts at openjdk.java.net
>> Envoy?: Vendredi 12 Mai 2017 01:17:45
>> Objet: A compromise approach on module isolation
> 
>> In the wake of the PR vote, one of the most oft-cited unmet expectations
>> was the lack of package namespace isolation among modules.  I'd like to
>> propose a patch [1] which would allow this type of module isolation to
>> be used by the run time on an opt-in or opt-out basis.
>>
>> Justification:
>>
>> ? Large applications are more likely to encounter concealed and
>> non-concealed package conflicts.
>> ? By making the change opt-in or opt-out, any application which expects
>> all of its modules to be loaded by a single class loader can still
>> function, while allowing more complex applications (which already would
>> necessarily have required at least some separated class loaders) to
>> continue to work.
>> ? The patch seems to have a small complexity and maintenance footprint.
>>
>> Drawbacks:
>>
>> None that I am aware of.
>>
>> Patch information:
>>
>> The approach of this patch simply looks for a system property and,
>> depending on the value of this property, establishes isolated module
>> loaders.  I relinquish any copyright claim to the patch in this mail.
>> Please do not get bogged down by any formatting problems introduced by
>> the mailing list; the purpose of directly including the patch is to give
>> a clear, unambiguous subject for discussion.  I can provide a proper
>> webrev (or whatever other form is requested) if needed.
>>
>> Other than the system property (which is just one possible approach), no
>> API or specification changes should be needed.  This version uses the
>> property "jdk.module.isolated" defaulting to "false".
>>
>> [1]:
>>
>> diff --git
>> a/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
>> b/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
>> index d976aab..f06adfb 100644
>> --- a/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
>> +++ b/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
>> @@ -112,6 +112,15 @@ public static ClassLoader appClassLoader() {
>>       }
>>
>>       /**
>> +     * Returns a new isolated application module class loader.
>> +     *
>> +     * @return the new application module class loader (not {@code null})
>> +     */
>> +    public static ClassLoader appModuleClassLoader() {
>> +        return new AppModuleClassLoader(APP_LOADER);
>> +    }
>> +
>> +    /**
>>        * The class loader that is used to find resources in modules
>> defined to
>>        * the boot class loader. It is not used for class loading.
>>        */
>> @@ -220,6 +229,21 @@ protected Package defineOrCheckPackage(String pn,
>> Manifest man, URL url) {
>>       }
>>
>>       /**
>> +     * An isolated application module class loader that is a {@code
>> BuiltinClassLoader} with
>> +     * the application class loader as parent.
>> +     */
>> +    private static class AppModuleClassLoader extends BuiltinClassLoader {
>> +        static {
>> +            if (!ClassLoader.registerAsParallelCapable())
>> +                throw new InternalError();
>> +        }
>> +
>> +        AppModuleClassLoader(AppClassLoader parent) {
>> +            super("app", parent, null);
>> +        }
>> +    }
>> +
>> +    /**
>>        * Returns a {@code URLClassPath} of file URLs to each of the
>> elements in
>>        * the given class path.
>>        */
>> diff --git
>> a/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java
>> b/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java
>> index b0f465d..cc2bdb6 100644
>> ---
>> a/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java
>> +++
>> b/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java
>> @@ -85,6 +85,8 @@ public ClassLoader apply(String name) {
>>           return platformModules;
>>       }
>>
>> +    private static final boolean ISOLATED_MODULE_PATH =
>> Boolean.parseBoolean(System.getProperty("jdk.module.isolated", "false"));
>> +
>>       /**
>>        * Returns the function to map modules in the given configuration
>> to the
>>        * built-in class loaders.
>> @@ -102,6 +104,8 @@ public ClassLoader apply(String name) {
>>               if (!bootModules.contains(mn)) {
>>                   if (platformModules.contains(mn)) {
>>                       map.put(mn, platformClassLoader);
>> +                } else if (ISOLATED_MODULE_PATH) {
>> +                    map.put(mn, ClassLoaders.appModuleClassLoader());
>>                   } else {
>>                       map.put(mn, appClassLoader);
>>                   }
>>
>>
>> --
>> - DML


-- 
- DML

From david.lloyd at redhat.com  Fri May 12 16:45:18 2017
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Fri, 12 May 2017 11:45:18 -0500
Subject: A compromise approach on module isolation
In-Reply-To: <35f5ab53-43ff-0645-09f6-dab09c2a2c86@redhat.com>
References: <e57dc552-d365-156b-1307-245dff80e656@redhat.com>
 <142082553.2488054.1494577688036.JavaMail.zimbra@u-pem.fr>
 <35f5ab53-43ff-0645-09f6-dab09c2a2c86@redhat.com>
Message-ID: <79d0798a-cece-b81e-0d2d-ef8f2369084b@redhat.com>

Here's a commit which does so (and also covers the circularity case), if 
you want to play with it a bit:

https://github.com/dmlloyd/openjdk-modules/commit/ffbca0555aeb934181ac92ec3a2bf48fae7efe04

On 05/12/2017 08:45 AM, David M. Lloyd wrote:
> Hi R?mi, thanks for the feedback.
> 
> At the moment you can fix this by editing the launcher scripts and 
> adding "-Djdk.module.isolated=true" to the end of the 
> "JLINK_VM_OPTIONS=" line (and similarly for Windows).
> 
> I think the tool can do this automatically with only minimal changes. 
> I'll have a look at it.
> 
> On 05/12/2017 03:28 AM, Remi Forax wrote:
>> Hi David,
>> with this patch, there is an issue if people (like me :) ) are using 
>> jlink.
>>
>> With jlink, application modules becomes system modules so depending if 
>> the application is delivered as a bunch of jars or as a custom jlink 
>> image, so behavior will be different because the application modules 
>> deploy with jlink will not be sandboxed.
>>
>> Maybe it means that jlink has to be changed, i do not know, but i 
>> think i would prefer that the module itself should decide using the 
>> module-info if it should run its own classloader or not, at least 
>> jlink can refuse to work if a module ask for being deployed in its own 
>> classloader
>>
>> And i'm still not sure that this kind of code has to be in the jdk, it 
>> can be easily written outside without putting the burden of having to 
>> maintain it in the JDK.
>>
>> regards,
>> R?mi
>>
>>
>> ----- Mail original -----
>>> De: "David M. Lloyd" <david.lloyd at redhat.com>
>>> ?: jpms-spec-experts at openjdk.java.net
>>> Envoy?: Vendredi 12 Mai 2017 01:17:45
>>> Objet: A compromise approach on module isolation
>>
>>> In the wake of the PR vote, one of the most oft-cited unmet expectations
>>> was the lack of package namespace isolation among modules.  I'd like to
>>> propose a patch [1] which would allow this type of module isolation to
>>> be used by the run time on an opt-in or opt-out basis.
>>>
>>> Justification:
>>>
>>> ? Large applications are more likely to encounter concealed and
>>> non-concealed package conflicts.
>>> ? By making the change opt-in or opt-out, any application which expects
>>> all of its modules to be loaded by a single class loader can still
>>> function, while allowing more complex applications (which already would
>>> necessarily have required at least some separated class loaders) to
>>> continue to work.
>>> ? The patch seems to have a small complexity and maintenance footprint.
>>>
>>> Drawbacks:
>>>
>>> None that I am aware of.
>>>
>>> Patch information:
>>>
>>> The approach of this patch simply looks for a system property and,
>>> depending on the value of this property, establishes isolated module
>>> loaders.  I relinquish any copyright claim to the patch in this mail.
>>> Please do not get bogged down by any formatting problems introduced by
>>> the mailing list; the purpose of directly including the patch is to give
>>> a clear, unambiguous subject for discussion.  I can provide a proper
>>> webrev (or whatever other form is requested) if needed.
>>>
>>> Other than the system property (which is just one possible approach), no
>>> API or specification changes should be needed.  This version uses the
>>> property "jdk.module.isolated" defaulting to "false".
>>>
>>> [1]:
>>>
>>> diff --git
>>> a/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
>>> b/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
>>> index d976aab..f06adfb 100644
>>> --- 
>>> a/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
>>> +++ 
>>> b/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
>>> @@ -112,6 +112,15 @@ public static ClassLoader appClassLoader() {
>>>       }
>>>
>>>       /**
>>> +     * Returns a new isolated application module class loader.
>>> +     *
>>> +     * @return the new application module class loader (not {@code 
>>> null})
>>> +     */
>>> +    public static ClassLoader appModuleClassLoader() {
>>> +        return new AppModuleClassLoader(APP_LOADER);
>>> +    }
>>> +
>>> +    /**
>>>        * The class loader that is used to find resources in modules
>>> defined to
>>>        * the boot class loader. It is not used for class loading.
>>>        */
>>> @@ -220,6 +229,21 @@ protected Package defineOrCheckPackage(String pn,
>>> Manifest man, URL url) {
>>>       }
>>>
>>>       /**
>>> +     * An isolated application module class loader that is a {@code
>>> BuiltinClassLoader} with
>>> +     * the application class loader as parent.
>>> +     */
>>> +    private static class AppModuleClassLoader extends 
>>> BuiltinClassLoader {
>>> +        static {
>>> +            if (!ClassLoader.registerAsParallelCapable())
>>> +                throw new InternalError();
>>> +        }
>>> +
>>> +        AppModuleClassLoader(AppClassLoader parent) {
>>> +            super("app", parent, null);
>>> +        }
>>> +    }
>>> +
>>> +    /**
>>>        * Returns a {@code URLClassPath} of file URLs to each of the
>>> elements in
>>>        * the given class path.
>>>        */
>>> diff --git
>>> a/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java 
>>>
>>> b/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java 
>>>
>>> index b0f465d..cc2bdb6 100644
>>> ---
>>> a/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java 
>>>
>>> +++
>>> b/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java 
>>>
>>> @@ -85,6 +85,8 @@ public ClassLoader apply(String name) {
>>>           return platformModules;
>>>       }
>>>
>>> +    private static final boolean ISOLATED_MODULE_PATH =
>>> Boolean.parseBoolean(System.getProperty("jdk.module.isolated", 
>>> "false"));
>>> +
>>>       /**
>>>        * Returns the function to map modules in the given configuration
>>> to the
>>>        * built-in class loaders.
>>> @@ -102,6 +104,8 @@ public ClassLoader apply(String name) {
>>>               if (!bootModules.contains(mn)) {
>>>                   if (platformModules.contains(mn)) {
>>>                       map.put(mn, platformClassLoader);
>>> +                } else if (ISOLATED_MODULE_PATH) {
>>> +                    map.put(mn, ClassLoaders.appModuleClassLoader());
>>>                   } else {
>>>                       map.put(mn, appClassLoader);
>>>                   }
>>>
>>>
>>> -- 
>>> - DML
> 
> 


-- 
- DML

From forax at univ-mlv.fr  Fri May 12 18:54:17 2017
From: forax at univ-mlv.fr (Remi Forax)
Date: Fri, 12 May 2017 18:54:17 +0000
Subject: A compromise approach on module isolation
In-Reply-To: <79d0798a-cece-b81e-0d2d-ef8f2369084b@redhat.com>
References: <e57dc552-d365-156b-1307-245dff80e656@redhat.com>
 <142082553.2488054.1494577688036.JavaMail.zimbra@u-pem.fr>
 <35f5ab53-43ff-0645-09f6-dab09c2a2c86@redhat.com>
 <79d0798a-cece-b81e-0d2d-ef8f2369084b@redhat.com>
Message-ID: <A7891D21-7470-41F8-976C-14C6671EFB87@univ-mlv.fr>

It doesn't work because application modules becomes system modules,
so even if you propagate the flags, an application modules becomes a system module hence is not deployed in a class loader anymore.

R?mi


On May 12, 2017 6:45:18 PM GMT+02:00, "David M. Lloyd" <david.lloyd at redhat.com> wrote:
>Here's a commit which does so (and also covers the circularity case),
>if 
>you want to play with it a bit:
>
>https://github.com/dmlloyd/openjdk-modules/commit/ffbca0555aeb934181ac92ec3a2bf48fae7efe04
>
>On 05/12/2017 08:45 AM, David M. Lloyd wrote:
>> Hi R?mi, thanks for the feedback.
>> 
>> At the moment you can fix this by editing the launcher scripts and 
>> adding "-Djdk.module.isolated=true" to the end of the 
>> "JLINK_VM_OPTIONS=" line (and similarly for Windows).
>> 
>> I think the tool can do this automatically with only minimal changes.
>
>> I'll have a look at it.
>> 
>> On 05/12/2017 03:28 AM, Remi Forax wrote:
>>> Hi David,
>>> with this patch, there is an issue if people (like me :) ) are using
>
>>> jlink.
>>>
>>> With jlink, application modules becomes system modules so depending
>if 
>>> the application is delivered as a bunch of jars or as a custom jlink
>
>>> image, so behavior will be different because the application modules
>
>>> deploy with jlink will not be sandboxed.
>>>
>>> Maybe it means that jlink has to be changed, i do not know, but i 
>>> think i would prefer that the module itself should decide using the 
>>> module-info if it should run its own classloader or not, at least 
>>> jlink can refuse to work if a module ask for being deployed in its
>own 
>>> classloader
>>>
>>> And i'm still not sure that this kind of code has to be in the jdk,
>it 
>>> can be easily written outside without putting the burden of having
>to 
>>> maintain it in the JDK.
>>>
>>> regards,
>>> R?mi
>>>
>>>
>>> ----- Mail original -----
>>>> De: "David M. Lloyd" <david.lloyd at redhat.com>
>>>> ?: jpms-spec-experts at openjdk.java.net
>>>> Envoy?: Vendredi 12 Mai 2017 01:17:45
>>>> Objet: A compromise approach on module isolation
>>>
>>>> In the wake of the PR vote, one of the most oft-cited unmet
>expectations
>>>> was the lack of package namespace isolation among modules.  I'd
>like to
>>>> propose a patch [1] which would allow this type of module isolation
>to
>>>> be used by the run time on an opt-in or opt-out basis.
>>>>
>>>> Justification:
>>>>
>>>> ? Large applications are more likely to encounter concealed and
>>>> non-concealed package conflicts.
>>>> ? By making the change opt-in or opt-out, any application which
>expects
>>>> all of its modules to be loaded by a single class loader can still
>>>> function, while allowing more complex applications (which already
>would
>>>> necessarily have required at least some separated class loaders) to
>>>> continue to work.
>>>> ? The patch seems to have a small complexity and maintenance
>footprint.
>>>>
>>>> Drawbacks:
>>>>
>>>> None that I am aware of.
>>>>
>>>> Patch information:
>>>>
>>>> The approach of this patch simply looks for a system property and,
>>>> depending on the value of this property, establishes isolated
>module
>>>> loaders.  I relinquish any copyright claim to the patch in this
>mail.
>>>> Please do not get bogged down by any formatting problems introduced
>by
>>>> the mailing list; the purpose of directly including the patch is to
>give
>>>> a clear, unambiguous subject for discussion.  I can provide a
>proper
>>>> webrev (or whatever other form is requested) if needed.
>>>>
>>>> Other than the system property (which is just one possible
>approach), no
>>>> API or specification changes should be needed.  This version uses
>the
>>>> property "jdk.module.isolated" defaulting to "false".
>>>>
>>>> [1]:
>>>>
>>>> diff --git
>>>>
>a/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
>>>>
>b/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
>>>> index d976aab..f06adfb 100644
>>>> --- 
>>>>
>a/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
>>>> +++ 
>>>>
>b/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
>>>> @@ -112,6 +112,15 @@ public static ClassLoader appClassLoader() {
>>>>       }
>>>>
>>>>       /**
>>>> +     * Returns a new isolated application module class loader.
>>>> +     *
>>>> +     * @return the new application module class loader (not {@code
>
>>>> null})
>>>> +     */
>>>> +    public static ClassLoader appModuleClassLoader() {
>>>> +        return new AppModuleClassLoader(APP_LOADER);
>>>> +    }
>>>> +
>>>> +    /**
>>>>        * The class loader that is used to find resources in modules
>>>> defined to
>>>>        * the boot class loader. It is not used for class loading.
>>>>        */
>>>> @@ -220,6 +229,21 @@ protected Package defineOrCheckPackage(String
>pn,
>>>> Manifest man, URL url) {
>>>>       }
>>>>
>>>>       /**
>>>> +     * An isolated application module class loader that is a
>{@code
>>>> BuiltinClassLoader} with
>>>> +     * the application class loader as parent.
>>>> +     */
>>>> +    private static class AppModuleClassLoader extends 
>>>> BuiltinClassLoader {
>>>> +        static {
>>>> +            if (!ClassLoader.registerAsParallelCapable())
>>>> +                throw new InternalError();
>>>> +        }
>>>> +
>>>> +        AppModuleClassLoader(AppClassLoader parent) {
>>>> +            super("app", parent, null);
>>>> +        }
>>>> +    }
>>>> +
>>>> +    /**
>>>>        * Returns a {@code URLClassPath} of file URLs to each of the
>>>> elements in
>>>>        * the given class path.
>>>>        */
>>>> diff --git
>>>>
>a/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java
>
>>>>
>>>>
>b/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java
>
>>>>
>>>> index b0f465d..cc2bdb6 100644
>>>> ---
>>>>
>a/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java
>
>>>>
>>>> +++
>>>>
>b/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java
>
>>>>
>>>> @@ -85,6 +85,8 @@ public ClassLoader apply(String name) {
>>>>           return platformModules;
>>>>       }
>>>>
>>>> +    private static final boolean ISOLATED_MODULE_PATH =
>>>> Boolean.parseBoolean(System.getProperty("jdk.module.isolated", 
>>>> "false"));
>>>> +
>>>>       /**
>>>>        * Returns the function to map modules in the given
>configuration
>>>> to the
>>>>        * built-in class loaders.
>>>> @@ -102,6 +104,8 @@ public ClassLoader apply(String name) {
>>>>               if (!bootModules.contains(mn)) {
>>>>                   if (platformModules.contains(mn)) {
>>>>                       map.put(mn, platformClassLoader);
>>>> +                } else if (ISOLATED_MODULE_PATH) {
>>>> +                    map.put(mn,
>ClassLoaders.appModuleClassLoader());
>>>>                   } else {
>>>>                       map.put(mn, appClassLoader);
>>>>                   }
>>>>
>>>>
>>>> -- 
>>>> - DML
>> 
>> 
>
>
>-- 
>- DML

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

From david.lloyd at redhat.com  Fri May 12 18:56:26 2017
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Fri, 12 May 2017 13:56:26 -0500
Subject: A compromise approach on module isolation
In-Reply-To: <A7891D21-7470-41F8-976C-14C6671EFB87@univ-mlv.fr>
References: <e57dc552-d365-156b-1307-245dff80e656@redhat.com>
 <142082553.2488054.1494577688036.JavaMail.zimbra@u-pem.fr>
 <35f5ab53-43ff-0645-09f6-dab09c2a2c86@redhat.com>
 <79d0798a-cece-b81e-0d2d-ef8f2369084b@redhat.com>
 <A7891D21-7470-41F8-976C-14C6671EFB87@univ-mlv.fr>
Message-ID: <391853ee-966e-f670-9003-b7f720cd9b19@redhat.com>

It seems to work in my tests.  Can you explain what you mean?

$ jlink --module-path $JAVA_HOME/jmods:out/production --add-modules 
org.module.two,org.module.one --output new-test 
-J-Djdk.module.detect-cycles=false -J-Djdk.module.isolated=true 
--launcher one=org.module.one/org.module.one.Main --launcher 
two=org.module.two/org.module.two.Main
$ new-test/bin/two
This is module two, my 
cl=jdk.internal.loader.ClassLoaders$AppModuleClassLoader at 2b05039f

On 05/12/2017 01:54 PM, Remi Forax wrote:
> It doesn't work because application modules becomes system modules,
> so even if you propagate the flags, an application modules becomes a 
> system module hence is not deployed in a class loader anymore.
> 
> R?mi
> 
> 
> On May 12, 2017 6:45:18 PM GMT+02:00, "David M. Lloyd" 
> <david.lloyd at redhat.com> wrote:
> 
>     Here's a commit which does so (and also covers the circularity case), if
>     you want to play with it a bit:
> 
>     https://github.com/dmlloyd/openjdk-modules/commit/ffbca0555aeb934181ac92ec3a2bf48fae7efe04
> 
>     On 05/12/2017 08:45 AM, David M. Lloyd wrote:
> 
>         Hi R?mi, thanks for the feedback.
> 
>         At the moment you can fix this by editing the launcher scripts and
>         adding "-Djdk.module.isolated=true" to the end of the
>         "JLINK_VM_OPTIONS=" line (and similarly for Windows).
> 
>         I think the tool can do this automatically with only minimal
>         changes.
>         I'll have a look at it.
> 
>         On 05/12/2017 03:28 AM, Remi Forax wrote:
> 
>             Hi David,
>             with this patch, there is an issue if people (like me :) )
>             are using
>             jlink.
> 
>             With jlink, application modules becomes system modules so
>             depending if
>             the application is delivered as a bunch of jars or as a
>             custom jlink
>             image, so behavior will be different because the application
>             modules
>             deploy with jlink will not be sandboxed.
> 
>             Maybe it means that jlink has to be changed, i do not know,
>             but i
>             think i would prefer that the module itself should decide
>             using the
>             module-info if it should run its own classloader or not, at
>             least
>             jlink can refuse to work if a module ask for being deployed
>             in its own
>             classloader
> 
>             And i'm still not sure that this kind of code has to be in
>             the jdk, it
>             can be easily written outside without putting the burden of
>             having to
>             maintain it in the JDK.
> 
>             regards,
>             R?mi
> 
> 
>             ----- Mail original -----
> 
>                 De: "David M. Lloyd" <david.lloyd at redhat.com>
>                 ?: jpms-spec-experts at openjdk.java.net
>                 Envoy?: Vendredi 12 Mai 2017 01:17:45
>                 Objet: A compromise approach on module isolation
> 
> 
>                 In the wake of the PR vote, one of the most oft-cited
>                 unmet expectations
>                 was the lack of package namespace isolation among
>                 modules. I'd like to
>                 propose a patch [1] which would allow this type of
>                 module isolation to
>                 be used by the run time on an opt-in or opt-out basis.
> 
>                 Justification:
> 
>                 ? Large applications are more likely to encounter
>                 concealed and
>                 non-concealed package conflicts.
>                 ? By making the change opt-in or opt-out, any
>                 application which expects
>                 all of its modules to be loaded by a single class loader
>                 can still
>                 function, while allowing more complex applications
>                 (which already would
>                 necessarily have required at least some separated class
>                 loaders) to
>                 continue to work.
>                 ? The patch seems to have a small complexity and
>                 maintenance footprint.
> 
>                 Drawbacks:
> 
>                 None that I am aware of.
> 
>                 Patch information:
> 
>                 The approach of this patch simply looks for a system
>                 property and,
>                 depending on the value of this property, establishes
>                 isolated module
>                 loaders. I relinquish any copyright claim to the patch
>                 in this mail.
>                 Please do not get bogged down by any formatting problems
>                 introduced by
>                 the mailing list; the purpose of directly including the
>                 patch is to give
>                 a clear, unambiguous subject for discussion. I can
>                 provide a proper
>                 webrev (or whatever other form is requested) if needed.
> 
>                 Other than the system property (which is just one
>                 possible approach), no
>                 API or specification changes should be needed. This
>                 version uses the
>                 property "jdk.module.isolated" defaulting to "false".
> 
>                 [1]:
> 
>                 diff --git
>                 a/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
>                 <http://ClassLoaders.java>
>                 b/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
>                 <http://ClassLoaders.java>
>                 index d976aab..f06adfb 100644
>                 ---
>                 a/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
>                 <http://ClassLoaders.java>
>                 +++
>                 b/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
>                 <http://ClassLoaders.java>
>                 @@ -112,6 +112,15 @@ public static ClassLoader
>                 appClassLoader() {
>                 }
> 
>                 /**
>                 + * Returns a new isolated application module class loader.
>                 + *
>                 + * @return the new application module class loader (not
>                 {@code
>                 null})
>                 + */
>                 + public static ClassLoader appModuleClassLoader() {
>                 + return new AppModuleClassLoader(APP_LOADER);
>                 + }
>                 +
>                 + /**
>                 * The class loader that is used to find resources in modules
>                 defined to
>                 * the boot class loader. It is not used for class loading.
>                 */
>                 @@ -220,6 +229,21 @@ protected Package
>                 defineOrCheckPackage(String pn,
>                 Manifest man, URL url) {
>                 }
> 
>                 /**
>                 + * An isolated application module class loader that is
>                 a {@code
>                 BuiltinClassLoader} with
>                 + * the application class loader as parent.
>                 + */
>                 + private static class AppModuleClassLoader extends
>                 BuiltinClassLoader {
>                 + static {
>                 + if (!ClassLoader.registerAsParallelCapable())
>                 + throw new InternalError();
>                 + }
>                 +
>                 + AppModuleClassLoader(AppClassLoader parent) {
>                 + super("app", parent, null);
>                 + }
>                 + }
>                 +
>                 + /**
>                 * Returns a {@code URLClassPath} of file URLs to each of the
>                 elements in
>                 * the given class path.
>                 */
>                 diff --git
>                 a/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java
>                 <http://ModuleLoaderMap.java>
> 
>                 b/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java
>                 <http://ModuleLoaderMap.java>
> 
>                 index b0f465d..cc2bdb6 100644
>                 ---
>                 a/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java
>                 <http://ModuleLoaderMap.java>
> 
>                 +++
>                 b/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java
>                 <http://ModuleLoaderMap.java>
> 
>                 @@ -85,6 +85,8 @@ public ClassLoader apply(String name) {
>                 return platformModules;
>                 }
> 
>                 + private static final boolean ISOLATED_MODULE_PATH =
>                 Boolean.parseBoolean(System.getProperty("jdk.module.isolated",
> 
>                 "false"));
>                 +
>                 /**
>                 * Returns the function to map modules in the given
>                 configuration
>                 to the
>                 * built-in class loaders.
>                 @@ -102,6 +104,8 @@ public ClassLoader apply(String name) {
>                 if (!bootModules.contains(mn)) {
>                 if (platformModules.contains(mn)) {
>                 map.put(mn, platformClassLoader);
>                 + } else if (ISOLATED_MODULE_PATH) {
>                 + map.put(mn, ClassLoaders.appModuleClassLoader());
>                 } else {
>                 map.put(mn, appClassLoader);
>                 }
> 
> 
>                 -- 
>                 - DML
> 
> 
> 
> 
> 
> -- 
> Sent from my Android device with K-9 Mail. Please excuse my brevity.


-- 
- DML

From forax at univ-mlv.fr  Fri May 12 19:21:19 2017
From: forax at univ-mlv.fr (Remi Forax)
Date: Fri, 12 May 2017 19:21:19 +0000
Subject: An alternative to "restricted keywords"
In-Reply-To: <7d3d6f28-5560-6e00-81fc-b7562a3d148d@gmail.com>
References: <2bdf5cbb-b3a1-dc16-702f-af7fc9b4c645@berlin.de>
 <641041596.2374187.1494569855008.JavaMail.zimbra@u-pem.fr>
 <7d3d6f28-5560-6e00-81fc-b7562a3d148d@gmail.com>
Message-ID: <6ED388E7-1274-4813-9FE9-A2757F6A409E@univ-mlv.fr>

Hi Peter,

On May 12, 2017 6:08:58 PM GMT+02:00, Peter Levart <peter.levart at gmail.com> wrote:
>Hi Remi,
>
>On 05/12/2017 08:17 AM, Remi Forax wrote:
>> [CC JPMS expert mailing list because, it's an important issue IMO]
>>
>> I've a counter proposition.
>>
>> I do not like your proposal because from the user point of view, '^'
>looks like a hack, it's not used anywhere else in the grammar.
>> I agree that restricted keywords are not properly specified in JLS.
>Reading your mail, i've discovered that what i was calling restricted
>keywords is not what javac implements :(
>> I agree that restricted keywords should be only enabled when parsing
>module-info.java
>> I agree that doing error recovery on the way the grammar for
>module-info is currently implemented in javac leads to less than ideal
>error messages.
>>
>> In my opinion, both
>>     module m { requires transitive transitive; }
>>     module m { requires transitive; }
>> should be rejected because what javac implements something more close
>to the javascript ASI rules than restricted keywords as currently
>specified by Alex.
>>
>> For me, a restricted keyword is a keyword which is activated if you
>are at a position in the grammar where it can be recognized and because
>it's a keyword, it tooks over an identifier.
>> by example for
>>    module m {
>> if the next token is 'requires', it should be recognized as a keyword
>because you can parse a directive 'required ...' so there is a
>production that will starts with the 'required' keyword.
>>
>> so
>>    module m { requires transitive; }
>> should be rejected because transitive should be recognized as a
>keyword after requires and the compiler should report a missing module
>name.
>>   
>> and
>>    module m { requires transitive transitive; }
>> should be rejected because the grammar that parse the modifiers is
>defined as "a loop" so from the grammar point of view it's like
>>    module m { requires Modifier Modifier; }
>> so the the front end of the compiler should report a missing module
>name and a later phase should report that there is twice the same
>modifier 'transitive'.
>>
>> I believe that with this definition of 'restricted keyword', compiler
>can recover error more easily and offers meaningful error message and
>the module-info part of the grammar is LR(1).
>
>This will make "requires", "uses", "provides", "with", "to", "static", 
>"transitive", "exports", etc .... all illegal module names. Ok, no big 
>deal, because there are no module names yet (apart from JDK modules and
>
>those are named differently). But...

you should use reverse DNS naming for module name, so no problem. 

>
>What about:
>
>module m { exports transitive; }
>
>Here 'transitive' is an existing package name for example. Who 
>guarantees that there are no packages out there with names matching 
>restricted keywords? Current restriction for modules is that they can 
>not have an unnamed package. Do we want to restrict package names a 
>module can export too?

you should use reverse DNS naming for package so no problem :)

>
>Stephan's solution does not have this problem.
>
>Regards, Peter

I think those issues are not real problem. 

R?mi 

>
>>
>> regards,
>> R?mi
>>
>> ----- Mail original -----
>>> De: "Stephan Herrmann" <stephan.herrmann at berlin.de>
>>> ?: jigsaw-dev at openjdk.java.net
>>> Envoy?: Mardi 9 Mai 2017 16:56:11
>>> Objet: An alternative to "restricted keywords"
>>> (1) I understand the need for avoiding that new module-related
>>> keywords conflict with existing code, where these words may be used
>>> as identifiers. Moreover, it must be possible for a module
>declaration
>>> to refer to packages or types thusly named.
>>>
>>> However,
>>>
>>> (2) The currently proposed "restricted keywords" are not
>appropriately
>>> specified in JLS.
>>>
>>> (3) The currently proposed "restricted keywords" pose difficulties
>to
>>> the implementation of all tools that need to parse a module
>declaration.
>>>
>>> (4) A simple alternative to "restricted keywords" exists, which has
>not
>>> received the attention it deserves.
>>>
>>> Details:
>>>
>>> (2) The current specification implicitly violates the assumption
>that
>>> parsing can be performed on the basis of a token stream produced by
>>> a scanner (aka lexer). From discussion on this list we learned that
>>> the following examples are intended to be syntactically legal:
>>>     module m { requires transitive transitive; }
>>>     module m { requires transitive; }
>>> (Please for the moment disregard heuristic solutions, while we are
>>>   investigating whether generally "restricted keywords" is a
>well-defined
>>>   concept, or not.)
>>> Of the three occurrences of "transitive", #1 is a keyword, the
>others
>>> are identifiers. At the point when the parser has consumed
>"requires"
>>> and now asks about classification of the word "transitive", the
>scanner
>>> cannot possible answer this classification. It can only answer for
>sure,
>>> after the *parser* has accepted the full declaration. Put
>differently,
>>> the parser must consume more tokens than have been classified by the
>>> Scanner. Put differently, to faithfully parse arbitrary grammars
>using
>>> a concept of "restricted keywords", scanners must provide
>speculative
>>> answers, which may later need to be revised by backtracking or
>similar
>>> exhaustive exploration of the space of possible interpretations.
>>>
>>> The specification is totally silent about this fundamental change.
>>>
>>>
>>> (3) "restricted keywords" pose three problems to tool
>implementations:
>>>
>>> (3.a) Any known practical approach to implement a parser with
>>> "restricted keywords" requires to leverage heuristics, which are
>based
>>> on the exact set of rules defined in the grammar. Such heuristics
>>> reduce the look-ahead that needs to be performed by the scanner,
>>> in order to avoid the full exhaustive exploration mentioned above.
>>> A set of such heuristic is extremely fragile and can easily break
>when
>>> later more rules are added to the grammar. This means small future
>>> language changes can easily break any chosen strategy.
>>>
>>> (3.b) If parsing works for error-free input, this doesn't imply that
>>> a parser will be able to give any useful answer for input with
>syntax
>>> errors. As a worst-case example consider an arbitrary input sequence
>>> consisting of just the two words "requires" and "transitive" in
>random
>>> order and with no punctuation.
>>> A parser will not be able to detect any structure in this sequence.
>>> By comparison, normal keywords serve as a baseline, where parsing
>>> typically can resume regardless of any leading garbage.
>>> While this is not relevant for normal compilation, it is paramount
>>> for assistive functions, which most of the time operate on
>incomplete
>>> text, likely to contain even syntax errors.
>>> Strictly speaking, any "module declaration" with syntax errors is
>>> not a ModuleDeclaration, and thus none of the "restrictive keywords"
>>> can be interpreted as keywords (which per JLS can only happen inside
>>> a ModuleDeclaration).
>>> All this means, that functionality like code completion is
>>> systematically broken in a language using "restricted keywords".
>>>
>>> (3.c) Other IDE functionality assumes that small fragments of the
>>> input text can be scanned out of context. The classical example here
>>> is syntax highlighting but there are more examples.
>>> Any such functionality has to be re-implemented, replacing the
>>> highly efficient local scanning with full parsing of the input text.
>>> For functionality that is implicitly invoked per keystroke, or on
>>> mouse hover etc, this difference in efficiency negatively affects
>>> the overall user experience of an IDE.
>>>
>>>
>>> (4) The following proposal avoids all difficulties described above:
>>>
>>> * open, module, requires, transitive, exports, opens, to, uses,
>>>     provides, and with are "module words", to which the following
>>>     interpretation is applied:
>>>     * within any ordinary compilation unit, a module word is a
>normal
>>>       identifier.
>>>     * within a modular compilation unit, all module words are
>>>       (unconditional) keywords.
>>> * We introduce three new auxiliary non-terminals:
>>>       LegacyPackageName:
>>>           LegacyIdentifier
>>>           LegacyPackageName . LegacyIdentifier
>>>       LegacyTypeName:
>>>           LegacyIdentifier
>>>           LegacyTypeName . LegacyIdentifier
>>>       LegacyIdentifier:
>>>           Identifier
>>>           ^open
>>>           ^module
>>>           ...
>>>           ^with
>>> * We modify all productions in 7.7, replacing PackageName with
>>>    LegacyPackageName and replacing TypeName with LegacyTypeName.
>>> * After parsing, each of the words '^open', '^module' etc.
>>>    is interpreted by removing the leading '^' (escape character).
>>>
>>> Here, '^' is chosen as the escape character following the precedent
>>> of Xtext. Plenty of other options for this purpose are possible,
>too.
>>>
>>>
>>>
>>> This proposal completely satisfies the requirements (1), and avoids
>>> all of the problems (2) and (3). There's an obvious price to pay:
>>> users will have to add the escape character when referring to code
>>> that uses a module word as a package name or type name.
>>>
>>> Not only is this a very low price compared to the benefits; one can
>>> even argue that it also helps the human reader of a module
>declaration,
>>> because it clearly marks which occurrences of a module word are
>indeed
>>> identifiers.
>>>
>>> An IDE can easily help in interactively adding escapes where
>necessary.
>>>
>>> Finally, in this trade-off it is relevant to consider the expected
>>> frequencies: legacy names (needing escape) will surely be the
>exception
>>> - by magnitudes. So, the little price needing to be paid, will only
>>> affect a comparatively small number of locations.
>>>
>>>
>>> Stephan

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

From rfscholte at apache.org  Fri May 12 20:27:24 2017
From: rfscholte at apache.org (Robert Scholte)
Date: Fri, 12 May 2017 22:27:24 +0200
Subject: Improve JPMS specifications
Message-ID: <op.yz4632nfkdkhrr@desktop-2khsk44>

In order to get more consensus within the EG I like to reach out regarding
to the specification descriptions in general.

# When to use Jigsaw

You could in some way say that Jigsaw introduces new rules during the game.
In the original game, having split packages could be a wise decision, just
like having an interface and its default implementation within the same  
package.
Rearranging classes over jars could result in duplicate classes or missing  
classes
after the dependency resolution for the artifacts on the  
classpath/modulepath.
It is likely that projects cannot move towards modularization and the  
message should
be that this is fine. The advice should be that *new* projects should try  
to
follow these new rules and depending on the state of its dependencies  
choose
when to add module descriptors.

With this assumption I think we should go for the more strict, though  
still usable
solution right now until it is proven that things are too strict. Up until  
now the
concerns are mainly based on the way of development done now  without  
restrictions
as they are defined by Jigsaw and the experience we have with it.

It must be made clear that it is not always possible to switch to a modular
project due to changed rules, which is fine. However, new next generation
Java applications should be advised to start as modular Java project or at
least keep in mind the new specifications.

# Provide explicit descriptions regarding applications and libraries

The way applications and libraries are developed are quite different, just  
like the
advice how to apply the modular system. It would help readers a lot if the  
specs mention these concepts separately.

e.g. public libraries should only refer to modular jars and to automodules  
with a Automatic-Module-Name attribute in the MANIFEST file. In other  
cases don't add a module descriptor.
The attribute Automatic-Module-Name should *only* be used if the Java  
project is JPMS-ready but can't provide a module descriptor yet.
For applications it doesn't make sense to add the Automatic-Module-Name to  
the MANIFEST file.
For application it doesn't make sense to export packages.

# Avoid the term "migration"

Migration suggest there's a path to slowly adopt Jigsaw functionality, but
IMHO there's not. You simply choose between using or not using.
(Public) Library development will still need the bottom up approach before  
being able to become modular.
Application development can make use automodules.

So the returning term "easy of migration" doesn't sounds correct anymore  
to me. Auto modules are there to make your application compile/run with a  
module descriptor.
Libraries still follow the bottom up principal and require additional  
action from their dependency maintainers, either by providing a jar with a  
module descriptor or a Automatic-Module-Name.

These are some of the topics I'd like to see improved

thanks,
Robert

From tprintezis at twitter.com  Fri May 12 20:38:39 2017
From: tprintezis at twitter.com (Tony Printezis)
Date: Fri, 12 May 2017 13:38:39 -0700
Subject: A compromise approach on module isolation
In-Reply-To: <391853ee-966e-f670-9003-b7f720cd9b19@redhat.com>
References: <e57dc552-d365-156b-1307-245dff80e656@redhat.com>
 <142082553.2488054.1494577688036.JavaMail.zimbra@u-pem.fr>
 <35f5ab53-43ff-0645-09f6-dab09c2a2c86@redhat.com>
 <79d0798a-cece-b81e-0d2d-ef8f2369084b@redhat.com>
 <A7891D21-7470-41F8-976C-14C6671EFB87@univ-mlv.fr>
 <391853ee-966e-f670-9003-b7f720cd9b19@redhat.com>
Message-ID: <CAOzU2imUx9TwAai+VdocxaRjJiGSqitd1c3H9mGA3jVkJ11FYA@mail.gmail.com>

David, Remi, and all,

I have a few questions (and a concern),

- This seems like a straightforward change and I?m sure it must have been
proposed in the past. And I guess it was rejected for specific reasons.
What are the gotchas of this approach? I.e. Will it break some code? Will
it require some code to be changed? And will it bake-in some behavior that
we might not want in the long term?

- (With my JVM/GC implementer hat on) I really don?t think launching a
class loader per module is a great idea. There are overheads in the JVM for
each class loader you introduce (performance - the GC has to do some extra
work per class loader, footprint - there are data structures maintained per
class loader and space could be wasted due to fragmentation in the meta
space of each class loader). FWIW, at Twitter, most of our services do not
use custom class loaders and we get a lot of benefit out of that. It?d be a
step backwards for us if somehow our JVMs got bloated with potentially
1,000s of class loaders.

- Using class loaders is one way to isolate modules. Has anyone considered
alternatives? For example, doing automatic shading of non-exported packages
in the JVM (lots of hand-waving here)?

Tony


?????
Tony Printezis | @TonyPrintezis | tprintezis at twitter.com


On May 12, 2017 at 2:56:40 PM, David M. Lloyd (david.lloyd at redhat.com)
wrote:

It seems to work in my tests. Can you explain what you mean?

$ jlink --module-path $JAVA_HOME/jmods:out/production --add-modules
org.module.two,org.module.one --output new-test
-J-Djdk.module.detect-cycles=false -J-Djdk.module.isolated=true
--launcher one=org.module.one/org.module.one.Main --launcher
two=org.module.two/org.module.two.Main
$ new-test/bin/two
This is module two, my
cl=jdk.internal.loader.ClassLoaders$AppModuleClassLoader at 2b05039f

On 05/12/2017 01:54 PM, Remi Forax wrote:
> It doesn't work because application modules becomes system modules,
> so even if you propagate the flags, an application modules becomes a
> system module hence is not deployed in a class loader anymore.
>
> R?mi
>
>
> On May 12, 2017 6:45:18 PM GMT+02:00, "David M. Lloyd"
> <david.lloyd at redhat.com> wrote:
>
> Here's a commit which does so (and also covers the circularity case), if
> you want to play with it a bit:
>
>
https://github.com/dmlloyd/openjdk-modules/commit/ffbca0555aeb934181ac92ec3a2bf48fae7efe04
>
> On 05/12/2017 08:45 AM, David M. Lloyd wrote:
>
> Hi R?mi, thanks for the feedback.
>
> At the moment you can fix this by editing the launcher scripts and
> adding "-Djdk.module.isolated=true" to the end of the
> "JLINK_VM_OPTIONS=" line (and similarly for Windows).
>
> I think the tool can do this automatically with only minimal
> changes.
> I'll have a look at it.
>
> On 05/12/2017 03:28 AM, Remi Forax wrote:
>
> Hi David,
> with this patch, there is an issue if people (like me :) )
> are using
> jlink.
>
> With jlink, application modules becomes system modules so
> depending if
> the application is delivered as a bunch of jars or as a
> custom jlink
> image, so behavior will be different because the application
> modules
> deploy with jlink will not be sandboxed.
>
> Maybe it means that jlink has to be changed, i do not know,
> but i
> think i would prefer that the module itself should decide
> using the
> module-info if it should run its own classloader or not, at
> least
> jlink can refuse to work if a module ask for being deployed
> in its own
> classloader
>
> And i'm still not sure that this kind of code has to be in
> the jdk, it
> can be easily written outside without putting the burden of
> having to
> maintain it in the JDK.
>
> regards,
> R?mi
>
>
> ----- Mail original -----
>
> De: "David M. Lloyd" <david.lloyd at redhat.com>
> ?: jpms-spec-experts at openjdk.java.net
> Envoy?: Vendredi 12 Mai 2017 01:17:45
> Objet: A compromise approach on module isolation
>
>
> In the wake of the PR vote, one of the most oft-cited
> unmet expectations
> was the lack of package namespace isolation among
> modules. I'd like to
> propose a patch [1] which would allow this type of
> module isolation to
> be used by the run time on an opt-in or opt-out basis.
>
> Justification:
>
> ? Large applications are more likely to encounter
> concealed and
> non-concealed package conflicts.
> ? By making the change opt-in or opt-out, any
> application which expects
> all of its modules to be loaded by a single class loader
> can still
> function, while allowing more complex applications
> (which already would
> necessarily have required at least some separated class
> loaders) to
> continue to work.
> ? The patch seems to have a small complexity and
> maintenance footprint.
>
> Drawbacks:
>
> None that I am aware of.
>
> Patch information:
>
> The approach of this patch simply looks for a system
> property and,
> depending on the value of this property, establishes
> isolated module
> loaders. I relinquish any copyright claim to the patch
> in this mail.
> Please do not get bogged down by any formatting problems
> introduced by
> the mailing list; the purpose of directly including the
> patch is to give
> a clear, unambiguous subject for discussion. I can
> provide a proper
> webrev (or whatever other form is requested) if needed.
>
> Other than the system property (which is just one
> possible approach), no
> API or specification changes should be needed. This
> version uses the
> property "jdk.module.isolated" defaulting to "false".
>
> [1]:
>
> diff --git
> a/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
> <http://ClassLoaders.java>
> b/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
> <http://ClassLoaders.java>
> index d976aab..f06adfb 100644
> ---
> a/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
> <http://ClassLoaders.java>
> +++
> b/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
> <http://ClassLoaders.java>
> @@ -112,6 +112,15 @@ public static ClassLoader
> appClassLoader() {
> }
>
> /**
> + * Returns a new isolated application module class loader.
> + *
> + * @return the new application module class loader (not
> {@code
> null})
> + */
> + public static ClassLoader appModuleClassLoader() {
> + return new AppModuleClassLoader(APP_LOADER);
> + }
> +
> + /**
> * The class loader that is used to find resources in modules
> defined to
> * the boot class loader. It is not used for class loading.
> */
> @@ -220,6 +229,21 @@ protected Package
> defineOrCheckPackage(String pn,
> Manifest man, URL url) {
> }
>
> /**
> + * An isolated application module class loader that is
> a {@code
> BuiltinClassLoader} with
> + * the application class loader as parent.
> + */
> + private static class AppModuleClassLoader extends
> BuiltinClassLoader {
> + static {
> + if (!ClassLoader.registerAsParallelCapable())
> + throw new InternalError();
> + }
> +
> + AppModuleClassLoader(AppClassLoader parent) {
> + super("app", parent, null);
> + }
> + }
> +
> + /**
> * Returns a {@code URLClassPath} of file URLs to each of the
> elements in
> * the given class path.
> */
> diff --git
>
a/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java
> <http://ModuleLoaderMap.java>
>
>
b/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java
> <http://ModuleLoaderMap.java>
>
> index b0f465d..cc2bdb6 100644
> ---
>
a/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java
> <http://ModuleLoaderMap.java>
>
> +++
>
b/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java
> <http://ModuleLoaderMap.java>
>
> @@ -85,6 +85,8 @@ public ClassLoader apply(String name) {
> return platformModules;
> }
>
> + private static final boolean ISOLATED_MODULE_PATH =
> Boolean.parseBoolean(System.getProperty("jdk.module.isolated",
>
> "false"));
> +
> /**
> * Returns the function to map modules in the given
> configuration
> to the
> * built-in class loaders.
> @@ -102,6 +104,8 @@ public ClassLoader apply(String name) {
> if (!bootModules.contains(mn)) {
> if (platformModules.contains(mn)) {
> map.put(mn, platformClassLoader);
> + } else if (ISOLATED_MODULE_PATH) {
> + map.put(mn, ClassLoaders.appModuleClassLoader());
> } else {
> map.put(mn, appClassLoader);
> }
>
>
> --
> - DML
>
>
>
>
>
> --
> Sent from my Android device with K-9 Mail. Please excuse my brevity.


-- 
- DML

From rfscholte at apache.org  Fri May 12 20:40:04 2017
From: rfscholte at apache.org (Robert Scholte)
Date: Fri, 12 May 2017 22:40:04 +0200
Subject: Jigsaw specific commandline arguments
Message-ID: <op.yz47o2rmkdkhrr@desktop-2khsk44>

A lot of type commandline arguments have been introduced at the beginning
of the implementation. With the experience we have reach now I would
advise to go over all the arguments again.

For instance: for libraries I would prefer not to add specific arguments
because might mean that all its users need to do the same.

We also shouldn't over-complicate things:
Accept that not all Java projects can be transformed to a modular project  
(e.g. it is very likely that Maven cannot become modular) and avoid  
command line arguments which still try to help with this.

Now would be the correct moment to look at these extra arguments and  
verify if they all makes sense. Keep in mind that adding options is easier  
compared to removing options.

This brings me to an extra feature request:

# Embed jigsaw specific arguments (e.g. inside compiled module descriptor)

One of the issues I recognize is that during compilation of a library some
additional arguments are added to make it compile. It is very likely that
some or all of these arguments are required at runtime as well when used
by an application. To improve the adoption I would advice to store JPMS  
specific
information in the module descriptor, something we could access in the
future.

Up until now most javac arguments are standalone and their effect are  
often good visible (source/target/release option, -g flag). Apart from *a*  
classpath their values don't have to be repeated to make it run. For the  
JPMS specific arguments it is a different story. Especially when using  
libraries which have been compiled with extra flags it would be good to  
know there original arguments.

thanks,
Robert

From david.lloyd at redhat.com  Fri May 12 21:09:54 2017
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Fri, 12 May 2017 16:09:54 -0500
Subject: A compromise approach on module isolation
In-Reply-To: <CAOzU2imUx9TwAai+VdocxaRjJiGSqitd1c3H9mGA3jVkJ11FYA@mail.gmail.com>
References: <e57dc552-d365-156b-1307-245dff80e656@redhat.com>
 <142082553.2488054.1494577688036.JavaMail.zimbra@u-pem.fr>
 <35f5ab53-43ff-0645-09f6-dab09c2a2c86@redhat.com>
 <79d0798a-cece-b81e-0d2d-ef8f2369084b@redhat.com>
 <A7891D21-7470-41F8-976C-14C6671EFB87@univ-mlv.fr>
 <391853ee-966e-f670-9003-b7f720cd9b19@redhat.com>
 <CAOzU2imUx9TwAai+VdocxaRjJiGSqitd1c3H9mGA3jVkJ11FYA@mail.gmail.com>
Message-ID: <ef982b92-f958-6728-e8ae-f12b91245504@redhat.com>

On 05/12/2017 03:38 PM, Tony Printezis wrote:
> David, Remi, and all,
> 
> I have a few questions (and a concern),
> 
> - This seems like a straightforward change and I?m sure it must have 
> been proposed in the past. And I guess it was rejected for specific 
> reasons. What are the gotchas of this approach? I.e. Will it break some 
> code? Will it require some code to be changed?

Hi Tony,

Theoretically, any code which assumes that the class loader is going to 
be the same across modules will be disappointed if this setting is 
enabled.  On the other hand, there are no modules yet, so one would hope 
that this would not be a problem; and also, the feature is switchable.

> And will it bake-in some behavior that we might not want in the long term?

As long as it is switchable (and possibly non-normative), it's 
definitely impossible.  Otherwise I don't know of any specific case 
where this would create a long-term issue (unless, hypothetically, it is 
a goal of the project to, say, eliminate ClassLoader altogether someday, 
or if some equally drastic long-term plan is in the works that I don't 
know of).

> - (With my JVM/GC implementer hat on) I really don?t think launching a 
> class loader per module is a great idea. There are overheads in the JVM 
> for each class loader you introduce (performance - the GC has to do some 
> extra work per class loader, footprint - there are data structures 
> maintained per class loader and space could be wasted due to 
> fragmentation in the meta space of each class loader). FWIW, at Twitter, 
> most of our services do not use custom class loaders and we get a lot of 
> benefit out of that. It?d be a step backwards for us if somehow our JVMs 
> got bloated with potentially 1,000s of class loaders.

It would be a tradeoff, for sure.  We don't see too much negative impact 
derived from having many class loaders, but I agree that it is possible 
that a large number of them might cause an issue.  I'm not sure that 
adding N class loaders on to N modules (which also have internal data 
structures and classes associated with them) will create a large % 
increase though; it would have to be tested.  And of course it's 
switchable in the event that isolated package spaces are not needed.

If there is in fact an impacting cost difference, another variation 
might be to allow isolation to be configured on a per-module basis 
(maybe using a module attribute or annotation, or a more complex 
command-line argument arrangement), or to allow the establishment of 
isolated groups, or other similar modifications.  I believe these 
options should all be doable in a short time frame as well.

> - Using class loaders is one way to isolate modules. Has anyone 
> considered alternatives? For example, doing automatic shading of 
> non-exported packages in the JVM (lots of hand-waving here)?


One can always shade at build time, though this may be non-ideal for 
other reasons.  This type of shading could be done at run time as you 
suggest, maybe using bytecode transformation, but that might be 
confusing for stack traces and so forth, and could possibly introduce 
other problems where class and package names vary unexpectedly.  Still 
it could be a viable solution in some cases.

Using class loaders is the only solution I can think of that doesn't 
introduce substantial overhead or involve rewriting the JVM's class 
loader implementation to redefine how it indexes packages (and updating 
the JVM and platform specs accordingly).  At some point, adding 
complexity to reduce overhead may become zero-sum.  :-)  But I am of 
course open to other ideas as well.

> On May 12, 2017 at 2:56:40 PM, David M. Lloyd (david.lloyd at redhat.com 
> <mailto:david.lloyd at redhat.com>) wrote:
> 
>> It seems to work in my tests. Can you explain what you mean?
>>
>> $ jlink --module-path $JAVA_HOME/jmods:out/production --add-modules
>> org.module.two,org.module.one --output new-test
>> -J-Djdk.module.detect-cycles=false -J-Djdk.module.isolated=true
>> --launcher one=org.module.one/org.module.one.Main --launcher
>> two=org.module.two/org.module.two.Main
>> $ new-test/bin/two
>> This is module two, my
>> cl=jdk.internal.loader.ClassLoaders$AppModuleClassLoader at 2b05039f
>>
>> On 05/12/2017 01:54 PM, Remi Forax wrote:
>> > It doesn't work because application modules becomes system modules,
>> > so even if you propagate the flags, an application modules becomes a  
>> > system module hence is not deployed in a class loader anymore.
>> >  
>> > R?mi
>> >  
>> >  
>> > On May 12, 2017 6:45:18 PM GMT+02:00, "David M. Lloyd"  
>> > <david.lloyd at redhat.com <mailto:david.lloyd at redhat.com>> wrote:
>> >  
>> >     Here's a commit which does so (and also covers the circularity case), if
>> >     you want to play with it a bit:
>> >  
>> >     https://github.com/dmlloyd/openjdk-modules/commit/ffbca0555aeb934181ac92ec3a2bf48fae7efe04 
>>
>> >  
>> >     On 05/12/2017 08:45 AM, David M. Lloyd wrote:
>> >  
>> >         Hi R?mi, thanks for the feedback.
>> >  
>> >         At the moment you can fix this by editing the launcher scripts and
>> >         adding "-Djdk.module.isolated=true" to the end of the
>> >         "JLINK_VM_OPTIONS=" line (and similarly for Windows).
>> >  
>> >         I think the tool can do this automatically with only minimal
>> >         changes.
>> >         I'll have a look at it.
>> >  
>> >         On 05/12/2017 03:28 AM, Remi Forax wrote:
>> >  
>> >             Hi David,
>> >             with this patch, there is an issue if people (like me :) )
>> >             are using
>> >             jlink.
>> >  
>> >             With jlink, application modules becomes system modules so
>> >             depending if
>> >             the application is delivered as a bunch of jars or as a
>> >             custom jlink
>> >             image, so behavior will be different because the application
>> >             modules
>> >             deploy with jlink will not be sandboxed.
>> >  
>> >             Maybe it means that jlink has to be changed, i do not know,
>> >             but i
>> >             think i would prefer that the module itself should decide
>> >             using the
>> >             module-info if it should run its own classloader or not, at
>> >             least
>> >             jlink can refuse to work if a module ask for being deployed
>> >             in its own
>> >             classloader
>> >  
>> >             And i'm still not sure that this kind of code has to be in
>> >             the jdk, it
>> >             can be easily written outside without putting the burden of
>> >             having to
>> >             maintain it in the JDK.
>> >  
>> >             regards,
>> >             R?mi
>> >  
>> >  
>> >             ----- Mail original -----
>> >  
>> >                 De: "David M. Lloyd" <david.lloyd at redhat.com <mailto:david.lloyd at redhat.com>>
>> >                 ?: jpms-spec-experts at openjdk.java.net 
>> <mailto:jpms-spec-experts at openjdk.java.net>
>> >                 Envoy?: Vendredi 12 Mai 2017 01:17:45
>> >                 Objet: A compromise approach on module isolation
>> >  
>> >  
>> >                 In the wake of the PR vote, one of the most oft-cited
>> >                 unmet expectations
>> >                 was the lack of package namespace isolation among
>> >                 modules. I'd like to
>> >                 propose a patch [1] which would allow this type of
>> >                 module isolation to
>> >                 be used by the run time on an opt-in or opt-out basis.
>> >  
>> >                 Justification:
>> >  
>> >                 ? Large applications are more likely to encounter
>> >                 concealed and
>> >                 non-concealed package conflicts.
>> >                 ? By making the change opt-in or opt-out, any
>> >                 application which expects
>> >                 all of its modules to be loaded by a single class loader
>> >                 can still
>> >                 function, while allowing more complex applications
>> >                 (which already would
>> >                 necessarily have required at least some separated class
>> >                 loaders) to
>> >                 continue to work.
>> >                 ? The patch seems to have a small complexity and
>> >                 maintenance footprint.
>> >  
>> >                 Drawbacks:
>> >  
>> >                 None that I am aware of.
>> >  
>> >                 Patch information:
>> >  
>> >                 The approach of this patch simply looks for a system
>> >                 property and,
>> >                 depending on the value of this property, establishes
>> >                 isolated module
>> >                 loaders. I relinquish any copyright claim to the patch
>> >                 in this mail.
>> >                 Please do not get bogged down by any formatting problems
>> >                 introduced by
>> >                 the mailing list; the purpose of directly including the
>> >                 patch is to give
>> >                 a clear, unambiguous subject for discussion. I can
>> >                 provide a proper
>> >                 webrev (or whatever other form is requested) if needed.
>> >  
>> >                 Other than the system property (which is just one
>> >                 possible approach), no
>> >                 API or specification changes should be needed. This
>> >                 version uses the
>> >                 property "jdk.module.isolated" defaulting to "false".
>> >  
>> >                 [1]:
>> >  
>> >                 diff --git
>> >                 a/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
>> >                 <http://ClassLoaders.java>
>> >                 b/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
>> >                 <http://ClassLoaders.java>
>> >                 index d976aab..f06adfb 100644
>> >                 ---
>> >                 a/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
>> >                 <http://ClassLoaders.java>
>> >                 +++
>> >                 b/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
>> >                 <http://ClassLoaders.java>
>> >                 @@ -112,6 +112,15 @@ public static ClassLoader
>> >                 appClassLoader() {
>> >                 }
>> >  
>> >                 /**
>> >                 + * Returns a new isolated application module class loader.
>> >                 + *
>> >                 + * @return the new application module class loader (not
>> >                 {@code
>> >                 null})
>> >                 + */
>> >                 + public static ClassLoader appModuleClassLoader() {
>> >                 + return new AppModuleClassLoader(APP_LOADER);
>> >                 + }
>> >                 +
>> >                 + /**
>> >                 * The class loader that is used to find resources in modules
>> >                 defined to
>> >                 * the boot class loader. It is not used for class loading.
>> >                 */
>> >                 @@ -220,6 +229,21 @@ protected Package
>> >                 defineOrCheckPackage(String pn,
>> >                 Manifest man, URL url) {
>> >                 }
>> >  
>> >                 /**
>> >                 + * An isolated application module class loader that is
>> >                 a {@code
>> >                 BuiltinClassLoader} with
>> >                 + * the application class loader as parent.
>> >                 + */
>> >                 + private static class AppModuleClassLoader extends
>> >                 BuiltinClassLoader {
>> >                 + static {
>> >                 + if (!ClassLoader.registerAsParallelCapable())
>> >                 + throw new InternalError();
>> >                 + }
>> >                 +
>> >                 + AppModuleClassLoader(AppClassLoader parent) {
>> >                 + super("app", parent, null);
>> >                 + }
>> >                 + }
>> >                 +
>> >                 + /**
>> >                 * Returns a {@code URLClassPath} of file URLs to each of the
>> >                 elements in
>> >                 * the given class path.
>> >                 */
>> >                 diff --git
>> >                 a/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java
>> >                 <http://ModuleLoaderMap.java>
>> >  
>> >                 b/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java
>> >                 <http://ModuleLoaderMap.java>
>> >  
>> >                 index b0f465d..cc2bdb6 100644
>> >                 ---
>> >                 a/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java
>> >                 <http://ModuleLoaderMap.java>
>> >  
>> >                 +++
>> >                 b/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java
>> >                 <http://ModuleLoaderMap.java>
>> >  
>> >                 @@ -85,6 +85,8 @@ public ClassLoader apply(String name) {
>> >                 return platformModules;
>> >                 }
>> >  
>> >                 + private static final boolean ISOLATED_MODULE_PATH =
>> >                 Boolean.parseBoolean(System.getProperty("jdk.module.isolated",
>> >  
>> >                 "false"));
>> >                 +
>> >                 /**
>> >                 * Returns the function to map modules in the given
>> >                 configuration
>> >                 to the
>> >                 * built-in class loaders.
>> >                 @@ -102,6 +104,8 @@ public ClassLoader apply(String name) {
>> >                 if (!bootModules.contains(mn)) {
>> >                 if (platformModules.contains(mn)) {
>> >                 map.put(mn, platformClassLoader);
>> >                 + } else if (ISOLATED_MODULE_PATH) {
>> >                 + map.put(mn, ClassLoaders.appModuleClassLoader());
>> >                 } else {
>> >                 map.put(mn, appClassLoader);
>> >                 }
>> >  
>> >  
>> >                 --  
>> >                 - DML
>> >  
>> >  
>> >  
>> >  
>> >  
>> > --  
>> > Sent from my Android device with K-9 Mail. Please excuse my brevity.
>>
>>
>> -- 
>> - DML


-- 
- DML

From volker.berlin at goebel-clan.de  Mon May 15 19:38:54 2017
From: volker.berlin at goebel-clan.de (Volker Berlin)
Date: Mon, 15 May 2017 21:38:54 +0200
Subject: The critical missing pieces and a path forward
In-Reply-To: <5914DBDA.7000300@oracle.com>
References: <3b52b3b6-d95d-dacf-06ce-8fceaa1a8cf5@redhat.com>
 <20170508145736.531AFC4422@eggemoggin.niobe.net>
 <f503147c-01c0-45b1-583e-3b7a4d3df206@goebel-clan.de>
 <5914DBDA.7000300@oracle.com>
Message-ID: <61d7a6e4-5441-9561-5713-5fa2e0374746@goebel-clan.de>


>
> It sounds like:
>
> - The implementation of "Logging" accesses the API of "Configuration", 
> to determine how much logging has been configured; and
>
> - The implementation of "Configuration" accesses the API of "Logging", 
> to record errors in config files.
>
> The cycle can be broken by refactoring either "Configuration" or 
> "Logging". In the spirit of the recent jigsaw-dev discussion about 
> Log4j [1], let's refactor "Logging" into "LoggingIntf" and "LoggingImpl":
>
> - "LoggingIntf" 'exports' an API for general use, so "Configuration" 
> 'requires' "LoggingIntf".
>
> - "LoggingIntf" also 'exports' an SPI.
>
> - "LoggingImpl" 'requires' "LoggingIntf" in order to gain access to 
> the SPI, and 'provides' an implementation of the SPI.
>
> - "LoggingIntf" 'uses' its own SPI and discovers providers via 
> ServiceLoader.
>
> - It may be that "LoggingImpl" 'requires' many other modules, and they 
> in turn have their own 'requires', eventually leading back to 
> "LoggingIntf", and that's fine. The dependencies of "LoggingImpl" may 
> even have their own 'uses' and 'provides' directives, all of which 
> will be bound in the same fashion as "LoggingImpl"-to-"LoggingIntf".
>
> Alex
>
> [1] 
> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012562.html
>
Thanks for the answer. What you suggest is:

  * Splitting modules in interfaces and implementation. This can explode
    the count of modules in a complex application from 300 modules to
    1000 modules. You lost maintainability.
  * In your construct you lost also the declaration of runtime
    dependency to "LoggingImpl". This dependency exists in real. You
    have only removed it from compile time. The application will not
    work if "LoggingImpl" is not present. This reduce also the
    maintainability.

This work nice if you have only the 2 modules. In real live application 
with 300 modules and more there can be 2-3 thousands dependencies. 
Without a full declared graph of runtime dependencies you can't manage 
it anymore.

Volker


From david.lloyd at redhat.com  Mon May 15 14:14:33 2017
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Mon, 15 May 2017 09:14:33 -0500
Subject: A compromise approach on module isolation
In-Reply-To: <c5b14bf4-47e6-e6ee-fdeb-78fdcd4a2088@redhat.com>
References: <e57dc552-d365-156b-1307-245dff80e656@redhat.com>
 <142082553.2488054.1494577688036.JavaMail.zimbra@u-pem.fr>
 <35f5ab53-43ff-0645-09f6-dab09c2a2c86@redhat.com>
 <79d0798a-cece-b81e-0d2d-ef8f2369084b@redhat.com>
 <A7891D21-7470-41F8-976C-14C6671EFB87@univ-mlv.fr>
 <391853ee-966e-f670-9003-b7f720cd9b19@redhat.com>
 <CAOzU2imUx9TwAai+VdocxaRjJiGSqitd1c3H9mGA3jVkJ11FYA@mail.gmail.com>
 <ef982b92-f958-6728-e8ae-f12b91245504@redhat.com>
 <c5b14bf4-47e6-e6ee-fdeb-78fdcd4a2088@redhat.com>
Message-ID: <fbc44ae4-a88a-0da2-59e0-8a0e39134343@redhat.com>

Corrected link for [2]: 
https://github.com/dmlloyd/openjdk-modules/commit/117a1cd6a8c1c44715

On 05/15/2017 09:12 AM, David M. Lloyd wrote:
> I've added a couple of commits [1] [2] that explore an alternative 
> approach where the property value can be a comma-separated list of 
> module names to isolate, or the special token "ALL" that puts all 
> application modules in isolated class loaders.
> 
> This allows fine-grained control over whether a module is isolated, 
> however it will not always be obvious which modules *need* to be 
> isolated (as the user may have to isolate modules that reference modules 
> with conflicting packages, in some cases).  So I would expect that users 
> who run into isolation problems will use "ALL" unless they have special 
> memory requirements (class loaders are not free, but they're not 
> ginormous either, so for most people and most projects this is probably 
> OK).
> 
> I also considered using a "ALL-MODULE-PATH" token instead as that 
> matches an existing convention, but thought maybe that is a bit more 
> specific than strictly necessary so I went for the shortest option 
> first.  There is a possible weakness in the jlink part that a space in 
> the list could cause argument parsing problems.  I didn't want to spend 
> time fixing that unless the basic concept is validated.
> 
> I envision that this could be translated to an "--isolate-modules=" 
> command line argument.
> 
> [1] https://github.com/dmlloyd/openjdk-modules/commit/941c86f760654eb
> [2] https://github.com/dmlloyd/openjdk-modules/commit/fb645cd681551eb
> 
> On 05/12/2017 04:09 PM, David M. Lloyd wrote:
>> On 05/12/2017 03:38 PM, Tony Printezis wrote:
>>> David, Remi, and all,
>>>
>>> I have a few questions (and a concern),
>>>
>>> - This seems like a straightforward change and I?m sure it must have 
>>> been proposed in the past. And I guess it was rejected for specific 
>>> reasons. What are the gotchas of this approach? I.e. Will it break 
>>> some code? Will it require some code to be changed?
>>
>> Hi Tony,
>>
>> Theoretically, any code which assumes that the class loader is going 
>> to be the same across modules will be disappointed if this setting is 
>> enabled.  On the other hand, there are no modules yet, so one would 
>> hope that this would not be a problem; and also, the feature is 
>> switchable.
>>
>>> And will it bake-in some behavior that we might not want in the long 
>>> term?
>>
>> As long as it is switchable (and possibly non-normative), it's 
>> definitely impossible.  Otherwise I don't know of any specific case 
>> where this would create a long-term issue (unless, hypothetically, it 
>> is a goal of the project to, say, eliminate ClassLoader altogether 
>> someday, or if some equally drastic long-term plan is in the works 
>> that I don't know of).
>>
>>> - (With my JVM/GC implementer hat on) I really don?t think launching 
>>> a class loader per module is a great idea. There are overheads in the 
>>> JVM for each class loader you introduce (performance - the GC has to 
>>> do some extra work per class loader, footprint - there are data 
>>> structures maintained per class loader and space could be wasted due 
>>> to fragmentation in the meta space of each class loader). FWIW, at 
>>> Twitter, most of our services do not use custom class loaders and we 
>>> get a lot of benefit out of that. It?d be a step backwards for us if 
>>> somehow our JVMs got bloated with potentially 1,000s of class loaders.
>>
>> It would be a tradeoff, for sure.  We don't see too much negative 
>> impact derived from having many class loaders, but I agree that it is 
>> possible that a large number of them might cause an issue.  I'm not 
>> sure that adding N class loaders on to N modules (which also have 
>> internal data structures and classes associated with them) will create 
>> a large % increase though; it would have to be tested.  And of course 
>> it's switchable in the event that isolated package spaces are not needed.
>>
>> If there is in fact an impacting cost difference, another variation 
>> might be to allow isolation to be configured on a per-module basis 
>> (maybe using a module attribute or annotation, or a more complex 
>> command-line argument arrangement), or to allow the establishment of 
>> isolated groups, or other similar modifications.  I believe these 
>> options should all be doable in a short time frame as well.
>>
>>> - Using class loaders is one way to isolate modules. Has anyone 
>>> considered alternatives? For example, doing automatic shading of 
>>> non-exported packages in the JVM (lots of hand-waving here)?
>>
>>
>> One can always shade at build time, though this may be non-ideal for 
>> other reasons.  This type of shading could be done at run time as you 
>> suggest, maybe using bytecode transformation, but that might be 
>> confusing for stack traces and so forth, and could possibly introduce 
>> other problems where class and package names vary unexpectedly.  Still 
>> it could be a viable solution in some cases.
>>
>> Using class loaders is the only solution I can think of that doesn't 
>> introduce substantial overhead or involve rewriting the JVM's class 
>> loader implementation to redefine how it indexes packages (and 
>> updating the JVM and platform specs accordingly).  At some point, 
>> adding complexity to reduce overhead may become zero-sum.  :-)  But I 
>> am of course open to other ideas as well.
>>
>>> On May 12, 2017 at 2:56:40 PM, David M. Lloyd (david.lloyd at redhat.com 
>>> <mailto:david.lloyd at redhat.com>) wrote:
>>>
>>>> It seems to work in my tests. Can you explain what you mean?
>>>>
>>>> $ jlink --module-path $JAVA_HOME/jmods:out/production --add-modules
>>>> org.module.two,org.module.one --output new-test
>>>> -J-Djdk.module.detect-cycles=false -J-Djdk.module.isolated=true
>>>> --launcher one=org.module.one/org.module.one.Main --launcher
>>>> two=org.module.two/org.module.two.Main
>>>> $ new-test/bin/two
>>>> This is module two, my
>>>> cl=jdk.internal.loader.ClassLoaders$AppModuleClassLoader at 2b05039f
>>>>
>>>> On 05/12/2017 01:54 PM, Remi Forax wrote:
>>>> > It doesn't work because application modules becomes system modules,
>>>> > so even if you propagate the flags, an application modules becomes 
>>>> a > system module hence is not deployed in a class loader anymore.
>>>> > > R?mi
>>>> > > > On May 12, 2017 6:45:18 PM GMT+02:00, "David M. Lloyd" > 
>>>> <david.lloyd at redhat.com <mailto:david.lloyd at redhat.com>> wrote:
>>>> > >     Here's a commit which does so (and also covers the 
>>>> circularity case), if
>>>> >     you want to play with it a bit:
>>>> > > 
>>>> https://github.com/dmlloyd/openjdk-modules/commit/ffbca0555aeb934181ac92ec3a2bf48fae7efe04 
>>>>
>>>> > >     On 05/12/2017 08:45 AM, David M. Lloyd wrote:
>>>> > >         Hi R?mi, thanks for the feedback.
>>>> > >         At the moment you can fix this by editing the launcher 
>>>> scripts and
>>>> >         adding "-Djdk.module.isolated=true" to the end of the
>>>> >         "JLINK_VM_OPTIONS=" line (and similarly for Windows).
>>>> > >         I think the tool can do this automatically with only 
>>>> minimal
>>>> >         changes.
>>>> >         I'll have a look at it.
>>>> > >         On 05/12/2017 03:28 AM, Remi Forax wrote:
>>>> > >             Hi David,
>>>> >             with this patch, there is an issue if people (like me 
>>>> :) )
>>>> >             are using
>>>> >             jlink.
>>>> > >             With jlink, application modules becomes system 
>>>> modules so
>>>> >             depending if
>>>> >             the application is delivered as a bunch of jars or as a
>>>> >             custom jlink
>>>> >             image, so behavior will be different because the 
>>>> application
>>>> >             modules
>>>> >             deploy with jlink will not be sandboxed.
>>>> > >             Maybe it means that jlink has to be changed, i do 
>>>> not know,
>>>> >             but i
>>>> >             think i would prefer that the module itself should decide
>>>> >             using the
>>>> >             module-info if it should run its own classloader or 
>>>> not, at
>>>> >             least
>>>> >             jlink can refuse to work if a module ask for being 
>>>> deployed
>>>> >             in its own
>>>> >             classloader
>>>> > >             And i'm still not sure that this kind of code has to 
>>>> be in
>>>> >             the jdk, it
>>>> >             can be easily written outside without putting the 
>>>> burden of
>>>> >             having to
>>>> >             maintain it in the JDK.
>>>> > >             regards,
>>>> >             R?mi
>>>> > > >             ----- Mail original -----
>>>> > >                 De: "David M. Lloyd" <david.lloyd at redhat.com 
>>>> <mailto:david.lloyd at redhat.com>>
>>>> >                 ?: jpms-spec-experts at openjdk.java.net 
>>>> <mailto:jpms-spec-experts at openjdk.java.net>
>>>> >                 Envoy?: Vendredi 12 Mai 2017 01:17:45
>>>> >                 Objet: A compromise approach on module isolation
>>>> > > >                 In the wake of the PR vote, one of the most 
>>>> oft-cited
>>>> >                 unmet expectations
>>>> >                 was the lack of package namespace isolation among
>>>> >                 modules. I'd like to
>>>> >                 propose a patch [1] which would allow this type of
>>>> >                 module isolation to
>>>> >                 be used by the run time on an opt-in or opt-out 
>>>> basis.
>>>> > >                 Justification:
>>>> > >                 ? Large applications are more likely to encounter
>>>> >                 concealed and
>>>> >                 non-concealed package conflicts.
>>>> >                 ? By making the change opt-in or opt-out, any
>>>> >                 application which expects
>>>> >                 all of its modules to be loaded by a single class 
>>>> loader
>>>> >                 can still
>>>> >                 function, while allowing more complex applications
>>>> >                 (which already would
>>>> >                 necessarily have required at least some separated 
>>>> class
>>>> >                 loaders) to
>>>> >                 continue to work.
>>>> >                 ? The patch seems to have a small complexity and
>>>> >                 maintenance footprint.
>>>> > >                 Drawbacks:
>>>> > >                 None that I am aware of.
>>>> > >                 Patch information:
>>>> > >                 The approach of this patch simply looks for a 
>>>> system
>>>> >                 property and,
>>>> >                 depending on the value of this property, establishes
>>>> >                 isolated module
>>>> >                 loaders. I relinquish any copyright claim to the 
>>>> patch
>>>> >                 in this mail.
>>>> >                 Please do not get bogged down by any formatting 
>>>> problems
>>>> >                 introduced by
>>>> >                 the mailing list; the purpose of directly 
>>>> including the
>>>> >                 patch is to give
>>>> >                 a clear, unambiguous subject for discussion. I can
>>>> >                 provide a proper
>>>> >                 webrev (or whatever other form is requested) if 
>>>> needed.
>>>> > >                 Other than the system property (which is just one
>>>> >                 possible approach), no
>>>> >                 API or specification changes should be needed. This
>>>> >                 version uses the
>>>> >                 property "jdk.module.isolated" defaulting to "false".
>>>> > >                 [1]:
>>>> > >                 diff --git
>>>> > 
>>>> a/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
>>>> >                 <http://ClassLoaders.java>
>>>> > 
>>>> b/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
>>>> >                 <http://ClassLoaders.java>
>>>> >                 index d976aab..f06adfb 100644
>>>> >                 ---
>>>> > 
>>>> a/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
>>>> >                 <http://ClassLoaders.java>
>>>> >                 +++
>>>> > 
>>>> b/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
>>>> >                 <http://ClassLoaders.java>
>>>> >                 @@ -112,6 +112,15 @@ public static ClassLoader
>>>> >                 appClassLoader() {
>>>> >                 }
>>>> > >                 /**
>>>> >                 + * Returns a new isolated application module 
>>>> class loader.
>>>> >                 + *
>>>> >                 + * @return the new application module class 
>>>> loader (not
>>>> >                 {@code
>>>> >                 null})
>>>> >                 + */
>>>> >                 + public static ClassLoader appModuleClassLoader() {
>>>> >                 + return new AppModuleClassLoader(APP_LOADER);
>>>> >                 + }
>>>> >                 +
>>>> >                 + /**
>>>> >                 * The class loader that is used to find resources 
>>>> in modules
>>>> >                 defined to
>>>> >                 * the boot class loader. It is not used for class 
>>>> loading.
>>>> >                 */
>>>> >                 @@ -220,6 +229,21 @@ protected Package
>>>> >                 defineOrCheckPackage(String pn,
>>>> >                 Manifest man, URL url) {
>>>> >                 }
>>>> > >                 /**
>>>> >                 + * An isolated application module class loader 
>>>> that is
>>>> >                 a {@code
>>>> >                 BuiltinClassLoader} with
>>>> >                 + * the application class loader as parent.
>>>> >                 + */
>>>> >                 + private static class AppModuleClassLoader extends
>>>> >                 BuiltinClassLoader {
>>>> >                 + static {
>>>> >                 + if (!ClassLoader.registerAsParallelCapable())
>>>> >                 + throw new InternalError();
>>>> >                 + }
>>>> >                 +
>>>> >                 + AppModuleClassLoader(AppClassLoader parent) {
>>>> >                 + super("app", parent, null);
>>>> >                 + }
>>>> >                 + }
>>>> >                 +
>>>> >                 + /**
>>>> >                 * Returns a {@code URLClassPath} of file URLs to 
>>>> each of the
>>>> >                 elements in
>>>> >                 * the given class path.
>>>> >                 */
>>>> >                 diff --git
>>>> > 
>>>> a/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java 
>>>>
>>>> >                 <http://ModuleLoaderMap.java>
>>>> > > 
>>>> b/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java 
>>>>
>>>> >                 <http://ModuleLoaderMap.java>
>>>> > >                 index b0f465d..cc2bdb6 100644
>>>> >                 ---
>>>> > 
>>>> a/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java 
>>>>
>>>> >                 <http://ModuleLoaderMap.java>
>>>> > >                 +++
>>>> > 
>>>> b/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java 
>>>>
>>>> >                 <http://ModuleLoaderMap.java>
>>>> > >                 @@ -85,6 +85,8 @@ public ClassLoader 
>>>> apply(String name) {
>>>> >                 return platformModules;
>>>> >                 }
>>>> > >                 + private static final boolean 
>>>> ISOLATED_MODULE_PATH =
>>>> > Boolean.parseBoolean(System.getProperty("jdk.module.isolated",
>>>> > >                 "false"));
>>>> >                 +
>>>> >                 /**
>>>> >                 * Returns the function to map modules in the given
>>>> >                 configuration
>>>> >                 to the
>>>> >                 * built-in class loaders.
>>>> >                 @@ -102,6 +104,8 @@ public ClassLoader 
>>>> apply(String name) {
>>>> >                 if (!bootModules.contains(mn)) {
>>>> >                 if (platformModules.contains(mn)) {
>>>> >                 map.put(mn, platformClassLoader);
>>>> >                 + } else if (ISOLATED_MODULE_PATH) {
>>>> >                 + map.put(mn, ClassLoaders.appModuleClassLoader());
>>>> >                 } else {
>>>> >                 map.put(mn, appClassLoader);
>>>> >                 }
>>>> > > >                 -- >                 - DML
>>>> > > > > > > -- > Sent from my Android device with K-9 Mail. Please 
>>>> excuse my brevity.
>>>>
>>>>
>>>> -- 
>>>> - DML
>>
>>
> 
> 


-- 
- DML

From Alan.Bateman at oracle.com  Tue May 16 07:32:55 2017
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Tue, 16 May 2017 08:32:55 +0100
Subject: A compromise approach on module isolation
In-Reply-To: <CAOzU2imUx9TwAai+VdocxaRjJiGSqitd1c3H9mGA3jVkJ11FYA@mail.gmail.com>
References: <e57dc552-d365-156b-1307-245dff80e656@redhat.com>
 <142082553.2488054.1494577688036.JavaMail.zimbra@u-pem.fr>
 <35f5ab53-43ff-0645-09f6-dab09c2a2c86@redhat.com>
 <79d0798a-cece-b81e-0d2d-ef8f2369084b@redhat.com>
 <A7891D21-7470-41F8-976C-14C6671EFB87@univ-mlv.fr>
 <391853ee-966e-f670-9003-b7f720cd9b19@redhat.com>
 <CAOzU2imUx9TwAai+VdocxaRjJiGSqitd1c3H9mGA3jVkJ11FYA@mail.gmail.com>
Message-ID: <69b5d4c3-9ffd-2482-5901-ebb163471f01@oracle.com>

On 12/05/2017 21:38, Tony Printezis wrote:

> David, Remi, and all,
>
> I have a few questions (and a concern),
>
> - This seems like a straightforward change and I?m sure it must have been
> proposed in the past. And I guess it was rejected for specific reasons.
> What are the gotchas of this approach? I.e. Will it break some code? Will
> it require some code to be changed? And will it bake-in some behavior that
> we might not want in the long term?
>
> - (With my JVM/GC implementer hat on) I really don?t think launching a
> class loader per module is a great idea. There are overheads in the JVM for
> each class loader you introduce (performance - the GC has to do some extra
> work per class loader, footprint - there are data structures maintained per
> class loader and space could be wasted due to fragmentation in the meta
> space of each class loader). FWIW, at Twitter, most of our services do not
> use custom class loaders and we get a lot of benefit out of that. It?d be a
> step backwards for us if somehow our JVMs got bloated with potentially
> 1,000s of class loaders.
>
> - Using class loaders is one way to isolate modules. Has anyone considered
> alternatives? For example, doing automatic shading of non-exported packages
> in the JVM (lots of hand-waving here)?
>
FWIW, we've been encouraging the use of shading on jigsaw-dev for a long 
time. When it works then it eliminates a lot of the potential for issues 
that otherwise arise when someone takes a copy of a library to include 
in their own library.

In any case, and at a high-level, then this topic of isolation or 
interference can be viewed as a failure of class loaders. Going there 
would be a huge undertaking, esp. when you start to think about 
identity. At least in Project Jigsaw, we chose not to go there.

Using class loaders for isolation is of course possible and has always 
been possible via the API (there is even a convenience method in 
ModuleLayer to do just that). The discussion here is therefore around 
the application module path that you specify to the `java` launcher or 
to the JNI invocation API. So it's mostly a JDK issue rather than a JSR 
376 issue. Alternatives to the current simple implementation have of 
course been explored -- this includes spilling when there are 
conflicting packages or just assigning the explicit modules on the 
module path to their own class loader. As expected, it brings many 
issues to the surface, particularly around the interoperability the 
class path, upgraded modules, qualified exports, and issues with legacy 
APIs (esp. resources) that arise with changes to visibility. These all 
seem like issues to look at in the future and what we have in JDK 9 
today doesn't preclude this.

-Alan

From david.lloyd at redhat.com  Mon May 15 14:12:45 2017
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Mon, 15 May 2017 09:12:45 -0500
Subject: A compromise approach on module isolation
In-Reply-To: <ef982b92-f958-6728-e8ae-f12b91245504@redhat.com>
References: <e57dc552-d365-156b-1307-245dff80e656@redhat.com>
 <142082553.2488054.1494577688036.JavaMail.zimbra@u-pem.fr>
 <35f5ab53-43ff-0645-09f6-dab09c2a2c86@redhat.com>
 <79d0798a-cece-b81e-0d2d-ef8f2369084b@redhat.com>
 <A7891D21-7470-41F8-976C-14C6671EFB87@univ-mlv.fr>
 <391853ee-966e-f670-9003-b7f720cd9b19@redhat.com>
 <CAOzU2imUx9TwAai+VdocxaRjJiGSqitd1c3H9mGA3jVkJ11FYA@mail.gmail.com>
 <ef982b92-f958-6728-e8ae-f12b91245504@redhat.com>
Message-ID: <c5b14bf4-47e6-e6ee-fdeb-78fdcd4a2088@redhat.com>

I've added a couple of commits [1] [2] that explore an alternative 
approach where the property value can be a comma-separated list of 
module names to isolate, or the special token "ALL" that puts all 
application modules in isolated class loaders.

This allows fine-grained control over whether a module is isolated, 
however it will not always be obvious which modules *need* to be 
isolated (as the user may have to isolate modules that reference modules 
with conflicting packages, in some cases).  So I would expect that users 
who run into isolation problems will use "ALL" unless they have special 
memory requirements (class loaders are not free, but they're not 
ginormous either, so for most people and most projects this is probably OK).

I also considered using a "ALL-MODULE-PATH" token instead as that 
matches an existing convention, but thought maybe that is a bit more 
specific than strictly necessary so I went for the shortest option 
first.  There is a possible weakness in the jlink part that a space in 
the list could cause argument parsing problems.  I didn't want to spend 
time fixing that unless the basic concept is validated.

I envision that this could be translated to an "--isolate-modules=" 
command line argument.

[1] https://github.com/dmlloyd/openjdk-modules/commit/941c86f760654eb
[2] https://github.com/dmlloyd/openjdk-modules/commit/fb645cd681551eb

On 05/12/2017 04:09 PM, David M. Lloyd wrote:
> On 05/12/2017 03:38 PM, Tony Printezis wrote:
>> David, Remi, and all,
>>
>> I have a few questions (and a concern),
>>
>> - This seems like a straightforward change and I?m sure it must have 
>> been proposed in the past. And I guess it was rejected for specific 
>> reasons. What are the gotchas of this approach? I.e. Will it break 
>> some code? Will it require some code to be changed?
> 
> Hi Tony,
> 
> Theoretically, any code which assumes that the class loader is going to 
> be the same across modules will be disappointed if this setting is 
> enabled.  On the other hand, there are no modules yet, so one would hope 
> that this would not be a problem; and also, the feature is switchable.
> 
>> And will it bake-in some behavior that we might not want in the long 
>> term?
> 
> As long as it is switchable (and possibly non-normative), it's 
> definitely impossible.  Otherwise I don't know of any specific case 
> where this would create a long-term issue (unless, hypothetically, it is 
> a goal of the project to, say, eliminate ClassLoader altogether someday, 
> or if some equally drastic long-term plan is in the works that I don't 
> know of).
> 
>> - (With my JVM/GC implementer hat on) I really don?t think launching a 
>> class loader per module is a great idea. There are overheads in the 
>> JVM for each class loader you introduce (performance - the GC has to 
>> do some extra work per class loader, footprint - there are data 
>> structures maintained per class loader and space could be wasted due 
>> to fragmentation in the meta space of each class loader). FWIW, at 
>> Twitter, most of our services do not use custom class loaders and we 
>> get a lot of benefit out of that. It?d be a step backwards for us if 
>> somehow our JVMs got bloated with potentially 1,000s of class loaders.
> 
> It would be a tradeoff, for sure.  We don't see too much negative impact 
> derived from having many class loaders, but I agree that it is possible 
> that a large number of them might cause an issue.  I'm not sure that 
> adding N class loaders on to N modules (which also have internal data 
> structures and classes associated with them) will create a large % 
> increase though; it would have to be tested.  And of course it's 
> switchable in the event that isolated package spaces are not needed.
> 
> If there is in fact an impacting cost difference, another variation 
> might be to allow isolation to be configured on a per-module basis 
> (maybe using a module attribute or annotation, or a more complex 
> command-line argument arrangement), or to allow the establishment of 
> isolated groups, or other similar modifications.  I believe these 
> options should all be doable in a short time frame as well.
> 
>> - Using class loaders is one way to isolate modules. Has anyone 
>> considered alternatives? For example, doing automatic shading of 
>> non-exported packages in the JVM (lots of hand-waving here)?
> 
> 
> One can always shade at build time, though this may be non-ideal for 
> other reasons.  This type of shading could be done at run time as you 
> suggest, maybe using bytecode transformation, but that might be 
> confusing for stack traces and so forth, and could possibly introduce 
> other problems where class and package names vary unexpectedly.  Still 
> it could be a viable solution in some cases.
> 
> Using class loaders is the only solution I can think of that doesn't 
> introduce substantial overhead or involve rewriting the JVM's class 
> loader implementation to redefine how it indexes packages (and updating 
> the JVM and platform specs accordingly).  At some point, adding 
> complexity to reduce overhead may become zero-sum.  :-)  But I am of 
> course open to other ideas as well.
> 
>> On May 12, 2017 at 2:56:40 PM, David M. Lloyd (david.lloyd at redhat.com 
>> <mailto:david.lloyd at redhat.com>) wrote:
>>
>>> It seems to work in my tests. Can you explain what you mean?
>>>
>>> $ jlink --module-path $JAVA_HOME/jmods:out/production --add-modules
>>> org.module.two,org.module.one --output new-test
>>> -J-Djdk.module.detect-cycles=false -J-Djdk.module.isolated=true
>>> --launcher one=org.module.one/org.module.one.Main --launcher
>>> two=org.module.two/org.module.two.Main
>>> $ new-test/bin/two
>>> This is module two, my
>>> cl=jdk.internal.loader.ClassLoaders$AppModuleClassLoader at 2b05039f
>>>
>>> On 05/12/2017 01:54 PM, Remi Forax wrote:
>>> > It doesn't work because application modules becomes system modules,
>>> > so even if you propagate the flags, an application modules becomes 
>>> a > system module hence is not deployed in a class loader anymore.
>>> > > R?mi
>>> > > > On May 12, 2017 6:45:18 PM GMT+02:00, "David M. Lloyd" > 
>>> <david.lloyd at redhat.com <mailto:david.lloyd at redhat.com>> wrote:
>>> > >     Here's a commit which does so (and also covers the 
>>> circularity case), if
>>> >     you want to play with it a bit:
>>> > >     
>>> https://github.com/dmlloyd/openjdk-modules/commit/ffbca0555aeb934181ac92ec3a2bf48fae7efe04 
>>>
>>> > >     On 05/12/2017 08:45 AM, David M. Lloyd wrote:
>>> > >         Hi R?mi, thanks for the feedback.
>>> > >         At the moment you can fix this by editing the launcher 
>>> scripts and
>>> >         adding "-Djdk.module.isolated=true" to the end of the
>>> >         "JLINK_VM_OPTIONS=" line (and similarly for Windows).
>>> > >         I think the tool can do this automatically with only minimal
>>> >         changes.
>>> >         I'll have a look at it.
>>> > >         On 05/12/2017 03:28 AM, Remi Forax wrote:
>>> > >             Hi David,
>>> >             with this patch, there is an issue if people (like me :) )
>>> >             are using
>>> >             jlink.
>>> > >             With jlink, application modules becomes system 
>>> modules so
>>> >             depending if
>>> >             the application is delivered as a bunch of jars or as a
>>> >             custom jlink
>>> >             image, so behavior will be different because the 
>>> application
>>> >             modules
>>> >             deploy with jlink will not be sandboxed.
>>> > >             Maybe it means that jlink has to be changed, i do not 
>>> know,
>>> >             but i
>>> >             think i would prefer that the module itself should decide
>>> >             using the
>>> >             module-info if it should run its own classloader or 
>>> not, at
>>> >             least
>>> >             jlink can refuse to work if a module ask for being 
>>> deployed
>>> >             in its own
>>> >             classloader
>>> > >             And i'm still not sure that this kind of code has to 
>>> be in
>>> >             the jdk, it
>>> >             can be easily written outside without putting the 
>>> burden of
>>> >             having to
>>> >             maintain it in the JDK.
>>> > >             regards,
>>> >             R?mi
>>> > > >             ----- Mail original -----
>>> > >                 De: "David M. Lloyd" <david.lloyd at redhat.com 
>>> <mailto:david.lloyd at redhat.com>>
>>> >                 ?: jpms-spec-experts at openjdk.java.net 
>>> <mailto:jpms-spec-experts at openjdk.java.net>
>>> >                 Envoy?: Vendredi 12 Mai 2017 01:17:45
>>> >                 Objet: A compromise approach on module isolation
>>> > > >                 In the wake of the PR vote, one of the most 
>>> oft-cited
>>> >                 unmet expectations
>>> >                 was the lack of package namespace isolation among
>>> >                 modules. I'd like to
>>> >                 propose a patch [1] which would allow this type of
>>> >                 module isolation to
>>> >                 be used by the run time on an opt-in or opt-out basis.
>>> > >                 Justification:
>>> > >                 ? Large applications are more likely to encounter
>>> >                 concealed and
>>> >                 non-concealed package conflicts.
>>> >                 ? By making the change opt-in or opt-out, any
>>> >                 application which expects
>>> >                 all of its modules to be loaded by a single class 
>>> loader
>>> >                 can still
>>> >                 function, while allowing more complex applications
>>> >                 (which already would
>>> >                 necessarily have required at least some separated 
>>> class
>>> >                 loaders) to
>>> >                 continue to work.
>>> >                 ? The patch seems to have a small complexity and
>>> >                 maintenance footprint.
>>> > >                 Drawbacks:
>>> > >                 None that I am aware of.
>>> > >                 Patch information:
>>> > >                 The approach of this patch simply looks for a system
>>> >                 property and,
>>> >                 depending on the value of this property, establishes
>>> >                 isolated module
>>> >                 loaders. I relinquish any copyright claim to the patch
>>> >                 in this mail.
>>> >                 Please do not get bogged down by any formatting 
>>> problems
>>> >                 introduced by
>>> >                 the mailing list; the purpose of directly including 
>>> the
>>> >                 patch is to give
>>> >                 a clear, unambiguous subject for discussion. I can
>>> >                 provide a proper
>>> >                 webrev (or whatever other form is requested) if 
>>> needed.
>>> > >                 Other than the system property (which is just one
>>> >                 possible approach), no
>>> >                 API or specification changes should be needed. This
>>> >                 version uses the
>>> >                 property "jdk.module.isolated" defaulting to "false".
>>> > >                 [1]:
>>> > >                 diff --git
>>> >                 
>>> a/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
>>> >                 <http://ClassLoaders.java>
>>> >                 
>>> b/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
>>> >                 <http://ClassLoaders.java>
>>> >                 index d976aab..f06adfb 100644
>>> >                 ---
>>> >                 
>>> a/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
>>> >                 <http://ClassLoaders.java>
>>> >                 +++
>>> >                 
>>> b/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java
>>> >                 <http://ClassLoaders.java>
>>> >                 @@ -112,6 +112,15 @@ public static ClassLoader
>>> >                 appClassLoader() {
>>> >                 }
>>> > >                 /**
>>> >                 + * Returns a new isolated application module class 
>>> loader.
>>> >                 + *
>>> >                 + * @return the new application module class loader 
>>> (not
>>> >                 {@code
>>> >                 null})
>>> >                 + */
>>> >                 + public static ClassLoader appModuleClassLoader() {
>>> >                 + return new AppModuleClassLoader(APP_LOADER);
>>> >                 + }
>>> >                 +
>>> >                 + /**
>>> >                 * The class loader that is used to find resources 
>>> in modules
>>> >                 defined to
>>> >                 * the boot class loader. It is not used for class 
>>> loading.
>>> >                 */
>>> >                 @@ -220,6 +229,21 @@ protected Package
>>> >                 defineOrCheckPackage(String pn,
>>> >                 Manifest man, URL url) {
>>> >                 }
>>> > >                 /**
>>> >                 + * An isolated application module class loader 
>>> that is
>>> >                 a {@code
>>> >                 BuiltinClassLoader} with
>>> >                 + * the application class loader as parent.
>>> >                 + */
>>> >                 + private static class AppModuleClassLoader extends
>>> >                 BuiltinClassLoader {
>>> >                 + static {
>>> >                 + if (!ClassLoader.registerAsParallelCapable())
>>> >                 + throw new InternalError();
>>> >                 + }
>>> >                 +
>>> >                 + AppModuleClassLoader(AppClassLoader parent) {
>>> >                 + super("app", parent, null);
>>> >                 + }
>>> >                 + }
>>> >                 +
>>> >                 + /**
>>> >                 * Returns a {@code URLClassPath} of file URLs to 
>>> each of the
>>> >                 elements in
>>> >                 * the given class path.
>>> >                 */
>>> >                 diff --git
>>> >                 
>>> a/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java 
>>>
>>> >                 <http://ModuleLoaderMap.java>
>>> > >                 
>>> b/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java 
>>>
>>> >                 <http://ModuleLoaderMap.java>
>>> > >                 index b0f465d..cc2bdb6 100644
>>> >                 ---
>>> >                 
>>> a/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java 
>>>
>>> >                 <http://ModuleLoaderMap.java>
>>> > >                 +++
>>> >                 
>>> b/jdk/src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java 
>>>
>>> >                 <http://ModuleLoaderMap.java>
>>> > >                 @@ -85,6 +85,8 @@ public ClassLoader apply(String 
>>> name) {
>>> >                 return platformModules;
>>> >                 }
>>> > >                 + private static final boolean 
>>> ISOLATED_MODULE_PATH =
>>> >                 
>>> Boolean.parseBoolean(System.getProperty("jdk.module.isolated",
>>> > >                 "false"));
>>> >                 +
>>> >                 /**
>>> >                 * Returns the function to map modules in the given
>>> >                 configuration
>>> >                 to the
>>> >                 * built-in class loaders.
>>> >                 @@ -102,6 +104,8 @@ public ClassLoader apply(String 
>>> name) {
>>> >                 if (!bootModules.contains(mn)) {
>>> >                 if (platformModules.contains(mn)) {
>>> >                 map.put(mn, platformClassLoader);
>>> >                 + } else if (ISOLATED_MODULE_PATH) {
>>> >                 + map.put(mn, ClassLoaders.appModuleClassLoader());
>>> >                 } else {
>>> >                 map.put(mn, appClassLoader);
>>> >                 }
>>> > > >                 -- >                 - DML
>>> > > > > > > -- > Sent from my Android device with K-9 Mail. Please 
>>> excuse my brevity.
>>>
>>>
>>> -- 
>>> - DML
> 
> 


-- 
- DML

From alex.buckley at oracle.com  Tue May 16 22:34:03 2017
From: alex.buckley at oracle.com (Alex Buckley)
Date: Tue, 16 May 2017 15:34:03 -0700
Subject: The critical missing pieces and a path forward
In-Reply-To: <61d7a6e4-5441-9561-5713-5fa2e0374746@goebel-clan.de>
References: <3b52b3b6-d95d-dacf-06ce-8fceaa1a8cf5@redhat.com>
 <20170508145736.531AFC4422@eggemoggin.niobe.net>
 <f503147c-01c0-45b1-583e-3b7a4d3df206@goebel-clan.de>
 <5914DBDA.7000300@oracle.com>
 <61d7a6e4-5441-9561-5713-5fa2e0374746@goebel-clan.de>
Message-ID: <591B7E5B.4040009@oracle.com>

On 5/15/2017 12:38 PM, Volker Berlin wrote:
> Thanks for the answer. What you suggest is:
>
>   * Splitting modules in interfaces and implementation. This can explode
>     the count of modules in a complex application from 300 modules to
>     1000 modules. You lost maintainability.

Only one module that participates in the cycle needs to be split.

>   * In your construct you lost also the declaration of runtime
>     dependency to "LoggingImpl". This dependency exists in real. You
>     have only removed it from compile time. The application will not
>     work if "LoggingImpl" is not present. This reduce also the
>     maintainability.

The "LoggingIntf" module that I suggested would discover providers via 
ServiceLoader can easily choose to stop if zero providers are found, or 
if providers other than the critical "LoggingImpl" module are found.

> This work nice if you have only the 2 modules. In real live application
> with 300 modules and more there can be 2-3 thousands dependencies.
> Without a full declared graph of runtime dependencies you can't manage
> it anymore.

I get the feeling there is some kind of disconnect as to how services 
work in the Java Platform Module System. The consumption and provision 
of services is denoted explicitly in the module declarations of 
"LoggingIntf" and "LoggingImpl", so it's easy to see how modules might 
bind to each other at run time.

I would also point out that code on the classpath can consume services 
provided by modules, and modules can consume services provided by code 
on the classpath (META-INF/services), so you can modularize the existing 
300-JAR codebase in phases.

Alex

From mark.reinhold at oracle.com  Wed May 17 02:45:12 2017
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Tue, 16 May 2017 19:45:12 -0700 (PDT)
Subject: JPMS (JSR 376) EG video conference: Thursday 18 May, 15:00 UTC
Message-ID: <20170517024512.8106BD30D0@eggemoggin.niobe.net>

As you know, the JCP Executive Committee voted last week [1] not to
approve this JSR for the next stage of the process.  We now have until
7 June to submit a revised Specification [2].

Time is short, so in order to move quickly I suggest that we hold a
series of video-conference calls to discuss issues raised by the EC, the
few remaining open issues on the list [3], and other issues about which
new information has recently become available.

I propose that we meet at 15:00 UTC this Thursday, 18 May [4] for one
hour, and again at that same time each business day as needed.

==> Please let me know by 23:00 UTC tomorrow, 17 May [5], if this
    schedule is workable for you.  I will send logistical information
    to attendees privately.

In order to preserve the integrity of our discussions, these meetings
should be the sole channel for EG communication once they begin.  The
jpms-spec-experts list will therefore be closed except for the posting
of the minutes of each meeting and for updates to the issue list and
the draft Specification as may be agreed during these meetings.  All EG
members will be asked to refrain from posting to other public fora such
as, e.g., the jigsaw-dev list, until this series of meetings is over.

These meetings are open to current EG members.  Each member may invite
one additional technical representative from that member's organization.

Thanks,
- Mark


[1] https://jcp.org/en/jsr/results?id=5959
[2] https://www.jcp.org/en/procedures/jcp2#3.4.5
[3] http://openjdk.java.net/projects/jigsaw/spec/issues/
[4] https://time.is/1500_18_May_2017_in_UTC/PDT/EDT/BST/CET?JPMS_EG_Call
[5] https://time.is/2300_18_May_2017_in_UTC/PDT/EDT/BST/CET

From Tim_Ellison at uk.ibm.com  Wed May 17 12:17:45 2017
From: Tim_Ellison at uk.ibm.com (Tim Ellison)
Date: Wed, 17 May 2017 13:17:45 +0100
Subject: JPMS (JSR 376) EG video conference: Thursday 18 May, 15:00 UTC
In-Reply-To: <20170517024512.8106BD30D0@eggemoggin.niobe.net>
References: <20170517024512.8106BD30D0@eggemoggin.niobe.net>
Message-ID: <OF1B5CD3C2.37BF1D12-ON80258123.00431896-80258123.00438AE9@notes.na.collabserv.com>

Thanks Mark.  The proposed time works for me.

FWIW I don't think it is necessary to shut down the alternate EG
communication channels.  I believe we'll made good progress on the calls
and will naturally use the other channels to summarize that discussion
and provide supporting materials for posterity.

Regards,
Tim

mark.reinhold at oracle.com wrote on 17/05/2017 03:45:12:
> To: jpms-spec-experts at openjdk.java.net
> Date: 17/05/2017 03:45
> Subject: JPMS (JSR 376) EG video conference: Thursday 18 May, 15:00 UTC
> Sent by: "jpms-spec-experts" 
<jpms-spec-experts-bounces at openjdk.java.net>
> 
> As you know, the JCP Executive Committee voted last week [1] not to
> approve this JSR for the next stage of the process.  We now have until
> 7 June to submit a revised Specification [2].
> 
> Time is short, so in order to move quickly I suggest that we hold a
> series of video-conference calls to discuss issues raised by the EC, the
> few remaining open issues on the list [3], and other issues about which
> new information has recently become available.
> 
> I propose that we meet at 15:00 UTC this Thursday, 18 May [4] for one
> hour, and again at that same time each business day as needed.
> 
> ==> Please let me know by 23:00 UTC tomorrow, 17 May [5], if this
>     schedule is workable for you.  I will send logistical information
>     to attendees privately.
> 
> In order to preserve the integrity of our discussions, these meetings
> should be the sole channel for EG communication once they begin.  The
> jpms-spec-experts list will therefore be closed except for the posting
> of the minutes of each meeting and for updates to the issue list and
> the draft Specification as may be agreed during these meetings.  All EG
> members will be asked to refrain from posting to other public fora such
> as, e.g., the jigsaw-dev list, until this series of meetings is over.
> 
> These meetings are open to current EG members.  Each member may invite
> one additional technical representative from that member's organization.
> 
> Thanks,
> - Mark
> 
> 
> [1] https://jcp.org/en/jsr/results?id=5959
> [2] https://www.jcp.org/en/procedures/jcp2#3.4.5
> [3] http://openjdk.java.net/projects/jigsaw/spec/issues/
> [4] https://time.is/1500_18_May_2017_in_UTC/PDT/EDT/BST/CET?JPMS_EG_Call
> [5] https://time.is/2300_18_May_2017_in_UTC/PDT/EDT/BST/CET
> 

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

From david.lloyd at redhat.com  Wed May 17 14:03:01 2017
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Wed, 17 May 2017 09:03:01 -0500
Subject: JPMS (JSR 376) EG video conference: Thursday 18 May, 15:00 UTC
In-Reply-To: <20170517024512.8106BD30D0@eggemoggin.niobe.net>
References: <20170517024512.8106BD30D0@eggemoggin.niobe.net>
Message-ID: <cae03b1c-ed3b-face-508c-c7ebe4d1676d@redhat.com>

On 05/16/2017 09:45 PM, mark.reinhold at oracle.com wrote:
> As you know, the JCP Executive Committee voted last week [1] not to
> approve this JSR for the next stage of the process.  We now have until
> 7 June to submit a revised Specification [2].
> 
> Time is short, so in order to move quickly I suggest that we hold a
> series of video-conference calls to discuss issues raised by the EC, the
> few remaining open issues on the list [3], and other issues about which
> new information has recently become available.
> 
> I propose that we meet at 15:00 UTC this Thursday, 18 May [4] for one
> hour, and again at that same time each business day as needed.
> 
> ==> Please let me know by 23:00 UTC tomorrow, 17 May [5], if this
>      schedule is workable for you.  I will send logistical information
>      to attendees privately.

It is workable for me.  Face-to-face meeting should help us move through 
issues much more quickly.

> In order to preserve the integrity of our discussions, these meetings
> should be the sole channel for EG communication once they begin.  The
> jpms-spec-experts list will therefore be closed except for the posting
> of the minutes of each meeting and for updates to the issue list and
> the draft Specification as may be agreed during these meetings.  All EG
> members will be asked to refrain from posting to other public fora such
> as, e.g., the jigsaw-dev list, until this series of meetings is over.

I do not think this is necessary, helpful, or even possible.  For 
various reasons, not the least of which is the fundamental obligation of 
EG and EC members to do so, Red Hat continuously engages with our 
community and customers on this and many other issues, sometimes in 
public, and must continue to do so.  In addition, public communications 
are essential for researching the kinds of technical issues we face, 
which will no doubt be necessary as these meetings progress.  EG members 
must have the latitude to perform such research and solicit opinions, 
ideas, and feedback from the community.

I would request that you rely on our dedication to reaching a positive 
outcome in this regard.  Undermining a productive technical discussion 
is not something that will lead us to a "yes" vote, and public 
communications are not something that will inherently undermine a 
productive technical discussion.

> These meetings are open to current EG members.  Each member may invite
> one additional technical representative from that member's organization.
OK.

[1] https://www.jcp.org/en/procedures/jcp2#3.2.1

-- 
- DML

From mark.reinhold at oracle.com  Wed May 17 16:49:22 2017
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Wed, 17 May 2017 09:49:22 -0700
Subject: JPMS (JSR 376) EG video conference: Thursday 18 May, 15:00 UTC
In-Reply-To: <OF1B5CD3C2.37BF1D12-ON80258123.00431896-80258123.00438AE9@notes.na.collabserv.com>
References: <20170517024512.8106BD30D0@eggemoggin.niobe.net>
 <OF1B5CD3C2.37BF1D12-ON80258123.00431896-80258123.00438AE9@notes.na.collabserv.com>
Message-ID: <20170517094922.486798781@eggemoggin.niobe.net>

2017/5/17 5:17:45 -0700, tim_ellison at uk.ibm.com:
> Thanks Mark.  The proposed time works for me.
> 
> FWIW I don't think it is necessary to shut down the alternate EG
> communication channels.  I believe we'll made good progress on the calls
> and will naturally use the other channels to summarize that discussion
> and provide supporting materials for posterity.

My request that EG members refrain from participating in public fora
until these meetings are over is in response to the suggestion made by
some JCP EC members that direct discussions amongst EG members will be
more productive than indirect discussions via public e-mail messages,
articles, and blog entries.  EG members are of course free to discuss
JPMS and Java SE 9 issues as needed on a private basis with colleagues,
users, and customers, and to discuss unrelated topics publicly.

These meetings will themselves be private, but to ensure transparency
I will ask a colleague to listen in and take detailed minutes.  I will
post those minutes to the public EG list promptly, as soon as they are
reviewed and approved by all meeting participants.

- Mark

From rfscholte at apache.org  Wed May 17 17:31:33 2017
From: rfscholte at apache.org (Robert Scholte)
Date: Wed, 17 May 2017 19:31:33 +0200
Subject: JPMS (JSR 376) EG video conference: Thursday 18 May, 15:00 UTC
In-Reply-To: <20170517024512.8106BD30D0@eggemoggin.niobe.net>
References: <20170517024512.8106BD30D0@eggemoggin.niobe.net>
Message-ID: <op.y0d8avgskdkhrr@desktop-2khsk44>

With a bit of adjustment of my working hours I should be able to join.
Brian Fox can join as well on behalf of Apache Maven.

thanks,
Robert

On Wed, 17 May 2017 04:45:12 +0200, <mark.reinhold at oracle.com> wrote:

> As you know, the JCP Executive Committee voted last week [1] not to
> approve this JSR for the next stage of the process.  We now have until
> 7 June to submit a revised Specification [2].
>
> Time is short, so in order to move quickly I suggest that we hold a
> series of video-conference calls to discuss issues raised by the EC, the
> few remaining open issues on the list [3], and other issues about which
> new information has recently become available.
>
> I propose that we meet at 15:00 UTC this Thursday, 18 May [4] for one
> hour, and again at that same time each business day as needed.
>
> ==> Please let me know by 23:00 UTC tomorrow, 17 May [5], if this
>     schedule is workable for you.  I will send logistical information
>     to attendees privately.
>
> In order to preserve the integrity of our discussions, these meetings
> should be the sole channel for EG communication once they begin.  The
> jpms-spec-experts list will therefore be closed except for the posting
> of the minutes of each meeting and for updates to the issue list and
> the draft Specification as may be agreed during these meetings.  All EG
> members will be asked to refrain from posting to other public fora such
> as, e.g., the jigsaw-dev list, until this series of meetings is over.
>
> These meetings are open to current EG members.  Each member may invite
> one additional technical representative from that member's organization.
>
> Thanks,
> - Mark
>
>
> [1] https://jcp.org/en/jsr/results?id=5959
> [2] https://www.jcp.org/en/procedures/jcp2#3.4.5
> [3] http://openjdk.java.net/projects/jigsaw/spec/issues/
> [4] https://time.is/1500_18_May_2017_in_UTC/PDT/EDT/BST/CET?JPMS_EG_Call
> [5] https://time.is/2300_18_May_2017_in_UTC/PDT/EDT/BST/CET

From mark.reinhold at oracle.com  Wed May 17 18:18:50 2017
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Wed, 17 May 2017 11:18:50 -0700
Subject: JPMS (JSR 376) EG video conference: Thursday 18 May, 15:00 UTC
In-Reply-To: <cae03b1c-ed3b-face-508c-c7ebe4d1676d@redhat.com>
References: <20170517024512.8106BD30D0@eggemoggin.niobe.net>
 <cae03b1c-ed3b-face-508c-c7ebe4d1676d@redhat.com>
Message-ID: <20170517111850.199408908@eggemoggin.niobe.net>

2017/5/17 7:03:01 -0700, david.lloyd at redhat.com:
> On 05/16/2017 09:45 PM, mark.reinhold at oracle.com wrote:
>> ...
>>
>> In order to preserve the integrity of our discussions, these meetings
>> should be the sole channel for EG communication once they begin.  The
>> jpms-spec-experts list will therefore be closed except for the posting
>> of the minutes of each meeting and for updates to the issue list and
>> the draft Specification as may be agreed during these meetings.  All EG
>> members will be asked to refrain from posting to other public fora such
>> as, e.g., the jigsaw-dev list, until this series of meetings is over.
> 
> I do not think this is necessary, helpful, or even possible.  For 
> various reasons, not the least of which is the fundamental obligation of 
> EG and EC members to do so, Red Hat continuously engages with our 
> community and customers on this and many other issues, sometimes in 
> public, and must continue to do so.  In addition, public communications 
> are essential for researching the kinds of technical issues we face, 
> which will no doubt be necessary as these meetings progress.  EG members 
> must have the latitude to perform such research and solicit opinions, 
> ideas, and feedback from the community.

As I wrote in my reply to Tim nearby, my request that EG members refrain
from participating in public fora is in response to the suggestion made
by some JCP EC members that direct discussions amongst EG members will
be more productive than indirect discussions via public e-mail messages,
articles, and blog entries.  EG members are of course free to discuss
JPMS and Java SE 9 issues as needed on a private basis with colleagues,
users, and customers, and to discuss unrelated topics publicly.

If you cannot agree to this request then as an alternative, and if
everyone else who wants to participate agrees, we could hold a single
marathon meeting starting at 15:00 UTC any day this week or next.

What is your preference?

- Mark

From mark.reinhold at oracle.com  Wed May 17 18:32:56 2017
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Wed, 17 May 2017 11:32:56 -0700 (PDT)
Subject: New issue: #ModuleIdentifiers
Message-ID: <20170517183256.B9C60D31F8@eggemoggin.niobe.net>

Over the last few weeks it's become clear that some members of the Java
community are disappointed that JPMS does not solve the "multiple-version
problem".  This was, of course, a deliberate decision, as captured in the
agreed multiple-versions non-requirement [1].  In my proposal to close
the related #MultipleModuleVersions issue [2] I wrote [3]:

  These overlapping issues do reflect actual, practical problems.  There
  are, however, already effective -- if somewhat crude -- solutions to
  these problems via techniques such as shading (in Maven) and shadowing
  (Gradle).  More sophisticated solutions could be designed and
  implemented in a future release.

  The lack of immediate solutions to these problems should not block a
  developer who wants to modularize an existing class-path application.
  If such an application works properly on the class path today then it
  likely does not have conflicting packages anyway, since conflicting
  packages on the class path almost always lead to trouble.

If we do address this problem in a future release then it's highly likely
that modules will need to be identified not just by names but also by
version strings or some other information that discriminates between
different modules of the same name.  To leave a path to that potential
future I suggest that the JPMS Specification address the following new
issue [4]:

  #ModuleIdentifiers -- To leave open the possibility in a future release
  of identifying modules not just by their names but also by some sort of
  versioning information, refactor the `java.lang.module` API to
  introduce a `ModuleId` class and to use instances of that class rather
  than bare `String` objects wherever modules are named.  This class
  will, for now, simply wrap a string.  In a future release it can be
  enhanced, if needed, also to carry versioning information.

- Mark


[1] http://openjdk.java.net/projects/jigsaw/spec/reqs/#multiple-versions
[2] http://openjdk.java.net/projects/jigsaw/spec/issues/#MultipleModuleVersions
[3] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000611.html
[4] http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleIdentifiers

From david.lloyd at redhat.com  Wed May 17 18:33:50 2017
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Wed, 17 May 2017 13:33:50 -0500
Subject: JPMS (JSR 376) EG video conference: Thursday 18 May, 15:00 UTC
In-Reply-To: <20170517111850.199408908@eggemoggin.niobe.net>
References: <20170517024512.8106BD30D0@eggemoggin.niobe.net>
 <cae03b1c-ed3b-face-508c-c7ebe4d1676d@redhat.com>
 <20170517111850.199408908@eggemoggin.niobe.net>
Message-ID: <0ce876ba-c1fe-c5fa-f7fd-bcea68eec218@redhat.com>

On 05/17/2017 01:18 PM, mark.reinhold at oracle.com wrote:
> 2017/5/17 7:03:01 -0700, david.lloyd at redhat.com:
>> On 05/16/2017 09:45 PM, mark.reinhold at oracle.com wrote:
>>> ...
>>>
>>> In order to preserve the integrity of our discussions, these meetings
>>> should be the sole channel for EG communication once they begin.  The
>>> jpms-spec-experts list will therefore be closed except for the posting
>>> of the minutes of each meeting and for updates to the issue list and
>>> the draft Specification as may be agreed during these meetings.  All EG
>>> members will be asked to refrain from posting to other public fora such
>>> as, e.g., the jigsaw-dev list, until this series of meetings is over.
>>
>> I do not think this is necessary, helpful, or even possible.  For
>> various reasons, not the least of which is the fundamental obligation of
>> EG and EC members to do so, Red Hat continuously engages with our
>> community and customers on this and many other issues, sometimes in
>> public, and must continue to do so.  In addition, public communications
>> are essential for researching the kinds of technical issues we face,
>> which will no doubt be necessary as these meetings progress.  EG members
>> must have the latitude to perform such research and solicit opinions,
>> ideas, and feedback from the community.
> 
> As I wrote in my reply to Tim nearby, my request that EG members refrain
> from participating in public fora is in response to the suggestion made
> by some JCP EC members that direct discussions amongst EG members will
> be more productive than indirect discussions via public e-mail messages,
> articles, and blog entries.  EG members are of course free to discuss
> JPMS and Java SE 9 issues as needed on a private basis with colleagues,
> users, and customers, and to discuss unrelated topics publicly.
> 
> If you cannot agree to this request then as an alternative, and if
> everyone else who wants to participate agrees, we could hold a single
> marathon meeting starting at 15:00 UTC any day this week or next.
> 
> What is your preference?

Please allow me to clarify what you are asking.

Are you saying that, from this moment forward, no technical discussion 
about JPMS or Java SE 9 will allowed, under threat of effective eviction 
from the EG, by any EG member (including Oracle) or any of their 
(possibly thousands of) employees, in any public forum?  Or are you 
merely talking about the experts and not their coworkers?

What about technical questions?  Bug reports?  Hey I found a typo in 
xxx.java?

-- 
- DML

From mark.reinhold at oracle.com  Wed May 17 18:33:56 2017
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Wed, 17 May 2017 11:33:56 -0700 (PDT)
Subject: New issues: #CompilationWithConcealedPackages,
 #ResolutionAtCompileTime, & #RestrictedKeywords
Message-ID: <20170517183356.BC553D31FC@eggemoggin.niobe.net>

Over on the jigsaw-dev list the maintainers of the Eclipse "ecj" compiler
have recently raised several issues, which they've summarized [1].  I've
created the following JPMS issues in response:

  #CompilationWithConcealedPackages [2] -- An exported class can rely on
  the name of a concealed class, i.e., a class in a non-exported package,
  in various ways.  An exported class can, e.g., declare that it
  `extends` a concealed class, or that one of its `public` methods has a
  formal parameter whose type is concealed.  When a compiler processes
  the source code of a module M that refers to an exported class of
  module N, to what degree must the compiler distinguish N's _concealed_
  class (needed by N's exported class) from a class in M with the same
  fully-qualified name?  In other words, given that a compiler knows that
  the "same" package exists in M as well as concealed in N, to what
  degree must the compiler differentiate the packages?  [Stephan Herrmann
  [3]]

  #ResolutionAtCompileTime [4] -- When a compiler processes the source
  code of a module M, the JLS mandates that the compiler use the Java
  Platform Module System to "resolve" M's dependences and ultimately
  compute the modules that M "reads".  However, the Java Platform Module
  System is not specified within the JLS, but rather in the
  `java.lang.module` API specification.  Which parts of the API
  specification have bearing on the resolution of dependences at compile
  time, and where is the definition of "reads" in the API specification?
  [Jayaprakash Artanareeswaran [5], Stephan Herrmann [6]]

  #RestrictedKeywords [7] -- The grammar of module declarations includes
  several tokens, such as `module` and `transitive`, that were
  traditionally lexed as identifiers.  In support of backward
  compatibility, the JLS deems these tokens to be lexed as identifiers
  unless they appear as "restricted keywords" in a module declaration, in
  which case they are to be lexed as keywords.  This rule can, however,
  be difficult to implement because it is context-sensitive, and hence
  beyond the capabilities of some parser generators.  [Jayaprakash
  Artanareeswaran [8], Stephan Herrmann [9]]

- Mark


[1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012524.html
[2] http://openjdk.java.net/projects/jigsaw/spec/issues/#CompilationWithConcealedPackages
[3] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012404.html
[4] http://openjdk.java.net/projects/jigsaw/spec/issues/#ResolutionAtCompileTime
[5] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012301.html
[6] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012425.html
[7] http://openjdk.java.net/projects/jigsaw/spec/issues/#RestrictedKeywords
[8] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012464.html]
[9] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012498.html]

From rfscholte at apache.org  Wed May 17 18:52:34 2017
From: rfscholte at apache.org (Robert Scholte)
Date: Wed, 17 May 2017 20:52:34 +0200
Subject: JPMS (JSR 376) EG video conference: Thursday 18 May, 15:00 UTC
In-Reply-To: <20170517024512.8106BD30D0@eggemoggin.niobe.net>
References: <20170517024512.8106BD30D0@eggemoggin.niobe.net>
Message-ID: <op.y0eb1wy6kdkhrr@desktop-2khsk44>

With a bit of adjustment of my working hours I should be able to join.
Brian Fox can join as well on behalf of Apache Maven.

thanks,
Robert

On Wed, 17 May 2017 04:45:12 +0200, <mark.reinhold at oracle.com> wrote:

> As you know, the JCP Executive Committee voted last week [1] not to
> approve this JSR for the next stage of the process.  We now have until
> 7 June to submit a revised Specification [2].
>
> Time is short, so in order to move quickly I suggest that we hold a
> series of video-conference calls to discuss issues raised by the EC, the
> few remaining open issues on the list [3], and other issues about which
> new information has recently become available.
>
> I propose that we meet at 15:00 UTC this Thursday, 18 May [4] for one
> hour, and again at that same time each business day as needed.
>
> ==> Please let me know by 23:00 UTC tomorrow, 17 May [5], if this
>     schedule is workable for you.  I will send logistical information
>     to attendees privately.
>
> In order to preserve the integrity of our discussions, these meetings
> should be the sole channel for EG communication once they begin.  The
> jpms-spec-experts list will therefore be closed except for the posting
> of the minutes of each meeting and for updates to the issue list and
> the draft Specification as may be agreed during these meetings.  All EG
> members will be asked to refrain from posting to other public fora such
> as, e.g., the jigsaw-dev list, until this series of meetings is over.
>
> These meetings are open to current EG members.  Each member may invite
> one additional technical representative from that member's organization.
>
> Thanks,
> - Mark
>
>
> [1] https://jcp.org/en/jsr/results?id=5959
> [2] https://www.jcp.org/en/procedures/jcp2#3.4.5
> [3] http://openjdk.java.net/projects/jigsaw/spec/issues/
> [4] https://time.is/1500_18_May_2017_in_UTC/PDT/EDT/BST/CET?JPMS_EG_Call
> [5] https://time.is/2300_18_May_2017_in_UTC/PDT/EDT/BST/CET

From mark.reinhold at oracle.com  Wed May 17 20:18:40 2017
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Wed, 17 May 2017 13:18:40 -0700
Subject: JPMS (JSR 376) EG video conference: Thursday 18 May, 15:00 UTC
In-Reply-To: <0ce876ba-c1fe-c5fa-f7fd-bcea68eec218@redhat.com>
References: <20170517024512.8106BD30D0@eggemoggin.niobe.net>
 <cae03b1c-ed3b-face-508c-c7ebe4d1676d@redhat.com>
 <20170517111850.199408908@eggemoggin.niobe.net>
 <0ce876ba-c1fe-c5fa-f7fd-bcea68eec218@redhat.com>
Message-ID: <20170517131840.294149658@eggemoggin.niobe.net>

2017/5/17 11:33:50 -0700, david.lloyd at redhat.com:
> On 05/17/2017 01:18 PM, mark.reinhold at oracle.com wrote:
>> ...
>> 
>> As I wrote in my reply to Tim nearby, my request that EG members refrain
>> from participating in public fora is in response to the suggestion made
>> by some JCP EC members that direct discussions amongst EG members will
>> be more productive than indirect discussions via public e-mail messages,
>> articles, and blog entries.  EG members are of course free to discuss
>> JPMS and Java SE 9 issues as needed on a private basis with colleagues,
>> users, and customers, and to discuss unrelated topics publicly.
>> 
>> If you cannot agree to this request then as an alternative, and if
>> everyone else who wants to participate agrees, we could hold a single
>> marathon meeting starting at 15:00 UTC any day this week or next.
>> 
>> What is your preference?
> 
> Please allow me to clarify what you are asking.
> 
> Are you saying that, from this moment forward, no technical discussion 
> about JPMS or Java SE 9 will allowed, under threat of effective eviction 
> from the EG, by any EG member (including Oracle) or any of their 
> (possibly thousands of) employees, in any public forum?  Or are you 
> merely talking about the experts and not their coworkers?

I don't see much point in trying to negotiate legalistic rules around
all of this.

I'm simply asking that, in the spirit of the suggestion made by members
of the JCP EC, members of this EG and their colleagues exercise good
professional judgement.  Let's limit our discussions to these meetings
for the duration, rather than to a confusing mix of private meetings and
public e-mail messages, and let's agree not to attempt to influence the
discussions from outside via public channels.

- Mark

From david.lloyd at redhat.com  Wed May 17 20:20:53 2017
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Wed, 17 May 2017 15:20:53 -0500
Subject: JPMS (JSR 376) EG video conference: Thursday 18 May, 15:00 UTC
In-Reply-To: <20170517131840.294149658@eggemoggin.niobe.net>
References: <20170517024512.8106BD30D0@eggemoggin.niobe.net>
 <cae03b1c-ed3b-face-508c-c7ebe4d1676d@redhat.com>
 <20170517111850.199408908@eggemoggin.niobe.net>
 <0ce876ba-c1fe-c5fa-f7fd-bcea68eec218@redhat.com>
 <20170517131840.294149658@eggemoggin.niobe.net>
Message-ID: <d82b855d-f5b6-97f0-8549-0d50d6b496d9@redhat.com>

On 05/17/2017 03:18 PM, mark.reinhold at oracle.com wrote:
> 2017/5/17 11:33:50 -0700, david.lloyd at redhat.com:
>> On 05/17/2017 01:18 PM, mark.reinhold at oracle.com wrote:
>>> ...
>>>
>>> As I wrote in my reply to Tim nearby, my request that EG members refrain
>>> from participating in public fora is in response to the suggestion made
>>> by some JCP EC members that direct discussions amongst EG members will
>>> be more productive than indirect discussions via public e-mail messages,
>>> articles, and blog entries.  EG members are of course free to discuss
>>> JPMS and Java SE 9 issues as needed on a private basis with colleagues,
>>> users, and customers, and to discuss unrelated topics publicly.
>>>
>>> If you cannot agree to this request then as an alternative, and if
>>> everyone else who wants to participate agrees, we could hold a single
>>> marathon meeting starting at 15:00 UTC any day this week or next.
>>>
>>> What is your preference?
>>
>> Please allow me to clarify what you are asking.
>>
>> Are you saying that, from this moment forward, no technical discussion
>> about JPMS or Java SE 9 will allowed, under threat of effective eviction
>> from the EG, by any EG member (including Oracle) or any of their
>> (possibly thousands of) employees, in any public forum?  Or are you
>> merely talking about the experts and not their coworkers?
> 
> I don't see much point in trying to negotiate legalistic rules around
> all of this.
> 
> I'm simply asking that, in the spirit of the suggestion made by members
> of the JCP EC, members of this EG and their colleagues exercise good
> professional judgement.  Let's limit our discussions to these meetings
> for the duration, rather than to a confusing mix of private meetings and
> public e-mail messages, and let's agree not to attempt to influence the
> discussions from outside via public channels.

OK, in that case I think we are in agreement, thanks.

-- 
- DML

From martijnverburg at gmail.com  Wed May 17 21:12:17 2017
From: martijnverburg at gmail.com (Martijn Verburg)
Date: Wed, 17 May 2017 22:12:17 +0100
Subject: JPMS (JSR 376) EG video conference: Thursday 18 May, 15:00 UTC
In-Reply-To: <d82b855d-f5b6-97f0-8549-0d50d6b496d9@redhat.com>
References: <20170517024512.8106BD30D0@eggemoggin.niobe.net>
 <cae03b1c-ed3b-face-508c-c7ebe4d1676d@redhat.com>
 <20170517111850.199408908@eggemoggin.niobe.net>
 <0ce876ba-c1fe-c5fa-f7fd-bcea68eec218@redhat.com>
 <20170517131840.294149658@eggemoggin.niobe.net>
 <d82b855d-f5b6-97f0-8549-0d50d6b496d9@redhat.com>
Message-ID: <CAP7YuAQ6ssv8891Q5V4wnJEDxdgH6nqGw2i9j7qmWftsLVpZfw@mail.gmail.com>

Hi all,

EC member here.  The intent here is that the EC knows 30 days is a short
period of time and that higher bandwidth communication is advisable in
order to speed up resolutions.  We're certainly *not* expecting there to be
restrictions on other forms of communication.  If that's what came across
then that certainly was not our intention.

Cheers,
Martijn

On 17 May 2017 at 21:20, David M. Lloyd <david.lloyd at redhat.com> wrote:

> On 05/17/2017 03:18 PM, mark.reinhold at oracle.com wrote:
>
>> 2017/5/17 11:33:50 -0700, david.lloyd at redhat.com:
>>
>>> On 05/17/2017 01:18 PM, mark.reinhold at oracle.com wrote:
>>>
>>>> ...
>>>>
>>>> As I wrote in my reply to Tim nearby, my request that EG members refrain
>>>> from participating in public fora is in response to the suggestion made
>>>> by some JCP EC members that direct discussions amongst EG members will
>>>> be more productive than indirect discussions via public e-mail messages,
>>>> articles, and blog entries.  EG members are of course free to discuss
>>>> JPMS and Java SE 9 issues as needed on a private basis with colleagues,
>>>> users, and customers, and to discuss unrelated topics publicly.
>>>>
>>>> If you cannot agree to this request then as an alternative, and if
>>>> everyone else who wants to participate agrees, we could hold a single
>>>> marathon meeting starting at 15:00 UTC any day this week or next.
>>>>
>>>> What is your preference?
>>>>
>>>
>>> Please allow me to clarify what you are asking.
>>>
>>> Are you saying that, from this moment forward, no technical discussion
>>> about JPMS or Java SE 9 will allowed, under threat of effective eviction
>>> from the EG, by any EG member (including Oracle) or any of their
>>> (possibly thousands of) employees, in any public forum?  Or are you
>>> merely talking about the experts and not their coworkers?
>>>
>>
>> I don't see much point in trying to negotiate legalistic rules around
>> all of this.
>>
>> I'm simply asking that, in the spirit of the suggestion made by members
>> of the JCP EC, members of this EG and their colleagues exercise good
>> professional judgement.  Let's limit our discussions to these meetings
>> for the duration, rather than to a confusing mix of private meetings and
>> public e-mail messages, and let's agree not to attempt to influence the
>> discussions from outside via public channels.
>>
>
> OK, in that case I think we are in agreement, thanks.
>
> --
> - DML
>

From forax at univ-mlv.fr  Wed May 17 21:34:58 2017
From: forax at univ-mlv.fr (Remi Forax)
Date: Wed, 17 May 2017 23:34:58 +0200 (CEST)
Subject: JPMS (JSR 376) EG video conference: Thursday 18 May, 15:00 UTC
In-Reply-To: <20170517024512.8106BD30D0@eggemoggin.niobe.net>
References: <20170517024512.8106BD30D0@eggemoggin.niobe.net>
Message-ID: <519571390.2040911.1495056898585.JavaMail.zimbra@u-pem.fr>

I will be happy to attend to the meeting,
at least this week.

R?mi

----- Mail original -----
> De: "mark reinhold" <mark.reinhold at oracle.com>
> ?: jpms-spec-experts at openjdk.java.net
> Envoy?: Mercredi 17 Mai 2017 04:45:12
> Objet: JPMS (JSR 376) EG video conference: Thursday 18 May, 15:00 UTC

> As you know, the JCP Executive Committee voted last week [1] not to
> approve this JSR for the next stage of the process.  We now have until
> 7 June to submit a revised Specification [2].
> 
> Time is short, so in order to move quickly I suggest that we hold a
> series of video-conference calls to discuss issues raised by the EC, the
> few remaining open issues on the list [3], and other issues about which
> new information has recently become available.
> 
> I propose that we meet at 15:00 UTC this Thursday, 18 May [4] for one
> hour, and again at that same time each business day as needed.
> 
> ==> Please let me know by 23:00 UTC tomorrow, 17 May [5], if this
>    schedule is workable for you.  I will send logistical information
>    to attendees privately.
> 
> In order to preserve the integrity of our discussions, these meetings
> should be the sole channel for EG communication once they begin.  The
> jpms-spec-experts list will therefore be closed except for the posting
> of the minutes of each meeting and for updates to the issue list and
> the draft Specification as may be agreed during these meetings.  All EG
> members will be asked to refrain from posting to other public fora such
> as, e.g., the jigsaw-dev list, until this series of meetings is over.
> 
> These meetings are open to current EG members.  Each member may invite
> one additional technical representative from that member's organization.
> 
> Thanks,
> - Mark
> 
> 
> [1] https://jcp.org/en/jsr/results?id=5959
> [2] https://www.jcp.org/en/procedures/jcp2#3.4.5
> [3] http://openjdk.java.net/projects/jigsaw/spec/issues/
> [4] https://time.is/1500_18_May_2017_in_UTC/PDT/EDT/BST/CET?JPMS_EG_Call
> [5] https://time.is/2300_18_May_2017_in_UTC/PDT/EDT/BST/CET

From mark.reinhold at oracle.com  Thu May 18 23:20:18 2017
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Thu, 18 May 2017 16:20:18 -0700
Subject: Updated working-draft JPMS specification
Message-ID: <20170518162018.565498010@eggemoggin.niobe.net>

I've updated the draft specification, here:

    http://cr.openjdk.java.net/~mr/jigsaw/spec/

It now includes changes to the `ModuleFinder` interface for the
`Automatic-Module-Name` attribute [1], and updates to the JLS that
are intended to address #CompilationWithConcealedPackages [2].

- Mark


[1] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-May/000687.html
[2] http://openjdk.java.net/projects/jigsaw/spec/issues/#CompilationWithConcealedPackages

From aguibert at us.ibm.com  Tue May 23 00:34:13 2017
From: aguibert at us.ibm.com (Andrew Guibert)
Date: Mon, 22 May 2017 19:34:13 -0500
Subject: JPMS EG meeting minutes?
Message-ID: <OFC3089ACC.E81D267C-ON00258129.0002A0E9-86258129.00032221@notes.na.collabserv.com>


Last I heard from the jpms-spec-experts mailing list all of the EG members
were going to hold video conferences each day to accelerate the resolution
of JPMS design.
While these video conferences are private, Mark indicated that detailed
meeting minutes would be posted to the public EG list [1].

Where can we find these meeting minutes?  I don't see any on the
jpms-spec-experts public mail archives.

From mark.reinhold at oracle.com  Tue May 23 00:39:27 2017
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Mon, 22 May 2017 17:39:27 -0700
Subject: JPMS EG meeting minutes?
In-Reply-To: <OFC3089ACC.E81D267C-ON00258129.0002A0E9-86258129.00032221@notes.na.collabserv.com>
References: <OFC3089ACC.E81D267C-ON00258129.0002A0E9-86258129.00032221@notes.na.collabserv.com>
Message-ID: <20170522173927.805722328@eggemoggin.niobe.net>

2017/5/22 17:34:13 -0700, Andrew Guibert <aguibert at us.ibm.com>:
> Last I heard from the jpms-spec-experts mailing list all of the EG members
> were going to hold video conferences each day to accelerate the resolution
> of JPMS design.
> While these video conferences are private, Mark indicated that detailed
> meeting minutes would be posted to the public EG list [1].
> 
> Where can we find these meeting minutes?  I don't see any on the
> jpms-spec-experts public mail archives.

That's because they haven't been posted yet.  The minutes for each
meeting will be posted only after all participants have had a chance
to review them and offer corrections.  I expect the minutes for the
first meeting to be posted tomorrow.  They will be announced on the
jpms-spec-experts list.

- Mark

From mark.reinhold at oracle.com  Wed May 24 18:40:15 2017
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Wed, 24 May 2017 11:40:15 -0700
Subject: Java Platform Module System (JSR 376) EG minutes: 2017/5/18
Message-ID: <20170524114015.293009101@eggemoggin.niobe.net>

I've published the minutes from last Thursday's meeting, here:

    http://openjdk.java.net/projects/jigsaw/spec/minutes/2017-05-18

- Mark

From mark.reinhold at oracle.com  Wed May 24 21:59:08 2017
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Wed, 24 May 2017 14:59:08 -0700
Subject: Updated working-draft JPMS specification and issue list
Message-ID: <20170524145908.851502882@eggemoggin.niobe.net>

I've updated the working-draft specification and the issue list to
reflect our recent discussions:

  http://cr.openjdk.java.net/~mr/jigsaw/spec/
  http://openjdk.java.net/projects/jigsaw/spec/issues/

The specification now includes changes to address these issues:

  #ResolutionAtCompileTime [1]
  #RestrictedKeywords [2]

The changes to the specification are summarized in more detail in the
history subsection of the front page [3].

I've added links from the issue list to relevant portions of the minutes
of last Thursday's meeting [4].  I'll add similar links after the minutes
of later meetings are published.

- Mark


[1] http://openjdk.java.net/projects/jigsaw/spec/issues/#ResolutionAtCompileTime
[2] http://openjdk.java.net/projects/jigsaw/spec/issues/#RestrictedKeywords
[3] http://cr.openjdk.java.net/~mr/jigsaw/spec/#History
[4] http://openjdk.java.net/projects/jigsaw/spec/minutes/2017-05-18

From stuhood at twitter.com  Thu May 25 03:29:56 2017
From: stuhood at twitter.com (Stu Hood)
Date: Wed, 24 May 2017 20:29:56 -0700
Subject: Java Platform Module System (JSR 376) EG minutes: 2017/5/18
In-Reply-To: <20170524114015.293009101@eggemoggin.niobe.net>
References: <20170524114015.293009101@eggemoggin.niobe.net>
Message-ID: <CAFu+-T7YLEgAe9G9PiG-K+oVmM-vE_w0fnKFGRJKF-AQpVcoTw@mail.gmail.com>

Thanks for these notes! I wanted to highlight one quote here that seems
relevant to two important issues.

Under #CompilationWithConcealedPackages:

> Brian recapped the issue saying that there is agreement in spirit that the
> identity of a class should include provenance.


This is extremely interesting, because if including the module in a class's
identity is feasible/agreeable (in more than just "spirit"!), then it seems
like it would resolve both #CompilationWithConcealedPackages
and #AvoidConcealedPackageConflicts in one fell swoop. Collisions between
classes defined in different modules would be impossible at both compile
and runtime.

On Wed, May 24, 2017 at 11:40 AM, <mark.reinhold at oracle.com> wrote:

> I've published the minutes from last Thursday's meeting, here:
>
>     http://openjdk.java.net/projects/jigsaw/spec/minutes/2017-05-18
>
> - Mark
>



-- 
@stuhood <https://twitter.com/stuhood>

From mark.reinhold at oracle.com  Thu May 25 14:23:40 2017
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Thu, 25 May 2017 07:23:40 -0700
Subject: Java Platform Module System (JSR 376) EG minutes: 2017/5/22
Message-ID: <20170525072340.469340015@eggemoggin.niobe.net>

Minutes from our meeting this past Monday:

  http://openjdk.java.net/projects/jigsaw/spec/minutes/2017-05-22

- Mark

From misterm at gmail.com  Thu May 25 14:56:53 2017
From: misterm at gmail.com (Michael Nascimento)
Date: Thu, 25 May 2017 11:56:53 -0300
Subject: Java Platform Module System (JSR 376) EG minutes: 2017/5/22
In-Reply-To: <20170525072340.469340015@eggemoggin.niobe.net>
References: <20170525072340.469340015@eggemoggin.niobe.net>
Message-ID: <CAAWA2oUzBZiQ7OKsLbbFJKKdn=MWBVF6fUwO9WSLOkv7vxo7QA@mail.gmail.com>

I can't understand from the minutes how #CyclicDependences was resolved as
LATER given only Oracle seemed to like this idea. Could you explain if you
are doing a voting to come to resolution or whether it's a top-down
decision from Oracle?

Regards,
Michael

On Thu, May 25, 2017 at 11:23 AM, <mark.reinhold at oracle.com> wrote:

> Minutes from our meeting this past Monday:
>
>   http://openjdk.java.net/projects/jigsaw/spec/minutes/2017-05-22
>
> - Mark
>

From mark.reinhold at oracle.com  Thu May 25 16:27:48 2017
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Thu, 25 May 2017 09:27:48 -0700
Subject: Updated JPMS issue list
Message-ID: <20170525092748.344281557@eggemoggin.niobe.net>

I've updated the issue list to reflect Monday's discussion [1]:

  http://openjdk.java.net/projects/jigsaw/spec/issues/

The #ModuleIdentifiers [2] and #VersionSyntax [3] issues are now marked
as resolved.  I'll update the #LayerPrimitives issue after the minutes
from Tuesday's meeting are posted, since the discussion of that issue
started on Monday but did not finish.

No changes to the specification arose from Monday's discussion.

- Mark


[1] http://openjdk.java.net/projects/jigsaw/spec/minutes/2017-05-22
[2] http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleIdentifiers
[3] http://openjdk.java.net/projects/jigsaw/spec/issues/#VersionSyntax

From mark.reinhold at oracle.com  Fri May 26 19:37:14 2017
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Fri, 26 May 2017 12:37:14 -0700
Subject: Java Platform Module System (JSR 376) EG minutes: 2017/5/23
Message-ID: <20170526123714.200867933@eggemoggin.niobe.net>

Minutes from our meeting this past Tuesday:

  http://openjdk.java.net/projects/jigsaw/spec/minutes/2017-05-23

I also made a minor update to the Monday minutes, to make the title
of the automatic-module usage issue suggested by Robert consistent
with the title used in Tuesday's minutes.

I'll update the working-draft specification and the issue list to
reflect the decisions recorded these minutes next week.

Thanks very much to Iris Clark for producing the minutes in such
a timely fashion!

- Mark

From mark.reinhold at oracle.com  Tue May 30 16:24:41 2017
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Tue, 30 May 2017 09:24:41 -0700
Subject: Updated working-draft JPMS specification and issue list
Message-ID: <20170530092441.41284603@eggemoggin.niobe.net>

I've updated the working-draft specification and the issue list to
reflect our final meeting last week [1]:

  http://cr.openjdk.java.net/~mr/jigsaw/spec/
  http://openjdk.java.net/projects/jigsaw/spec/issues/

The specification now includes the `ModuleLayer.Controller::addExports`
method [2], and ?7.7 of the draft JLS was revised to distinguish clearly
between access at compile time, access at run time, and reflective
access, in response to a comment from Volker Simonis [3].

- Mark


[1] http://openjdk.java.net/projects/jigsaw/spec/minutes/2017-05-23
[2] http://openjdk.java.net/projects/jigsaw/spec/issues/#LayerPrimitives
[3] http://mail.openjdk.java.net/pipermail/jpms-spec-comments/2017-May/000090.html