Alternatives for naming automatic modules, and a proposal (#AutomaticModuleNames)

Wed Apr 26 17:35:00 UTC 2017

On 26 April 2017 at 17:27,  <mark.reinhold at oracle.com> wrote:
> 2017/4/25 5:08:21 -0700, Stephen Colebourne <scolebourne at joda.org>:
>> As discussed before, removing automatic modules and allowing modules
>> to have partially specified dependencies [1] 1b allows projects to
>> migrate to modules immediately. This would be a huge win. Any project
>> that is interested could write module-info immediately, even if all it
>> was doing was specify the exported packages. They would release within
>> their normal release cycles, and depend on whatever other modules are
>> already released at the time of release. As more and more modules
>> become available, the partial requirements will turn to complete
>> requirements, and thus explicit modules.
>
> This approach is only a little better than that of the `Module-Name`
> manifest attribute.  At least it doesn't establish two different ways
> to name a module, but it still makes it easy for people other than the
> maintainer of a component to try to establish that component's module
> name.  This approach would, moreover, still require existing projects
> to do new releases, which as you indicate above can be very costly and
> in many cases unlikely to happen.

Perhaps this concept is not fully understood. The concept is:

- there are two types of module - "complete" and "incomplete"
- a module can only declare a "requires" clause on another module, ie.
one with a module declaration
- a complete module fully defines all its dependencies
- an incomplete module defines as many dependencies as it can
- an incomplete module also depends on the classpath
- the current automatic module concept ceases to exist
(an incomplete module is not an automatic module, as it doesn't get to
read all other modules. It only gets to read the classpath)

Note that at no stage does a module declaration author attempt to
guess a module name. The module declaration author is only able to
depend on things with known names.

The set of modules sits on the modulepath. The set of non-modules sits
on the classpath. The two do not mix.

Stephen
(I'll reply to some other points later when I'm not heading for the train...)

> 2017/4/25 5:08:21 -0700, Stephen Colebourne <scolebourne at joda.org>:
>> On 24 April 2017 at 19:54,  <mark.reinhold at oracle.com> wrote:
>>> An explicit module that depends upon one or more modules that are
>>> automatic today is, itself, no more stable than those automatic modules.
>>> It could be broken when those automatic modules are modularized
>>> explicitly, and if it `requires transitive` an automatic module then any
>>> modules that depend upon it could be broken when that automatic module
>>> is modularized explicitly.
>>
>> I find this to be a completely artificial pure view of the world. Most
>> projects do not have internal packages, and if they do, most end-users
>> do not use them. When developers upgrade an artifact in Maven, they
>> expect to have incompatibilities, so a smaller set of packages is just
>> fine. Its a normal part of software development today, not an
>> aberration.
>
> If developers expect to have to fix incompatibilities when they upgrade
> to a newer version of an artifact then what's wrong, after all, with
> publishing an explicit module that depends upon non-modularized
> components in the form of automatic modules?
>
> If one of those automatic modules is modularized later on, and given a
> different name, then how is having to fix that materially different from
> having to fix code that was using a package that's no longer exported?
> If anything it might actually be easier to cope with a module-name
> change, since all you have to do is edit a `requires` directive; code
> that refers to a package that's now encapsulated might require a
> non-trivial rewrite.
>
> I think I need to reconsider my previous conclusion that explicit modules
> that depend upon automatic modules should never be published for broad
> use [2].  Sure, they have unstable names and unstable APIs but if, as you
> say, people are generally used to fixing minor problems when they upgrade
> then this instability may well be tolerable -- especially since it would
> allow maintainers to modularize whenever they want to rather than only
> after all of their dependencies have been modularized.
>
>> ...
>>
>> On another thread you wrote:
>> On 24 April 2017 at 16:46,  <mark.reinhold at oracle.com> wrote:
>>> Automatic modules are best viewed as a transitional tool that allows you
>>> to modularize an isolated, unpublished set of components in a top-down
>>> fashion over time.  You can use existing published JAR files, from Maven
>>> Central or elsewhere, as automatic modules.  The names and APIs of those
>>> automatic modules are unstable and will change as they are modularized
>>> explicitly, but in an isolated set of components you can easily adjust
>>> your own `module-info.java` files as needed to cope with those changes.
>>
>> And for what benefit? Why would a company put itself through the pain
>> of relying on potentially thousands of external dependencies via
>> automatic modules, taking a hit every time one gets converted to a
>> real module. Where is the payback for that company? What is the
>> benefit they gain?
>
> Assuming that their long-term goal is to fully modularize their code base
> then automatic modules allow them to get started now, before all of the
> relevant dependencies have been modularized.  They can write module
> declarations for all of their components, deal with any split packages
> or cyclic dependences, and count on reliable configuration and strong
> encapsulation for their own code, though not their dependencies.
>
>> Unfortunately, the concept of bottom-up full modularization simply
>> won't work, no matter how much the Jigsaw team hopes it will. The
>> process would take forever, may not be possible for some projects,
>> will be side-tracked into the release cycles of larger projects, be
>> blocked by dead projects and for many other reasons just stall.
>
> I don't know of anyone working on Jigsaw who thinks that a complete
> bottom-up modularization of the entire ecosystem will ever actually
> happen.  It won't.
>
>> ...
>>
>> For example, there are 42 separate projects at Apache Commons, some of
>> which have multiple release streams - thats an awful lot of work
>> you're expecting unpaid volunteers to do, particularly when each has
>> to be released in order. (A typical release at Apache Commons is a
>> multi-week affair, so even if Apache Commons had the energy to release
>> everything, it would take all their energy for well over a year.)
>>
>> There are even 11 different Joda projects to work on, with active ones
>> depending on less active ones, I would be forced to do artificial
>> releases just to satisfy a bottom-up migration.
>>
>> The only approach that makes any sense to migration is to drop the
>> artificial purity goal. We already have build tools (Maven, Gradle)
>> that manage versions, locking them in to form a working graph of
>> artifacts. When a version is changed, sometimes things break, and we
>> fix them. This is all normal software development. It is not something
>> for JPMS to try and fight or fix.
>
> One of the long-standing goals of the module system is to fit in with
> existing tools and developer workflows rather than supplant them, or
> fight them, or fix them.  If, in that light, occasional breakage is
> better than "artificial purity" then I don't see what's wrong with
> encouraging the publication of explicit modules that depend upon
> automatic modules.
>
> The only remaining objection seems to be the aesthetic one, i.e., the
> fact that the name of an automatic module is derived from the artifact
> that defines it rather than from some intrinsic property of its content
> or an explicit declaration by its author.  I understand and share this
> concern.  I completely agree that modules are not artifacts, as I've
> written before [3], but I don't see a better alternative [4].
>
> If conflating modules with artifacts in the case of automatic modules
> enables more projects to modularize more quickly, then I can live with
> that.
>
>> As discussed before, removing automatic modules and allowing modules
>> to have partially specified dependencies [1] 1b allows projects to
>> migrate to modules immediately. This would be a huge win. Any project
>> that is interested could write module-info immediately, even if all it
>> was doing was specify the exported packages. They would release within
>> their normal release cycles, and depend on whatever other modules are
>> already released at the time of release. As more and more modules
>> become available, the partial requirements will turn to complete
>> requirements, and thus explicit modules.
>
> This approach is only a little better than that of the `Module-Name`
> manifest attribute.  At least it doesn't establish two different ways
> to name a module, but it still makes it easy for people other than the
> maintainer of a component to try to establish that component's module
> name.  This approach would, moreover, still require existing projects
> to do new releases, which as you indicate above can be very costly and
> in many cases unlikely to happen.
>
>> ...
>>
>> Worrying about getting incomplete "impure" modules in Maven Central is
>> simply the wrong concern. If we keep the current design, and insist on
>> no automatic module dependencies in Maven Central, then JPMS will
>> simply not be adopted outside a few private niches.
>
> Very well, then: Let's allow automatic-module dependences in Central.
>
> - Mark
>
>
> [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011686.html
> [2] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-April/000682.html
> [3] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-February/000582.html
> [4] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-April/000666.html