Alternatives for naming automatic modules, and a proposal (#AutomaticModuleNames)

Wed Apr 26 16:27:02 UTC 2017

2017/4/25 5:08:21 -0700, Stephen Colebourne <scolebourne at joda.org>:
> On 24 April 2017 at 19:54,  <mark.reinhold at oracle.com> wrote:
>> An explicit module that depends upon one or more modules that are
>> automatic today is, itself, no more stable than those automatic modules.
>> It could be broken when those automatic modules are modularized
>> explicitly, and if it `requires transitive` an automatic module then any
>> modules that depend upon it could be broken when that automatic module
>> is modularized explicitly.
> 
> I find this to be a completely artificial pure view of the world. Most
> projects do not have internal packages, and if they do, most end-users
> do not use them. When developers upgrade an artifact in Maven, they
> expect to have incompatibilities, so a smaller set of packages is just
> fine. Its a normal part of software development today, not an
> aberration.

If developers expect to have to fix incompatibilities when they upgrade
to a newer version of an artifact then what's wrong, after all, with
publishing an explicit module that depends upon non-modularized
components in the form of automatic modules?

If one of those automatic modules is modularized later on, and given a
different name, then how is having to fix that materially different from
having to fix code that was using a package that's no longer exported?
If anything it might actually be easier to cope with a module-name
change, since all you have to do is edit a `requires` directive; code
that refers to a package that's now encapsulated might require a
non-trivial rewrite.

I think I need to reconsider my previous conclusion that explicit modules
that depend upon automatic modules should never be published for broad
use [2].  Sure, they have unstable names and unstable APIs but if, as you
say, people are generally used to fixing minor problems when they upgrade
then this instability may well be tolerable -- especially since it would
allow maintainers to modularize whenever they want to rather than only
after all of their dependencies have been modularized.

> ...
> 
> On another thread you wrote:
> On 24 April 2017 at 16:46,  <mark.reinhold at oracle.com> wrote:
>> Automatic modules are best viewed as a transitional tool that allows you
>> to modularize an isolated, unpublished set of components in a top-down
>> fashion over time.  You can use existing published JAR files, from Maven
>> Central or elsewhere, as automatic modules.  The names and APIs of those
>> automatic modules are unstable and will change as they are modularized
>> explicitly, but in an isolated set of components you can easily adjust
>> your own `module-info.java` files as needed to cope with those changes.
> 
> And for what benefit? Why would a company put itself through the pain
> of relying on potentially thousands of external dependencies via
> automatic modules, taking a hit every time one gets converted to a
> real module. Where is the payback for that company? What is the
> benefit they gain?

Assuming that their long-term goal is to fully modularize their code base
then automatic modules allow them to get started now, before all of the
relevant dependencies have been modularized.  They can write module
declarations for all of their components, deal with any split packages
or cyclic dependences, and count on reliable configuration and strong
encapsulation for their own code, though not their dependencies.

> Unfortunately, the concept of bottom-up full modularization simply
> won't work, no matter how much the Jigsaw team hopes it will. The
> process would take forever, may not be possible for some projects,
> will be side-tracked into the release cycles of larger projects, be
> blocked by dead projects and for many other reasons just stall.

I don't know of anyone working on Jigsaw who thinks that a complete
bottom-up modularization of the entire ecosystem will ever actually
happen.  It won't.

> ...
> 
> For example, there are 42 separate projects at Apache Commons, some of
> which have multiple release streams - thats an awful lot of work
> you're expecting unpaid volunteers to do, particularly when each has
> to be released in order. (A typical release at Apache Commons is a
> multi-week affair, so even if Apache Commons had the energy to release
> everything, it would take all their energy for well over a year.)
> 
> There are even 11 different Joda projects to work on, with active ones
> depending on less active ones, I would be forced to do artificial
> releases just to satisfy a bottom-up migration.
> 
> The only approach that makes any sense to migration is to drop the
> artificial purity goal. We already have build tools (Maven, Gradle)
> that manage versions, locking them in to form a working graph of
> artifacts. When a version is changed, sometimes things break, and we
> fix them. This is all normal software development. It is not something
> for JPMS to try and fight or fix.

One of the long-standing goals of the module system is to fit in with
existing tools and developer workflows rather than supplant them, or
fight them, or fix them.  If, in that light, occasional breakage is
better than "artificial purity" then I don't see what's wrong with
encouraging the publication of explicit modules that depend upon
automatic modules.

The only remaining objection seems to be the aesthetic one, i.e., the
fact that the name of an automatic module is derived from the artifact
that defines it rather than from some intrinsic property of its content
or an explicit declaration by its author.  I understand and share this
concern.  I completely agree that modules are not artifacts, as I've
written before [3], but I don't see a better alternative [4].

If conflating modules with artifacts in the case of automatic modules
enables more projects to modularize more quickly, then I can live with
that.

> As discussed before, removing automatic modules and allowing modules
> to have partially specified dependencies [1] 1b allows projects to
> migrate to modules immediately. This would be a huge win. Any project
> that is interested could write module-info immediately, even if all it
> was doing was specify the exported packages. They would release within
> their normal release cycles, and depend on whatever other modules are
> already released at the time of release. As more and more modules
> become available, the partial requirements will turn to complete
> requirements, and thus explicit modules.

This approach is only a little better than that of the `Module-Name`
manifest attribute.  At least it doesn't establish two different ways
to name a module, but it still makes it easy for people other than the
maintainer of a component to try to establish that component's module
name.  This approach would, moreover, still require existing projects
to do new releases, which as you indicate above can be very costly and
in many cases unlikely to happen.

> ...
> 
> Worrying about getting incomplete "impure" modules in Maven Central is
> simply the wrong concern. If we keep the current design, and insist on
> no automatic module dependencies in Maven Central, then JPMS will
> simply not be adopted outside a few private niches.

Very well, then: Let's allow automatic-module dependences in Central.

- Mark

[1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011686.html
[2] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-April/000682.html
[3] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-February/000582.html
[4] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-April/000666.html