Maven Central will never be the universal Jigsaw module repository (Re: Advice + proposals regarding automodule naming)

[David M. Lloyd]

On 01/20/2017 05:08 AM, Robert Scholte wrote:
> I think we need to clarify the term "ease of transition". What are we
> expecting and why?
> It looks to me the expectation is that every current existing Java
> project should be able to have a module-info where every dependency is
> specified as a requirement. We all agree (and have accepted) that in
> case of split packages this will not work, so we drop a little in
> percentages. Brian and I will go one step beyond: you cannot require
> unnamed / automatic modules.
> Even if we are going to consider synthetic modules, that won't solve the
> problem. This would require the developer to specify the module name for
> all his requirements AND including the transitive ones. The latter is
> required because there's no way to pass this kind of information to
> depending projects.
> Stability is the keyword here, and the automatic modules cannot
> guarantee stability.
>
> Some developers have asked me: why do we need this? Tools like Maven are
> already superb in selecting all required dependencies for both compile
> time and runtime. Results are stable. And those few times there are
> issues, I was able to fix it by adding or excluding dependencies.
>
> With the module-info the result should be at least as stable as done
> with Maven. In case of automodules you could think of 2 options: #1 add
> as requires AND configure aliases for this synthetic module AND all its
> transitive unnamed modules. #2 drop requires for this module. I know
> which one I would choose.
>
> Also keep in mind: Maven Central also started empty. I don't know which
> jar was the first, but I'm pretty sure it can still be used. Give it
> time for jars to become modules. Ease of transition is already there
> because applications can refer to jars containing a module-info file,
> even if they don't use it.

The critical flaw in this analogy (which I'm afraid departs a little 
from the automatic module concept) is that the single, global module 
namespace that necessarily will have to exist in order for any 
centralized module repository cannot be met by Maven Central without a 
fundamental and complex change to the way that submissions are curated

The reason for this is that today, a Maven artifact in Maven Central 
only has to resolve consistently relative to the set of artifacts it 
consumes, and (to a lesser extent because there's some flexibility here) 
the set of artifacts it is likely to coexist with.  This flexibility and 
relativity goes most of the way to mitigate the fact that many Maven 
artifacts have conflicting packages and version requirements.  Because 
of this, most of the time this is invisible to average users.

In the modular world though, not only must you resolve a set of 
artifacts that resolve in a mutually consistent way, but they also have 
to be 100% non-conflicting in terms of module specification, and more 
problematically, they have to be 100% mutually consistent in terms of 
dependency mesh.  In order to have any sort of guarantee of consistency 
for any given module artifact, consistency must be guaranteed for *all* 
artifacts.

The Maven Central model for artifacts fails in this regard for the exact 
same reason that there isn't, for example, one unified Linux package 
"mega-repository".  Packaging issues aside, there are many competing 
implementations of the same specifications and solutions to the same 
problems; these things have rippling effects on compatibility.  In order 
to create one, single, unified module repository for *everything* in 
Maven Central that is internally consistent would be a behemoth 
undertaking and a major maintenance burden.

Thus the alternative is as I've expressed many times before.  The 
ecosystem of artifacts remains an ecosystem of artifacts.  Ecosystems of 
modules will be a new entity, a subset of available artifacts designed 
to solve a specific problem; some module ecosystems will be produced as 
single applications and others as development platform distributions 
targeted at various audiences and maintained by different entities with 
different goals.

As a consequence, build systems which work to contribute these 
ecosystems necessarily will operate in one of two possible ways.

The first way is to consume artifacts like one does today, and within 
the build environment and using ecosystem-specific metadata, wire it in 
to the module graph, before (probably) CI testing the resultant 
combination and committing it into the distribution.

The second way is to consume sources as artifacts and use the same 
ecosystem-specific metadata to compile the sources, wiring it in as above.

Expecting that we can start from an empty repository and build up The 
One Single Central module repository is unrealistic because such a 
repository either must be too constrained to be generally useful in the 
way that Maven Central is useful, or it must be too inconsistent to be 
useful in any nontrivial project.

I think that because a lot of users are still in the beginning or 
experimental stages of modularization, these realities are not yet 
obvious, and I hate to defeat optimism in this regard but we have been 
modularizing Maven artifacts for many years now with our own module 
system, so we know firsthand how difficult it is to mesh hundreds of 
artifacts into a single distribution, let alone the many thousands that 
exist in Maven Central.  I think that anyone developing nontrivial 
distributions or applications will encounter these realities sooner or 
later so I hope that the experience we've gained will inform a more 
sensible approach to module distribution than "throw it all in Maven 
Central, it'll be fine".

-- 
- DML