Maven Central will never be the universal Jigsaw module repository (Re: Advice + proposals regarding automodule naming)

[Remi Forax]

----- Mail original -----
> De: "David M. Lloyd" <david.lloyd at redhat.com>
> À: jpms-spec-experts at openjdk.java.net
> Envoyé: Vendredi 20 Janvier 2017 23:48:54
> Objet: Maven Central will never be the universal Jigsaw module repository	(Re: Advice + proposals regarding automodule
> naming)

> On 01/20/2017 05:08 AM, Robert Scholte wrote:
>> I think we need to clarify the term "ease of transition". What are we
>> expecting and why?
>> It looks to me the expectation is that every current existing Java
>> project should be able to have a module-info where every dependency is
>> specified as a requirement. We all agree (and have accepted) that in
>> case of split packages this will not work, so we drop a little in
>> percentages. Brian and I will go one step beyond: you cannot require
>> unnamed / automatic modules.
>> Even if we are going to consider synthetic modules, that won't solve the
>> problem. This would require the developer to specify the module name for
>> all his requirements AND including the transitive ones. The latter is
>> required because there's no way to pass this kind of information to
>> depending projects.
>> Stability is the keyword here, and the automatic modules cannot
>> guarantee stability.
>>
>> Some developers have asked me: why do we need this? Tools like Maven are
>> already superb in selecting all required dependencies for both compile
>> time and runtime. Results are stable. And those few times there are
>> issues, I was able to fix it by adding or excluding dependencies.
>>
>> With the module-info the result should be at least as stable as done
>> with Maven. In case of automodules you could think of 2 options: #1 add
>> as requires AND configure aliases for this synthetic module AND all its
>> transitive unnamed modules. #2 drop requires for this module. I know
>> which one I would choose.
>>
>> Also keep in mind: Maven Central also started empty. I don't know which
>> jar was the first, but I'm pretty sure it can still be used. Give it
>> time for jars to become modules. Ease of transition is already there
>> because applications can refer to jars containing a module-info file,
>> even if they don't use it.
> 
> The critical flaw in this analogy (which I'm afraid departs a little
> from the automatic module concept) is that the single, global module
> namespace that necessarily will have to exist in order for any
> centralized module repository cannot be met by Maven Central without a
> fundamental and complex change to the way that submissions are curated
> 
> The reason for this is that today, a Maven artifact in Maven Central
> only has to resolve consistently relative to the set of artifacts it
> consumes, and (to a lesser extent because there's some flexibility here)
> the set of artifacts it is likely to coexist with.  This flexibility and
> relativity goes most of the way to mitigate the fact that many Maven
> artifacts have conflicting packages and version requirements.  Because
> of this, most of the time this is invisible to average users.
> 
> In the modular world though, not only must you resolve a set of
> artifacts that resolve in a mutually consistent way, but they also have
> to be 100% non-conflicting in terms of module specification, and more
> problematically, they have to be 100% mutually consistent in terms of
> dependency mesh.  In order to have any sort of guarantee of consistency
> for any given module artifact, consistency must be guaranteed for *all*
> artifacts.

You can imagine island of artifacts with no connection, OSGI bundles are these kind of island, but i agree.

> 
> The Maven Central model for artifacts fails in this regard for the exact
> same reason that there isn't, for example, one unified Linux package
> "mega-repository".  Packaging issues aside, there are many competing
> implementations of the same specifications and solutions to the same
> problems; these things have rippling effects on compatibility.  In order
> to create one, single, unified module repository for *everything* in
> Maven Central that is internally consistent would be a behemoth
> undertaking and a major maintenance burden.

What save Java when compared to linux distributions is that you have only one Java platform from the module point of view, there are no or a loosely defined spec for linux. 
But, i agree with you that we will have to be stricter on what can be published on term of binary compatibility and dependency.

My dream is to have a build tool that increment my module version automatically depending how i break the compatibility from the previously published version.

> 
> Thus the alternative is as I've expressed many times before.  The
> ecosystem of artifacts remains an ecosystem of artifacts.  Ecosystems of
> modules will be a new entity, a subset of available artifacts designed
> to solve a specific problem; some module ecosystems will be produced as
> single applications and others as development platform distributions
> targeted at various audiences and maintained by different entities with
> different goals.
> 
> As a consequence, build systems which work to contribute these
> ecosystems necessarily will operate in one of two possible ways.
> 
> The first way is to consume artifacts like one does today, and within
> the build environment and using ecosystem-specific metadata, wire it in
> to the module graph, before (probably) CI testing the resultant
> combination and committing it into the distribution.
> 
> The second way is to consume sources as artifacts and use the same
> ecosystem-specific metadata to compile the sources, wiring it in as above.

for me, we will transitioning from 1 to 2.

> 
> Expecting that we can start from an empty repository and build up The
> One Single Central module repository is unrealistic because such a
> repository either must be too constrained to be generally useful in the
> way that Maven Central is useful, or it must be too inconsistent to be
> useful in any nontrivial project.

given that the module-info is not read for Java < 9, you can consider that currently Maven Central is empty and that when jigsaw compatible will be commited in Maven Central, it will build a new respository of compatible modules from bottom to the top. The fact that these modules will be more or less compatible with Java < 9 (the classpath world) is just a nice bonus.

> 
> I think that because a lot of users are still in the beginning or
> experimental stages of modularization, these realities are not yet
> obvious, and I hate to defeat optimism in this regard but we have been
> modularizing Maven artifacts for many years now with our own module
> system, so we know firsthand how difficult it is to mesh hundreds of
> artifacts into a single distribution, let alone the many thousands that
> exist in Maven Central.  I think that anyone developing nontrivial
> distributions or applications will encounter these realities sooner or
> later so I hope that the experience we've gained will inform a more
> sensible approach to module distribution than "throw it all in Maven
> Central, it'll be fine".

And as i said above, from the module perspective of the Java module, maven Central is currently empty (if build tool do not allow to mix modulepath and classpath).
In my opinion, we will see more backpressure that we have now, you will not be able to push something not compatible to your previous release to Maven Central without a major internet outrage.

> 
> --
> - DML

Rémi