Module descriptions versus module descriptors

[David M. Lloyd]

I think that this use case is supported by the requirements.  While the 
requirements explicitly do not support multiple versions of modules, it 
is easy enough to package each version as a separate module with a 
different name, e.g. "org.apache.commons.collections.3.2" vs. 
"org.apache.commons.collections.3.3" or some such, optionally using a 
module alias to choose one of these to be the proper 
"org.apache.commons.collections" module.

According to the requirements, the module implementation may forbid this 
arrangement if it determines that the modules "interfere" with each 
other.  However, as long as nothing is re-exported, this should not be 
the case.

On 12/15/2015 10:32 AM, Paul Benedict wrote:
> I've been ruminating over the Commons Collections example. Here is a
> typical use case. Presume all libraries are properly modularized:
>
> 1) Module "org.apache.commons.collections" is uploaded into Maven for
> Commons Collections 3.2.2
> 2) Same module name is also uploaded for 3.2.3, 3.2.4, 3.3.0, etc.
> 3) App depends on module Commons Collections 3.2.2 (not re-exported)
> 4) App depends on module XYZ which depends on module Commons Collections
> 3.3.0 (not re-exported)
>
> Some things to note:
>
> a) We should expect module names to be reused for any version of a product.
> b) An app should be able to require version A and transitively require
> version B. Duplicate packages shouldn't be a problem here because of the
> strong encapsulation and lack of re-exporting.
> c) What is supposed to be done with the duplicate module names?
>
> Is this typical use case supported by the current requirements?
>
> Cheers,
> Paul
>
> On Thu, Dec 10, 2015 at 9:02 AM, David M. Lloyd <david.lloyd at redhat.com
> <mailto:david.lloyd at redhat.com>> wrote:
>
>     On 12/09/2015 07:26 PM, mark.reinhold at oracle.com
>     <mailto:mark.reinhold at oracle.com> wrote:
>
>         2015/10/20 4:27 -0700, david.lloyd at redhat.com
>         <mailto:david.lloyd at redhat.com>:
>
>             I see no logical path that leads from the requirements as
>             specified
>             exclusively to the assumption that the descriptor must be
>             bytecode, let
>             alone part of the JVM and/or language specification. All the
>             reasons
>             given appear to be self-justifying or based on abstract
>             assumptions,
>             e.g. "modules are... a new kind of Java program component...
>             therefore
>             [Jigsaw] treats them as such".
>
>
>         I agree that there are other ways to represent module descriptions.
>         I've never stated otherwise.
>
>         If module boundaries, however they're expressed, are to be
>         enforced by
>         the compiler and the VM -- as you've agreed elsewhere -- then their
>         manner of expression is inevitably going to be a topic for the Java
>         language and VM specifications.  This is not avoidable.
>
>
>     Actually that does not logically follow.  I could, for example,
>     "express" module boundaries in the VM by creating a hard equivalence
>     between modules and class loaders, and in the compiler via
>     command-line arguments.  In this way, no langauge or VM
>     specifications must change; in fact only the specification of the
>     compiler tool itself *must* change.  Granted it is *likely* that in
>     such a case, one would go further and maybe make some small
>     modifications to the javax.tools API as well to facilitate the
>     provision of resources on a per-module basis.
>
>                         ... it seems to me that a significant part of
>             the Jigsaw
>             design justification for its handling of module metadata
>             hinges around
>             the conflation of the description of a module, and the
>             descriptor used
>             by the static module loading implementation.  This raises a
>             red flag for
>             me because it fundamentally locks the capabilities of module
>             descriptions to whatever makes sense to express in the
>             descriptor, and
>             then in turn constrains these things to the language and JVM
>             specification.
>
>             In our (JBoss) module system, these concepts are decoupled:
>             a filesystem
>             module's descriptor is read and parsed into a description
>             which is then
>             consumed by the module system to create the module.  ...
>
>
>         If module boundaries are to be enforced by the compiler and the
>         VM then,
>         in such a decoupled system, where and in what form does the compiler
>         locate the information that describes the module being compiled,
>         and any
>         other modules required in order to compile that module?
>
>
>     You (or rather, the build tool) would tell it.
>
>     In real nontrivial systems, the universe of compiled and packaged
>     (and versioned) artifacts is what you use when you're building
>     another piece of the system.  The installed set of modules is
>     necessarily a subset of this total universe.  Because of this, if
>     I'm building something I generally cannot say "I have a build
>     dependency upon org.apache.commons.collections".  I must say "Build
>     against org.apache.commons.collections, version 3.2.2", and
>     furthermore I might have to also specify whose build of such I use,
>     and where to get it from.  And, I commonly must build different
>     pieces of my universe against different versions of the same thing,
>     especially when pieces come from a third party.  I will likely not
>     use the same version I built against in the final system, especially
>     in an evolving system, which I believe is a critical use case.
>     Finally, I may use the final built artifact in a variety of systems,
>     whereupon it is necessary to modify the run time name and/or linkage
>     information depending on the requirements of the target system.  I
>     will almost certainly not even test the two together until a later
>     integration testing phase, which (in probable addition to an ABI
>     compatibility checking tool) will be the last stage before accepting
>     the new module into the installed set.
>
>     In other words, it is often necessary to enforce different module
>     boundaries depending on circumstances related to build, install, and
>     run, independently of the rest of the source code.  Much of this
>     information may not be available at all when the original artifact
>     is being built.  Any bundling of module descriptor information
>     inside of the module contents will have the effect of rendering that
>     module useless in any module environment that is sufficiently unlike
>     the one for/in which the module was built (unless you modify the
>     module itself after the fact).  It is not realistic to assume that
>     the version of a module that is compiled against is the version that
>     will be run against, and it's misleading to "newbies" who might
>     expect some kind of global cohesion between modules without any kind
>     of limiting context (e.g. a module distribution, or a specific
>     application).
>
>     The power that a modular Java gives you is that you can leverage
>     what is arguably Java's best feature - the ability to easily provide
>     perfect ABI stability between versions, and therefore the ability to
>     substitute modules in the run-time environment - in order to
>     maintain a growing and evolving ecosystems, or indeed many such
>     ecosystems.  This is not unlike what many OS distributors can do,
>     and for similar reasons in a similar way (though we have the sizable
>     advantage of Java's more powerful linking capabilities at our
>     disposal, which makes some hard things easy, and some impossible
>     things possible).  I think that the module system should do what it
>     can to enable this, and to me that means: external module
>     descriptions, i.e. the compiled artifact (which consumes and/or
>     provides specific ABI(s)) should be separate from the install/run
>     linkage (which specifies how to tie modules together in such a way
>     that all necessary ABI requirements are fulfilled).  This is because
>     the artifact build dependency information is superficially similar
>     to, but fundamentally not the same as, the run time linkage
>     information - and this will be true no matter what we do or what
>     constraints we place. The only thing we can influence in this regard
>     is, how hard do we make these use cases to achieve?
>
>     There are other reasons that I prefer this approach as well, but
>     this is the reason that directly answers your question.  And there
>     are possibly other approaches that would also solve the same
>     problems - for example, a hybrid approach where exported and
>     non-exported packages are listed and annotated within the module,
>     but the dependence information is external and provided separately
>     at build time and run time (in a manner suitable to each situation).
>
>     --
>     - DML
>
>

-- 
- DML