Discussion: #NonHierarchicalLayers

[David M. Lloyd]

On 07/13/2016 09:28 AM, mark.reinhold at oracle.com wrote:
> Reference: http://openjdk.java.net/projects/jigsaw/spec/issues/#NonHierarchicalLayers
>
> 2016/3/2 16:07:06 -0800, david.lloyd at redhat.com:
>> If there is any expectation of moving existing systems over to Jigsaw, I
>> think that the strict hierarchical layer system should be abandoned in
>> favor of a graph-oriented system.  This allows containers to maintain
>> separate module graphs with separate namespaces which nevertheless
>> interconnect.
>
> I suspect that generalizing the design to allow non-hierarchical layers
> is possible, but tricky.  Both configurations and layers would be able to
> have multiple parents, so the API would become more complex, but perhaps
> that's okay since this is more for expert than casual use.  To avoid
> having to deal with conflicts between parents (i.e., the diamond-import
> problem), it'd likely be best to make the collections of parents be lists
> rather than sets, so that they're searched in a determined order.

That's good news.

>> I believe that this is a necessary requirement if (in particular) there
>> is an expectation that Java EE 9 (or another Java EE specification)
>> would be reframed in terms of Jigsaw-style modules, since Java EE module
>> names have many fewer restrictions, and it is possible for (for example)
>> more than one application to contain modules of the same name.
>
> I don't understand how these two observations motivate non-hierarchical
> layers, but maybe I'm missing something.
>
> Module names are an entirely distinct issue [1].  The constraints on
> module names could be relaxed regardless of whether or not layers are
> hierarchical.

Right, I shouldn't have conflated these issues.  Names are relevant to 
this issue only insofar as sibling layers allow for module name 
duplication between them.

> It's already possible to have multiple modules of the same name with
> simple hierarchical layers.  Two hosted applications can contain a module
> of the same name as long as they're loaded into distinct, unrelated
> layers.  (Even two layers that are related can contain modules of the
> same name, in which case the resolver chooses the module in the nearest
> layer.)
>
> Are there other reasons why a hypothetical future Java EE specification,
> which allows applications to be built from collections of JPMS modules,
> might necessitate non-hierarchical layers?

It's not the specification that would require them, but the 
implementation, which may allow OSGi-style dynamic addition and changing 
of installed dependencies, especially from different (possibly 
overlapping) module namespaces.  Such dynamic changes can currently only 
be modeled using incrementally-added layers, but this fails if a 
previously loaded module must be updated to refer to a later one, or if 
there is more than one module namespace.

Another case is where there are two separate deployment systems with 
isolated name spaces (for example, OSGi and Java EE) which may be 
cross-linked.

In our current solution, we don't actually layer the namespaces - in the 
internal APIs we actually identify what module space the proposed 
dependency exists in, along with the name of the module to find.  This 
allows us to (for example) have more than one declaration syntax for 
dependencies.

In pre-Java 9 terms, a good metaphor is Extension-list: versus 
Class-Path: MANIFEST headers - each comes from a separate namespace, but 
each declares where the dependency information comes from.  We also add 
a Dependencies: header, which allows (the equivalent of boot layer) 
module names to be specified as dependencies for a deployment, 
separately from the old extension list and class path headers (which we 
use to resolve modules from different namespaces).  I believe that it is 
presently our intention to continue to support extension lists even 
after that is dropped from the spec; the namespace for extensions is 
isolated from the namespace of static modules, but deployment modules 
may depend on either or both (and/or additional module spaces like OSGi, 
which may also have conflicting namespaces) at their option using the 
appropriate MANIFEST header or other deployment descriptor.

So I would hypothetically imagine a layer for our static modules (though 
this depends on the resolution or lack thereof of a couple other 
issues), then another for extensions, another for EE deployments, 
another sibling for filesystem-based "automatic" modules that we 
generate for absolute class-path refs, and another sibling for each 
auxiliary add-on system (like OSGi), all of which have the ability to 
arbitrarily cross-reference to the other sibling layers.  Some possible 
configurations can be interpreted as a tree, but some configurations 
must be a DAG or even a fully cyclic graph in order to work.  Note also 
that this might all only be meaningful hand-in-hand with a resolution to 
#MutableConfigurations and #LazyConfigurationAndInstantiation.

Note that we could actually get by *without* non-hierarchical layers, 
*if* I can explicitly establish module dependences which link between 
sibling custom layers: I actually view this as a better, more general 
approach, and one which we use internally.  So that's another option.

>> If on the other hand there is an expectation that Java EE is not going
>> to be retrofitted in a way which maps EE applications and modules to
>> Jigsaw modules, then this should be clearly declared.
>
> As I wrote earlier, we definitely do have that expectation -- that's why
> the requirements include an entire section on dynamic configuration.
>
> - Mark
>
>
> [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleNameCharacters
>

-- 
- DML