Configurations... (Was: Re: Module-system requirements)

[David M. Lloyd]

On 03/09/2015 03:55 PM, mark.reinhold at oracle.com wrote:
> 2015/2/16 6:12 -0800, david.lloyd at redhat.com:
>> On 02/14/2015 03:33 PM, mark.reinhold at oracle.com wrote:
>>> 2015/2/12 7:35 -0800, david.lloyd at redhat.com:
>>>> I'm a little concerned that the configurations section implies a strict
>>>> hierarchy:
>>>>
>>>> "Nested dynamic configurations — An application must be able to create a
>>>> dynamic configuration that relates to an existing dynamic configuration
>>>> rather than to the application’s initial configuration."
>>>>
>>>> Why can a configuration not relate to more than one other configurations?
>>>
>>> A tree of configurations rather than a simple linear list is more
>>> difficult to reason about and trickier to implement (you must, e.g.,
>>> cope with the diamond-import problem).
>>>
>>> The feedback we've received so far has not suggested that this
>>> limitation will be a problem in practice.  Do you have use cases
>>> which require this flexibility?
>>
>> We implement not just a tree, but a general directed graph (even
>> allowing cycles) among modules and among their configurations (module
>> loaders),
>
> A configuration, as defined in the requirements document, is a group of
> modules related by resolution to each other and, if needed, to modules in
> one or more enclosing configurations.  If you look just at the graph of
> modules in a running system then it's at least a DAG, and possibly even
> cyclic, but that doesn't necessarily mean that the configurations
> themselves need to be related to each other in a structure any richer
> than a tree.

OK I guess I need to back up a little.  Our module loaders do have an 
arbitrarily complex logical relationship, but this stems from the fact 
that they're actually physically fully isolated from one another, and 
free to delegate module loading decisions arbitrarily.  Any module's 
(programmatic representation of its) dependency declarations may consist 
of a module loader + identifier combination, and any module loader may 
choose to delegate part or all of its namespace to any other module loader.

>>            which enables us to easily support OSGi and Java EE, as well
>> as tying these things to filesystem JARs (e.g. for EE deployments which
>> have Class-Path declarations that have an absolute path name), and the
>> legacy extensions mechanism which is presently also a part of Java EE
>> (though this is going/has gone away, I understand), and also allows
>> tying in other systems as well.
>
> Does doing these things require configurations to be related in a
> directed graph, or is a tree sufficient?

The way we interoperate between extensions, external filesystem class 
path entries, and EE modules, by using the above mechanism we can (for 
example) create a dependence relationship from the EE module to an 
extension module, from the extension module to a filesystem class path 
module, from a filesystem class path module back to an extension module, 
and from any of these three module types back to the static platform 
module set.  Really to us a ModuleLoader is just a namespace with some 
policy connected to it; the actual loading/resolving relationships are 
simply arrows to specific other modules, optionally in specific other 
ModuleLoaders.

>> Due to the way we support imports, diamonds are really no problem.  Each
>> module's dependency list is interpreted in order, so the set of imported
>> resources is always the union of everything exported by each dependency.
>>    So if you acquire a different set of items via two different "paths"
>> to a module, you get the union of those sets in the end.  Duplicated
>> paths are removed such that the first occurrence is used.
>
> Ah, so the order of dependence declarations is significant in your
> system.  Interesting, but I wonder if it's wise.

So far it hasn't caused us any problems; on the contrary it's been 
proven to be a quite useful trait.  Though in retrospect I think it 
would have been better to spend more time trying to prevent package 
overlap, as people tend to use this to implement dirty (fragile) tricks 
(as people do).  It was only due to technical reasons that we can't 
effectively distinguish between a package and a resource directory in 
all the cases that we need to be able to do so; this may have been 
resolvable given a bit more perseverance.

> Anyway, the diamond-import problem I was thinking of has to do with
> configurations rather than modules.  It can be solved in a manner similar
> to what you describe for resources, i.e., by forcing an application that
> wants to do this to place an order on the configurations to which a new
> configuration is to be related.  When looking for a module the resolver
> would consider the configurations in that order, and since only one
> module can satisfy any particular dependence it would just stop at the
> first one that it finds.
>
> I'm still concerned, however, that this could lead to some unpleasant
> implementation complexities, and also be more difficult to use in
> practice, so unless there's a clear need to relate configurations in
> graphs then I'd prefer to limit them to trees.

Maybe what we need is a clearer definition of "Configuration" as it is 
only defined as a side-effect of Resolution: "The resulting 
configuration contains all of the code and data required to compile or 
run the initial module."   And Linking is AFAICT the first section that 
implies that Configurations actually contain modules in any way.  This 
might help me understand what a tree topology actually means.

-- 
- DML