Dynamic modules and bidirectional interoperation

[mark.reinhold at oracle.com]

2016/10/28 12:20:51 -0700, Thomas Watson <tjwatson at us.ibm.com>:
> ...
> 
> I don't have near the history with JPMS as this group. I'm going out on a 
> limb talking about the definitive behavior of the JMPS runtime, but here 
> goes. Today layers are searched for required modules in a specific single 
> parent order.  First the local layer is searched, if not found look in the 
> parent layer, do this until the empty layer is reached.

This isn't quite right ...

>                                                          For example, look 
> at the following layers:
> 
>  A -> B -> C 
> 
> These are simply searched in order A, B, C.
> Now we introduce multiple parents:
> 
>  A -> B -> C
>   \-> D -> E
> 
> These are searched in order A, B, C, D, E.  From the perspective of layer 
> A I would expect a consistent resolution result as if you had a single 
> parent hierarchy.
> 
>  A -> B -> C -> D -> E

Yes, that's what I expect also.

> The problem I do see though is what happens if you have the following:
> 
>  layer B has modules X and Z
>  layer D has modules Y and Z 
>  both modules X and Y requires public Z
>  layer A has module W which requires both X and Y

(Reminder: `requires public` is being renamed to `requires transitive`.)

Graphically:

  A { W requires X & Y }
  -> B { X requires transitive Z, Z }
  -> D { Y requires transitive Z, Z }

> This better result in some kind of error regardless if single or 
> multi-parent layers are used because in both cases module W is getting 
> exposed to two copies of the Z module (through dependencies X.B->Z.B and 
> Y.D-> Z.D).  I'm basing this on the javadoc of Configuration for 
> findModule(String name) and resolveRequires(ModuleFinder before, 
> ModuleFinder after, Collection<String> roots).

Whether or not this results in an error depends upon how you configure
layer B.

To compute a configuration the resolver takes two module finders, a
"before" finder and an "after" finder.  To locate a module by name it
consults the "before" finder, the parent configuration (and then its
parents recursively), and the "after" finder, in that order, until it
locates a module or fails.

If Z.B is in B's "after" finder (the common case) then no error will be
reported.  Module X in layer B will read the copy of Z in layer D (X.B ->
Z.D), i.e., there will be no Z.B in the resulting module graph.  Even if
you ask for Z explicitly when configuring B, i.e., specify it as a root,
you'll get Z.D rather than Z.D.

If Z.B is in B's "before" finder (the uncommon case, principally to
support `--upgrade-module-path`) then an error will be reported since W
would have to read both Z.B and Z.D due to the read edges implied by the
`requires transitive Z` directives in the definitions of modules X and Y.

(Corner case: If the two Z modules do not define any packages of the same
 name then right now no error is reported; this is probably a bug.)

>                                                 In my opinion Z.B is the 
> only Z module which module W should ever be able to wire to because if it 
> requires Z directly it could only ever wire to Z.B.

No, if W required Z directly then it would still get Z.D since there
wouldn't be a Z.B in B's configuration.

>                                                      But this may be 
> over-restrictive. For example, what if W only requires Y? In that case it 
> seems perfectly reasonable to allow W to wire to Y.D and Z.D.

If W only requires Y then you'd still get W -> Y.D, Z.D.

>                                                               Is this 
> problem made worse with multiple parent layers?

The semantics seem clear enough; whether they're "worse" depends upon
whether you consider the current behavior to be a problem.  The error
scenario described above can now occur between two modules of the same
name in different parents, so in theory at least there would be more
opportunities for this kind of situation to arise.  Whether that would
turn out to be true in practice, I'm not sure.

> This approach of using #NonHierarchicalLayers is suggested as a tool to 
> solve the problem of pinning modules that are dynamically updated or 
> uninstalled in a dynamic module system like OSGi.  Another possible 
> solution to that is to use a solution to #DiscardableModules, but that 
> does require an ever growing chain of parent Layers to be created as 
> modules are updated in the dynamic module system.

Right.

> 2016/10/21 6:24:38 -0700, Remi Forax <forax at univ-mlv.fr>,
> jpms-spec-observers:
>> ...
>> 
>> Let's try again, currently a way to solve your issue is to allow a
>> Layer to have several parents, i really dislike this idea because it
>> introduces an arbitrary order that is hidden deep in the code thus
>> makes really hard to reason about which Layer resolve what, for me
>> it's not different to add a kind of classpath resolution in the
>> middle of the module resolution.
>> 
>> Currently, you control the classloaders, so you can load a hand
>> crafted class in the module (you just have to patch the class name
>> to have the right package, you can use the patching capabilities of
>> defineAnonymousClass for that, but you can also call defineClass on
>> the right classloader as again, you control the classloaders).  Once
>> you are able to execute arbitrary code in a module you can call
>> addReads, addExports (and soon addOpens) on the j.l.r.Module.
> 
> I agree with you this could be used instead of a solution to 
> #ReadabilityAddedByLayerCreator, but I think it is a bit messy when it 
> doesn't have to be.  A solution to #ReadabilityAddedByLayerCreator should 
> not be too controversial. I also think this is an orthogonal issue to the 
> pinning stale modules issue. I don't think you are suggesting this 
> approach to somehow reach into the Layer implementation to pluck out the 
> stale modules ourselves. That is where the monkey-patching feels more like 
> gorilla-patching to me.

Agreed, though I'm not sure that's what Rémi is trying to suggest.

Perhaps a more pertinent problem with Rémi's general idea is that it
would only modify the run-time module graph, as instantiated via layers
and updated via the j.l.r.Module API.  Such updates would have no effect
on the later resolution of JPMS modules since resolution refers to the
unchanging configurations of the relevant layers rather than to the
run-time module graph.  Depending on the details I suspect this would
make it difficult, if not impossible, to resolve JPMS modules against
OSGi bundles surfaced as JPMS modules in the way that I suggested.

- Mark