Proposal: #NonHierarchicalLayers

[David M. Lloyd]

On 02/15/2017 01:51 PM, mark.reinhold at oracle.com wrote:
> 2017/1/30 7:13:31 -0800, david.lloyd at redhat.com:
>> On 01/26/2017 04:34 PM, mark.reinhold at oracle.com wrote:
>>> 2016/12/13 6:47:53 -0800, david.lloyd at redhat.com:
>>>> On 12/12/2016 05:23 PM, mark.reinhold at oracle.com wrote:
>>>>> 2016/12/8 5:46:24 -0800, david.lloyd at redhat.com:
>>>>>>  ...  I've added the following methods to Layer.Controller:
>>>>>>
>>>>>> public Controller addPackage(Module source, String pn) { ... }
>>>>>> public Controller addOpens(Module source, String pn, Module target) { ... }
>>>>>> public Controller addOpensToAll(Module source, String pn) { ... }
>>>>>> public Controller addOpensToAllUnnamed(Module source, String pn) { ... }
>>>>>> public Controller addExports(Module source, String pn, Module target) {
>>>>>> ... }
>>>>>> public Controller addExportsToAll(Module source, String pn) { ... }
>>>>>> public Controller addExportsToAllUnnamed(Module source, String pn) { ... }
>>>>>> public Controller addUses(Module source, Class<?> service) { ... }
>>>>>
>>>>> Can you explain exactly why you need all these methods?
>>>>>
>>>>> I can see why you might need the qualified `addExports` method, akin
>>>>> to the existing `addOpens` method, if you're doing some form of module
>>>>> resolution on your own that's somehow taking named layers, or whatever,
>>>>> into account.
>>>>
>>>> Yeah we're assembling the module structure in a multi-stage lazy
>>>> resolution process, thus we don't know exactly what we're opening or
>>>> exporting until after all contents and dependencies are defined (and
>>>> this can change over time).
>>>
>>> Hmm.  This seems to be a fundamental difference between JBoss modules and
>>> OSGi.  Once an OSGi bundle is loaded then its exports don't change, at
>>> least not until it's updated, and this is part of what enables Watson's
>>> JPMS embedding to work.
>>
>> OSGi does support dynamic attachments of fragments.  The current
>> prototype cannot do this but on Jan. 4 Thomas expressed that being able
>> to add packages would enable this part of the specification.  Re-linking
>> everything is (according to this email) an alternative that comes at the
>> cost of not supporting this feature.
>
> True, but Thomas later recommended against complicating JPMS in order to
> support this feature of OSGi [1].  He can live with having to re-resolve
> bundles when fragments add API packages, which is not common and arguably
> an anti-pattern.

Sure, it's possible that OSGi implementations can get away with it, as 
few users seem to care much about such strict spec adherence and perhaps 
the spec will be modified to take the new restrictions into account.

>>> For any kind of Watson-style embedding of JBoss modules in JPMS to work
>>> then there has to be a point in time at which you can say "okay, this
>>> JBoss module is in a stable state, let's spin up a JPMS module descriptor
>>> for it so that the JPMS resolver can resolve some JPMS modules against
>>> it."  If the JBoss module is updated later on then you create a new JPMS
>>> module descriptor for it and re-resolve any JPMS modules that used to
>>> depend upon it.
>>
>> In order to do this we have to shut down all existing deployments in the
>> container that have any kind of dependency relationship, and restart
>> them.  This is not really acceptable as it defeats the purpose of the
>> mechanism.
>>
>>> Can you, in general, identify such points in time for JBoss modules?
>>
>> Any time the user deploys additional or replacement content is a point
>> where a module may need to be updated.
>>
>>> If so, then I don't think you need the above methods.  The JPMS module
>>> descriptor that you construct needn't capture all the details of the
>>> corresponding JBoss module.  In fact it can't, since JBoss modules can
>>> relate to each other in ways that aren't expressible in JPMS.  That's
>>> fine, though -- the module descriptor need only expose the information
>>> that the JPMS resolver needs for actual JPMS modules, mainly `opens`,
>>> `exports`, and `requires transitive`.
>>>
>>> If not, then adding the above methods won't help you anyway.  They only
>>> affect the run-time state of modules, i.e., the state known to the JVM
>>> and surfaced in `java.lang.reflect.Module`.  The JPMS resolver doesn't
>>> read the run-time state but, rather, the configurations computed by
>>> earlier invocations of the resolver, and ultimately those are all based
>>> on module descriptors, over in the `java.lang.module` package.
>>
>> Currently the prototype code does not use the JPMS resolver.  From my
>> understanding, the JVM tables mediate access control, but the actual
>> linkage between classes at run time happens via
>> java.lang.ClassLoader#findClass(java.lang.String, java.lang.String).  As
>> long as this method resolves
>
> This sentence is incomplete, so I'm not sure what you meant to say here.

Something like "as long as this method resolves classes in modules then 
we'll be fine".

> At any rate, if you're not eventually going to use the JPMS resolver to
> resolve some JPMS modules against some JBoss modules, in order to obtain
> bidirectional interoperation as is now possible with OSGi [2], then why
> bother with any of this?

I don't want to.  Even with all of our outstanding issues resolved, this 
end product is not what we'd hoped for and is at best a partial 
compromise, and in my mind represents far too radical a change in far 
too short of a time frame.  In many ways it runs flatly counter to what 
we've learned and accomplished these past several years in our own 
architecture.

The reality though is that if the platform includes a new deployment 
format with new behaviors, some users will soon expect that this format 
and its corresponding behaviors to be supported by containers, because 
it will be viewed by many to be a standard.

It's possible that we can support this expectation to an extent simply 
by adding a separated JPMS container, and that would be a good option 
for that purpose.  However that leaves our existing deployment model 
forever deficient because of its relative lack of encapsulation 
features, and to a lesser extent, reliance on "fake" module diagnostics, 
to speak nothing of compatibility oddities for users going from one to 
the other.  If we want to bring our existing deployment model with all 
its features up to date with the new encapsulation capabilities, we 
either need to find a way to utilize Jigsaw to do it (which is what I'm 
trying to accomplish here), or else we need some new API which allows us 
to establish similar behaviors at a class loader level.

Otherwise, you're right: if we can't take advantage of the useful 
capabilities of JPMS on our own terms, then why would we vote to support 
what amounts to a competing module architecture and deployment format, 
which actively undermines alternative systems which can't or won't abide 
by its restrictions (by the combination of establishing that the JPMS is 
not a meta-module system, yet also restricting key JVM features to only 
that system, and ensuring that differing behavioral characteristics 
(like version numbers or descriptor formats for example) either conform 
or are relegated to second-class behaviors), and which from our point of 
view has several critical deficiencies and substantial compatibility 
problems compared to what we have today?

The only reason we would do, for the lack (via the above reasons) of 
inherently representing a good forward evolution for us, is if it 
provided a way for us to coexist while also giving us similar 
capabilities.  Otherwise we frankly shouldn't support such a system at 
all, and I would hope to persuade others of the same.

-- 
- DML