Updates on Interoperability

[Bryan Atsatt]

Richard S. Hall wrote:
> Sam Pullara wrote:
>> Did we decide which way it will be compatible?  I'm a little 
>> concerned that implementing something that can use OSGi modules is a 
>> far bigger task that implementing something that OSGi can use.  If we 
>> are going for total compatibility it seems like we should just use 
>> OSGi.  Personally I wanted something in Java that was simpler and 
>> cleaner that OSGi-users could leverage if they wanted.  But I'm 
>> coming at this from the we-need-something-like-maven/gem/ivy camp.
>>
>> The vast majority of people that are going to use this system will 
>> have never heard of OSGI nor care about any of their more subtle 
>> features and just want an easy way to leverage the vast amount of 
>> libraries available (virtually all of them are not OSGi modules today 
>> but are jar files in maven repositories).  We shouldn't be specing 
>> out some sort of uber container but rather a simple way to tie code 
>> to its dependencies.
>
> If this is all that people wanted, then we could just define 
> repositories and associated metadata and forget about all runtime 
> modularity support, but I strongly disagree that this is all people 
> want or else there wouldn't be such an upsurge in OSGi interest and 
> adoption.
>
>> I strongly suggest KISS applies to our effort.  I write this knowing 
>> that the committee seems to be heavily in favor of this alternate 
>> view that we implement the kitchen sink.
>
> I think our interest is similar, but perhaps our conclusions are 
> different. Initially, I think my comments were construed as an 
> argument against split packages in general, but I am really arguing 
> the same as you (I think) that we don't really need split packages to 
> be a concept in our "interoperability API", if that is how we are to 
> view the 277 API. Of course, I am also willing to have the former 
> argument too, but I will leave that for another time. :-)
>
> From my point of view, I do not think it is wholly reasonable to 
> assume that we want to try to accomplish interoperability across 
> module systems, where that includes arbitrarily sharing split packages 
> across module systems.
Nor do I, but I'm trying to understand the implications if we assume 
that OSGi will continue to split packages.
>
> Let me try to accurately conjure the peculiarities of split packages 
> in the OSGi module layer...
>
> We support split packages in two ways, via required bundles (i.e., 
> module dependencies) and via bundle fragments. In general, we 
> recommend that if you split your packages that you attach mandatory 
> attributes on them so that importers do not get them by mistake. By 
> requiring a bundle, you can ignore mandatory attributes and get 
> everything a bundle has to offer and combine it with other required 
> bundles that might be exporting other parts of the split packages. 
> Given this, there is a semi-reasonable chance* that we can tell you 
> which bundles are contributing to the "complete" packages, but we have 
> no way of knowing if/when combined split packages are complete. 
> Further, we have to assume that split package parts offered by 
> different bundles do not overlap, because if we allowed them to 
> overlap then we would have shadowing and ordering issues. Again we 
> have no easy way to verify that they do not overlap, so we just assume 
> they don't.
>
> Fragments on the other hand are a little trickier, since split 
> packages from them are loaded from the same class loader to provide 
> package private access (which is not afforded for split packages in 
> the require bundle approach). The "host" bundle doesn't really know 
> about fragments and it is possible for fragments to shadow or extend 
> packages in the "host" bundle. Since fragments provide their split 
> packages in an implicit way (i.e., they largely just become part of 
> the host bundle's class path), there is no easy way to determine if a 
> fragment is contributing to a given package or not. (* This feature of 
> fragments also makes it so it is not really possible to know who is 
> contributing classes to a split package in the require bundle case.)
Ok, so first I assume it would make sense for an OSGi Repository to 
simply never return a ModuleDefinition for a fragment.

And the "you don't know what you've got till it's resolved" problem is 
going to be present in any module system for which package name is not a 
required component of a dependency expression. But as long as re-export 
is not the default behavior (it isn't), doesn't this really just boil 
down to the leakage problem? And this is addressed in 277 the same as it 
is in OSGi: you must declare such "leaks" as re-exports ("uses"). Not 
perfect, but probably good enough.

So I think we're left with the issue of a single search visiting 
multiple loaders that can return the same package. This won't happen in 
the fragment case, since the "split" gets mended at runtime, but it will 
in the require-bundle case. And this will manifest as multiple entries 
in the import list which export the same package.

But even this isn't an issue, as long as dependency resolution relies 
solely on declared exports and re-exports of a module, and not on what 
is available by browsing the imports of that module.

So I'm back to my original statement that we just need to acknowledge 
that package duplication may exist in an import list.

Or am I missing something here?
>
> I think that is fairly complete description of the situation, sorry if 
> it is somewhat terse. In summary, I wouldn't assume that we can get 
> reasonable interoperability at this level. If you need these types of 
> features, then you should stick to a single modularity 
> framework...preferably OSGi. ;-)
>
> But my overall point is, I believe that if we can support 
> interoperability at module- and package-level dependencies across 
> module systems, then I think we would hit most use cases.
>
> -> richard
>
>>
>> Sam
>>
>> On Apr 25, 2008, at 1:57 PM, Richard S. Hall wrote:
>>
>>> Bryan Atsatt wrote:
>>>> Richard S. Hall wrote:
>>>>> Gordon Hirsch wrote:
>>>>>>> 1. Map its dependency declarations into a standard runtime 
>>>>>>> representation.
>>>>>>> 2. Support a standard search model over its stored modules, 
>>>>>>> using the runtime dependency expressions.
>>>>>>> 3. Map its stored module data into a standard runtime 
>>>>>>> representation that can be returned by the search.
>>>>>>
>>>>>> One challenge lies in defining the "standard runtime 
>>>>>> representations" and "standard search model" in a universal 
>>>>>> enough way to encompass OSGi and other module systems. This 
>>>>>> implies embracing concepts (in these standard representations and 
>>>>>> search model) that were not universally liked by the EG early on. 
>>>>>> (Split packages and package-level import/export come to mind.)
>>>>>
>>>>> Package-level import/export seem like a must, but I still don't 
>>>>> see split packages as a necessity.
>>>> Strongly agreed on import-by-package, and that ModuleDefinition 
>>>> requires a method to enumerate exported packages (and probably an 
>>>> exportsPackage(String packageName) method to enable an efficient 
>>>> Query).
>>>> On the split package front, however, it seems a little fuzzy to me. 
>>>> If an OSGi implementation of 277 is also going to remain OSGi Rx 
>>>> compliant, it will still need to support split packages, right? If 
>>>> so, don't we need to surface this fact at the 277 level? Perhaps 
>>>> that is as simple as acknowledging that Module.getImportedModules() 
>>>> may return more than one instance that exports a given package.
>>>
>>> That all depends on if the 277 API is a subset of OSGi functionality 
>>> or equal to it, I suppose.
>>>
>>> -> richard
>>>>
>>>> // Bryan
>