Relationship to JSR 291 [was: Re: Bryan's comments]

[Richard S. Hall]

Bryan Atsatt wrote:
> Stanley M. Ho wrote:
>> Glyn Normington wrote:
>>>
>>> *Bryan Atsatt <bryan.atsatt at ORACLE.COM>* wrote on 30/05/2007 07:57:59:
>>> ...
>>>  > So the open issue is the richness of the import "language": must we
>>>  > support only lowest-common-denominator, or can we do better without
>>>  > over-complicating the design?
>>>  >
>>>  > I for one would like to be able to have a single module express
>>>  > dependencies on modules from both the same and different
>>> ModuleSystems,
>>>  > *using the standard semantics of each*. This may be reaching too
>>> far,
>>>  > but we should at least explore it seriously while we figure out what
>>>  > interop means here...
>>>
>>> At this point, I feel that is likely to be reaching too far, but I'm
>>> happy to play along and see what we can learn along the way.
>>
>> I agreed with Glyn that this might be reaching too far. Before we dive
>> too much into how to implement interoperability, I think one of the
>> outstanding questions we should answer first is what degree of import
>> interoperability we want to offer. There are four possibilities:
>>
>> 1. JSR 277 module imports OSGi module by module name
>> 2. OSGi module imports JSR 277 module by module name
>> 3. OSGi module imports JSR 277 module by package name
>> 4. JSR 277 module imports OSGi module by package name
>>
>> Let's ignore the 294 issues and the module initialization issues for now
>> to simplify this discussion.
>>
>> I think we all agreed #1 and #2 are important to support and the
>> reflective APIs already enable these (of course, there are minor issues
>> we still have to address as we evolve the APIs.)
>>
>> That said, it is unclear to me how important it is to support #3 and #4,
>> so I think the first question for this EG is whether we want to support
>> #3 and #4 at all. However, let's pretend #3 and #4 are important for
>> this specific discussion.
>>
>> I think #3 is already possible since the OSGi framework could look up
>> the appropriate module which exported the package in the repository,
>> using the reflective APIs. Afterwards, the OSGi framework can then do
>> the necessary wiring. Yes, the reflective APIs can be further refined to
>> make it easier for the OSGi framework to query modules by exported
>> package name from the repository, but this is a minor issue that we can
>> address easily.
>>
>> #4 is a bit complicated, because there is no import-by-package semantic
>> in the current module layer's APIs and I think we all agreed that we
>> don't want to support this semantic directly in the module system
>> defined by JSR 277.
>
> Not me.
>
> I do understand your point that this is more difficult, but I don't
> really care--it is our job to do the Right Thing here. Sure, if it is
> not feasible in the time frame we have, then so be it. But we haven't
> even tried!
>
> If we're forced to make a choice between the two models, I'd pick
> import-by-package with zero hesitation.
>
> OSGi has a lot of experience here, and we shouldn't ignore it.
> Import-by-name was added late in the game, and is now seriously
> downplayed, for good reason.

Just for information, import-by-name (or module dependencies) was not
added to OSGi R4 because they were thought to be widely needed or
useful, adding them was always somewhat contentious which is why the R4
spec contained a section saying why you shouldn't use them (this section
has been further expanded in R4.1). The main motivation for adding them
was certainly legacy situation and for tightly coupled sub-systems.

All in all, I agree with everything that Bryan is saying and I argued
these same points early on; however, I was under the impression that we
had long since lost that battle.

Bryan, it is unclear to me whether your view is that we need to make
sure that 277 supports other modules systems the use import-by-package
or if you think that 277 should somehow directly support
import-by-package (i.e., explicitly expose such concepts).

-> richard

>
> And Oracle has a lot of relevant experience as well, since the shared
> loader mechanism I created for our AS stack ("shared libraries")
> supports only the import-by-name model. We have *lots* of
> shared-libraries (I've seen 100+ running simultaneously), used by many
> different components in the stack, in addition to customer applications.
> It is a tremendous pain when refactoring or even simple renaming has to
> be done (I hope to shift this implementation onto 277 to solve this
> problem, rather than inventing an interim approach). And this kind of
> change happens far more frequently than you probably expect.
>
> I don't want to repeat the same mistake here. Nor do I think that
> leaving a long trail of extraneous "view" modules behind for
> compatibility is a good solution.
>
> Worse still, if the 277 APIs don't support import-by-package, then the
> compiler won't support it either. If the compiler doesn't support it,
> then we've lost a golden opportunity to move to a better world, one in
> which packaging is irrelevant.
>
> We're supposed to be eliminating "jar-hell" here. Let's not simply
> replace it with "module-hell".
>
>> In this context, the question is really about what
>> it would take to make #4 possible if we want to support it, and so far
>> there are two different approaches we have discussed:
>>
>> a. Treat import-by-package separately from import-by-module in the
>> reflective APIs and make the module layer fully aware of the
>> import-by-package concept.
>>
>> b. Expose exported-package using the ModuleDefinition abstraction, and
>> provide necessary hooks (e.g. consistency checking) for the module
>> system to resolve this kind of dependency appropriately. The module
>> layer is not aware of the import-by-package concept at all. New import
>> dependency granularity could also be introduced in a similar manner in
>> the future without requiring significant changes in the module layer.
>>
>>
>> As Richard and I discussed in previous emails, it seems to make the most
>> sense for JSR 277 to define the minimal set of features possible in the
>> module layer to accomplish what needs to be accomplished to provide core
>> modularity support in the Java platform. I think (b) follows this
>> principle nicely while (a) does not. If we decide to support #3 and #4,
>> then how we want to address #4 would impact our overall design
>> significantly.
>>
>> Is there anyone in the EG provide good reasons why #3 and #4 are
>> important to be supported? If so, is there anyone in the EG think (b) is
>> not feasible at all and we should do (a) instead to support #4? Can you
>> explain your rationale?
>
> Yes, and yes. See above for rationale (ease-of-refactoring and compiler
> support).
>
>>
>>
>>>  > RESOLUTION MODELS
>>>  >
>>>  > The current design requires that each ModuleSystem provide its own
>>>  > resolution logic, and that each definition will be resolved by its
>>>  > owning ModuleSystem. This model appears to provide flexibility for
>>>  > significant differences in implementation, but we really don't know
>>>  > enough at this point. Perhaps only an actual second implementation
>>> will
>>>  > tell us if this provides useful flexibility.
>>
>> In the existing module systems we are aware, e.g. OSGi and NetBeans, we
>> already know their resolution logics are different from each other. Even
>> if they are migrated/re-based to JSR 277 in the future, I think we
>> should expect their resolution logic would remain the same to maintain
>> backward compatibility with their existing modules/bundles. In other
>> words, their resolution logic would still be different from each other.
>
> And this is very likely to be the source of subtle but very painful
> interop issues. Which is why I also suggested that we should consider
> moving to a canonical, though evolvable, resolution mechanism.
>
>>
>>> A phased approach would be particularly beneficial if the initial phase
>>> could be delivered as part of Java 7 and subsequent phases implemented
>>> strictly on top of Java 7. But getting the API right up front might be
>>> tricky unless we can spot some really good abstractions or prototype
>>> the
>>> later phases sufficiently well. Is that the kind of phasing you had in
>>> mind?
>>
>> Once an API is put into the SE platform, it will stick around almost
>> forever. I think if we go with the phased approach on interoperability
>> support (I believe this is what Bryan suggested), it would be much
>> better to simply left out those features/APIs which do not have
>> implementations to back them up from JDK 7, rather than putting the
>> half-baked features/APIs into the release and regret later. ;) We can
>> always add new APIs in subsequence releases if we decide to offer better
>> degree of interoperability in the future.
>
> I couldn't agree more that we don't want to put half-baked APIs into SE!
> If we cannot expect significant validation of our interoperation model
> (whatever we settle on) from NetBeans and OSGi, then we shouldn't expose
> these APIs at all. Period.
>
> But this doesn't just apply to import-by-package. We haven't even
> defined how class and resource sharing will occur. I won't be a bit
> surprised if we want one or more new types here.
>
>
> And, last thoughts for the day...
>
> Even if you are adamantly opposed to import-by-package, please take a
> good look at the other ideas buried in my message from yesterday,
> particularly the ModuleContext proposal.
>
> It is completely independent of import-by-package, and seems to me at
> least to be a concise, flexible model for caching/isolating/finding
> Module instances. (The idea of exposing a resolve() method on it also
> makes a clean API, even if it delegates to ModuleSystem instead of a new
> resolver type.)
>
> There are other potentially interesting tidbits there as well
> (PlatformBinding exposure on definition, Query support for current
> platform, ImportDependency *as* Query).
>
> I'd hate for the baby to be thrown out with the bath water :^)
>
> // Bryan
>
>>
>> - Stanley
>>