Relationship to JSR 291 [was: Re: Bryan's comments]

[Bryan Atsatt]

Responses inline, and a few clarifications here (I was a bit tired when
I finished this last night :^)...

The main point I was trying to make is that resolution must occur within
a specific context, but I don't think my example APIs showed that well.
I was assuming that ImportResolver had a ModuleContext as a field, but
we can make this much cleaner and easier to understand by passing it as
an argument:

public abstract class ImportResolver {
     public abstract Module resolve(ModuleDefinition def,
                                    ModuleContext ctx);
}

And we can really tie this together by adding a convenience method to
ModuleContext:

public abstract class ModuleContext {
     ...
     public Module resolve(ModuleDefinition def) {
         return getImportResolver().resolve(def, this);
     }
}

Now resolution becomes simply:

     context.resolve(definition);


(I also left out any use of the ImportPolicy, as it isn't yet clear to
me that it should remain a separate type in this model.)

// Bryan

Glyn Normington wrote:
>
> *Bryan Atsatt <bryan.atsatt at ORACLE.COM>* wrote on 30/05/2007 07:57:59:
>
>  > Andy Piper wrote:
>  >  > At 23:19 25/05/2007, Stanley M. Ho wrote:
>  >  >> Anyway, it seems the EG consensus so far is to not add import package
>  >  >> support. If any EG member disagrees, please speak up.
>  >  >
>  >  > Well, it depends on what the solution for enabling interoperation
>  >  > with JSR 291 is.
>  >  > Our requirement is that there must be a solution, if that requires
>  >  > import package, so be it. If not then not.
>  >
>  > Exactly.
>  >
>  > I think we can all agree that, at minimum, interoperation means that
>  > classes and resources are sharable *across* ModuleSystems at runtime.
>  >
>  > Which implies that *import dependencies must be resolvable across
>  > multiple ModuleSystem instances*. (BTW, I think we should change the
>  > state name "PREPARING" to "RESOLVING" in 7.2.1.)
>
> Agreed. We must avoid the trap of thinking that module system interop.
> can be achieved by exposing class loaders (as loadClass will happily
> load unexported classes).
>
>  >
>  > So the open issue is the richness of the import "language": must we
>  > support only lowest-common-denominator, or can we do better without
>  > over-complicating the design?
>  >
>  > I for one would like to be able to have a single module express
>  > dependencies on modules from both the same and different ModuleSystems,
>  > *using the standard semantics of each*. This may be reaching too far,
>  > but we should at least explore it seriously while we figure out what
>  > interop means here...
>
> At this point, I feel that is likely to be reaching too far, but I'm
> happy to play along and see what we can learn along the way.
>
>  >
>  >
>  > BASICS
>  >
>  > So far, we only know of two different import semantics: module-name, and
>  > package-name. For discussion, let's call these:
>  >
>  > a. import-module
>  > b. import-package
>  >
>  > So, to start, we could:
>  >
>  > 1. Support declaration of both import types. If 294 supports imports at
>  > all, it should be relatively easy to support both, since a superpackage
>  >   name is a module name, and it contains member package names. (Compiler
>  > support is clearly the critical issue here, but it will obviously
>  > require use of the 277 runtime, so the import *type* should be
>  > transparent to it.) At worst, we'd need two new annotation types.
>
> A superpackage name is a deployment module name in the JSR 277 model of
> one superpackage per deployment module, but I don't see any reason why a
> JSR 291 deployment module should not contain more than one superpackage.
> So if 294 were to support import, then its import-module would really be
> a superpackage import rather than a development module import.

If we end up with nested superpackages, might it make sense to model
multiple superpackages by enclosing them in a single top-level one?

>
>  >
>  > 2. Provide API for both import types (e.g. ImportDependency has
>  > getModuleName() and getPackageName() methods, one of which will return
>  > null on a given instance).
>  >
>  > However, we know these are necessary but not sufficient. Leaving aside
>  > the resolution issue for a moment, support for import-package also
>  > suggests that we:
>  >
>  > 3. Enable a single module to declare different versions for each of its
>  > member packages.
>  >
>  > 4. Enable efficient Repository lookup by package name.
>  >
>  > I think these are relatively easy (but *could* be considered optional).
>  >
>  > We also need:
>  >
>  > 5. Standard Query types for lookup by module and package name.
>  >
>  >
>  > EXISTING DEPENDENCY RESOLUTION MODEL
>  >
>  > The more interesting issue is dependency resolution. But this hasn't
>  > been discussed in any real detail, so lets do so before talking further
>  > about import-package. To simplify this discussion, I'm ignoring
>  > bundled/custom import policies for now...
>  >
>  > Resolution in the current spec is delegated to the associated
>  > ModuleSystem instance (7.2.2 #8). While the details are not spelled out,
>  > the expectation appears to be that
>  > ModuleSystem.getModule(ModuleDefinition) must:
>  >
>  > - Select an initial repository. Call getRepository() on the input
> parameter.
>  >
>  > Then, for each ImportDependency of the definition:
>  >
>  > - Select a matching definition. Construct a Query from the
>  > ImportDependency and use Repository.find() to lookup a matching
>  > ModuleDefinition.
>  >
>  > - Get an instance. Use def.getModuleSystem().getModule(def). The
>  > ModuleSystem is expected to return a cached instance if available, or
>  > create/cache/return one if not.
>
> I think there also needs to be some 'resolution context' object which
> explicitly denotes a particular resolution so that each module system
> can keep track of the state of a resolution. This is required when two
> or more imports of a given module from another module system need to
> resolve to the same module instance. A resolution context may also be
> needed for back-tracking when a set of module instances created earlier
> in resolution turn out not to satisfy all the necessary constraints.

Yes. I had (briefly :^) thought that we need only a List<Module> as
field in the ImportResolver to hold the resolved modules as we go. But
probably we need to keep some additional state along with each Module,
so a new type may be required. Let's call it ResolutionContext, and
create one at the start of each resolution:

public abstract class ImportResolver {

     public Module resolve(ModuleDefinition def, ModuleContext modCtx) {
         ResolutionContext resCtx = createResolutionContext();
         return resolve(def, modCtx, resCtx);
     }

     protected abstract ResolutionContext createResolutionContext();

     protected abstract Module resolve(ModuleDefinition def,
                                       ModuleContext modCtx,
                                       ResolutionContext resCtx);
}

Does this approach make sense? Any thoughts on details for
ResolutionContext?

>
>  >
>  >
>  > (TBD: The PlatformBinding must be taken into account somehow during
>  > selection. ModuleDefinition must include an accessor for it, and either
>  > Repository.find() should implicitly filter them, or the caller must
>  > construct a Query which will do so. I think we should add a
>  > CURRENT_PLATFORM constant to Query, which will evaluate to true if no
>  > binding is present in a definition.)
>  >
>  > (The spec also talks about Repository as the mechanism of isolation
>  > (6.4). This was the case in the prototype, where the repository itself
>  > provided caching. It doesn't appear to work with the current design.
>  > There is no need that I can see to isolate ModuleDefinition
>  > instances--it is Module instances with their associated loaders that may
>  > require isolation.)
>  >
>  > (Also note that if ImportDependency was itself a Query subclass, there
>  > would be no need to do any mapping. And since the ModuleDefinition
>  > subclass must produce ImportDependency instances, it can even produce
>  > more specialized Query instances if desired.)
>  >
>  >
>  > REFINEMENT
>  >
>  > I think we can improve on the existing model in several ways:
>  >
>  > A. Provide a model for Module isolation (e.g. for EE, Applets, etc).
>  >
>  > B. Encapsulate all selection logic in a single mechanism.
>  >
>  > C. Eliminate the overhead of the repository lookup when a cached
>  > instance exists.
>  >
>  > Let me propose a new class that encapsulates the caching logic, enables
>  > lookup using Query, and supports multiple instances for isolation:
>  >
>  > public abstract class ModuleContext {
>  >
>  >      // Get the context used to define JRE modules.
>  >      public static ModuleContext getBootstrapContext(){...};
>  >
>  >      // Get the context used to define the main module.
>  >      public static ModuleContext getSystemContext(){...};
>  >
>  >      // Get all contexts.
>  >      public static List<ModuleContext> getContexts() {...};
>  >
>  >      // Add a new context.
>  >      public static void addContext(ModuleContext ctx) {...}
>  >
>  >      // Remove a context (error if == default).
>  >      public static boolean removeContext(ModuleContext ctx) {...}
>  >
>  >      // Get the parent context (null if bootstrap).
>  >      public ModuleContext getParentContext(){...}
>  >
>  >      // Get the name of this context.
>  >      public String getContextName() {...}
>  >
>  >      // Create a new Module instance and store it in the cache.
>  >      public abstract Module createModule(ModuleDefinition def);
>  >
>  >      // Find cached Module instances. Must check parent first.
>  >      public abstract List<Module> findModules(Query query);
>  >
>  >      // Set the context used for JRE modules.
>  >      static void setBootstrapContext(ModuleContext ctx){...}
>  >
>  >      // Set the context used to define the main module.
>  >      static void setSystemContext(ModuleContext ctx){...}
>  > }
>  >
>  > The JVM will create an appropriate subtype and call
>  > setBootstrapContext(). The launcher will create a child context and call
>  > setSystemContext(). An EE (or similar) environment can create/remove new
>  > contexts as needed for application isolation.
>  >
>  > And the resolution algorithm can now check the cache *first*, before
>  > doing a repository lookup, using the same mechanism in both. Query
>  > should be used to express *all* selection criteria (including
>  > attributes, which we have not yet directly supported).
>  >
>  > Caches are no longer tied to ModuleSystem instances.
>  > ModuleSystem.getModule() can become simply createModule(). The normal
>  > implementation of ModuleContext.createModule() just calls
>  > ModuleSystem.createModule() and caches/returns the result.
>  >
>  >
>  > This class could easily be made concrete, but it may be useful to
>  > support subtypes for specialization (e.g. event generation, lifecycle
>  > management, specialized diagnostics, etc).
>  >
>  >
>  > RESOLUTION MODELS
>  >
>  > The current design requires that each ModuleSystem provide its own
>  > resolution logic, and that each definition will be resolved by its
>  > owning ModuleSystem. This model appears to provide flexibility for
>  > significant differences in implementation, but we really don't know
>  > enough at this point. Perhaps only an actual second implementation will
>  > tell us if this provides useful flexibility.
>  >
>  > It wouldn't surprise me if we have to keep tightening the spec as we go,
>  > in order to remove inconsistencies that arise from the separate
>  > algorithms. And this may eliminate flexibility to the point where it is
>  > no longer useful. Much worse, we may not even discover this until after
>  > the spec and RI are released, if that second implementation (e.g. OSGi)
>  > is not completed beforehand.
>  >
>  > We should at least consider the obvious alternative: one algorithm (to
>  > rule them all :^). And I don't mean one hard-coded algorithm, I mean one
>  >   replaceable, extensible class, such as:
>  >
>  > public abstract class ImportResolver {
>  >      public abstract Module resolve(ModuleDefinition def);
>  > }
>  >
>  > And we add a method to ModuleContext to retrieve an instance:
>  >
>  > public abstract class ModuleContext {
>  >      ...
>  >      public abstract ImportResolver getImportResolver();
>  > }
>  >
>  > (Note that ImportResolver is now in a position to subsume the
>  > functionality of both VisibilityPolicy and ImportOverridePolicy.)
>  >
>  > Repository usage is encapsulated within the implementation of the
>  > resolve() method. The full resolution algorithm becomes:
>  >
>  >     context.getImportResolver().resolve(definition);
>  >
>  > The launcher uses the "system" context for this. EE, Applets, etc. make
>  > and use their own distinct, isolated instances.
>  >
>  >
>  > RESOLUTION WITH IMPORT-NAME AND IMPORT-PACKAGE
>  >
>  > With this scaffolding in place we can easily take a phased approach to
>  > supporting import-package: the initial implementation simply does not
>  > support it at runtime.
>
> A phased approach would be particularly beneficial if the initial phase
> could be delivered as part of Java 7 and subsequent phases implemented
> strictly on top of Java 7. But getting the API right up front might be
> tricky unless we can spot some really good abstractions or prototype the
> later phases sufficiently well. Is that the kind of phasing you had in
> mind?

Yes. It would be *extremely* useful to have an OSGi
implementation/prototype well under way before Java 7 is completed, so
that we can fine tune the model as we learn.

>
>  >
>  > A subsequent implementation may support import-package, but only within
>  > the boundaries of the same ModuleSystem.
>  >
>  > And a full blown implementation may support import-package across
>  > ModuleSystems.
>  >
>  > We can build in support for selecting/configuring the ImportResolver as
>  > a service, just as we plan to do with ModuleSystem (and Repository, I
>  > presume).
>  >
>  >
>  > And maybe, just maybe, we can find a way to abstract and re-use the
>  > mature resolution logic from the OSGi reference implementation *as* the
>  > one implementation to rule them all.
>  >
>  > // Bryan
>
> Glyn
>
>
> ------------------------------------------------------------------------
>
> /
> /
>
> /Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU/
>
>
>
>
>
>