Relationship to JSR 291 [was: Re: Bryan's comments]

[Bryan Atsatt]

Andy Piper wrote:
 > At 23:19 25/05/2007, Stanley M. Ho wrote:
 >> Anyway, it seems the EG consensus so far is to not add import package
 >> support. If any EG member disagrees, please speak up.
 >
 > Well, it depends on what the solution for enabling interoperation
 > with JSR 291 is.
 > Our requirement is that there must be a solution, if that requires
 > import package, so be it. If not then not.

Exactly.

I think we can all agree that, at minimum, interoperation means that
classes and resources are sharable *across* ModuleSystems at runtime.

Which implies that *import dependencies must be resolvable across
multiple ModuleSystem instances*. (BTW, I think we should change the
state name "PREPARING" to "RESOLVING" in 7.2.1.)

So the open issue is the richness of the import "language": must we
support only lowest-common-denominator, or can we do better without
over-complicating the design?

I for one would like to be able to have a single module express
dependencies on modules from both the same and different ModuleSystems,
*using the standard semantics of each*. This may be reaching too far,
but we should at least explore it seriously while we figure out what
interop means here...


BASICS

So far, we only know of two different import semantics: module-name, and
package-name. For discussion, let's call these:

a. import-module
b. import-package

So, to start, we could:

1. Support declaration of both import types. If 294 supports imports at
all, it should be relatively easy to support both, since a superpackage
  name is a module name, and it contains member package names. (Compiler
support is clearly the critical issue here, but it will obviously
require use of the 277 runtime, so the import *type* should be
transparent to it.) At worst, we'd need two new annotation types.

2. Provide API for both import types (e.g. ImportDependency has
getModuleName() and getPackageName() methods, one of which will return
null on a given instance).

However, we know these are necessary but not sufficient. Leaving aside
the resolution issue for a moment, support for import-package also
suggests that we:

3. Enable a single module to declare different versions for each of its
member packages.

4. Enable efficient Repository lookup by package name.

I think these are relatively easy (but *could* be considered optional).

We also need:

5. Standard Query types for lookup by module and package name.


EXISTING DEPENDENCY RESOLUTION MODEL

The more interesting issue is dependency resolution. But this hasn't
been discussed in any real detail, so lets do so before talking further
about import-package. To simplify this discussion, I'm ignoring
bundled/custom import policies for now...

Resolution in the current spec is delegated to the associated
ModuleSystem instance (7.2.2 #8). While the details are not spelled out,
the expectation appears to be that
ModuleSystem.getModule(ModuleDefinition) must:

- Select an initial repository. Call getRepository() on the input parameter.

Then, for each ImportDependency of the definition:

- Select a matching definition. Construct a Query from the
ImportDependency and use Repository.find() to lookup a matching
ModuleDefinition.

- Get an instance. Use def.getModuleSystem().getModule(def). The
ModuleSystem is expected to return a cached instance if available, or
create/cache/return one if not.


(TBD: The PlatformBinding must be taken into account somehow during
selection. ModuleDefinition must include an accessor for it, and either
Repository.find() should implicitly filter them, or the caller must
construct a Query which will do so. I think we should add a
CURRENT_PLATFORM constant to Query, which will evaluate to true if no
binding is present in a definition.)

(The spec also talks about Repository as the mechanism of isolation
(6.4). This was the case in the prototype, where the repository itself
provided caching. It doesn't appear to work with the current design.
There is no need that I can see to isolate ModuleDefinition
instances--it is Module instances with their associated loaders that may
require isolation.)

(Also note that if ImportDependency was itself a Query subclass, there
would be no need to do any mapping. And since the ModuleDefinition
subclass must produce ImportDependency instances, it can even produce
more specialized Query instances if desired.)


REFINEMENT

I think we can improve on the existing model in several ways:

A. Provide a model for Module isolation (e.g. for EE, Applets, etc).

B. Encapsulate all selection logic in a single mechanism.

C. Eliminate the overhead of the repository lookup when a cached
instance exists.

Let me propose a new class that encapsulates the caching logic, enables
lookup using Query, and supports multiple instances for isolation:

public abstract class ModuleContext {

     // Get the context used to define JRE modules.
     public static ModuleContext getBootstrapContext(){...};

     // Get the context used to define the main module.
     public static ModuleContext getSystemContext(){...};

     // Get all contexts.
     public static List<ModuleContext> getContexts() {...};

     // Add a new context.
     public static void addContext(ModuleContext ctx) {...}

     // Remove a context (error if == default).
     public static boolean removeContext(ModuleContext ctx) {...}

     // Get the parent context (null if bootstrap).
     public ModuleContext getParentContext(){...}

     // Get the name of this context.
     public String getContextName() {...}

     // Create a new Module instance and store it in the cache.
     public abstract Module createModule(ModuleDefinition def);

     // Find cached Module instances. Must check parent first.
     public abstract List<Module> findModules(Query query);

     // Set the context used for JRE modules.
     static void setBootstrapContext(ModuleContext ctx){...}

     // Set the context used to define the main module.
     static void setSystemContext(ModuleContext ctx){...}
}

The JVM will create an appropriate subtype and call
setBootstrapContext(). The launcher will create a child context and call
setSystemContext(). An EE (or similar) environment can create/remove new
contexts as needed for application isolation.

And the resolution algorithm can now check the cache *first*, before
doing a repository lookup, using the same mechanism in both. Query
should be used to express *all* selection criteria (including
attributes, which we have not yet directly supported).

Caches are no longer tied to ModuleSystem instances.
ModuleSystem.getModule() can become simply createModule(). The normal
implementation of ModuleContext.createModule() just calls
ModuleSystem.createModule() and caches/returns the result.


This class could easily be made concrete, but it may be useful to
support subtypes for specialization (e.g. event generation, lifecycle
management, specialized diagnostics, etc).


RESOLUTION MODELS

The current design requires that each ModuleSystem provide its own
resolution logic, and that each definition will be resolved by its
owning ModuleSystem. This model appears to provide flexibility for
significant differences in implementation, but we really don't know
enough at this point. Perhaps only an actual second implementation will
tell us if this provides useful flexibility.

It wouldn't surprise me if we have to keep tightening the spec as we go,
in order to remove inconsistencies that arise from the separate
algorithms. And this may eliminate flexibility to the point where it is
no longer useful. Much worse, we may not even discover this until after
the spec and RI are released, if that second implementation (e.g. OSGi)
is not completed beforehand.

We should at least consider the obvious alternative: one algorithm (to
rule them all :^). And I don't mean one hard-coded algorithm, I mean one
  replaceable, extensible class, such as:

public abstract class ImportResolver {
     public abstract Module resolve(ModuleDefinition def);
}

And we add a method to ModuleContext to retrieve an instance:

public abstract class ModuleContext {
     ...
     public abstract ImportResolver getImportResolver();
}

(Note that ImportResolver is now in a position to subsume the
functionality of both VisibilityPolicy and ImportOverridePolicy.)

Repository usage is encapsulated within the implementation of the
resolve() method. The full resolution algorithm becomes:

    context.getImportResolver().resolve(definition);

The launcher uses the "system" context for this. EE, Applets, etc. make
and use their own distinct, isolated instances.


RESOLUTION WITH IMPORT-NAME AND IMPORT-PACKAGE

With this scaffolding in place we can easily take a phased approach to
supporting import-package: the initial implementation simply does not
support it at runtime.

A subsequent implementation may support import-package, but only within
the boundaries of the same ModuleSystem.

And a full blown implementation may support import-package across
ModuleSystems.

We can build in support for selecting/configuring the ImportResolver as
a service, just as we plan to do with ModuleSystem (and Repository, I
presume).


And maybe, just maybe, we can find a way to abstract and re-use the
mature resolution logic from the OSGi reference implementation *as* the
one implementation to rule them all.

// Bryan