From glyn_normington at UK.IBM.COM  Fri Jun  1 05:47:53 2007
From: glyn_normington at UK.IBM.COM (Glyn Normington)
Date: Fri, 01 Jun 2007 13:47:53 +0100
Subject: Module isolation
In-Reply-To: <465F5198.2090805@oracle.com>
Message-ID: <OFD20EE2C8.4BD4A1F7-ON802572ED.00360624-802572ED.00464DBD@uk.ibm.com>

Bryan Atsatt <bryan.atsatt at ORACLE.COM> wrote on 31/05/2007 23:52:08:

> I'd like to continue this discussion separately from the
> 291/import-by-package thread :^).
>
> To recap, the spec (sections 1.2, 6.1, 6.4) talks about using separate
> Repository instances as a means to provide isolation for environments
> like EE and Applets.
>
> I believe this is an oversight--actually, a holdover from the ideas I
> put forth in the prototype, where Repository cached Module instances.
>
> But in the current design, that is no longer the case. Repositories hold
> ModuleDefinitions, not Modules, so they are not a suitable isolation
> mechanism. It is *Module* instances with their associated loaders that
> may require isolation.

Yes, I think you're right.

>
> The current design also assumes that ModuleSystems cache Module
> instances. This certainly makes more sense than Repository. However, it
> means that systems that require isolation must arrange for separate
> ModuleSystem instances. This seems quite awkward, and a poor
abstraction.
>
> So I think we need to introduce an explicit notion of an isolation
> context that:
>
> 1. Acts as a cache for Module instances.
> 2. Enables multiple instances, each forming a separate Module namespace.
> 3. Provides out-of-the box instances for bootstrap and "system" modules.
> 4. Provides a delegation model to link instances.
> 5. Provides a lookup mechanism identical to that of Repository.
>
> Any given resolution would take place within a single context. That is,
> an initial context is selected, and while existing instances can be
> retrieved from delegated contexts, *new* instances must be created in
> the initial context.

This seems like a module analogue of a class loader except that new module
instances are created in the initial context, which seems a bit peculiar.
Wouldn't this mean that Java platform module instances could get created
in an application context?

>
> To enforce this model, and to simplify the api, I believe the context
> should directly expose a "resolve" convenience method (which simply
> delegates the actual implementation).
>
> Here is a strawman API, with a simple parent-first delegation model:
>
> public abstract class ModuleContext {
>
>      // Get the context used to resolve JRE modules.
>      public static ModuleContext getBootstrapContext(){...};
>
>      // Get the context used to resolve the main module.
>      public static ModuleContext getSystemContext(){...};
>
>      // Get all contexts.
>      public static List<ModuleContext> getContexts() {...};
>
>      // Add a new context.
>      public static void addContext(ModuleContext ctx) {...}
>
>      // Remove a context (error if == default).
>      public static boolean removeContext(ModuleContext ctx) {...}
>
>      // Get the parent context (null if bootstrap).
>      public ModuleContext getParentContext(){...}
>
>      // Get the name of this context.
>      public String getContextName() {...}
>
>      // Add a new Module instance to the cache. Throw an error
>      // if an existing module contains the same definition.
>      public abstract void addModule(Module module);
>
>      // Find cached Module instances. Must check parent first.
>      public abstract List<Module> findModules(Query query);
>
>      // Remove a Module.
>      public abstract boolean remove(Module module);
>
>      // Remove all Modules created from the specified definition.
>      public static void removeAll(ModuleDefinition def) {...}
>
>      // Resolve a definition.
>      public Module resolve(ModuleDefinition def) {
>          ModuleSystem ms = def.getModuleSystem();
>          return ms.resolve(def, this);
>      }

Using module context as a resolution context seems to imply that
concurrent resolutions need to be serialised to avoid interference and so
some synchronisation needs adding. Or maybe the idea is to use a module
context instance per resolution, but that wouldn't fit with a module
context being a cache spanning multiple resolutions.

>
>      // Set the context used for JRE modules.
>      static void setBootstrapContext(ModuleContext ctx){...}
>
>      // Set the context used to define the main module.
>      static void setSystemContext(ModuleContext ctx){...}
> }
>
> The JVM would create an appropriate subtype and call
> setBootstrapContext(). JRE module instances would be cached here.
>
> The launcher would create a child context and call setSystemContext().
> The main module and any of its dependencies would be resolved here.
>
> An EE/Applet (or similar) environment can create/remove/use new contexts
> as needed for application isolation. Note that the these new contexts
> need not have the system context in their parent chain. But they do need
> the bootstrap context so the code should enforce that.
>
> The resolve() method here assumes that ModuleSystem.getModule() is
> replaced by:
>
> ModuleSystem {
>     ...
>     public abstract Module resolve(ModuleDefinition def,
>                                    ModuleContext ctx);
> }
>
> The ModuleSystem is no longer required to directly cache Modules, but to
> implement the resolution algorithm using the supplied context.
>
> ModuleSystem releaseModule() and disableModuleDefinition() methods just
> use remove() and removeAll(), respectively.
>
> This class could easily be made concrete, as the implementations are
> obviously simple. Regardless, for extensibility the actual types used at
> runtime should be under the control of the JVM/JRE and those systems
> that need to create new contexts.
>
> And, we might want to tie other existing ideas in here. For example,
> ModuleContext could be the home for "default" policies:
>
>     public abstract ImportPolicy getImportPolicy();
>     public abstract ImportOverridePolicy getImportOverridePolicy();
>
> Thoughts?

As I've pointed out on this list before, custom import policies have many
disadvantages including making it very hard to write management systems
that can infer dependencies before modules are installed in the module
system. Moving custom import policies out of individual modules into the
repository (which came up in discussion with Stanley and Michal at
JavaOne) or module context, leaving purely declarative metadata behind in
individual modules, would solve this problem. Then the default policy
would simply be the one returned by default. (I guess the same is true for
custom import override policies, which I'm also not fond of, particularly
at the module level.)

>
> // Bryan

Glyn





Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU





-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/attachments/20070601/3268311a/attachment.html 

From bryan.atsatt at oracle.com  Fri Jun  1 14:00:09 2007
From: bryan.atsatt at oracle.com (Bryan Atsatt)
Date: Fri, 01 Jun 2007 14:00:09 -0700
Subject: Module isolation
In-Reply-To: <OFD20EE2C8.4BD4A1F7-ON802572ED.00360624-802572ED.00464DBD@uk.ibm.com>
References: <OFD20EE2C8.4BD4A1F7-ON802572ED.00360624-802572ED.00464DBD@uk.ibm.com>
Message-ID: <466088D9.9020903@oracle.com>

Glyn Normington wrote:
>
> *Bryan Atsatt <bryan.atsatt at ORACLE.COM>* wrote on 31/05/2007 23:52:08:
>
>  > I'd like to continue this discussion separately from the
>  > 291/import-by-package thread :^).
>  >
>  > To recap, the spec (sections 1.2, 6.1, 6.4) talks about using separate
>  > Repository instances as a means to provide isolation for environments
>  > like EE and Applets.
>  >
>  > I believe this is an oversight--actually, a holdover from the ideas I
>  > put forth in the prototype, where Repository cached Module instances.
>  >
>  > But in the current design, that is no longer the case. Repositories hold
>  > ModuleDefinitions, not Modules, so they are not a suitable isolation
>  > mechanism. It is *Module* instances with their associated loaders that
>  > may require isolation.
>
> Yes, I think you're right.
>
>  >
>  > The current design also assumes that ModuleSystems cache Module
>  > instances. This certainly makes more sense than Repository. However, it
>  > means that systems that require isolation must arrange for separate
>  > ModuleSystem instances. This seems quite awkward, and a poor abstraction.
>  >
>  > So I think we need to introduce an explicit notion of an isolation
>  > context that:
>  >
>  > 1. Acts as a cache for Module instances.
>  > 2. Enables multiple instances, each forming a separate Module namespace.
>  > 3. Provides out-of-the box instances for bootstrap and "system" modules.
>  > 4. Provides a delegation model to link instances.
>  > 5. Provides a lookup mechanism identical to that of Repository.
>  >
>  > Any given resolution would take place within a single context. That is,
>  > an initial context is selected, and while existing instances can be
>  > retrieved from delegated contexts, *new* instances must be created in
>  > the initial context.
>
> This seems like a module analogue of a class loader except that new
> module instances are created in the initial context, which seems a bit
> peculiar. Wouldn't this mean that Java platform module instances could
> get created in an application context?


Argh. You're right, and this isn't appropriate for normal sharing. I was
thinking about this from the other perspective: isolation, where new
Module instances should end up in the new context.

But even there, we don't want to allow bootstrap Modules to be
re-instantiated (although this would be useful in some cases, if it
could be made to work).

Hmm. In my original model, binding a Module to a "context" (Repository,
in that case) was natural because, like Class.getClassLoader(),
ModuleDefinition.getRepository() allowed you to get the "defining"
context easily, so recursive resolution could always use the correct
context.

Given the current design, my premise was that it doesn't seem to make
any sense to bind a Module cache directly to a Repository when
Repository.find() returns ModuleDefinition. And ModuleSystem.getModule()
requires caching behavior, so I think this might have thrown me off track.

I'm guessing now that I've just not thinking about this the right way
(and the spec is misleading?). I had forgotten a conversation Stanley
and I had on this topic, which may be lurking somewhere between the
lines of the spec.

Stanley's idea was that a Repository implementation could return a
*copy* of a ModuleDefinition created by a different Repository. Such a
Repository would in fact form an isolation context. And caching Module
instances becomes trivial: each definition can just have one as a field.

While there are definite lifecycle issues here between such repository
instances, they are probably solvable.

And then we really *do* have an exact analogue of the loader delegation
model, in which:

    Repository       ~= ClassLoader
    ModuleDefinition ~= Class
    Module           ~= Object

Isolation of classes requires different Class instances. So isolation of
  modules requires different ModuleDefinition instances.

But this puts us back awfully close to my original design.

>
>  >
>  > To enforce this model, and to simplify the api, I believe the context
>  > should directly expose a "resolve" convenience method (which simply
>  > delegates the actual implementation).
>  >
>  > Here is a strawman API, with a simple parent-first delegation model:
>  >
>  > public abstract class ModuleContext {
>  >
>  >      // Get the context used to resolve JRE modules.
>  >      public static ModuleContext getBootstrapContext(){...};
>  >
>  >      // Get the context used to resolve the main module.
>  >      public static ModuleContext getSystemContext(){...};
>  >
>  >      // Get all contexts.
>  >      public static List<ModuleContext> getContexts() {...};
>  >
>  >      // Add a new context.
>  >      public static void addContext(ModuleContext ctx) {...}
>  >
>  >      // Remove a context (error if == default).
>  >      public static boolean removeContext(ModuleContext ctx) {...}
>  >
>  >      // Get the parent context (null if bootstrap).
>  >      public ModuleContext getParentContext(){...}
>  >
>  >      // Get the name of this context.
>  >      public String getContextName() {...}
>  >
>  >      // Add a new Module instance to the cache. Throw an error
>  >      // if an existing module contains the same definition.
>  >      public abstract void addModule(Module module);
>  >
>  >      // Find cached Module instances. Must check parent first.
>  >      public abstract List<Module> findModules(Query query);
>  >
>  >      // Remove a Module.
>  >      public abstract boolean remove(Module module);
>  >
>  >      // Remove all Modules created from the specified definition.
>  >      public static void removeAll(ModuleDefinition def) {...}
>  >
>  >      // Resolve a definition.
>  >      public Module resolve(ModuleDefinition def) {
>  >          ModuleSystem ms = def.getModuleSystem();
>  >          return ms.resolve(def, this);
>  >      }
>
> Using module context as a resolution context seems to imply that
> concurrent resolutions need to be serialised to avoid interference and
> so some synchronisation needs adding. Or maybe the idea is to use a
> module context instance per resolution, but that wouldn't fit with a
> module context being a cache spanning multiple resolutions.

Well, regardless of the resolution mechanism, there must be some
serialization in the process, else how will sharing occur? If two
modules both depend on X, and X hasn't been instantiated, we must ensure
that only 1 thread does the instantiation, and any others see the result.

>
>  >
>  >      // Set the context used for JRE modules.
>  >      static void setBootstrapContext(ModuleContext ctx){...}
>  >
>  >      // Set the context used to define the main module.
>  >      static void setSystemContext(ModuleContext ctx){...}
>  > }
>  >
>  > The JVM would create an appropriate subtype and call
>  > setBootstrapContext(). JRE module instances would be cached here.
>  >
>  > The launcher would create a child context and call setSystemContext().
>  > The main module and any of its dependencies would be resolved here.
>  >
>  > An EE/Applet (or similar) environment can create/remove/use new contexts
>  > as needed for application isolation. Note that the these new contexts
>  > need not have the system context in their parent chain. But they do need
>  > the bootstrap context so the code should enforce that.
>  >
>  > The resolve() method here assumes that ModuleSystem.getModule() is
>  > replaced by:
>  >
>  > ModuleSystem {
>  >     ...
>  >     public abstract Module resolve(ModuleDefinition def,
>  >                                    ModuleContext ctx);
>  > }
>  >
>  > The ModuleSystem is no longer required to directly cache Modules, but to
>  > implement the resolution algorithm using the supplied context.
>  >
>  > ModuleSystem releaseModule() and disableModuleDefinition() methods just
>  > use remove() and removeAll(), respectively.
>  >
>  > This class could easily be made concrete, as the implementations are
>  > obviously simple. Regardless, for extensibility the actual types used at
>  > runtime should be under the control of the JVM/JRE and those systems
>  > that need to create new contexts.
>  >
>  > And, we might want to tie other existing ideas in here. For example,
>  > ModuleContext could be the home for "default" policies:
>  >
>  >     public abstract ImportPolicy getImportPolicy();
>  >     public abstract ImportOverridePolicy getImportOverridePolicy();
>  >
>  > Thoughts?
>
> As I've pointed out on this list before, custom import policies have
> many disadvantages including making it very hard to write management
> systems that can infer dependencies before modules are installed in the
> module system. Moving custom import policies out of individual modules
> into the repository (which came up in discussion with Stanley and Michal
> at JavaOne) or module context, leaving purely declarative metadata
> behind in individual modules, would solve this problem. Then the default
> policy would simply be the one returned by default. (I guess the same is
> true for custom import override policies, which I'm also not fond of,
> particularly at the module level.)

Yep, I have also suggested moving this kind of logic to Repository. I
don't much like bundled ImportPolicies, even though Stanley has defended
most of the issues well, simply because of the extra complexity.
>
>  >
>  > // Bryan
>
> Glyn
>
>
> ------------------------------------------------------------------------
>
> /
> /
>
> /Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU/
>
>
>
>
>
>


From bryan.atsatt at oracle.com  Fri Jun  1 14:48:24 2007
From: bryan.atsatt at oracle.com (Bryan Atsatt)
Date: Fri, 01 Jun 2007 14:48:24 -0700
Subject: Exported resources
In-Reply-To: <465F68C8.4000103@sun.com>
References: <4654109D.40205@sun.com> <46548736.7020507@oracle.com>
	<465629EA.2050008@sun.com> <465658CE.5020409@oracle.com>
	<465DCE51.4010304@sun.com> <465DDCD0.8090703@oracle.com>
	<465DF6D6.5050608@sun.com> <465E0201.2030308@oracle.com>
	<465F68C8.4000103@sun.com>
Message-ID: <46609428.4000909@oracle.com>

Hi Stanley,

Sorry if I'm not being clear. Let me try to summarize:

- I *do* want 277 to support private (non-exported) resources, but

- I do not think we should use permissions as the enforcement mechanism.

- I believe the 'caller-class-is-member' enforcement mechanism is
sufficient, and

- I do realize that this means ResourceBundle.getBundle() (or similar)
will fail to find private resources; these must be made public.


Let me give more detailed reasoning for this position...

You are of course correct that a class loader is free to add permissions
to any class by assigning a ProtectionDomain with those permissions
during defineClass(). (Let's call these "hard-wired" permissions.)

So permission to access private resources could easily be hard-wired for
other classes *in the same module*.

To this point, there really isn't any meaningful difference between
using permissions or using my approach: both allow module classes to
directly access their private resources.

But if we want to grant that same permission to *any* class outside of
the module, it gets far more interesting. And it is this problem that
I'm concerned with.

For ResourceBundle.getBundle(), for example, to access a private
resource, we must grant permission to the *ResourceBundle* class. And
how do we do that?

We could hard-wire permission for ResourceBundle, but... what about
ServiceLoader? Ok, so let's just hard-wire permission for any JRE module.

Problem solved? Not at all: what about all the non JRE frameworks out in
the world that need the same kind of access?

Clearly we cannot hard-wire permissions for these. So now we need to use
policy file entries to do so. We could of course take this approach, but
it either leaves some poor admin running around to update policy files
as things break, or some fancy mechanism to grant access during
framework module install. Ick.

And, hard-wired or not, permissions *are* a friend model.

But 294 is not likely to provide such a model for classes.

So, even if we hard-wire permission for ResourceBundle to access a
private resource, how is it going to access a private ListResourceBundle
class? Or how will the ServiceLoader gain access to the service provider
class? How will third-party frameworks solve this problem?

It is hard for me to see how all this complexity is justified, simply to
hide resources that have never been hidden before.

This doesn't mean there is no place for private resources! It just means
that they can only be used by the module classes themselves. And this is
still a very valuable feature: developers can restrict access to
resources that are strictly implementation details.

// Bryan

Stanley M. Ho wrote:
> Hi Bryan,
>
> Bryan Atsatt wrote:
>> ...
>> No, I don't agree. That's what I said before :^). I *really* don't like
>> the idea that I'm going to have to explicitly grant permission to some
>> module to access my resource. This *significantly* complicates
>
> No, this was not what I suggested. If some other modules want to access
> your resource, your resources need to be exported. However, if you want
> to access your own private resource in your module (call
> ClassLoader.getResource*() directly or call ResourceBundle.getBundle(),
> etc.), then your code would require additional permissions.
>
>> deployment. How will the grant occur, especially given that each
>> environment can use an entirely different permission storage mechanism?
>>
>> Or are you thinking that the loaders in the module system will construct
>> ProtectionDomains pre-wired with permissions? If so, exactly which
>> entities will be blessed? Just the JRE code? What about all the existing
>> non-JRE frameworks in use?
>
> When your module instance is initialized, the module system can
> construct the ProtectionDomain with the appropriate permissions and
> associate that with the classes in your module, so your classes in your
> module will always have the necessary permissions to access your own
> resources.
>
>> Sure, resources have never had JVM access control applied to them. But I
>> don't think we should go to the other extreme (permissions) just so that
>> I can give "friend" access to private resources. I'd much prefer that
>> such resources simply be exported.
>>
>> And I don't see this as a particular problem, either. We've gotten along
>> just fine without private resources so far (or maybe there is some big
>> demand for this that I'm missing?).
>>
>> And what about the mismatch issue I brought up? Lots of existing
>> frameworks use the "services" pattern, in which:
>>
>> 1. A well known resource name is used in a getResource() call.
>> 2. The resource contains a string naming a provider class.
>> 3. The provider class is loaded using Class.forName().
>>
>> This is a simple but very powerful pattern, but it requires that *both*
>> the resource and the named class be accessible to the framework. If we
>> require a permission grant for the resource, but not for the class, it
>> will be very easy to get access to one and not the other. Is it really
>> worth opening up this potential headache to enable a friend model for
>> resources?
>>
>> // Bryan
>
> Note that the security permission approach can be abused if a module has
> all permissions, it can access even the private resources in other
> modules. This is the friend access as you mentioned, but this is not my
> intention of using the security permission approach.
>
> Sorry, I have to go and I can't response in details here. Are you saying
> that we should probably give up the idea of exported/private resources
> and make all resources in a module to be always public and accessible?
> Or are you saying that you still like the idea of exported/private
> resources but you don't like to address it using the security permission
> approach?
>
> - Stanley
>


From bryan.atsatt at oracle.com  Mon Jun  4 11:38:18 2007
From: bryan.atsatt at oracle.com (Bryan Atsatt)
Date: Mon, 04 Jun 2007 11:38:18 -0700
Subject: Import constraints
Message-ID: <46645C1A.2090809@oracle.com>

Currently the spec provides a somewhat odd mix of functionality here...

1. Module name. We have declaration syntax (superpackage name) and
runtime support for using a module name as a constraint
(ImportDependency class and Query.name()).

2. Versions. We have declaration syntax (VersionConstraint annotation)
and runtime support for using a collection of version/version-ranges as
a constraint (VersionConstraint/ImportDependency classes, and
Query.version()).

3. Attributes. We have declaration syntax for *assigning* attributes to
a definition (ModuleAttributes annotation), but no declaration syntax
for using them as constraints (a custom ImportPolicy is required). We
have runtime support for using them as constraints (Query.attribute()).

4. Extensible constraints. We have no declaration syntax, but we *do*
have runtime support for expressing any type of constraint and combining
them with boolean operators (Query).


I'd like to emphasize that Query should be seen *as the runtime
representation of a constraint*, and that:

     Any import declaration must be expressible as a Query.

This includes the module name (and potentially a package name, TBD).

This last is an important point: other than optimizing searches, a
module name is not special! It's just another Query element (ditto for a
package name Query), evaluated along with any others in a compound Query
(i.e. Query elements combined with boolean operators). A definition can
even be looked up *without* a module name, using nothing but attribute
matching.

When I initially designed Query (and Repository.find()), I wanted
something very flexible and extensible, with the expectation that it
could be used to represent the entire constraint model. But I don't
think we're using it as effectively as we can...

Ok, so?

Well, first, I think we ought to generalize the runtime dependency type
to fully leverage Query. So change ImportDependency to:

public class ImportDependency extends Query {
     private Query constraint;
     private boolean optional;
     private boolean reExport;

     public ImportDependency (Query constraint,
                              boolean optional,
                              boolean reExport) {
         this.constraint = constraint;
         this.optional = optional;
         this.reExport = reExport;
     }

     public boolean match(ModuleDefinition target) {
         return constraint.match(target);
     }

     public boolean isOptional() {
         return optional;
     }

     public boolean isReExported() {
         return reExport;
     }

     public String toString() {
         ...
     }
}

This way, we rely on the flexibility of Query to express *all*
constraints, and have a single concrete type to represent dependencies.

By having Dependency extend Query, instances can be directly passed to
Repository.find(); no conversion required. We could obviously just make
a getter for the Query if we want, but why not make this convenient to use?

And, more importantly, ModuleDefinition.getImportDependency()
implementations are free to return any kind of constraints they want,
simply by varying the Query.

Module name, if present, is just a Query element. Again, I think the
only place this (and/or package name) is explicitly needed is to enable
search *optimization*: if we want to avoid linear searches in
Repository.findModuleDefinition(Query) implementations, there needs to
be a way to narrow the search efficiently. But it's just an
optimization, and a topic for a different thread.


Second, I think we should eliminate VersionConstraint as a separate
class. It's functionality is easy to replace with Query elements that
match Version or VersionRange, strung together with appropriate boolean
operators.


Third, we should add a getExportedPackages() method to ModuleDefinition.
This way a Query can easily be created to search by package name.
Whether or not we choose to surface import-by-package declaratively, it
should at least be possible for a ModuleSystem to use such a Query.


Fourth, we should add support for simple attribute constraint
declarations. Enabling simple equality constraints ought to be easy
enough: an AttributeConstraint annotation that takes key/value lists
(key1==value1;key2==value2;...). We already have support for expressing
these as Query instances.

This simple type of attribute constraint is supported by OSGi. I don't
have a good sense of how common its usage is (Glyn/Richard?), but some
will likely view the absence of this support in 277 as an
interoperability issue.

More complex attribute comparisons could still be left up to custom
import policies. Or...


<dreaming-out-loud>

We could instead create a generic *query* declaration syntax, with
support for expressing module-name, package-name, version,
version-range, attribute matching, as well as boolean operators and
precedence.

This would eliminate the need for VersionConstraint/AttributeConstraint
annotations. The current VersionConstraint syntax using ";" separators
would no longer be required, as the boolean operators could be used instead.

And the expressiveness might reduce or eliminate the need for many
custom import policies.

</dreaming-out-loud>

Finally, Query ought to have a toString() implementation so that we can
log them, examine in a debugger, put them in error messages, etc.


Regardless of the approach, it would certainly be nice to provide some
symmetry between our declarative syntax and our runtime capabilities.

Thoughts?

// Bryan


From bryan.atsatt at oracle.com  Mon Jun  4 12:42:40 2007
From: bryan.atsatt at oracle.com (Bryan Atsatt)
Date: Mon, 04 Jun 2007 12:42:40 -0700
Subject: JSR 277 EG observer mailing list
In-Reply-To: <4648BA20.5070206@sun.com>
References: <46400CFB.1020004@sun.com> <46412161.9020902@sun.com>
	<4648BA20.5070206@sun.com>
Message-ID: <46646B30.2030500@oracle.com>

How are people supposed to find this? Shouldn't there be an easily
spotted link somewhere on the public 277 page?

// Bryan

Stanley M. Ho wrote:
> Hi JSR 277 experts,
>
> Just an update. This EG mailing list has been hooked up with the
> observer list over the weekend, and you will find the public mailing
> list archive here:
>
> http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/
>
> - Stanley
>


From bryan.atsatt at ORACLE.COM  Mon Jun  4 12:21:29 2007
From: bryan.atsatt at ORACLE.COM (Bryan Atsatt)
Date: Mon, 04 Jun 2007 12:21:29 -0700
Subject: Relationship to JSR 291 [was: Re: Bryan's comments]
In-Reply-To: <465F6157.7070706@sun.com>
References: <OF62C11B58.0C61920A-ON802572EB.0055969A-802572EB.005897E5@uk.ibm.com>
	<465DF2E4.4090603@sun.com> <465E2051.7020008@oracle.com>
	<465EBE00.2060306@ungoverned.org> <465F0D8C.5010606@oracle.com>
	<465F6157.7070706@sun.com>
Message-ID: <46646639.7080606@oracle.com>

Stanley M. Ho wrote:
> Hi Bryan,
>
> I don't think I will have time to response to all your comments this
> week, so my response would focus on the import-by-package discussion.
>
> First, the only import dependency we have consensus in this EG so far is
> import-by-module-name (or import-by-superpackage-name), and this was
> also what's in the EDR. That said, it is by no mean that we can't
> consider other import dependency granularities (e.g.
> import-by-package-name, or import-by-contract) in addition to
> import-by-module-name. I would like to focus this discussion more on
> whether it makes sense for JSR 277 to support additional import
> dependency granularity, mainly import-by-package-name, and if so, in
> what context.
>
> There are three cases we should consider:
>
> 1. 277 module imports another 277 module by package name
>
> I think our consensus so far is that this is nice-to-have or
> non-critical, given we already have support for import-by-module-name.
> To keep this discussion in focus, let's not dive into
> import-by-package-name v.s. import-by-module-name, and let's assume
> there is no support for #1 for now.

Agreed.

>
> 2. OSGi module imports 277 module by package name
>
> As I discussed this with Richard in previous emails, majority of the
> works for this fall under the OSGi framework if it wants to enable this
> kind of wiring, but we could probably make the implementor's life easier
> by provide the appropriate 277 reflective API if necessary.

Yes, the API would be useful.

>
> 3. 277 module imports OSGi module by package name
>
> If I understand you correctly, I think what you really want is the
> support for this case, so the existing OSGi bundles can be leveraged by
> 277 modules in a nature way.
>
> Let me say it up front that I don't oppose the idea of #3 in the context
> of interoperability. If #3 is what you really want, I think where we
> diverge is simply how to make it possible.
>
> Do you agree so far?

Yes.

>
> Bryan Atsatt wrote:
>> ...
>> I want 277 to explicitly expose the concept of import-by-package through
>> appropriate APIs.
>>
>> It would be very nice if the 277ModuleSystem implementation itself
>> directly supports import-by-package (case #3). But this is less critical
>> to me than case #4...
>>
>> I want a 277ModuleDefinition to be able to import-by-package an
>> OSGiModuleDefinition (i.e. a bundle exposed via the 277 APIs). I believe
>> this case will be a particularly important one. By the time SE 7 ships,
> ...
>
> As I described several emails back:
>
>  > This doesn't mean supporting the notion of import by package is
>  > impossible. What OSGi will expose from its repository is simply module
>  > definitions, and module definition is basically an abstraction.
>  > Therefore, it should be possible to treat an OSGi bundle as a module
>  > definition, as well as to treat each exported package in an OSGi
>  > bundle a module definition as well. The dependency supported in JSR
>  > 277 is import-by-module. In the JSR 277 module system, this is mapped
>  > to import-by-superpackage. In the OSGi module system, this can be
>  > mapped to import-by-OSGi-bundle or import-by-package. If other module
>  > systems want to support different import granularity or if OSGi
>  > evolves to add new import granularity in the future, this model should
>  > still work.
>
> Rather than surfacing the import-by-package at the API level, I think we
> can solve #3 by generalizing the import dependency concept as
> import-by-name, and this name can be mapped to superpackage-name in JSR
> 277 and OSGi-bundle-name or OSGi-exported-package-name in OSGi (i.e.
> exposing OSGi bundles and exported packages through implementing
> OSGiBundleModuleDefinition and OSGiPackageModuleDefinition). I think
> this is also what you intended to say at one point (correct me if I
> misunderstand you) by suggesting to change
> ImportDependency.getModuleName() into something like
> ImportDependency.getName().

I believe we are now thinking along the same lines.

I just sent an email on the topic of generalizing the import dependency
concept, with the subject "Import constraints". With the approach I
suggest there, the distinctions between module-name and package-name are
mostly transparent to the runtime (declarative support for package-name
is not addressed).

That isn't to say that there aren't resolution issues to be worked out
for package-name, but its a start.

>
> We expect the module-aware compiler would somehow recognize the import
> dependency defined in 277. In other words, if the OSGi framework exposes
> the module definitions appropriately, you'll get the compiler support.
> Since the import dependency is generic, the compiler and other tools ed
> to do extra work to support other forms of import dependency.

Yep. As I've said from the very beginning, I think the compiler is going
to have to rely on the 277 runtime to resolve *external* (i.e.
non-source) dependencies. So as long as the runtime can deal with
package-name, the compiler will too.

(Dependencies resolvable via included source will, I assume, just
*directly* resolve, without applying any constraint logic. That is, if a
class refers to a class in another superpackage, and that class &
superpackage are available in the same compilation, the compiler will
simply select them without applying *any* constraints. This may be too
simple of a model, but maybe it is sufficient.)

>
> In case if the 277 module would like to import these OSGi exported
> packages using an OSGi-like resolution algorithm, it should be possible
> for it to use a custom import policy (or import override policy or maybe
> some new override mechanisms) to get the behavior it wants.
>
> Do you think this is a workable approach to address #3?

I don't think custom code should be required; we need to make it easier
than that or it doesn't qualify as "first-class" citizenship, at least
not to me :^).


I guess I am still missing something fundamental--I keep thinking that
import-by-package is actually not so hard. There are three main components:

1. Declaration.
2. Runtime infrastructure.
3. Resolution algorithm.

I'm pretty sure that #2 is easy (and covered in my "Import constraints"
thread).

And we don't yet have *any* import declaration mechanism (yes, we're
assuming that this will be handled in 294, but it hasn't been brought up
yet). I don't see this as a difficult issue either way. It would be
natural to have both import-by-name and import-by-package in
superpackages, if we have import at all. If we need to use annotations
instead, it is easy to have both types.

So I can only assume that the *real* concern here is #3. And I can't
tell if this is a valid concern or not. I hope Glyn and Richard will
point out what I'm missing here...

What is the difference between selecting a ModuleDefinition based on its
name, or selecting it based on its contents? Either way, "shallow"
validation must still occur.

// Bryan

>
> - Stanley
>


From Stanley.Ho at sun.com  Mon Jun  4 18:56:01 2007
From: Stanley.Ho at sun.com (Stanley M. Ho)
Date: Mon, 04 Jun 2007 18:56:01 -0700
Subject: Exported resources
In-Reply-To: <46609428.4000909@oracle.com>
References: <4654109D.40205@sun.com> <46548736.7020507@oracle.com>
	<465629EA.2050008@sun.com> <465658CE.5020409@oracle.com>
	<465DCE51.4010304@sun.com> <465DDCD0.8090703@oracle.com>
	<465DF6D6.5050608@sun.com> <465E0201.2030308@oracle.com>
	<465F68C8.4000103@sun.com> <46609428.4000909@oracle.com>
Message-ID: <4664C2B1.1050109@sun.com>

Hi Bryan,

I would like to get closures on a few open issues first, so my responses
will be limited to those threads rather than all the new threads that
have been started recently.

Bryan Atsatt wrote:
> Hi Stanley,
>
> Sorry if I'm not being clear. Let me try to summarize:
>
> - I *do* want 277 to support private (non-exported) resources, but

Good.

> - I do not think we should use permissions as the enforcement mechanism.
>
> - I believe the 'caller-class-is-member' enforcement mechanism is
> sufficient, and
>
> - I do realize that this means ResourceBundle.getBundle() (or similar)
> will fail to find private resources; these must be made public.
>
> Let me give more detailed reasoning for this position...
>
> You are of course correct that a class loader is free to add permissions
> to any class by assigning a ProtectionDomain with those permissions
> during defineClass(). (Let's call these "hard-wired" permissions.)
>
> So permission to access private resources could easily be hard-wired for
> other classes *in the same module*.
>
> To this point, there really isn't any meaningful difference between
> using permissions or using my approach: both allow module classes to
> directly access their private resources.
>
> But if we want to grant that same permission to *any* class outside of
> the module, it gets far more interesting. And it is this problem that
> I'm concerned with.
>
> For ResourceBundle.getBundle(), for example, to access a private
> resource, we must grant permission to the *ResourceBundle* class. And
> how do we do that?
>
> We could hard-wire permission for ResourceBundle, but... what about
> ServiceLoader? Ok, so let's just hard-wire permission for any JRE module.

The system classes are granted with all permissions, so they will have
sufficient permissions to access the private resources in a module. I
don't think this is a real issue.

> Problem solved? Not at all: what about all the non JRE frameworks out in
> the world that need the same kind of access?

I agreed that using other frameworks might be a concern.

> Clearly we cannot hard-wire permissions for these. So now we need to use
> policy file entries to do so. We could of course take this approach, but
> it either leaves some poor admin running around to update policy files
> as things break, or some fancy mechanism to grant access during
> framework module install. Ick.

If the framework is bundled as part of your module, it will have the
same set of permissions as the other code in your module, so it will
have sufficient permissions to allow your code to access the private
resources. If the framework is deployed through the typical mechanisms
(e.g. extension classpath, etc.), it will have all permissions, hence it
will also have sufficient permissions to access the private resources.
Further, if the framework is deployed as another module in one of the
system's repositories, it will likely have all permissions to allow your
code to access the private resources in your module as well. Moreover,
if the framework JAR/module is signed and is trusted, the system will
typically grant all permissions to the code in the framework when the
code is loaded. In other words, there is no need to explicitly grant
permissions in most scenarios.

The scenario where this may be an issue is when the framework is
deployed through some other means (e.g. other modules in a
URLRepository, JARs in custom classloaders) *and* the framework is
unsigned (or if the framework is signed but not trusted for some
reasons), it may not have sufficient permissions to allow your code to
access the private resources through the framework API. However, it is
unclear to me if there are frameworks deployed like this for this issue
to become a real problem. Do you have any example that shows this is a
real problem?

> And, hard-wired or not, permissions *are* a friend model.
>
> But 294 is not likely to provide such a model for classes.
>
> So, even if we hard-wire permission for ResourceBundle to access a
> private resource, how is it going to access a private ListResourceBundle
> class? Or how will the ServiceLoader gain access to the service provider
> class? How will third-party frameworks solve this problem?

In a typical classloader, all public and private classes are visible
(return from loadClass()) externally but only the public ones are
accessible (invoke without access control failure). When a module uses
the ResourceBundle API to load the ListResourceBundle from the module
itself, the ResourceBundle API only loads the class from the module
classloader and the class is returned to the code in the module for
actual access. ServiceLoader works in similar way that the ServiceLoader
only loads the service-provider class but it's the actual caller who
accesses the class. Therefore, I don't think there is a real problem here.

> It is hard for me to see how all this complexity is justified, simply to
> hide resources that have never been hidden before.
>
> This doesn't mean there is no place for private resources! It just means
> that they can only be used by the module classes themselves. And this is
> still a very valuable feature: developers can restrict access to
> resources that are strictly implementation details.

I don't think the security permission approach is actually complex, and
I do have two concerns with the approach you suggested:

1. A module has to export its own supposed-to-be-private resources in
the same module in order to use the ResourceBundle API to load them.

2. It does not allow private resources in a module to be accessed
externally by any mean.

My main concern is around #1 'cause it doesn't sound right to me - I
think a private resource should stay non-exported. #2 is an issue if we
want to support a way for external code to access the private resources.
There is already a setAccessible() method that allows you to override
the default access control and to access private methods/fields in
classes, so I think it would also make sense to have a way for external
code to access the private resources when necessary (e.g. debugging,
diagnosis, etc.).

- Stanley


From Stanley.Ho at sun.com  Mon Jun  4 19:17:05 2007
From: Stanley.Ho at sun.com (Stanley M. Ho)
Date: Mon, 04 Jun 2007 19:17:05 -0700
Subject: Query...
In-Reply-To: <46548491.7040408@oracle.com>
References: <46548491.7040408@oracle.com>
Message-ID: <4664C7A1.1030303@sun.com>

Bryan Atsatt wrote:
> And we haven't done so in the spec, either. I think we should.

Query was not declared as Serializable in the spec because it was
unclear if we wanted to allow custom Query implementations and how it
might impact these custom Query implementations.

Now that we have decided to allow custom Query implementations, if we
make the Query serializable, the custom implementations will have to
implement serialization properly, but I think this is still okay.
However, if the query is serialized for it to be sent over the wire to a
remote repository, then its usefulness is pretty much limited to the
default query, because it is very unlikely that the remote repository
will happen to have the custom implementation classes in place for the
remote repository to deserialize the custom query properly.

If the EG still thinks this is a useful thing to add, I can incorporate
it into the next revision of the specification.

- Stanley


From Stanley.Ho at sun.com  Mon Jun  4 20:16:33 2007
From: Stanley.Ho at sun.com (Stanley M. Ho)
Date: Mon, 04 Jun 2007 20:16:33 -0700
Subject: Relationship to JSR 291 [was: Re: Bryan's comments]
In-Reply-To: <2071f55c84384fb5d141568f491677c9@ungoverned.org>
References: <OF62C11B58.0C61920A-ON802572EB.0055969A-802572EB.005897E5@uk.ibm.com>
	<465DF2E4.4090603@sun.com>
	<9E52FB97-F312-4A4F-A096-ECA0430F9D7C@ungoverned.org>
	<465F4D94.9060809@sun.com>
	<0e79b722e610c3de839326fb68539389@ungoverned.org>
	<465F62FC.7070201@sun.com>
	<2071f55c84384fb5d141568f491677c9@ungoverned.org>
Message-ID: <4664D591.1060608@sun.com>

Hi Richard,

Richard S.Hall wrote:
> ...
> Perhaps so.
>
> I am operating off my [possibly dated] recollection that 277 modules
> explicitly list the classes that they export, which meant that they may
> expose arbitrary public classes from a package. If this is no longer
> the case, then this is fine.
>
> The point remains, all legacy OSGi dependencies on a package assume
> that all public classes are exported from that package. An OSGi
> dependency on a given package cannot be resolved to a 277 module whose
> "exported public" classes are not the same as all "public" classes in
> the specific package.

I was under the impression that the class filtering is also in effect to
determine which public classes are exported from that package in OSGi
and that the OSGi dependency on a package already has a similar
"exported public" concept.

Anyway, these are all related to how the OSGi framework might import 277
modules by package name. I think it would be good to know if this is
actually a sensible thing that the OSGi framework wants to support
before we dive too deep on this. Glyn/Richard, is this something that
you foresee the OSGi framework will support?

- Stanley


From bryan.atsatt at oracle.com  Mon Jun  4 21:42:46 2007
From: bryan.atsatt at oracle.com (Bryan Atsatt)
Date: Mon, 04 Jun 2007 21:42:46 -0700
Subject: Exported resources
In-Reply-To: <4664C2B1.1050109@sun.com>
References: <4654109D.40205@sun.com> <46548736.7020507@oracle.com>
	<465629EA.2050008@sun.com> <465658CE.5020409@oracle.com>
	<465DCE51.4010304@sun.com> <465DDCD0.8090703@oracle.com>
	<465DF6D6.5050608@sun.com> <465E0201.2030308@oracle.com>
	<465F68C8.4000103@sun.com> <46609428.4000909@oracle.com>
	<4664C2B1.1050109@sun.com>
Message-ID: <4664E9C6.30608@oracle.com>

Hey Stanley,

Yeah, sorry about the flood of mail; I've had a little time to catch up.

You seem to have a JRE-centric perspective here ;^). Third-party
frameworks/libraries are precisely what a module system is for: to allow
their clients to *avoid* bundling the jars. And they aren't likely to be
deployed as extensions, either; not when there's this nice repository
model. Consider Spring. Or any of the hundreds of other third party
frameworks/libraries out there. We *want* these to be deployed as
normal, separate modules.

And signing a jar by itself doesn't grant any additional permissions, it
simply enables verification of the contained classes. Some environments
may support UI to grant additional permissions, of course, but that is
orthogonal.

But, lets leave aside the whole grant issue for now, and assume that we
can grant permissions as needed to access module private resources.

There is a much bigger issue, which is that this whole model falls apart
simply because we can't grant this type of "friend" access to module
private classes...

Your description of how ResourceBundle or ServiceLoader accesses the
class contains a big assumption, one that *could* be correct for such
JRE code, but is almost certainly wrong for third-party code.

These types of frameworks call Class.forName() or loader.loadClass().
And then they call either Class.newInstance() or Constructor.newInstance().

Regardless of whether or not they invoke any other methods, JSR 294 will
require an access control check at the point of instantiation, extending
those it performs today. Look at how Class.newInstance() and
Constructor.newInstance() both use:

     Reflection.ensureMemberAccess()

This is the logical point at which to inject the new module private
access control check.

Could this be made to succeed for JRE callers? Of course (but it
wouldn't be right, IMO).

But, more importantly: it won't, and cannot, succeed for third-party
callers without some sort of friend model for classes. Which isn't on
the table.


So, again, if we're not going to do this for classes, why should we do
it for resources?

// Bryan

Stanley M. Ho wrote:
> Hi Bryan,
>
> I would like to get closures on a few open issues first, so my responses
> will be limited to those threads rather than all the new threads that
> have been started recently.
>
> Bryan Atsatt wrote:
>> Hi Stanley,
>>
>> Sorry if I'm not being clear. Let me try to summarize:
>>
>> - I *do* want 277 to support private (non-exported) resources, but
>
> Good.
>
>> - I do not think we should use permissions as the enforcement mechanism.
>>
>> - I believe the 'caller-class-is-member' enforcement mechanism is
>> sufficient, and
>>
>> - I do realize that this means ResourceBundle.getBundle() (or similar)
>> will fail to find private resources; these must be made public.
>>
>> Let me give more detailed reasoning for this position...
>>
>> You are of course correct that a class loader is free to add permissions
>> to any class by assigning a ProtectionDomain with those permissions
>> during defineClass(). (Let's call these "hard-wired" permissions.)
>>
>> So permission to access private resources could easily be hard-wired for
>> other classes *in the same module*.
>>
>> To this point, there really isn't any meaningful difference between
>> using permissions or using my approach: both allow module classes to
>> directly access their private resources.
>>
>> But if we want to grant that same permission to *any* class outside of
>> the module, it gets far more interesting. And it is this problem that
>> I'm concerned with.
>>
>> For ResourceBundle.getBundle(), for example, to access a private
>> resource, we must grant permission to the *ResourceBundle* class. And
>> how do we do that?
>>
>> We could hard-wire permission for ResourceBundle, but... what about
>> ServiceLoader? Ok, so let's just hard-wire permission for any JRE module.
>
> The system classes are granted with all permissions, so they will have
> sufficient permissions to access the private resources in a module. I
> don't think this is a real issue.
>
>> Problem solved? Not at all: what about all the non JRE frameworks out in
>> the world that need the same kind of access?
>
> I agreed that using other frameworks might be a concern.
>
>> Clearly we cannot hard-wire permissions for these. So now we need to use
>> policy file entries to do so. We could of course take this approach, but
>> it either leaves some poor admin running around to update policy files
>> as things break, or some fancy mechanism to grant access during
>> framework module install. Ick.
>
> If the framework is bundled as part of your module, it will have the
> same set of permissions as the other code in your module, so it will
> have sufficient permissions to allow your code to access the private
> resources. If the framework is deployed through the typical mechanisms
> (e.g. extension classpath, etc.), it will have all permissions, hence it
> will also have sufficient permissions to access the private resources.
> Further, if the framework is deployed as another module in one of the
> system's repositories, it will likely have all permissions to allow your
> code to access the private resources in your module as well. Moreover,
> if the framework JAR/module is signed and is trusted, the system will
> typically grant all permissions to the code in the framework when the
> code is loaded. In other words, there is no need to explicitly grant
> permissions in most scenarios.
>
> The scenario where this may be an issue is when the framework is
> deployed through some other means (e.g. other modules in a
> URLRepository, JARs in custom classloaders) *and* the framework is
> unsigned (or if the framework is signed but not trusted for some
> reasons), it may not have sufficient permissions to allow your code to
> access the private resources through the framework API. However, it is
> unclear to me if there are frameworks deployed like this for this issue
> to become a real problem. Do you have any example that shows this is a
> real problem?
>
>> And, hard-wired or not, permissions *are* a friend model.
>>
>> But 294 is not likely to provide such a model for classes.
>>
>> So, even if we hard-wire permission for ResourceBundle to access a
>> private resource, how is it going to access a private ListResourceBundle
>> class? Or how will the ServiceLoader gain access to the service provider
>> class? How will third-party frameworks solve this problem?
>
> In a typical classloader, all public and private classes are visible
> (return from loadClass()) externally but only the public ones are
> accessible (invoke without access control failure). When a module uses
> the ResourceBundle API to load the ListResourceBundle from the module
> itself, the ResourceBundle API only loads the class from the module
> classloader and the class is returned to the code in the module for
> actual access. ServiceLoader works in similar way that the ServiceLoader
> only loads the service-provider class but it's the actual caller who
> accesses the class. Therefore, I don't think there is a real problem here.
>
>> It is hard for me to see how all this complexity is justified, simply to
>> hide resources that have never been hidden before.
>>
>> This doesn't mean there is no place for private resources! It just means
>> that they can only be used by the module classes themselves. And this is
>> still a very valuable feature: developers can restrict access to
>> resources that are strictly implementation details.
>
> I don't think the security permission approach is actually complex, and
> I do have two concerns with the approach you suggested:
>
> 1. A module has to export its own supposed-to-be-private resources in
> the same module in order to use the ResourceBundle API to load them.
>
> 2. It does not allow private resources in a module to be accessed
> externally by any mean.
>
> My main concern is around #1 'cause it doesn't sound right to me - I
> think a private resource should stay non-exported. #2 is an issue if we
> want to support a way for external code to access the private resources.
> There is already a setAccessible() method that allows you to override
> the default access control and to access private methods/fields in
> classes, so I think it would also make sense to have a way for external
> code to access the private resources when necessary (e.g. debugging,
> diagnosis, etc.).
>
> - Stanley
>


From bryan.atsatt at oracle.com  Mon Jun  4 22:37:55 2007
From: bryan.atsatt at oracle.com (Bryan Atsatt)
Date: Mon, 04 Jun 2007 22:37:55 -0700
Subject: Query...
In-Reply-To: <4664C7A1.1030303@sun.com>
References: <46548491.7040408@oracle.com> <4664C7A1.1030303@sun.com>
Message-ID: <4664F6B3.7010109@oracle.com>

Sure, but it isn't like this problem has no solution. RMI (JRMP or IIOP)
will marshall classes, and surely some enterprising developer will
implement a solution using our very own Repository model (marshall an
"import"+class-name+repository-name back to the originator, or better
yet, to a large centralized repository, etc.).

// Bryan

Stanley M. Ho wrote:
> Bryan Atsatt wrote:
>> And we haven't done so in the spec, either. I think we should.
>
> Query was not declared as Serializable in the spec because it was
> unclear if we wanted to allow custom Query implementations and how it
> might impact these custom Query implementations.
>
> Now that we have decided to allow custom Query implementations, if we
> make the Query serializable, the custom implementations will have to
> implement serialization properly, but I think this is still okay.
> However, if the query is serialized for it to be sent over the wire to a
> remote repository, then its usefulness is pretty much limited to the
> default query, because it is very unlikely that the remote repository
> will happen to have the custom implementation classes in place for the
> remote repository to deserialize the custom query properly.
>
> If the EG still thinks this is a useful thing to add, I can incorporate
> it into the next revision of the specification.
>
> - Stanley
>


From Stanley.Ho at sun.com  Mon Jun  4 22:49:46 2007
From: Stanley.Ho at sun.com (Stanley M. Ho)
Date: Mon, 04 Jun 2007 22:49:46 -0700
Subject: Relationship to JSR 291 [was: Re: Bryan's comments]
In-Reply-To: <46646639.7080606@oracle.com>
References: <OF62C11B58.0C61920A-ON802572EB.0055969A-802572EB.005897E5@uk.ibm.com>
	<465DF2E4.4090603@sun.com> <465E2051.7020008@oracle.com>
	<465EBE00.2060306@ungoverned.org> <465F0D8C.5010606@oracle.com>
	<465F6157.7070706@sun.com> <46646639.7080606@oracle.com>
Message-ID: <4664F97A.20007@sun.com>

Hi Bryan,

Bryan Atsatt wrote:
>> ...
>> Rather than surfacing the import-by-package at the API level, I think we
>> can solve #3 by generalizing the import dependency concept as
>> import-by-name, and this name can be mapped to superpackage-name in JSR
>> 277 and OSGi-bundle-name or OSGi-exported-package-name in OSGi (i.e.
>> exposing OSGi bundles and exported packages through implementing
>> OSGiBundleModuleDefinition and OSGiPackageModuleDefinition). I think
>> this is also what you intended to say at one point (correct me if I
>> misunderstand you) by suggesting to change
>> ImportDependency.getModuleName() into something like
>> ImportDependency.getName().
>
> I believe we are now thinking along the same lines.

Good to know.

> I just sent an email on the topic of generalizing the import dependency
> concept, with the subject "Import constraints". With the approach I
> suggest there, the distinctions between module-name and package-name are
> mostly transparent to the runtime (declarative support for package-name
> is not addressed).

I will reply to that thread later.

>> In case if the 277 module would like to import these OSGi exported
>> packages using an OSGi-like resolution algorithm, it should be possible
>> for it to use a custom import policy (or import override policy or maybe
>> some new override mechanisms) to get the behavior it wants.
>>
>> Do you think this is a workable approach to address #3?
>
> I don't think custom code should be required; we need to make it easier
> than that or it doesn't qualify as "first-class" citizenship, at least
> not to me :^).

The OSGi resolution algorithm has evolved significantly between R3 and
R4, and it will likely continue to evolve in certain degree in future
releases (Glyn/Richard, please correct me if I'm wrong). From this
perspective, I don't think it makes sense to have build-in OSGi-like
resolution algorithm in 277, because it will stick forever once 277 is
part of the SE platform and cannot be evolve easily.

I think it would makes more sense to allow OSGi-like resolution logic to
be plugged into the 277 module which imports OSGi bundles/packages. We
could argue whether it should be plugged in per-module in the form of
custom import policy, or per-system(or per-repository?) in the form of
import override policy, or maybe even have a new mechanism to allow
certain class of modules (e.g. 277 modules importing OSGi modules, or
277 modules importing NetBeans modules, etc.) to always use a specific
import policy automatically.  But my point is that OSGi-like resolution
logic is something that will need to evolve in the future and should not
be built-in. I think if there is a way to plug in OSGi-like resolution
logic into 277 module which imports OSGi bundles, it would still qualify
as "first-class" citizenship to me. ;)

> And we don't yet have *any* import declaration mechanism (yes, we're
> assuming that this will be handled in 294, but it hasn't been brought up
> yet). I don't see this as a difficult issue either way. It would be
> natural to have both import-by-name and import-by-package in
> superpackages, if we have import at all. If we need to use annotations
> instead, it is easy to have both types.

As we discussed (and I think you also agreed), we can make the import
dependency to be generic with a name, and the name could be a module
name (or a package name in the context of OSGi interoperability). We'll
only need to have a single import-by-name either in the superpackage or
in the form of annotation.

- Stanley


From glyn_normington at UK.IBM.COM  Fri Jun  1 02:43:27 2007
From: glyn_normington at UK.IBM.COM (Glyn Normington)
Date: Fri, 01 Jun 2007 10:43:27 +0100
Subject: Relationship to JSR 291 [was: Re: Bryan's comments]
In-Reply-To: <OF084663C8.EE3A0E10-ON802572EC.0058BFEB-802572EC.00594B64@uk.ibm.com>
Message-ID: <OF89FAD0A0.1337D999-ON802572ED.0034FF2E-802572ED.00356AED@uk.ibm.com>

After writing:

>I think ResolutionContext doesn't need any detail

I remembered the class in the interoperation kernel prototype did have a
little detail (but not much):

public interface ResolutionContext extends KernelRelated {

    /**
     *  Notify completion.
     */
    void complete();

}

I blogged ([1]) the README and class diagrams from the prototype for ease
of reference.

Glyn
[1]
http://underlap.blogspot.com/2007/06/module-system-interoperation-kernel.html

glyn_normington at uk.ibm.com wrote on 31/05/2007 17:15:19:

>
> Bryan Atsatt <bryan.atsatt at ORACLE.COM> wrote on 30/05/2007 19:11:02:
>
> > Responses inline, and a few clarifications here (I was a bit tired
when
> > I finished this last night :^)...
> >
> > The main point I was trying to make is that resolution must occur
within
> > a specific context, but I don't think my example APIs showed that
well.
> > I was assuming that ImportResolver had a ModuleContext as a field, but
> > we can make this much cleaner and easier to understand by passing it
as
> > an argument:
> >
> > public abstract class ImportResolver {
> >      public abstract Module resolve(ModuleDefinition def,
> >                                     ModuleContext ctx);
> > }
> >
> > And we can really tie this together by adding a convenience method to
> > ModuleContext:
> >
> > public abstract class ModuleContext {
> >      ...
> >      public Module resolve(ModuleDefinition def) {
> >          return getImportResolver().resolve(def, this);
> >      }
> > }
> >
> > Now resolution becomes simply:
> >
> >      context.resolve(definition);
> >
> >
> > (I also left out any use of the ImportPolicy, as it isn't yet clear to
> > me that it should remain a separate type in this model.)
> >
> > // Bryan
> >
> > Glyn Normington wrote:
> > >
> > > *Bryan Atsatt <bryan.atsatt at ORACLE.COM>* wrote on 30/05/2007
07:57:59:
> > >
> > >  > Andy Piper wrote:
> > >  >  > At 23:19 25/05/2007, Stanley M. Ho wrote:
> > >  >  >> Anyway, it seems the EG consensus so far is to not add
> import package
> > >  >  >> support. If any EG member disagrees, please speak up.
> > >  >  >
> > >  >  > Well, it depends on what the solution for enabling
interoperation
> > >  >  > with JSR 291 is.
> > >  >  > Our requirement is that there must be a solution, if that
requires
> > >  >  > import package, so be it. If not then not.
> > >  >
> > >  > Exactly.
> > >  >
> > >  > I think we can all agree that, at minimum, interoperation means
that
> > >  > classes and resources are sharable *across* ModuleSystems at
runtime.
> > >  >
> > >  > Which implies that *import dependencies must be resolvable across
> > >  > multiple ModuleSystem instances*. (BTW, I think we should change
the
> > >  > state name "PREPARING" to "RESOLVING" in 7.2.1.)
> > >
> > > Agreed. We must avoid the trap of thinking that module system
interop.
> > > can be achieved by exposing class loaders (as loadClass will happily
> > > load unexported classes).
> > >
> > >  >
> > >  > So the open issue is the richness of the import "language": must
we
> > >  > support only lowest-common-denominator, or can we do better
without
> > >  > over-complicating the design?
> > >  >
> > >  > I for one would like to be able to have a single module express
> > >  > dependencies on modules from both the same and different
ModuleSystems,
> > >  > *using the standard semantics of each*. This may be reaching too
far,
> > >  > but we should at least explore it seriously while we figure out
what
> > >  > interop means here...
> > >
> > > At this point, I feel that is likely to be reaching too far, but I'm
> > > happy to play along and see what we can learn along the way.
> > >
> > >  >
> > >  >
> > >  > BASICS
> > >  >
> > >  > So far, we only know of two different import semantics:
> module-name, and
> > >  > package-name. For discussion, let's call these:
> > >  >
> > >  > a. import-module
> > >  > b. import-package
> > >  >
> > >  > So, to start, we could:
> > >  >
> > >  > 1. Support declaration of both import types. If 294 supports
imports at
> > >  > all, it should be relatively easy to support both, since a
superpackage
> > >  >   name is a module name, and it contains member package
> names. (Compiler
> > >  > support is clearly the critical issue here, but it will obviously
> > >  > require use of the 277 runtime, so the import *type* should be
> > >  > transparent to it.) At worst, we'd need two new annotation types.
> > >
> > > A superpackage name is a deployment module name in the JSR 277 model
of
> > > one superpackage per deployment module, but I don't see any reason
why a
> > > JSR 291 deployment module should not contain more than one
superpackage.
> > > So if 294 were to support import, then its import-module would
really be
> > > a superpackage import rather than a development module import.
> >
> > If we end up with nested superpackages, might it make sense to model
> > multiple superpackages by enclosing them in a single top-level one?
>
> That is an option, but of course each nested superpackage has to
> name its parent, so it wouldn't be possible to combine superpackages
> from independent groups or sources without either modifying their
> superpackage declarations or getting them to agree on the name of
> the parent superpackage and code it themselves.
>
> >
> > >
> > >  >
> > >  > 2. Provide API for both import types (e.g. ImportDependency has
> > >  > getModuleName() and getPackageName() methods, one of which will
return
> > >  > null on a given instance).
> > >  >
> > >  > However, we know these are necessary but not sufficient. Leaving
aside
> > >  > the resolution issue for a moment, support for import-package
also
> > >  > suggests that we:
> > >  >
> > >  > 3. Enable a single module to declare different versions for each
of its
> > >  > member packages.
> > >  >
> > >  > 4. Enable efficient Repository lookup by package name.
> > >  >
> > >  > I think these are relatively easy (but *could* be considered
optional).
> > >  >
> > >  > We also need:
> > >  >
> > >  > 5. Standard Query types for lookup by module and package name.
> > >  >
> > >  >
> > >  > EXISTING DEPENDENCY RESOLUTION MODEL
> > >  >
> > >  > The more interesting issue is dependency resolution. But this
hasn't
> > >  > been discussed in any real detail, so lets do so before talking
further
> > >  > about import-package. To simplify this discussion, I'm ignoring
> > >  > bundled/custom import policies for now...
> > >  >
> > >  > Resolution in the current spec is delegated to the associated
> > >  > ModuleSystem instance (7.2.2 #8). While the details are not
> spelled out,
> > >  > the expectation appears to be that
> > >  > ModuleSystem.getModule(ModuleDefinition) must:
> > >  >
> > >  > - Select an initial repository. Call getRepository() on the input
> > > parameter.
> > >  >
> > >  > Then, for each ImportDependency of the definition:
> > >  >
> > >  > - Select a matching definition. Construct a Query from the
> > >  > ImportDependency and use Repository.find() to lookup a matching
> > >  > ModuleDefinition.
> > >  >
> > >  > - Get an instance. Use def.getModuleSystem().getModule(def). The
> > >  > ModuleSystem is expected to return a cached instance if
available, or
> > >  > create/cache/return one if not.
> > >
> > > I think there also needs to be some 'resolution context' object
which
> > > explicitly denotes a particular resolution so that each module
system
> > > can keep track of the state of a resolution. This is required when
two
> > > or more imports of a given module from another module system need to
> > > resolve to the same module instance. A resolution context may also
be
> > > needed for back-tracking when a set of module instances created
earlier
> > > in resolution turn out not to satisfy all the necessary constraints.
> >
> > Yes. I had (briefly :^) thought that we need only a List<Module> as
> > field in the ImportResolver to hold the resolved modules as we go. But
> > probably we need to keep some additional state along with each Module,
> > so a new type may be required. Let's call it ResolutionContext, and
> > create one at the start of each resolution:
> >
> > public abstract class ImportResolver {
> >
> >      public Module resolve(ModuleDefinition def, ModuleContext modCtx)
{
> >          ResolutionContext resCtx = createResolutionContext();
> >          return resolve(def, modCtx, resCtx);
> >      }
> >
> >      protected abstract ResolutionContext createResolutionContext();
> >
> >      protected abstract Module resolve(ModuleDefinition def,
> >                                        ModuleContext modCtx,
> >                                        ResolutionContext resCtx);
> > }
> >
> > Does this approach make sense? Any thoughts on details for
> > ResolutionContext?
>
> Seems reasonable. I think ResolutionContext doesn't need any detail
> - it can simply be used as the key of a map recording the
> resolutions that a module system is currently aware of.
>
> >
> > >
> > >  >
> > >  >
> > >  > (TBD: The PlatformBinding must be taken into account somehow
during
> > >  > selection. ModuleDefinition must include an accessor for it, and
either
> > >  > Repository.find() should implicitly filter them, or the caller
must
> > >  > construct a Query which will do so. I think we should add a
> > >  > CURRENT_PLATFORM constant to Query, which will evaluate to true
if no
> > >  > binding is present in a definition.)
> > >  >
> > >  > (The spec also talks about Repository as the mechanism of
isolation
> > >  > (6.4). This was the case in the prototype, where the repository
itself
> > >  > provided caching. It doesn't appear to work with the current
design.
> > >  > There is no need that I can see to isolate ModuleDefinition
> > >  > instances--it is Module instances with their associated
> loaders that may
> > >  > require isolation.)
> > >  >
> > >  > (Also note that if ImportDependency was itself a Query subclass,
there
> > >  > would be no need to do any mapping. And since the
ModuleDefinition
> > >  > subclass must produce ImportDependency instances, it can even
produce
> > >  > more specialized Query instances if desired.)
> > >  >
> > >  >
> > >  > REFINEMENT
> > >  >
> > >  > I think we can improve on the existing model in several ways:
> > >  >
> > >  > A. Provide a model for Module isolation (e.g. for EE, Applets,
etc).
> > >  >
> > >  > B. Encapsulate all selection logic in a single mechanism.
> > >  >
> > >  > C. Eliminate the overhead of the repository lookup when a cached
> > >  > instance exists.
> > >  >
> > >  > Let me propose a new class that encapsulates the caching logic,
enables
> > >  > lookup using Query, and supports multiple instances for
isolation:
> > >  >
> > >  > public abstract class ModuleContext {
> > >  >
> > >  >      // Get the context used to define JRE modules.
> > >  >      public static ModuleContext getBootstrapContext(){...};
> > >  >
> > >  >      // Get the context used to define the main module.
> > >  >      public static ModuleContext getSystemContext(){...};
> > >  >
> > >  >      // Get all contexts.
> > >  >      public static List<ModuleContext> getContexts() {...};
> > >  >
> > >  >      // Add a new context.
> > >  >      public static void addContext(ModuleContext ctx) {...}
> > >  >
> > >  >      // Remove a context (error if == default).
> > >  >      public static boolean removeContext(ModuleContext ctx) {...}
> > >  >
> > >  >      // Get the parent context (null if bootstrap).
> > >  >      public ModuleContext getParentContext(){...}
> > >  >
> > >  >      // Get the name of this context.
> > >  >      public String getContextName() {...}
> > >  >
> > >  >      // Create a new Module instance and store it in the cache.
> > >  >      public abstract Module createModule(ModuleDefinition def);
> > >  >
> > >  >      // Find cached Module instances. Must check parent first.
> > >  >      public abstract List<Module> findModules(Query query);
> > >  >
> > >  >      // Set the context used for JRE modules.
> > >  >      static void setBootstrapContext(ModuleContext ctx){...}
> > >  >
> > >  >      // Set the context used to define the main module.
> > >  >      static void setSystemContext(ModuleContext ctx){...}
> > >  > }
> > >  >
> > >  > The JVM will create an appropriate subtype and call
> > >  > setBootstrapContext(). The launcher will create a child
> context and call
> > >  > setSystemContext(). An EE (or similar) environment can
> create/remove new
> > >  > contexts as needed for application isolation.
> > >  >
> > >  > And the resolution algorithm can now check the cache *first*,
before
> > >  > doing a repository lookup, using the same mechanism in both.
Query
> > >  > should be used to express *all* selection criteria (including
> > >  > attributes, which we have not yet directly supported).
> > >  >
> > >  > Caches are no longer tied to ModuleSystem instances.
> > >  > ModuleSystem.getModule() can become simply createModule(). The
normal
> > >  > implementation of ModuleContext.createModule() just calls
> > >  > ModuleSystem.createModule() and caches/returns the result.
> > >  >
> > >  >
> > >  > This class could easily be made concrete, but it may be useful to
> > >  > support subtypes for specialization (e.g. event generation,
lifecycle
> > >  > management, specialized diagnostics, etc).
> > >  >
> > >  >
> > >  > RESOLUTION MODELS
> > >  >
> > >  > The current design requires that each ModuleSystem provide its
own
> > >  > resolution logic, and that each definition will be resolved by
its
> > >  > owning ModuleSystem. This model appears to provide flexibility
for
> > >  > significant differences in implementation, but we really don't
know
> > >  > enough at this point. Perhaps only an actual second
implementation will
> > >  > tell us if this provides useful flexibility.
> > >  >
> > >  > It wouldn't surprise me if we have to keep tightening the
> spec as we go,
> > >  > in order to remove inconsistencies that arise from the separate
> > >  > algorithms. And this may eliminate flexibility to the point where
it is
> > >  > no longer useful. Much worse, we may not even discover this until
after
> > >  > the spec and RI are released, if that second implementation (e.g.
OSGi)
> > >  > is not completed beforehand.
> > >  >
> > >  > We should at least consider the obvious alternative: one
algorithm (to
> > >  > rule them all :^). And I don't mean one hard-coded algorithm,I
mean one
> > >  >   replaceable, extensible class, such as:
> > >  >
> > >  > public abstract class ImportResolver {
> > >  >      public abstract Module resolve(ModuleDefinition def);
> > >  > }
> > >  >
> > >  > And we add a method to ModuleContext to retrieve an instance:
> > >  >
> > >  > public abstract class ModuleContext {
> > >  >      ...
> > >  >      public abstract ImportResolver getImportResolver();
> > >  > }
> > >  >
> > >  > (Note that ImportResolver is now in a position to subsume the
> > >  > functionality of both VisibilityPolicy and ImportOverridePolicy.)
> > >  >
> > >  > Repository usage is encapsulated within the implementation of the
> > >  > resolve() method. The full resolution algorithm becomes:
> > >  >
> > >  >     context.getImportResolver().resolve(definition);
> > >  >
> > >  > The launcher uses the "system" context for this. EE, Applets,etc.
make
> > >  > and use their own distinct, isolated instances.
> > >  >
> > >  >
> > >  > RESOLUTION WITH IMPORT-NAME AND IMPORT-PACKAGE
> > >  >
> > >  > With this scaffolding in place we can easily take a phased
approach to
> > >  > supporting import-package: the initial implementation simply does
not
> > >  > support it at runtime.
> > >
> > > A phased approach would be particularly beneficial if the initial
phase
> > > could be delivered as part of Java 7 and subsequent phases
implemented
> > > strictly on top of Java 7. But getting the API right up front might
be
> > > tricky unless we can spot some really good abstractions or prototype
the
> > > later phases sufficiently well. Is that the kind of phasing you had
in
> > > mind?
> >
> > Yes. It would be *extremely* useful to have an OSGi
> > implementation/prototype well under way before Java 7 is completed, so
> > that we can fine tune the model as we learn.
> >
> > >
> > >  >
> > >  > A subsequent implementation may support import-package, but only
within
> > >  > the boundaries of the same ModuleSystem.
> > >  >
> > >  > And a full blown implementation may support import-package across
> > >  > ModuleSystems.
> > >  >
> > >  > We can build in support for selecting/configuring the
ImportResolver as
> > >  > a service, just as we plan to do with ModuleSystem (and
Repository, I
> > >  > presume).
> > >  >
> > >  >
> > >  > And maybe, just maybe, we can find a way to abstract and re-use
the
> > >  > mature resolution logic from the OSGi reference implementation
*as* the
> > >  > one implementation to rule them all.
> > >  >
> > >  > // Bryan
> > >
> > > Glyn
> > >
> > >
> > >
------------------------------------------------------------------------
> > >
> > > /
> > > /
> > >
> > > /Unless stated otherwise above:
> > > IBM United Kingdom Limited - Registered in England and Wales with
number
> > > 741598.
> > > Registered office: PO Box 41, North Harbour, Portsmouth,
> Hampshire PO6 3AU/
> > >
> > >
> > >
> > >
> > >
> > >
>
> Glyn
>
>
>

> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with
> number 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU
>
>
>
>






Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU





-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/attachments/20070601/2ffb304f/attachment.html 

From glyn_normington at UK.IBM.COM  Tue Jun  5 02:27:07 2007
From: glyn_normington at UK.IBM.COM (Glyn Normington)
Date: Tue, 05 Jun 2007 10:27:07 +0100
Subject: Relationship to JSR 291 [was: Re: Bryan's comments]
In-Reply-To: <46646639.7080606@oracle.com>
Message-ID: <OFC12639E7.6C1E83B9-ON802572F1.00310AA2-802572F1.0033EC6C@uk.ibm.com>

Bryan Atsatt <bryan.atsatt at ORACLE.COM> wrote on 04/06/2007 20:21:29:

> Stanley M. Ho wrote:
> > Hi Bryan,
> >
> > I don't think I will have time to response to all your comments this
> > week, so my response would focus on the import-by-package discussion.
> >
> > First, the only import dependency we have consensus in this EG so far
is
> > import-by-module-name (or import-by-superpackage-name), and this was
> > also what's in the EDR. That said, it is by no mean that we can't
> > consider other import dependency granularities (e.g.
> > import-by-package-name, or import-by-contract) in addition to
> > import-by-module-name. I would like to focus this discussion more on
> > whether it makes sense for JSR 277 to support additional import
> > dependency granularity, mainly import-by-package-name, and if so, in
> > what context.
> >
> > There are three cases we should consider:
> >
> > 1. 277 module imports another 277 module by package name
> >
> > I think our consensus so far is that this is nice-to-have or
> > non-critical, given we already have support for import-by-module-name.
> > To keep this discussion in focus, let's not dive into
> > import-by-package-name v.s. import-by-module-name, and let's assume
> > there is no support for #1 for now.
>
> Agreed.
>
> >
> > 2. OSGi module imports 277 module by package name
> >
> > As I discussed this with Richard in previous emails, majority of the
> > works for this fall under the OSGi framework if it wants to enable
this
> > kind of wiring, but we could probably make the implementor's life
easier
> > by provide the appropriate 277 reflective API if necessary.
>
> Yes, the API would be useful.
>
> >
> > 3. 277 module imports OSGi module by package name
> >
> > If I understand you correctly, I think what you really want is the
> > support for this case, so the existing OSGi bundles can be leveraged
by
> > 277 modules in a nature way.
> >
> > Let me say it up front that I don't oppose the idea of #3 in the
context
> > of interoperability. If #3 is what you really want, I think where we
> > diverge is simply how to make it possible.
> >
> > Do you agree so far?
>
> Yes.
>
> >
> > Bryan Atsatt wrote:
> >> ...
> >> I want 277 to explicitly expose the concept of import-by-package
through
> >> appropriate APIs.
> >>
> >> It would be very nice if the 277ModuleSystem implementation itself
> >> directly supports import-by-package (case #3). But this is less
critical
> >> to me than case #4...
> >>
> >> I want a 277ModuleDefinition to be able to import-by-package an
> >> OSGiModuleDefinition (i.e. a bundle exposed via the 277 APIs). I
believe
> >> this case will be a particularly important one. By the time SE 7
ships,
> > ...
> >
> > As I described several emails back:
> >
> >  > This doesn't mean supporting the notion of import by package is
> >  > impossible. What OSGi will expose from its repository is simply
module
> >  > definitions, and module definition is basically an abstraction.
> >  > Therefore, it should be possible to treat an OSGi bundle as a
module
> >  > definition, as well as to treat each exported package in an OSGi
> >  > bundle a module definition as well. The dependency supported in JSR
> >  > 277 is import-by-module. In the JSR 277 module system, this is
mapped
> >  > to import-by-superpackage. In the OSGi module system, this can be
> >  > mapped to import-by-OSGi-bundle or import-by-package. If other
module
> >  > systems want to support different import granularity or if OSGi
> >  > evolves to add new import granularity in the future, this model
should
> >  > still work.
> >
> > Rather than surfacing the import-by-package at the API level, I think
we
> > can solve #3 by generalizing the import dependency concept as
> > import-by-name, and this name can be mapped to superpackage-name in
JSR
> > 277 and OSGi-bundle-name or OSGi-exported-package-name in OSGi (i.e.
> > exposing OSGi bundles and exported packages through implementing
> > OSGiBundleModuleDefinition and OSGiPackageModuleDefinition). I think
> > this is also what you intended to say at one point (correct me if I
> > misunderstand you) by suggesting to change
> > ImportDependency.getModuleName() into something like
> > ImportDependency.getName().
>
> I believe we are now thinking along the same lines.
>
> I just sent an email on the topic of generalizing the import dependency
> concept, with the subject "Import constraints". With the approach I
> suggest there, the distinctions between module-name and package-name are
> mostly transparent to the runtime (declarative support for package-name
> is not addressed).
>
> That isn't to say that there aren't resolution issues to be worked out
> for package-name, but its a start.
>
> >
> > We expect the module-aware compiler would somehow recognize the import
> > dependency defined in 277. In other words, if the OSGi framework
exposes
> > the module definitions appropriately, you'll get the compiler support.
> > Since the import dependency is generic, the compiler and other tools
ed
> > to do extra work to support other forms of import dependency.
>
> Yep. As I've said from the very beginning, I think the compiler is going
> to have to rely on the 277 runtime to resolve *external* (i.e.
> non-source) dependencies. So as long as the runtime can deal with
> package-name, the compiler will too.
>
> (Dependencies resolvable via included source will, I assume, just
> *directly* resolve, without applying any constraint logic. That is, if a
> class refers to a class in another superpackage, and that class &
> superpackage are available in the same compilation, the compiler will
> simply select them without applying *any* constraints. This may be too
> simple of a model, but maybe it is sufficient.)
>
> >
> > In case if the 277 module would like to import these OSGi exported
> > packages using an OSGi-like resolution algorithm, it should be
possible
> > for it to use a custom import policy (or import override policy or
maybe
> > some new override mechanisms) to get the behavior it wants.
> >
> > Do you think this is a workable approach to address #3?
>
> I don't think custom code should be required; we need to make it easier
> than that or it doesn't qualify as "first-class" citizenship, at least
> not to me :^).
>
>
> I guess I am still missing something fundamental--I keep thinking that
> import-by-package is actually not so hard. There are three main
components:
>
> 1. Declaration.
> 2. Runtime infrastructure.
> 3. Resolution algorithm.
>
> I'm pretty sure that #2 is easy (and covered in my "Import constraints"
> thread).
>
> And we don't yet have *any* import declaration mechanism (yes, we're
> assuming that this will be handled in 294, but it hasn't been brought up
> yet). I don't see this as a difficult issue either way. It would be
> natural to have both import-by-name and import-by-package in
> superpackages, if we have import at all. If we need to use annotations
> instead, it is easy to have both types.
>
> So I can only assume that the *real* concern here is #3. And I can't
> tell if this is a valid concern or not. I hope Glyn and Richard will
> point out what I'm missing here...

Sounds right to me. However, the JSR 291 support for import-package is
more subtle, and I would say was generally harder to get right (over
several versions and several years), than require-bundle which was added
fairly simply (despite the semantic rough edges, pointed out in the
spec.). Although JSR 277 has paid lip service to learning from OSGi, the
design is radically different in several ways:

* dependencies expressed in metadata and code rather than purely in
metadata
* split packages not allowed
* shallow/deep validation to avoid class space inconsistency rather than
metadata expressing package dependencies and a system provided resolver
that aims to find a "best fit"

so my feeling is that we've got a quite a lot of work to do to support
something equivalent to import-by-package. Of course, it may turn out to
be simpler than I expect...

>
> What is the difference between selecting a ModuleDefinition based on its
> name, or selecting it based on its contents? Either way, "shallow"
> validation must still occur.
>
> // Bryan
>
> >
> > - Stanley
> >

Glyn





Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU





-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/attachments/20070605/c58a9202/attachment.html 

From heavy at UNGOVERNED.ORG  Tue Jun  5 05:23:15 2007
From: heavy at UNGOVERNED.ORG (Richard S. Hall)
Date: Tue, 05 Jun 2007 08:23:15 -0400
Subject: Relationship to JSR 291 [was: Re: Bryan's comments]
In-Reply-To: <4664D591.1060608@sun.com>
References: <OF62C11B58.0C61920A-ON802572EB.0055969A-802572EB.005897E5@uk.ibm.com>
	<465DF2E4.4090603@sun.com>
	<9E52FB97-F312-4A4F-A096-ECA0430F9D7C@ungoverned.org>
	<465F4D94.9060809@sun.com>
	<0e79b722e610c3de839326fb68539389@ungoverned.org>
	<465F62FC.7070201@sun.com>
	<2071f55c84384fb5d141568f491677c9@ungoverned.org>
	<4664D591.1060608@sun.com>
Message-ID: <466555B3.9000903@ungoverned.org>

Stanley M. Ho wrote:
> Hi Richard,
>
> Richard S.Hall wrote:
>> ...
>> Perhaps so.
>>
>> I am operating off my [possibly dated] recollection that 277 modules
>> explicitly list the classes that they export, which meant that they may
>> expose arbitrary public classes from a package. If this is no longer
>> the case, then this is fine.
>>
>> The point remains, all legacy OSGi dependencies on a package assume
>> that all public classes are exported from that package. An OSGi
>> dependency on a given package cannot be resolved to a 277 module whose
>> "exported public" classes are not the same as all "public" classes in
>> the specific package.
>
> I was under the impression that the class filtering is also in effect to
> determine which public classes are exported from that package in OSGi
> and that the OSGi dependency on a package already has a similar
> "exported public" concept.

This is definitely true, but this was only introduced in R4 and is still
intended to be the exception, not the norm...from my understanding, 277
defined the norm in the opposite direction.

Ultimately, if we are making the same assumption, which is that people
will generally be exporting all public classes from a package and only
occasionally filtering some classes that should not be public, and we
are promoting this as the best practice, then we are at least in the
same ball park.

> Anyway, these are all related to how the OSGi framework might import 277
> modules by package name. I think it would be good to know if this is
> actually a sensible thing that the OSGi framework wants to support
> before we dive too deep on this. Glyn/Richard, is this something that
> you foresee the OSGi framework will support?

Well, effectively all OSGi dependency resolution comes down to packages
at some point, so if we are going to use 277 modules at all in the OSGi
framework, then we have to have some way to determine their set of
exported packages, which is necessary for determining consistency.

-> richard


From bryan.atsatt at oracle.com  Tue Jun  5 14:58:39 2007
From: bryan.atsatt at oracle.com (Bryan Atsatt)
Date: Tue, 05 Jun 2007 14:58:39 -0700
Subject: Relationship to JSR 291 [was: Re: Bryan's comments]
In-Reply-To: <OFC12639E7.6C1E83B9-ON802572F1.00310AA2-802572F1.0033EC6C@uk.ibm.com>
References: <OFC12639E7.6C1E83B9-ON802572F1.00310AA2-802572F1.0033EC6C@uk.ibm.com>
Message-ID: <4665DC8F.8000402@oracle.com>

Glyn Normington wrote:
>
> *Bryan Atsatt <bryan.atsatt at ORACLE.COM>* wrote on 04/06/2007 20:21:29:
>
>  > Stanley M. Ho wrote:
>  > > Hi Bryan,
>  > >
>  > > I don't think I will have time to response to all your comments this
>  > > week, so my response would focus on the import-by-package discussion.
>  > >
>  > > First, the only import dependency we have consensus in this EG so
> far is
>  > > import-by-module-name (or import-by-superpackage-name), and this was
>  > > also what's in the EDR. That said, it is by no mean that we can't
>  > > consider other import dependency granularities (e.g.
>  > > import-by-package-name, or import-by-contract) in addition to
>  > > import-by-module-name. I would like to focus this discussion more on
>  > > whether it makes sense for JSR 277 to support additional import
>  > > dependency granularity, mainly import-by-package-name, and if so, in
>  > > what context.
>  > >
>  > > There are three cases we should consider:
>  > >
>  > > 1. 277 module imports another 277 module by package name
>  > >
>  > > I think our consensus so far is that this is nice-to-have or
>  > > non-critical, given we already have support for import-by-module-name.
>  > > To keep this discussion in focus, let's not dive into
>  > > import-by-package-name v.s. import-by-module-name, and let's assume
>  > > there is no support for #1 for now.
>  >
>  > Agreed.
>  >
>  > >
>  > > 2. OSGi module imports 277 module by package name
>  > >
>  > > As I discussed this with Richard in previous emails, majority of the
>  > > works for this fall under the OSGi framework if it wants to enable this
>  > > kind of wiring, but we could probably make the implementor's life
> easier
>  > > by provide the appropriate 277 reflective API if necessary.
>  >
>  > Yes, the API would be useful.
>  >
>  > >
>  > > 3. 277 module imports OSGi module by package name
>  > >
>  > > If I understand you correctly, I think what you really want is the
>  > > support for this case, so the existing OSGi bundles can be leveraged by
>  > > 277 modules in a nature way.
>  > >
>  > > Let me say it up front that I don't oppose the idea of #3 in the
> context
>  > > of interoperability. If #3 is what you really want, I think where we
>  > > diverge is simply how to make it possible.
>  > >
>  > > Do you agree so far?
>  >
>  > Yes.
>  >
>  > >
>  > > Bryan Atsatt wrote:
>  > >> ...
>  > >> I want 277 to explicitly expose the concept of import-by-package
> through
>  > >> appropriate APIs.
>  > >>
>  > >> It would be very nice if the 277ModuleSystem implementation itself
>  > >> directly supports import-by-package (case #3). But this is less
> critical
>  > >> to me than case #4...
>  > >>
>  > >> I want a 277ModuleDefinition to be able to import-by-package an
>  > >> OSGiModuleDefinition (i.e. a bundle exposed via the 277 APIs). I
> believe
>  > >> this case will be a particularly important one. By the time SE 7
> ships,
>  > > ...
>  > >
>  > > As I described several emails back:
>  > >
>  > >  > This doesn't mean supporting the notion of import by package is
>  > >  > impossible. What OSGi will expose from its repository is simply
> module
>  > >  > definitions, and module definition is basically an abstraction.
>  > >  > Therefore, it should be possible to treat an OSGi bundle as a module
>  > >  > definition, as well as to treat each exported package in an OSGi
>  > >  > bundle a module definition as well. The dependency supported in JSR
>  > >  > 277 is import-by-module. In the JSR 277 module system, this is
> mapped
>  > >  > to import-by-superpackage. In the OSGi module system, this can be
>  > >  > mapped to import-by-OSGi-bundle or import-by-package. If other
> module
>  > >  > systems want to support different import granularity or if OSGi
>  > >  > evolves to add new import granularity in the future, this model
> should
>  > >  > still work.
>  > >
>  > > Rather than surfacing the import-by-package at the API level, I
> think we
>  > > can solve #3 by generalizing the import dependency concept as
>  > > import-by-name, and this name can be mapped to superpackage-name in JSR
>  > > 277 and OSGi-bundle-name or OSGi-exported-package-name in OSGi (i.e.
>  > > exposing OSGi bundles and exported packages through implementing
>  > > OSGiBundleModuleDefinition and OSGiPackageModuleDefinition). I think
>  > > this is also what you intended to say at one point (correct me if I
>  > > misunderstand you) by suggesting to change
>  > > ImportDependency.getModuleName() into something like
>  > > ImportDependency.getName().
>  >
>  > I believe we are now thinking along the same lines.
>  >
>  > I just sent an email on the topic of generalizing the import dependency
>  > concept, with the subject "Import constraints". With the approach I
>  > suggest there, the distinctions between module-name and package-name are
>  > mostly transparent to the runtime (declarative support for package-name
>  > is not addressed).
>  >
>  > That isn't to say that there aren't resolution issues to be worked out
>  > for package-name, but its a start.
>  >
>  > >
>  > > We expect the module-aware compiler would somehow recognize the import
>  > > dependency defined in 277. In other words, if the OSGi framework
> exposes
>  > > the module definitions appropriately, you'll get the compiler support.
>  > > Since the import dependency is generic, the compiler and other tools ed
>  > > to do extra work to support other forms of import dependency.
>  >
>  > Yep. As I've said from the very beginning, I think the compiler is going
>  > to have to rely on the 277 runtime to resolve *external* (i.e.
>  > non-source) dependencies. So as long as the runtime can deal with
>  > package-name, the compiler will too.
>  >
>  > (Dependencies resolvable via included source will, I assume, just
>  > *directly* resolve, without applying any constraint logic. That is, if a
>  > class refers to a class in another superpackage, and that class &
>  > superpackage are available in the same compilation, the compiler will
>  > simply select them without applying *any* constraints. This may be too
>  > simple of a model, but maybe it is sufficient.)
>  >
>  > >
>  > > In case if the 277 module would like to import these OSGi exported
>  > > packages using an OSGi-like resolution algorithm, it should be possible
>  > > for it to use a custom import policy (or import override policy or
> maybe
>  > > some new override mechanisms) to get the behavior it wants.
>  > >
>  > > Do you think this is a workable approach to address #3?
>  >
>  > I don't think custom code should be required; we need to make it easier
>  > than that or it doesn't qualify as "first-class" citizenship, at least
>  > not to me :^).
>  >
>  >
>  > I guess I am still missing something fundamental--I keep thinking that
>  > import-by-package is actually not so hard. There are three main
> components:
>  >
>  > 1. Declaration.
>  > 2. Runtime infrastructure.
>  > 3. Resolution algorithm.
>  >
>  > I'm pretty sure that #2 is easy (and covered in my "Import constraints"
>  > thread).
>  >
>  > And we don't yet have *any* import declaration mechanism (yes, we're
>  > assuming that this will be handled in 294, but it hasn't been brought up
>  > yet). I don't see this as a difficult issue either way. It would be
>  > natural to have both import-by-name and import-by-package in
>  > superpackages, if we have import at all. If we need to use annotations
>  > instead, it is easy to have both types.
>  >
>  > So I can only assume that the *real* concern here is #3. And I can't
>  > tell if this is a valid concern or not. I hope Glyn and Richard will
>  > point out what I'm missing here...
>
> Sounds right to me. However, the JSR 291 support for import-package is
> more subtle, and I would say was generally harder to get right (over
> several versions and several years), than require-bundle which was added
> fairly simply (despite the semantic rough edges, pointed out in the
> spec.). Although JSR 277 has paid lip service to learning from OSGi, the
> design is radically different in several ways:
>
> * dependencies expressed in metadata and code rather than purely in
> metadata
> * split packages not allowed
> * shallow/deep validation to avoid class space inconsistency rather than
> metadata expressing package dependencies and a system provided resolver
> that aims to find a "best fit"
>
> so my feeling is that we've got a quite a lot of work to do to support
> something equivalent to import-by-package. Of course, it may turn out to
> be simpler than I expect...

And that's what I'm struggling to determine. I do understand that 277
has taken a different approach, and so there is some cognitive
dissonance here.

But, the list you cite above seems a bit hand-wavy to me (sorry :^),
perhaps just distinctions without a difference.

My question below still stands. In general, it seems to me that
resolution consists of:

1. Selection of candidate module/bundles based on import declarations.

2. Class space consistency validation (which may narrow the list if
multiple choices, or result in a failure).

I don't understand what possible difference it could make whether we use
a module name or a package name to select candidates in step 1. Or even
if some custom code makes that selection (though I'm not a fan of this
either.)

Can anyone explain what difference it might make?

The selection criteria are entirely up to the developer/deployer; our
job is to make the best choices *within* that scope at runtime, or fail
if required if they violate the rules.

// Bryan

>  > What is the difference between selecting a ModuleDefinition based on its
>  > name, or selecting it based on its contents? Either way, "shallow"
>  > validation must still occur.
>  >
>  > // Bryan
>  >
>  > >
>  > > - Stanley
>  > >
>
> Glyn
>
>
> ------------------------------------------------------------------------
>
> /
> /
>
> /Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU/
>
>
>
>
>
>


From heavy at UNGOVERNED.ORG  Tue Jun  5 15:33:41 2007
From: heavy at UNGOVERNED.ORG (Richard S.Hall)
Date: Tue, 05 Jun 2007 18:33:41 -0400
Subject: Relationship to JSR 291 [was: Re: Bryan's comments]
In-Reply-To: <4665DC8F.8000402@oracle.com>
References: <OFC12639E7.6C1E83B9-ON802572F1.00310AA2-802572F1.0033EC6C@uk.ibm.com>
	<4665DC8F.8000402@oracle.com>
Message-ID: <81d16f0ad9ea912c29d9ba241ef0ce33@ungoverned.org>

On Jun 5, 2007, at 5:58 PM, Bryan Atsatt wrote:

> Glyn Normington wrote:
>>
>> *Bryan Atsatt <bryan.atsatt at ORACLE.COM>* wrote on 04/06/2007 20:21:29:
>>
>>  > Stanley M. Ho wrote:
>>  > > Hi Bryan,
>>  > >
>>  > > I don't think I will have time to response to all your comments
>> this
>>  > > week, so my response would focus on the import-by-package
>> discussion.
>>  > >
>>  > > First, the only import dependency we have consensus in this EG so
>> far is
>>  > > import-by-module-name (or import-by-superpackage-name), and this
>> was
>>  > > also what's in the EDR. That said, it is by no mean that we can't
>>  > > consider other import dependency granularities (e.g.
>>  > > import-by-package-name, or import-by-contract) in addition to
>>  > > import-by-module-name. I would like to focus this discussion
>> more on
>>  > > whether it makes sense for JSR 277 to support additional import
>>  > > dependency granularity, mainly import-by-package-name, and if
>> so, in
>>  > > what context.
>>  > >
>>  > > There are three cases we should consider:
>>  > >
>>  > > 1. 277 module imports another 277 module by package name
>>  > >
>>  > > I think our consensus so far is that this is nice-to-have or
>>  > > non-critical, given we already have support for
>> import-by-module-name.
>>  > > To keep this discussion in focus, let's not dive into
>>  > > import-by-package-name v.s. import-by-module-name, and let's
>> assume
>>  > > there is no support for #1 for now.
>>  >
>>  > Agreed.
>>  >
>>  > >
>>  > > 2. OSGi module imports 277 module by package name
>>  > >
>>  > > As I discussed this with Richard in previous emails, majority of
>> the
>>  > > works for this fall under the OSGi framework if it wants to
>> enable this
>>  > > kind of wiring, but we could probably make the implementor's life
>> easier
>>  > > by provide the appropriate 277 reflective API if necessary.
>>  >
>>  > Yes, the API would be useful.
>>  >
>>  > >
>>  > > 3. 277 module imports OSGi module by package name
>>  > >
>>  > > If I understand you correctly, I think what you really want is
>> the
>>  > > support for this case, so the existing OSGi bundles can be
>> leveraged by
>>  > > 277 modules in a nature way.
>>  > >
>>  > > Let me say it up front that I don't oppose the idea of #3 in the
>> context
>>  > > of interoperability. If #3 is what you really want, I think
>> where we
>>  > > diverge is simply how to make it possible.
>>  > >
>>  > > Do you agree so far?
>>  >
>>  > Yes.
>>  >
>>  > >
>>  > > Bryan Atsatt wrote:
>>  > >> ...
>>  > >> I want 277 to explicitly expose the concept of import-by-package
>> through
>>  > >> appropriate APIs.
>>  > >>
>>  > >> It would be very nice if the 277ModuleSystem implementation
>> itself
>>  > >> directly supports import-by-package (case #3). But this is less
>> critical
>>  > >> to me than case #4...
>>  > >>
>>  > >> I want a 277ModuleDefinition to be able to import-by-package an
>>  > >> OSGiModuleDefinition (i.e. a bundle exposed via the 277 APIs). I
>> believe
>>  > >> this case will be a particularly important one. By the time SE 7
>> ships,
>>  > > ...
>>  > >
>>  > > As I described several emails back:
>>  > >
>>  > >  > This doesn't mean supporting the notion of import by package
>> is
>>  > >  > impossible. What OSGi will expose from its repository is
>> simply
>> module
>>  > >  > definitions, and module definition is basically an
>> abstraction.
>>  > >  > Therefore, it should be possible to treat an OSGi bundle as a
>> module
>>  > >  > definition, as well as to treat each exported package in an
>> OSGi
>>  > >  > bundle a module definition as well. The dependency supported
>> in JSR
>>  > >  > 277 is import-by-module. In the JSR 277 module system, this is
>> mapped
>>  > >  > to import-by-superpackage. In the OSGi module system, this
>> can be
>>  > >  > mapped to import-by-OSGi-bundle or import-by-package. If other
>> module
>>  > >  > systems want to support different import granularity or if
>> OSGi
>>  > >  > evolves to add new import granularity in the future, this
>> model
>> should
>>  > >  > still work.
>>  > >
>>  > > Rather than surfacing the import-by-package at the API level, I
>> think we
>>  > > can solve #3 by generalizing the import dependency concept as
>>  > > import-by-name, and this name can be mapped to superpackage-name
>> in JSR
>>  > > 277 and OSGi-bundle-name or OSGi-exported-package-name in OSGi
>> (i.e.
>>  > > exposing OSGi bundles and exported packages through implementing
>>  > > OSGiBundleModuleDefinition and OSGiPackageModuleDefinition). I
>> think
>>  > > this is also what you intended to say at one point (correct me
>> if I
>>  > > misunderstand you) by suggesting to change
>>  > > ImportDependency.getModuleName() into something like
>>  > > ImportDependency.getName().
>>  >
>>  > I believe we are now thinking along the same lines.
>>  >
>>  > I just sent an email on the topic of generalizing the import
>> dependency
>>  > concept, with the subject "Import constraints". With the approach I
>>  > suggest there, the distinctions between module-name and
>> package-name are
>>  > mostly transparent to the runtime (declarative support for
>> package-name
>>  > is not addressed).
>>  >
>>  > That isn't to say that there aren't resolution issues to be worked
>> out
>>  > for package-name, but its a start.
>>  >
>>  > >
>>  > > We expect the module-aware compiler would somehow recognize the
>> import
>>  > > dependency defined in 277. In other words, if the OSGi framework
>> exposes
>>  > > the module definitions appropriately, you'll get the compiler
>> support.
>>  > > Since the import dependency is generic, the compiler and other
>> tools ed
>>  > > to do extra work to support other forms of import dependency.
>>  >
>>  > Yep. As I've said from the very beginning, I think the compiler is
>> going
>>  > to have to rely on the 277 runtime to resolve *external* (i.e.
>>  > non-source) dependencies. So as long as the runtime can deal with
>>  > package-name, the compiler will too.
>>  >
>>  > (Dependencies resolvable via included source will, I assume, just
>>  > *directly* resolve, without applying any constraint logic. That
>> is, if a
>>  > class refers to a class in another superpackage, and that class &
>>  > superpackage are available in the same compilation, the compiler
>> will
>>  > simply select them without applying *any* constraints. This may be
>> too
>>  > simple of a model, but maybe it is sufficient.)
>>  >
>>  > >
>>  > > In case if the 277 module would like to import these OSGi
>> exported
>>  > > packages using an OSGi-like resolution algorithm, it should be
>> possible
>>  > > for it to use a custom import policy (or import override policy
>> or
>> maybe
>>  > > some new override mechanisms) to get the behavior it wants.
>>  > >
>>  > > Do you think this is a workable approach to address #3?
>>  >
>>  > I don't think custom code should be required; we need to make it
>> easier
>>  > than that or it doesn't qualify as "first-class" citizenship, at
>> least
>>  > not to me :^).
>>  >
>>  >
>>  > I guess I am still missing something fundamental--I keep thinking
>> that
>>  > import-by-package is actually not so hard. There are three main
>> components:
>>  >
>>  > 1. Declaration.
>>  > 2. Runtime infrastructure.
>>  > 3. Resolution algorithm.
>>  >
>>  > I'm pretty sure that #2 is easy (and covered in my "Import
>> constraints"
>>  > thread).
>>  >
>>  > And we don't yet have *any* import declaration mechanism (yes,
>> we're
>>  > assuming that this will be handled in 294, but it hasn't been
>> brought up
>>  > yet). I don't see this as a difficult issue either way. It would be
>>  > natural to have both import-by-name and import-by-package in
>>  > superpackages, if we have import at all. If we need to use
>> annotations
>>  > instead, it is easy to have both types.
>>  >
>>  > So I can only assume that the *real* concern here is #3. And I
>> can't
>>  > tell if this is a valid concern or not. I hope Glyn and Richard
>> will
>>  > point out what I'm missing here...
>>
>> Sounds right to me. However, the JSR 291 support for import-package is
>> more subtle, and I would say was generally harder to get right (over
>> several versions and several years), than require-bundle which was
>> added
>> fairly simply (despite the semantic rough edges, pointed out in the
>> spec.). Although JSR 277 has paid lip service to learning from OSGi,
>> the
>> design is radically different in several ways:
>>
>> * dependencies expressed in metadata and code rather than purely in
>> metadata
>> * split packages not allowed
>> * shallow/deep validation to avoid class space inconsistency rather
>> than
>> metadata expressing package dependencies and a system provided
>> resolver
>> that aims to find a "best fit"
>>
>> so my feeling is that we've got a quite a lot of work to do to support
>> something equivalent to import-by-package. Of course, it may turn out
>> to
>> be simpler than I expect...
>
> And that's what I'm struggling to determine. I do understand that 277
> has taken a different approach, and so there is some cognitive
> dissonance here.
>
> But, the list you cite above seems a bit hand-wavy to me (sorry :^),
> perhaps just distinctions without a difference.
>
> My question below still stands. In general, it seems to me that
> resolution consists of:
>
> 1. Selection of candidate module/bundles based on import declarations.
>
> 2. Class space consistency validation (which may narrow the list if
> multiple choices, or result in a failure).
>
> I don't understand what possible difference it could make whether we
> use
> a module name or a package name to select candidates in step 1. Or even
> if some custom code makes that selection (though I'm not a fan of this
> either.)

In general, I can say your two steps above are correct...Felix actually
has what amounts to a two-pass resolve that does (1) from above in the
first step and (2) from above in the second step. However, it turns out
that (2) is actually quite difficult and difficult to do in an
efficient way.

Perhaps it is more difficult for the OSGi framework due to some of the
sophistication of its model, e.g., package substitutability, fragments,
split packages, package-level uses constraints, etc.

All I know is that we are still finding cases that need to be clarified
in the OSGi framework for this process...

I won't say that I am disagreeing with you...if we can strip away some
of the complexity of the OSGi model, then I am sure you could implement
a reasonably straightforward resolver. That won't necessarily help us
in achieving interoperability, though.

-> richard


>
> Can anyone explain what difference it might make?
>
> The selection criteria are entirely up to the developer/deployer; our
> job is to make the best choices *within* that scope at runtime, or fail
> if required if they violate the rules.
>
> // Bryan
>
>>  > What is the difference between selecting a ModuleDefinition based
>> on its
>>  > name, or selecting it based on its contents? Either way, "shallow"
>>  > validation must still occur.
>>  >
>>  > // Bryan
>>  >
>>  > >
>>  > > - Stanley
>>  > >
>>
>> Glyn
>>
>>
>> ----------------------------------------------------------------------
>> --
>>
>> /
>> /
>>
>> /Unless stated otherwise above:
>> IBM United Kingdom Limited - Registered in England and Wales with
>> number
>> 741598.
>> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire
>> PO6 3AU/
>>
>>
>>
>>
>>
>>


From Stanley.Ho at sun.com  Wed Jun  6 14:51:26 2007
From: Stanley.Ho at sun.com (Stanley M. Ho)
Date: Wed, 06 Jun 2007 14:51:26 -0700
Subject: JSR 277 EG observer mailing list
In-Reply-To: <46646B30.2030500@oracle.com>
References: <46400CFB.1020004@sun.com> <46412161.9020902@sun.com>
	<4648BA20.5070206@sun.com> <46646B30.2030500@oracle.com>
Message-ID: <46672C5E.9070607@sun.com>

Hi Bryan,

Bryan Atsatt wrote:
> How are people supposed to find this? Shouldn't there be an easily
> spotted link somewhere on the public 277 page?

Yes, it is mentioned in the JSR 277 community update page:

http://jcp.org/en/egc/view?id=277

but unfortunately it requires JCP member login.

It is also mentioned in the OpenJDK project which will cover the
reference implementation (RI) of JSR 277 and JSR 294, and hopefully
people will be able to find it:

http://openjdk.java.net/projects/modules

- Stanley


From Stanley.Ho at sun.com  Wed Jun  6 15:02:28 2007
From: Stanley.Ho at sun.com (Stanley M. Ho)
Date: Wed, 06 Jun 2007 15:02:28 -0700
Subject: Relationship to JSR 291 [was: Re: Bryan's comments]
In-Reply-To: <466555B3.9000903@ungoverned.org>
References: <OF62C11B58.0C61920A-ON802572EB.0055969A-802572EB.005897E5@uk.ibm.com>
	<465DF2E4.4090603@sun.com>
	<9E52FB97-F312-4A4F-A096-ECA0430F9D7C@ungoverned.org>
	<465F4D94.9060809@sun.com>
	<0e79b722e610c3de839326fb68539389@ungoverned.org>
	<465F62FC.7070201@sun.com>
	<2071f55c84384fb5d141568f491677c9@ungoverned.org>
	<4664D591.1060608@sun.com> <466555B3.9000903@ungoverned.org>
Message-ID: <46672EF4.7060201@sun.com>

Hi Richard,

Richard S. Hall wrote:
> This is definitely true, but this was only introduced in R4 and is still
> intended to be the exception, not the norm...from my understanding, 277
> defined the norm in the opposite direction.
>
> Ultimately, if we are making the same assumption, which is that people
> will generally be exporting all public classes from a package and only
> occasionally filtering some classes that should not be public, and we
> are promoting this as the best practice, then we are at least in the
> same ball park.

Whether people will be exporting individual exported public classes or
exporting all public classes from a package but excluding a few specific
classes is simply a syntax issue; the result is still a set of "exported
public classes." I think as long as we have the same concept of
"exported public classes," we are in the same ball park.

> Well, effectively all OSGi dependency resolution comes down to packages
> at some point, so if we are going to use 277 modules at all in the OSGi
> framework, then we have to have some way to determine their set of
> exported packages, which is necessary for determining consistency.

Sounds reasonable.

- Stanley


From Stanley.Ho at sun.com  Wed Jun  6 16:17:45 2007
From: Stanley.Ho at sun.com (Stanley M. Ho)
Date: Wed, 06 Jun 2007 16:17:45 -0700
Subject: Exported resources
In-Reply-To: <4664E9C6.30608@oracle.com>
References: <4654109D.40205@sun.com> <46548736.7020507@oracle.com>
	<465629EA.2050008@sun.com> <465658CE.5020409@oracle.com>
	<465DCE51.4010304@sun.com> <465DDCD0.8090703@oracle.com>
	<465DF6D6.5050608@sun.com> <465E0201.2030308@oracle.com>
	<465F68C8.4000103@sun.com> <46609428.4000909@oracle.com>
	<4664C2B1.1050109@sun.com> <4664E9C6.30608@oracle.com>
Message-ID: <46674099.5040908@sun.com>

Hi Bryan,

Bryan Atsatt wrote:
> ...
> frameworks/libraries are precisely what a module system is for: to allow
> their clients to *avoid* bundling the jars. And they aren't likely to be
> deployed as extensions, either; not when there's this nice repository
> model. Consider Spring. Or any of the hundreds of other third party
> frameworks/libraries out there. We *want* these to be deployed as
> normal, separate modules.

I agreed. My point was that in many cases, these frameworks/libraries
would be deployed into the repository and loaded with all permissions at
runtime, so this should not be an issue even they are deployed as
normal, separate modules.

> There is a much bigger issue, which is that this whole model falls apart
> simply because we can't grant this type of "friend" access to module
> private classes...
>
> Your description of how ResourceBundle or ServiceLoader accesses the
> class contains a big assumption, one that *could* be correct for such
> JRE code, but is almost certainly wrong for third-party code.
>
> These types of frameworks call Class.forName() or loader.loadClass().
> And then they call either Class.newInstance() or Constructor.newInstance().
>
> Regardless of whether or not they invoke any other methods, JSR 294 will
> require an access control check at the point of instantiation, extending
> those it performs today. Look at how Class.newInstance() and
> Constructor.newInstance() both use:
>
>     Reflection.ensureMemberAccess()
>
> This is the logical point at which to inject the new module private
> access control check.
>
> Could this be made to succeed for JRE callers? Of course (but it
> wouldn't be right, IMO).
>
> But, more importantly: it won't, and cannot, succeed for third-party
> callers without some sort of friend model for classes. Which isn't on
> the table.
>
> So, again, if we're not going to do this for classes, why should we do
> it for resources?

Sorry, I still don't think I fully understand your point. From my
perspective, the access model for exported classes and exported
resources are orthogonal. I agreed that it would be nice to have some
symmetries between them, but I think we already concluded why this is
not feasible because of their fundamental differences. The fact that
this is not how we support classes today doesn't mean we could not do it
for resources.

Perhaps it would help to pull out the following question you had couple
emails back for our discussion:

Bryan Atsatt wrote:
 > And what about the mismatch issue I brought up? Lots of existing
 > frameworks use the "services" pattern, in which:
 >
 > 1. A well known resource name is used in a getResource() call.
 > 2. The resource contains a string naming a provider class.
 > 3. The provider class is loaded using Class.forName().
 >
 > This is a simple but very powerful pattern, but it requires that *both*
 > the resource and the named class be accessible to the framework. If we
 > require a permission grant for the resource, but not for the class, it
 > will be very easy to get access to one and not the other. Is it really
 > worth opening up this potential headache to enable a friend model for
 > resources?

In today's model, if the resource is available, #1 will always succeed.
If the provider class is available, then #3 will also succeed, and
whether the class is accessible will be determined by the JVM. The
entire "exported resources" issue is basically around the behavioral
change in #1; this is orthogonal to #2 and #3, and I think we should
ignore the entire "exported classes" issue for the moment to keep this
discussion focus.

That said, I realize that we might have been thinking about two very
different use cases (correct me if I misunderstand you) here:

a. Some service-related frameworks (e.g. ServiceLoader) need to retrieve
resources like META-INF/services/<service-provider> from the module
classloader.
b. A module uses the ResourceBundle APIs to retrieve property-based
resources or list-based resources from its own module.

For (a) alone, I would agree with you that this kind of resource might
simply be exported. In fact, I don't think this should even be
considered a private resource because its content is intended for
external consumption.

For (b), I don't think it makes sense to require a module exporting the
private resources for its own consumption, and this is where I hope the
security permission approach would help to make it work.

Suppose we say that resources like (a) should be exported while
resources like (b) should remain private (and we'll use the security
permission approach to allow module to access its own private
resources), do you think you can live with this?

- Stanley


From bryan.atsatt at oracle.com  Wed Jun  6 16:48:07 2007
From: bryan.atsatt at oracle.com (Bryan Atsatt)
Date: Wed, 06 Jun 2007 16:48:07 -0700
Subject: Exported resources
In-Reply-To: <46674099.5040908@sun.com>
References: <4654109D.40205@sun.com> <46548736.7020507@oracle.com>
	<465629EA.2050008@sun.com> <465658CE.5020409@oracle.com>
	<465DCE51.4010304@sun.com> <465DDCD0.8090703@oracle.com>
	<465DF6D6.5050608@sun.com> <465E0201.2030308@oracle.com>
	<465F68C8.4000103@sun.com> <46609428.4000909@oracle.com>
	<4664C2B1.1050109@sun.com> <4664E9C6.30608@oracle.com>
	<46674099.5040908@sun.com>
Message-ID: <466747B7.2030709@oracle.com>

Stanley M. Ho wrote:
> Hi Bryan,
>
> Bryan Atsatt wrote:
>> ...
>> frameworks/libraries are precisely what a module system is for: to allow
>> their clients to *avoid* bundling the jars. And they aren't likely to be
>> deployed as extensions, either; not when there's this nice repository
>> model. Consider Spring. Or any of the hundreds of other third party
>> frameworks/libraries out there. We *want* these to be deployed as
>> normal, separate modules.
>
> I agreed. My point was that in many cases, these frameworks/libraries
> would be deployed into the repository and loaded with all permissions at
> runtime,

That last phrase seems like a huge leap to me. How exactly would this
grant all permissions? And, even if it did, it seems like an *extremely*
large security hole!

(But, again, I don't feel the need to dwell on the grant issue at the
moment--we have bigger fish to fry.)

> so this should not be an issue even they are deployed as
> normal, separate modules.
>
>> There is a much bigger issue, which is that this whole model falls apart
>> simply because we can't grant this type of "friend" access to module
>> private classes...
>>
>> Your description of how ResourceBundle or ServiceLoader accesses the
>> class contains a big assumption, one that *could* be correct for such
>> JRE code, but is almost certainly wrong for third-party code.
>>
>> These types of frameworks call Class.forName() or loader.loadClass().
>> And then they call either Class.newInstance() or
>> Constructor.newInstance().
>>
>> Regardless of whether or not they invoke any other methods, JSR 294 will
>> require an access control check at the point of instantiation, extending
>> those it performs today. Look at how Class.newInstance() and
>> Constructor.newInstance() both use:
>>
>>     Reflection.ensureMemberAccess()
>>
>> This is the logical point at which to inject the new module private
>> access control check.
>>
>> Could this be made to succeed for JRE callers? Of course (but it
>> wouldn't be right, IMO).
>>
>> But, more importantly: it won't, and cannot, succeed for third-party
>> callers without some sort of friend model for classes. Which isn't on
>> the table.
>>
>> So, again, if we're not going to do this for classes, why should we do
>> it for resources?
>
> Sorry, I still don't think I fully understand your point. From my
> perspective, the access model for exported classes and exported
> resources are orthogonal. I agreed that it would be nice to have some
> symmetries between them, but I think we already concluded why this is
> not feasible because of their fundamental differences. The fact that
> this is not how we support classes today doesn't mean we could not do it
> for resources.
>
> Perhaps it would help to pull out the following question you had couple
> emails back for our discussion:
>
> Bryan Atsatt wrote:
>  > And what about the mismatch issue I brought up? Lots of existing
>  > frameworks use the "services" pattern, in which:
>  >
>  > 1. A well known resource name is used in a getResource() call.
>  > 2. The resource contains a string naming a provider class.
>  > 3. The provider class is loaded using Class.forName().
>  >
>  > This is a simple but very powerful pattern, but it requires that *both*
>  > the resource and the named class be accessible to the framework. If we
>  > require a permission grant for the resource, but not for the class, it
>  > will be very easy to get access to one and not the other. Is it really
>  > worth opening up this potential headache to enable a friend model for
>  > resources?
>
> In today's model, if the resource is available, #1 will always succeed.
> If the provider class is available, then #3 will also succeed, and
> whether the class is accessible will be determined by the JVM. The
> entire "exported resources" issue is basically around the behavioral
> change in #1; this is orthogonal to #2 and #3, and I think we should
> ignore the entire "exported classes" issue for the moment to keep this
> discussion focus.
>
> That said, I realize that we might have been thinking about two very
> different use cases (correct me if I misunderstand you) here:
>
> a. Some service-related frameworks (e.g. ServiceLoader) need to retrieve
> resources like META-INF/services/<service-provider> from the module
> classloader.
> b. A module uses the ResourceBundle APIs to retrieve property-based
> resources or list-based resources from its own module.
>
> For (a) alone, I would agree with you that this kind of resource might
> simply be exported. In fact, I don't think this should even be
> considered a private resource because its content is intended for
> external consumption.
>
> For (b), I don't think it makes sense to require a module exporting the
> private resources for its own consumption, and this is where I hope the
> security permission approach would help to make it work.
>
> Suppose we say that resources like (a) should be exported while
> resources like (b) should remain private (and we'll use the security
> permission approach to allow module to access its own private
> resources), do you think you can live with this?

It isn't just me that can't live with it :^). You are correct that
private *property-based* resources could be retrieved this way, but
incorrect about *list-based* resources.

"List-based" resources are... *Class* instances. Specifically, they are
subclasses of ListResourceBundle. They must be loaded and instantiated
by ResourceBundle.

If ListResourceBundle subclasses are module private,
ResourceBundle.getBundle() will fail. And we have no mechanism to grant
access.

// Bryan

>
> - Stanley
>


From Stanley.Ho at sun.com  Wed Jun  6 18:36:52 2007
From: Stanley.Ho at sun.com (Stanley M. Ho)
Date: Wed, 06 Jun 2007 18:36:52 -0700
Subject: Exported resources
In-Reply-To: <466747B7.2030709@oracle.com>
References: <4654109D.40205@sun.com> <46548736.7020507@oracle.com>
	<465629EA.2050008@sun.com> <465658CE.5020409@oracle.com>
	<465DCE51.4010304@sun.com> <465DDCD0.8090703@oracle.com>
	<465DF6D6.5050608@sun.com> <465E0201.2030308@oracle.com>
	<465F68C8.4000103@sun.com> <46609428.4000909@oracle.com>
	<4664C2B1.1050109@sun.com> <4664E9C6.30608@oracle.com>
	<46674099.5040908@sun.com> <466747B7.2030709@oracle.com>
Message-ID: <46676134.8020206@sun.com>

Hi Bryan,

Bryan Atsatt wrote:
>> Suppose we say that resources like (a) should be exported while
>> resources like (b) should remain private (and we'll use the security
>> permission approach to allow module to access its own private
>> resources), do you think you can live with this?
>
> It isn't just me that can't live with it :^). You are correct that
> private *property-based* resources could be retrieved this way, but
> incorrect about *list-based* resources.
>
> "List-based" resources are... *Class* instances. Specifically, they are
> subclasses of ListResourceBundle. They must be loaded and instantiated
> by ResourceBundle.

Oops... I missed that. ;-)

> If ListResourceBundle subclasses are module private,
> ResourceBundle.getBundle() will fail. And we have no mechanism to grant
> access.

If some class needs access to non-public/exported details of another
class, you can always use privileged reflection. That is the solution
today and I expect it will be available with superpackages too. In other
words, it is possible for ResourceBundle.getBundle() to succeed. Of
course, it is still debatable whether changing ResourceBundle to use
privileged reflection to access ListResourceBundle in a module is the
right thing to do.

I think I begin to see your point. To elaborate on what you said, if
ListResourceBundle must be made exported, then it probably makes sense
to require property-based resources to be exported from the module as
well. If we go down this path, then all resources that are accessible by
the module itself or other modules should be made exported, and it won't
require the security permission approach to support it. Actually,
calling it "exported resource" would be misleading; what it really means
is "public resource" or "accessible resource" which can be accessed by
anyone, but nobody could access the "private resource" including the
module itself.

- Stanley


From bryan.atsatt at oracle.com  Wed Jun  6 18:59:56 2007
From: bryan.atsatt at oracle.com (Bryan Atsatt)
Date: Wed, 06 Jun 2007 18:59:56 -0700
Subject: Exported resources
In-Reply-To: <46676134.8020206@sun.com>
References: <4654109D.40205@sun.com> <46548736.7020507@oracle.com>
	<465629EA.2050008@sun.com> <465658CE.5020409@oracle.com>
	<465DCE51.4010304@sun.com> <465DDCD0.8090703@oracle.com>
	<465DF6D6.5050608@sun.com> <465E0201.2030308@oracle.com>
	<465F68C8.4000103@sun.com> <46609428.4000909@oracle.com>
	<4664C2B1.1050109@sun.com> <4664E9C6.30608@oracle.com>
	<46674099.5040908@sun.com> <466747B7.2030709@oracle.com>
	<46676134.8020206@sun.com>
Message-ID: <4667669C.7060708@oracle.com>

Yes, I think we're getting close now :^)

(By "privileged reflection", you mean having permission to call
AccessibleObject.setAccessible(true), right? If so, then we're back to
the grant issue!)

I still think we can provide "private", (i.e. non-exported) resources
which *are* available to the module classes, but to no class outside of it.

Without using permissions.

Remember this?

   private static class StackAccessor extends SecurityManager {
       public Class[] getStack() {
           return getClassContext();
       }
   }

   private static final STACK_ACCESSOR = new StackAccessor();

If a resource is found, and is not-exported, a module class loader can
use this (or similar) to get the immediate caller's class (at a fixed
index).

If the caller class' Module does not match that for the loader, we don't
return the resource.

Which is what I did in the prototype.

// Bryan

Stanley M. Ho wrote:
> Hi Bryan,
>
> Bryan Atsatt wrote:
>>> Suppose we say that resources like (a) should be exported while
>>> resources like (b) should remain private (and we'll use the security
>>> permission approach to allow module to access its own private
>>> resources), do you think you can live with this?
>>
>> It isn't just me that can't live with it :^). You are correct that
>> private *property-based* resources could be retrieved this way, but
>> incorrect about *list-based* resources.
>>
>> "List-based" resources are... *Class* instances. Specifically, they are
>> subclasses of ListResourceBundle. They must be loaded and instantiated
>> by ResourceBundle.
>
> Oops... I missed that. ;-)
>
>> If ListResourceBundle subclasses are module private,
>> ResourceBundle.getBundle() will fail. And we have no mechanism to grant
>> access.
>
> If some class needs access to non-public/exported details of another
> class, you can always use privileged reflection. That is the solution
> today and I expect it will be available with superpackages too. In other
> words, it is possible for ResourceBundle.getBundle() to succeed. Of
> course, it is still debatable whether changing ResourceBundle to use
> privileged reflection to access ListResourceBundle in a module is the
> right thing to do.
>
> I think I begin to see your point. To elaborate on what you said, if
> ListResourceBundle must be made exported, then it probably makes sense
> to require property-based resources to be exported from the module as
> well. If we go down this path, then all resources that are accessible by
> the module itself or other modules should be made exported, and it won't
> require the security permission approach to support it. Actually,
> calling it "exported resource" would be misleading; what it really means
> is "public resource" or "accessible resource" which can be accessed by
> anyone, but nobody could access the "private resource" including the
> module itself.
>
> - Stanley
>


From bryan.atsatt at oracle.com  Thu Jun  7 16:53:16 2007
From: bryan.atsatt at oracle.com (Bryan Atsatt)
Date: Thu, 07 Jun 2007 16:53:16 -0700
Subject: Relationship to JSR 291 [was: Re: Bryan's comments]
In-Reply-To: <81d16f0ad9ea912c29d9ba241ef0ce33@ungoverned.org>
References: <OFC12639E7.6C1E83B9-ON802572F1.00310AA2-802572F1.0033EC6C@uk.ibm.com>
	<4665DC8F.8000402@oracle.com>
	<81d16f0ad9ea912c29d9ba241ef0ce33@ungoverned.org>
Message-ID: <46689A6C.5070207@oracle.com>

[snip]
Richard S.Hall wrote:

>> My question below still stands. In general, it seems to me that
>> resolution consists of:
>>
>> 1. Selection of candidate module/bundles based on import declarations.
>>
>> 2. Class space consistency validation (which may narrow the list if
>> multiple choices, or result in a failure).
>>
>> I don't understand what possible difference it could make whether we
>> use
>> a module name or a package name to select candidates in step 1. Or even
>> if some custom code makes that selection (though I'm not a fan of this
>> either.)
>
> In general, I can say your two steps above are correct...Felix actually
> has what amounts to a two-pass resolve that does (1) from above in the
> first step and (2) from above in the second step. However, it turns out
> that (2) is actually quite difficult and difficult to do in an
> efficient way.
>
> Perhaps it is more difficult for the OSGi framework due to some of the
> sophistication of its model, e.g., package substitutability, fragments,
> split packages, package-level uses constraints, etc.
>
> All I know is that we are still finding cases that need to be clarified
> in the OSGi framework for this process...
>
> I won't say that I am disagreeing with you...if we can strip away some
> of the complexity of the OSGi model, then I am sure you could implement
> a reasonably straightforward resolver.

Right, that is what I'm trying to get at here. I believe *very* strongly
that import-by-name is a bad idea. Period. It is just plain wrong to do
the equivalent of "import foo.jar". We can do this already, with
manifest Class-Path. Simply layering version support on it does *not*
eliminate jar-hell!

It is far too brittle, raising all kinds of maintenance issues. Under
import-by-name, refactoring *requires* introducing a "facade" module to
avoid breaking importers. And this facade module now becomes *another*
product artifact to be managed: documented, maintained, distributed,
deprecated, and ultimately perhaps removed.

I truly believe that the Java world will be far better served by having
277 support ONLY import-by-package!

But there has been this background fear that import-by-package is
somehow terribly complex. I can certainly believe that the union of all
the OSGi features *does* result in a lot of complexity.

I just think that is almost entirely orthogonal to *this* domain, since
we are not trying to expose all those other features.

> That won't necessarily help us in achieving interoperability, though.

Absolutely; regardless of the outcome of the import-by-package issue, we
still need to deal with exactly how the loaders can share classes and
resources...

// Bryan


From heavy at UNGOVERNED.ORG  Thu Jun  7 17:08:36 2007
From: heavy at UNGOVERNED.ORG (Richard S.Hall)
Date: Thu, 07 Jun 2007 20:08:36 -0400
Subject: Relationship to JSR 291 [was: Re: Bryan's comments]
In-Reply-To: <46689A6C.5070207@oracle.com>
References: <OFC12639E7.6C1E83B9-ON802572F1.00310AA2-802572F1.0033EC6C@uk.ibm.com>
	<4665DC8F.8000402@oracle.com>
	<81d16f0ad9ea912c29d9ba241ef0ce33@ungoverned.org>
	<46689A6C.5070207@oracle.com>
Message-ID: <4779f81c025b414f3da25e1b6789eadd@ungoverned.org>

On Jun 7, 2007, at 7:53 PM, Bryan Atsatt wrote:

> [snip]
> Richard S.Hall wrote:
>
>>> My question below still stands. In general, it seems to me that
>>> resolution consists of:
>>>
>>> 1. Selection of candidate module/bundles based on import
>>> declarations.
>>>
>>> 2. Class space consistency validation (which may narrow the list if
>>> multiple choices, or result in a failure).
>>>
>>> I don't understand what possible difference it could make whether we
>>> use
>>> a module name or a package name to select candidates in step 1. Or
>>> even
>>> if some custom code makes that selection (though I'm not a fan of
>>> this
>>> either.)
>>
>> In general, I can say your two steps above are correct...Felix
>> actually
>> has what amounts to a two-pass resolve that does (1) from above in the
>> first step and (2) from above in the second step. However, it turns
>> out
>> that (2) is actually quite difficult and difficult to do in an
>> efficient way.
>>
>> Perhaps it is more difficult for the OSGi framework due to some of the
>> sophistication of its model, e.g., package substitutability,
>> fragments,
>> split packages, package-level uses constraints, etc.
>>
>> All I know is that we are still finding cases that need to be
>> clarified
>> in the OSGi framework for this process...
>>
>> I won't say that I am disagreeing with you...if we can strip away some
>> of the complexity of the OSGi model, then I am sure you could
>> implement
>> a reasonably straightforward resolver.
>
> Right, that is what I'm trying to get at here. I believe *very*
> strongly
> that import-by-name is a bad idea. Period. It is just plain wrong to do
> the equivalent of "import foo.jar". We can do this already, with
> manifest Class-Path. Simply layering version support on it does *not*
> eliminate jar-hell!
>
> It is far too brittle, raising all kinds of maintenance issues. Under
> import-by-name, refactoring *requires* introducing a "facade" module to
> avoid breaking importers. And this facade module now becomes *another*
> product artifact to be managed: documented, maintained, distributed,
> deprecated, and ultimately perhaps removed.
>
> I truly believe that the Java world will be far better served by having
> 277 support ONLY import-by-package!
>
> But there has been this background fear that import-by-package is
> somehow terribly complex. I can certainly believe that the union of all
> the OSGi features *does* result in a lot of complexity.
>
> I just think that is almost entirely orthogonal to *this* domain, since
> we are not trying to expose all those other features.

Well, you are preaching to the choir with me. I argued long and hard to
keep require-bundle out of OSGi R4 for the same reasons you list above
(I lost that battle). I also encourage everyone who wants to use
require-bundle to read the section in the OSGi spec that lists many of
its limitations and issues (which were extended in OSGi R4.1).

I would also be happy if 277 only supported import by package, but I
stopped arguing for it since it was kiboshed early in the EG
discussions.

>> That won't necessarily help us in achieving interoperability, though.
>
> Absolutely; regardless of the outcome of the import-by-package issue,
> we
> still need to deal with exactly how the loaders can share classes and
> resources...

So, I guess we are in agreement, even if the rest of the EG isn't. :-)

-> richard


From Stanley.Ho at sun.com  Thu Jun  7 17:46:23 2007
From: Stanley.Ho at sun.com (Stanley M. Ho)
Date: Thu, 07 Jun 2007 17:46:23 -0700
Subject: Exported resources
In-Reply-To: <4667669C.7060708@oracle.com>
References: <4654109D.40205@sun.com> <46548736.7020507@oracle.com>
	<465629EA.2050008@sun.com> <465658CE.5020409@oracle.com>
	<465DCE51.4010304@sun.com> <465DDCD0.8090703@oracle.com>
	<465DF6D6.5050608@sun.com> <465E0201.2030308@oracle.com>
	<465F68C8.4000103@sun.com> <46609428.4000909@oracle.com>
	<4664C2B1.1050109@sun.com> <4664E9C6.30608@oracle.com>
	<46674099.5040908@sun.com> <466747B7.2030709@oracle.com>
	<46676134.8020206@sun.com> <4667669C.7060708@oracle.com>
Message-ID: <4668A6DF.1040906@sun.com>

Hi Bryan,

Bryan Atsatt wrote:
> Yes, I think we're getting close now :^)
>
> (By "privileged reflection", you mean having permission to call
> AccessibleObject.setAccessible(true), right? If so, then we're back to
> the grant issue!)

Yes. In this case, it's the ResourceBundle API calling it to retrieve
ListResourceBundle, so it'll have the sufficient permissions to do that.
Anyway, I think we should leave this issue aside for now.

> I still think we can provide "private", (i.e. non-exported) resources
> which *are* available to the module classes, but to no class outside of it.
>
> Without using permissions.
>
> Remember this?
>
>   private static class StackAccessor extends SecurityManager {
>       public Class[] getStack() {
>           return getClassContext();
>       }
>   }
>
>   private static final STACK_ACCESSOR = new StackAccessor();
>
> If a resource is found, and is not-exported, a module class loader can
> use this (or similar) to get the immediate caller's class (at a fixed
> index).
>
> If the caller class' Module does not match that for the loader, we don't
> return the resource.

I think we should step back to rethink the use cases a bit.

Typically, a module calls some APIs (e.g. ResourceBundle, AWTToolkit,
etc.) and these APIs might then retrieve resources through the module's
ClassLoader.getResource*(). If we are saying that a module is required
to make these resources exported in order for these use cases to work,
then what are the actual use cases we want to support by hiding
resources from getResource*()?

I think there remains a common use case where hiding resources would be
a good thing if developers want to hide all the resources that are not
used by anyone through ClassLoader.getResource*() (e.g. all the .class
files). On the other hand, I think the use case where the code in the
module calls its own ClassLoader.getResource*() to retrieve private
resources directly is not very common; If this is a rare use case, I
don't think it makes sense to introduce this notion of "private
resources". If we are forcing developers to export resources for the
common use cases (like ResourceBundle and ServiceLoader) anyway, then
why not require them to export resources in other cases as well?

It would help if there are some real use cases to support why it is
important to have this notion of "private resources".

- Stanley


From Stanley.Ho at Sun.COM  Thu Jun  7 18:33:44 2007
From: Stanley.Ho at Sun.COM (Stanley M. Ho)
Date: Thu, 07 Jun 2007 18:33:44 -0700
Subject: Import constraints
In-Reply-To: <46645C1A.2090809@oracle.com>
References: <46645C1A.2090809@oracle.com>
Message-ID: <4668B1F8.1040503@sun.com>

Hi Bryan,

Bryan Atsatt wrote:
> ...
> By having Dependency extend Query, instances can be directly passed to
> Repository.find(); no conversion required. We could obviously just make
> a getter for the Query if we want, but why not make this convenient to use?

An import dependency describes the relationship between two modules. It
is true that we may want to query a module described in the import
dependency, but an import dependency itself is not a query. In fact,
someone may even want to query a module which contains the specific
import dependency, although I think this use case is very rare. Anyway,
it sounded like the main benefit to have import dependency extend query
is to make it easier to pass into Repository.find(). Unfortunately, once
a class is extended from another, all the public/protected
methods/fields from the parent class would automatically become part of
the signature of the child class. I don't think we want to pollute the
class hierarchy and potentially confuse developers just for a bit of
convenient. ;-)

> Second, I think we should eliminate VersionConstraint as a separate
> class. It's functionality is easy to replace with Query elements that
> match Version or VersionRange, strung together with appropriate boolean
> operators.

There are methods in VersionConstraint for determining if it contains a
specified version/version range/version constraint. These methods are
used heavily for version constraint override in module
initialization/import policy/import override policy, and they are not
suitable to be moved into the Query class. I think VersionConstraint
should stay as a separate class.

> Third, we should add a getExportedPackages() method to ModuleDefinition.
> This way a Query can easily be created to search by package name.
> Whether or not we choose to surface import-by-package declaratively, it
> should at least be possible for a ModuleSystem to use such a Query.

Having something like getExportedPackages() sounds reasonable from the
perspective of interoperability.

> Fourth, we should add support for simple attribute constraint
> declarations. Enabling simple equality constraints ought to be easy
> enough: an AttributeConstraint annotation that takes key/value lists
> (key1==value1;key2==value2;...). We already have support for expressing
> these as Query instances.

The Query class currently only supports querying arbitrary attributes at
the module level. Are you suggesting that we should let developers to
declare arbitrary attributes at the import? Could you come up with real
use cases for this in the context of 277 alone?

> Finally, Query ought to have a toString() implementation so that we can
> log them, examine in a debugger, put them in error messages, etc.

The Query.toString() will output a human readable string, and this is
already implemented in the RI.

- Stanley


From andyp at bea.com  Fri Jun  8 01:53:51 2007
From: andyp at bea.com (Andy Piper)
Date: Fri, 08 Jun 2007 09:53:51 +0100
Subject: Relationship to JSR 291 [was: Re: Bryan's comments]
In-Reply-To: <46689A6C.5070207@oracle.com>
References: <OFC12639E7.6C1E83B9-ON802572F1.00310AA2-802572F1.0033EC6C@uk.ibm.com>
	<4665DC8F.8000402@oracle.com>
	<81d16f0ad9ea912c29d9ba241ef0ce33@ungoverned.org>
	<46689A6C.5070207@oracle.com>
Message-ID: <6.2.5.6.2.20070608093830.02c499a8@bea.com>

At 00:53 08/06/2007, Bryan Atsatt wrote:
>I truly believe that the Java world will be far better served by having
>277 support ONLY import-by-package!
>
>But there has been this background fear that import-by-package is
>somehow terribly complex. I can certainly believe that the union of all
>the OSGi features *does* result in a lot of complexity.

Just to express a usage datapoint here. Having used OSGi for a while
now I am finding import-by-package too hard to use in certain
circumstances (but in general it is a nice flexible model), but its
not an issue that import-by-name really fixes in my opinion.

The basic issue is that to use something I am interested in the union
of packages that make up the "feature". I actually don't care about
the individual packages. Take my current bugbear, cglib in Spring for
example, to use this "feature" the *proxied* class bundle needs to
import the following packages:

  net.sf.cglib.proxy,
  net.sf.cglib.core,
  net.sf.cglib.reflect,
  org.aopalliance.aop,
  org.springframework.aop.framework,
  org.springframework.aop,

not an obvious set, nor a set of packages that I care much about.
Worse, the proxied class does not know ahead of time that it needs to
import these packages (this is I believe because cglib tries to
create the proxy in the classloader of the proxied class by default).

Ideally to solve this I would like two things:

1. The ability to express a feature in terms of its constituent
packages. Note that import-by-name doesn't really help here since the
actual packages are distributed across multiple modules. What I
really want is some kind of feature module that exports these
packages to you when you ask for the feature.

2. Some way of augmenting a module *after* the fact with a particular
set of packages, so that the module doesn't need to do dynamic-import
* everywhere.

I don't think either if these is possible in OSGi right now (I would
be happy to be corrected!). I don't think JSR 277 solves these either
(but again I would be happy to be corrected).

Just a datapoint from real-world usage.

andy


Notice:  This email message, together with any attachments, may contain information  of  BEA Systems,  Inc.,  its subsidiaries  and  affiliated entities,  that may be confidential,  proprietary,  copyrighted  and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by email and then delete it.


From heavy at UNGOVERNED.ORG  Fri Jun  8 05:31:23 2007
From: heavy at UNGOVERNED.ORG (Richard S. Hall)
Date: Fri, 08 Jun 2007 08:31:23 -0400
Subject: Relationship to JSR 291 [was: Re: Bryan's comments]
In-Reply-To: <6.2.5.6.2.20070608093830.02c499a8@bea.com>
References: <OFC12639E7.6C1E83B9-ON802572F1.00310AA2-802572F1.0033EC6C@uk.ibm.com>
	<4665DC8F.8000402@oracle.com>
	<81d16f0ad9ea912c29d9ba241ef0ce33@ungoverned.org>
	<46689A6C.5070207@oracle.com>
	<6.2.5.6.2.20070608093830.02c499a8@bea.com>
Message-ID: <46694C1B.3000107@ungoverned.org>

Andy Piper wrote:
> At 00:53 08/06/2007, Bryan Atsatt wrote:
>
>> I truly believe that the Java world will be far better served by having
>> 277 support ONLY import-by-package!
>>
>> But there has been this background fear that import-by-package is
>> somehow terribly complex. I can certainly believe that the union of all
>> the OSGi features *does* result in a lot of complexity.
>>
>
> Just to express a usage datapoint here. Having used OSGi for a while
> now I am finding import-by-package too hard to use in certain
> circumstances (but in general it is a nice flexible model), but its
> not an issue that import-by-name really fixes in my opinion.
>
> The basic issue is that to use something I am interested in the union
> of packages that make up the "feature". I actually don't care about
> the individual packages. Take my current bugbear, cglib in Spring for
> example, to use this "feature" the *proxied* class bundle needs to
> import the following packages:
>
>   net.sf.cglib.proxy,
>   net.sf.cglib.core,
>   net.sf.cglib.reflect,
>   org.aopalliance.aop,
>   org.springframework.aop.framework,
>   org.springframework.aop,
>
> not an obvious set, nor a set of packages that I care much about.
> Worse, the proxied class does not know ahead of time that it needs to
> import these packages (this is I believe because cglib tries to
> create the proxy in the classloader of the proxied class by default).
>
> Ideally to solve this I would like two things:
>
> 1. The ability to express a feature in terms of its constituent
> packages. Note that import-by-name doesn't really help here since the
> actual packages are distributed across multiple modules. What I
> really want is some kind of feature module that exports these
> packages to you when you ask for the feature.
>
> 2. Some way of augmenting a module *after* the fact with a particular
> set of packages, so that the module doesn't need to do dynamic-import
> * everywhere.
>
> I don't think either if these is possible in OSGi right now (I would
> be happy to be corrected!). I don't think JSR 277 solves these either
> (but again I would be happy to be corrected).
>

Yes, you are correct that both of these are not currently
possible...well, I guess (2) is technically possible since you could
grab the original bundle JAR file and rewrite its manifest then install
it, but I know this is not what you want.

Both of these have been discussed at one point in time or the other.
Peter Kriens was recently talking to me about something similar to (1),
so we might be doing some experiments in this area. I think something
like (1) could fit in the OSGi model fairly well, but more investigation
is needed to be sure.

-> richard

> Just a datapoint from real-world usage.
>
> andy
>
>
> Notice:  This email message, together with any attachments, may contain information  of  BEA Systems,  Inc.,  its subsidiaries  and  affiliated entities,  that may be confidential,  proprietary,  copyrighted  and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by email and then delete it.
>


From glyn_normington at UK.IBM.COM  Fri Jun  8 08:58:21 2007
From: glyn_normington at UK.IBM.COM (Glyn Normington)
Date: Fri, 08 Jun 2007 16:58:21 +0100
Subject: JSR 277 EG observer mailing list
In-Reply-To: <46672C5E.9070607@sun.com>
Message-ID: <OFB42F1F41.14FA9781-ON802572F4.0056F42A-802572F4.0057BD3E@uk.ibm.com>

Hi Stanley

Any chance of updating the public part of the JSR 277 page?

Don't forget your blog links to it the observer list, as does mine now.
;-)

Glyn

"Stanley M. Ho" <Stanley.Ho at SUN.COM> wrote on 06/06/2007 10:51:26 PM:

> Hi Bryan,
>
> Bryan Atsatt wrote:
> > How are people supposed to find this? Shouldn't there be an easily
> > spotted link somewhere on the public 277 page?
>
> Yes, it is mentioned in the JSR 277 community update page:
>
> http://jcp.org/en/egc/view?id=277
>
> but unfortunately it requires JCP member login.
>
> It is also mentioned in the OpenJDK project which will cover the
> reference implementation (RI) of JSR 277 and JSR 294, and hopefully
> people will be able to find it:
>
> http://openjdk.java.net/projects/modules
>
> - Stanley






Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU





-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/attachments/20070608/08982b0d/attachment.html 

From sam at SAMPULLARA.COM  Fri Jun  8 11:12:39 2007
From: sam at SAMPULLARA.COM (Sam Pullara)
Date: Fri, 08 Jun 2007 11:12:39 -0700
Subject: Relationship to JSR 291 [was: Re: Bryan's comments]
In-Reply-To: <6.2.5.6.2.20070608093830.02c499a8@bea.com>
References: <OFC12639E7.6C1E83B9-ON802572F1.00310AA2-802572F1.0033EC6C@uk.ibm.com>
	<4665DC8F.8000402@oracle.com>
	<81d16f0ad9ea912c29d9ba241ef0ce33@ungoverned.org>
	<46689A6C.5070207@oracle.com>
	<6.2.5.6.2.20070608093830.02c499a8@bea.com>
Message-ID: <BEF4551C-0B5A-4618-B282-28DFF215F198@sampullara.com>

My aghast reaction to Andy's mail is that if we aren't solving this
simple issue, what is it that we are solving?  This use case is far
more like what I would want in the real world.  If we don't get
something out of this JSR and the language JSR that approximates a
runtime version of 'gem' (from the ruby camp) we might as well open
up the next JSR for really solving the problems that we have.  People
want to import functionality, not packages, e.g. import ejb3, not
org.hibernate.*, javax.ejb.*, etc.

Sam

On Jun 8, 2007, at 1:53 AM, Andy Piper wrote:

> At 00:53 08/06/2007, Bryan Atsatt wrote:
>> I truly believe that the Java world will be far better served by
>> having
>> 277 support ONLY import-by-package!
>>
>> But there has been this background fear that import-by-package is
>> somehow terribly complex. I can certainly believe that the union
>> of all
>> the OSGi features *does* result in a lot of complexity.
>
> Just to express a usage datapoint here. Having used OSGi for a while
> now I am finding import-by-package too hard to use in certain
> circumstances (but in general it is a nice flexible model), but its
> not an issue that import-by-name really fixes in my opinion.
>
> The basic issue is that to use something I am interested in the union
> of packages that make up the "feature". I actually don't care about
> the individual packages. Take my current bugbear, cglib in Spring for
> example, to use this "feature" the *proxied* class bundle needs to
> import the following packages:
>
>   net.sf.cglib.proxy,
>   net.sf.cglib.core,
>   net.sf.cglib.reflect,
>   org.aopalliance.aop,
>   org.springframework.aop.framework,
>   org.springframework.aop,
>
> not an obvious set, nor a set of packages that I care much about.
> Worse, the proxied class does not know ahead of time that it needs to
> import these packages (this is I believe because cglib tries to
> create the proxy in the classloader of the proxied class by default).
>
> Ideally to solve this I would like two things:
>
> 1. The ability to express a feature in terms of its constituent
> packages. Note that import-by-name doesn't really help here since the
> actual packages are distributed across multiple modules. What I
> really want is some kind of feature module that exports these
> packages to you when you ask for the feature.
>
> 2. Some way of augmenting a module *after* the fact with a particular
> set of packages, so that the module doesn't need to do dynamic-import
> * everywhere.
>
> I don't think either if these is possible in OSGi right now (I would
> be happy to be corrected!). I don't think JSR 277 solves these either
> (but again I would be happy to be corrected).
>
> Just a datapoint from real-world usage.
>
> andy
>
>
> Notice:  This email message, together with any attachments, may
> contain information  of  BEA Systems,  Inc.,  its subsidiaries
> and  affiliated entities,  that may be confidential,  proprietary,
> copyrighted  and/or legally privileged, and is intended solely for
> the use of the individual or entity named in this message. If you
> are not the intended recipient, and have received this message in
> error, please immediately return this by email and then delete it.


From Stanley.Ho at sun.com  Fri Jun  8 14:23:57 2007
From: Stanley.Ho at sun.com (Stanley M. Ho)
Date: Fri, 08 Jun 2007 14:23:57 -0700
Subject: Relationship to JSR 291 [was: Re: Bryan's comments]
In-Reply-To: <6.2.5.6.2.20070608093830.02c499a8@bea.com>
References: <OFC12639E7.6C1E83B9-ON802572F1.00310AA2-802572F1.0033EC6C@uk.ibm.com>
	<4665DC8F.8000402@oracle.com>
	<81d16f0ad9ea912c29d9ba241ef0ce33@ungoverned.org>
	<46689A6C.5070207@oracle.com>
	<6.2.5.6.2.20070608093830.02c499a8@bea.com>
Message-ID: <4669C8ED.1030506@sun.com>

Andy Piper wrote:
> ...
> 1. The ability to express a feature in terms of its constituent
> packages. Note that import-by-name doesn't really help here since the
> actual packages are distributed across multiple modules. What I
> really want is some kind of feature module that exports these
> packages to you when you ask for the feature.

Unless I miss something, this is the main use case that import-by-name
would address. Developers can package all the classes related to a
feature (e.g. networking, xml, swing, ejb, etc.) into a module, and it
exports (or reexports from other modules) all the things you need; other
developers who want to use the feature would simply import that module
by name. In fact, this is how we intend to use the module system to
modularize various components in the SE platform.

- Stanley


From sam at SAMPULLARA.COM  Fri Jun  8 14:40:54 2007
From: sam at SAMPULLARA.COM (Sam Pullara)
Date: Fri, 08 Jun 2007 14:40:54 -0700
Subject: Relationship to JSR 291 [was: Re: Bryan's comments]
In-Reply-To: <4669C8ED.1030506@sun.com>
References: <OFC12639E7.6C1E83B9-ON802572F1.00310AA2-802572F1.0033EC6C@uk.ibm.com>
	<4665DC8F.8000402@oracle.com>
	<81d16f0ad9ea912c29d9ba241ef0ce33@ungoverned.org>
	<46689A6C.5070207@oracle.com>
	<6.2.5.6.2.20070608093830.02c499a8@bea.com>
	<4669C8ED.1030506@sun.com>
Message-ID: <A808386A-7992-4CC9-B2B0-3DFECBC6F692@sampullara.com>

Whew.  I thought something had gone south.

Sam

On Jun 8, 2007, at 2:23 PM, Stanley M. Ho wrote:

> Andy Piper wrote:
>> ...
>> 1. The ability to express a feature in terms of its constituent
>> packages. Note that import-by-name doesn't really help here since the
>> actual packages are distributed across multiple modules. What I
>> really want is some kind of feature module that exports these
>> packages to you when you ask for the feature.
>
> Unless I miss something, this is the main use case that import-by-name
> would address. Developers can package all the classes related to a
> feature (e.g. networking, xml, swing, ejb, etc.) into a module, and it
> exports (or reexports from other modules) all the things you need;
> other
> developers who want to use the feature would simply import that module
> by name. In fact, this is how we intend to use the module system to
> modularize various components in the SE platform.
>
> - Stanley


From heavy at UNGOVERNED.ORG  Mon Jun 11 08:54:20 2007
From: heavy at UNGOVERNED.ORG (Richard S. Hall)
Date: Mon, 11 Jun 2007 11:54:20 -0400
Subject: Relationship to JSR 291 [was: Re: Bryan's comments]
In-Reply-To: <4669C8ED.1030506@sun.com>
References: <OFC12639E7.6C1E83B9-ON802572F1.00310AA2-802572F1.0033EC6C@uk.ibm.com>
	<4665DC8F.8000402@oracle.com>
	<81d16f0ad9ea912c29d9ba241ef0ce33@ungoverned.org>
	<46689A6C.5070207@oracle.com>
	<6.2.5.6.2.20070608093830.02c499a8@bea.com>
	<4669C8ED.1030506@sun.com>
Message-ID: <466D702C.2020105@ungoverned.org>

Stanley M. Ho wrote:
> Andy Piper wrote:
>> ...
>> 1. The ability to express a feature in terms of its constituent
>> packages. Note that import-by-name doesn't really help here since the
>> actual packages are distributed across multiple modules. What I
>> really want is some kind of feature module that exports these
>> packages to you when you ask for the feature.
>
> Unless I miss something, this is the main use case that import-by-name
> would address. Developers can package all the classes related to a
> feature (e.g. networking, xml, swing, ejb, etc.) into a module, and it
> exports (or reexports from other modules) all the things you need; other
> developers who want to use the feature would simply import that module
> by name. In fact, this is how we intend to use the module system to
> modularize various components in the SE platform.

It certainly is possible to use the import-by-name approach to imitate
some sort of import-by-feature approach, but this assumes that you are
always able to create modules that have a one-to-one correspondence with
some higher level "feature" concept. While this may work in some cases,
e.g., replaceable modules implementing standard XML parsers, it is not
likely to be possible in all cases -- the fact that it is the only
mechanism available virtually guarantees that a majority of modules will
not meet this criterion.

Ultimately, this approach still falls short for the following reasons:

    * No matter how you describe it, the import-by-name approach ==
      import-by-module, since name == module name in 277. Thus, you are
      still saying who gives you what you want, rather than what you want.
    * As a byproduct of the above, you are still defining dependencies
      too broadly, since import-by-name says that you need everything
      from this named module, even though you might only need some very
      small subset, which will lead to dependency fanout.

Further, I would say that it is more wishful thinking rather than actual
reality to believe that a higher level concept even helps here, since in
the end for Java it always comes down to importing packages and using
them in your source code. The higher level concept does not exist in
source code nor does the programmer know anything about it; the
programmer does, however, know about packages and the associated API
since this is required to use anything in Java.

The real value of this sort of higher level concept is further
diminished when you introduce tooling support that can pull these sorts
of dependencies from the byte code and automatically generate dependency
metadata, such as BND does for OSGi bundles.

People may like the "import-by-name" approach when they are developing
and need to create a class path for their IDE, but in reality there is
no reason why they should have to do this for their IDE either. Assuming
that a module repository exists, if I start using classes from
Hibernate, for example, there is no reason why my IDE cannot see that
this package is not in my class path and offer to fix it by searching
the module repo for providers of the Hibernate package, similar to how
IDEs currently fix imports in the source code...in this case it could
even allow you to select from multiple versions. Then the issue of "ease
of use" goes away completely.

So, in short, call it what you want, but the solution offered by 277 is
import-by-module with all of the associated shortcomings.

-> richard


From andyp at bea.com  Mon Jun 11 05:55:13 2007
From: andyp at bea.com (Andy Piper)
Date: Mon, 11 Jun 2007 13:55:13 +0100
Subject: Relationship to JSR 291 [was: Re: Bryan's comments]
In-Reply-To: <4669C8ED.1030506@sun.com>
References: <OFC12639E7.6C1E83B9-ON802572F1.00310AA2-802572F1.0033EC6C@uk.ibm.com>
	<4665DC8F.8000402@oracle.com>
	<81d16f0ad9ea912c29d9ba241ef0ce33@ungoverned.org>
	<46689A6C.5070207@oracle.com>
	<6.2.5.6.2.20070608093830.02c499a8@bea.com>
	<4669C8ED.1030506@sun.com>
Message-ID: <6.2.5.6.2.20070611135135.028a97d8@bea.com>

At 10:23 PM 6/8/2007, Stanley M. Ho wrote:
>Unless I miss something, this is the main use case that import-by-name
>would address. Developers can package all the classes related to a
>feature (e.g. networking, xml, swing, ejb, etc.) into a module, and it
>exports (or reexports from other modules) all the things you need; other
>developers who want to use the feature would simply import that module
>by name. In fact, this is how we intend to use the module system to
>modularize various components in the SE platform.

Ok so re-export is the key difference from OSGi here then. But this
raises the question, you now have two modules exporting the same
package, so how do you resolve this? Does the module system correctly
reconcile the classes as coming from the same source?

andy



Notice:  This email message, together with any attachments, may contain information  of  BEA Systems,  Inc.,  its subsidiaries  and  affiliated entities,  that may be confidential,  proprietary,  copyrighted  and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by email and then delete it.


From bryan.atsatt at oracle.com  Mon Jun 11 15:22:14 2007
From: bryan.atsatt at oracle.com (Bryan Atsatt)
Date: Mon, 11 Jun 2007 15:22:14 -0700
Subject: Query optimization
Message-ID: <466DCB16.6020007@oracle.com>

Given a list of ModuleDefinition instances, it is obvious how a
Repository implementation can do a *linear* search using Query:

private List<ModuleDefinition> definitions;

public List<ModuleDefinition> findModuleDefinitions(Query constraint) {
     List<ModuleDefinition> result = new ArrayList<ModuleDefinition>();
     for(ModuleDefinition def : definitions) {
         if (constraint.match(def)) {
             result.add(def);
         }
     }
}


We ought to be able to do much better than this, however, and make it
easy to use one or more indices to narrow the search. And I think we
should. Sure, small collections won't benefit hugely from it. But large
ones will, and, even more importantly, *remote* ones can by using a
local index. URLRepository is a perfect candidate, as the contents of
the repository-metadata.xml file can be turned into an index.

And consider a large, remote, *centralized* repository (perhaps even
hosted by Google ;^)


Given the current import-by-name syntax, the least we can do is to
leverage that name, if present in the Query, to use an index:

private Map<String, List<ModuleDefinition>> byNameIndex;

public List<ModuleDefinition> findModuleDefinitions(Query constraint) {

     List<ModuleDefinition> result = EMPTY_LIST;

     // Extract the module name from the query somehow (TBD)
     String moduleName = constraint.getModuleName();

     if (moduleName == null) {

         // Fall back on linear search.
         result = find(constraint, definitions);

     } else {

         // Use index to narrow search.
         List<ModuleDefinition> list = definitions.get(moduleName);
         if (list != null) {
             result = find(constraint, list);
         }
     }
     return result;
}

private List<ModuleDefinition> find(Query constraint,
                                     List<ModuleDefinition> context) {
     List<ModuleDefinition> result = new ArrayList<ModuleDefinition>();
     for(ModuleDefinition def : context) {
         if (constraint.match(def)) {
             result.add(def);
         }
     }
     return result;
}


Used this way, module-name becomes a primary key. It should also be
possible to have package-name as a primary key.

We could hard-wire support for common primary keys. Better, though,
would be a general mechanism for this, so that Repository
implementations can index on whatever makes sense. I think this can
actually be accomplished reasonably easily...

First, note that the index above only works if the node performs the
*same* value comparison operation, in this case String.equals(). This is
important, since if the node actually does a !String.equals(), our use
of the index is just plain wrong.

Here are some elements of the problem as things currently stand:

1. Iteration. How do we access the contents of a Query?

2. Value types. How do we determine the type of a Query value?

3. Comparison operations. How do we determine the comparison operation?

4. Extraction. How do we extract a value from a primary key node?

5. Identifier(s). How do we specify what can be used as a primary key?

The visitor pattern is an easy solution to #1, but is awkward to use.
Far better would be if we can make Query implement Iterable<Query> so
that for-each loops can be used. This requires a slightly more
complicated implementation, but I think the usability is probably worth it:

public abstract class Query implements Iterable<Query> {
     public Iterator<Query> iterator() {...}
     public boolean hasChildren() {return false;}
     public List<Query> getChildren() {return EMPTY_LIST};

     ...
}

The boolean node types just override the has/getChildren() methods.

Ok, so now that we can walk query nodes, how do we tackle the other
issues? It seems to me that there is one relatively simple path that
resolves all of the remaining issues at the same time: explicit, public
Query subclasses. For example:

public class ModuleNameEquals extends Query {
     public ModuleNameEquals (String name) {...}
     public String getName() {...}
     public boolean match(ModuleDefinition def) {
         return getName().equals(def.getName());
     }
}

public class AttributeEquals extends Query {
     public AttributeEquals (String key, String value) {...}
     public String getKey() {...}
     public String getValue() {...}
     public boolean match(ModuleDefinition def) {
         String v = def.getAttribute(getKey());
         return getValue().equals(v);
     }
}

public class ExportsPackage extends Query {
     public ExportsPackage (String name) {...}
     public String getName() {...}
     public boolean match(ModuleDefinition def) {
         Set<String> pkgs = def.getExportedPackages();
         return pkgs.contains(getName());
     }
}

public class VersionEquals extends Query {
     public VersionEquals (Version version) {...}
     public Version getVersion() {...}
     public boolean match(ModuleDefinition def) {
         return def.getVersion().equals(getVersion());
     }
}

public class VersionInRange extends Query {
     public VersionInRange (VersionRange range) {...}
     public VersionRange getRange() {...}
     public boolean match(ModuleDefinition def) {
         return getRange().contains(def.getVersion());
     }
}

...

Now we can iterate across any query looking for types that match any
index we've created, and extract values to use against them. And the
model is easily extensible if we want to support other types in the
future (e.g. ModuleSystemNameEquals, AttributeGreaterThan, etc.).


But, we're not done yet; knowing that a Query contains a
ModuleNameEquals instance, for example, isn't enough. What if the
instance is a child of a Not instance (i.e.
Query.not(Query.name("foo")))? Our use of the name against our
byNameIndex example above would then fail utterly.

In the spirit of making "simple things easy, and hard things possible",
I think we should provide convenience methods for the common cases.
(Which will also help guard against repository implementors missing the
Not case.)

A simple way to expose this is to add a method to each of the explicit
types for which we'd like to support indexing. For example:

public class ModuleNameEquals {
    public static List<String> getIndexableNames(Query q) {...}
    ...
}

This method would iterate the query looking for instances of its own
class, and collect up the names. More importantly, it would ignore an
instance if a Not node is found as a *parent*. (It could even be made
smart enough to deal with double negatives.)

We can add similar methods for the other types.

(We could also get fancy and add support for multiple indices and do the
  intersection or union, but I think that is better left to repository
implementors :^)


Thoughts?

// Bryan


From heavy at UNGOVERNED.ORG  Mon Jun 11 16:44:32 2007
From: heavy at UNGOVERNED.ORG (Richard S.Hall)
Date: Mon, 11 Jun 2007 19:44:32 -0400
Subject: Relationship to JSR 291 [was: Re: Bryan's comments]
In-Reply-To: <6.2.5.6.2.20070611135135.028a97d8@bea.com>
References: <OFC12639E7.6C1E83B9-ON802572F1.00310AA2-802572F1.0033EC6C@uk.ibm.com>
	<4665DC8F.8000402@oracle.com>
	<81d16f0ad9ea912c29d9ba241ef0ce33@ungoverned.org>
	<46689A6C.5070207@oracle.com>
	<6.2.5.6.2.20070608093830.02c499a8@bea.com>
	<4669C8ED.1030506@sun.com> <6.2.5.6.2.20070611135135.028a97d8@bea.com>
Message-ID: <9afdda27152c90e233d38d9804a2a7ae@ungoverned.org>

On Jun 11, 2007, at 8:55 AM, Andy Piper wrote:

> At 10:23 PM 6/8/2007, Stanley M. Ho wrote:
>> Unless I miss something, this is the main use case that import-by-name
>> would address. Developers can package all the classes related to a
>> feature (e.g. networking, xml, swing, ejb, etc.) into a module, and it
>> exports (or reexports from other modules) all the things you need;
>> other
>> developers who want to use the feature would simply import that module
>> by name. In fact, this is how we intend to use the module system to
>> modularize various components in the SE platform.
>
> Ok so re-export is the key difference from OSGi here then. But this
> raises the question, you now have two modules exporting the same
> package, so how do you resolve this? Does the module system correctly
> reconcile the classes as coming from the same source?

One thing to point out is that OSGi's import-by-module also supports
re-export, so this is not a difference. The main difference between the
two import-by-module concepts is that the OSGi import-by-module also
supports split packages across imported modules, which complicates
things a little more.

As far as figuring out where packages come from, this is somewhat
complicated for the OSGi framework due to split packages, but if it did
not support split packages, then it would be reasonably
straightforward.

-> richard

>
> andy
>
>
>
> Notice:  This email message, together with any attachments, may
> contain information  of  BEA Systems,  Inc.,  its subsidiaries  and
> affiliated entities,  that may be confidential,  proprietary,
> copyrighted  and/or legally privileged, and is intended solely for the
> use of the individual or entity named in this message. If you are not
> the intended recipient, and have received this message in error,
> please immediately return this by email and then delete it.


From bryan.atsatt at oracle.com  Mon Jun 11 18:45:09 2007
From: bryan.atsatt at oracle.com (Bryan Atsatt)
Date: Mon, 11 Jun 2007 18:45:09 -0700
Subject: Exported resources
In-Reply-To: <4668A6DF.1040906@sun.com>
References: <4654109D.40205@sun.com> <46548736.7020507@oracle.com>
	<465629EA.2050008@sun.com> <465658CE.5020409@oracle.com>
	<465DCE51.4010304@sun.com> <465DDCD0.8090703@oracle.com>
	<465DF6D6.5050608@sun.com> <465E0201.2030308@oracle.com>
	<465F68C8.4000103@sun.com> <46609428.4000909@oracle.com>
	<4664C2B1.1050109@sun.com> <4664E9C6.30608@oracle.com>
	<46674099.5040908@sun.com> <466747B7.2030709@oracle.com>
	<46676134.8020206@sun.com> <4667669C.7060708@oracle.com>
	<4668A6DF.1040906@sun.com>
Message-ID: <466DFAA5.6000905@oracle.com>

Stanley M. Ho wrote:
> Hi Bryan,
>
> Bryan Atsatt wrote:
>> Yes, I think we're getting close now :^)
>>
>> (By "privileged reflection", you mean having permission to call
>> AccessibleObject.setAccessible(true), right? If so, then we're back to
>> the grant issue!)
>
> Yes. In this case, it's the ResourceBundle API calling it to retrieve
> ListResourceBundle, so it'll have the sufficient permissions to do that.
> Anyway, I think we should leave this issue aside for now.
>
>> I still think we can provide "private", (i.e. non-exported) resources
>> which *are* available to the module classes, but to no class outside
>> of it.
>>
>> Without using permissions.
>>
>> Remember this?
>>
>>   private static class StackAccessor extends SecurityManager {
>>       public Class[] getStack() {
>>           return getClassContext();
>>       }
>>   }
>>
>>   private static final STACK_ACCESSOR = new StackAccessor();
>>
>> If a resource is found, and is not-exported, a module class loader can
>> use this (or similar) to get the immediate caller's class (at a fixed
>> index).
>>
>> If the caller class' Module does not match that for the loader, we don't
>> return the resource.
>
> I think we should step back to rethink the use cases a bit.
>
> Typically, a module calls some APIs (e.g. ResourceBundle, AWTToolkit,
> etc.) and these APIs might then retrieve resources through the module's
> ClassLoader.getResource*(). If we are saying that a module is required
> to make these resources exported in order for these use cases to work,
> then what are the actual use cases we want to support by hiding
> resources from getResource*()?
>
> I think there remains a common use case where hiding resources would be
> a good thing if developers want to hide all the resources that are not
> used by anyone through ClassLoader.getResource*() (e.g. all the .class
> files). On the other hand, I think the use case where the code in the
> module calls its own ClassLoader.getResource*() to retrieve private
> resources directly is not very common; If this is a rare use case, I
> don't think it makes sense to introduce this notion of "private
> resources". If we are forcing developers to export resources for the
> common use cases (like ResourceBundle and ServiceLoader) anyway, then
> why not require them to export resources in other cases as well?
>
> It would help if there are some real use cases to support why it is
> important to have this notion of "private resources".

One very common use case comes from the EE world: web modules containing
html/gif/xml/jsp files, all of which are implementation details.

I would assume that most any app with a UI would have similar resources.

>
> - Stanley
>


From bryan.atsatt at oracle.com  Tue Jun 12 10:55:57 2007
From: bryan.atsatt at oracle.com (Bryan Atsatt)
Date: Tue, 12 Jun 2007 10:55:57 -0700
Subject: Exported resources
In-Reply-To: <466DFAA5.6000905@oracle.com>
References: <4654109D.40205@sun.com> <46548736.7020507@oracle.com>
	<465629EA.2050008@sun.com> <465658CE.5020409@oracle.com>
	<465DCE51.4010304@sun.com> <465DDCD0.8090703@oracle.com>
	<465DF6D6.5050608@sun.com> <465E0201.2030308@oracle.com>
	<465F68C8.4000103@sun.com> <46609428.4000909@oracle.com>
	<4664C2B1.1050109@sun.com> <4664E9C6.30608@oracle.com>
	<46674099.5040908@sun.com> <466747B7.2030709@oracle.com>
	<46676134.8020206@sun.com> <4667669C.7060708@oracle.com>
	<4668A6DF.1040906@sun.com> <466DFAA5.6000905@oracle.com>
Message-ID: <466EDE2D.9020508@oracle.com>

Whoops, .jsp files are not candidates for private resources, since the
*container* must have access to them. But I stand by the gif/html/xml
comment :^).

Another candidate is property files (thanks to Stephen McConnell for
reminding me), when used as default configuration.

Bryan Atsatt wrote:
> Stanley M. Ho wrote:
>> Hi Bryan,
>>
>> Bryan Atsatt wrote:
>>> Yes, I think we're getting close now :^)
>>>
>>> (By "privileged reflection", you mean having permission to call
>>> AccessibleObject.setAccessible(true), right? If so, then we're back to
>>> the grant issue!)
>>
>> Yes. In this case, it's the ResourceBundle API calling it to retrieve
>> ListResourceBundle, so it'll have the sufficient permissions to do that.
>> Anyway, I think we should leave this issue aside for now.
>>
>>> I still think we can provide "private", (i.e. non-exported) resources
>>> which *are* available to the module classes, but to no class outside
>>> of it.
>>>
>>> Without using permissions.
>>>
>>> Remember this?
>>>
>>>   private static class StackAccessor extends SecurityManager {
>>>       public Class[] getStack() {
>>>           return getClassContext();
>>>       }
>>>   }
>>>
>>>   private static final STACK_ACCESSOR = new StackAccessor();
>>>
>>> If a resource is found, and is not-exported, a module class loader can
>>> use this (or similar) to get the immediate caller's class (at a fixed
>>> index).
>>>
>>> If the caller class' Module does not match that for the loader, we don't
>>> return the resource.
>>
>> I think we should step back to rethink the use cases a bit.
>>
>> Typically, a module calls some APIs (e.g. ResourceBundle, AWTToolkit,
>> etc.) and these APIs might then retrieve resources through the module's
>> ClassLoader.getResource*(). If we are saying that a module is required
>> to make these resources exported in order for these use cases to work,
>> then what are the actual use cases we want to support by hiding
>> resources from getResource*()?
>>
>> I think there remains a common use case where hiding resources would be
>> a good thing if developers want to hide all the resources that are not
>> used by anyone through ClassLoader.getResource*() (e.g. all the .class
>> files). On the other hand, I think the use case where the code in the
>> module calls its own ClassLoader.getResource*() to retrieve private
>> resources directly is not very common; If this is a rare use case, I
>> don't think it makes sense to introduce this notion of "private
>> resources". If we are forcing developers to export resources for the
>> common use cases (like ResourceBundle and ServiceLoader) anyway, then
>> why not require them to export resources in other cases as well?
>>
>> It would help if there are some real use cases to support why it is
>> important to have this notion of "private resources".
>
> One very common use case comes from the EE world: web modules containing
> html/gif/xml/jsp files, all of which are implementation details.
>
> I would assume that most any app with a UI would have similar resources.
>
>>
>> - Stanley
>>
>


From bryan.atsatt at oracle.com  Tue Jun 12 15:44:51 2007
From: bryan.atsatt at oracle.com (Bryan Atsatt)
Date: Tue, 12 Jun 2007 15:44:51 -0700
Subject: Import specifications (was Re: Import constraints)
In-Reply-To: <4668B1F8.1040503@sun.com>
References: <46645C1A.2090809@oracle.com> <4668B1F8.1040503@sun.com>
Message-ID: <466F21E3.5060304@oracle.com>

Stanley M. Ho wrote:
> Hi Bryan,
>
> Bryan Atsatt wrote:
>> ...
>> By having Dependency extend Query, instances can be directly passed to
>> Repository.find(); no conversion required. We could obviously just make
>> a getter for the Query if we want, but why not make this convenient to
>> use?
>
> An import dependency describes the relationship between two modules.

Agreed. And I think we would be far better off thinking of and modeling
this relationship as a general *requirement specification*, rather than
using the very narrow definition:

    name + version(s)

Clearly these are important elements of an import specification. But the
design should allow for and even support others.

One concrete example is attributes. What is the point of declaring them
on a module if they cannot be used in an import specification? And I
think these could be very valuable.

The term "constraint" makes sense when you think of name as the primary
specification, and everything else as modifiers of it.

But this puts all the emphasis on the wrong syLLable, IMO. What will we
do when a "contract" model is introduced, where attributes and
version(s) are all that is required to specify the desired contract?

Module name won't be required at all in such a specification.


So far, we have three distinct actors in the resolution process:

1. Developer: creates import specifications, using whatever selection
criteria makes sense.

2. Admin: creates import specification *filter* (ImportOverridePolicy),
mapping developer specification to fit the local environment. (Can also
use a VisibilityPolicy, but it isn't clear to me why that same
functionality can't be achieved in the override policy.)

3. Module System: locates definitions that match specifications.

All of these actors collaborate to produce a single list of candidate
definitions at runtime. If that list contains duplicate packages, the
module system must select the best fit, or fail.

Clearly the Query mechanism provides an extensible solution to the
problem in #3. And annotations give us an extensible solution to the
problem in #1.

But, just as clearly, we have a big gap in the middle: the filter model
is not a general, extensible mechanism. And I think we need to fix this.

It seems to me that the best solution is to use the *same* model to
filter as we use for lookup: Query.

(Please see the "Query optimization" thread for my proposal to change
Query from an opaque to a transparent type.)

So, rather than pass VersionConstraint instances through the narrow()
method, we can pass Query instances. Yes, this does make it a bit more
work to implement such a filter, but it is then a completely extensible,
symmetric model.

And this is also why I proposed that we change ImportDependency to
simply take a Query as a ctor argument:

   public ImportDependency (Query spec,
                            boolean optional,
                            boolean reExport) {...}

(I should have sent my "optimization" proposal before I suggested the
above change, so it would've made more sense. Really this would be so
much easier if you could just read my mind :^)


Further, I don't understand the need to limit the filter to only
"narrowing" operations. For example, why shouldn't an admin be able to
map a module *name* as well? This could be really handy (and might even
allow for elimination of the refactoring/name problem).

In fact, why not allow the admin to be able to filter *any* part of the
import specification? I would prefer to see a more generic model than
the current ImportOverridePolicy, for example:

public class ImportOverridePolicy {
    public List<ImportDependency> getImports(ModuleDefinition def) {
        return def.getImportDependencies();
    }

    public static ImportOverridePolicy getPolicy() {...}
    static void setPolicy(ImportOverridePolicy policy) {...}
}

The default implementation does nothing, but subtypes can perform any
transformation they want. The ModuleSystem just calls this method
instead of calling the definition directly.

> It is true that we may want to query a module described in the import
> dependency, but an import dependency itself is not a query. In fact,
> someone may even want to query a module which contains the specific
> import dependency, although I think this use case is very rare. Anyway,
> it sounded like the main benefit to have import dependency extend query
> is to make it easier to pass into Repository.find(). Unfortunately, once
> a class is extended from another, all the public/protected
> methods/fields from the parent class would automatically become part of
> the signature of the child class.

No kidding? :^)

> I don't think we want to pollute the
> class hierarchy and potentially confuse developers just for a bit of
> convenient. ;-)

This is a style issue, and not one I care a great deal about--it
certainly is not the focus of my original message! But I will say that
convenience as a factor in design decisions should not be taken lightly.
To me, a clumsy usage model is a strong smell indicating that the
abstractions are not quite correct.

>
>> Second, I think we should eliminate VersionConstraint as a separate
>> class. It's functionality is easy to replace with Query elements that
>> match Version or VersionRange, strung together with appropriate boolean
>> operators.
>
> There are methods in VersionConstraint for determining if it contains a
> specified version/version range/version constraint. These methods are
> used heavily for version constraint override in module
> initialization/import policy/import override policy, and they are not
> suitable to be moved into the Query class. I think VersionConstraint
> should stay as a separate class.
>
>> Third, we should add a getExportedPackages() method to ModuleDefinition.
>> This way a Query can easily be created to search by package name.
>> Whether or not we choose to surface import-by-package declaratively, it
>> should at least be possible for a ModuleSystem to use such a Query.
>
> Having something like getExportedPackages() sounds reasonable from the
> perspective of interoperability.
>
>> Fourth, we should add support for simple attribute constraint
>> declarations. Enabling simple equality constraints ought to be easy
>> enough: an AttributeConstraint annotation that takes key/value lists
>> (key1==value1;key2==value2;...). We already have support for expressing
>> these as Query instances.
>
> The Query class currently only supports querying arbitrary attributes at
> the module level. Are you suggesting that we should let developers to
> declare arbitrary attributes at the import?

Yes.

> Could you come up with real use cases for this in the context of 277 alone?

Sure: import-by-contract. One such contract is a specification name;
together with version info, a Specification-Name attribute could be used
to select a definition.

Out current spec describes "attribute matching" as one use case for
implementing a custom ImportPolicy. If there is any value in enabling
this, then we should probably support it directly "at the import".


>
>> Finally, Query ought to have a toString() implementation so that we can
>> log them, examine in a debugger, put them in error messages, etc.
>
> The Query.toString() will output a human readable string, and this is
> already implemented in the RI.
>
> - Stanley
>


From Stanley.Ho at sun.com  Tue Jun 12 16:39:46 2007
From: Stanley.Ho at sun.com (Stanley M. Ho)
Date: Tue, 12 Jun 2007 16:39:46 -0700
Subject: Exported resources
In-Reply-To: <466EDE2D.9020508@oracle.com>
References: <4654109D.40205@sun.com> <46548736.7020507@oracle.com>
	<465629EA.2050008@sun.com> <465658CE.5020409@oracle.com>
	<465DCE51.4010304@sun.com> <465DDCD0.8090703@oracle.com>
	<465DF6D6.5050608@sun.com> <465E0201.2030308@oracle.com>
	<465F68C8.4000103@sun.com> <46609428.4000909@oracle.com>
	<4664C2B1.1050109@sun.com> <4664E9C6.30608@oracle.com>
	<46674099.5040908@sun.com> <466747B7.2030709@oracle.com>
	<46676134.8020206@sun.com> <4667669C.7060708@oracle.com>
	<4668A6DF.1040906@sun.com> <466DFAA5.6000905@oracle.com>
	<466EDE2D.9020508@oracle.com>
Message-ID: <466F2EC2.8050107@sun.com>

Hi Bryan,

I would like to have some closures on the older threads first before
responding to the newly created ones, so don't take my silent as
ignoring those new threads. ;-)

Bryan Atsatt wrote:
> Whoops, .jsp files are not candidates for private resources, since the
> *container* must have access to them. But I stand by the gif/html/xml
> comment :^).
>
> Another candidate is property files (thanks to Stephen McConnell for
> reminding me), when used as default configuration.

In the context of the EE environment, my question is that who exactly
are we trying to protect these resources from? If the answer is the
container, making these resources public would also be acceptable 'cause
I don't think we expect the container would mess around these resources
if they remain public.

Based on what we discussed so far, my impression is that we can
implement the notion of "private resources" but it won't solve the
general cases like ResourceBundle; having the notion of "private
resources" would be nice-to-have in some cases, but they are not critical.

I think we have dragged on this thread for too long. Perhaps a way out
is to go with the simple notion of exported-means-visible-and-accessible
for now (i.e. exported resources are visible and accessible by everyone,
non-exported are visible and accessible by nobody). We could revisit
this issue around "private resources" after we gather more feedbacks
when the updated specification is published. What do you think?

- Stanley


From bryan.atsatt at ORACLE.COM  Tue Jun 12 16:39:52 2007
From: bryan.atsatt at ORACLE.COM (Bryan Atsatt)
Date: Tue, 12 Jun 2007 16:39:52 -0700
Subject: Import specifications (was Re: Import constraints)
In-Reply-To: <466F21E3.5060304@oracle.com>
References: <46645C1A.2090809@oracle.com> <4668B1F8.1040503@sun.com>
	<466F21E3.5060304@oracle.com>
Message-ID: <466F2EC8.4060805@oracle.com>

Bryan Atsatt wrote:
> Stanley M. Ho wrote:
>> Hi Bryan,
>>
>> Bryan Atsatt wrote:
>>> ...
>>> By having Dependency extend Query, instances can be directly passed to
>>> Repository.find(); no conversion required. We could obviously just make
>>> a getter for the Query if we want, but why not make this convenient to
>>> use?
>>
>> An import dependency describes the relationship between two modules.
>
> Agreed. And I think we would be far better off thinking of and modeling
> this relationship as a general *requirement specification*, rather than
> using the very narrow definition:
>
>    name + version(s)
>
> Clearly these are important elements of an import specification. But the
> design should allow for and even support others.
>
> One concrete example is attributes. What is the point of declaring them
> on a module if they cannot be used in an import specification?

Sorry, I meant to change this before sending. I do know that you meant
for attributes to be used by higher level mechanisms (e.g.
ServiceLoader). And that is certainly a valid use case (though it isn't
entirely clear to me how this works other than by polling).

But enabling them as import specifiers seems like an even more natural
use case.

> And I think these could be very valuable.
>
> The term "constraint" makes sense when you think of name as the primary
> specification, and everything else as modifiers of it.
>
> But this puts all the emphasis on the wrong syLLable, IMO. What will we
> do when a "contract" model is introduced, where attributes and
> version(s) are all that is required to specify the desired contract?
>
> Module name won't be required at all in such a specification.
>
>
> So far, we have three distinct actors in the resolution process:
>
> 1. Developer: creates import specifications, using whatever selection
> criteria makes sense.
>
> 2. Admin: creates import specification *filter* (ImportOverridePolicy),
> mapping developer specification to fit the local environment. (Can also
> use a VisibilityPolicy, but it isn't clear to me why that same
> functionality can't be achieved in the override policy.)
>
> 3. Module System: locates definitions that match specifications.
>
> All of these actors collaborate to produce a single list of candidate
> definitions at runtime. If that list contains duplicate packages, the
> module system must select the best fit, or fail.
>
> Clearly the Query mechanism provides an extensible solution to the
> problem in #3. And annotations give us an extensible solution to the
> problem in #1.
>
> But, just as clearly, we have a big gap in the middle: the filter model
> is not a general, extensible mechanism. And I think we need to fix this.
>
> It seems to me that the best solution is to use the *same* model to
> filter as we use for lookup: Query.
>
> (Please see the "Query optimization" thread for my proposal to change
> Query from an opaque to a transparent type.)
>
> So, rather than pass VersionConstraint instances through the narrow()
> method, we can pass Query instances. Yes, this does make it a bit more
> work to implement such a filter, but it is then a completely extensible,
> symmetric model.
>
> And this is also why I proposed that we change ImportDependency to
> simply take a Query as a ctor argument:
>
>   public ImportDependency (Query spec,
>                            boolean optional,
>                            boolean reExport) {...}
>
> (I should have sent my "optimization" proposal before I suggested the
> above change, so it would've made more sense. Really this would be so
> much easier if you could just read my mind :^)
>
>
> Further, I don't understand the need to limit the filter to only
> "narrowing" operations. For example, why shouldn't an admin be able to
> map a module *name* as well? This could be really handy (and might even
> allow for elimination of the refactoring/name problem).
>
> In fact, why not allow the admin to be able to filter *any* part of the
> import specification? I would prefer to see a more generic model than
> the current ImportOverridePolicy, for example:
>
> public class ImportOverridePolicy {
>    public List<ImportDependency> getImports(ModuleDefinition def) {
>        return def.getImportDependencies();
>    }
>
>    public static ImportOverridePolicy getPolicy() {...}
>    static void setPolicy(ImportOverridePolicy policy) {...}
> }
>
> The default implementation does nothing, but subtypes can perform any
> transformation they want. The ModuleSystem just calls this method
> instead of calling the definition directly.
>
>> It is true that we may want to query a module described in the import
>> dependency, but an import dependency itself is not a query. In fact,
>> someone may even want to query a module which contains the specific
>> import dependency, although I think this use case is very rare. Anyway,
>> it sounded like the main benefit to have import dependency extend query
>> is to make it easier to pass into Repository.find(). Unfortunately, once
>> a class is extended from another, all the public/protected
>> methods/fields from the parent class would automatically become part of
>> the signature of the child class.
>
> No kidding? :^)
>
>> I don't think we want to pollute the
>> class hierarchy and potentially confuse developers just for a bit of
>> convenient. ;-)
>
> This is a style issue, and not one I care a great deal about--it
> certainly is not the focus of my original message! But I will say that
> convenience as a factor in design decisions should not be taken lightly.
> To me, a clumsy usage model is a strong smell indicating that the
> abstractions are not quite correct.
>
>>
>>> Second, I think we should eliminate VersionConstraint as a separate
>>> class. It's functionality is easy to replace with Query elements that
>>> match Version or VersionRange, strung together with appropriate boolean
>>> operators.
>>
>> There are methods in VersionConstraint for determining if it contains a
>> specified version/version range/version constraint. These methods are
>> used heavily for version constraint override in module
>> initialization/import policy/import override policy, and they are not
>> suitable to be moved into the Query class. I think VersionConstraint
>> should stay as a separate class.
>>
>>> Third, we should add a getExportedPackages() method to ModuleDefinition.
>>> This way a Query can easily be created to search by package name.
>>> Whether or not we choose to surface import-by-package declaratively, it
>>> should at least be possible for a ModuleSystem to use such a Query.
>>
>> Having something like getExportedPackages() sounds reasonable from the
>> perspective of interoperability.
>>
>>> Fourth, we should add support for simple attribute constraint
>>> declarations. Enabling simple equality constraints ought to be easy
>>> enough: an AttributeConstraint annotation that takes key/value lists
>>> (key1==value1;key2==value2;...). We already have support for expressing
>>> these as Query instances.
>>
>> The Query class currently only supports querying arbitrary attributes at
>> the module level. Are you suggesting that we should let developers to
>> declare arbitrary attributes at the import?
>
> Yes.
>
>> Could you come up with real use cases for this in the context of 277
>> alone?
>
> Sure: import-by-contract. One such contract is a specification name;
> together with version info, a Specification-Name attribute could be used
> to select a definition.
>
> Out current spec describes "attribute matching" as one use case for
> implementing a custom ImportPolicy. If there is any value in enabling
> this, then we should probably support it directly "at the import".
>
>
>>
>>> Finally, Query ought to have a toString() implementation so that we can
>>> log them, examine in a debugger, put them in error messages, etc.
>>
>> The Query.toString() will output a human readable string, and this is
>> already implemented in the RI.
>>
>> - Stanley
>>
>


From bryan.atsatt at oracle.com  Wed Jun 13 11:17:24 2007
From: bryan.atsatt at oracle.com (Bryan Atsatt)
Date: Wed, 13 Jun 2007 11:17:24 -0700
Subject: Exported resources
In-Reply-To: <466F2EC2.8050107@sun.com>
References: <4654109D.40205@sun.com> <46548736.7020507@oracle.com>
	<465629EA.2050008@sun.com> <465658CE.5020409@oracle.com>
	<465DCE51.4010304@sun.com> <465DDCD0.8090703@oracle.com>
	<465DF6D6.5050608@sun.com> <465E0201.2030308@oracle.com>
	<465F68C8.4000103@sun.com> <46609428.4000909@oracle.com>
	<4664C2B1.1050109@sun.com> <4664E9C6.30608@oracle.com>
	<46674099.5040908@sun.com> <466747B7.2030709@oracle.com>
	<46676134.8020206@sun.com> <4667669C.7060708@oracle.com>
	<4668A6DF.1040906@sun.com> <466DFAA5.6000905@oracle.com>
	<466EDE2D.9020508@oracle.com> <466F2EC2.8050107@sun.com>
Message-ID: <467034B4.9050506@oracle.com>

(resend)

No problem, tackle these however you want. I've just had some time and
have taken the opportunity to air out some of the ideas that have been
lurking in my brain...

// Bryan

Stanley M. Ho wrote:
> Hi Bryan,
>
> I would like to have some closures on the older threads first before
> responding to the newly created ones, so don't take my silent as
> ignoring those new threads. ;-)
>
> Bryan Atsatt wrote:
>> Whoops, .jsp files are not candidates for private resources, since the
>> *container* must have access to them. But I stand by the gif/html/xml
>> comment :^).
>>
>> Another candidate is property files (thanks to Stephen McConnell for
>> reminding me), when used as default configuration.
>
> In the context of the EE environment, my question is that who exactly
> are we trying to protect these resources from? If the answer is the
> container, making these resources public would also be acceptable 'cause
> I don't think we expect the container would mess around these resources
> if they remain public.
>
> Based on what we discussed so far, my impression is that we can
> implement the notion of "private resources" but it won't solve the
> general cases like ResourceBundle; having the notion of "private
> resources" would be nice-to-have in some cases, but they are not critical.
>
> I think we have dragged on this thread for too long. Perhaps a way out
> is to go with the simple notion of exported-means-visible-and-accessible
> for now (i.e. exported resources are visible and accessible by everyone,
> non-exported are visible and accessible by nobody). We could revisit
> this issue around "private resources" after we gather more feedbacks
> when the updated specification is published. What do you think?
>
> - Stanley
>


From Stanley.Ho at sun.com  Wed Jun 13 11:19:10 2007
From: Stanley.Ho at sun.com (Stanley M. Ho)
Date: Wed, 13 Jun 2007 11:19:10 -0700
Subject: Strawman: Services and service-providers support
In-Reply-To: <466F3529.90002@sun.com>
References: <465374B2.7060501@sun.com> <466F3529.90002@sun.com>
Message-ID: <4670351E.40109@sun.com>

My original email did not seem to go through ... resend.

- Stanley

-------- Original Message --------
Subject: Re: Strawman: Services and service-providers support
Date: Tue, 12 Jun 2007 17:07:05 -0700
From: Stanley M. Ho <Stanley.Ho at Sun.COM>
Organization: Sun Microsystems, Inc.
To: Java Community Process JSR #277 Expert List <JSR-277-EG at JCP.ORG>

Hi JSR 277 experts,

Since I have not heard any further input on the services and
service-providers strawman, I suppose the EG is fine with the strawman
overall except the issue raised by Richard. Unless I hear any objection,
I will incorporate the appropriate portion of the strawman based on the
feedback you have provided into the next revision of the specification.

Thanks,
- Stanley

P.S. I am still working on getting the strawman out to the observers
through the infrastructure on openjdk.org, and it will take some time.


From Stanley.Ho at Sun.COM  Wed Jun 13 11:23:00 2007
From: Stanley.Ho at Sun.COM (Stanley M. Ho)
Date: Wed, 13 Jun 2007 11:23:00 -0700
Subject: Module isolation
In-Reply-To: <466088D9.9020903@oracle.com>
References: <OFD20EE2C8.4BD4A1F7-ON802572ED.00360624-802572ED.00464DBD@uk.ibm.com>
	<466088D9.9020903@oracle.com>
Message-ID: <46703604.5040203@sun.com>

Resend.

Bryan Atsatt wrote:
> ...
> Stanley's idea was that a Repository implementation could return a
> *copy* of a ModuleDefinition created by a different Repository. Such a
> Repository would in fact form an isolation context. And caching Module
> instances becomes trivial: each definition can just have one as a field.
>
> While there are definite lifecycle issues here between such repository
> instances, they are probably solvable.
>
> And then we really *do* have an exact analogue of the loader delegation
> model, in which:
>
>    Repository       ~= ClassLoader
>    ModuleDefinition ~= Class
>    Module           ~= Object
>
> Isolation of classes requires different Class instances. So isolation of
>  modules requires different ModuleDefinition instances.

Yes, that's the basic idea for isolation. In addition, the ModuleSystem
object is the one actually handling the module instances' instantiation,
initialization, and release, so the repository implementors do not have
to be aware of most of the module runtime's complexities.

- Stanley


From Stanley.Ho at sun.com  Wed Jun 13 11:24:24 2007
From: Stanley.Ho at sun.com (Stanley M. Ho)
Date: Wed, 13 Jun 2007 11:24:24 -0700
Subject: JSR 277 EG observer mailing list
In-Reply-To: <OFB42F1F41.14FA9781-ON802572F4.0056F42A-802572F4.0057BD3E@uk.ibm.com>
References: <OFB42F1F41.14FA9781-ON802572F4.0056F42A-802572F4.0057BD3E@uk.ibm.com>
Message-ID: <46703658.2020403@sun.com>

This is a resend.

Glyn Normington wrote:
>
> Hi Stanley
>
> Any chance of updating the public part of the JSR 277 page?

The content of the public page is for the original JSR proposal, and is
administrated and maintained by the JCP PMO office. I have contacted the
office and the answer is no. Anyway, I think mentioning the observer
list in the openjdk page + blogs + google should be sufficient for most
people.

- Stanley


From bryan.atsatt at oracle.com  Wed Jun 13 12:17:18 2007
From: bryan.atsatt at oracle.com (Bryan Atsatt)
Date: Wed, 13 Jun 2007 12:17:18 -0700
Subject: Module isolation
In-Reply-To: <46703604.5040203@sun.com>
References: <OFD20EE2C8.4BD4A1F7-ON802572ED.00360624-802572ED.00464DBD@uk.ibm.com>
	<466088D9.9020903@oracle.com> <46703604.5040203@sun.com>
Message-ID: <467042BE.3090307@oracle.com>

Stanley M. Ho wrote:
> Resend.
>
> Bryan Atsatt wrote:
>> ...
>> Stanley's idea was that a Repository implementation could return a
>> *copy* of a ModuleDefinition created by a different Repository. Such a
>> Repository would in fact form an isolation context. And caching Module
>> instances becomes trivial: each definition can just have one as a field.
>>
>> While there are definite lifecycle issues here between such repository
>> instances, they are probably solvable.
>>
>> And then we really *do* have an exact analogue of the loader delegation
>> model, in which:
>>
>>    Repository       ~= ClassLoader
>>    ModuleDefinition ~= Class
>>    Module           ~= Object
>>
>> Isolation of classes requires different Class instances. So isolation of
>>  modules requires different ModuleDefinition instances.
>
> Yes, that's the basic idea for isolation. In addition, the ModuleSystem
> object is the one actually handling the module instances' instantiation,
> initialization, and release, so the repository implementors do not have
> to be aware of most of the module runtime's complexities.

It seems to me that if we're going to follow this model, then:

1. There is a 1:1 relationship between ModuleDefinition instances and
Module instances.

2. Given #1, why does the getModuleInstance() method on ModuleDefinition
need to call through to the ModuleSystem? As it is, the ModuleSystem
must keep a separate map, which seems like overkill. The definition type
will be specific to the module system, so it can always instantiate the
correct type, and simply cache it in a field.

3. The ModuleDefinition copy operation and semantics must be defined. We
could simply implement Cloneable, or create a copy() operation. Either
way, it needs to be clear *what* is copied; I would think we would copy
all of the data *except* for the ModuleDefinitionContent.

>
> - Stanley
>


From Stanley.Ho at sun.com  Wed Jun 13 18:02:37 2007
From: Stanley.Ho at sun.com (Stanley M. Ho)
Date: Wed, 13 Jun 2007 18:02:37 -0700
Subject: Module isolation
In-Reply-To: <467081E3.60504@sun.com>
References: <OFD20EE2C8.4BD4A1F7-ON802572ED.00360624-802572ED.00464DBD@uk.ibm.com>
	<466088D9.9020903@oracle.com> <46703604.5040203@sun.com>
	<467042BE.3090307@oracle.com> <467081E3.60504@sun.com>
Message-ID: <467093AD.5020900@sun.com>

Resend.

Bryan Atsatt wrote:
 >
 > It seems to me that if we're going to follow this model, then:
 >
 > 1. There is a 1:1 relationship between ModuleDefinition instances and
 > Module instances.

ModuleSystem is the one actually handling the module instances'
instantiation, initialization, and release. Typically, instantiating a
Module from a ModuleDefinition through the ModuleSystem will return the
same cached Module instance for sharing, and it's 1:1. That said, the
ModuleSystem also has a releaseModule() method (see Section 7.1.4 in the
spec) that allows Module instance to be released from the cache when
necessary, and that Module instance will be GCed eventually. If a
ModuleDefinition is instantiated multiple times and each time
ModuleSystem.releaseModule() is also called for that ModuleDefinition,
it is possible to have multiple Module instances in memory for a given
ModuleDefinition, although calling getModuleInstance() method on
ModuleDefinition will only return the Module instance in the
ModuleSystem's cache.

 > 2. Given #1, why does the getModuleInstance() method on ModuleDefinition
 > need to call through to the ModuleSystem? As it is, the ModuleSystem
 > must keep a separate map, which seems like overkill. The definition type
 > will be specific to the module system, so it can always instantiate the
 > correct type, and simply cache it in a field.

That's because ModuleSystem does much more than instantiating a Module
instance from a ModuleDefinition. ModuleSystem is also responsible for
initializing the Module instance to interconnect the imported modules
(and resolve potential cyclic dependencies) and to perform type
consistency checking. To do module initialization properly, ModuleSystem
needs to be aware of all the cached Module instances (so they can be
reused for sharing) and all the Module instances that are in the process
of instantiation, initialization, and release.

The getModuleInstance() method on ModuleDefinition is simply a
convenience method to get to the Module instance from the ModuleSystem,
and ModuleDefinition does not keep track of any Module instance itself.
It is the ModuleSystem doing all the actual works.

 > 3. The ModuleDefinition copy operation and semantics must be defined. We
 > could simply implement Cloneable, or create a copy() operation. Either
 > way, it needs to be clear *what* is copied; I would think we would copy
 > all of the data *except* for the ModuleDefinitionContent.

It is unclear to me which use case you have in mind that requires
cloning of ModuleDefinition. At a high level, the information in
ModuleDefinition can be retrieved from ModuleDefinitionContent. If you
have a ModuleDefinitionContent instance, you could simply construct new
ModuleDefinition instances with it, and you don't need cloning.

- Stanley


From Stanley.Ho at Sun.COM  Wed Jun 13 16:11:38 2007
From: Stanley.Ho at Sun.COM (Stanley M. Ho)
Date: Wed, 13 Jun 2007 16:11:38 -0700
Subject: Module isolation
In-Reply-To: <467042BE.3090307@oracle.com>
References: <OFD20EE2C8.4BD4A1F7-ON802572ED.00360624-802572ED.00464DBD@uk.ibm.com>
	<466088D9.9020903@oracle.com> <46703604.5040203@sun.com>
	<467042BE.3090307@oracle.com>
Message-ID: <467079AA.6050805@sun.com>

Hi Bryan,

Bryan Atsatt wrote:
> ...
> It seems to me that if we're going to follow this model, then:
>
> 1. There is a 1:1 relationship between ModuleDefinition instances and
> Module instances.

ModuleSystem is the one actually handling the module instances'
instantiation, initialization, and release. Typically, instantiating a
Module from a ModuleDefinition through the ModuleSystem will return the
same cached Module instance for sharing, and it's 1:1. That said, the
ModuleSystem also has a releaseModule() method (see Section 7.1.4 in the
spec) that allows Module instance to be released from the cache when
necessary, and that Module instance will be GCed eventually. If a
ModuleDefinition is instantiated multiple times and each time
ModuleSystem.releaseModule() is also called for that ModuleDefinition,
it is possible to have multiple Module instances in memory for a given
ModuleDefinition, although calling getModuleInstance() method on
ModuleDefinition will only return the Module instance in the
ModuleSystem's cache.

> 2. Given #1, why does the getModuleInstance() method on ModuleDefinition
> need to call through to the ModuleSystem? As it is, the ModuleSystem
> must keep a separate map, which seems like overkill. The definition type
> will be specific to the module system, so it can always instantiate the
> correct type, and simply cache it in a field.

That's because ModuleSystem does much more than instantiating a Module
instance from a ModuleDefinition. ModuleSystem is also responsible for
initializing the Module instance to interconnect the imported modules
(and resolve potential cyclic dependencies) and to perform type
consistency checking. To do module initialization properly, ModuleSystem
needs to be aware of all the cached Module instances (so they can be
reused for sharing) and all the Module instances that are in the process
of instantiation, initialization, and release.

The getModuleInstance() method on ModuleDefinition is simply a
convenience method to get to the Module instance from the ModuleSystem,
and ModuleDefinition does not keep track of any Module instance itself.
It is the ModuleSystem doing all the actual works.

> 3. The ModuleDefinition copy operation and semantics must be defined. We
> could simply implement Cloneable, or create a copy() operation. Either
> way, it needs to be clear *what* is copied; I would think we would copy
> all of the data *except* for the ModuleDefinitionContent.

It is unclear to me which use case you have in mind that requires
cloning of ModuleDefinition. At a high level, the information in
ModuleDefinition can be retrieved from ModuleDefinitionContent. If you
have a ModuleDefinitionContent instance, you could simply construct new
ModuleDefinition instances with it, and you don't need cloning.

- Stanley


From glyn_normington at UK.IBM.COM  Thu Jun 14 03:07:06 2007
From: glyn_normington at UK.IBM.COM (Glyn Normington)
Date: Thu, 14 Jun 2007 11:07:06 +0100
Subject: Strawman: Services and service-providers support
In-Reply-To: <4670351E.40109@sun.com>
Message-ID: <OF0B6E6977.474612FD-ON802572FA.0035881E-802572FA.003794D4@uk.ibm.com>

"Stanley M. Ho" <Stanley.Ho at SUN.COM> wrote on 13/06/2007 07:19:10 PM:

> Since I have not heard any further input on the services and
> service-providers strawman, I suppose the EG is fine with the strawman
> overall except the issue raised by Richard. Unless I hear any objection,
> I will incorporate the appropriate portion of the strawman based on the
> feedback you have provided into the next revision of the specification.

I am very concerned that the scope of JSR 277 is being expanded
considerably without much attention being paid to the state of the art
(particularly Spring-OSGi and Declarative Services). If we could implement
good interoperation with JSR 291, we could delegate the complexities of
supporting services to JSR 291 and technologies like Spring-OSGi that
layer nicely on top of JSR 291.

Apart from that, the support for services in the strawman has some obvious
holes, so I don't think it is ready to be incorporated into the JSR 277
specification:

1. It seems to be lacking any form of dependency injection.

2. The namespace of services is global, but not partitioned by service
interface version. The effect of this is that a module could import v1 of
a service interface class and obtain an instance of the service that
implements v2 of the service interface and get a class cast exception.

3. There is no support for dynamic updates of service providers and
notification of service updates to service consumers. (This is consistent
with JSR 277's static nature, but I point it out as this is an obvious
future requirement based on our experience in OSGi.)

4. There seems to be some confusion in the strawman between loading of
service interfaces/implementations and construction and publication of
service instances.

I wonder what other Expert Group members think of this strawman. Silence
does not necessarily indicate happiness, so it would be good to have more
feedback.

Glyn





Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU





-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/attachments/20070614/eaa6ad16/attachment.html 

From Stanley.Ho at sun.com  Wed Jun 13 16:46:43 2007
From: Stanley.Ho at sun.com (Stanley M. Ho)
Date: Wed, 13 Jun 2007 16:46:43 -0700
Subject: Module isolation
In-Reply-To: <467042BE.3090307@oracle.com>
References: <OFD20EE2C8.4BD4A1F7-ON802572ED.00360624-802572ED.00464DBD@uk.ibm.com>
	<466088D9.9020903@oracle.com> <46703604.5040203@sun.com>
	<467042BE.3090307@oracle.com>
Message-ID: <467081E3.60504@sun.com>

Resend.

Bryan Atsatt wrote:
>
> It seems to me that if we're going to follow this model, then:
>
> 1. There is a 1:1 relationship between ModuleDefinition instances and
> Module instances.

ModuleSystem is the one actually handling the module instances'
instantiation, initialization, and release. Typically, instantiating a
Module from a ModuleDefinition through the ModuleSystem will return the
same cached Module instance for sharing, and it's 1:1. That said, the
ModuleSystem also has a releaseModule() method (see Section 7.1.4 in the
spec) that allows Module instance to be released from the cache when
necessary, and that Module instance will be GCed eventually. If a
ModuleDefinition is instantiated multiple times and each time
ModuleSystem.releaseModule() is also called for that ModuleDefinition,
it is possible to have multiple Module instances in memory for a given
ModuleDefinition, although calling getModuleInstance() method on
ModuleDefinition will only return the Module instance in the
ModuleSystem's cache.

> 2. Given #1, why does the getModuleInstance() method on ModuleDefinition
> need to call through to the ModuleSystem? As it is, the ModuleSystem
> must keep a separate map, which seems like overkill. The definition type
> will be specific to the module system, so it can always instantiate the
> correct type, and simply cache it in a field.

That's because ModuleSystem does much more than instantiating a Module
instance from a ModuleDefinition. ModuleSystem is also responsible for
initializing the Module instance to interconnect the imported modules
(and resolve potential cyclic dependencies) and to perform type
consistency checking. To do module initialization properly, ModuleSystem
needs to be aware of all the cached Module instances (so they can be
reused for sharing) and all the Module instances that are in the process
of instantiation, initialization, and release.

The getModuleInstance() method on ModuleDefinition is simply a
convenience method to get to the Module instance from the ModuleSystem,
and ModuleDefinition does not keep track of any Module instance itself.
It is the ModuleSystem doing all the actual works.

> 3. The ModuleDefinition copy operation and semantics must be defined. We
> could simply implement Cloneable, or create a copy() operation. Either
> way, it needs to be clear *what* is copied; I would think we would copy
> all of the data *except* for the ModuleDefinitionContent.

It is unclear to me which use case you have in mind that requires
cloning of ModuleDefinition. At a high level, the information in
ModuleDefinition can be retrieved from ModuleDefinitionContent. If you
have a ModuleDefinitionContent instance, you could simply construct new
ModuleDefinition instances with it, and you don't need cloning.

- Stanley


From andyp at bea.com  Thu Jun 14 04:43:45 2007
From: andyp at bea.com (Andy Piper)
Date: Thu, 14 Jun 2007 12:43:45 +0100
Subject: Strawman: Services and service-providers support
In-Reply-To: <OF0B6E6977.474612FD-ON802572FA.0035881E-802572FA.003794D4@uk.ibm.com>
References: <4670351E.40109@sun.com>
	<OF0B6E6977.474612FD-ON802572FA.0035881E-802572FA.003794D4@uk.ibm.com>
Message-ID: <6.2.5.6.2.20070614124314.028d9c28@bea.com>

I share Glyn's concerns, but can someone remind me where the strawman is?!

Thanks!

andy

At 11:07 AM 6/14/2007, Glyn Normington wrote:

>"Stanley M. Ho" <Stanley.Ho at SUN.COM> wrote on 13/06/2007 07:19:10 PM:
>
> > Since I have not heard any further input on the services and
> > service-providers strawman, I suppose the EG is fine with the strawman
> > overall except the issue raised by Richard. Unless I hear any objection,
> > I will incorporate the appropriate portion of the strawman based on the
> > feedback you have provided into the next revision of the specification.
>
>I am very concerned that the scope of JSR 277 is being expanded
>considerably without much attention being paid to the state of the
>art (particularly Spring-OSGi and Declarative Services). If we could
>implement good interoperation with JSR 291, we could delegate the
>complexities of supporting services to JSR 291 and technologies like
>Spring-OSGi that layer nicely on top of JSR 291.
>
>Apart from that, the support for services in the strawman has some
>obvious holes, so I don't think it is ready to be incorporated into
>the JSR 277 specification:
>
>1. It seems to be lacking any form of dependency injection.
>
>2. The namespace of services is global, but not partitioned by
>service interface version. The effect of this is that a module could
>import v1 of a service interface class and obtain an instance of the
>service that implements v2 of the service interface and get a class
>cast exception.
>
>3. There is no support for dynamic updates of service providers and
>notification of service updates to service consumers. (This is
>consistent with JSR 277's static nature, but I point it out as this
>is an obvious future requirement based on our experience in OSGi.)
>
>4. There seems to be some confusion in the strawman between loading
>of service interfaces/implementations and construction and
>publication of service instances.
>
>I wonder what other Expert Group members think of this strawman.
>Silence does not necessarily indicate happiness, so it would be good
>to have more feedback.
>
>Glyn
>
>
>
>----------
>
>
>Unless stated otherwise above:
>IBM United Kingdom Limited - Registered in England and Wales with
>number 741598.
>Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>
>
>
>
>



Notice:  This email message, together with any attachments, may contain information  of  BEA Systems,  Inc.,  its subsidiaries  and  affiliated entities,  that may be confidential,  proprietary,  copyrighted  and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by email and then delete it.


From glyn_normington at UK.IBM.COM  Thu Jun 14 05:51:36 2007
From: glyn_normington at UK.IBM.COM (Glyn Normington)
Date: Thu, 14 Jun 2007 13:51:36 +0100
Subject: Strawman: Services and service-providers support
In-Reply-To: <6.2.5.6.2.20070614124314.028d9c28@bea.com>
Message-ID: <OFD51BDD42.6E4C5EA0-ON802572FA.00468254-802572FA.0046A4AB@uk.ibm.com>

http://jcp.org/en/eg/download/jsr-277-service-provider-strawman-05222007.pdf?id=277&fileId=3438

Glyn

Andy Piper <andyp at BEA.COM> wrote on 14/06/2007 12:43:45 PM:

> I share Glyn's concerns, but can someone remind me where the strawman
is?!
>
> Thanks!
>
> andy
>
> At 11:07 AM 6/14/2007, Glyn Normington wrote:
>
> >"Stanley M. Ho" <Stanley.Ho at SUN.COM> wrote on 13/06/2007 07:19:10 PM:
> >
> > > Since I have not heard any further input on the services and
> > > service-providers strawman, I suppose the EG is fine with the
strawman
> > > overall except the issue raised by Richard. Unless I hear any
objection,
> > > I will incorporate the appropriate portion of the strawman based on
the
> > > feedback you have provided into the next revision of the
specification.
> >
> >I am very concerned that the scope of JSR 277 is being expanded
> >considerably without much attention being paid to the state of the
> >art (particularly Spring-OSGi and Declarative Services). If we could
> >implement good interoperation with JSR 291, we could delegate the
> >complexities of supporting services to JSR 291 and technologies like
> >Spring-OSGi that layer nicely on top of JSR 291.
> >
> >Apart from that, the support for services in the strawman has some
> >obvious holes, so I don't think it is ready to be incorporated into
> >the JSR 277 specification:
> >
> >1. It seems to be lacking any form of dependency injection.
> >
> >2. The namespace of services is global, but not partitioned by
> >service interface version. The effect of this is that a module could
> >import v1 of a service interface class and obtain an instance of the
> >service that implements v2 of the service interface and get a class
> >cast exception.
> >
> >3. There is no support for dynamic updates of service providers and
> >notification of service updates to service consumers. (This is
> >consistent with JSR 277's static nature, but I point it out as this
> >is an obvious future requirement based on our experience in OSGi.)
> >
> >4. There seems to be some confusion in the strawman between loading
> >of service interfaces/implementations and construction and
> >publication of service instances.
> >
> >I wonder what other Expert Group members think of this strawman.
> >Silence does not necessarily indicate happiness, so it would be good
> >to have more feedback.
> >
> >Glyn
> >
> >
> >
> >----------
> >
> >
> >Unless stated otherwise above:
> >IBM United Kingdom Limited - Registered in England and Wales with
> >number 741598.
> >Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU
> >
> >
> >
> >
> >
>
>
>
> Notice:  This email message, together with any attachments, may
> contain information  of  BEA Systems,  Inc.,  its subsidiaries  and
> affiliated entities,  that may be confidential,  proprietary,
> copyrighted  and/or legally privileged, and is intended solely for
> the use of the individual or entity named in this message. If you
> are not the intended recipient, and have received this message in
> error, please immediately return this by email and then delete it.






Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU





-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/attachments/20070614/6a67bd80/attachment.html 

From Stanley.Ho at Sun.COM  Thu Jun 14 09:53:58 2007
From: Stanley.Ho at Sun.COM (Stanley M. Ho)
Date: Thu, 14 Jun 2007 09:53:58 -0700
Subject: Strawman: Services and service-providers support
In-Reply-To: <OF0B6E6977.474612FD-ON802572FA.0035881E-802572FA.003794D4@uk.ibm.com>
References: <OF0B6E6977.474612FD-ON802572FA.0035881E-802572FA.003794D4@uk.ibm.com>
Message-ID: <467172A6.9000801@sun.com>

Hi Glyn,

The existing service/service-provider model has been used in various SE
components for loading available providers from JARs in the installed
optional packages and classpath. Since JSR 277 will define a set of
repositories that allow multiple versions of modules to coexist and
these modules may define services and/or service-providers, what the
strawman describes is basically how JSR 277 will fit into this existing
service/service-provider model and provide support for it.

While I agreed with you that enhancing the existing
service/service-provider model is desirable (like #1 and #2 you pointed
out), it is outside the scope of JSR 277 (i.e. JSR 277 is not chartered
to change the service/service-provider model.) Therefore, I suggest we
should focus our discussion on issues around how to support the existing
service/service-provider model.

#3 you mentioned is somewhat related to another topic I would like to
bring up, and I will start the discussion in another thread. For #4, it
is unclear to me what the issues are, could you elaborate?

Thanks,
- Stanley


Glyn Normington wrote:
>
> I am very concerned that the scope of JSR 277 is being expanded
> considerably without much attention being paid to the state of the art
> (particularly Spring-OSGi and Declarative Services). If we could
> implement good interoperation with JSR 291, we could delegate the
> complexities of supporting services to JSR 291 and technologies like
> Spring-OSGi that layer nicely on top of JSR 291.
>
> Apart from that, the support for services in the strawman has some
> obvious holes, so I don't think it is ready to be incorporated into the
> JSR 277 specification:
>
> 1. It seems to be lacking any form of dependency injection.
>
> 2. The namespace of services is global, but not partitioned by service
> interface version. The effect of this is that a module could import v1
> of a service interface class and obtain an instance of the service that
> implements v2 of the service interface and get a class cast exception.
>
> 3. There is no support for dynamic updates of service providers and
> notification of service updates to service consumers. (This is
> consistent with JSR 277's static nature, but I point it out as this is
> an obvious future requirement based on our experience in OSGi.)
>
> 4. There seems to be some confusion in the strawman between loading of
> service interfaces/implementations and construction and publication of
> service instances.
>
> I wonder what other Expert Group members think of this strawman. Silence
> does not necessarily indicate happiness, so it would be good to have
> more feedback.
>
> Glyn


From bryan.atsatt at oracle.com  Thu Jun 14 10:41:09 2007
From: bryan.atsatt at oracle.com (Bryan Atsatt)
Date: Thu, 14 Jun 2007 10:41:09 -0700
Subject: Module isolation
In-Reply-To: <467079AA.6050805@sun.com>
References: <OFD20EE2C8.4BD4A1F7-ON802572ED.00360624-802572ED.00464DBD@uk.ibm.com>
	<466088D9.9020903@oracle.com> <46703604.5040203@sun.com>
	<467042BE.3090307@oracle.com> <467079AA.6050805@sun.com>
Message-ID: <46717DB5.1080407@oracle.com>

Stanley M. Ho wrote:
> Hi Bryan,
>
> Bryan Atsatt wrote:
>> ...
>> It seems to me that if we're going to follow this model, then:
>>
>> 1. There is a 1:1 relationship between ModuleDefinition instances and
>> Module instances.
>
> ModuleSystem is the one actually handling the module instances'
> instantiation, initialization, and release. Typically, instantiating a
> Module from a ModuleDefinition through the ModuleSystem will return the
> same cached Module instance for sharing, and it's 1:1. That said, the
> ModuleSystem also has a releaseModule() method (see Section 7.1.4 in the
> spec) that allows Module instance to be released from the cache when
> necessary, and that Module instance will be GCed eventually. If a
> ModuleDefinition is instantiated multiple times and each time
> ModuleSystem.releaseModule() is also called for that ModuleDefinition,
> it is possible to have multiple Module instances in memory for a given
> ModuleDefinition, although calling getModuleInstance() method on
> ModuleDefinition will only return the Module instance in the
> ModuleSystem's cache.

Apparently I don't understand the use case here. Though releaseModule()
is exactly like the detach() method I put in the prototype, I realized
long ago that this pattern does not work well for isolation, as you
cannot control access to the original instances *or* the subsequent
instances--there is no "transaction" model.

A separate, private, Repository instance, OTOH, does provide such a model.

Can you please explain the use case for releaseModule()?

If there is no strong case for this, I would much prefer to stick with
the very simple 1:1 model.

>
>> 2. Given #1, why does the getModuleInstance() method on ModuleDefinition
>> need to call through to the ModuleSystem? As it is, the ModuleSystem
>> must keep a separate map, which seems like overkill. The definition type
>> will be specific to the module system, so it can always instantiate the
>> correct type, and simply cache it in a field.
>
> That's because ModuleSystem does much more than instantiating a Module
> instance from a ModuleDefinition. ModuleSystem is also responsible for
> initializing the Module instance to interconnect the imported modules
> (and resolve potential cyclic dependencies) and to perform type
> consistency checking. To do module initialization properly, ModuleSystem
> needs to be aware of all the cached Module instances (so they can be
> reused for sharing) and all the Module instances that are in the process
> of instantiation, initialization, and release.

Of course ModuleSystem must be involved, but that is orthogonal to
*where* the instance is cached.

Would you agree that if we did not support releaseModule(), caching the
Module instance in the ModuleDefinition would eliminate some complexity?

>
> The getModuleInstance() method on ModuleDefinition is simply a
> convenience method to get to the Module instance from the ModuleSystem,
> and ModuleDefinition does not keep track of any Module instance itself.
> It is the ModuleSystem doing all the actual works.
>
>> 3. The ModuleDefinition copy operation and semantics must be defined. We
>> could simply implement Cloneable, or create a copy() operation. Either
>> way, it needs to be clear *what* is copied; I would think we would copy
>> all of the data *except* for the ModuleDefinitionContent.
>
> It is unclear to me which use case you have in mind that requires
> cloning of ModuleDefinition. At a high level, the information in
> ModuleDefinition can be retrieved from ModuleDefinitionContent. If you
> have a ModuleDefinitionContent instance, you could simply construct new
> ModuleDefinition instances with it, and you don't need cloning.

Isolation requires creating a separate repository instance that wraps
another, and hands out *copies* of definitions from the underlying instance.

We simply need a standard way to make such a copy, one that does not
require knowing the actual subtype.

>
> - Stanley
>


From Stanley.Ho at sun.com  Thu Jun 14 17:13:41 2007
From: Stanley.Ho at sun.com (Stanley M. Ho)
Date: Thu, 14 Jun 2007 17:13:41 -0700
Subject: Module isolation
In-Reply-To: <46717DB5.1080407@oracle.com>
References: <OFD20EE2C8.4BD4A1F7-ON802572ED.00360624-802572ED.00464DBD@uk.ibm.com>
	<466088D9.9020903@oracle.com> <46703604.5040203@sun.com>
	<467042BE.3090307@oracle.com> <467079AA.6050805@sun.com>
	<46717DB5.1080407@oracle.com>
Message-ID: <4671D9B5.20909@sun.com>

Hi Bryan,

Bryan Atsatt wrote:
>
> Can you please explain the use case for releaseModule()?

If a repository instance is long-lived, the instantiated module
instances from the module definitions in the repository would be cached
for a long period of time; calling releaseModule() could help freeing up
resources when necessary, without requiring the repository instance to
be shutdown. This is also a use case from NetBeans.

Also, when a module definition is reloaded/uninstalled, or when the
repository is shutdown, we will disable the module definitions in the
repository from the module system so any existing module instance would
be released and no new module instance can be instantiated. However,
releasing the module instance alone is not sufficient because that
module instance may be referenced by some importers. Hence, we want to
call releaseModule() on the importers as well for these module instances
to eventually become unreachable and be GCed.

> Would you agree that if we did not support releaseModule(), caching the
> Module instance in the ModuleDefinition would eliminate some complexity?

No. I actually think caching the Module instance in the ModuleDefinition
would add complexity to the design and the implementation. If the module
instances are cached in the ModuleSystem, the ModuleSystem could fully
control the access of these module instances and update/change them
internally. ModuleDefinition is supposed to be stateless, and I think it
should be kept this way.

> Isolation requires creating a separate repository instance that wraps
> another, and hands out *copies* of definitions from the underlying
> instance.
>
> We simply need a standard way to make such a copy, one that does not
> require knowing the actual subtype.

If I understand what you are trying to do, you want to construct a new
repository instance with module definitions that are based on some
module definitions in an existing repository instance. To do that, we
could either go with the approach of constructing new ModuleDefinition,
or as you suggested - cloning. However, I don't think cloning is the
right general solution here - there are many module definitions that are
unsuitable for cloning (e.g. platform module definitions). My assumption
is that the developer creating the separate repository instance is
possibly the one creating the original repository instance, so I don't
understand why he can't simply construct a new ModuleDefinition with an
existing ModuleDefinitionContent.

- Stanley


From bryan.atsatt at oracle.com  Thu Jun 14 18:57:42 2007
From: bryan.atsatt at oracle.com (Bryan Atsatt)
Date: Thu, 14 Jun 2007 18:57:42 -0700
Subject: Module isolation
In-Reply-To: <4671D9B5.20909@sun.com>
References: <OFD20EE2C8.4BD4A1F7-ON802572ED.00360624-802572ED.00464DBD@uk.ibm.com>
	<466088D9.9020903@oracle.com> <46703604.5040203@sun.com>
	<467042BE.3090307@oracle.com> <467079AA.6050805@sun.com>
	<46717DB5.1080407@oracle.com> <4671D9B5.20909@sun.com>
Message-ID: <4671F216.3050506@oracle.com>

Stanley M. Ho wrote:
> Hi Bryan,
>
> Bryan Atsatt wrote:
>>
>> Can you please explain the use case for releaseModule()?
>
> If a repository instance is long-lived, the instantiated module
> instances from the module definitions in the repository would be cached
> for a long period of time; calling releaseModule() could help freeing up
> resources when necessary, without requiring the repository instance to
> be shutdown. This is also a use case from NetBeans.

Sorry, but this is just a bit fuzzy to me. I suspect that a concrete use
case is an application server, when it has stopped an application,
*could* release any modules used by that application.

But this approach doesn't really work very well. What happens when,
moments after releasing a module, another application is deployed that
needs the same module? It gets a different instance. And that could
easily lead to ClassCastExceptions and/or LinkageErrors.

How is it possible to know when it is safe to "free up resources"?

I don't think you can have it both ways. Either classes are shared for
the lifetime of the process, or they are shared *only* within a well
bounded scope. Not both.

To me, this is exactly why we need an isolation model: to provide a well
defined scope whose lifecycle can be correctly managed.


>
> Also, when a module definition is reloaded/uninstalled, or when the
> repository is shutdown, we will disable the module definitions in the
> repository from the module system so any existing module instance would
> be released and no new module instance can be instantiated. However,
> releasing the module instance alone is not sufficient because that
> module instance may be referenced by some importers. Hence, we want to
> call releaseModule() on the importers as well for these module instances
> to eventually become unreachable and be GCed.
>
>> Would you agree that if we did not support releaseModule(), caching the
>> Module instance in the ModuleDefinition would eliminate some complexity?
>
> No. I actually think caching the Module instance in the ModuleDefinition
> would add complexity to the design and the implementation. If the module
> instances are cached in the ModuleSystem, the ModuleSystem could fully
> control the access of these module instances and update/change them
> internally. ModuleDefinition is supposed to be stateless, and I think it
> should be kept this way.

Well, I agree in theory. But... I am struggling to understand how we
provide an isolation model. If:

- There is a 1:1 relation for ModuleDefinition<->Module instance, and
- Isolation requires separate Module instances, then
- Isolation requires separate ModuleDefinition instances

If this is our isolation model, then how does a ModuleSystem instance
support this? Clearly, it would need to keep a mapping from each
ModuleDefinition to its Module instance. How is this simpler, or better?

In your current model (just as in my detach() model), there is still a
1:1 from *at any given moment*, at least from the perspective of the
definition.

Are you thinking that the ModuleSystem would have to keep track of
released modules?

>
>> Isolation requires creating a separate repository instance that wraps
>> another, and hands out *copies* of definitions from the underlying
>> instance.
>>
>> We simply need a standard way to make such a copy, one that does not
>> require knowing the actual subtype.
>
> If I understand what you are trying to do, you want to construct a new
> repository instance with module definitions that are based on some
> module definitions in an existing repository instance. To do that, we
> could either go with the approach of constructing new ModuleDefinition,
> or as you suggested - cloning. However, I don't think cloning is the
> right general solution here - there are many module definitions that are
> unsuitable for cloning (e.g. platform module definitions). My assumption
> is that the developer creating the separate repository instance is
> possibly the one creating the original repository instance, so I don't
> understand why he can't simply construct a new ModuleDefinition with an
> existing ModuleDefinitionContent.

But this approach means that the wrapper must be tightly coupled to the
wrappee. While I can imagine this holding true in some cases, it
certainly doesn't seem like the common case.

Why should an applet container, for example, be required to know the
*type* of ModuleDefinition subclasses in the repository?

Providing some sort of copy() operation on the base class eliminates
this kind of coupling.

>
> - Stanley
>


From glyn_normington at UK.IBM.COM  Fri Jun 15 03:07:38 2007
From: glyn_normington at UK.IBM.COM (Glyn Normington)
Date: Fri, 15 Jun 2007 11:07:38 +0100
Subject: Strawman: Services and service-providers support
In-Reply-To: <467172A6.9000801@sun.com>
Message-ID: <OF7EE8721A.05DA767D-ON802572FB.003119D4-802572FB.0037A18F@uk.ibm.com>

Hi Stanley

"Stanley M. Ho" <Stanley.Ho at SUN.COM> wrote on 14/06/2007 05:53:58 PM:

> Hi Glyn,
>
> The existing service/service-provider model has been used in various SE
> components for loading available providers from JARs in the installed
> optional packages and classpath. Since JSR 277 will define a set of
> repositories that allow multiple versions of modules to coexist and
> these modules may define services and/or service-providers, what the
> strawman describes is basically how JSR 277 will fit into this existing
> service/service-provider model and provide support for it.
>
> While I agreed with you that enhancing the existing
> service/service-provider model is desirable (like #1 and #2 you pointed
> out), it is outside the scope of JSR 277 (i.e. JSR 277 is not chartered
> to change the service/service-provider model.) Therefore, I suggest we
> should focus our discussion on issues around how to support the existing
> service/service-provider model.

Fine.

>
> #3 you mentioned is somewhat related to another topic I would like to
> bring up, and I will start the discussion in another thread. For #4, it
> is unclear to me what the issues are, could you elaborate?

Maybe it's just a matter of presentation in the strawman, but it confused
me somewhat and the net effect was that I wasn't sure what pieces JSR 277
will be providing and what pieces are upgrades of existing facilities
(since I'm not familiar with the existing service/service-provider model)
to take advantage of JSR 277.

1. The opening paragraph of the Background section defines a service as "a
well-known set of interfaces and (usually abstract) classes" - no mention
of instances,

2. The third paragraph of the same section say "a service is represented
by a single type ...". Again no mention of instances.

3. The requirements section talks about the module system making services
available to the Java platform. This is using 'service' in the above
sense, but I'm not sure what it means to make an interface "available to
the Java platform" unless it means "available for modules to import".

4. Section 6 talks about an application module importing a service module
if and only if there is at least one suitable service provider. I guess
the rationale is to make the application module fail to resolve if it
hasn't a (current) hope of obtaining the service it needs. But from my
perspective this is odd. The service provider could be available without
it being possible to obtain a service instance, which is the thing the
application really needs. So rather than being an import-time check, the
availability of critical services should be checked when the application
module starts (for static applications that don't need to react
dynamically to services coming online, that is).

Glyn

>
> Thanks,
> - Stanley
>
>
> Glyn Normington wrote:
> >
> > I am very concerned that the scope of JSR 277 is being expanded
> > considerably without much attention being paid to the state of the art
> > (particularly Spring-OSGi and Declarative Services). If we could
> > implement good interoperation with JSR 291, we could delegate the
> > complexities of supporting services to JSR 291 and technologies like
> > Spring-OSGi that layer nicely on top of JSR 291.
> >
> > Apart from that, the support for services in the strawman has some
> > obvious holes, so I don't think it is ready to be incorporated into
the
> > JSR 277 specification:
> >
> > 1. It seems to be lacking any form of dependency injection.
> >
> > 2. The namespace of services is global, but not partitioned by service
> > interface version. The effect of this is that a module could import v1
> > of a service interface class and obtain an instance of the service
that
> > implements v2 of the service interface and get a class cast exception.
> >
> > 3. There is no support for dynamic updates of service providers and
> > notification of service updates to service consumers. (This is
> > consistent with JSR 277's static nature, but I point it out as this is
> > an obvious future requirement based on our experience in OSGi.)
> >
> > 4. There seems to be some confusion in the strawman between loading of
> > service interfaces/implementations and construction and publication of
> > service instances.
> >
> > I wonder what other Expert Group members think of this strawman.
Silence
> > does not necessarily indicate happiness, so it would be good to have
> > more feedback.
> >
> > Glyn





Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU





-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/attachments/20070615/f83a31d0/attachment.html 

From bryan.atsatt at oracle.com  Fri Jun 15 10:11:34 2007
From: bryan.atsatt at oracle.com (Bryan Atsatt)
Date: Fri, 15 Jun 2007 10:11:34 -0700
Subject: Strawman: Services and service-providers support
In-Reply-To: <OF0B6E6977.474612FD-ON802572FA.0035881E-802572FA.003794D4@uk.ibm.com>
References: <OF0B6E6977.474612FD-ON802572FA.0035881E-802572FA.003794D4@uk.ibm.com>
Message-ID: <4672C846.1020104@oracle.com>

While I agree that the service loader framework appears to be lacking
(very) important features, if it is going to be part of the SE APIs then
it seems reasonable to support it.

The ServiceLoader's polling model is OK as far as it goes, but I much
prefer the "activator" model, where a service module could announce the
availability of services during module initialization.

// Bryan

Glyn Normington wrote:
>
> *"Stanley M. Ho" <Stanley.Ho at SUN.COM>* wrote on 13/06/2007 07:19:10 PM:
>
>  > Since I have not heard any further input on the services and
>  > service-providers strawman, I suppose the EG is fine with the strawman
>  > overall except the issue raised by Richard. Unless I hear any objection,
>  > I will incorporate the appropriate portion of the strawman based on the
>  > feedback you have provided into the next revision of the specification.
>
> I am very concerned that the scope of JSR 277 is being expanded
> considerably without much attention being paid to the state of the art
> (particularly Spring-OSGi and Declarative Services). If we could
> implement good interoperation with JSR 291, we could delegate the
> complexities of supporting services to JSR 291 and technologies like
> Spring-OSGi that layer nicely on top of JSR 291.
>
> Apart from that, the support for services in the strawman has some
> obvious holes, so I don't think it is ready to be incorporated into the
> JSR 277 specification:
>
> 1. It seems to be lacking any form of dependency injection.
>
> 2. The namespace of services is global, but not partitioned by service
> interface version. The effect of this is that a module could import v1
> of a service interface class and obtain an instance of the service that
> implements v2 of the service interface and get a class cast exception.
>
> 3. There is no support for dynamic updates of service providers and
> notification of service updates to service consumers. (This is
> consistent with JSR 277's static nature, but I point it out as this is
> an obvious future requirement based on our experience in OSGi.)
>
> 4. There seems to be some confusion in the strawman between loading of
> service interfaces/implementations and construction and publication of
> service instances.
>
> I wonder what other Expert Group members think of this strawman. Silence
> does not necessarily indicate happiness, so it would be good to have
> more feedback.
>
> Glyn
>
>
> ------------------------------------------------------------------------
>
> /
> /
>
> /Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU/
>
>
>
>
>
>


From Stanley.Ho at Sun.COM  Fri Jun 15 17:19:58 2007
From: Stanley.Ho at Sun.COM (Stanley M. Ho)
Date: Fri, 15 Jun 2007 17:19:58 -0700
Subject: Module isolation
In-Reply-To: <4671F216.3050506@oracle.com>
References: <OFD20EE2C8.4BD4A1F7-ON802572ED.00360624-802572ED.00464DBD@uk.ibm.com>
	<466088D9.9020903@oracle.com> <46703604.5040203@sun.com>
	<467042BE.3090307@oracle.com> <467079AA.6050805@sun.com>
	<46717DB5.1080407@oracle.com> <4671D9B5.20909@sun.com>
	<4671F216.3050506@oracle.com>
Message-ID: <46732CAE.6070207@sun.com>

Hello Bryan,

Bryan Atsatt wrote:
> ...
> But this approach doesn't really work very well. What happens when,
> moments after releasing a module, another application is deployed that
> needs the same module? It gets a different instance. And that could
> easily lead to ClassCastExceptions and/or LinkageErrors.
>
> How is it possible to know when it is safe to "free up resources"?

The use case I have is that sometimes we might want to make a module
temporary unavailable (e.g. turning off a plugin in the IDE), without
shutting down the repository (possibly with hundreds/thousands of other
modules) or uninstall the module. In this case, the container will
trigger not only the release of the existing module instance (so it will
have a chance to be GCed eventually), but it will also make the module
definition invisible (through visibility policy) from other modules.
Without the second part, it has the potential problem you described.

Note that I don't think this is a common thing many developers want to
do. In fact, I think we should discourage most developers from calling
releaseModule() because of the potential consequences. On the other
hand, we shouldn't preclude this use case either. If you have better
suggestion to address this use case, I would like to hear it.

> Well, I agree in theory. But... I am struggling to understand how we
> provide an isolation model. If:
>
> - There is a 1:1 relation for ModuleDefinition<->Module instance, and
> - Isolation requires separate Module instances, then
> - Isolation requires separate ModuleDefinition instances
>
> If this is our isolation model, then how does a ModuleSystem instance
> support this? Clearly, it would need to keep a mapping from each
> ModuleDefinition to its Module instance. How is this simpler, or better?
>
> In your current model (just as in my detach() model), there is still a
> 1:1 from *at any given moment*, at least from the perspective of the
> definition.
>
> Are you thinking that the ModuleSystem would have to keep track of
> released modules?

The ModuleSystem instance would have a <ModuleDefinition, Module>
mapping, and this should be a very simple thing to support and maintain.
Also, the ModuleSystem needs to maintain this information anyway to
avoid multiple Module instances to be instantiated for a given
ModuleDefinition, or to avoid a Module instance to be instantiated if a
ModuleDefinition has been uninstalled or belongs to a repository which
has been shutdown. And yes, the ModuleSystem would have another map to
keep track of all the ModuleDefinitions that are no longer
usable/instantiated. Having the information centralized in one place has
other benefits too, e.g. if we want to find out what the outstanding
module instances from module definitions in all repositories, the
ModuleSystem can provide the answer easily.

My view is that ModuleSystem needs to keep track of various runtime
information related to ModuleDefinition and Module anyway, so I don't
see clear benefit in moving part of that information into other classes.

> But this approach means that the wrapper must be tightly coupled to the
> wrappee. While I can imagine this holding true in some cases, it
> certainly doesn't seem like the common case.
>
> Why should an applet container, for example, be required to know the
> *type* of ModuleDefinition subclasses in the repository?
>
> Providing some sort of copy() operation on the base class eliminates
> this kind of coupling.

Perhaps I still don't fully understand how you will want to create a new
repository for isolation, and why having the knowledge of the
ModuleDefinition subclasses is not acceptable in this context.

As I previously hinted, having some sort of copy() operation in the base
class is not feasible because many module definitions does not make
sense to be cloned. Cloning also implies that the underlying lifetime of
the ModuleDefinition (and the ModuleDefinitionContent) from two
different repositories could be arbitrarily tied, and I don't think it
makes sense unless the repositories have the same owner.

In the case of an applet container, my expectation is that the container
will construct some kind of AppletURLRespository for each codebase with
some ModuleDefinition subclass, and each ModuleDefinition is
instantiated with a custom ModuleDefinitionContent implementation with
the applet cache as the backing store. In other words, the applet
container or the AppletURLRepository already has knowledge about the
ModuleDefinition subclass. If the applet container needs a new
repository for isolation, then it would construct another
AppletURLRespository, and this new AppletURLRespository could construct
each ModuleDefinition using the existing ModuleDefinitionContent instance.

- Stanley


From bryan.atsatt at oracle.com  Mon Jun 18 14:09:13 2007
From: bryan.atsatt at oracle.com (Bryan Atsatt)
Date: Mon, 18 Jun 2007 14:09:13 -0700
Subject: Module isolation
In-Reply-To: <46732CAE.6070207@sun.com>
References: <OFD20EE2C8.4BD4A1F7-ON802572ED.00360624-802572ED.00464DBD@uk.ibm.com>
	<466088D9.9020903@oracle.com> <46703604.5040203@sun.com>
	<467042BE.3090307@oracle.com> <467079AA.6050805@sun.com>
	<46717DB5.1080407@oracle.com> <4671D9B5.20909@sun.com>
	<4671F216.3050506@oracle.com> <46732CAE.6070207@sun.com>
Message-ID: <4676F479.9070400@oracle.com>

Stanley M. Ho wrote:
> Hello Bryan,
>
> Bryan Atsatt wrote:
>> ...
>> But this approach doesn't really work very well. What happens when,
>> moments after releasing a module, another application is deployed that
>> needs the same module? It gets a different instance. And that could
>> easily lead to ClassCastExceptions and/or LinkageErrors.
>>
>> How is it possible to know when it is safe to "free up resources"?
>
> The use case I have is that sometimes we might want to make a module
> temporary unavailable (e.g. turning off a plugin in the IDE), without
> shutting down the repository (possibly with hundreds/thousands of other
> modules) or uninstall the module. In this case, the container will
> trigger not only the release of the existing module instance (so it will
> have a chance to be GCed eventually), but it will also make the module
> definition invisible (through visibility policy) from other modules.
> Without the second part, it has the potential problem you described.

This use case seems to presume that the IDE can/will ensure that there
is only *one* consumer of the module: itself. What if a different plugin
has a dependency on it, and has already been resolved? Is the intention
that the IDE will be wired into the module system deeply enough to
manage this correctly?

Again, this is why we need either a real, general, isolation model, or
an access control model:

Containers, of any kind, must be able to explicitly control access to
"private" modules.


An application server, for example, will need to keep one application
from accessing the modules of another. And it must *know* that there is
no sharing, so that the application lifecycles can remain independent.

So we need some notion of a context. A purely private repository
instance is one (probably good) possibility. Another is the wrapper
Repository approach, but this requires definition copies (and management
of sharing content, lifecycle, etc).

I started this thread with another, explicit type (ModuleContext), but
it isn't obvious how to use such beast correctly.

An access control model would also work, where all parties can share
Repository instances, but we somehow discriminate between *callers* to
return different results.

We may even want to support some mixture of private + access control.

Regardless of what approach we take, the releaseModule() idea is too
simplistic. Having originally created the detach() method, thinking
along similar lines as you are with the plugin case, I do understand the
idea; I just no longer think it is sufficient :^).

The only "safe" time to release a module is when there are *zero*
importers, and, even then, you must hide/release atomically, ensuring
that no new imports spring up during the operation.

>
> Note that I don't think this is a common thing many developers want to
> do. In fact, I think we should discourage most developers from calling
> releaseModule() because of the potential consequences. On the other
> hand, we shouldn't preclude this use case either. If you have better
> suggestion to address this use case, I would like to hear it.
>
>> Well, I agree in theory. But... I am struggling to understand how we
>> provide an isolation model. If:
>>
>> - There is a 1:1 relation for ModuleDefinition<->Module instance, and
>> - Isolation requires separate Module instances, then
>> - Isolation requires separate ModuleDefinition instances
>>
>> If this is our isolation model, then how does a ModuleSystem instance
>> support this? Clearly, it would need to keep a mapping from each
>> ModuleDefinition to its Module instance. How is this simpler, or better?
>>
>> In your current model (just as in my detach() model), there is still a
>> 1:1 from *at any given moment*, at least from the perspective of the
>> definition.
>>
>> Are you thinking that the ModuleSystem would have to keep track of
>> released modules?
>
> The ModuleSystem instance would have a <ModuleDefinition, Module>
> mapping, and this should be a very simple thing to support and maintain.
> Also, the ModuleSystem needs to maintain this information anyway to
> avoid multiple Module instances to be instantiated for a given
> ModuleDefinition, or to avoid a Module instance to be instantiated if a
> ModuleDefinition has been uninstalled or belongs to a repository which
> has been shutdown. And yes, the ModuleSystem would have another map to
> keep track of all the ModuleDefinitions that are no longer
> usable/instantiated. Having the information centralized in one place has
> other benefits too, e.g. if we want to find out what the outstanding
> module instances from module definitions in all repositories, the
> ModuleSystem can provide the answer easily.

Sure. (And the ModuleSystem could not use strong references to hold
released Module instances, else it would prevent GC.)

> My view is that ModuleSystem needs to keep track of various runtime
> information related to ModuleDefinition and Module anyway, so I don't
> see clear benefit in moving part of that information into other classes.

Other than Module instances, what other "runtime information" is there
to keep track of? Caches of exported packages?

Understand here that I am *not* focused on the idea of caching Module
instances on ModuleDefinition; I do understand the desire to avoid
polluting a stateless type.

I am trying to figure out if we *could* do so, as a means of pinning
down precisely what our model is.

For example, if we were to eliminate the releaseModule() method (in
favor of some more complete mechanism), then there really is always a
1:1 for Module:ModuleDefinition, and the model is simple and obvious.
(And therefore a field cache *could* be used).

>> But this approach means that the wrapper must be tightly coupled to the
>> wrappee. While I can imagine this holding true in some cases, it
>> certainly doesn't seem like the common case.
>>
>> Why should an applet container, for example, be required to know the
>> *type* of ModuleDefinition subclasses in the repository?
>>
>> Providing some sort of copy() operation on the base class eliminates
>> this kind of coupling.
>
> Perhaps I still don't fully understand how you will want to create a new
> repository for isolation, and why having the knowledge of the
> ModuleDefinition subclasses is not acceptable in this context.
>
> As I previously hinted, having some sort of copy() operation in the base
> class is not feasible because many module definitions does not make
> sense to be cloned. Cloning also implies that the underlying lifetime of
> the ModuleDefinition (and the ModuleDefinitionContent) from two
> different repositories could be arbitrarily tied, and I don't think it
> makes sense unless the repositories have the same owner.

I completely agree that cloning raises concurrency and lifecycle issues.
But these don't go away just because you know the actual type of a
ModuleDefinition!

I am not in any way wedded to the copy idea. I am just trying to find
*some* solution that enables private Module instances; the copy idea was
actually yours :^).

> In the case of an applet container, my expectation is that the container
> will construct some kind of AppletURLRespository for each codebase with
> some ModuleDefinition subclass, and each ModuleDefinition is
> instantiated with a custom ModuleDefinitionContent implementation with
> the applet cache as the backing store. In other words, the applet
> container or the AppletURLRepository already has knowledge about the
> ModuleDefinition subclass. If the applet container needs a new
> repository for isolation, then it would construct another
> AppletURLRespository, and this new AppletURLRespository could construct
> each ModuleDefinition using the existing ModuleDefinitionContent instance.

So, in effect, we have 100% *private repository* instances.

I've been thinking that we need an intermediate somewhere between a
shared/public repo instance and an entirely private one, but... that now
strikes me as too fuzzy, and I can't see a real use case :^)

So an application server would have to create, say, a private
LocalRepository instance to hold the modules of a single application.
And it would have to ensure that no other application could get it's
grubby paws on that repository instance.

Ok. That works for me. And it eliminates the need for cloning AND for
releaseModule():

1. Any given Repository instance is either 100% shared or 100% private,
with *no* in-between.

2. The lifecycle of a shared repository instance is that of the process.

3. The lifecycle of a private repository instance is entirely up to the
creator of that instance.

4. The lifecycle of a ModuleDefinition/Module is at most that of the
enclosing Repository instance, and at least is bounded by
install/uninstall (no finer granularity).


This seems like a clean simple model: private Modules via private
Repositories.

And a private repository can work for the Applet, IDE plugin or EE
application cases just fine.

It does leave open the issue of dependencies *within* a private
repository. The simple model would be to treat the entire repository as
atomic, with any change requiring a new Repository instance. This is
probably too simplistic, however.

In an EE app, web-modules are supposed to be isolated from each other
and from other parts of the app (ejb, connectors, etc.). So this
requires either a further partitioning of the app into multiple
repositories, or some form of access control.

Further, it is possible to re-start or re-deploy/re-start only a single
web-module, *without* restarting the rest of the app. The re-start case
could use releaseModule(), though a "real" stop method would be
preferable. But this is a very special case in which the specs
essentially dictate the possible dependencies between modules. In the
general case where the dependencies are not dictated, releaseModule() is
problematic.

The re-deploy case would use uninstall/install (but would still like stop!).


If you *really* want the releaseModule() functionality, I would suggest
that we introduce a PrivateRepository type, and support release *only*
on that type.

// Bryan

>
> - Stanley
>


From abrock at REDHAT.COM  Tue Jun 19 05:29:37 2007
From: abrock at REDHAT.COM (Adrian)
Date: Tue, 19 Jun 2007 14:29:37 +0200
Subject: Application Isolation in App Servers was Re: Module isolation
In-Reply-To: <4676F479.9070400@oracle.com>
References: <OFD20EE2C8.4BD4A1F7-ON802572ED.00360624-802572ED.00464DBD@uk.ibm.com>
	<466088D9.9020903@oracle.com> <46703604.5040203@sun.com>
	<467042BE.3090307@oracle.com> <467079AA.6050805@sun.com>
	<46717DB5.1080407@oracle.com> <4671D9B5.20909@sun.com>
	<4671F216.3050506@oracle.com> <46732CAE.6070207@sun.com>
	<4676F479.9070400@oracle.com>
Message-ID: <1182256177.3510.21.camel@warjort>

On Mon, 2007-06-18 at 14:09 -0700, Bryan Atsatt wrote:
> So an application server would have to create, say, a private
> LocalRepository instance to hold the modules of a single application.
> And it would have to ensure that no other application could get it's
> grubby paws on that repository instance.
>
> Ok. That works for me. And it eliminates the need for cloning AND for
> releaseModule():
>
> 1. Any given Repository instance is either 100% shared or 100% private,
> with *no* in-between.

Since we are talking about appservers, I'll just point out
that this doesn't cover what JBoss allows you to do.

In JBoss we have a notion of what I like to call "ClassLoader Domains"
although they are actually called "LoaderRepositorys" after
the JMX name (note the JMX spec only has a singleton repository).

These are "isolated" classloading systems that sit on top
of a "Default Domain" and you can choose whether to do
parent/child delegation in either
* standard J2SE order (parent first)
* servlet style (child first).

When you deploy an application you get to choose the domain.
There is nothing to stop you putting multiple applications
into the same domain.

I would guess in your proposal they WOULD share the
same LocalRepository but only those named applications
would have access to it?

There are two main usecases for such a configuration.

1)  Optimization
You want to optimize the inter-application communication
between related applications. i.e. one application uses
components from another application and you don't want
to do call-by-value.
By having the applications in the same classloader domain
one application can lookup and use the local interfaces
of the other application.

2) Convenience
You deploy a rar inside an ear (making the rar classes
invisible to other applications)
but you want to configure the connection factories
of that rar outside the ear where it is easier to
edit the configuration file.

In this case you can specify that the connection factories
should use the classloading domain of the ear, making
its classes visible again.

e.g. something like:

<connection-factories>

<loader-repository>classloader:application=myapp.ear</loader-repository>

<etc/>

--
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Adrian Brock
Chief Scientist
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx


From bryan.atsatt at oracle.com  Tue Jun 19 12:17:11 2007
From: bryan.atsatt at oracle.com (Bryan Atsatt)
Date: Tue, 19 Jun 2007 12:17:11 -0700
Subject: Application Isolation in App Servers was Re: Module isolation
In-Reply-To: <1182256177.3510.21.camel@warjort>
References: <OFD20EE2C8.4BD4A1F7-ON802572ED.00360624-802572ED.00464DBD@uk.ibm.com>
	<466088D9.9020903@oracle.com> <46703604.5040203@sun.com>
	<467042BE.3090307@oracle.com> <467079AA.6050805@sun.com>
	<46717DB5.1080407@oracle.com> <4671D9B5.20909@sun.com>
	<4671F216.3050506@oracle.com> <46732CAE.6070207@sun.com>
	<4676F479.9070400@oracle.com> <1182256177.3510.21.camel@warjort>
Message-ID: <46782BB7.7020501@oracle.com>

Adrian wrote:
> On Mon, 2007-06-18 at 14:09 -0700, Bryan Atsatt wrote:
>> So an application server would have to create, say, a private
>> LocalRepository instance to hold the modules of a single application.
>> And it would have to ensure that no other application could get it's
>> grubby paws on that repository instance.
>>
>> Ok. That works for me. And it eliminates the need for cloning AND for
>> releaseModule():
>>
>> 1. Any given Repository instance is either 100% shared or 100% private,
>> with *no* in-between.
>
> Since we are talking about appservers, I'll just point out
> that this doesn't cover what JBoss allows you to do.
>
> In JBoss we have a notion of what I like to call "ClassLoader Domains"
> although they are actually called "LoaderRepositorys" after
> the JMX name (note the JMX spec only has a singleton repository).
>
> These are "isolated" classloading systems that sit on top
> of a "Default Domain" and you can choose whether to do
> parent/child delegation in either
> * standard J2SE order (parent first)
> * servlet style (child first).
>
> When you deploy an application you get to choose the domain.
> There is nothing to stop you putting multiple applications
> into the same domain.
>
> I would guess in your proposal they WOULD share the
> same LocalRepository but only those named applications
> would have access to it?

Right.

>
> There are two main usecases for such a configuration.
>
> 1)  Optimization
> You want to optimize the inter-application communication
> between related applications. i.e. one application uses
> components from another application and you don't want
> to do call-by-value.
> By having the applications in the same classloader domain
> one application can lookup and use the local interfaces
> of the other application.

Yep.

>
> 2) Convenience
> You deploy a rar inside an ear (making the rar classes
> invisible to other applications)
> but you want to configure the connection factories
> of that rar outside the ear where it is easier to
> edit the configuration file.
>
> In this case you can specify that the connection factories
> should use the classloading domain of the ear, making
> its classes visible again.
>
> e.g. something like:
>
> <connection-factories>
>
> <loader-repository>classloader:application=myapp.ear</loader-repository>
>
> <etc/>

This last example doesn't really apply in our server, as the config is
just as easy to edit inside as outside an ear. But I see your point.


Most apps don't need to share classes in this way, and need only a
mixture of global and purely local classes. Clearly any module system
will be able to satisfy the first requirement. We need to make sure we
can also satisfy the second.

But you are right, of course, that there exist more complex scenarios
that require an intermediate form of sharing. Your named domains seems
to satisfy that requirement. Can you restrict access to domains?

And, as you know, even *within* an application, web-modules need to be
isolated. So the private repository approach breaks down a bit here, or
must be taken to the extreme of a single module repository instance.
This is where an access control model might be a better solution.


Our server supports three models for class sharing:

1. Global. All apps share a single version of the classes. This is a
deprecated model.

2. Shared. Our "Shared Libraries" are named/versioned loaders configured
at the server level, each with one or more code-sources. Each can
optionally import other shared-libraries. Import declarations are by
name and version range. A default set of imports is configured by the
container for all application instances, but each is app free to modify
that set via config.

(This is obviously very much like the 277/OSGi model.)

3. Application group. Applications are hierarchical: each app is
deployed under some parent, and can access the classes of that parent
(excluding web-modules). So several apps can be deployed under a common
parent, and classes visible at the parent are visible to the children.
The global model in #1 simply means deploying classes into our "default"
application.


The shared-library approach has proved to be the most flexible and most
used (surprise). But this model does not support any access control, so
at the moment, if you want restricted sharing you must use the shared
parent approach.

And while we do support execution under a SecurityManager, *very* few
companies are willing to pay the runtime overhead for this. Sure,
permissions can be applied without a SecurityManager, but this incurs
the administrative headache of granting permissions correctly.


So I'd rather steer clear of a permission based approach to restricting
access to modules, if we want to go down this path at all.

// Bryan

> --
> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
> Adrian Brock
> Chief Scientist
> JBoss, a division of Red Hat
> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>


From bryan.atsatt at oracle.com  Tue Jun 19 16:31:15 2007
From: bryan.atsatt at oracle.com (Bryan Atsatt)
Date: Tue, 19 Jun 2007 16:31:15 -0700
Subject: Application Isolation in App Servers was Re: Module isolation
In-Reply-To: <46782BB7.7020501@oracle.com>
References: <OFD20EE2C8.4BD4A1F7-ON802572ED.00360624-802572ED.00464DBD@uk.ibm.com>
	<466088D9.9020903@oracle.com> <46703604.5040203@sun.com>
	<467042BE.3090307@oracle.com> <467079AA.6050805@sun.com>
	<46717DB5.1080407@oracle.com> <4671D9B5.20909@sun.com>
	<4671F216.3050506@oracle.com> <46732CAE.6070207@sun.com>
	<4676F479.9070400@oracle.com> <1182256177.3510.21.camel@warjort>
	<46782BB7.7020501@oracle.com>
Message-ID: <46786743.4080003@oracle.com>

Whoops, forgot to say one thing...

As you describe it, I would consider JBoss ClassLoader Domains to be
100% public as long as *any* application can import/subscribe/access it.

To me:

a. A public repository contains Modules that may be imported by any other.

b. A private repository contains Modules that may only be imported by
other modules from the *same* deployment unit (e.g. .ear file.)

Anything in between seems to cry out for an access control model.

// Bryan

Bryan Atsatt wrote:
> Adrian wrote:
>> On Mon, 2007-06-18 at 14:09 -0700, Bryan Atsatt wrote:
>>> So an application server would have to create, say, a private
>>> LocalRepository instance to hold the modules of a single application.
>>> And it would have to ensure that no other application could get it's
>>> grubby paws on that repository instance.
>>>
>>> Ok. That works for me. And it eliminates the need for cloning AND for
>>> releaseModule():
>>>
>>> 1. Any given Repository instance is either 100% shared or 100% private,
>>> with *no* in-between.
>>
>> Since we are talking about appservers, I'll just point out
>> that this doesn't cover what JBoss allows you to do.
>>
>> In JBoss we have a notion of what I like to call "ClassLoader Domains"
>> although they are actually called "LoaderRepositorys" after
>> the JMX name (note the JMX spec only has a singleton repository).
>>
>> These are "isolated" classloading systems that sit on top
>> of a "Default Domain" and you can choose whether to do
>> parent/child delegation in either
>> * standard J2SE order (parent first)
>> * servlet style (child first).
>>
>> When you deploy an application you get to choose the domain.
>> There is nothing to stop you putting multiple applications
>> into the same domain.
>>
>> I would guess in your proposal they WOULD share the
>> same LocalRepository but only those named applications
>> would have access to it?
>
> Right.
>
>>
>> There are two main usecases for such a configuration.
>>
>> 1)  Optimization
>> You want to optimize the inter-application communication
>> between related applications. i.e. one application uses
>> components from another application and you don't want
>> to do call-by-value.
>> By having the applications in the same classloader domain
>> one application can lookup and use the local interfaces
>> of the other application.
>
> Yep.
>
>>
>> 2) Convenience
>> You deploy a rar inside an ear (making the rar classes
>> invisible to other applications)
>> but you want to configure the connection factories
>> of that rar outside the ear where it is easier to
>> edit the configuration file.
>>
>> In this case you can specify that the connection factories
>> should use the classloading domain of the ear, making
>> its classes visible again.
>>
>> e.g. something like:
>>
>> <connection-factories>
>>
>> <loader-repository>classloader:application=myapp.ear</loader-repository>
>>
>> <etc/>
>
> This last example doesn't really apply in our server, as the config is
> just as easy to edit inside as outside an ear. But I see your point.
>
>
> Most apps don't need to share classes in this way, and need only a
> mixture of global and purely local classes. Clearly any module system
> will be able to satisfy the first requirement. We need to make sure we
> can also satisfy the second.
>
> But you are right, of course, that there exist more complex scenarios
> that require an intermediate form of sharing. Your named domains seems
> to satisfy that requirement. Can you restrict access to domains?
>
> And, as you know, even *within* an application, web-modules need to be
> isolated. So the private repository approach breaks down a bit here, or
> must be taken to the extreme of a single module repository instance.
> This is where an access control model might be a better solution.
>
>
> Our server supports three models for class sharing:
>
> 1. Global. All apps share a single version of the classes. This is a
> deprecated model.
>
> 2. Shared. Our "Shared Libraries" are named/versioned loaders configured
> at the server level, each with one or more code-sources. Each can
> optionally import other shared-libraries. Import declarations are by
> name and version range. A default set of imports is configured by the
> container for all application instances, but each is app free to modify
> that set via config.
>
> (This is obviously very much like the 277/OSGi model.)
>
> 3. Application group. Applications are hierarchical: each app is
> deployed under some parent, and can access the classes of that parent
> (excluding web-modules). So several apps can be deployed under a common
> parent, and classes visible at the parent are visible to the children.
> The global model in #1 simply means deploying classes into our "default"
> application.
>
>
> The shared-library approach has proved to be the most flexible and most
> used (surprise). But this model does not support any access control, so
> at the moment, if you want restricted sharing you must use the shared
> parent approach.
>
> And while we do support execution under a SecurityManager, *very* few
> companies are willing to pay the runtime overhead for this. Sure,
> permissions can be applied without a SecurityManager, but this incurs
> the administrative headache of granting permissions correctly.
>
>
> So I'd rather steer clear of a permission based approach to restricting
> access to modules, if we want to go down this path at all.
>
> // Bryan
>
>> --
>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>> Adrian Brock
>> Chief Scientist
>> JBoss, a division of Red Hat
>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>
>


From glyn_normington at UK.IBM.COM  Wed Jun 20 02:25:11 2007
From: glyn_normington at UK.IBM.COM (Glyn Normington)
Date: Wed, 20 Jun 2007 10:25:11 +0100
Subject: JSR 291 interoperation status
Message-ID: <OF311FDF59.E80066BE-ON80257300.0033032E-80257300.0033BEC7@uk.ibm.com>

Hi Stanley

Please could you give us an update on your work on interoperation with JSR
291 and some idea of when a strawman is likely to be available for initial
review by the JSR 277 Expert Group?

Like I said when we spoke at JavaOne, the approach you are planning on
taking *might* work, but it has significant challenges, so I'd like to
hear how it's coming along.

Thanks,

Glyn





Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU





-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/attachments/20070620/ce6cf41b/attachment.html 

From abrock at REDHAT.COM  Wed Jun 20 07:19:26 2007
From: abrock at REDHAT.COM (Adrian)
Date: Wed, 20 Jun 2007 16:19:26 +0200
Subject: Application Isolation in App Servers was Re: Module isolation
In-Reply-To: <46782BB7.7020501@oracle.com>
References: <OFD20EE2C8.4BD4A1F7-ON802572ED.00360624-802572ED.00464DBD@uk.ibm.com>
	<466088D9.9020903@oracle.com> <46703604.5040203@sun.com>
	<467042BE.3090307@oracle.com> <467079AA.6050805@sun.com>
	<46717DB5.1080407@oracle.com> <4671D9B5.20909@sun.com>
	<4671F216.3050506@oracle.com> <46732CAE.6070207@sun.com>
	<4676F479.9070400@oracle.com> <1182256177.3510.21.camel@warjort>
	<46782BB7.7020501@oracle.com>
Message-ID: <1182349166.3510.10.camel@warjort>

On Tue, 2007-06-19 at 12:17 -0700, Bryan Atsatt wrote:
> But you are right, of course, that there exist more complex scenarios
> that require an intermediate form of sharing. Your named domains seems
> to satisfy that requirement. Can you restrict access to domains?
>

Not really. We assume that if you have "createClassLoader" permission
then you can modify the classloading structure however you like.

> And, as you know, even *within* an application, web-modules need to be
> isolated. So the private repository approach breaks down a bit here, or
> must be taken to the extreme of a single module repository instance.
> This is where an access control model might be a better solution.

The big issue with web-apps is their "load locally first".
This gets people into all sorts of trouble when they start
adding random things to WEB-INF/lib that is also visible
elsewhere in the application.

e.g. You add commons logging to an ear and then put
it in WEB-INF/lib of a war also in the ear

Ideally you'd want them to share the same classes,
but sometimes it is done this way so each webapp can
have its own singletons.

--
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Adrian Brock
Chief Scientist
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx


From bryan.atsatt at oracle.com  Wed Jun 20 11:21:46 2007
From: bryan.atsatt at oracle.com (Bryan Atsatt)
Date: Wed, 20 Jun 2007 11:21:46 -0700
Subject: Application Isolation in App Servers was Re: Module isolation
In-Reply-To: <1182349166.3510.10.camel@warjort>
References: <OFD20EE2C8.4BD4A1F7-ON802572ED.00360624-802572ED.00464DBD@uk.ibm.com>
	<466088D9.9020903@oracle.com> <46703604.5040203@sun.com>
	<467042BE.3090307@oracle.com> <467079AA.6050805@sun.com>
	<46717DB5.1080407@oracle.com> <4671D9B5.20909@sun.com>
	<4671F216.3050506@oracle.com> <46732CAE.6070207@sun.com>
	<4676F479.9070400@oracle.com> <1182256177.3510.21.camel@warjort>
	<46782BB7.7020501@oracle.com> <1182349166.3510.10.camel@warjort>
Message-ID: <4679703A.7040202@oracle.com>

Adrian wrote:
> On Tue, 2007-06-19 at 12:17 -0700, Bryan Atsatt wrote:
>> But you are right, of course, that there exist more complex scenarios
>> that require an intermediate form of sharing. Your named domains seems
>> to satisfy that requirement. Can you restrict access to domains?
>>
>
> Not really. We assume that if you have "createClassLoader" permission
> then you can modify the classloading structure however you like.

Right. So, like our shared-libraries, these loaders are generally
available to any app.

>
>> And, as you know, even *within* an application, web-modules need to be
>> isolated. So the private repository approach breaks down a bit here, or
>> must be taken to the extreme of a single module repository instance.
>> This is where an access control model might be a better solution.
>
> The big issue with web-apps is their "load locally first".
> This gets people into all sorts of trouble when they start
> adding random things to WEB-INF/lib that is also visible
> elsewhere in the application.

Sure. This is in fact why we disable "child-first" behavior by default,
and strongly discourage it in our docs. This servlet spec
"recommendation" has been a nightmare! (The ironic part is that it
caused Sun's own CTS to break, as they had copies of EJB interfaces in
the wars! Turns out this was a really good thing, as it was a giant red
flag that caused me to set the default the other way, long before any of
this was really understood.)

Let's not repeat the same mistake here! There should not be any way to
invert the search order in a module system. Period. It was a *major
hack* and works only in carefully controlled circumstances.

The isolation problem needs a general solution.

> e.g. You add commons logging to an ear and then put
> it in WEB-INF/lib of a war also in the ear
>
> Ideally you'd want them to share the same classes,
> but sometimes it is done this way so each webapp can
> have its own singletons.

Yes, and this is exactly the sort of situation where you might want to
share a ModuleDefinition, but *not* a Module. This would be an enhanced
equivalent of child-first: enabling class duplication but only of a
*subset* of required classes (i.e. specific module/modules).

But, without the OSGi "uses" like functionality, I don't know that we
can implement a good enough validation model to detect the collision cases.

Personally, I'd rather we not support this kind of hack; it is nothing
but a way to work around a serious deficiency of some existing code that
uses statics poorly. That code should just be fixed!

And, while we wait for that fix, a fully private copy of the module is a
reasonable work-around. There aren't too many of these cases, at least
not that I'm aware of.


These additional complexities aside, we still need a solution for the
simple case of web-app loader isolation within an app.


>
> --
> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
> Adrian Brock
> Chief Scientist
> JBoss, a division of Red Hat
> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>


From Stanley.Ho at sun.com  Wed Jun 20 12:54:35 2007
From: Stanley.Ho at sun.com (Stanley M. Ho)
Date: Wed, 20 Jun 2007 12:54:35 -0700
Subject: Module isolation
In-Reply-To: <4676F479.9070400@oracle.com>
References: <OFD20EE2C8.4BD4A1F7-ON802572ED.00360624-802572ED.00464DBD@uk.ibm.com>
	<466088D9.9020903@oracle.com> <46703604.5040203@sun.com>
	<467042BE.3090307@oracle.com> <467079AA.6050805@sun.com>
	<46717DB5.1080407@oracle.com> <4671D9B5.20909@sun.com>
	<4671F216.3050506@oracle.com> <46732CAE.6070207@sun.com>
	<4676F479.9070400@oracle.com>
Message-ID: <467985FB.4060407@sun.com>

Hi Bryan,

Sorry, I was sidetracked in the last two days, will follow up other
threads soon.

Bryan Atsatt wrote:
> This use case seems to presume that the IDE can/will ensure that there
> is only *one* consumer of the module: itself. What if a different plugin
> has a dependency on it, and has already been resolved? Is the intention
> that the IDE will be wired into the module system deeply enough to
> manage this correctly?

It won't affect existing plugins that has also been initialized with the
dependency. My assumption is that the IDE should know what it's doing
when using the APIs.

> An application server, for example, will need to keep one application
> from accessing the modules of another. And it must *know* that there is
> no sharing, so that the application lifecycles can remain independent.
>
> So we need some notion of a context. A purely private repository
> instance is one (probably good) possibility. Another is the wrapper
> Repository approach, but this requires definition copies (and management
> of sharing content, lifecycle, etc).

The way I see it is that the repository is the context that you want to
use for isolation. The APIs allow developers only to walk up the parent
repository instance from a child repository instance, but not vice
versa. If the repository instance is only used in a specific domain
(e.g. webapp) and the repository instance is hidden from other
applications (e.g. other webapps), this would effectively hide the
modules in that repository and the repository could be considered
private. Whether such repository is constructed using the wrapper
approach or reusing existing ModuleDefinitionContent is an
implementation detail. Do you agree?

> Regardless of what approach we take, the releaseModule() idea is too
> simplistic. Having originally created the detach() method, thinking
> along similar lines as you are with the plugin case, I do understand the
> idea; I just no longer think it is sufficient :^).
>
> The only "safe" time to release a module is when there are *zero*
> importers, and, even then, you must hide/release atomically, ensuring
> that no new imports spring up during the operation.

Releasing a module simply means that its reference is released from the
module system's cache; it does not affect any existing importer. Also,
hide and release does not need to happen atomically - I think hiding it
first would be sufficient to prevent new imports to be resolved, but
I'll need to double check it with the RI.

Note that I'm not arguing the releaseModule() approach is perfect, but I
do think this is a use case that we can ignore; I would welcome better
suggestion to handle this.

> Other than Module instances, what other "runtime information" is there
> to keep track of? Caches of exported packages?

The Module instances that have been initialized, the ModuleDefinitions
that are being instantiated and the corresponding Module instances that
are being initialized (could be triggered simultaneously from multiple
threads), and the ModuleDefinitions that have been disabled (e.g. the
repository has been shutdown and no new Module instance should be created).

> For example, if we were to eliminate the releaseModule() method (in
> favor of some more complete mechanism), then there really is always a
> 1:1 for Module:ModuleDefinition, and the model is simple and obvious.
> (And therefore a field cache *could* be used).

To keep this discussion focus, I think we could assume
Module:ModuleDefinition is 1:1 for now. releaseModule() is just a
special case. That said, even if the relationship is 1:1, it still
doesn't mean we *should* put the state in ModuleDefinition, see above. ;-)

> So, in effect, we have 100% *private repository* instances.
>
> I've been thinking that we need an intermediate somewhere between a
> shared/public repo instance and an entirely private one, but... that now
> strikes me as too fuzzy, and I can't see a real use case :^)
>
> So an application server would have to create, say, a private
> LocalRepository instance to hold the modules of a single application.
> And it would have to ensure that no other application could get it's
> grubby paws on that repository instance.
>
> Ok. That works for me. And it eliminates the need for cloning AND for
> releaseModule():
>
> 1. Any given Repository instance is either 100% shared or 100% private,
> with *no* in-between.
>
> 2. The lifecycle of a shared repository instance is that of the process.
>
> 3. The lifecycle of a private repository instance is entirely up to the
> creator of that instance.

I agreed that distinguishing the repository instances for sharing or
private usages is more easy to understand, but the notion of shared and
private really depends on the usage context, and it's not an attribute
of the repository itself. Suppose there is a repository with two child
repositories; this repository would be considered "shared" from the
perspective of the modules in the child repositories, but this
repository might still be considered private from the other applications
in the same JVM.

I think your first three points can be combined as follows:

1. Any given Repository instance could be used for sharing or private
purpose.

2. The lifetime of a repository instance is managed by its creator.

> 4. The lifecycle of a ModuleDefinition/Module is at most that of the
> enclosing Repository instance, and at least is bounded by
> install/uninstall (no finer granularity).

Yes.

> It does leave open the issue of dependencies *within* a private
> repository. The simple model would be to treat the entire repository as
> atomic, with any change requiring a new Repository instance. This is
> probably too simplistic, however.
>
> In an EE app, web-modules are supposed to be isolated from each other
> and from other parts of the app (ejb, connectors, etc.). So this
> requires either a further partitioning of the app into multiple
> repositories, or some form of access control.

The web-modules should probably be in its own repository if isolation is
needed. There is no access control for ModuleDefinitions - accessibility
of a ModuleDefinition is the same as visibility of ModuleDefinition;
this is also a very different issue. I think we should focus our
discussion on using repository instance for isolation unless we find
this approach insufficient.

> Further, it is possible to re-start or re-deploy/re-start only a single
> web-module, *without* restarting the rest of the app. The re-start case
> could use releaseModule(), though a "real" stop method would be
> preferable. But this is a very special case in which the specs
> essentially dictate the possible dependencies between modules. In the
> general case where the dependencies are not dictated, releaseModule() is
> problematic.
> The re-deploy case would use uninstall/install (but would still like
> stop!).

Whether the solution is stop() or releaseModule(), it still shows that
we need to support the use case of long-lived repository with
short-lived modules. Do you agree?

> If you *really* want the releaseModule() functionality, I would suggest
> that we introduce a PrivateRepository type, and support release *only*
> on that type.

The notion of private or shared depends on the usage context of the
repository instance, so it won't be appropriate to surface the concept
at the API level.

Bryan, there are many topics in this thread, but I think what you really
want is to discuss the notion of module isolation. As we discussed so
far, I think we all agreed that repository is the appropriate context to
make module isolation possible. Could you summarize any outstanding
design concern you have with this approach in a paragraph or two?

Thanks,
- Stanley


From Stanley.Ho at sun.com  Wed Jun 20 15:16:01 2007
From: Stanley.Ho at sun.com (Stanley M. Ho)
Date: Wed, 20 Jun 2007 15:16:01 -0700
Subject: Strawman: Services and service-providers support
In-Reply-To: <OF7EE8721A.05DA767D-ON802572FB.003119D4-802572FB.0037A18F@uk.ibm.com>
References: <OF7EE8721A.05DA767D-ON802572FB.003119D4-802572FB.0037A18F@uk.ibm.com>
Message-ID: <4679A721.6050908@sun.com>

Hi Glyn,

Glyn Normington wrote:
> Maybe it's just a matter of presentation in the strawman, but it
> confused me somewhat and the net effect was that I wasn't sure what
> pieces JSR 277 will be providing and what pieces are upgrades of
> existing facilities (since I'm not familiar with the existing
> service/service-provider model) to take advantage of JSR 277.
>
> 1. The opening paragraph of the Background section defines a service as
> "a well-known set of interfaces and (usually abstract) classes" - no
> mention of instances,
>
> 2. The third paragraph of the same section say "a service is represented
> by a single type ...". Again no mention of instances.

These text are borrowed from the ServiceLoader API. I will look into
clarifying it.

> 3. The requirements section talks about the module system making
> services available to the Java platform. This is using 'service' in the
> above sense, but I'm not sure what it means to make an interface
> "available to the Java platform" unless it means "available for modules
> to import".

It basically means two things:

1. The service module is available for other modules to import.
2. If the service module is deployed in one of the system repositories,
the service in the service module is available to both the module-based
and non-module-based applications when they call ServiceLoader.load().

> 4. Section 6 talks about an application module importing a service
> module if and only if there is at least one suitable service provider. I
> guess the rationale is to make the application module fail to resolve if
> it hasn't a (current) hope of obtaining the service it needs. But from
> my perspective this is odd. The service provider could be available
> without it being possible to obtain a service instance, which is the
> thing the application really needs. So rather than being an import-time
> check, the availability of critical services should be checked when the
> application module starts (for static applications that don't need to
> react dynamically to services coming online, that is).

Right, in typical cases, the availability should be checked at runtime
instead of during initialization. What I wanted to show in the strawman
was how it might be supported if we wanted to do the checking during
initialization using the declarative approach, but the strawman also
suggested that this was probably not a good idea and the custom import
policy should be the right way to go.

Most of the Section 4,5,6, and 8 are what I have in mind that belong to
277. I think it will become more clear after I extract the pieces into
the next revision of the spec so the EG can review and discuss.

- Stanley


From Stanley.Ho at sun.com  Wed Jun 20 15:43:14 2007
From: Stanley.Ho at sun.com (Stanley M. Ho)
Date: Wed, 20 Jun 2007 15:43:14 -0700
Subject: JSR 291 interoperation status
In-Reply-To: <OF311FDF59.E80066BE-ON80257300.0033032E-80257300.0033BEC7@uk.ibm.com>
References: <OF311FDF59.E80066BE-ON80257300.0033032E-80257300.0033BEC7@uk.ibm.com>
Message-ID: <4679AD82.3080602@sun.com>

Hi Glyn,

That strawman is in progress. As you mentioned, it has significant
challenges, so it takes more time to produce compared to other
strawmans. Please stay tuned. ;-)

- Stanley


Glyn Normington wrote:
> Please could you give us an update on your work on interoperation with
> JSR 291 and some idea of when a strawman is likely to be available for
> initial review by the JSR 277 Expert Group?
>
> Like I said when we spoke at JavaOne, the approach you are planning on
> taking *might* work, but it has significant challenges, so I'd like to
> hear how it's coming along.
>
> Thanks,
>
> Glyn


From bryan.atsatt at oracle.com  Wed Jun 20 16:11:40 2007
From: bryan.atsatt at oracle.com (Bryan Atsatt)
Date: Wed, 20 Jun 2007 16:11:40 -0700
Subject: Module isolation
In-Reply-To: <467985FB.4060407@sun.com>
References: <OFD20EE2C8.4BD4A1F7-ON802572ED.00360624-802572ED.00464DBD@uk.ibm.com>
	<466088D9.9020903@oracle.com> <46703604.5040203@sun.com>
	<467042BE.3090307@oracle.com> <467079AA.6050805@sun.com>
	<46717DB5.1080407@oracle.com> <4671D9B5.20909@sun.com>
	<4671F216.3050506@oracle.com> <46732CAE.6070207@sun.com>
	<4676F479.9070400@oracle.com> <467985FB.4060407@sun.com>
Message-ID: <4679B42C.6020305@oracle.com>

Stanley M. Ho wrote:
> Hi Bryan,
>
> Sorry, I was sidetracked in the last two days, will follow up other
> threads soon.
>
> Bryan Atsatt wrote:
>> This use case seems to presume that the IDE can/will ensure that there
>> is only *one* consumer of the module: itself. What if a different plugin
>> has a dependency on it, and has already been resolved? Is the intention
>> that the IDE will be wired into the module system deeply enough to
>> manage this correctly?
>
> It won't affect existing plugins that has also been initialized with the
> dependency. My assumption is that the IDE should know what it's doing
> when using the APIs.
>
>> An application server, for example, will need to keep one application
>> from accessing the modules of another. And it must *know* that there is
>> no sharing, so that the application lifecycles can remain independent.
>>
>> So we need some notion of a context. A purely private repository
>> instance is one (probably good) possibility. Another is the wrapper
>> Repository approach, but this requires definition copies (and management
>> of sharing content, lifecycle, etc).
>
> The way I see it is that the repository is the context that you want to
> use for isolation. The APIs allow developers only to walk up the parent
> repository instance from a child repository instance, but not vice
> versa. If the repository instance is only used in a specific domain
> (e.g. webapp) and the repository instance is hidden from other
> applications (e.g. other webapps), this would effectively hide the
> modules in that repository and the repository could be considered
> private. Whether such repository is constructed using the wrapper
> approach or reusing existing ModuleDefinitionContent is an
> implementation detail. Do you agree?

Yes, but it is a very critical detail! The wrapper approach, if
required, drags in the copy, concurrency and lifecycle issues; all of
which need solutions. I am groping here for a model in which none of
this is required under normal circumstances.

I think the purely private repository model works fine, where the
definitions and module instances are also private.

And, as I said in the app server thread, using the private repository
approach for web-modules literally means a repository containing a
*single definition*. This may be an acceptable solution, but I want to
ensure that everyone is aware of this implication.


>> Regardless of what approach we take, the releaseModule() idea is too
>> simplistic. Having originally created the detach() method, thinking
>> along similar lines as you are with the plugin case, I do understand the
>> idea; I just no longer think it is sufficient :^).
>>
>> The only "safe" time to release a module is when there are *zero*
>> importers, and, even then, you must hide/release atomically, ensuring
>> that no new imports spring up during the operation.
>
> Releasing a module simply means that its reference is released from the
> module system's cache; it does not affect any existing importer. Also,
> hide and release does not need to happen atomically - I think hiding it
> first would be sufficient to prevent new imports to be resolved, but
> I'll need to double check it with the RI.

I understand what release does :^). The issue is not the release itself,
but the potential subsequent instantiation of a Module from that same
definition. This instance is now a duplicate, with duplicate classes,
which can easily lead to failures: the original instance may still be in
use.

>
> Note that I'm not arguing the releaseModule() approach is perfect, but I
> do think this is a use case that we can ignore; I would welcome better
> suggestion to handle this.
>
>> Other than Module instances, what other "runtime information" is there
>> to keep track of? Caches of exported packages?
>
> The Module instances that have been initialized, the ModuleDefinitions
> that are being instantiated and the corresponding Module instances that
> are being initialized (could be triggered simultaneously from multiple
> threads), and the ModuleDefinitions that have been disabled (e.g. the
> repository has been shutdown and no new Module instance should be created).

Right. But only the first of these is long lived, the others are
transient, needed only during resolution. So again, if releaseModule()
either did not exist, *or* if we did not keep references to released
instances, a simple field cache could be used. Just trying to see
through some of the fog here :^)

>
>> For example, if we were to eliminate the releaseModule() method (in
>> favor of some more complete mechanism), then there really is always a
>> 1:1 for Module:ModuleDefinition, and the model is simple and obvious.
>> (And therefore a field cache *could* be used).
>
> To keep this discussion focus, I think we could assume
> Module:ModuleDefinition is 1:1 for now. releaseModule() is just a
> special case. That said, even if the relationship is 1:1, it still
> doesn't mean we *should* put the state in ModuleDefinition, see above. ;-)

Sure. (OTOH, it *would* make ModuleSystem a bit simpler by eliminating
the need for a silly mapping, and would bake the 1:1 model into the
design. And stashing a field like this isn't much different than an
immutable String having a hash field that is lazily stored. But clearly
if you want to track released/stopped instances, this becomes more
involved! I'm going to shut up about this aspect now ;^)

>
>> So, in effect, we have 100% *private repository* instances.
>>
>> I've been thinking that we need an intermediate somewhere between a
>> shared/public repo instance and an entirely private one, but... that now
>> strikes me as too fuzzy, and I can't see a real use case :^)
>>
>> So an application server would have to create, say, a private
>> LocalRepository instance to hold the modules of a single application.
>> And it would have to ensure that no other application could get it's
>> grubby paws on that repository instance.
>>
>> Ok. That works for me. And it eliminates the need for cloning AND for
>> releaseModule():
>>
>> 1. Any given Repository instance is either 100% shared or 100% private,
>> with *no* in-between.
>>
>> 2. The lifecycle of a shared repository instance is that of the process.
>>
>> 3. The lifecycle of a private repository instance is entirely up to the
>> creator of that instance.
>
> I agreed that distinguishing the repository instances for sharing or
> private usages is more easy to understand, but the notion of shared and
> private really depends on the usage context, and it's not an attribute
> of the repository itself. Suppose there is a repository with two child
> repositories; this repository would be considered "shared" from the
> perspective of the modules in the child repositories, but this
> repository might still be considered private from the other applications
> in the same JVM.
>
> I think your first three points can be combined as follows:
>
> 1. Any given Repository instance could be used for sharing or private
> purpose.
>
> 2. The lifetime of a repository instance is managed by its creator.

Yes, but this doesn't go far enough. For example, the JRE will create
the system repository, and, by this rule, could shut it down at any
time. Clearly this would cause major havoc. Unless we have a complete
lifecycle model for Modules, so that it is possible to ensure that no
active user exists, such "global" modules must live for the lifetime of
the process. Otherwise, we risk collision failures.

>
>> 4. The lifecycle of a ModuleDefinition/Module is at most that of the
>> enclosing Repository instance, and at least is bounded by
>> install/uninstall (no finer granularity).
>
> Yes.

But releaseModule() violates #4. Use of that method enables unlimited
numbers of Module instances from the same definition, during the
lifetime of the enclosing repository.

>
>> It does leave open the issue of dependencies *within* a private
>> repository. The simple model would be to treat the entire repository as
>> atomic, with any change requiring a new Repository instance. This is
>> probably too simplistic, however.
>>
>> In an EE app, web-modules are supposed to be isolated from each other
>> and from other parts of the app (ejb, connectors, etc.). So this
>> requires either a further partitioning of the app into multiple
>> repositories, or some form of access control.
>
> The web-modules should probably be in its own repository if isolation is
> needed. There is no access control for ModuleDefinitions - accessibility
> of a ModuleDefinition is the same as visibility of ModuleDefinition;
> this is also a very different issue. I think we should focus our
> discussion on using repository instance for isolation unless we find
> this approach insufficient.

Sure, again, as long as a single module repository is deemed an
acceptable solution.

>
>> Further, it is possible to re-start or re-deploy/re-start only a single
>> web-module, *without* restarting the rest of the app. The re-start case
>> could use releaseModule(), though a "real" stop method would be
>> preferable. But this is a very special case in which the specs
>> essentially dictate the possible dependencies between modules. In the
>> general case where the dependencies are not dictated, releaseModule() is
>> problematic.
>> The re-deploy case would use uninstall/install (but would still like
>> stop!).
>
> Whether the solution is stop() or releaseModule(), it still shows that
> we need to support the use case of long-lived repository with
> short-lived modules. Do you agree?

Sure.

>
>> If you *really* want the releaseModule() functionality, I would suggest
>> that we introduce a PrivateRepository type, and support release *only*
>> on that type.
>
> The notion of private or shared depends on the usage context of the
> repository instance, so it won't be appropriate to surface the concept
> at the API level.

If the semantics of releaseModule() limit its use to very special
circumstances, I think it would be appropriate to capture this in the API.

In the spec, you rule out the use of this method against any java.* or
bootstrap repository module.

But is it ever appropriate to call releaseModule() on a Module from the
system repository? Or any public, global repository?

I was simply exploring the possibility that this functionality only
makes sense on a private repository instance. But I don't care if we
generalize that to say that the creator should be able to control this
behavior.

If so, I'd like that to be manifest in the API somehow. I don't really
care how. As another possibility, why not expose this an attribute of
Repository, just as we do read-only status?

    public abstract boolean supportsRelease();

And then document ModuleSystem.releaseModule() as a no-op or failure
when the underlying repo returns false.

>
> Bryan, there are many topics in this thread, but I think what you really
> want is to discuss the notion of module isolation. As we discussed so
> far, I think we all agreed that repository is the appropriate context to
> make module isolation possible. Could you summarize any outstanding
> design concern you have with this approach in a paragraph or two?

My only remaining design concern (on the topic of isolation) is the
semantics of releaseModule(). As long as we nail down the *correct*
usage of this method, I'll be happy.

Since the current design does not provide a means to determine or
control use of released modules, class duplication failures and/or large
memory leaks remain a strong possibility. As I've said from the very
beginning, we have solved this problem in our application server by
choosing to disable the class loaders of "released" modules, and it has
proven extremely useful. I would like to see this supported here as an
optional behavior.

>
> Thanks,
> - Stanley
>


From Stanley.Ho at sun.com  Wed Jun 20 17:58:16 2007
From: Stanley.Ho at sun.com (Stanley M. Ho)
Date: Wed, 20 Jun 2007 17:58:16 -0700
Subject: Module isolation
In-Reply-To: <4679B42C.6020305@oracle.com>
References: <OFD20EE2C8.4BD4A1F7-ON802572ED.00360624-802572ED.00464DBD@uk.ibm.com>
	<466088D9.9020903@oracle.com> <46703604.5040203@sun.com>
	<467042BE.3090307@oracle.com> <467079AA.6050805@sun.com>
	<46717DB5.1080407@oracle.com> <4671D9B5.20909@sun.com>
	<4671F216.3050506@oracle.com> <46732CAE.6070207@sun.com>
	<4676F479.9070400@oracle.com> <467985FB.4060407@sun.com>
	<4679B42C.6020305@oracle.com>
Message-ID: <4679CD28.3000603@sun.com>

Hi Bryan,

Bryan Atsatt wrote:
> I think the purely private repository model works fine, where the
> definitions and module instances are also private.
>
> And, as I said in the app server thread, using the private repository
> approach for web-modules literally means a repository containing a
> *single definition*. This may be an acceptable solution, but I want to
> ensure that everyone is aware of this implication.

Sure. I think we're more or less on the same page now. ;-)

> If the semantics of releaseModule() limit its use to very special
> circumstances, I think it would be appropriate to capture this in the API.

Agreed.

> In the spec, you rule out the use of this method against any java.* or
> bootstrap repository module.
>
> But is it ever appropriate to call releaseModule() on a Module from the
> system repository? Or any public, global repository?

Most likely not, but I see your point.

> I was simply exploring the possibility that this functionality only
> makes sense on a private repository instance. But I don't care if we
> generalize that to say that the creator should be able to control this
> behavior.
>
> If so, I'd like that to be manifest in the API somehow. I don't really
> care how. As another possibility, why not expose this an attribute of
> Repository, just as we do read-only status?
>
>    public abstract boolean supportsRelease();
>
> And then document ModuleSystem.releaseModule() as a no-op or failure
> when the underlying repo returns false.

This sounds reasonable. I will look into this for the next revision of
the API spec.

>> Bryan, there are many topics in this thread, but I think what you really
>> want is to discuss the notion of module isolation. As we discussed so
>> far, I think we all agreed that repository is the appropriate context to
>> make module isolation possible. Could you summarize any outstanding
>> design concern you have with this approach in a paragraph or two?
>
> My only remaining design concern (on the topic of isolation) is the
> semantics of releaseModule(). As long as we nail down the *correct*
> usage of this method, I'll be happy.

Good. I will clarify the semantics of releaseModule() in the javadoc.

> Since the current design does not provide a means to determine or
> control use of released modules, class duplication failures and/or large
> memory leaks remain a strong possibility. As I've said from the very
> beginning, we have solved this problem in our application server by
> choosing to disable the class loaders of "released" modules, and it has
> proven extremely useful. I would like to see this supported here as an
> optional behavior.

Disabling the classloaders is a different issue. I will start a new
thread on a topic related to this issue soon.

- Stanley


From glyn_normington at UK.IBM.COM  Thu Jun 21 01:59:02 2007
From: glyn_normington at UK.IBM.COM (Glyn Normington)
Date: Thu, 21 Jun 2007 09:59:02 +0100
Subject: JSR 291 interoperation status
In-Reply-To: <4679AD82.3080602@sun.com>
Message-ID: <OF2B4928D6.712CA1A8-ON80257301.002C75D2-80257301.003159D6@uk.ibm.com>

Hi Stanley

Thanks for the status. Please could you indicate roughly when the strawman
will be ready for initial review by this EG? Hopefully early feedback from
the Expert Group would be a help...

Glyn

"Stanley M. Ho" <Stanley.Ho at SUN.COM> wrote on 20/06/2007 11:43:14 PM:

> Hi Glyn,
>
> That strawman is in progress. As you mentioned, it has significant
> challenges, so it takes more time to produce compared to other
> strawmans. Please stay tuned. ;-)
>
> - Stanley
>
>
> Glyn Normington wrote:
> > Please could you give us an update on your work on interoperation with
> > JSR 291 and some idea of when a strawman is likely to be available for
> > initial review by the JSR 277 Expert Group?
> >
> > Like I said when we spoke at JavaOne, the approach you are planning on
> > taking *might* work, but it has significant challenges, so I'd like to
> > hear how it's coming along.
> >
> > Thanks,
> >
> > Glyn






Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU





-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/attachments/20070621/1daafef1/attachment.html 

From Stanley.Ho at sun.com  Thu Jun 21 18:10:02 2007
From: Stanley.Ho at sun.com (Stanley M. Ho)
Date: Thu, 21 Jun 2007 18:10:02 -0700
Subject: Module system notification mechanism
Message-ID: <467B216A.70109@sun.com>

Hi JSR 277 experts,

Since we have been discussing some issues around the module instances'
lifetime lately, I think it's probably a good time to bring up a related
topic for discussion.

As I reviewed the feedbacks from the EDR comments, from our previous
discussions, as well as from my discussions with the teams in SE and EE,
there were a few suggestions related to the module instances' lifetime:

1. The module system shall provide a way to monitor various events, e.g.
module initialized, module released, etc.
2. The module system shall allow a module to have activator code. The
activator code would be executed right before the module is fully
initialized and when the instance is released.
3. A module shall have a way to be stopped.

Having these suggestions don't mean we have to do all of them, and I
would like to get your inputs.


 From my perspective, having a way to monitor module system's events
(i.e. #1) seems very reasonable and useful, especially the use cases are
very common. In fact, many teams in SE have expressed the needs in
monitoring the module system's events in their class libraries in some
degrees, so these libraries would react and behave appropriately. There
are also other class libraries sitting on top of the JRE that have
similar needs.

For #2, this is a use case I gathered from EE, and this would be used
mainly for registering and unregistering services when a module has been
initialized or is released. Not that I think this is unimportant, but I
am not yet convinced this is something we need to support directly at
the module system level. For instance, if the module system notification
mechanism (i.e. #1) is available, it should be possible for the EE
server (or other apps that require similar functionality) to build a
simple activator layer on top of the module system.

For #3, the use case is that some EE servers might want to have the
ability to "stop" a module by disabling the module's classloader when
the module instance is released. In general, disabling a classloader is
an uncommon and dangerous operation to perform, and it also violates the
current classloading spec. While I agreed we should make this use case
possible, I don't think this is something we want to push into the
module system.; I believe there are alternatives we could consider to
achieve the same result. For example, suppose there are APIs available
to disable a classloader (might come from the classloading project) and
the module system notification mechanism (i.e. #1) is available, it
should be possible for the EE servers to hook into the notification
mechanism, and disable the specific classloader it wants when a module
instance is released from the module system.


To keep things simple, I suggest we should support #1, and I hope this
should be sufficient to enable other applications (e.g. EE servers) to
support #2 and #3. I would like to hear what your thoughts are.

- Stanley


From bryan.atsatt at oracle.com  Thu Jun 21 18:23:58 2007
From: bryan.atsatt at oracle.com (Bryan Atsatt)
Date: Thu, 21 Jun 2007 18:23:58 -0700
Subject: Module system notification mechanism
In-Reply-To: <467B216A.70109@sun.com>
References: <467B216A.70109@sun.com>
Message-ID: <467B24AE.5090602@oracle.com>

Stanley M. Ho wrote:
> Hi JSR 277 experts,
>
> Since we have been discussing some issues around the module instances'
> lifetime lately, I think it's probably a good time to bring up a related
> topic for discussion.
>
> As I reviewed the feedbacks from the EDR comments, from our previous
> discussions, as well as from my discussions with the teams in SE and EE,
> there were a few suggestions related to the module instances' lifetime:
>
> 1. The module system shall provide a way to monitor various events, e.g.
> module initialized, module released, etc.
> 2. The module system shall allow a module to have activator code. The
> activator code would be executed right before the module is fully
> initialized and when the instance is released.
> 3. A module shall have a way to be stopped.
>
> Having these suggestions don't mean we have to do all of them, and I
> would like to get your inputs.
>
>
>  From my perspective, having a way to monitor module system's events
> (i.e. #1) seems very reasonable and useful, especially the use cases are
> very common. In fact, many teams in SE have expressed the needs in
> monitoring the module system's events in their class libraries in some
> degrees, so these libraries would react and behave appropriately. There
> are also other class libraries sitting on top of the JRE that have
> similar needs.
>
> For #2, this is a use case I gathered from EE, and this would be used
> mainly for registering and unregistering services when a module has been
> initialized or is released. Not that I think this is unimportant, but I
> am not yet convinced this is something we need to support directly at
> the module system level. For instance, if the module system notification
> mechanism (i.e. #1) is available, it should be possible for the EE
> server (or other apps that require similar functionality) to build a
> simple activator layer on top of the module system.
>
> For #3, the use case is that some EE servers might want to have the
> ability to "stop" a module by disabling the module's classloader when
> the module instance is released. In general, disabling a classloader is
> an uncommon and dangerous operation to perform, and it also violates the
> current classloading spec. While I agreed we should make this use case
> possible, I don't think this is something we want to push into the
> module system.; I believe there are alternatives we could consider to
> achieve the same result. For example, suppose there are APIs available
> to disable a classloader (might come from the classloading project) and
> the module system notification mechanism (i.e. #1) is available, it
> should be possible for the EE servers to hook into the notification
> mechanism, and disable the specific classloader it wants when a module
> instance is released from the module system.
>
>
> To keep things simple, I suggest we should support #1, and I hope this
> should be sufficient to enable other applications (e.g. EE servers) to
> support #2 and #3. I would like to hear what your thoughts are.

As long as there is some API support for disabling the loader, I'm fine
with this approach.

However, this is not a feature I would ever advocate adding to
java.lang.ClassLoader(!), since it really only makes sense in the
context of a managed loader. In a managed environment with a true
lifecycle, it is important to be able to *find* ongoing uses of a
supposedly stopped module. The only way we've ever been able to do that
is to ensure that any use of such a loader will cause an exception to be
thrown.

We have had great success with this model in finding code that does not
participate correctly in the container lifecycle. The most common case
is threads that aren't stopped when they should be. Leaked references
are another huge headache.

>
> - Stanley
>


From heavy at UNGOVERNED.ORG  Fri Jun 22 05:21:48 2007
From: heavy at UNGOVERNED.ORG (Richard S. Hall)
Date: Fri, 22 Jun 2007 08:21:48 -0400
Subject: Module system notification mechanism
In-Reply-To: <467B216A.70109@sun.com>
References: <467B216A.70109@sun.com>
Message-ID: <467BBEDC.8080801@ungoverned.org>

I think your suggestion is reasonable.

-> richard

Stanley M. Ho wrote:
> Hi JSR 277 experts,
>
> Since we have been discussing some issues around the module instances'
> lifetime lately, I think it's probably a good time to bring up a related
> topic for discussion.
>
> As I reviewed the feedbacks from the EDR comments, from our previous
> discussions, as well as from my discussions with the teams in SE and EE,
> there were a few suggestions related to the module instances' lifetime:
>
> 1. The module system shall provide a way to monitor various events, e.g.
> module initialized, module released, etc.
> 2. The module system shall allow a module to have activator code. The
> activator code would be executed right before the module is fully
> initialized and when the instance is released.
> 3. A module shall have a way to be stopped.
>
> Having these suggestions don't mean we have to do all of them, and I
> would like to get your inputs.
>
>
> From my perspective, having a way to monitor module system's events
> (i.e. #1) seems very reasonable and useful, especially the use cases are
> very common. In fact, many teams in SE have expressed the needs in
> monitoring the module system's events in their class libraries in some
> degrees, so these libraries would react and behave appropriately. There
> are also other class libraries sitting on top of the JRE that have
> similar needs.
>
> For #2, this is a use case I gathered from EE, and this would be used
> mainly for registering and unregistering services when a module has been
> initialized or is released. Not that I think this is unimportant, but I
> am not yet convinced this is something we need to support directly at
> the module system level. For instance, if the module system notification
> mechanism (i.e. #1) is available, it should be possible for the EE
> server (or other apps that require similar functionality) to build a
> simple activator layer on top of the module system.
>
> For #3, the use case is that some EE servers might want to have the
> ability to "stop" a module by disabling the module's classloader when
> the module instance is released. In general, disabling a classloader is
> an uncommon and dangerous operation to perform, and it also violates the
> current classloading spec. While I agreed we should make this use case
> possible, I don't think this is something we want to push into the
> module system.; I believe there are alternatives we could consider to
> achieve the same result. For example, suppose there are APIs available
> to disable a classloader (might come from the classloading project) and
> the module system notification mechanism (i.e. #1) is available, it
> should be possible for the EE servers to hook into the notification
> mechanism, and disable the specific classloader it wants when a module
> instance is released from the module system.
>
>
> To keep things simple, I suggest we should support #1, and I hope this
> should be sufficient to enable other applications (e.g. EE servers) to
> support #2 and #3. I would like to hear what your thoughts are.
>
> - Stanley


From glyn_normington at UK.IBM.COM  Fri Jun 22 06:48:07 2007
From: glyn_normington at UK.IBM.COM (Glyn Normington)
Date: Fri, 22 Jun 2007 14:48:07 +0100
Subject: Module system notification mechanism
In-Reply-To: <467BBEDC.8080801@ungoverned.org>
Message-ID: <OF7A1035B2.CDF4066F-ON80257302.00470CA1-80257302.004BD172@uk.ibm.com>

Hmmm. I thought so too (apart from reinventing the OSGi lifecycle event,
of course), but I wonder if the module system needs to be directly
responsible for driving an activator in case it fails to terminate (in
which case the module needs to enter some kind of error or inactive
state)?

Glyn

"Richard S. Hall" <heavy at UNGOVERNED.ORG> wrote on 22/06/2007 01:21:48 PM:

> I think your suggestion is reasonable.
>
> -> richard
>
> Stanley M. Ho wrote:
> > Hi JSR 277 experts,
> >
> > Since we have been discussing some issues around the module instances'
> > lifetime lately, I think it's probably a good time to bring up a
related
> > topic for discussion.
> >
> > As I reviewed the feedbacks from the EDR comments, from our previous
> > discussions, as well as from my discussions with the teams in SE and
EE,
> > there were a few suggestions related to the module instances'
lifetime:
> >
> > 1. The module system shall provide a way to monitor various events,
e.g.
> > module initialized, module released, etc.
> > 2. The module system shall allow a module to have activator code. The
> > activator code would be executed right before the module is fully
> > initialized and when the instance is released.
> > 3. A module shall have a way to be stopped.
> >
> > Having these suggestions don't mean we have to do all of them, and I
> > would like to get your inputs.
> >
> >
> > From my perspective, having a way to monitor module system's events
> > (i.e. #1) seems very reasonable and useful, especially the use cases
are
> > very common. In fact, many teams in SE have expressed the needs in
> > monitoring the module system's events in their class libraries in some
> > degrees, so these libraries would react and behave appropriately.
There
> > are also other class libraries sitting on top of the JRE that have
> > similar needs.
> >
> > For #2, this is a use case I gathered from EE, and this would be used
> > mainly for registering and unregistering services when a module has
been
> > initialized or is released. Not that I think this is unimportant, but
I
> > am not yet convinced this is something we need to support directly at
> > the module system level. For instance, if the module system
notification
> > mechanism (i.e. #1) is available, it should be possible for the EE
> > server (or other apps that require similar functionality) to build a
> > simple activator layer on top of the module system.
> >
> > For #3, the use case is that some EE servers might want to have the
> > ability to "stop" a module by disabling the module's classloader when
> > the module instance is released. In general, disabling a classloader
is
> > an uncommon and dangerous operation to perform, and it also violates
the
> > current classloading spec. While I agreed we should make this use case
> > possible, I don't think this is something we want to push into the
> > module system.; I believe there are alternatives we could consider to
> > achieve the same result. For example, suppose there are APIs available
> > to disable a classloader (might come from the classloading project)
and
> > the module system notification mechanism (i.e. #1) is available, it
> > should be possible for the EE servers to hook into the notification
> > mechanism, and disable the specific classloader it wants when a module
> > instance is released from the module system.
> >
> >
> > To keep things simple, I suggest we should support #1, and I hope this
> > should be sufficient to enable other applications (e.g. EE servers) to
> > support #2 and #3. I would like to hear what your thoughts are.
> >
> > - Stanley






Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU





-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/attachments/20070622/f16e3a26/attachment.html 

From bryan.atsatt at oracle.com  Fri Jun 22 10:12:48 2007
From: bryan.atsatt at oracle.com (Bryan Atsatt)
Date: Fri, 22 Jun 2007 10:12:48 -0700
Subject: Module system notification mechanism
In-Reply-To: <OF7A1035B2.CDF4066F-ON80257302.00470CA1-80257302.004BD172@uk.ibm.com>
References: <OF7A1035B2.CDF4066F-ON80257302.00470CA1-80257302.004BD172@uk.ibm.com>
Message-ID: <467C0310.8090702@oracle.com>

I think that is part of Stanley's point here: the module system isn't
responsible, as it merely generates the events. Some *other* system
could then take on lifecycle based on these events.

Glyn Normington wrote:
>
> Hmmm. I thought so too (apart from reinventing the OSGi lifecycle event,
> of course), but I wonder if the module system needs to be directly
> responsible for driving an activator in case it fails to terminate (in
> which case the module needs to enter some kind of error or inactive state)?
>
> Glyn
>
> *"Richard S. Hall" <heavy at UNGOVERNED.ORG>* wrote on 22/06/2007 01:21:48 PM:
>
>  > I think your suggestion is reasonable.
>  >
>  > -> richard
>  >
>  > Stanley M. Ho wrote:
>  > > Hi JSR 277 experts,
>  > >
>  > > Since we have been discussing some issues around the module instances'
>  > > lifetime lately, I think it's probably a good time to bring up a
> related
>  > > topic for discussion.
>  > >
>  > > As I reviewed the feedbacks from the EDR comments, from our previous
>  > > discussions, as well as from my discussions with the teams in SE
> and EE,
>  > > there were a few suggestions related to the module instances' lifetime:
>  > >
>  > > 1. The module system shall provide a way to monitor various events,
> e.g.
>  > > module initialized, module released, etc.
>  > > 2. The module system shall allow a module to have activator code. The
>  > > activator code would be executed right before the module is fully
>  > > initialized and when the instance is released.
>  > > 3. A module shall have a way to be stopped.
>  > >
>  > > Having these suggestions don't mean we have to do all of them, and I
>  > > would like to get your inputs.
>  > >
>  > >
>  > > From my perspective, having a way to monitor module system's events
>  > > (i.e. #1) seems very reasonable and useful, especially the use
> cases are
>  > > very common. In fact, many teams in SE have expressed the needs in
>  > > monitoring the module system's events in their class libraries in some
>  > > degrees, so these libraries would react and behave appropriately. There
>  > > are also other class libraries sitting on top of the JRE that have
>  > > similar needs.
>  > >
>  > > For #2, this is a use case I gathered from EE, and this would be used
>  > > mainly for registering and unregistering services when a module has
> been
>  > > initialized or is released. Not that I think this is unimportant, but I
>  > > am not yet convinced this is something we need to support directly at
>  > > the module system level. For instance, if the module system
> notification
>  > > mechanism (i.e. #1) is available, it should be possible for the EE
>  > > server (or other apps that require similar functionality) to build a
>  > > simple activator layer on top of the module system.
>  > >
>  > > For #3, the use case is that some EE servers might want to have the
>  > > ability to "stop" a module by disabling the module's classloader when
>  > > the module instance is released. In general, disabling a classloader is
>  > > an uncommon and dangerous operation to perform, and it also
> violates the
>  > > current classloading spec. While I agreed we should make this use case
>  > > possible, I don't think this is something we want to push into the
>  > > module system.; I believe there are alternatives we could consider to
>  > > achieve the same result. For example, suppose there are APIs available
>  > > to disable a classloader (might come from the classloading project) and
>  > > the module system notification mechanism (i.e. #1) is available, it
>  > > should be possible for the EE servers to hook into the notification
>  > > mechanism, and disable the specific classloader it wants when a module
>  > > instance is released from the module system.
>  > >
>  > >
>  > > To keep things simple, I suggest we should support #1, and I hope this
>  > > should be sufficient to enable other applications (e.g. EE servers) to
>  > > support #2 and #3. I would like to hear what your thoughts are.
>  > >
>  > > - Stanley
>
>
>
> ------------------------------------------------------------------------
>
> /
> /
>
> /Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU/
>
>
>
>
>
>


From glyn_normington at UK.IBM.COM  Mon Jun 25 01:54:54 2007
From: glyn_normington at UK.IBM.COM (Glyn Normington)
Date: Mon, 25 Jun 2007 09:54:54 +0100
Subject: Module system notification mechanism
In-Reply-To: <467C0310.8090702@oracle.com>
Message-ID: <OF690447A6.FB5D18DD-ON80257305.002F4A57-80257305.0030F908@uk.ibm.com>

Yes, but my point was that separating lifecycle out in that way would make
it harder to enforce constraints like "if a module's state is initialised,
the module's activator completed successfully".

JSR 291 solved this problem with additional module states and error
handling. It also supports lazy module initialisation triggered by the
first class load from a module. How much of this should JSR 277 re-invent?

Glyn

Bryan Atsatt <bryan.atsatt at ORACLE.COM> wrote on 22/06/2007 06:12:48 PM:

> I think that is part of Stanley's point here: the module system isn't
> responsible, as it merely generates the events. Some *other* system
> could then take on lifecycle based on these events.
>
> Glyn Normington wrote:
> >
> > Hmmm. I thought so too (apart from reinventing the OSGi lifecycle
event,
> > of course), but I wonder if the module system needs to be directly
> > responsible for driving an activator in case it fails to terminate (in
> > which case the module needs to enter some kind of error or inactive
state)?
> >
> > Glyn
> >
> > *"Richard S. Hall" <heavy at UNGOVERNED.ORG>* wrote on 22/06/2007
01:21:48 PM:
> >
> >  > I think your suggestion is reasonable.
> >  >
> >  > -> richard
> >  >
> >  > Stanley M. Ho wrote:
> >  > > Hi JSR 277 experts,
> >  > >
> >  > > Since we have been discussing some issues around the module
instances'
> >  > > lifetime lately, I think it's probably a good time to bring up a
> > related
> >  > > topic for discussion.
> >  > >
> >  > > As I reviewed the feedbacks from the EDR comments, from our
previous
> >  > > discussions, as well as from my discussions with the teams in SE
> > and EE,
> >  > > there were a few suggestions related to the module
instances'lifetime:
> >  > >
> >  > > 1. The module system shall provide a way to monitor various
events,
> > e.g.
> >  > > module initialized, module released, etc.
> >  > > 2. The module system shall allow a module to have activator code.
The
> >  > > activator code would be executed right before the module is fully
> >  > > initialized and when the instance is released.
> >  > > 3. A module shall have a way to be stopped.
> >  > >
> >  > > Having these suggestions don't mean we have to do all of them,
and I
> >  > > would like to get your inputs.
> >  > >
> >  > >
> >  > > From my perspective, having a way to monitor module system's
events
> >  > > (i.e. #1) seems very reasonable and useful, especially the use
> > cases are
> >  > > very common. In fact, many teams in SE have expressed the needs
in
> >  > > monitoring the module system's events in their class libraries in
some
> >  > > degrees, so these libraries would react and behave appropriately.
There
> >  > > are also other class libraries sitting on top of the JRE that
have
> >  > > similar needs.
> >  > >
> >  > > For #2, this is a use case I gathered from EE, and this would be
used
> >  > > mainly for registering and unregistering services when a module
has
> > been
> >  > > initialized or is released. Not that I think this is unimportant,
but I
> >  > > am not yet convinced this is something we need to support
directly at
> >  > > the module system level. For instance, if the module system
> > notification
> >  > > mechanism (i.e. #1) is available, it should be possible for the
EE
> >  > > server (or other apps that require similar functionality) to
build a
> >  > > simple activator layer on top of the module system.
> >  > >
> >  > > For #3, the use case is that some EE servers might want to have
the
> >  > > ability to "stop" a module by disabling the module's classloader
when
> >  > > the module instance is released. In general, disabling a
classloader is
> >  > > an uncommon and dangerous operation to perform, and it also
> > violates the
> >  > > current classloading spec. While I agreed we should make this use
case
> >  > > possible, I don't think this is something we want to push into
the
> >  > > module system.; I believe there are alternatives we could
consider to
> >  > > achieve the same result. For example, suppose there are APIs
available
> >  > > to disable a classloader (might come from the classloading
project) and
> >  > > the module system notification mechanism (i.e. #1) is available,
it
> >  > > should be possible for the EE servers to hook into the
notification
> >  > > mechanism, and disable the specific classloader it wants when a
module
> >  > > instance is released from the module system.
> >  > >
> >  > >
> >  > > To keep things simple, I suggest we should support #1, and I hope
this
> >  > > should be sufficient to enable other applications (e.g. EE
servers) to
> >  > > support #2 and #3. I would like to hear what your thoughts are.
> >  > >
> >  > > - Stanley
> >
> >
> >
> >
------------------------------------------------------------------------
> >
> > /
> > /
> >
> > /Unless stated otherwise above:
> > IBM United Kingdom Limited - Registered in England and Wales with
number
> > 741598.
> > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU/
> >
> >
> >
> >
> >
> >






Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU





-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/attachments/20070625/fbb1b1e7/attachment.html 

From bryan.atsatt at oracle.com  Mon Jun 25 10:00:12 2007
From: bryan.atsatt at oracle.com (Bryan Atsatt)
Date: Mon, 25 Jun 2007 10:00:12 -0700
Subject: Module system notification mechanism
In-Reply-To: <OF690447A6.FB5D18DD-ON80257305.002F4A57-80257305.0030F908@uk.ibm.com>
References: <OF690447A6.FB5D18DD-ON80257305.002F4A57-80257305.0030F908@uk.ibm.com>
Message-ID: <467FF49C.1000002@oracle.com>

It seems simple enough to put the module in a failed state if a
module-init event handler throws an exception. Is this not sufficient?

The lazy init feature sounds like a nice optimization, but doesn't
strike me as a must-have. Have you seen use-cases that require it? Are
they init order issues?


(I am sensitive to your meta point here, but... If 277 is going to
exist, there are some rather obvious, desirable features which it will
need, just as OSGi needed them. I _don't_ like the fact that we are
re-inventing rather than re-using OSGi, but that is how it has played
out, despite all protestations. As long as we can still attain the goal
of meaningful interoperation, it seems irresponsible to leave out
features like this. And yes, we must nail down the interoperation model
before we go much/any farther.)

Glyn Normington wrote:
>
> Yes, but my point was that separating lifecycle out in that way would
> make it harder to enforce constraints like "if a module's state is
> initialised, the module's activator completed successfully".
>
> JSR 291 solved this problem with additional module states and error
> handling. It also supports lazy module initialisation triggered by the
> first class load from a module. How much of this should JSR 277 re-invent?
>
> Glyn
>
> *Bryan Atsatt <bryan.atsatt at ORACLE.COM>* wrote on 22/06/2007 06:12:48 PM:
>
>  > I think that is part of Stanley's point here: the module system isn't
>  > responsible, as it merely generates the events. Some *other* system
>  > could then take on lifecycle based on these events.
>  >
>  > Glyn Normington wrote:
>  > >
>  > > Hmmm. I thought so too (apart from reinventing the OSGi lifecycle
> event,
>  > > of course), but I wonder if the module system needs to be directly
>  > > responsible for driving an activator in case it fails to terminate (in
>  > > which case the module needs to enter some kind of error or inactive
> state)?
>  > >
>  > > Glyn
>  > >
>  > > *"Richard S. Hall" <heavy at UNGOVERNED.ORG>* wrote on 22/06/2007
> 01:21:48 PM:
>  > >
>  > >  > I think your suggestion is reasonable.
>  > >  >
>  > >  > -> richard
>  > >  >
>  > >  > Stanley M. Ho wrote:
>  > >  > > Hi JSR 277 experts,
>  > >  > >
>  > >  > > Since we have been discussing some issues around the module
> instances'
>  > >  > > lifetime lately, I think it's probably a good time to bring up a
>  > > related
>  > >  > > topic for discussion.
>  > >  > >
>  > >  > > As I reviewed the feedbacks from the EDR comments, from our
> previous
>  > >  > > discussions, as well as from my discussions with the teams in SE
>  > > and EE,
>  > >  > > there were a few suggestions related to the module
> instances'lifetime:
>  > >  > >
>  > >  > > 1. The module system shall provide a way to monitor various
> events,
>  > > e.g.
>  > >  > > module initialized, module released, etc.
>  > >  > > 2. The module system shall allow a module to have activator
> code. The
>  > >  > > activator code would be executed right before the module is fully
>  > >  > > initialized and when the instance is released.
>  > >  > > 3. A module shall have a way to be stopped.
>  > >  > >
>  > >  > > Having these suggestions don't mean we have to do all of them,
> and I
>  > >  > > would like to get your inputs.
>  > >  > >
>  > >  > >
>  > >  > > From my perspective, having a way to monitor module system's
> events
>  > >  > > (i.e. #1) seems very reasonable and useful, especially the use
>  > > cases are
>  > >  > > very common. In fact, many teams in SE have expressed the needs in
>  > >  > > monitoring the module system's events in their class libraries
> in some
>  > >  > > degrees, so these libraries would react and behave
> appropriately. There
>  > >  > > are also other class libraries sitting on top of the JRE that have
>  > >  > > similar needs.
>  > >  > >
>  > >  > > For #2, this is a use case I gathered from EE, and this would
> be used
>  > >  > > mainly for registering and unregistering services when a
> module has
>  > > been
>  > >  > > initialized or is released. Not that I think this is
> unimportant, but I
>  > >  > > am not yet convinced this is something we need to support
> directly at
>  > >  > > the module system level. For instance, if the module system
>  > > notification
>  > >  > > mechanism (i.e. #1) is available, it should be possible for the EE
>  > >  > > server (or other apps that require similar functionality) to
> build a
>  > >  > > simple activator layer on top of the module system.
>  > >  > >
>  > >  > > For #3, the use case is that some EE servers might want to
> have the
>  > >  > > ability to "stop" a module by disabling the module's
> classloader when
>  > >  > > the module instance is released. In general, disabling a
> classloader is
>  > >  > > an uncommon and dangerous operation to perform, and it also
>  > > violates the
>  > >  > > current classloading spec. While I agreed we should make this
> use case
>  > >  > > possible, I don't think this is something we want to push into the
>  > >  > > module system.; I believe there are alternatives we could
> consider to
>  > >  > > achieve the same result. For example, suppose there are APIs
> available
>  > >  > > to disable a classloader (might come from the classloading
> project) and
>  > >  > > the module system notification mechanism (i.e. #1) is
> available, it
>  > >  > > should be possible for the EE servers to hook into the
> notification
>  > >  > > mechanism, and disable the specific classloader it wants when
> a module
>  > >  > > instance is released from the module system.
>  > >  > >
>  > >  > >
>  > >  > > To keep things simple, I suggest we should support #1, and I
> hope this
>  > >  > > should be sufficient to enable other applications (e.g. EE
> servers) to
>  > >  > > support #2 and #3. I would like to hear what your thoughts are.
>  > >  > >
>  > >  > > - Stanley
>  > >
>  > >
>  > >
>  > >
> ------------------------------------------------------------------------
>  > >
>  > > /
>  > > /
>  > >
>  > > /Unless stated otherwise above:
>  > > IBM United Kingdom Limited - Registered in England and Wales with
> number
>  > > 741598.
>  > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire
> PO6 3AU/
>  > >
>  > >
>  > >
>  > >
>  > >
>  > >
>
>
>
> ------------------------------------------------------------------------
>
> /
> /
>
> /Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU/
>
>
>
>
>
>


From Stanley.Ho at sun.com  Mon Jun 25 13:54:15 2007
From: Stanley.Ho at sun.com (Stanley M. Ho)
Date: Mon, 25 Jun 2007 13:54:15 -0700
Subject: Module system notification mechanism
In-Reply-To: <467B24AE.5090602@oracle.com>
References: <467B216A.70109@sun.com> <467B24AE.5090602@oracle.com>
Message-ID: <46802B77.4050909@sun.com>

Hi Bryan,

Bryan Atsatt wrote:
> As long as there is some API support for disabling the loader, I'm fine
> with this approach.

Good.

> However, this is not a feature I would ever advocate adding to
> java.lang.ClassLoader(!), since it really only makes sense in the
> context of a managed loader. In a managed environment with a true
> lifecycle, it is important to be able to *find* ongoing uses of a
> supposedly stopped module. The only way we've ever been able to do that
> is to ensure that any use of such a loader will cause an exception to be
> thrown.
>
> We have had great success with this model in finding code that does not
> participate correctly in the container lifecycle. The most common case
> is threads that aren't stopped when they should be. Leaked references
> are another huge headache.

I also don't think it should be in java.lang.ClassLoader, but we should
leave it up to the classloading folks to decide. In case the
functionality ends up in a public API in a vendor specific package (e.g.
com.sun.* or something similar), I think this should still work for you.

- Stanley


From Stanley.Ho at sun.com  Mon Jun 25 14:04:08 2007
From: Stanley.Ho at sun.com (Stanley M. Ho)
Date: Mon, 25 Jun 2007 14:04:08 -0700
Subject: Module system notification mechanism
In-Reply-To: <OF690447A6.FB5D18DD-ON80257305.002F4A57-80257305.0030F908@uk.ibm.com>
References: <OF690447A6.FB5D18DD-ON80257305.002F4A57-80257305.0030F908@uk.ibm.com>
Message-ID: <46802DC8.8040704@sun.com>

Hi Glyn,

Glyn Normington wrote:
>
> Yes, but my point was that separating lifecycle out in that way would
> make it harder to enforce constraints like "if a module's state is
> initialised, the module's activator completed successfully".

The mechanism I suggested is simply for informing the application that
something has happened. This is a notification only and does not change
the state of the module system in any way.

If we want to enforce constraints like you mentioned, one approach is to
execute the activator code in the custom import policy. If this is not
sufficient, then we'll probably need a different mechanism to handle
this use case.

- Stanley


From glyn_normington at UK.IBM.COM  Tue Jun 26 01:53:51 2007
From: glyn_normington at UK.IBM.COM (Glyn Normington)
Date: Tue, 26 Jun 2007 09:53:51 +0100
Subject: Module system notification mechanism
In-Reply-To: <46802DC8.8040704@sun.com>
Message-ID: <OFA89CDC08.EE5BAC69-ON80257306.002F09E3-80257306.0030E070@uk.ibm.com>

Hi Stanley

"Stanley M. Ho" <Stanley.Ho at SUN.COM> wrote on 25/06/2007 10:04:08 PM:

> Hi Glyn,
>
> Glyn Normington wrote:
> >
> > Yes, but my point was that separating lifecycle out in that way would
> > make it harder to enforce constraints like "if a module's state is
> > initialised, the module's activator completed successfully".
>
> The mechanism I suggested is simply for informing the application that
> something has happened. This is a notification only and does not change
> the state of the module system in any way.

That's what I expected.

>
> If we want to enforce constraints like you mentioned, one approach is to
> execute the activator code in the custom import policy. If this is not
> sufficient, then we'll probably need a different mechanism to handle
> this use case.

I don't think calling the activator from the custom import policy would
work.

The main problem is that it would require any module needing an activator
to supply a custom import policy, which we want to avoid in most cases.

Also, the custom import policy runs in a peculiar 'half-resolved' state
(the module is unresolved at the start and hopefully fully resolved at the
end, so on average, it's half-resolved ;-) ) and we wouldn't want
application code proper, such as an activator, to have to support this
state.

>
> - Stanley

Glyn






Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU





-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/attachments/20070626/e0c8f649/attachment.html 

From glyn_normington at UK.IBM.COM  Tue Jun 26 02:17:29 2007
From: glyn_normington at UK.IBM.COM (Glyn Normington)
Date: Tue, 26 Jun 2007 10:17:29 +0100
Subject: Module system notification mechanism
In-Reply-To: <467FF49C.1000002@oracle.com>
Message-ID: <OF10E06F12.E5C86EC2-ON80257306.00314A67-80257306.00330A21@uk.ibm.com>

Bryan Atsatt <bryan.atsatt at ORACLE.COM> wrote on 25/06/2007 06:00:12 PM:

> It seems simple enough to put the module in a failed state if a
> module-init event handler throws an exception. Is this not sufficient?

No, I don't think so.

What state would a module be in when its module-init event is published?
If potential consumers of the module listen for the event, then they'll be
interested in the module being initialised (ideally meaning its activator
has already completed successfully) and ready for use. But the module
can't be in the initalised state if it might transition to a failed state
if one of the event handlers calls the module's activator and that fails.

Also, why should the failure of a 3rd party listener put the module in a
failed state? One buggy or malicious module could impact the whole system.

>
> The lazy init feature sounds like a nice optimization, but doesn't
> strike me as a must-have. Have you seen use-cases that require it? Are
> they init order issues?

I agree it's not a must have as JSR 291 handles that.

>
>
> (I am sensitive to your meta point here, but... If 277 is going to
> exist, there are some rather obvious, desirable features which it will
> need, just as OSGi needed them. I _don't_ like the fact that we are
> re-inventing rather than re-using OSGi, but that is how it has played
> out, despite all protestations. As long as we can still attain the goal
> of meaningful interoperation, it seems irresponsible to leave out
> features like this. And yes, we must nail down the interoperation model
> before we go much/any farther.)

We *have* to find some way of JSR 277 being able to 'delegate' some
responsibilities to JSR 291 rather than having to reproduce the whole
shooting match which would literally take years (to produce something as
good). I'd like to see JSR 277 providing some kind of basic support -- at
least the repository -- with JSR 291 providing the bells and whistles.

Glyn

>
> Glyn Normington wrote:
> >
> > Yes, but my point was that separating lifecycle out in that way would
> > make it harder to enforce constraints like "if a module's state is
> > initialised, the module's activator completed successfully".
> >
> > JSR 291 solved this problem with additional module states and error
> > handling. It also supports lazy module initialisation triggered by the
> > first class load from a module. How much of this should JSR 277
re-invent?
> >
> > Glyn
> >
> > *Bryan Atsatt <bryan.atsatt at ORACLE.COM>* wrote on 22/06/2007 06:12:48
PM:
> >
> >  > I think that is part of Stanley's point here: the module system
isn't
> >  > responsible, as it merely generates the events. Some *other* system
> >  > could then take on lifecycle based on these events.
> >  >
> >  > Glyn Normington wrote:
> >  > >
> >  > > Hmmm. I thought so too (apart from reinventing the OSGi lifecycle
> > event,
> >  > > of course), but I wonder if the module system needs to be
directly
> >  > > responsible for driving an activator in case it fails to
terminate (in
> >  > > which case the module needs to enter some kind of error or
inactive
> > state)?
> >  > >
> >  > > Glyn
> >  > >
> >  > > *"Richard S. Hall" <heavy at UNGOVERNED.ORG>* wrote on 22/06/2007
> > 01:21:48 PM:
> >  > >
> >  > >  > I think your suggestion is reasonable.
> >  > >  >
> >  > >  > -> richard
> >  > >  >
> >  > >  > Stanley M. Ho wrote:
> >  > >  > > Hi JSR 277 experts,
> >  > >  > >
> >  > >  > > Since we have been discussing some issues around the module
> > instances'
> >  > >  > > lifetime lately, I think it's probably a good time to bring
up a
> >  > > related
> >  > >  > > topic for discussion.
> >  > >  > >
> >  > >  > > As I reviewed the feedbacks from the EDR comments, from our
> > previous
> >  > >  > > discussions, as well as from my discussions with the teams
in SE
> >  > > and EE,
> >  > >  > > there were a few suggestions related to the module
> > instances'lifetime:
> >  > >  > >
> >  > >  > > 1. The module system shall provide a way to monitor various
> > events,
> >  > > e.g.
> >  > >  > > module initialized, module released, etc.
> >  > >  > > 2. The module system shall allow a module to have activator
> > code. The
> >  > >  > > activator code would be executed right before the module is
fully
> >  > >  > > initialized and when the instance is released.
> >  > >  > > 3. A module shall have a way to be stopped.
> >  > >  > >
> >  > >  > > Having these suggestions don't mean we have to do all of
them,
> > and I
> >  > >  > > would like to get your inputs.
> >  > >  > >
> >  > >  > >
> >  > >  > > From my perspective, having a way to monitor module system's
> > events
> >  > >  > > (i.e. #1) seems very reasonable and useful, especially the
use
> >  > > cases are
> >  > >  > > very common. In fact, many teams in SE have expressed the
needs in
> >  > >  > > monitoring the module system's events in their class
libraries
> > in some
> >  > >  > > degrees, so these libraries would react and behave
> > appropriately. There
> >  > >  > > are also other class libraries sitting on top of the JREthat
have
> >  > >  > > similar needs.
> >  > >  > >
> >  > >  > > For #2, this is a use case I gathered from EE, and this
would
> > be used
> >  > >  > > mainly for registering and unregistering services when a
> > module has
> >  > > been
> >  > >  > > initialized or is released. Not that I think this is
> > unimportant, but I
> >  > >  > > am not yet convinced this is something we need to support
> > directly at
> >  > >  > > the module system level. For instance, if the module system
> >  > > notification
> >  > >  > > mechanism (i.e. #1) is available, it should be possible for
the EE
> >  > >  > > server (or other apps that require similar functionality) to
> > build a
> >  > >  > > simple activator layer on top of the module system.
> >  > >  > >
> >  > >  > > For #3, the use case is that some EE servers might want to
> > have the
> >  > >  > > ability to "stop" a module by disabling the module's
> > classloader when
> >  > >  > > the module instance is released. In general, disabling a
> > classloader is
> >  > >  > > an uncommon and dangerous operation to perform, and it also
> >  > > violates the
> >  > >  > > current classloading spec. While I agreed we should make
this
> > use case
> >  > >  > > possible, I don't think this is something we want to push
into the
> >  > >  > > module system.; I believe there are alternatives we could
> > consider to
> >  > >  > > achieve the same result. For example, suppose there are APIs
> > available
> >  > >  > > to disable a classloader (might come from the classloading
> > project) and
> >  > >  > > the module system notification mechanism (i.e. #1) is
> > available, it
> >  > >  > > should be possible for the EE servers to hook into the
> > notification
> >  > >  > > mechanism, and disable the specific classloader it wants
when
> > a module
> >  > >  > > instance is released from the module system.
> >  > >  > >
> >  > >  > >
> >  > >  > > To keep things simple, I suggest we should support #1, and I
> > hope this
> >  > >  > > should be sufficient to enable other applications (e.g. EE
> > servers) to
> >  > >  > > support #2 and #3. I would like to hear what your thoughts
are.
> >  > >  > >
> >  > >  > > - Stanley
> >  > >
> >  > >
> >  > >
> >  > >
> >
------------------------------------------------------------------------
> >  > >
> >  > > /
> >  > > /
> >  > >
> >  > > /Unless stated otherwise above:
> >  > > IBM United Kingdom Limited - Registered in England and Wales with
> > number
> >  > > 741598.
> >  > > Registered office: PO Box 41, North Harbour, Portsmouth,
Hampshire
> > PO6 3AU/
> >  > >
> >  > >
> >  > >
> >  > >
> >  > >
> >  > >
> >
> >
> >
> >
------------------------------------------------------------------------
> >
> > /
> > /
> >
> > /Unless stated otherwise above:
> > IBM United Kingdom Limited - Registered in England and Wales with
number
> > 741598.
> > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU/
> >
> >
> >
> >
> >
> >






Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU





-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/attachments/20070626/77af7cd1/attachment.html 

From Stanley.Ho at sun.com  Thu Jun 28 00:38:55 2007
From: Stanley.Ho at sun.com (Stanley M. Ho)
Date: Thu, 28 Jun 2007 00:38:55 -0700
Subject: Early snapshot of JSR 277 and 294 implementation
Message-ID: <4683658F.4000500@sun.com>

Hi experts and observers,

We have just made an early snapshot of the JSR 277 and 294
implementation available through the Modules project on openjdk:

    http://openjdk.java.net/projects/modules/

This snapshot basically covers the basic features described in the
updated specification in JSR 277, and the reflective APIs defined in JSR
294. You should expect the snapshot will be updated periodically going
forwards, as we continue to evolve the implementation and adding many
more functionalities in the next few months.

We will continue to focus our discussions in this EG mailing list for
the JSR 277 specification. For those of you interested in discussing the
implementation in the Modules project, we have made two discussion lists
available to the Java community:

* modules-dev at openjdk.java.net
Technical discussion about the implementation of the Modules project

* modules-discuss at openjdk.java.net
General discussion of the Modules project and how to use modules

If you are interested in discussing the Modules project with its
developers, or in discussing how to use JSR 277 and JSR 294, be a
subscriber today!

- Stanley


From bryan.atsatt at oracle.com  Thu Jun 28 12:10:04 2007
From: bryan.atsatt at oracle.com (Bryan Atsatt)
Date: Thu, 28 Jun 2007 12:10:04 -0700
Subject: Module system notification mechanism
In-Reply-To: <OF10E06F12.E5C86EC2-ON80257306.00314A67-80257306.00330A21@uk.ibm.com>
References: <OF10E06F12.E5C86EC2-ON80257306.00314A67-80257306.00330A21@uk.ibm.com>
Message-ID: <4684078C.6030907@oracle.com>

Glyn Normington wrote:
>
> *Bryan Atsatt <bryan.atsatt at ORACLE.COM>* wrote on 25/06/2007 06:00:12 PM:
>
>  > It seems simple enough to put the module in a failed state if a
>  > module-init event handler throws an exception. Is this not sufficient?
>
> No, I don't think so.
>
> What state would a module be in when its module-init event is published?
> If potential consumers of the module listen for the event, then they'll
> be interested in the module being initialised (ideally meaning its
> activator has already completed successfully) and ready for use. But the
> module can't be in the initalised state if it might transition to a
> failed state if one of the event handlers calls the module's activator
> and that fails.
>
> Also, why should the failure of a 3rd party listener put the module in a
> failed state? One buggy or malicious module could impact the whole system.

Whoops. Yes, of course you are correct. I was not paying quite enough
attention here, and thinking of one of our own event models, in which
only the "module" itself can receive events. And that is very much like
the OSGi bundle activator model.

Stanley, I assume you are proposing a general mechanism in which any
code can register listeners? If so, then I have to agree with Glyn that
this is not sufficient to implement a module "activator".

>
>  >
>  > The lazy init feature sounds like a nice optimization, but doesn't
>  > strike me as a must-have. Have you seen use-cases that require it? Are
>  > they init order issues?
>
> I agree it's not a must have as JSR 291 handles that.
>
>  >
>  >
>  > (I am sensitive to your meta point here, but... If 277 is going to
>  > exist, there are some rather obvious, desirable features which it will
>  > need, just as OSGi needed them. I _don't_ like the fact that we are
>  > re-inventing rather than re-using OSGi, but that is how it has played
>  > out, despite all protestations. As long as we can still attain the goal
>  > of meaningful interoperation, it seems irresponsible to leave out
>  > features like this. And yes, we must nail down the interoperation model
>  > before we go much/any farther.)
>
> We *have* to find some way of JSR 277 being able to 'delegate' some
> responsibilities to JSR 291 rather than having to reproduce the whole
> shooting match which would literally take years (to produce something as
> good). I'd like to see JSR 277 providing some kind of basic support --
> at least the repository -- with JSR 291 providing the bells and whistles.
>
> Glyn
>
>  >
>  > Glyn Normington wrote:
>  > >
>  > > Yes, but my point was that separating lifecycle out in that way would
>  > > make it harder to enforce constraints like "if a module's state is
>  > > initialised, the module's activator completed successfully".
>  > >
>  > > JSR 291 solved this problem with additional module states and error
>  > > handling. It also supports lazy module initialisation triggered by the
>  > > first class load from a module. How much of this should JSR 277
> re-invent?
>  > >
>  > > Glyn
>  > >
>  > > *Bryan Atsatt <bryan.atsatt at ORACLE.COM>* wrote on 22/06/2007
> 06:12:48 PM:
>  > >
>  > >  > I think that is part of Stanley's point here: the module system
> isn't
>  > >  > responsible, as it merely generates the events. Some *other* system
>  > >  > could then take on lifecycle based on these events.
>  > >  >
>  > >  > Glyn Normington wrote:
>  > >  > >
>  > >  > > Hmmm. I thought so too (apart from reinventing the OSGi lifecycle
>  > > event,
>  > >  > > of course), but I wonder if the module system needs to be directly
>  > >  > > responsible for driving an activator in case it fails to
> terminate (in
>  > >  > > which case the module needs to enter some kind of error or
> inactive
>  > > state)?
>  > >  > >
>  > >  > > Glyn
>  > >  > >
>  > >  > > *"Richard S. Hall" <heavy at UNGOVERNED.ORG>* wrote on 22/06/2007
>  > > 01:21:48 PM:
>  > >  > >
>  > >  > >  > I think your suggestion is reasonable.
>  > >  > >  >
>  > >  > >  > -> richard
>  > >  > >  >
>  > >  > >  > Stanley M. Ho wrote:
>  > >  > >  > > Hi JSR 277 experts,
>  > >  > >  > >
>  > >  > >  > > Since we have been discussing some issues around the module
>  > > instances'
>  > >  > >  > > lifetime lately, I think it's probably a good time to
> bring up a
>  > >  > > related
>  > >  > >  > > topic for discussion.
>  > >  > >  > >
>  > >  > >  > > As I reviewed the feedbacks from the EDR comments, from our
>  > > previous
>  > >  > >  > > discussions, as well as from my discussions with the
> teams in SE
>  > >  > > and EE,
>  > >  > >  > > there were a few suggestions related to the module
>  > > instances'lifetime:
>  > >  > >  > >
>  > >  > >  > > 1. The module system shall provide a way to monitor various
>  > > events,
>  > >  > > e.g.
>  > >  > >  > > module initialized, module released, etc.
>  > >  > >  > > 2. The module system shall allow a module to have activator
>  > > code. The
>  > >  > >  > > activator code would be executed right before the module
> is fully
>  > >  > >  > > initialized and when the instance is released.
>  > >  > >  > > 3. A module shall have a way to be stopped.
>  > >  > >  > >
>  > >  > >  > > Having these suggestions don't mean we have to do all of
> them,
>  > > and I
>  > >  > >  > > would like to get your inputs.
>  > >  > >  > >
>  > >  > >  > >
>  > >  > >  > > From my perspective, having a way to monitor module system's
>  > > events
>  > >  > >  > > (i.e. #1) seems very reasonable and useful, especially
> the use
>  > >  > > cases are
>  > >  > >  > > very common. In fact, many teams in SE have expressed the
> needs in
>  > >  > >  > > monitoring the module system's events in their class
> libraries
>  > > in some
>  > >  > >  > > degrees, so these libraries would react and behave
>  > > appropriately. There
>  > >  > >  > > are also other class libraries sitting on top of the
> JREthat have
>  > >  > >  > > similar needs.
>  > >  > >  > >
>  > >  > >  > > For #2, this is a use case I gathered from EE, and this would
>  > > be used
>  > >  > >  > > mainly for registering and unregistering services when a
>  > > module has
>  > >  > > been
>  > >  > >  > > initialized or is released. Not that I think this is
>  > > unimportant, but I
>  > >  > >  > > am not yet convinced this is something we need to support
>  > > directly at
>  > >  > >  > > the module system level. For instance, if the module system
>  > >  > > notification
>  > >  > >  > > mechanism (i.e. #1) is available, it should be possible
> for the EE
>  > >  > >  > > server (or other apps that require similar functionality) to
>  > > build a
>  > >  > >  > > simple activator layer on top of the module system.
>  > >  > >  > >
>  > >  > >  > > For #3, the use case is that some EE servers might want to
>  > > have the
>  > >  > >  > > ability to "stop" a module by disabling the module's
>  > > classloader when
>  > >  > >  > > the module instance is released. In general, disabling a
>  > > classloader is
>  > >  > >  > > an uncommon and dangerous operation to perform, and it also
>  > >  > > violates the
>  > >  > >  > > current classloading spec. While I agreed we should make this
>  > > use case
>  > >  > >  > > possible, I don't think this is something we want to push
> into the
>  > >  > >  > > module system.; I believe there are alternatives we could
>  > > consider to
>  > >  > >  > > achieve the same result. For example, suppose there are APIs
>  > > available
>  > >  > >  > > to disable a classloader (might come from the classloading
>  > > project) and
>  > >  > >  > > the module system notification mechanism (i.e. #1) is
>  > > available, it
>  > >  > >  > > should be possible for the EE servers to hook into the
>  > > notification
>  > >  > >  > > mechanism, and disable the specific classloader it wants when
>  > > a module
>  > >  > >  > > instance is released from the module system.
>  > >  > >  > >
>  > >  > >  > >
>  > >  > >  > > To keep things simple, I suggest we should support #1, and I
>  > > hope this
>  > >  > >  > > should be sufficient to enable other applications (e.g. EE
>  > > servers) to
>  > >  > >  > > support #2 and #3. I would like to hear what your
> thoughts are.
>  > >  > >  > >
>  > >  > >  > > - Stanley
>  > >  > >
>  > >  > >
>  > >  > >
>  > >  > >
>  > >
> ------------------------------------------------------------------------
>  > >  > >
>  > >  > > /
>  > >  > > /
>  > >  > >
>  > >  > > /Unless stated otherwise above:
>  > >  > > IBM United Kingdom Limited - Registered in England and Wales with
>  > > number
>  > >  > > 741598.
>  > >  > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire
>  > > PO6 3AU/
>  > >  > >
>  > >  > >
>  > >  > >
>  > >  > >
>  > >  > >
>  > >  > >
>  > >
>  > >
>  > >
>  > >
> ------------------------------------------------------------------------
>  > >
>  > > /
>  > > /
>  > >
>  > > /Unless stated otherwise above:
>  > > IBM United Kingdom Limited - Registered in England and Wales with
> number
>  > > 741598.
>  > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire
> PO6 3AU/
>  > >
>  > >
>  > >
>  > >
>  > >
>  > >
>
>
>
> ------------------------------------------------------------------------
>
> /
> /
>
> /Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU/
>
>
>
>
>
>


From Stanley.Ho at sun.com  Thu Jun 28 16:21:03 2007
From: Stanley.Ho at sun.com (Stanley M. Ho)
Date: Thu, 28 Jun 2007 16:21:03 -0700
Subject: Module system notification mechanism
In-Reply-To: <4684078C.6030907@oracle.com>
References: <OF10E06F12.E5C86EC2-ON80257306.00314A67-80257306.00330A21@uk.ibm.com>
	<4684078C.6030907@oracle.com>
Message-ID: <4684425F.4060308@sun.com>

Bryan Atsatt wrote:
> Stanley, I assume you are proposing a general mechanism in which any
> code can register listeners? If so, then I have to agree with Glyn that
> this is not sufficient to implement a module "activator".

The mechanism I have in mind can be registered only by code that have
the appropriate security permissions, and untrusted code in the sandbox
won't be able to listen to the notification.

I am still a bit puzzled by why the module's state needs to be coupled
directly to the result from the module's activator because this looks to
me more like a solution to a problem than a requirement. For that model
to work, I think there are couple underlying assumptions, but they might
not be valid in our case:

1. If a module's activator is executed before the module is fully
initialized, it can safely register something into some service lookup
mechanism in the system.
2. The service lookup mechanism will make sure the registered thing from
that module is not usable by other modules unless that module becomes
fully initialized.
3. If the module's activator fails to execute when the module is being
initialized, the module system knows how to work with the service lookup
mechanism to undo all the unregisterations automatically and correctly.

In this case, the module system probably needs to be highly integrated
with the service lookup mechanism, and that mechanism should be the only
one used by all the modules' activators. However, in practice, multiple
service lookup mechanisms exist and can be used in the Java platform,
and I don't think we should expect the module system will know how to
work with these arbitrary mechanisms to provide the suggested module's
state semantic in conjunction with the module's activator. Also, from
our previous discussions, I think our consensus is to avoid pushing the
service mechanism down to the module layer if possible.

If the use case is that the module's activator should be called by the
container when the module is fully initialized or is released, the
notification mechanism should be sufficient for this purpose. If the use
case is that a module instance should be released if its module's
activator fails to start, the notification mechanism should still be
sufficient, assuming the container will call
ModuleSystem.releaseModule() after the container has received the module
initialized notification but fails to start the activator. (Yes, there
are side effects with releaseModule() as we discussed before, but it's a
problem the container can probably deal with.)

Is there any other use case that you think is important to consider but
not allowed by this notification mechanism?

- Stanley