From Stanley.Ho at Sun.COM Mon May 14 12:17:01 2007 From: Stanley.Ho at Sun.COM (Stanley M. Ho) Date: Mon, 14 May 2007 12:17:01 -0700 Subject: Reminder: Updated JSR-277 specification (04/19/2007) In-Reply-To: <46319FA2.10102@sun.com> References: <46282FC0.1040100@sun.com> <46319FA2.10102@sun.com> Message-ID: <4648B5AD.7030801@sun.com> Hi JSR 277 experts, If you still have any input on the updated early draft specification, please send them in as soon as possible. Thanks, - Stanley From Stanley.Ho at sun.com Mon May 14 12:36:00 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Mon, 14 May 2007 12:36:00 -0700 Subject: JSR 277 EG observer mailing list In-Reply-To: <46412161.9020902@sun.com> References: <46400CFB.1020004@sun.com> <46412161.9020902@sun.com> Message-ID: <4648BA20.5070206@sun.com> Hi JSR 277 experts, Just an update. This EG mailing list has been hooked up with the observer list over the weekend, and you will find the public mailing list archive here: http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/ - Stanley From Stanley.Ho at sun.com Mon May 14 14:54:43 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Mon, 14 May 2007 14:54:43 -0700 Subject: Reminder: i18n strawman Message-ID: <4648DAA3.9030502@sun.com> Hi JSR 277 experts, This is a kindly reminder. If you have inputs on the i18n strawman, please send them in by 5/21/2007. Even if you have reviewed the strawman and have no comments, it would still be good to let us know. Thanks, - Stanley -------- Original Message -------- Subject: Strawman: i18n support Date: Mon, 07 May 2007 17:09:41 -0700 From: Stanley M. Ho Reply-To: Java Community Process JSR #277 Expert List Organization: Sun Microsystems, Inc. To: JSR-277-EG at JCP.ORG Hi JSR 277 experts, I've been working with Masayoshi Okutsu (i18n lead) and Naoto Sato in the Java SE team to come up with a strawman for the i18n support. This strawman was developed based on the initial i18n requirements we discussed in the EG several weeks ago, as well as on the assumption that we simply want to reuse the existing module distribution format (i.e. jam files) and repository infrastructure for packaging and deploying resources. I have uploaded the strawman [1] to the expert group home page for download. If you prefer to receive them in email, please let me know and I'll send it to you directly. I would like to get your inputs by 5/21/2007, so we can determine how to refine the strawman further. Thanks, - Stanley [1] i18n strawman http://www.jcp.org/en/eg/download/jsr-277-i18n-strawman-05022007.pdf?id=277&fileId=3417 From cierniak at google.com Mon May 14 15:01:55 2007 From: cierniak at google.com (Michal Cierniak) Date: Mon, 14 May 2007 15:01:55 -0700 Subject: Reminder: i18n strawman In-Reply-To: <4648DAA3.9030502@sun.com> References: <4648DAA3.9030502@sun.com> Message-ID: <237294540705141501v6c42ec97i35145e94c2e81be0@mail.gmail.com> Stanley, It is not clear to me from the doc if i8n relies on the assumption that a package may be split across multiple modules. Can you clarify? Thanks, Michal On 5/14/07, Stanley M. Ho wrote: > Hi JSR 277 experts, > > This is a kindly reminder. If you have inputs on the i18n strawman, > please send them in by 5/21/2007. Even if you have reviewed the strawman > and have no comments, it would still be good to let us know. > > Thanks, > - Stanley From cierniak at google.com Mon May 14 15:40:08 2007 From: cierniak at google.com (Michal Cierniak) Date: Mon, 14 May 2007 15:40:08 -0700 Subject: Re-export (was: Updated JSR-277 specification (04/19/2007)) Message-ID: <237294540705141540j3e19bea6oa9697343c6533a29@mail.gmail.com> Stanley, Do you also think that JSR-294 will have to understand re-exports to correctly enforce information hiding at compile time? If so, do you expect that if module A imports and re-exports module B then all of B's exports will be added to A's exports? If this question is answered in the draft, I missed the answer. Independently of 294's decision, do you think that this should happen for 277 modules? I don't have a strong preference one way or another but I think that it may be a little better if we required that B's exported classes are added to A's export list. The benefit is that some of the resolution and validation algorithms might become simpler. The downside is that we will repeat information in multiple places. We would have to decide what happens if the B's exports as seen at the time of building A are different form the actual exports of the version of B that is wired at runtime and from this point of view, I can see arguments pro and con (personally I think that this will be better overall but I can imagine that others may have a different opinion). Michal On 5/14/07, Stanley M. Ho wrote: > Hi JSR 277 experts, > > If you still have any input on the updated early draft specification, > please send them in as soon as possible. > > Thanks, > - Stanley > From Stanley.Ho at sun.com Mon May 14 16:18:24 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Mon, 14 May 2007 16:18:24 -0700 Subject: Reminder: i18n strawman In-Reply-To: <237294540705141501v6c42ec97i35145e94c2e81be0@mail.gmail.com> References: <4648DAA3.9030502@sun.com> <237294540705141501v6c42ec97i35145e94c2e81be0@mail.gmail.com> Message-ID: <4648EE40.5050802@sun.com> Hi Michal, Yes, the i18n strawman currently assumes that the class-based resource bundles in multiple modules are in the same Java package (but each with different superpackage membership), and this is consistent with the existing practice in using the java.util.ResourceBundle API. Based on the earlier EG discussions, we have agreed to explicitly disallow split packages during shallow validation so developers are not encouraged to use them and we don't have to deal with all the issues that split packages may cause. However, the existing i18n practice somewhat contradicts what we encourage developers to do. That said, this may not be a big problem after all, since split packages would still be disallowed during module initialization. This also implies that a target module can't import resource modules directly and must rely on the ResourceBundle API to look up resources, and personally I think this is acceptable (if other EG members have different opinions, I would like to hear them). Therefore, we could still discourage split packages in general and only let developers use them in this special case. The obvious alternative is to require class-based resource bundles in multiple modules to be in different Java packages. Unfortunately, this is incompatible with the existing practice in translating resource bundles, and I'm concerned that this would hurt adoption. What do you think? - Stanley Michal Cierniak wrote: > Stanley, > > It is not clear to me from the doc if i8n relies on the assumption > that a package may be split across multiple modules. Can you clarify? > > Thanks, > Michal From cierniak at google.com Mon May 14 16:36:51 2007 From: cierniak at google.com (Michal Cierniak) Date: Mon, 14 May 2007 16:36:51 -0700 Subject: Reminder: i18n strawman In-Reply-To: <4648EE40.5050802@sun.com> References: <4648DAA3.9030502@sun.com> <237294540705141501v6c42ec97i35145e94c2e81be0@mail.gmail.com> <4648EE40.5050802@sun.com> Message-ID: <237294540705141636q2832e6bcy2de28f633235cdc4@mail.gmail.com> I think that it's reasonable if we decide that the only way that split packages may manifest themselves is as resources via the ResourceBundle API. I was worried about code for the same package coming from multiple modules but it seems that you're not suggesting this. Thanks! Michal On 5/14/07, Stanley M. Ho wrote: > Hi Michal, > > Yes, the i18n strawman currently assumes that the class-based resource > bundles in multiple modules are in the same Java package (but each with > different superpackage membership), and this is consistent with the > existing practice in using the java.util.ResourceBundle API. > > Based on the earlier EG discussions, we have agreed to explicitly > disallow split packages during shallow validation so developers are not > encouraged to use them and we don't have to deal with all the issues > that split packages may cause. However, the existing i18n practice > somewhat contradicts what we encourage developers to do. > > That said, this may not be a big problem after all, since split packages > would still be disallowed during module initialization. This also > implies that a target module can't import resource modules directly and > must rely on the ResourceBundle API to look up resources, and personally > I think this is acceptable (if other EG members have different opinions, > I would like to hear them). Therefore, we could still discourage split > packages in general and only let developers use them in this special case. > > The obvious alternative is to require class-based resource bundles in > multiple modules to be in different Java packages. Unfortunately, this > is incompatible with the existing practice in translating resource > bundles, and I'm concerned that this would hurt adoption. > > What do you think? > > - Stanley > > > Michal Cierniak wrote: > > Stanley, > > > > It is not clear to me from the doc if i8n relies on the assumption > > that a package may be split across multiple modules. Can you clarify? > > > > Thanks, > > Michal > From Stanley.Ho at sun.com Mon May 14 18:12:52 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Mon, 14 May 2007 18:12:52 -0700 Subject: Re-export (was: Updated JSR-277 specification (04/19/2007)) In-Reply-To: <237294540705141540j3e19bea6oa9697343c6533a29@mail.gmail.com> References: <237294540705141540j3e19bea6oa9697343c6533a29@mail.gmail.com> Message-ID: <46490914.9010809@sun.com> Hi Michal, Michal Cierniak wrote: > Stanley, > > Do you also think that JSR-294 will have to understand re-exports to > correctly enforce information hiding at compile time? If so, do you > expect that if module A imports and re-exports module B then all of > B's exports will be added to A's exports? If this question is > answered in the draft, I missed the answer. I think the module-aware compiler will definitely need to understand re-exports for proper compilation. That said, whether this is done at the JSR 294's level will depend on the outcome of the "import" discussion in the JSR 294 EG, which I expect will happen sometimes soon. > Independently of 294's decision, do you think that this should happen > for 277 modules? I don't have a strong preference one way or another > but I think that it may be a little better if we required that B's > exported classes are added to A's export list. The benefit is that > some of the resolution and validation algorithms might become simpler. > The downside is that we will repeat information in multiple places. > We would have to decide what happens if the B's exports as seen at the > time of building A are different form the actual exports of the > version of B that is wired at runtime and from this point of view, I > can see arguments pro and con (personally I think that this will be > better overall but I can imagine that others may have a different > opinion). > > Michal I certainly like the simplicity; unfortunately, I don't think this will work at runtime if I correctly understand what you proposed. :-( Supposes module A imports modules B and C, and module B also imports and re-exports module C. Using the expanded-export-list approach would cause module A fail to initialize - the shallow validation could detect duplicate types from the imported modules B and C but would be unable to determine if the duplicate types were originated from the same module C. Under the current algorithms in the updated specification, module A would be initialized successfully, which is the correct behavior. - Stanley From cierniak at google.com Mon May 14 22:13:45 2007 From: cierniak at google.com (Michal Cierniak) Date: Mon, 14 May 2007 22:13:45 -0700 Subject: Re-export (was: Updated JSR-277 specification (04/19/2007)) In-Reply-To: <46490914.9010809@sun.com> References: <237294540705141540j3e19bea6oa9697343c6533a29@mail.gmail.com> <46490914.9010809@sun.com> Message-ID: <237294540705142213o640b4c7fqcf918a7be9abf2d@mail.gmail.com> On 5/14/07, Stanley M. Ho wrote: > Supposes module A imports modules B and C, and module B also imports and > re-exports module C. Using the expanded-export-list approach would cause > module A fail to initialize - the shallow validation could detect > duplicate types from the imported modules B and C but would be unable to > determine if the duplicate types were originated from the same module C. > Under the current algorithms in the updated specification, module A > would be initialized successfully, which is the correct behavior. Well, this is true if we don't mark such types as "re-exported". What if C's exports that are added to B's are marked with a bit that means "re-exported"? Your example would still work and we would get better error diagnostic. For instance if your example was expanded to make B import and re-export C and D. If a signature of a method of one of B's exported types contained a type from C and the version of C that was wired to B at runtime lacked that type, we would have no idea why B failed to validate. Was it malformed to begin with? Or was it that one of its imports changed? If no one else thinks that this might be useful, I will not really press for such a feature. As I said in my previous email, I don't really have a strong opinion. I was asking to get a clarification what your intent was. Michal From glyn_normington at UK.IBM.COM Tue May 15 06:53:03 2007 From: glyn_normington at UK.IBM.COM (Glyn Normington) Date: Tue, 15 May 2007 14:53:03 +0100 Subject: Reminder: i18n strawman In-Reply-To: <237294540705141636q2832e6bcy2de28f633235cdc4@mail.gmail.com> Message-ID: The ResourceBundle API does not only use resource (*.properties) lookups. It also attempts class loads to find resource bundles, effectively needing to load classes from a split package. Glyn Michal Cierniak wrote on 15/05/2007 00:36:51: > I think that it's reasonable if we decide that the only way that split > packages may manifest themselves is as resources via the > ResourceBundle API. I was worried about code for the same package > coming from multiple modules but it seems that you're not suggesting > this. > > Thanks! > Michal > > On 5/14/07, Stanley M. Ho wrote: > > Hi Michal, > > > > Yes, the i18n strawman currently assumes that the class-based resource > > bundles in multiple modules are in the same Java package (but each with > > different superpackage membership), and this is consistent with the > > existing practice in using the java.util.ResourceBundle API. > > > > Based on the earlier EG discussions, we have agreed to explicitly > > disallow split packages during shallow validation so developers are not > > encouraged to use them and we don't have to deal with all the issues > > that split packages may cause. However, the existing i18n practice > > somewhat contradicts what we encourage developers to do. > > > > That said, this may not be a big problem after all, since split packages > > would still be disallowed during module initialization. This also > > implies that a target module can't import resource modules directly and > > must rely on the ResourceBundle API to look up resources, and personally > > I think this is acceptable (if other EG members have different opinions, > > I would like to hear them). Therefore, we could still discourage split > > packages in general and only let developers use them in this special case. > > > > The obvious alternative is to require class-based resource bundles in > > multiple modules to be in different Java packages. Unfortunately, this > > is incompatible with the existing practice in translating resource > > bundles, and I'm concerned that this would hurt adoption. > > > > What do you think? > > > > - Stanley > > > > > > Michal Cierniak wrote: > > > Stanley, > > > > > > It is not clear to me from the doc if i8n relies on the assumption > > > that a package may be split across multiple modules. Can you clarify? > > > > > > Thanks, > > > Michal > > Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/attachments/20070515/a54896ca/attachment.html From cierniak at google.com Tue May 15 07:22:49 2007 From: cierniak at google.com (Michal Cierniak) Date: Tue, 15 May 2007 07:22:49 -0700 Subject: Reminder: i18n strawman In-Reply-To: References: <237294540705141636q2832e6bcy2de28f633235cdc4@mail.gmail.com> Message-ID: <237294540705150722w7ae9a8b6v912d2cee962e11d7@mail.gmail.com> Glyn, thanks for pointing this out! Stanley, do you think that we could restrict the uses of the ResourceBundle API in this context to not load code? For the context: my recollection is that we had a consensus to not allow split packages but since this discussion took place a long time ago, it could be that I don't remember it well. Michal On 5/15/07, Glyn Normington wrote: > > The ResourceBundle API does not only use resource (*.properties) lookups. > It also attempts class loads to find resource bundles, effectively needing > to load classes from a split package. > > Glyn From bryan.atsatt at oracle.com Tue May 15 16:50:30 2007 From: bryan.atsatt at oracle.com (Bryan Atsatt) Date: Tue, 15 May 2007 16:50:30 -0700 Subject: Reminder: i18n strawman In-Reply-To: <237294540705150722w7ae9a8b6v912d2cee962e11d7@mail.gmail.com> References: <237294540705141636q2832e6bcy2de28f633235cdc4@mail.gmail.com> <237294540705150722w7ae9a8b6v912d2cee962e11d7@mail.gmail.com> Message-ID: <464A4746.70302@oracle.com> Yes, I was going to point this out as well. In fact, at Oracle, our translators *require* us to use ListResourceBundle subclasses. They will not translate property file based resource bundles; I've never gotten a full answer as to the reason for this, but I think it is a tools restriction. This same restriction was imposed when I was at IBM, so I would guess that it may be widespread... // Bryan Michal Cierniak wrote: > Glyn, thanks for pointing this out! > > Stanley, do you think that we could restrict the uses of the > ResourceBundle API in this context to not load code? > > For the context: my recollection is that we had a consensus to not > allow split packages but since this discussion took place a long time > ago, it could be that I don't remember it well. > > Michal > > On 5/15/07, Glyn Normington wrote: >> >> The ResourceBundle API does not only use resource (*.properties) lookups. >> It also attempts class loads to find resource bundles, effectively >> needing >> to load classes from a split package. >> >> Glyn > From stanley.ho at Sun.COM Tue May 15 23:35:31 2007 From: stanley.ho at Sun.COM (Stanley M. Ho) Date: Wed, 16 May 2007 00:35:31 -0600 Subject: Reminder: i18n strawman Message-ID: On Tue, 15 May 2007 07:22:49 -0700, Michal Cierniak wrote: >Glyn, thanks for pointing this out! > >Stanley, do you think that we could restrict the uses of the >ResourceBundle API in this context to not load code? Michal, since we will define the actual requirements for the ResourceBundle API changes in Java SE 7, we could certainly restrict the usage of ListResourceBundle in the ResourceBundle API when the target module attempts to load resource bundles from the resource modules. I have briefly talked to the folks in the i18n team in Java SE, and their impression was that PropertyResourceBundle is much more commonly used than ListResourceBundle among developers. However, there are some applications (including the JDK) relying on ListResourceBundle, so supporting PropertyResourceBundle alone in the module system may not be a sufficient solution. >For the context: my recollection is that we had a consensus to not >allow split packages but since this discussion took place a long time >ago, it could be that I don't remember it well. > >Michal Right. The EG has the consensus on not to support split packages in the module system. So far, we have only enforced the policy on shallow validation, so module cannot import other modules with split packages. I don't think we want to alter this policy in type consistency validation, as it does simplify our general design a bit. That said, I think the question for the EG is whether we want to enforce the no-split-packages policy across the board (and drop the ListResourceBundle support as a result), or enforce the policy only during type consistency validation and make split packages a special case in loading class-based resource bundles through the ResourceBundle API. - Stanley P.S. This is a repost. Apparently, I have email problems with jcp.org again, and please ignore if you receive more than one copy of my reply. From stanley.ho at sun.com Tue May 15 23:39:10 2007 From: stanley.ho at sun.com (Stanley M. Ho) Date: Wed, 16 May 2007 00:39:10 -0600 Subject: Legacy classes in a module Message-ID: Hi JSR 277 experts, This is regarding the semantic of legacy classes in a module, and is one of the outstanding issues in the updated specification (Section 2.14) that I would like to get your inputs. Typically, classes in a module are "module" classes - Java classes that have superpackage membership. However, it is also possible that developers may want to make use of legacy classes in a module, e.g. using an existing xml parser in a module. There are two possible use cases I can think of: 1. Developers simply make use of the legacy classes as implementation details in the module. In this case, the exported module classes neither expose nor reference the legacy classes in the export signatures, and the legacy classes in the module should not be made visible to the importing modules. 2. Developers use the legacy classes as part of the public API in the module. In this case, the legacy classes in the module should be made visible to the importing modules. The exported module classes may expose or reference the legacy classes in the export signatures, but this is not required. I think it's important for us to recognize and support both use cases. However, this may be difficult for us to determine which use case is the developers' real intention. Adding to the mix is that lack of module level access control for the legacy classes, and there is not much we can do once the legacy classes are leaked out from the module classloader. That said, I think it's still important for us to define the semantics for both use cases in the specification. Here is what I propose: a. Consider #1 to be the default. b. Add a new superpackage-level annotation (e.g. @ExportLegacyClasses) for developers to indicate the module is in the #2 category. c. If @ExportLegacyClasses is declared, the ModuleDefinition.getExportedClasses() would include the legacy classes. Note that unless this annotation is declared in the module definition, the importing modules will never attempt to load the legacy classes from the imported module during classloading delegation as the classloading delegation algorithm always checks getExportedClassed() first (see algorithm in 7.3.1, step 4). However, if other code obtains a reference of the module classloader somehow and invokes loadClass() directly with the legacy class name, the module classloader will still return the legacy class, and this is consistent with the existing classloader behavior. Let me know if you have inputs. If there are better ways to deal with this issue, I would like to hear them as well. - Stanley P.S. This is also a repost due to email problems. Please ignore if you receive more than one copy of this message. From glyn_normington at UK.IBM.COM Wed May 16 07:09:43 2007 From: glyn_normington at UK.IBM.COM (Glyn Normington) Date: Wed, 16 May 2007 15:09:43 +0100 Subject: Legacy classes in a module In-Reply-To: Message-ID: Hi Stanley "Stanley M. Ho" wrote on 16/05/2007 07:39:10: > Hi JSR 277 experts, > > This is regarding the semantic of legacy classes in a module, and is one of > the outstanding issues in the updated specification (Section 2.14) that I > would like to get your inputs. > > Typically, classes in a module are "module" classes - Java classes that have > superpackage membership. However, it is also possible that developers may > want to make use of legacy classes in a module, e.g. using an existing xml > parser in a module. There are two possible use cases I can think of: I can't tell from this whether the developer has the freedom to recompile the legacy classes, but I assume not otherwise it would be trivial to include them as superpackage members. (Also, there are use cases where a developer needs to use an existing JAR, but doesn't have the source or the rights to crack the JAR open and fiddle with the contents.) > > 1. Developers simply make use of the legacy classes as implementation > details in the module. In this case, the exported module classes neither > expose nor reference the legacy classes in the export signatures, and the > legacy classes in the module should not be made visible to the importing > modules. > > 2. Developers use the legacy classes as part of the public API in the > module. In this case, the legacy classes in the module should be made > visible to the importing modules. The exported module classes may expose or > reference the legacy classes in the export signatures, but this is > not required. > > I think it's important for us to recognize and support both use cases. > However, this may be difficult for us to determine which use case is the > developers' real intention. Adding to the mix is that lack of module level > access control for the legacy classes, and there is not much we can do once > the legacy classes are leaked out from the module classloader. > > That said, I think it's still important for us to define the semantics for > both use cases in the specification. Here is what I propose: > > a. Consider #1 to be the default. Agreed. > b. Add a new superpackage-level annotation (e.g. @ExportLegacyClasses) for > developers to indicate the module is in the #2 category. Note that this does not enable a developer to expose some but not all of the legacy classes in a module. > c. If @ExportLegacyClasses is declared, the > ModuleDefinition.getExportedClasses() would include the legacy classes. > > Note that unless this annotation is declared in the module definition, the > importing modules will never attempt to load the legacy classes from the > imported module during classloading delegation as the classloading > delegation algorithm always checks getExportedClassed() first (see algorithm > in 7.3.1, step 4). However, if other code obtains a reference of the module > classloader somehow and invokes loadClass() directly with the legacy class > name, the module classloader will still return the legacy class, and this is > consistent with the existing classloader behavior. Is this proposal meant to include the case of creating a module from legacy classes alone? If so, I guess the module would need an 'empty' superpackage definition such as: @Version("2.9.0") @ExportLegacyClasses superpackage org.apache.xerces {} > > Let me know if you have inputs. If there are better ways to deal with this > issue, I would like to hear them as well. > > - Stanley > > P.S. This is also a repost due to email problems. Please ignore if you > receive more than one copy of this message. Glyn Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/attachments/20070516/5d656731/attachment.html From bryan.atsatt at oracle.com Wed May 16 10:40:47 2007 From: bryan.atsatt at oracle.com (Bryan Atsatt) Date: Wed, 16 May 2007 10:40:47 -0700 Subject: Legacy classes in a module In-Reply-To: References: Message-ID: <464B421F.3050503@oracle.com> While the current strawman suggests that superpackage classes will have modified class files (e.g. naming their superpackage), there is a proposal on the table that eliminates this requirement. If eliminated, the only requirement to include "legacy" classes in a superpackage will be to have the superpackage artifact present. In a 277 module, any mixture of new and "legacy" classes (likely in separate jars) could then fall under the umbrella superpackage. And this whole issue is eliminated... // Bryan Stanley M. Ho wrote: > Hi JSR 277 experts, > > This is regarding the semantic of legacy classes in a module, and is one of > the outstanding issues in the updated specification (Section 2.14) that I > would like to get your inputs. > > Typically, classes in a module are "module" classes - Java classes that have > superpackage membership. However, it is also possible that developers may > want to make use of legacy classes in a module, e.g. using an existing xml > parser in a module. There are two possible use cases I can think of: > > 1. Developers simply make use of the legacy classes as implementation > details in the module. In this case, the exported module classes neither > expose nor reference the legacy classes in the export signatures, and the > legacy classes in the module should not be made visible to the importing > modules. > > 2. Developers use the legacy classes as part of the public API in the > module. In this case, the legacy classes in the module should be made > visible to the importing modules. The exported module classes may expose or > reference the legacy classes in the export signatures, but this is not required. > > I think it's important for us to recognize and support both use cases. > However, this may be difficult for us to determine which use case is the > developers' real intention. Adding to the mix is that lack of module level > access control for the legacy classes, and there is not much we can do once > the legacy classes are leaked out from the module classloader. > > That said, I think it's still important for us to define the semantics for > both use cases in the specification. Here is what I propose: > > a. Consider #1 to be the default. > b. Add a new superpackage-level annotation (e.g. @ExportLegacyClasses) for > developers to indicate the module is in the #2 category. > c. If @ExportLegacyClasses is declared, the > ModuleDefinition.getExportedClasses() would include the legacy classes. > > Note that unless this annotation is declared in the module definition, the > importing modules will never attempt to load the legacy classes from the > imported module during classloading delegation as the classloading > delegation algorithm always checks getExportedClassed() first (see algorithm > in 7.3.1, step 4). However, if other code obtains a reference of the module > classloader somehow and invokes loadClass() directly with the legacy class > name, the module classloader will still return the legacy class, and this is > consistent with the existing classloader behavior. > > Let me know if you have inputs. If there are better ways to deal with this > issue, I would like to hear them as well. > > - Stanley > > P.S. This is also a repost due to email problems. Please ignore if you > receive more than one copy of this message. > From bryan.atsatt at oracle.com Wed May 16 11:15:57 2007 From: bryan.atsatt at oracle.com (Bryan Atsatt) Date: Wed, 16 May 2007 11:15:57 -0700 Subject: Reminder: i18n strawman In-Reply-To: References: Message-ID: <464B4A5D.9060807@oracle.com> ListResourceBundle cannot be demoted to second-class citizen status in the module system (Perhaps you haven't seen my earlier response?). There is a solution to the split package problem, and, while it does change the development model slightly, I think that is in line with the more significant model change of moving to a module system... Introduce a package naming convention for resource bundles deployed as modules. The convention would allow ResourceBundle.getBundle() to easily map from the "base" name to the actual bundle package name. For example, if we introduce the convention that resource bundle "base" names are mapped such that "resources" is appended to the package name, then calling ResourceBundle.getBundle with: "com.acme.MyResources" Would actually search for resources with the base name: "com.acme.resources.MyResources" thus sidestepping the split package issue. Yes, this requires developers to actually define their resources using this naming convention, but I don't see that as a significant burden. Especially when you consider the benefit of not having to package all translations in the primary module. To avoid potential package name collisions, we should consider using a more distinct name than simply "resources", e.g.: "com.acme.resource_bundle.MyResources" "com.acme.module_resources.MyResources" "com.acme._resources_.MyResources" // Bryan Stanley M. Ho wrote: > On Tue, 15 May 2007 07:22:49 -0700, Michal Cierniak wrote: > >> Glyn, thanks for pointing this out! >> >> Stanley, do you think that we could restrict the uses of the >> ResourceBundle API in this context to not load code? > > Michal, since we will define the actual requirements for the ResourceBundle > API changes in Java SE 7, we could certainly restrict the usage of > ListResourceBundle in the ResourceBundle API when the target module attempts > to load resource bundles from the resource modules. > > I have briefly talked to the folks in the i18n team in Java SE, and their > impression was that PropertyResourceBundle is much more commonly used than > ListResourceBundle among developers. However, there are some applications > (including the JDK) relying on ListResourceBundle, so supporting > PropertyResourceBundle alone in the module system may not be a sufficient > solution. > >> For the context: my recollection is that we had a consensus to not >> allow split packages but since this discussion took place a long time >> ago, it could be that I don't remember it well. >> >> Michal > > Right. The EG has the consensus on not to support split packages in the > module system. So far, we have only enforced the policy on shallow > validation, so module cannot import other modules with split packages. I > don't think we want to alter this policy in type consistency validation, as > it does simplify our general design a bit. That said, I think the question > for the EG is whether we want to enforce the no-split-packages policy across > the board (and drop the ListResourceBundle support as a result), or enforce > the policy only during type consistency validation and make split packages a > special case in loading class-based resource bundles through the > ResourceBundle API. > > - Stanley > > P.S. This is a repost. Apparently, I have email problems with jcp.org again, > and please ignore if you receive more than one copy of my reply. > From cierniak at google.com Wed May 16 12:25:05 2007 From: cierniak at google.com (Michal Cierniak) Date: Wed, 16 May 2007 12:25:05 -0700 Subject: Reminder: i18n strawman In-Reply-To: <464B4A5D.9060807@oracle.com> References: <464B4A5D.9060807@oracle.com> Message-ID: <237294540705161225i7aa527f1ka093ddd7dc1d0e46@mail.gmail.com> I also think that this would not be too burdensome. On 5/16/07, Bryan Atsatt wrote: > ListResourceBundle cannot be demoted to second-class citizen status in > the module system (Perhaps you haven't seen my earlier response?). > > There is a solution to the split package problem, and, while it does > change the development model slightly, I think that is in line with the > more significant model change of moving to a module system... > > Introduce a package naming convention for resource bundles deployed as > modules. The convention would allow ResourceBundle.getBundle() to easily > map from the "base" name to the actual bundle package name. > > For example, if we introduce the convention that resource bundle "base" > names are mapped such that "resources" is appended to the package name, > then calling ResourceBundle.getBundle with: > > "com.acme.MyResources" > > Would actually search for resources with the base name: > > "com.acme.resources.MyResources" > > thus sidestepping the split package issue. > > Yes, this requires developers to actually define their resources using > this naming convention, but I don't see that as a significant burden. > Especially when you consider the benefit of not having to package all > translations in the primary module. > > To avoid potential package name collisions, we should consider using a > more distinct name than simply "resources", e.g.: > > "com.acme.resource_bundle.MyResources" > "com.acme.module_resources.MyResources" > "com.acme._resources_.MyResources" > > // Bryan From cierniak at google.com Wed May 16 14:17:54 2007 From: cierniak at google.com (Michal Cierniak) Date: Wed, 16 May 2007 14:17:54 -0700 Subject: Legacy classes in a module In-Reply-To: <464B421F.3050503@oracle.com> References: <464B421F.3050503@oracle.com> Message-ID: <237294540705161417o2199c3bbp26d2de3f68e1f2db@mail.gmail.com> I would like to add that even if superpackage classes do name their superpackage in the new world, it wouldn't be difficult to require the JRE to "rewrite" legacy classes on the fly if they are loaded from a JAR embedded in a JAM. In that case, the legacy classes would end up being members of the superpackage associated with the containing JAM. We might run into other problems because legacy classes do not necessarily play by the rules of the new module-aware world but this is independent of whether we rewrite them to add superpackage membership info. Michal On 5/16/07, Bryan Atsatt wrote: > While the current strawman suggests that superpackage classes will have > modified class files (e.g. naming their superpackage), there is a > proposal on the table that eliminates this requirement. > > If eliminated, the only requirement to include "legacy" classes in a > superpackage will be to have the superpackage artifact present. In a 277 > module, any mixture of new and "legacy" classes (likely in separate > jars) could then fall under the umbrella superpackage. And this whole > issue is eliminated... > > // Bryan > From stanley.ho at sun.com Thu May 17 12:11:07 2007 From: stanley.ho at sun.com (Stanley M. Ho) Date: Thu, 17 May 2007 13:11:07 -0600 Subject: Reminder: i18n strawman Message-ID: On Wed, 16 May 2007 11:15:57 -0700, Bryan Atsatt wrote: >ListResourceBundle cannot be demoted to second-class citizen status in >the module system (Perhaps you haven't seen my earlier response?). Yes, I agreed it's important for us to support ListResourceBundle. >There is a solution to the split package problem, and, while it does >change the development model slightly, I think that is in line with the >more significant model change of moving to a module system... > >Introduce a package naming convention for resource bundles deployed as >modules. The convention would allow ResourceBundle.getBundle() to easily >map from the "base" name to the actual bundle package name. > >For example, if we introduce the convention that resource bundle "base" >names are mapped such that "resources" is appended to the package name, >then calling ResourceBundle.getBundle with: > > "com.acme.MyResources" > >Would actually search for resources with the base name: > > "com.acme.resources.MyResources" > >thus sidestepping the split package issue. > >Yes, this requires developers to actually define their resources using >this naming convention, but I don't see that as a significant burden. >Especially when you consider the benefit of not having to package all >translations in the primary module. > >To avoid potential package name collisions, we should consider using a >more distinct name than simply "resources", e.g.: > > "com.acme.resource_bundle.MyResources" > "com.acme.module_resources.MyResources" > "com.acme._resources_.MyResources" > >// Bryan I was under the assumption that we didn't want to change the way developers define their resources. That said, I am fine with using some packaging naming convention to sidestep the split package issue. If we go down this path, I think we will need to expand Bryan's suggestion a bit. The ResourceBundle API may search multiple resource modules in order to look up the resources (from more restrictive locales to less restrictive locales), so it is important for resources in the resource modules for each locale to have unique package name to avoid potential conflict regardless the lookup order. e.g. "com.acme_en.MyResources" "com.acme_ja.MyResources_ja" "com.acme_fr.MyResources_fr" "com.acme_fr_CH.MyResources_fr_CH" Note that the locale information is repeated twice. If we want to simplify it further, we can require the locale information to be specified only as part of the package name of the resource, but not as part of the filename of the resource: "com.acme_en.MyResources" "com.acme_ja.MyResources" "com.acme_fr.MyResources" "com.acme_fr_CH.MyResources" - Stanley P.S. This is another repost. My email problems with jcp.org continue, and please ignore if you receive more than one copy of my reply. From stanley.ho at sun.com Thu May 17 13:48:47 2007 From: stanley.ho at sun.com (Stanley M. Ho) Date: Thu, 17 May 2007 14:48:47 -0600 Subject: Legacy classes in a module Message-ID: Hi Glyn, On Wed, 16 May 2007 15:09:43 +0100, Glyn Normington wrote: >I can't tell from this whether the developer has the freedom to recompile >the legacy classes, but I assume not otherwise it would be trivial to >include them as superpackage members. (Also, there are use cases where a >developer needs to use an existing JAR, but doesn't have the source or the >rights to crack the JAR open and fiddle with the contents.) Yes, this is to address the situation where the developer does not have the freedom to recompile the legacy classes but he still want to make use of them in the module. >> b. Add a new superpackage-level annotation (e.g. @ExportLegacyClasses) >for >> developers to indicate the module is in the #2 category. > >Note that this does not enable a developer to expose some but not all of >the legacy classes in a module. Right. It should be very easy if we want to change it to support finer granularity, but I would like to keep it simple for now. >Is this proposal meant to include the case of creating a module from >legacy classes alone? If so, I guess the module would need an 'empty' >superpackage definition such as: > >@Version("2.9.0") >@ExportLegacyClasses >superpackage org.apache.xerces {} No, this proposal is only intended to address the first bullet in Section 2.14 in the updated specification. If the developer has the rights to repackage the legacy classes with the module metadata, most likely he also has access to the source thus he has the freedom to recompile, and I think the recommendation would be for him to migrate the legacy JAR to be a simple module archive. - Stanley From stanley.ho at sun.com Thu May 17 14:11:55 2007 From: stanley.ho at sun.com (Stanley M. Ho) Date: Thu, 17 May 2007 15:11:55 -0600 Subject: Legacy classes in a module Message-ID: Regardless, I think we still need to distinguish whether the legacy classes should be considered module private (use case #1) or made visible externally (use case #2). I don't think we can really know the developer's intent by looking at the legacy classes alone. Even if rewriting is involved, we still need to distinguish these cases in order to have the proper export information in the superpackage. - Stanley On Wed, 16 May 2007 14:17:54 -0700, Michal Cierniak wrote: >I would like to add that even if superpackage classes do name their >superpackage in the new world, it wouldn't be difficult to require the >JRE to "rewrite" legacy classes on the fly if they are loaded from a >JAR embedded in a JAM. In that case, the legacy classes would end up >being members of the superpackage associated with the containing JAM. > >We might run into other problems because legacy classes do not >necessarily play by the rules of the new module-aware world but this >is independent of whether we rewrite them to add superpackage >membership info. > >Michal > >On 5/16/07, Bryan Atsatt wrote: >> While the current strawman suggests that superpackage classes will have >> modified class files (e.g. naming their superpackage), there is a >> proposal on the table that eliminates this requirement. >> >> If eliminated, the only requirement to include "legacy" classes in a >> superpackage will be to have the superpackage artifact present. In a 277 >> module, any mixture of new and "legacy" classes (likely in separate >> jars) could then fall under the umbrella superpackage. And this whole >> issue is eliminated... >> >> // Bryan >> From cierniak at google.com Thu May 17 15:21:20 2007 From: cierniak at google.com (Michal Cierniak) Date: Thu, 17 May 2007 15:21:20 -0700 Subject: Legacy classes in a module In-Reply-To: References: Message-ID: <237294540705171521t58613439hb7a2f59d0a23d39f@mail.gmail.com> Yes, I was only addressing one aspect of the problem. On 5/17/07, Stanley M. Ho wrote: > Regardless, I think we still need to distinguish whether the legacy classes > should be considered module private (use case #1) or made visible externally > (use case #2). I don't think we can really know the developer's intent by > looking at the legacy classes alone. Even if rewriting is involved, we still > need to distinguish these cases in order to have the proper export > information in the superpackage. > > - Stanley > > > On Wed, 16 May 2007 14:17:54 -0700, Michal Cierniak wrote: > > >I would like to add that even if superpackage classes do name their > >superpackage in the new world, it wouldn't be difficult to require the > >JRE to "rewrite" legacy classes on the fly if they are loaded from a > >JAR embedded in a JAM. In that case, the legacy classes would end up > >being members of the superpackage associated with the containing JAM. > > > >We might run into other problems because legacy classes do not > >necessarily play by the rules of the new module-aware world but this > >is independent of whether we rewrite them to add superpackage > >membership info. > > > >Michal > > > >On 5/16/07, Bryan Atsatt wrote: > >> While the current strawman suggests that superpackage classes will have > >> modified class files (e.g. naming their superpackage), there is a > >> proposal on the table that eliminates this requirement. > >> > >> If eliminated, the only requirement to include "legacy" classes in a > >> superpackage will be to have the superpackage artifact present. In a 277 > >> module, any mixture of new and "legacy" classes (likely in separate > >> jars) could then fall under the umbrella superpackage. And this whole > >> issue is eliminated... > >> > >> // Bryan > >> > From bryan.atsatt at oracle.com Thu May 17 19:01:09 2007 From: bryan.atsatt at oracle.com (Bryan Atsatt) Date: Thu, 17 May 2007 19:01:09 -0700 Subject: Legacy classes in a module In-Reply-To: References: Message-ID: <464D08E5.7070500@oracle.com> I am assuming that in this scenario the module developer would be allowed to make this explicit in the superpackage declaration itself. That is, superpackage member declarations could include packages from the "legacy" jars. 294 simply has to allow declared member packages to be AWOL at compile time. BTW: The proposal I referred to earlier assumes that the runtime artifact of a superpackage (e.g. a SuperPackage class instance) is *assigned* to each class during ClassLoader.defineClass(). This decouples the superpackage and class file artifacts while solving the superpackage "loading" problem for the VM. It obviously makes it trivial for "legacy" classes to be assigned into a superpackage as they are defined. (It also makes it possible to extend the superpackage at runtime, but that may be going too far.) // Bryan Stanley M. Ho wrote: > Regardless, I think we still need to distinguish whether the legacy classes > should be considered module private (use case #1) or made visible externally > (use case #2). I don't think we can really know the developer's intent by > looking at the legacy classes alone. Even if rewriting is involved, we still > need to distinguish these cases in order to have the proper export > information in the superpackage. > > - Stanley > > > On Wed, 16 May 2007 14:17:54 -0700, Michal Cierniak wrote: > >> I would like to add that even if superpackage classes do name their >> superpackage in the new world, it wouldn't be difficult to require the >> JRE to "rewrite" legacy classes on the fly if they are loaded from a >> JAR embedded in a JAM. In that case, the legacy classes would end up >> being members of the superpackage associated with the containing JAM. >> >> We might run into other problems because legacy classes do not >> necessarily play by the rules of the new module-aware world but this >> is independent of whether we rewrite them to add superpackage >> membership info. >> >> Michal >> >> On 5/16/07, Bryan Atsatt wrote: >>> While the current strawman suggests that superpackage classes will have >>> modified class files (e.g. naming their superpackage), there is a >>> proposal on the table that eliminates this requirement. >>> >>> If eliminated, the only requirement to include "legacy" classes in a >>> superpackage will be to have the superpackage artifact present. In a 277 >>> module, any mixture of new and "legacy" classes (likely in separate >>> jars) could then fall under the umbrella superpackage. And this whole >>> issue is eliminated... >>> >>> // Bryan >>> > From glyn_normington at UK.IBM.COM Fri May 18 00:38:00 2007 From: glyn_normington at UK.IBM.COM (Glyn Normington) Date: Fri, 18 May 2007 08:38:00 +0100 Subject: Legacy classes in a module In-Reply-To: <464D08E5.7070500@oracle.com> Message-ID: Hi Bryan Your proposal has considerable benefits. Do you have a feel for its acceptability to the JSR 294 Expert Group? I don't think we've discussed it (properly) over there. Glyn Bryan Atsatt wrote on 18/05/2007 03:01:09: > I am assuming that in this scenario the module developer would be > allowed to make this explicit in the superpackage declaration itself. > That is, superpackage member declarations could include packages from > the "legacy" jars. 294 simply has to allow declared member packages to > be AWOL at compile time. > > BTW: The proposal I referred to earlier assumes that the runtime > artifact of a superpackage (e.g. a SuperPackage class instance) is > *assigned* to each class during ClassLoader.defineClass(). This > decouples the superpackage and class file artifacts while solving the > superpackage "loading" problem for the VM. It obviously makes it trivial > for "legacy" classes to be assigned into a superpackage as they are > defined. (It also makes it possible to extend the superpackage at > runtime, but that may be going too far.) > > // Bryan > > Stanley M. Ho wrote: > > Regardless, I think we still need to distinguish whether the legacy classes > > should be considered module private (use case #1) or made visible externally > > (use case #2). I don't think we can really know the developer's intent by > > looking at the legacy classes alone. Even if rewriting is involved, we still > > need to distinguish these cases in order to have the proper export > > information in the superpackage. > > > > - Stanley > > > > > > On Wed, 16 May 2007 14:17:54 -0700, Michal Cierniak > wrote: > > > >> I would like to add that even if superpackage classes do name their > >> superpackage in the new world, it wouldn't be difficult to require the > >> JRE to "rewrite" legacy classes on the fly if they are loaded from a > >> JAR embedded in a JAM. In that case, the legacy classes would end up > >> being members of the superpackage associated with the containing JAM. > >> > >> We might run into other problems because legacy classes do not > >> necessarily play by the rules of the new module-aware world but this > >> is independent of whether we rewrite them to add superpackage > >> membership info. > >> > >> Michal > >> > >> On 5/16/07, Bryan Atsatt wrote: > >>> While the current strawman suggests that superpackage classes will have > >>> modified class files (e.g. naming their superpackage), there is a > >>> proposal on the table that eliminates this requirement. > >>> > >>> If eliminated, the only requirement to include "legacy" classes in a > >>> superpackage will be to have the superpackage artifact present. In a 277 > >>> module, any mixture of new and "legacy" classes (likely in separate > >>> jars) could then fall under the umbrella superpackage. And this whole > >>> issue is eliminated... > >>> > >>> // Bryan > >>> > > Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/attachments/20070518/e7722aff/attachment.html From bryan.atsatt at oracle.com Fri May 18 11:42:27 2007 From: bryan.atsatt at oracle.com (Bryan Atsatt) Date: Fri, 18 May 2007 11:42:27 -0700 Subject: Legacy classes in a module In-Reply-To: References: Message-ID: <464DF393.2060808@oracle.com> No, we haven't yet, but we should. Another benefit you have may have spotted is that an OSGi implementation of 277 modules could support "split superpackages" by having the loaders involved assign a common SuperPackage instance. While none of us likes split packages, if we are to truly make OSGi a first-class citizen in the 277/294 world, this side-effect of the proposal should be seriously considered. // Bryan Glyn Normington wrote: > > Hi Bryan > > Your proposal has considerable benefits. Do you have a feel for its > acceptability to the JSR 294 Expert Group? I don't think we've discussed > it (properly) over there. > > Glyn > > *Bryan Atsatt * wrote on 18/05/2007 03:01:09: > > > I am assuming that in this scenario the module developer would be > > allowed to make this explicit in the superpackage declaration itself. > > That is, superpackage member declarations could include packages from > > the "legacy" jars. 294 simply has to allow declared member packages to > > be AWOL at compile time. > > > > BTW: The proposal I referred to earlier assumes that the runtime > > artifact of a superpackage (e.g. a SuperPackage class instance) is > > *assigned* to each class during ClassLoader.defineClass(). This > > decouples the superpackage and class file artifacts while solving the > > superpackage "loading" problem for the VM. It obviously makes it trivial > > for "legacy" classes to be assigned into a superpackage as they are > > defined. (It also makes it possible to extend the superpackage at > > runtime, but that may be going too far.) > > > > // Bryan > > > > Stanley M. Ho wrote: > > > Regardless, I think we still need to distinguish whether the legacy > classes > > > should be considered module private (use case #1) or made visible > externally > > > (use case #2). I don't think we can really know the developer's > intent by > > > looking at the legacy classes alone. Even if rewriting is involved, > we still > > > need to distinguish these cases in order to have the proper export > > > information in the superpackage. > > > > > > - Stanley > > > > > > > > > On Wed, 16 May 2007 14:17:54 -0700, Michal Cierniak > > wrote: > > > > > >> I would like to add that even if superpackage classes do name their > > >> superpackage in the new world, it wouldn't be difficult to require the > > >> JRE to "rewrite" legacy classes on the fly if they are loaded from a > > >> JAR embedded in a JAM. In that case, the legacy classes would end up > > >> being members of the superpackage associated with the containing JAM. > > >> > > >> We might run into other problems because legacy classes do not > > >> necessarily play by the rules of the new module-aware world but this > > >> is independent of whether we rewrite them to add superpackage > > >> membership info. > > >> > > >> Michal > > >> > > >> On 5/16/07, Bryan Atsatt wrote: > > >>> While the current strawman suggests that superpackage classes > will have > > >>> modified class files (e.g. naming their superpackage), there is a > > >>> proposal on the table that eliminates this requirement. > > >>> > > >>> If eliminated, the only requirement to include "legacy" classes in a > > >>> superpackage will be to have the superpackage artifact present. > In a 277 > > >>> module, any mixture of new and "legacy" classes (likely in separate > > >>> jars) could then fall under the umbrella superpackage. And this whole > > >>> issue is eliminated... > > >>> > > >>> // Bryan > > >>> > > > > > > > ------------------------------------------------------------------------ > > / > / > > /Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU/ > > > > > > From stanley.ho at sun.com Fri May 18 12:27:21 2007 From: stanley.ho at sun.com (Stanley M. Ho) Date: Fri, 18 May 2007 13:27:21 -0600 Subject: Legacy classes in a module Message-ID: Hi Bryan, On Thu, 17 May 2007 19:01:09 -0700, Bryan Atsatt wrote: >I am assuming that in this scenario the module developer would be >allowed to make this explicit in the superpackage declaration itself. >That is, superpackage member declarations could include packages from >the "legacy" jars. 294 simply has to allow declared member packages to >be AWOL at compile time. >From the perspective of JSR 277, I think what we need is simply a way to determine from the module metadata whether/which legacy classes should be made visible externally or not, and this is orthogonal to how it is expressed in the syntax in JSR 294. Because it is unclear how legacy classes will be handled in JSR 294, I am assuming that the module developer would make this explicit in the form of metadata annotation with the superpackage. Do you have any problem if we go with the @ExportLegacyClasses proposal for now so we can move on to other issues? >BTW: The proposal I referred to earlier assumes that the runtime >artifact of a superpackage (e.g. a SuperPackage class instance) is >*assigned* to each class during ClassLoader.defineClass(). This >decouples the superpackage and class file artifacts while solving the >superpackage "loading" problem for the VM. It obviously makes it trivial >for "legacy" classes to be assigned into a superpackage as they are >defined. (It also makes it possible to extend the superpackage at >runtime, but that may be going too far.) > >// Bryan The proposal you referred to is in the scope of JSR 294 but out of the scope of JSR 277, so I think it will be more appropriate if you raise and discuss that proposal with the JSR 294 EG. ;) - Stanley From stanley.ho at sun.com Fri May 18 12:41:03 2007 From: stanley.ho at sun.com (Stanley M. Ho) Date: Fri, 18 May 2007 13:41:03 -0600 Subject: Another reminder: i18n strawman Message-ID: Hi JSR 277 experts, This is another reminder. If you still have inputs on the i18n strawman, please send them in as soon as possible. In case you need more time to review the strawman, please let me know. Otherwise, I would interpret the silent to mean either you are fine with the strawman's content or your concern has already been raised by other EG members. ;) Thanks, - Stanley On Mon, 14 May 2007 14:54:43 -0700, Stanley M. Ho wrote: >Hi JSR 277 experts, > >This is a kindly reminder. If you have inputs on the i18n strawman, >please send them in by 5/21/2007. Even if you have reviewed the strawman >and have no comments, it would still be good to let us know. > >Thanks, >- Stanley > >-------- Original Message -------- >Subject: Strawman: i18n support >Date: Mon, 07 May 2007 17:09:41 -0700 >From: Stanley M. Ho >Reply-To: Java Community Process JSR #277 Expert List >Organization: Sun Microsystems, Inc. >To: JSR-277-EG at JCP.ORG > >Hi JSR 277 experts, > >I've been working with Masayoshi Okutsu (i18n lead) and Naoto Sato in >the Java SE team to come up with a strawman for the i18n support. This >strawman was developed based on the initial i18n requirements we >discussed in the EG several weeks ago, as well as on the assumption that >we simply want to reuse the existing module distribution format (i.e. >jam files) and repository infrastructure for packaging and deploying >resources. > >I have uploaded the strawman [1] to the expert group home page for >download. If you prefer to receive them in email, please let me know and >I'll send it to you directly. > >I would like to get your inputs by 5/21/2007, so we can determine how to >refine the strawman further. > >Thanks, >- Stanley > >[1] i18n strawman >http://www.jcp.org/en/eg/download/jsr-277-i18n-strawman-05022007.pdf?id=277&fileId=3417 From bryan.atsatt at oracle.com Fri May 18 12:58:08 2007 From: bryan.atsatt at oracle.com (Bryan Atsatt) Date: Fri, 18 May 2007 12:58:08 -0700 Subject: Legacy classes in a module In-Reply-To: References: Message-ID: <464E0550.5080708@oracle.com> Of course--we can revisit this if 294 adopts my proposed model... // Bryan Stanley M. Ho wrote: > Hi Bryan, > > On Thu, 17 May 2007 19:01:09 -0700, Bryan Atsatt > wrote: > >> I am assuming that in this scenario the module developer would be >> allowed to make this explicit in the superpackage declaration itself. >> That is, superpackage member declarations could include packages from >> the "legacy" jars. 294 simply has to allow declared member packages to >> be AWOL at compile time. > >>From the perspective of JSR 277, I think what we need is simply a way to > determine from the module metadata whether/which legacy classes should be > made visible externally or not, and this is orthogonal to how it is > expressed in the syntax in JSR 294. > > Because it is unclear how legacy classes will be handled in JSR 294, I am > assuming that the module developer would make this explicit in the form of > metadata annotation with the superpackage. Do you have any problem if we go > with the @ExportLegacyClasses proposal for now so we can move on to other > issues? > >> BTW: The proposal I referred to earlier assumes that the runtime >> artifact of a superpackage (e.g. a SuperPackage class instance) is >> *assigned* to each class during ClassLoader.defineClass(). This >> decouples the superpackage and class file artifacts while solving the >> superpackage "loading" problem for the VM. It obviously makes it trivial >> for "legacy" classes to be assigned into a superpackage as they are >> defined. (It also makes it possible to extend the superpackage at >> runtime, but that may be going too far.) >> >> // Bryan > > The proposal you referred to is in the scope of JSR 294 but out of the scope > of JSR 277, so I think it will be more appropriate if you raise and discuss > that proposal with the JSR 294 EG. ;) > > - Stanley > From Stanley.Ho at sun.com Thu May 17 11:49:59 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Thu, 17 May 2007 11:49:59 -0700 Subject: Reminder: i18n strawman In-Reply-To: <464B4A5D.9060807@oracle.com> References: <464B4A5D.9060807@oracle.com> Message-ID: <464CA3D7.1040203@sun.com> Bryan Atsatt wrote: > ListResourceBundle cannot be demoted to second-class citizen status in > the module system (Perhaps you haven't seen my earlier response?). Yes, I agreed it's important for us to support ListResourceBundle. > There is a solution to the split package problem, and, while it does > change the development model slightly, I think that is in line with the > more significant model change of moving to a module system... > > Introduce a package naming convention for resource bundles deployed as > modules. The convention would allow ResourceBundle.getBundle() to easily > map from the "base" name to the actual bundle package name. > > For example, if we introduce the convention that resource bundle "base" > names are mapped such that "resources" is appended to the package name, > then calling ResourceBundle.getBundle with: > > "com.acme.MyResources" > > Would actually search for resources with the base name: > > "com.acme.resources.MyResources" > > thus sidestepping the split package issue. > > Yes, this requires developers to actually define their resources using > this naming convention, but I don't see that as a significant burden. > Especially when you consider the benefit of not having to package all > translations in the primary module. > > To avoid potential package name collisions, we should consider using a > more distinct name than simply "resources", e.g.: > > "com.acme.resource_bundle.MyResources" > "com.acme.module_resources.MyResources" > "com.acme._resources_.MyResources" > > // Bryan I was under the assumption that we didn't want to change the way developers define their resources. That said, I am fine with using some packaging naming convention to sidestep the split package issue. If we go down this path, I think we will need to expand Bryan's suggestion a bit. The ResourceBundle API may search multiple resource modules in order to look up the resources (from more restrictive locales to less restrictive locales), so it is important for resources in the resource modules for each locale to have unique package name to avoid potential conflict regardless the lookup order. e.g. "com.acme_en.MyResources" "com.acme_ja.MyResources_ja" "com.acme_fr.MyResources_fr" "com.acme_fr_CH.MyResources_fr_CH" Note that the locale information is repeated twice. If we want to simplify it further, we can require the locale information to be specified only as part of the package name of the resource, but not as part of the filename of the resource: "com.acme_en.MyResources" "com.acme_ja.MyResources" "com.acme_fr.MyResources" "com.acme_fr_CH.MyResources" - Stanley From Stanley.Ho at sun.com Fri May 18 12:23:27 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Fri, 18 May 2007 12:23:27 -0700 Subject: Legacy classes in a module In-Reply-To: <464D08E5.7070500@oracle.com> References: <464D08E5.7070500@oracle.com> Message-ID: <464DFD2F.3050109@sun.com> Hi Bryan, Bryan Atsatt wrote: > I am assuming that in this scenario the module developer would be > allowed to make this explicit in the superpackage declaration itself. > That is, superpackage member declarations could include packages from > the "legacy" jars. 294 simply has to allow declared member packages to > be AWOL at compile time. From the perspective of JSR 277, I think what we need is simply a way to determine from the module metadata whether/which legacy classes should be made visible externally or not, and this is orthogonal to how it is expressed in the syntax in JSR 294. Since it is unclear how legacy classes will be handled in JSR 294, I am assuming that the module developer would make this explicit in the form of metadata annotation with the superpackage. Do you have any problem if we go with the @ExportLegacyClasses proposal in JSR 277 for now so we can move on to other issues? > BTW: The proposal I referred to earlier assumes that the runtime > artifact of a superpackage (e.g. a SuperPackage class instance) is > *assigned* to each class during ClassLoader.defineClass(). This > decouples the superpackage and class file artifacts while solving the > superpackage "loading" problem for the VM. It obviously makes it trivial > for "legacy" classes to be assigned into a superpackage as they are > defined. (It also makes it possible to extend the superpackage at > runtime, but that may be going too far.) > > // Bryan The proposal you referred to is in the scope of JSR 294 but out of the scope of JSR 277, so I think it will be more appropriate if you raise and discuss the proposal with the JSR 294 EG. ;) - Stanley From Stanley.Ho at sun.com Tue May 15 18:48:45 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Tue, 15 May 2007 18:48:45 -0700 Subject: Reminder: i18n strawman In-Reply-To: <237294540705150722w7ae9a8b6v912d2cee962e11d7@mail.gmail.com> References: <237294540705141636q2832e6bcy2de28f633235cdc4@mail.gmail.com> <237294540705150722w7ae9a8b6v912d2cee962e11d7@mail.gmail.com> Message-ID: <464A62FD.1040404@sun.com> Michal Cierniak wrote: > Glyn, thanks for pointing this out! > > Stanley, do you think that we could restrict the uses of the > ResourceBundle API in this context to not load code? Michal, since we will define the actual requirements for the ResourceBundle API changes in Java SE 7, we could certainly restrict the usage of ListResourceBundle in the ResourceBundle API when the target module attempts to load resource bundles from the resource modules. I have briefly talked to the folks in the i18n team in Java SE, and their impression was that PropertyResourceBundle is much more commonly used than ListResourceBundle among developers. However, there are some applications (including the JDK) relying on ListResourceBundle, so supporting PropertyResourceBundle alone in the module system may not be a sufficient solution. > For the context: my recollection is that we had a consensus to not > allow split packages but since this discussion took place a long time > ago, it could be that I don't remember it well. Right. The EG has the consensus on not to support split packages in the module system. So far, we have only enforced the policy on the shallow validation, so module cannot import other modules with split packages. I don't think we want to alter this policy in type consistency validation, as this policy does simplify our general design a bit. That said, I think the question for the EG is whether we want to enforce the no-split-packages policy beyond type consistency validation (and drop the ListResourceBundle support as a result), or enforce the policy only during type consistency validation and make split packages a special case in loading class-based resource bundles through the ResourceBundle API. - Stanley From Stanley.Ho at sun.com Wed May 16 18:53:40 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Wed, 16 May 2007 18:53:40 -0700 Subject: Reminder: i18n strawman In-Reply-To: <464B4A5D.9060807@oracle.com> References: <464B4A5D.9060807@oracle.com> Message-ID: <464BB5A4.709@sun.com> Bryan Atsatt wrote: > ListResourceBundle cannot be demoted to second-class citizen status in > the module system (Perhaps you haven't seen my earlier response?). Sorry, I missed that, but I agreed it's important for us to support ListResourceBundle. > There is a solution to the split package problem, and, while it does > change the development model slightly, I think that is in line with the > more significant model change of moving to a module system... > > Introduce a package naming convention for resource bundles deployed as > modules. The convention would allow ResourceBundle.getBundle() to easily > map from the "base" name to the actual bundle package name. > > For example, if we introduce the convention that resource bundle "base" > names are mapped such that "resources" is appended to the package name, > then calling ResourceBundle.getBundle with: > > "com.acme.MyResources" > > Would actually search for resources with the base name: > > "com.acme.resources.MyResources" > > thus sidestepping the split package issue. > > Yes, this requires developers to actually define their resources using > this naming convention, but I don't see that as a significant burden. > Especially when you consider the benefit of not having to package all > translations in the primary module. > > To avoid potential package name collisions, we should consider using a > more distinct name than simply "resources", e.g.: > > "com.acme.resource_bundle.MyResources" > "com.acme.module_resources.MyResources" > "com.acme._resources_.MyResources" > > // Bryan I was under the assumption that we didn't want to change the way developers define their resources. That said, I am fine with using some packaging naming convention to sidestep the split package issue. If we go down this path, I think we will need to expand Bryan's suggestion a bit. The ResourceBundle API may search multiple resource modules in order to look up the resources (from more restrictive locales to less restrictive locales), so it is important for resources in the resource modules for each locale to have unique package name to avoid potential conflict regardless the lookup order. e.g. "com.acme.locale_en.MyResources" "com.acme.locale_ja.MyResources_ja" "com.acme.locale_fr.MyResources_fr" "com.acme.locale_fr_CH.MyResources_fr_CH" Note that the locale information is repeated twice. If we want to simplify it further, we can require the locale information to be specified only as part of the package name of the resource, but not as part of the filename of the resource: "com.acme.locale_en.MyResources" "com.acme.locale_ja.MyResources" "com.acme.locale_fr.MyResources" "com.acme.locale_fr_CH.MyResources" - Stanley From Stanley.Ho at sun.com Fri May 18 15:27:08 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Fri, 18 May 2007 15:27:08 -0700 Subject: Email problem Message-ID: <464E283C.4050207@sun.com> Hi, Apparently, my email problem has just been fixed, but you will now receive all the emails that I attempted to send/re-send in the last few days - please ignore them. Sorry for any confusion. - Stanley From Stanley.Ho at sun.com Tue May 15 19:01:08 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Tue, 15 May 2007 19:01:08 -0700 Subject: Legacy classes in a module Message-ID: <464A65E4.7030606@sun.com> Hi JSR 277 experts, This is regarding the semantic of legacy classes in a module, and is one of the outstanding issues in the updated specification (Section 2.14) that I would like to get your inputs. Typically, classes in a module are "module" classes - Java classes that have superpackage membership. However, it is also possible that developers may want to make use of legacy classes in a module, e.g. using an existing xml parser in a module. There are two possible use cases I can think of: 1. Developers simply make use of the legacy classes as implementation details in the module. In this case, the exported module classes neither expose nor reference the legacy classes in the export signatures, and the legacy classes in the module should not be made visible to the importing modules. 2. Developers use the legacy classes as part of the public API in the module. In this case, the legacy classes in the module should be made visible to the importing modules. The exported module classes may expose or reference the legacy classes in the export signatures, but this is not required. I think it's important for us to recognize and support both use cases. However, this may be difficult for us to determine which use case is the developers' real intention. Adding to the mix is that lack of module level access control for the legacy classes, and there is not much we can do once the legacy classes are leaked out from the module classloader. That said, I think it's still important for us to define the semantics for both use cases in the specification. Here is what I propose: a. Consider #1 to be the default. b. Add a new superpackage-level annotation (e.g. @ExportLegacyClasses) for developers to indicate the module is in the #2 category. c. If @ExportLegacyClasses is declared, the ModuleDefinition.getExportedClasses() would include the legacy classes. Note that unless this annotation is declared in the module definition, the importing modules will never attempt to load the legacy classes from the imported module during classloading delegation as the classloading delegation algorithm always checks getExportedClassed() first (see algorithm in 7.3.1, step 4). However, if other code obtains a reference of the module classloader somehow and invokes loadClass() directly with the legacy class name, the module classloader will still return the legacy class, and this is consistent with the existing classloader behavior. Let me know if you have inputs. If there are better ways to deal with this issue, I would like to hear them as well. - Stanley From Stanley.Ho at sun.com Fri May 18 16:39:49 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Fri, 18 May 2007 16:39:49 -0700 Subject: Legacy classes in a module In-Reply-To: <464E0550.5080708@oracle.com> References: <464E0550.5080708@oracle.com> Message-ID: <464E3945.50903@sun.com> Sounds good. Unless I hear any objection from the EG, I will go with the @ExportLegacyClasses proposal to handle the legacy classes for now. - Stanley Bryan Atsatt wrote: > Of course--we can revisit this if 294 adopts my proposed model... > > // Bryan From glyn_normington at UK.IBM.COM Sat May 19 10:52:16 2007 From: glyn_normington at UK.IBM.COM (Glyn Normington) Date: Sat, 19 May 2007 18:52:16 +0100 Subject: Legacy classes in a module In-Reply-To: <464DF393.2060808@oracle.com> Message-ID: Good point, so let's discuss the implications on the JSR 294 list in due course as it would change the class loading restrictions in the current 294 strawman. If you can expand on this aspect at all before floating your proposal, it would help us all to work through the implications. Glyn Bryan Atsatt wrote on 18/05/2007 19:42:27: > No, we haven't yet, but we should. > > Another benefit you have may have spotted is that an OSGi implementation > of 277 modules could support "split superpackages" by having the loaders > involved assign a common SuperPackage instance. While none of us likes > split packages, if we are to truly make OSGi a first-class citizen in > the 277/294 world, this side-effect of the proposal should be seriously > considered. > > // Bryan > > Glyn Normington wrote: > > > > Hi Bryan > > > > Your proposal has considerable benefits. Do you have a feel for its > > acceptability to the JSR 294 Expert Group? I don't think we've discussed > > it (properly) over there. > > > > Glyn > > > > *Bryan Atsatt * wrote on 18/05/2007 03:01:09: > > > > > I am assuming that in this scenario the module developer would be > > > allowed to make this explicit in the superpackage declaration itself. > > > That is, superpackage member declarations could include packages from > > > the "legacy" jars. 294 simply has to allow declared member packages to > > > be AWOL at compile time. > > > > > > BTW: The proposal I referred to earlier assumes that the runtime > > > artifact of a superpackage (e.g. a SuperPackage class instance) is > > > *assigned* to each class during ClassLoader.defineClass(). This > > > decouples the superpackage and class file artifacts while solving the > > > superpackage "loading" problem for the VM. It obviously makes it trivial > > > for "legacy" classes to be assigned into a superpackage as they are > > > defined. (It also makes it possible to extend the superpackage at > > > runtime, but that may be going too far.) > > > > > > // Bryan > > > > > > Stanley M. Ho wrote: > > > > Regardless, I think we still need to distinguish whether the legacy > > classes > > > > should be considered module private (use case #1) or made visible > > externally > > > > (use case #2). I don't think we can really know the developer's > > intent by > > > > looking at the legacy classes alone. Even if rewriting is involved, > > we still > > > > need to distinguish these cases in order to have the proper export > > > > information in the superpackage. > > > > > > > > - Stanley > > > > > > > > > > > > On Wed, 16 May 2007 14:17:54 -0700, Michal Cierniak > > > wrote: > > > > > > > >> I would like to add that even if superpackage classes do name their > > > >> superpackage in the new world, it wouldn't be difficult to require the > > > >> JRE to "rewrite" legacy classes on the fly if they are loaded from a > > > >> JAR embedded in a JAM. In that case, the legacy classes would end up > > > >> being members of the superpackage associated with the containing JAM. > > > >> > > > >> We might run into other problems because legacy classes do not > > > >> necessarily play by the rules of the new module-aware world but this > > > >> is independent of whether we rewrite them to add superpackage > > > >> membership info. > > > >> > > > >> Michal > > > >> > > > >> On 5/16/07, Bryan Atsatt wrote: > > > >>> While the current strawman suggests that superpackage classes > > will have > > > >>> modified class files (e.g. naming their superpackage), there is a > > > >>> proposal on the table that eliminates this requirement. > > > >>> > > > >>> If eliminated, the only requirement to include "legacy" classes in a > > > >>> superpackage will be to have the superpackage artifact present. > > In a 277 > > > >>> module, any mixture of new and "legacy" classes (likely in separate > > > >>> jars) could then fall under the umbrella superpackage. And this whole > > > >>> issue is eliminated... > > > >>> > > > >>> // Bryan > > > >>> > > > > > > > > > > > > ------------------------------------------------------------------------ > > > > / > > / > > > > /Unless stated otherwise above: > > IBM United Kingdom Limited - Registered in England and Wales with number > > 741598. > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU/ > > > > > > > > > > > > Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/attachments/20070519/86dabb40/attachment.html From Stanley.Ho at sun.com Mon May 21 14:50:09 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Mon, 21 May 2007 14:50:09 -0700 Subject: Another reminder: i18n strawman In-Reply-To: References: Message-ID: <46521411.8010101@sun.com> Hi JSR 277 experts, Since I have not heard any further input on the i18n strawman, I suppose the EG is fine with the strawman overall except the split package issue which has been raised and discussed. Unless I hear any objection, I will incorporate the i18n strawman along with the feedback you have provided into the next revision of the specification. Thanks, - Stanley From bryan.atsatt at oracle.com Mon May 21 18:21:12 2007 From: bryan.atsatt at oracle.com (Bryan Atsatt) Date: Mon, 21 May 2007 18:21:12 -0700 Subject: Another reminder: Updated JSR-277 specification (04/19/2007) In-Reply-To: <463F7C11.8090601@sun.com> References: <463A28ED.8080302@sun.com> <463A6323.1050307@oracle.com> <463F7C11.8090601@sun.com> Message-ID: <46524588.6070203@oracle.com> Hey Stanley, More comments inline. // Bryan Stanley M. Ho wrote: > Hi Bryan, > > Thanks for the inputs. See my comments inline. > > Bryan Atsatt wrote: >> Stanley, >> >> Here are my comments on both the spec and the apis. >> >> // Bryan >> >> SPEC >> ==== >> >> 1.5 >> >> You are assuming that module-name==superpackage-name, right? Otherwise, >> if we add import statements to 294, then these would have to specify >> module names, thus creating a circular dependency. > > Right. I will clarify it in the next revision of the spec. > >> 2.1 >> >> "is properly understood as a mathematical abstraction" >> >> This phrase seems unnecessary, and comes across as a bit highbrow to me. >> >> "A module archive is inherently sealed, and it does not reference any >> resource externally" >> >> The term "sealed" here could be confused with the manifest attribute. >> Perhaps a better term is "self-contained". And the second phrase is a >> bit confusing also, since imports do reference external "resources". >> Perhaps if you said "may not directly reference external classes or >> resources"? > > I will look for better words to describe it. > >> 2.2 >> >> Should state that module name must == superpackage name. >> >> 2.4 >> >> The members definition should be a list of *packages*, not classes. That >> is all the 294 proposal supports (I actually can't see it at the moment >> since my HD died, but this is my recollection!). I think it is >> reasonable to assume it will stay this way. > > The member declared in the source file is indeed a list of packages. > That said, based on my understanding of 294, the compiler would expand > the member packages to a list of classes in the artifact. I will clarify > it in the spec. > >> Version may be optional in a superpackage, but it is mandatory for a >> module, right? This should be clarified. > > "mandatory" would mean something that we can check and enforce. Since > version is declared using annotation in a superpackage, I don't think we > can enforce it at build time. That said, we can still enforce it at > deployment time if this is what you meant. Yes, the 277 spec should describe it as mandatory, and we should enforce it at runtime. If it becomes possible to enforce it a compile time, we would do so. > >> The example members list should be package names only. >> >> 2.7.3 >> >> I like this addition. > > I'm glad to know. > >> 2.7.4 >> >> There may well be cases in which a module wants to re-export a subset of >> imported classes/resources. We should consider supporting this case. > > Could you describe the actual use cases for this? Package a.b contains and exports classes C and D. Package x.y imports a.b, but only wants to re-export C. > >> Refactoring in this way results in a perhaps unexpected runtime/memory >> overhead. A pure wrapper module that re-exports must have its own class >> loader, and, even though it won't define any classes, the VM *cache* >> will be updated as that loader is used. I think we need to think hard about this issue. The OSGi model of import by *package name* decouples the importer from any explicit binding to a bundle/module name. Refactoring under that model is *much* cleaner, and far more natural. As is the usage model. After all, Foo.java import statements contain package/class names, *not* module names. Programmers think in terms of classes and packages. Peter makes this point pretty strongly, and I have to say I agree wholeheartedly: http://www.aqute.biz/Blog/2006-04-29 >> >> 4.1 >> >> Since not everyone uses the Sun terms (FCS, GA), we might want to also >> define qualifier here as "beta", or "pre-release", or >> "release-candidate". And a "release" version must not have a qualifier. > > I will clarify it. > >> 5.2.2 >> >> >> We should also make it clear that resources will also often be contained >> in "legacy" jars, for which no ClassesDirectoryPath is required. > > There is related to an open issue on how we want to support "legacy" > jars in the implementation. I will open up a new thread for the > discussion on this topic. > >> 5.6 >> >> Might be a good idea to explicitly state that the Class-Path attribute >> is ignored. > > Right. > >> #3 This would appear to rule out OSGi's support of split packages. This >> spec should enable each ModuleSystem to control this behavior. > > The JAM file is the distribution format for the module system defined by > 277, and I don't think the treatment of JAR manifest in this section > applies to OSGi. Sure, for the manifest of a .jam file. But the statement: "All packages defined in a module definition are inherently sealed, and this entry is ignored by the module system." is pretty broad, and would seem to indicate that it applies to the definitions produced by any module system. > >> 6.1 >> >> Third paragraph: Remove the leading "Besides, " >> >> 6.2.2 >> >> Repository shutdown must define the semantics for any module instances >> from that repository. >> >> I still strongly believe we need a module shutdown method that >> optionally disables the loader. >> >> And this should also be used during repository shutdown. See my comments >> below for section E.5. > > This is on my radar, and is briefly mentioned in E.4 and E.5. We will > need to discuss what kind of lifecycle support we want to provide, and I > will open a new thread for the discussion soon. > >> 6.2.3/6.3 >> >> For EE and other similar systems, it may be useful to have different >> VisibilityPolicy instances per Repository. We may want to have a >> getter/setter here, with the default implementation of get returning the >> default policy. > > Could you describe the use cases you had in mind for this? It creates a nice way to create wrapper repository instances that provide a customized view... EE systems are required to isolate applications from each other. And each may have very different external dependencies. If each repository instance can have its own VisibilityPolicy, then a wrapper Repository can be constructed for each application, using a different policy. Same goes for ImportOverridePolicy. > >> 6.2.6 >> >> Local repository instances will "know" when any of these events occurs >> (via their API). Remote ones will not, so reload() is a reasonable >> method. >> >> Should reload be *required* before any repository returns new/modified >> instances? > > My expectation is that most repository implementations would want to > optimize their performance by caching most things in memory and not poll > the underlying storages unless they have to, so I think "reload" should > be required. That said, if you have use cases that show otherwise, I > would be interested to know about them. I'm just considering the simple case of deploying a new module to a LocalRepository. I think it will be a bit surprising if some special act is required to make it available. For example, this would fail: localRepos.install(aModuleNamedSue); ModuleDefinition sue = localRepos.find("sue"); And it is pretty awkward to insert a call to reload() here. Given that the primary client of Repository is ModuleSystem, I guess the preparation logic can call reload() and re-try IFF a suitable definition cannot be found. > >> 6.5.1 >> >> It isn't clear from this that each repository instance sharing an >> interchange dir must have a unique "expansion" directory. >> >> While some variant subclass could be created that uses appropriate >> locking to share the expansion dir, the provided LocalRepository does >> not. > > I think this will be up to each repository implementation to decide. No > matter what the implementation strategy is, it should not affect the > repository's semantic. Yes, but LocalRepository is a specific implementation, and one that requires an expansion directory for each instance, right? We should make that clear. > >> 6.5.3 >> >> #5 Should be "shut down" not "shutted down". >> >> 7.1.1/7.1.2 >> >> Should say that a system property may be used to control the type >> returned by ModuleSystem.getDefault(). > > This method would be used to return the default module system which has > ties with the underlying JRE implementation. As we discussed, this is > not replaceable, so I don't think providing a system property for > override makes sense. ;) Thinking about this a bit more, the idea of a "default" module system is a bit odd. Certainly there will be one module system used by the JRE itself, and this is what you were thinking about here. And clearly LocalRepository and URLRepository are hard-wired to this same module system, so it makes some sense to be able to say in the javadoc for these classes that they use the "default" module (which needs to be added, btw). Do you have any other uses in mind? Perhaps we should be more explicit: public static ModuleSystem getJREModuleSystem(); Regardless, what I was really thinking about before is ModuleSystem initialization. ModuleDefinition has a getModuleSystem() method, but how is it implemented? Our model so far appears to assume that ModuleSystem instances will be shared, which is certainly reasonable, but... how? One simple model is that it is module system dependent, e.g. each module system implementation provides a singleton or equivalent. But this just pushes the problem up a layer, since the runtime type of ModuleDefinition will vary based on module system. I see that you added a ctor arg/getModuleSystem() method to Repository as well. In this model, the type problem is pushed up one more layer. It is the Repository implementation that must know the ModuleDefinition type, and therefore the ModuleSystem type. But we need to specify exactly how the initial repositories are configured/initialized, using the correct types, so that the JRE can instantiate them. (I took a cut at this in the class loading doc, if you recall.) This should be added to Appendix E--even if some of the details are JVM vendor specific, the requirements must at least be defined. But... didn't we decide long ago that a repository should not be restricted to a single module system? With a multi-repository search model, this is less of an issue, but why impose this limit? Composite implementations might have a hard time implementing getModuleSystem(). Perhaps we should consider a model in which we: 1. Require ModuleSystem implementations to have a globally unique name. (There aren't likely to be hundreds of them, so this shouldn't be much of a hardship!) 2. Have a persistent configuration mechanism for ModuleSystems, like the one for the initial Repository instances. In addition to the runtime types for ModuleSystem subclasses, the instances themselves will likely need configuration, just like Repositories (logging, security, import/visibility policies, etc.) 3. During startup, the JRE initializes the registered ModuleSystems, then the registered Repositories. 4. Add to ModuleSystem: public abstract String getName(); public static ModuleSystem getModuleSystem(String name); public static List getModuleSystems(); Now repository implementations are not involved in *instantiating* ModuleSystems; they look them up by name. (and having a centralized lookup mechanism may enable other interesting behaviors) > >> I think we need more flexible mechanisms to control this. At the very >> least, there should be a way to define it in the local JRE in a >> persistent fashion. >> >> But we also may want a way for the launcher to choose the default module >> system based on some delegation model. Perhaps the available systems can >> be specified in a config file, each in a standard 277 module. And the >> launcher's search algorithm could iterate across the ModuleSystem >> instances and ask each to try loading the main module. >> >> Obviously the specifics need to be worked out, but you get the idea. > > This will depend on the implementation logic in the launcher in the JDK, > but the launcher is out of scope in the 277 spec. On the other hand, if > the launcher requires APIs from 277 in order to do its work, then we > will need to consider the requirements. > >> 7.3.1 >> >> #4, bullet 2: loadClass() must not mask any thrown Error or >> RuntimeException by converting it to a ClassNotFoundException! This >> would make debugging the problem much more difficult, and would be a >> significant change in class loader behavior. > > I will make the logic more clear. > >> 7.3.2.2 >> >> The search order for resources MUST be the same as that for classes. I >> can support this statement if you like. > > How to actually support the notion of exported resources is another open > issue, and I will open a new thread on this soon. > >> We should also say something about the protocol for resource URLs. Shouldn't we? Like whether there will be a new protocol for them, or not. >> >> E.5 >> >> Again, module shutdown can only be performed within the scope of some >> larger construct, such as an EE application/container. >> >> Such an environment can and does have an explicit lifecycle model, >> including threading. >> >> Consider what can (and will!) happen after a module uninstall, if that >> module is left "alive"... >> >> If the uninstall removes the deployed module from the storage, then >> subsequent subsequent attempts to load classes/resources from that >> module can fail (depending on what is cached by the underlying >> ZipFile/file system/OS implementation). >> >> And it doesn't need to be a direct call to loadClass(). I have often >> seen this kind of failure happen with lazy dependency loading. >> >> So, the uninstall either must defer physically removing the deployed >> module until all instances are gone, or it must schedule them for >> removal on next startup, or whatever... crazy complexity. >> >> Can the storage of a shutdown repository be safely deleted? Not if any >> process is still using it. >> >> For all of these reasons, I strongly believe we need a module shutdown >> method. And yes, its use must be well documented! > > If I understand your requirements properly, I think what you are really > asking is a classloader disable/shutdown method, and such a method would > be called when a module is released (or shutdown) from the module system > in the context of EE servers. > >> API >> --- > > Since the APIs are still evolving significantly, I would like to defer > most of that discussions until the next revision of the spec and APIs > are available for the EG to review. That said, I will take your comments > into account when the APIs are refactored. > > - Stanley > > >> ModuleDefinition >> >> * The getModuleDefinitionContent() method must return a list of >> instances, since multiple jars/directories may be present. Might also >> make sense to shorten the name to getContents() since the return type >> makes it pretty clear what you are getting. >> >> >> ModuleDefinitionContent >> >> * The semantics of the open() method are fuzzy: who calls it, and when? >> Can it be safely called multiple times? What happens when the other >> methods are called and open() hasn't occurred? >> >> * Perhaps the open should be automatic on use of any accessor method if >> needed? >> >> * There should be an isOpen() method. >> >> * I think the name should be shortened to ModuleContent. It is not >> actually part of the definition. >> >> >> JamsModuleDefinition >> >> * Should be singular: JamModuleDefinition >> >> >> ImportPolicy >> >> * Should have a static getDefault() method. >> >> >> Version/VersionRange/VersionContstraint >> >> (See proposal in earlier email) >> >> >> PlatformBinding >> >> * Why not just call this "Platform"? It is no different than Version, >> just another attribute that can be used to filter during resolution. >> >> >> ModuleArchiveInfo >> >> * Need a way to get this from either a Module or a ModuleDefinition. >> >> >> Repository >> >> * Implementations of findModuleDefinition() will want to discover if the >> query contains a module name (equality test only!), so that the name can >> be used as a primary key. Either we should add a method to Query to test >> for this, or we should pass a name argument to this method. In the >> latter case, a null name must be allowed. > From heavy at UNGOVERNED.ORG Tue May 22 05:41:20 2007 From: heavy at UNGOVERNED.ORG (Richard S. Hall) Date: Tue, 22 May 2007 08:41:20 -0400 Subject: Another reminder: Updated JSR-277 specification (04/19/2007) In-Reply-To: <46524588.6070203@oracle.com> References: <463A28ED.8080302@sun.com> <463A6323.1050307@oracle.com> <463F7C11.8090601@sun.com> <46524588.6070203@oracle.com> Message-ID: <4652E4F0.3080200@ungoverned.org> Bryan Atsatt wrote: > Hey Stanley, > > More comments inline. > > // Bryan > > Stanley M. Ho wrote: >> Hi Bryan, >> >> Thanks for the inputs. See my comments inline. >> >> Bryan Atsatt wrote: >>> Stanley, >>> >>> Here are my comments on both the spec and the apis. >>> >>> // Bryan >>> >>> SPEC >>> ==== >>> >>> 1.5 >>> >>> You are assuming that module-name==superpackage-name, right? Otherwise, >>> if we add import statements to 294, then these would have to specify >>> module names, thus creating a circular dependency. >> >> Right. I will clarify it in the next revision of the spec. >> >>> 2.1 >>> >>> "is properly understood as a mathematical abstraction" >>> >>> This phrase seems unnecessary, and comes across as a bit highbrow to >>> me. >>> >>> "A module archive is inherently sealed, and it does not reference any >>> resource externally" >>> >>> The term "sealed" here could be confused with the manifest attribute. >>> Perhaps a better term is "self-contained". And the second phrase is a >>> bit confusing also, since imports do reference external "resources". >>> Perhaps if you said "may not directly reference external classes or >>> resources"? >> >> I will look for better words to describe it. >> >>> 2.2 >>> >>> Should state that module name must == superpackage name. >>> >>> 2.4 >>> >>> The members definition should be a list of *packages*, not classes. >>> That >>> is all the 294 proposal supports (I actually can't see it at the moment >>> since my HD died, but this is my recollection!). I think it is >>> reasonable to assume it will stay this way. >> >> The member declared in the source file is indeed a list of packages. >> That said, based on my understanding of 294, the compiler would expand >> the member packages to a list of classes in the artifact. I will clarify >> it in the spec. >> >>> Version may be optional in a superpackage, but it is mandatory for a >>> module, right? This should be clarified. >> >> "mandatory" would mean something that we can check and enforce. Since >> version is declared using annotation in a superpackage, I don't think we >> can enforce it at build time. That said, we can still enforce it at >> deployment time if this is what you meant. > > Yes, the 277 spec should describe it as mandatory, and we should enforce > it at runtime. If it becomes possible to enforce it a compile time, we > would do so. > >> >>> The example members list should be package names only. >>> >>> 2.7.3 >>> >>> I like this addition. >> >> I'm glad to know. >> >>> 2.7.4 >>> >>> There may well be cases in which a module wants to re-export a >>> subset of >>> imported classes/resources. We should consider supporting this case. >> >> Could you describe the actual use cases for this? > > Package a.b contains and exports classes C and D. > Package x.y imports a.b, but only wants to re-export C. I am not sure that this case is such a good idea. By allowing it, wouldn't consumers of package x.y then potentially be able to see a D from some other provider...this seems like a recipe for subtle bugs. You mention the OSGi model below, but such subsetting of packages really breaks the OSGi model of atomic packages. >>> Refactoring in this way results in a perhaps unexpected runtime/memory >>> overhead. A pure wrapper module that re-exports must have its own class >>> loader, and, even though it won't define any classes, the VM *cache* >>> will be updated as that loader is used. > > I think we need to think hard about this issue. The OSGi model of import > by *package name* decouples the importer from any explicit binding to a > bundle/module name. Refactoring under that model is *much* cleaner, and > far more natural. As is the usage model. After all, Foo.java import > statements contain package/class names, *not* module names. Programmers > think in terms of classes and packages. > > Peter makes this point pretty strongly, and I have to say I agree > wholeheartedly: > > http://www.aqute.biz/Blog/2006-04-29 Me too. -> richard >>> 4.1 >>> >>> Since not everyone uses the Sun terms (FCS, GA), we might want to also >>> define qualifier here as "beta", or "pre-release", or >>> "release-candidate". And a "release" version must not have a qualifier. >> >> I will clarify it. >> >>> 5.2.2 >>> >>> >>> We should also make it clear that resources will also often be >>> contained >>> in "legacy" jars, for which no ClassesDirectoryPath is required. >> >> There is related to an open issue on how we want to support "legacy" >> jars in the implementation. I will open up a new thread for the >> discussion on this topic. >> >>> 5.6 >>> >>> Might be a good idea to explicitly state that the Class-Path attribute >>> is ignored. >> >> Right. >> >>> #3 This would appear to rule out OSGi's support of split packages. This >>> spec should enable each ModuleSystem to control this behavior. >> >> The JAM file is the distribution format for the module system defined by >> 277, and I don't think the treatment of JAR manifest in this section >> applies to OSGi. > > Sure, for the manifest of a .jam file. But the statement: > > "All packages defined in a module definition are inherently sealed, and > this entry is ignored by the module system." > > is pretty broad, and would seem to indicate that it applies to the > definitions produced by any module system. > > >> >>> 6.1 >>> >>> Third paragraph: Remove the leading "Besides, " >>> >>> 6.2.2 >>> >>> Repository shutdown must define the semantics for any module instances >>> from that repository. >>> >>> I still strongly believe we need a module shutdown method that >>> optionally disables the loader. >>> >>> And this should also be used during repository shutdown. See my >>> comments >>> below for section E.5. >> >> This is on my radar, and is briefly mentioned in E.4 and E.5. We will >> need to discuss what kind of lifecycle support we want to provide, and I >> will open a new thread for the discussion soon. >> >>> 6.2.3/6.3 >>> >>> For EE and other similar systems, it may be useful to have different >>> VisibilityPolicy instances per Repository. We may want to have a >>> getter/setter here, with the default implementation of get returning >>> the >>> default policy. >> >> Could you describe the use cases you had in mind for this? > > It creates a nice way to create wrapper repository instances that > provide a customized view... > > EE systems are required to isolate applications from each other. And > each may have very different external dependencies. If each repository > instance can have its own VisibilityPolicy, then a wrapper Repository > can be constructed for each application, using a different policy. > > Same goes for ImportOverridePolicy. > >> >>> 6.2.6 >>> >>> Local repository instances will "know" when any of these events occurs >>> (via their API). Remote ones will not, so reload() is a reasonable >>> method. >>> >>> Should reload be *required* before any repository returns new/modified >>> instances? >> >> My expectation is that most repository implementations would want to >> optimize their performance by caching most things in memory and not poll >> the underlying storages unless they have to, so I think "reload" should >> be required. That said, if you have use cases that show otherwise, I >> would be interested to know about them. > > I'm just considering the simple case of deploying a new module to a > LocalRepository. I think it will be a bit surprising if some special act > is required to make it available. For example, this would fail: > > localRepos.install(aModuleNamedSue); > ModuleDefinition sue = localRepos.find("sue"); > > And it is pretty awkward to insert a call to reload() here. Given that > the primary client of Repository is ModuleSystem, I guess the > preparation logic can call reload() and re-try IFF a suitable definition > cannot be found. > >> >>> 6.5.1 >>> >>> It isn't clear from this that each repository instance sharing an >>> interchange dir must have a unique "expansion" directory. >>> >>> While some variant subclass could be created that uses appropriate >>> locking to share the expansion dir, the provided LocalRepository does >>> not. >> >> I think this will be up to each repository implementation to decide. No >> matter what the implementation strategy is, it should not affect the >> repository's semantic. > > Yes, but LocalRepository is a specific implementation, and one that > requires an expansion directory for each instance, right? We should make > that clear. > >> >>> 6.5.3 >>> >>> #5 Should be "shut down" not "shutted down". >>> >>> 7.1.1/7.1.2 >>> >>> Should say that a system property may be used to control the type >>> returned by ModuleSystem.getDefault(). >> >> This method would be used to return the default module system which has >> ties with the underlying JRE implementation. As we discussed, this is >> not replaceable, so I don't think providing a system property for >> override makes sense. ;) > > Thinking about this a bit more, the idea of a "default" module system is > a bit odd. Certainly there will be one module system used by the JRE > itself, and this is what you were thinking about here. And clearly > LocalRepository and URLRepository are hard-wired to this same module > system, so it makes some sense to be able to say in the javadoc for > these classes that they use the "default" module (which needs to be > added, btw). Do you have any other uses in mind? Perhaps we should be > more explicit: > > public static ModuleSystem getJREModuleSystem(); > > > Regardless, what I was really thinking about before is ModuleSystem > initialization. > > ModuleDefinition has a getModuleSystem() method, but how is it > implemented? Our model so far appears to assume that ModuleSystem > instances will be shared, which is certainly reasonable, but... how? One > simple model is that it is module system dependent, e.g. each module > system implementation provides a singleton or equivalent. But this just > pushes the problem up a layer, since the runtime type of > ModuleDefinition will vary based on module system. > > I see that you added a ctor arg/getModuleSystem() method to Repository > as well. In this model, the type problem is pushed up one more layer. It > is the Repository implementation that must know the ModuleDefinition > type, and therefore the ModuleSystem type. > > But we need to specify exactly how the initial repositories are > configured/initialized, using the correct types, so that the JRE can > instantiate them. (I took a cut at this in the class loading doc, if you > recall.) This should be added to Appendix E--even if some of the details > are JVM vendor specific, the requirements must at least be defined. > > > But... didn't we decide long ago that a repository should not be > restricted to a single module system? With a multi-repository search > model, this is less of an issue, but why impose this limit? Composite > implementations might have a hard time implementing getModuleSystem(). > > Perhaps we should consider a model in which we: > > 1. Require ModuleSystem implementations to have a globally unique name. > (There aren't likely to be hundreds of them, so this shouldn't be much > of a hardship!) > > 2. Have a persistent configuration mechanism for ModuleSystems, like the > one for the initial Repository instances. In addition to the runtime > types for ModuleSystem subclasses, the instances themselves will likely > need configuration, just like Repositories (logging, security, > import/visibility policies, etc.) > > 3. During startup, the JRE initializes the registered ModuleSystems, > then the registered Repositories. > > 4. Add to ModuleSystem: > > public abstract String getName(); > > public static ModuleSystem getModuleSystem(String name); > public static List getModuleSystems(); > > Now repository implementations are not involved in *instantiating* > ModuleSystems; they look them up by name. > > (and having a centralized lookup mechanism may enable other interesting > behaviors) > >> >>> I think we need more flexible mechanisms to control this. At the very >>> least, there should be a way to define it in the local JRE in a >>> persistent fashion. >>> >>> But we also may want a way for the launcher to choose the default >>> module >>> system based on some delegation model. Perhaps the available systems >>> can >>> be specified in a config file, each in a standard 277 module. And the >>> launcher's search algorithm could iterate across the ModuleSystem >>> instances and ask each to try loading the main module. >>> >>> Obviously the specifics need to be worked out, but you get the idea. >> >> This will depend on the implementation logic in the launcher in the JDK, >> but the launcher is out of scope in the 277 spec. On the other hand, if >> the launcher requires APIs from 277 in order to do its work, then we >> will need to consider the requirements. >> >>> 7.3.1 >>> >>> #4, bullet 2: loadClass() must not mask any thrown Error or >>> RuntimeException by converting it to a ClassNotFoundException! This >>> would make debugging the problem much more difficult, and would be a >>> significant change in class loader behavior. >> >> I will make the logic more clear. >> >>> 7.3.2.2 >>> >>> The search order for resources MUST be the same as that for classes. I >>> can support this statement if you like. >> >> How to actually support the notion of exported resources is another open >> issue, and I will open a new thread on this soon. >> >>> We should also say something about the protocol for resource URLs. > > Shouldn't we? Like whether there will be a new protocol for them, or not. > >>> >>> E.5 >>> >>> Again, module shutdown can only be performed within the scope of some >>> larger construct, such as an EE application/container. >>> >>> Such an environment can and does have an explicit lifecycle model, >>> including threading. >>> >>> Consider what can (and will!) happen after a module uninstall, if that >>> module is left "alive"... >>> >>> If the uninstall removes the deployed module from the storage, then >>> subsequent subsequent attempts to load classes/resources from that >>> module can fail (depending on what is cached by the underlying >>> ZipFile/file system/OS implementation). >>> >>> And it doesn't need to be a direct call to loadClass(). I have often >>> seen this kind of failure happen with lazy dependency loading. >>> >>> So, the uninstall either must defer physically removing the deployed >>> module until all instances are gone, or it must schedule them for >>> removal on next startup, or whatever... crazy complexity. >>> >>> Can the storage of a shutdown repository be safely deleted? Not if any >>> process is still using it. >>> >>> For all of these reasons, I strongly believe we need a module shutdown >>> method. And yes, its use must be well documented! >> >> If I understand your requirements properly, I think what you are really >> asking is a classloader disable/shutdown method, and such a method would >> be called when a module is released (or shutdown) from the module system >> in the context of EE servers. >> >>> API >>> --- >> >> Since the APIs are still evolving significantly, I would like to defer >> most of that discussions until the next revision of the spec and APIs >> are available for the EG to review. That said, I will take your comments >> into account when the APIs are refactored. >> >> - Stanley >> >> >>> ModuleDefinition >>> >>> * The getModuleDefinitionContent() method must return a list of >>> instances, since multiple jars/directories may be present. Might also >>> make sense to shorten the name to getContents() since the return type >>> makes it pretty clear what you are getting. >>> >>> >>> ModuleDefinitionContent >>> >>> * The semantics of the open() method are fuzzy: who calls it, and when? >>> Can it be safely called multiple times? What happens when the other >>> methods are called and open() hasn't occurred? >>> >>> * Perhaps the open should be automatic on use of any accessor method if >>> needed? >>> >>> * There should be an isOpen() method. >>> >>> * I think the name should be shortened to ModuleContent. It is not >>> actually part of the definition. >>> >>> >>> JamsModuleDefinition >>> >>> * Should be singular: JamModuleDefinition >>> >>> >>> ImportPolicy >>> >>> * Should have a static getDefault() method. >>> >>> >>> Version/VersionRange/VersionContstraint >>> >>> (See proposal in earlier email) >>> >>> >>> PlatformBinding >>> >>> * Why not just call this "Platform"? It is no different than Version, >>> just another attribute that can be used to filter during resolution. >>> >>> >>> ModuleArchiveInfo >>> >>> * Need a way to get this from either a Module or a ModuleDefinition. >>> >>> >>> Repository >>> >>> * Implementations of findModuleDefinition() will want to discover if >>> the >>> query contains a module name (equality test only!), so that the name >>> can >>> be used as a primary key. Either we should add a method to Query to >>> test >>> for this, or we should pass a name argument to this method. In the >>> latter case, a null name must be allowed. >> From Stanley.Ho at sun.com Tue May 22 12:55:46 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Tue, 22 May 2007 12:55:46 -0700 Subject: Another JSR-277 EDR comment Message-ID: <46534AC2.2060907@sun.com> Hi JSR 277 experts, I have received a new EDR comment [1] late last week, and I've uploaded the comment to the Confluence. The comment in [1] is around simplification in using annotations for declaring exported classes and exported resources. Since the syntax in declaring exported classes is in the scope of JSR 294 but outside the scope of JSR 277, I have forwarded them to the JSR 294 spec leads for consideration. For declaring exported resources, it is within the scope of JSR 277. However, after exchanging a few emails with the reviewer to clarify what he suggested, it is clear that the example he mentioned does not actually require resources to be exported in the first place, and it does not yield any new requirement that we need to address. If you disagree, please let me know. - Stanley [1] Early Draft Comment - 16 http://gee.cs.oswego.edu:8086/pages/viewpage.action?pageId=630 From bryan.atsatt at oracle.com Tue May 22 14:42:17 2007 From: bryan.atsatt at oracle.com (Bryan Atsatt) Date: Tue, 22 May 2007 14:42:17 -0700 Subject: Another reminder: Updated JSR-277 specification (04/19/2007) In-Reply-To: <4652E4F0.3080200@ungoverned.org> References: <463A28ED.8080302@sun.com> <463A6323.1050307@oracle.com> <463F7C11.8090601@sun.com> <46524588.6070203@oracle.com> <4652E4F0.3080200@ungoverned.org> Message-ID: <465363B9.2010504@oracle.com> Richard S. Hall wrote: > Bryan Atsatt wrote: >> Hey Stanley, >> >> More comments inline. >> >> // Bryan >> >> Stanley M. Ho wrote: >>> Hi Bryan, >>> >>> Thanks for the inputs. See my comments inline. >>> >>> Bryan Atsatt wrote: >>>> Stanley, >>>> >>>> Here are my comments on both the spec and the apis. >>>> >>>> // Bryan >>>> >>>> SPEC >>>> ==== >>>> >>>> 1.5 >>>> >>>> You are assuming that module-name==superpackage-name, right? Otherwise, >>>> if we add import statements to 294, then these would have to specify >>>> module names, thus creating a circular dependency. >>> >>> Right. I will clarify it in the next revision of the spec. >>> >>>> 2.1 >>>> >>>> "is properly understood as a mathematical abstraction" >>>> >>>> This phrase seems unnecessary, and comes across as a bit highbrow to >>>> me. >>>> >>>> "A module archive is inherently sealed, and it does not reference any >>>> resource externally" >>>> >>>> The term "sealed" here could be confused with the manifest attribute. >>>> Perhaps a better term is "self-contained". And the second phrase is a >>>> bit confusing also, since imports do reference external "resources". >>>> Perhaps if you said "may not directly reference external classes or >>>> resources"? >>> >>> I will look for better words to describe it. >>> >>>> 2.2 >>>> >>>> Should state that module name must == superpackage name. >>>> >>>> 2.4 >>>> >>>> The members definition should be a list of *packages*, not classes. >>>> That >>>> is all the 294 proposal supports (I actually can't see it at the moment >>>> since my HD died, but this is my recollection!). I think it is >>>> reasonable to assume it will stay this way. >>> >>> The member declared in the source file is indeed a list of packages. >>> That said, based on my understanding of 294, the compiler would expand >>> the member packages to a list of classes in the artifact. I will clarify >>> it in the spec. >>> >>>> Version may be optional in a superpackage, but it is mandatory for a >>>> module, right? This should be clarified. >>> >>> "mandatory" would mean something that we can check and enforce. Since >>> version is declared using annotation in a superpackage, I don't think we >>> can enforce it at build time. That said, we can still enforce it at >>> deployment time if this is what you meant. >> >> Yes, the 277 spec should describe it as mandatory, and we should enforce >> it at runtime. If it becomes possible to enforce it a compile time, we >> would do so. >> >>> >>>> The example members list should be package names only. >>>> >>>> 2.7.3 >>>> >>>> I like this addition. >>> >>> I'm glad to know. >>> >>>> 2.7.4 >>>> >>>> There may well be cases in which a module wants to re-export a >>>> subset of >>>> imported classes/resources. We should consider supporting this case. >>> >>> Could you describe the actual use cases for this? >> >> Package a.b contains and exports classes C and D. >> Package x.y imports a.b, but only wants to re-export C. > > I am not sure that this case is such a good idea. By allowing it, > wouldn't consumers of package x.y then potentially be able to see a D > from some other provider...this seems like a recipe for subtle bugs. You > mention the OSGi model below, but such subsetting of packages really > breaks the OSGi model of atomic packages. Right, I was thinking that we would enforce this during resolution, but... It is certainly simpler to simply support re-export of entire packages. It is a little weird for a.b to re-export D if C doesn't refer to D, and no classes in a.b refer to D, but this is perhaps too subtle. > >>>> Refactoring in this way results in a perhaps unexpected runtime/memory >>>> overhead. A pure wrapper module that re-exports must have its own class >>>> loader, and, even though it won't define any classes, the VM *cache* >>>> will be updated as that loader is used. >> >> I think we need to think hard about this issue. The OSGi model of import >> by *package name* decouples the importer from any explicit binding to a >> bundle/module name. Refactoring under that model is *much* cleaner, and >> far more natural. As is the usage model. After all, Foo.java import >> statements contain package/class names, *not* module names. Programmers >> think in terms of classes and packages. >> >> Peter makes this point pretty strongly, and I have to say I agree >> wholeheartedly: >> >> http://www.aqute.biz/Blog/2006-04-29 > > Me too. > > -> richard > >>>> 4.1 >>>> >>>> Since not everyone uses the Sun terms (FCS, GA), we might want to also >>>> define qualifier here as "beta", or "pre-release", or >>>> "release-candidate". And a "release" version must not have a qualifier. >>> >>> I will clarify it. >>> >>>> 5.2.2 >>>> >>>> >>>> We should also make it clear that resources will also often be >>>> contained >>>> in "legacy" jars, for which no ClassesDirectoryPath is required. >>> >>> There is related to an open issue on how we want to support "legacy" >>> jars in the implementation. I will open up a new thread for the >>> discussion on this topic. >>> >>>> 5.6 >>>> >>>> Might be a good idea to explicitly state that the Class-Path attribute >>>> is ignored. >>> >>> Right. >>> >>>> #3 This would appear to rule out OSGi's support of split packages. This >>>> spec should enable each ModuleSystem to control this behavior. >>> >>> The JAM file is the distribution format for the module system defined by >>> 277, and I don't think the treatment of JAR manifest in this section >>> applies to OSGi. >> >> Sure, for the manifest of a .jam file. But the statement: >> >> "All packages defined in a module definition are inherently sealed, and >> this entry is ignored by the module system." >> >> is pretty broad, and would seem to indicate that it applies to the >> definitions produced by any module system. >> >> >>> >>>> 6.1 >>>> >>>> Third paragraph: Remove the leading "Besides, " >>>> >>>> 6.2.2 >>>> >>>> Repository shutdown must define the semantics for any module instances >>>> from that repository. >>>> >>>> I still strongly believe we need a module shutdown method that >>>> optionally disables the loader. >>>> >>>> And this should also be used during repository shutdown. See my >>>> comments >>>> below for section E.5. >>> >>> This is on my radar, and is briefly mentioned in E.4 and E.5. We will >>> need to discuss what kind of lifecycle support we want to provide, and I >>> will open a new thread for the discussion soon. >>> >>>> 6.2.3/6.3 >>>> >>>> For EE and other similar systems, it may be useful to have different >>>> VisibilityPolicy instances per Repository. We may want to have a >>>> getter/setter here, with the default implementation of get returning >>>> the >>>> default policy. >>> >>> Could you describe the use cases you had in mind for this? >> >> It creates a nice way to create wrapper repository instances that >> provide a customized view... >> >> EE systems are required to isolate applications from each other. And >> each may have very different external dependencies. If each repository >> instance can have its own VisibilityPolicy, then a wrapper Repository >> can be constructed for each application, using a different policy. >> >> Same goes for ImportOverridePolicy. >> >>> >>>> 6.2.6 >>>> >>>> Local repository instances will "know" when any of these events occurs >>>> (via their API). Remote ones will not, so reload() is a reasonable >>>> method. >>>> >>>> Should reload be *required* before any repository returns new/modified >>>> instances? >>> >>> My expectation is that most repository implementations would want to >>> optimize their performance by caching most things in memory and not poll >>> the underlying storages unless they have to, so I think "reload" should >>> be required. That said, if you have use cases that show otherwise, I >>> would be interested to know about them. >> >> I'm just considering the simple case of deploying a new module to a >> LocalRepository. I think it will be a bit surprising if some special act >> is required to make it available. For example, this would fail: >> >> localRepos.install(aModuleNamedSue); >> ModuleDefinition sue = localRepos.find("sue"); >> >> And it is pretty awkward to insert a call to reload() here. Given that >> the primary client of Repository is ModuleSystem, I guess the >> preparation logic can call reload() and re-try IFF a suitable definition >> cannot be found. >> >>> >>>> 6.5.1 >>>> >>>> It isn't clear from this that each repository instance sharing an >>>> interchange dir must have a unique "expansion" directory. >>>> >>>> While some variant subclass could be created that uses appropriate >>>> locking to share the expansion dir, the provided LocalRepository does >>>> not. >>> >>> I think this will be up to each repository implementation to decide. No >>> matter what the implementation strategy is, it should not affect the >>> repository's semantic. >> >> Yes, but LocalRepository is a specific implementation, and one that >> requires an expansion directory for each instance, right? We should make >> that clear. >> >>> >>>> 6.5.3 >>>> >>>> #5 Should be "shut down" not "shutted down". >>>> >>>> 7.1.1/7.1.2 >>>> >>>> Should say that a system property may be used to control the type >>>> returned by ModuleSystem.getDefault(). >>> >>> This method would be used to return the default module system which has >>> ties with the underlying JRE implementation. As we discussed, this is >>> not replaceable, so I don't think providing a system property for >>> override makes sense. ;) >> >> Thinking about this a bit more, the idea of a "default" module system is >> a bit odd. Certainly there will be one module system used by the JRE >> itself, and this is what you were thinking about here. And clearly >> LocalRepository and URLRepository are hard-wired to this same module >> system, so it makes some sense to be able to say in the javadoc for >> these classes that they use the "default" module (which needs to be >> added, btw). Do you have any other uses in mind? Perhaps we should be >> more explicit: >> >> public static ModuleSystem getJREModuleSystem(); >> >> >> Regardless, what I was really thinking about before is ModuleSystem >> initialization. >> >> ModuleDefinition has a getModuleSystem() method, but how is it >> implemented? Our model so far appears to assume that ModuleSystem >> instances will be shared, which is certainly reasonable, but... how? One >> simple model is that it is module system dependent, e.g. each module >> system implementation provides a singleton or equivalent. But this just >> pushes the problem up a layer, since the runtime type of >> ModuleDefinition will vary based on module system. >> >> I see that you added a ctor arg/getModuleSystem() method to Repository >> as well. In this model, the type problem is pushed up one more layer. It >> is the Repository implementation that must know the ModuleDefinition >> type, and therefore the ModuleSystem type. >> >> But we need to specify exactly how the initial repositories are >> configured/initialized, using the correct types, so that the JRE can >> instantiate them. (I took a cut at this in the class loading doc, if you >> recall.) This should be added to Appendix E--even if some of the details >> are JVM vendor specific, the requirements must at least be defined. >> >> >> But... didn't we decide long ago that a repository should not be >> restricted to a single module system? With a multi-repository search >> model, this is less of an issue, but why impose this limit? Composite >> implementations might have a hard time implementing getModuleSystem(). >> >> Perhaps we should consider a model in which we: >> >> 1. Require ModuleSystem implementations to have a globally unique name. >> (There aren't likely to be hundreds of them, so this shouldn't be much >> of a hardship!) >> >> 2. Have a persistent configuration mechanism for ModuleSystems, like the >> one for the initial Repository instances. In addition to the runtime >> types for ModuleSystem subclasses, the instances themselves will likely >> need configuration, just like Repositories (logging, security, >> import/visibility policies, etc.) >> >> 3. During startup, the JRE initializes the registered ModuleSystems, >> then the registered Repositories. >> >> 4. Add to ModuleSystem: >> >> public abstract String getName(); >> >> public static ModuleSystem getModuleSystem(String name); >> public static List getModuleSystems(); >> >> Now repository implementations are not involved in *instantiating* >> ModuleSystems; they look them up by name. >> >> (and having a centralized lookup mechanism may enable other interesting >> behaviors) >> >>> >>>> I think we need more flexible mechanisms to control this. At the very >>>> least, there should be a way to define it in the local JRE in a >>>> persistent fashion. >>>> >>>> But we also may want a way for the launcher to choose the default >>>> module >>>> system based on some delegation model. Perhaps the available systems >>>> can >>>> be specified in a config file, each in a standard 277 module. And the >>>> launcher's search algorithm could iterate across the ModuleSystem >>>> instances and ask each to try loading the main module. >>>> >>>> Obviously the specifics need to be worked out, but you get the idea. >>> >>> This will depend on the implementation logic in the launcher in the JDK, >>> but the launcher is out of scope in the 277 spec. On the other hand, if >>> the launcher requires APIs from 277 in order to do its work, then we >>> will need to consider the requirements. >>> >>>> 7.3.1 >>>> >>>> #4, bullet 2: loadClass() must not mask any thrown Error or >>>> RuntimeException by converting it to a ClassNotFoundException! This >>>> would make debugging the problem much more difficult, and would be a >>>> significant change in class loader behavior. >>> >>> I will make the logic more clear. >>> >>>> 7.3.2.2 >>>> >>>> The search order for resources MUST be the same as that for classes. I >>>> can support this statement if you like. >>> >>> How to actually support the notion of exported resources is another open >>> issue, and I will open a new thread on this soon. >>> >>>> We should also say something about the protocol for resource URLs. >> >> Shouldn't we? Like whether there will be a new protocol for them, or not. >> >>>> >>>> E.5 >>>> >>>> Again, module shutdown can only be performed within the scope of some >>>> larger construct, such as an EE application/container. >>>> >>>> Such an environment can and does have an explicit lifecycle model, >>>> including threading. >>>> >>>> Consider what can (and will!) happen after a module uninstall, if that >>>> module is left "alive"... >>>> >>>> If the uninstall removes the deployed module from the storage, then >>>> subsequent subsequent attempts to load classes/resources from that >>>> module can fail (depending on what is cached by the underlying >>>> ZipFile/file system/OS implementation). >>>> >>>> And it doesn't need to be a direct call to loadClass(). I have often >>>> seen this kind of failure happen with lazy dependency loading. >>>> >>>> So, the uninstall either must defer physically removing the deployed >>>> module until all instances are gone, or it must schedule them for >>>> removal on next startup, or whatever... crazy complexity. >>>> >>>> Can the storage of a shutdown repository be safely deleted? Not if any >>>> process is still using it. >>>> >>>> For all of these reasons, I strongly believe we need a module shutdown >>>> method. And yes, its use must be well documented! >>> >>> If I understand your requirements properly, I think what you are really >>> asking is a classloader disable/shutdown method, and such a method would >>> be called when a module is released (or shutdown) from the module system >>> in the context of EE servers. >>> >>>> API >>>> --- >>> >>> Since the APIs are still evolving significantly, I would like to defer >>> most of that discussions until the next revision of the spec and APIs >>> are available for the EG to review. That said, I will take your comments >>> into account when the APIs are refactored. >>> >>> - Stanley >>> >>> >>>> ModuleDefinition >>>> >>>> * The getModuleDefinitionContent() method must return a list of >>>> instances, since multiple jars/directories may be present. Might also >>>> make sense to shorten the name to getContents() since the return type >>>> makes it pretty clear what you are getting. >>>> >>>> >>>> ModuleDefinitionContent >>>> >>>> * The semantics of the open() method are fuzzy: who calls it, and when? >>>> Can it be safely called multiple times? What happens when the other >>>> methods are called and open() hasn't occurred? >>>> >>>> * Perhaps the open should be automatic on use of any accessor method if >>>> needed? >>>> >>>> * There should be an isOpen() method. >>>> >>>> * I think the name should be shortened to ModuleContent. It is not >>>> actually part of the definition. >>>> >>>> >>>> JamsModuleDefinition >>>> >>>> * Should be singular: JamModuleDefinition >>>> >>>> >>>> ImportPolicy >>>> >>>> * Should have a static getDefault() method. >>>> >>>> >>>> Version/VersionRange/VersionContstraint >>>> >>>> (See proposal in earlier email) >>>> >>>> >>>> PlatformBinding >>>> >>>> * Why not just call this "Platform"? It is no different than Version, >>>> just another attribute that can be used to filter during resolution. >>>> >>>> >>>> ModuleArchiveInfo >>>> >>>> * Need a way to get this from either a Module or a ModuleDefinition. >>>> >>>> >>>> Repository >>>> >>>> * Implementations of findModuleDefinition() will want to discover if >>>> the >>>> query contains a module name (equality test only!), so that the name >>>> can >>>> be used as a primary key. Either we should add a method to Query to >>>> test >>>> for this, or we should pass a name argument to this method. In the >>>> latter case, a null name must be allowed. >>> > From Stanley.Ho at sun.com Tue May 22 15:54:42 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Tue, 22 May 2007 15:54:42 -0700 Subject: Strawman: Services and service-providers support Message-ID: <465374B2.7060501@sun.com> Hi JSR 277 experts, I've been working with Peter von de Ahe and Dave Bristor in the Java SE Tools and Libraries team to come up with a strawman for the services and service-providers support. This strawman was developed based on the initial requirements we discussed in the EG several weeks ago, and it defines how the module system will support the service/service-provider model defined in the java.util.ServiceLoader API. I have uploaded the strawman [1] to the expert group home page for download. If you prefer to receive them in email, please let me know and I'll send it to you directly. I would like to get your inputs by 6/5/2007, so we can determine how to refine the strawman further. Thanks, - Stanley [1] Services and service-providers strawman http://www.jcp.org/en/eg/download/jsr-277-service-provider-strawman-05222007.pdf?id=277&fileId=3438 From Stanley.Ho at sun.com Tue May 22 18:49:59 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Tue, 22 May 2007 18:49:59 -0700 Subject: Bryan's comments (was: Re: Another reminder...) In-Reply-To: <46524588.6070203@oracle.com> References: <463A28ED.8080302@sun.com> <463A6323.1050307@oracle.com> <463F7C11.8090601@sun.com> <46524588.6070203@oracle.com> Message-ID: <46539DC7.7090300@sun.com> Hi Bryan, Bryan Atsatt wrote: >> "mandatory" would mean something that we can check and enforce. Since >> version is declared using annotation in a superpackage, I don't think we >> can enforce it at build time. That said, we can still enforce it at >> deployment time if this is what you meant. > > Yes, the 277 spec should describe it as mandatory, and we should enforce > it at runtime. If it becomes possible to enforce it a compile time, we > would do so. Based on the updated specification, the module system will provide a default version for the module definition which has no version declared in the metadata, so each module definition will always have a version. Is there any strong reason why we should reject all the module definitions which have no version declared in the metadata rather than giving them a default version number? >>> 2.7.4 >>> >>> There may well be cases in which a module wants to re-export a subset of >>> imported classes/resources. We should consider supporting this case. >> >> Could you describe the actual use cases for this? > > Package a.b contains and exports classes C and D. > Package x.y imports a.b, but only wants to re-export C. I have the same concern as Richard. Unless there are real use cases for this, I suggest we should go with re-export at the module granularity as we discussed before. >>> Refactoring in this way results in a perhaps unexpected runtime/memory >>> overhead. A pure wrapper module that re-exports must have its own class >>> loader, and, even though it won't define any classes, the VM *cache* >>> will be updated as that loader is used. If there is no support for re-exporting subset of imported classes/resources, this becomes a non-issue. > I think we need to think hard about this issue. The OSGi model of import > by *package name* decouples the importer from any explicit binding to a > bundle/module name. Refactoring under that model is *much* cleaner, and > far more natural. As is the usage model. After all, Foo.java import > statements contain package/class names, *not* module names. Programmers > think in terms of classes and packages. > > Peter makes this point pretty strongly, and I have to say I agree > wholeheartedly: > > http://www.aqute.biz/Blog/2006-04-29 I agreed that in some situations it is much better to have dependency that is loosely coupled. You may want to check out the service-providers strawman that I just sent out, and it deals with the exact issue around API vs implementation. >>> 5.6 >>> > Sure, for the manifest of a .jam file. But the statement: > > "All packages defined in a module definition are inherently sealed, and > this entry is ignored by the module system." > > is pretty broad, and would seem to indicate that it applies to the > definitions produced by any module system. I will see if I can clarify it. >>> 6.2.3/6.3 >>> >>> For EE and other similar systems, it may be useful to have different >>> VisibilityPolicy instances per Repository. We may want to have a >>> getter/setter here, with the default implementation of get returning the >>> default policy. >> >> Could you describe the use cases you had in mind for this? > > It creates a nice way to create wrapper repository instances that > provide a customized view... > > EE systems are required to isolate applications from each other. And > each may have very different external dependencies. If each repository > instance can have its own VisibilityPolicy, then a wrapper Repository > can be constructed for each application, using a different policy. > > Same goes for ImportOverridePolicy. Interesting. I will look into this further. >>> 6.2.6 > > I'm just considering the simple case of deploying a new module to a > LocalRepository. I think it will be a bit surprising if some special act > is required to make it available. For example, this would fail: > > localRepos.install(aModuleNamedSue); > ModuleDefinition sue = localRepos.find("sue"); > > And it is pretty awkward to insert a call to reload() here. Given that > the primary client of Repository is ModuleSystem, I guess the > preparation logic can call reload() and re-try IFF a suitable definition > cannot be found. I see your point. The good news is that the updated specification does not require "reload" to be invoked in your example because a module definition would be available for subsequence searches after it has been installed, see Section 6.2.4 in the specification. I hope this would address your concern. >>> 6.5.1 > > Yes, but LocalRepository is a specific implementation, and one that > requires an expansion directory for each instance, right? We should make > that clear. No. If two LocalRepository instances load module definitions from the same source location, the implementation may share the same expansion directory, or maybe not. I think the specification should leave this up to the implementors, and I don't see good reason to dictate one implementation approach over the other. >>> 7.1.1/7.1.2 >>> > Thinking about this a bit more, the idea of a "default" module system is > a bit odd. Certainly there will be one module system used by the JRE > itself, and this is what you were thinking about here. And clearly > LocalRepository and URLRepository are hard-wired to this same module > system, so it makes some sense to be able to say in the javadoc for > these classes that they use the "default" module (which needs to be > added, btw). Do you have any other uses in mind? Perhaps we should be > more explicit: > > public static ModuleSystem getJREModuleSystem(); > > Regardless, what I was really thinking about before is ModuleSystem > initialization. > > ModuleDefinition has a getModuleSystem() method, but how is it > implemented? Our model so far appears to assume that ModuleSystem > instances will be shared, which is certainly reasonable, but... how? One > simple model is that it is module system dependent, e.g. each module > system implementation provides a singleton or equivalent. But this just > pushes the problem up a layer, since the runtime type of > ModuleDefinition will vary based on module system. As I mentioned previously, the APIs are still evolving significantly, and they do not fully reflect what I have in mind. More specifically, I've been planning to make ModuleSystem a service so its providers can be discovered through the java.util.ServiceLoader API. (Now the service providers strawman is available, it should help our discussion.) This will make the static method above go away, and the repository implementations can simply make use of the ServiceLoader API to discover the specific ModuleSystem provider they need. > I see that you added a ctor arg/getModuleSystem() method to Repository > as well. In this model, the type problem is pushed up one more layer. It > is the Repository implementation that must know the ModuleDefinition > type, and therefore the ModuleSystem type. > > But we need to specify exactly how the initial repositories are > configured/initialized, using the correct types, so that the JRE can > instantiate them. (I took a cut at this in the class loading doc, if you > recall.) This should be added to Appendix E--even if some of the details > are JVM vendor specific, the requirements must at least be defined. I expect the repository configuration in the default implementation will be documented as part of the JDK documents, similar to what has been done to the security policy and other configuration files in the JDK: http://java.sun.com/javase/6/docs/technotes/guides/security/PolicyFiles.html This way, different JRE implementations are allowed to innovate and we don't have to put these implementation-specific details into the specification. > But... didn't we decide long ago that a repository should not be > restricted to a single module system? With a multi-repository search > model, this is less of an issue, but why impose this limit? Composite > implementations might have a hard time implementing getModuleSystem(). Right. Keep in mind that the APIs are still evolving. I expect this will be changed when the interop proposal is in place. > Perhaps we should consider a model in which we: > > 1. Require ModuleSystem implementations to have a globally unique name. > (There aren't likely to be hundreds of them, so this shouldn't be much > of a hardship!) > > 2. Have a persistent configuration mechanism for ModuleSystems, like the > one for the initial Repository instances. In addition to the runtime > types for ModuleSystem subclasses, the instances themselves will likely > need configuration, just like Repositories (logging, security, > import/visibility policies, etc.) > > 3. During startup, the JRE initializes the registered ModuleSystems, > then the registered Repositories. > > 4. Add to ModuleSystem: > > public abstract String getName(); > > public static ModuleSystem getModuleSystem(String name); > public static List getModuleSystems(); > > Now repository implementations are not involved in *instantiating* > ModuleSystems; they look them up by name. > > (and having a centralized lookup mechanism may enable other interesting > behaviors) If we make ModuleSystem a service type, you will be able to perform this kind of discovery easily using the ServiceLoader API, without requiring a different set of API to be created for similar purpose. There is also no need to have persistent configuration mechanism for ModuleSystem; if a custom module system implementation is deployed as a service-provider module into the system repository, you would be able to use the API to look it up. >>> 7.3.2.2 >> >>> We should also say something about the protocol for resource URLs. > > Shouldn't we? Like whether there will be a new protocol for them, or not. My current thinking is that the protocol in the resource URLs would be entirely up to each repository implementation. Since there will be custom repository implementations, we can't foresee what their requirements are, so it would be difficult to define a standard protocol that is suitable across all possible repository implementations. If you think otherwise, let me know. I will clarify this in the next revision of the specification. - Stanley From Stanley.Ho at sun.com Tue May 22 23:16:14 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Tue, 22 May 2007 23:16:14 -0700 Subject: Traveling ... Message-ID: <4653DC2E.7030105@sun.com> Hi JSR 277 experts, I'll be away starting tomorrow for about two weeks. During this period, I'll be mostly traveling (a few days on vacation) and have limited access to emails. I will try to continue the discussion in the EG even when I am away, but it will take me longer time to response. - Stanley From Stanley.Ho at sun.com Wed May 23 02:59:57 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Wed, 23 May 2007 02:59:57 -0700 Subject: Exported resources Message-ID: <4654109D.40205@sun.com> Hi JSR 277 experts, This is regarding the semantic of exported resources in a module, and it is one of the outstanding issues in the updated specification (Section 7.3.2.2) that I would like to get your inputs. There are two aspects of the issue: 1. Search order for resources in module classloader This is related to what happens when a resource is requested through the module classloader: module.getClassLoader().getResource(String) module.getClassLoader().getResourceAsStream(String) module.getClassLoader().getResources(String) The updated specification already defines the search order for classes in the module classloader based on the declared import order and re-export (see Section 7.3.1), and classloading is delegated based on the exported classes from the imported modules. In our earlier discussion, I think many EG members expected that the search order for resources would be similar to that for classes. Therefore, I propose that the search order for resources in the module classloader would also be based on the declared import order and re-export similar to the algorithm described in Section 7.3.1, and resource loading would be delegated based on isResourceExported() from the imported modules accordingly. 2. Enforcement of access control There are two notions of resources: exported and private. Ideally, if a resource is exported, it would be visible and accessible through the module classloader regardless the caller; on the other hand, if a resource is private, it would be visible and accessible through the module classloader only if the caller is the module containing the resource. Unlike Java classes, there is no access control supported for resources in the JVM, so we will need to come up a scheme to support the semantic we want. Supporting exported resources should be straightforward - the module classloader can simply return the exported resources whenever they are requested; supporting private resources is a bit complicated, and this is what I want to go into more details. To enforce the access of private resources, one reasonable approach is based on security permission. Each module would be assigned with certain permission to access its own resources. When a private resource is requested though the getResource()/getResourceAsStream()/getResources() method of the module classloader, the module classloader would perform security check to determine if the caller has the appropriate permission before the private resource is returned; otherwise, the module classloader would pretend the requested resource does not exist. Apparently, the .NET Framework uses a similar approach to enforce access to resources embedded/linked in the Assembly: http://msdn2.microsoft.com/en-us/library/xc4235zt.aspx http://msdn2.microsoft.com/en-us/library/5kx66y1a.aspx This security permission approach should be sufficient to prevent untrusted code (e.g. applets, JNLP applications, etc.) to access private resources in other modules to exploit potential vulnerability. On the other hand, if the caller has all permissions, it can freely access the private resources in other modules, and this security permission approach would break down. That said, if the caller has all permissions, it can pretty much do whatever it wants anyway, and this is consistent with the existing behavior. Note that if a module has an exported class and one of its public method loads private resources internally, the developers might be required to put a doPrivileged() block around the resource loading code in this method for the private resources to be loaded successfully. Another implication is that the performance of accessing private resources would be slower compared to exported resources, because of the overhead of the permission check; this performance characteristic may not be what developers would expect. That's it. Let me know your inputs. If you have better suggestions to deal with this issue, I would definitely like to hear them. - Stanley From glyn_normington at UK.IBM.COM Wed May 23 07:35:34 2007 From: glyn_normington at UK.IBM.COM (Glyn Normington) Date: Wed, 23 May 2007 15:35:34 +0100 Subject: Strawman: Services and service-providers support In-Reply-To: <465374B2.7060501@sun.com> Message-ID: Hi Stanley Don't you think we should make the strawman available to observers of this list so they have the context to follow subsequent discussion of the strawman and send in any insights they may have? Glyn "Stanley M. Ho" wrote on 22/05/2007 23:54:42: > Hi JSR 277 experts, > > I've been working with Peter von de Ahe and Dave Bristor in the Java SE > Tools and Libraries team to come up with a strawman for the services and > service-providers support. This strawman was developed based on the > initial requirements we discussed in the EG several weeks ago, and it > defines how the module system will support the service/service-provider > model defined in the java.util.ServiceLoader API. > > I have uploaded the strawman [1] to the expert group home page for > download. If you prefer to receive them in email, please let me know and > I'll send it to you directly. > > I would like to get your inputs by 6/5/2007, so we can determine how to > refine the strawman further. > > Thanks, > - Stanley > > [1] Services and service-providers strawman > http://www.jcp.org/en/eg/download/jsr-277-service-provider- > strawman-05222007.pdf?id=277&fileId=3438 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/attachments/20070523/86c217ef/attachment.html From bryan.atsatt at oracle.com Wed May 23 11:14:41 2007 From: bryan.atsatt at oracle.com (Bryan Atsatt) Date: Wed, 23 May 2007 11:14:41 -0700 Subject: Query... Message-ID: <46548491.7040408@oracle.com> When I designed the Query/Repository model in the prototype, I had it firmly in my mind that Query should be Serializable, so that it could be sent over the wire to a remote repository. I've described it this way to a number of people. But apparently I forgot to actually declare it. :^) And we haven't done so in the spec, either. I think we should. // Bryan From bryan.atsatt at oracle.com Wed May 23 11:25:58 2007 From: bryan.atsatt at oracle.com (Bryan Atsatt) Date: Wed, 23 May 2007 11:25:58 -0700 Subject: Exported resources In-Reply-To: <4654109D.40205@sun.com> References: <4654109D.40205@sun.com> Message-ID: <46548736.7020507@oracle.com> 1. Definitely agree that resource search order should be identical to class search order. 2. Using permissions to limit access to private resources seems like overkill to me. The prototype implemented this in a very simple fashion: a. If resource is exported, return it, else a. Get the caller's Module (get class from stack, get module from it) b. If callerModule == this, return resource, else return null. // Bryan Stanley M. Ho wrote: > Hi JSR 277 experts, > > This is regarding the semantic of exported resources in a module, and it > is one of the outstanding issues in the updated specification (Section > 7.3.2.2) that I would like to get your inputs. There are two aspects of > the issue: > > 1. Search order for resources in module classloader > > This is related to what happens when a resource is requested through the > module classloader: > > module.getClassLoader().getResource(String) > module.getClassLoader().getResourceAsStream(String) > module.getClassLoader().getResources(String) > > The updated specification already defines the search order for classes > in the module classloader based on the declared import order and > re-export (see Section 7.3.1), and classloading is delegated based on > the exported classes from the imported modules. > > In our earlier discussion, I think many EG members expected that the > search order for resources would be similar to that for classes. > Therefore, I propose that the search order for resources in the module > classloader would also be based on the declared import order and > re-export similar to the algorithm described in Section 7.3.1, and > resource loading would be delegated based on isResourceExported() from > the imported modules accordingly. > > > 2. Enforcement of access control > > There are two notions of resources: exported and private. Ideally, if a > resource is exported, it would be visible and accessible through the > module classloader regardless the caller; on the other hand, if a > resource is private, it would be visible and accessible through the > module classloader only if the caller is the module containing the > resource. > > Unlike Java classes, there is no access control supported for resources > in the JVM, so we will need to come up a scheme to support the semantic > we want. Supporting exported resources should be straightforward - the > module classloader can simply return the exported resources whenever > they are requested; supporting private resources is a bit complicated, > and this is what I want to go into more details. > > To enforce the access of private resources, one reasonable approach is > based on security permission. Each module would be assigned with certain > permission to access its own resources. When a private resource is > requested though the getResource()/getResourceAsStream()/getResources() > method of the module classloader, the module classloader would perform > security check to determine if the caller has the appropriate permission > before the private resource is returned; otherwise, the module > classloader would pretend the requested resource does not exist. > Apparently, the .NET Framework uses a similar approach to enforce access > to resources embedded/linked in the Assembly: > > http://msdn2.microsoft.com/en-us/library/xc4235zt.aspx > http://msdn2.microsoft.com/en-us/library/5kx66y1a.aspx > > This security permission approach should be sufficient to prevent > untrusted code (e.g. applets, JNLP applications, etc.) to access private > resources in other modules to exploit potential vulnerability. On the > other hand, if the caller has all permissions, it can freely access the > private resources in other modules, and this security permission > approach would break down. That said, if the caller has all permissions, > it can pretty much do whatever it wants anyway, and this is consistent > with the existing behavior. Note that if a module has an exported class > and one of its public method loads private resources internally, the > developers might be required to put a doPrivileged() block around the > resource loading code in this method for the private resources to be > loaded successfully. Another implication is that the performance of > accessing private resources would be slower compared to exported > resources, because of the overhead of the permission check; this > performance characteristic may not be what developers would expect. > > That's it. Let me know your inputs. If you have better suggestions to > deal with this issue, I would definitely like to hear them. > > - Stanley > From bryan.atsatt at oracle.com Wed May 23 12:52:47 2007 From: bryan.atsatt at oracle.com (Bryan Atsatt) Date: Wed, 23 May 2007 12:52:47 -0700 Subject: Bryan's comments In-Reply-To: <46539DC7.7090300@sun.com> References: <463A28ED.8080302@sun.com> <463A6323.1050307@oracle.com> <463F7C11.8090601@sun.com> <46524588.6070203@oracle.com> <46539DC7.7090300@sun.com> Message-ID: <46549B8F.3090403@oracle.com> Stanley M. Ho wrote: > Hi Bryan, > > Bryan Atsatt wrote: >>> "mandatory" would mean something that we can check and enforce. Since >>> version is declared using annotation in a superpackage, I don't think we >>> can enforce it at build time. That said, we can still enforce it at >>> deployment time if this is what you meant. >> >> Yes, the 277 spec should describe it as mandatory, and we should enforce >> it at runtime. If it becomes possible to enforce it a compile time, we >> would do so. > > Based on the updated specification, the module system will provide a > default version for the module definition which has no version declared > in the metadata, so each module definition will always have a version. > Is there any strong reason why we should reject all the module > definitions which have no version declared in the metadata rather than > giving them a default version number? It is more philosophical than technical: if developers are forced to declare it, they must be conscious of the implications of that choice. > >>>> 2.7.4 >>>> >>>> There may well be cases in which a module wants to re-export a >>>> subset of >>>> imported classes/resources. We should consider supporting this case. >>> >>> Could you describe the actual use cases for this? >> >> Package a.b contains and exports classes C and D. >> Package x.y imports a.b, but only wants to re-export C. > > I have the same concern as Richard. Unless there are real use cases for > this, I suggest we should go with re-export at the module granularity as > we discussed before. > >>>> Refactoring in this way results in a perhaps unexpected runtime/memory >>>> overhead. A pure wrapper module that re-exports must have its own class >>>> loader, and, even though it won't define any classes, the VM *cache* >>>> will be updated as that loader is used. > > If there is no support for re-exporting subset of imported > classes/resources, this becomes a non-issue. Sorry, but this doesn't follow. The VM class cache is updated (at least) when: loader.defineClass() is called. VM calls loader.internalLoadClass() Class.forName() is called. The cache records that the initiating loader returned a specific class, regardless of what loader defined it. This enables loader.findLoadedClass() to work as expected. Given that: a. Module has-a single ClassLoader, and b. A specific Module re-exports classes from an imported Module. we have two scenarios to consider: 1. Module has both member and re-exported classes. The loader will both define member classes and the VM cache will be updated with mappings from Module loader to imported-module-Class instances. 2. Module has no member classes, only re-exported classes. The cache will be updated with mappings from Module loader to imported-module-Class instances. In the latter case, if there is only a single imported module, an optimization could be that this view module returns the imported module's loader rather than having it's own. But this is not possible when there are imports from more than one module: a unique loader instance must be created to satisfy the api contract module.getClassLoader(). Even though it will never define any classes, the VM will still update the cache. This latter case is what I am referring to as overhead. Of course, with appropriate updates to the VM/caching behavior, this overhead could be eliminated. But this would have to be addressed or the overhead *will* exist. > >> I think we need to think hard about this issue. The OSGi model of import >> by *package name* decouples the importer from any explicit binding to a >> bundle/module name. Refactoring under that model is *much* cleaner, and >> far more natural. As is the usage model. After all, Foo.java import >> statements contain package/class names, *not* module names. Programmers >> think in terms of classes and packages. >> >> Peter makes this point pretty strongly, and I have to say I agree >> wholeheartedly: >> >> http://www.aqute.biz/Blog/2006-04-29 > > I agreed that in some situations it is much better to have dependency > that is loosely coupled. You may want to check out the service-providers > strawman that I just sent out, and it deals with the exact issue around > API vs implementation. That's great, but sort of beside the point :^). I am specifically referring to the syntax and semantics of "import" declarations. At the moment, 277 supports only the "Reguire Bundle" style semantics defined by OSGi. This model is is inherently tightly-coupled, and its use is greatly discouraged in the OSGi world. It was not even present in the initial releases. I strongly believe that it would be a huge mistake for 277 to support only this model. If we want to simplify things and support only one model, then we should choose import by package name/version, *not* import by module name/version. > >>>> 5.6 >>>> >> Sure, for the manifest of a .jam file. But the statement: >> >> "All packages defined in a module definition are inherently sealed, and >> this entry is ignored by the module system." >> >> is pretty broad, and would seem to indicate that it applies to the >> definitions produced by any module system. > > I will see if I can clarify it. > >>>> 6.2.3/6.3 >>>> >>>> For EE and other similar systems, it may be useful to have different >>>> VisibilityPolicy instances per Repository. We may want to have a >>>> getter/setter here, with the default implementation of get returning >>>> the >>>> default policy. >>> >>> Could you describe the use cases you had in mind for this? >> >> It creates a nice way to create wrapper repository instances that >> provide a customized view... >> >> EE systems are required to isolate applications from each other. And >> each may have very different external dependencies. If each repository >> instance can have its own VisibilityPolicy, then a wrapper Repository >> can be constructed for each application, using a different policy. >> >> Same goes for ImportOverridePolicy. > > Interesting. I will look into this further. > >>>> 6.2.6 >> >> I'm just considering the simple case of deploying a new module to a >> LocalRepository. I think it will be a bit surprising if some special act >> is required to make it available. For example, this would fail: >> >> localRepos.install(aModuleNamedSue); >> ModuleDefinition sue = localRepos.find("sue"); >> >> And it is pretty awkward to insert a call to reload() here. Given that >> the primary client of Repository is ModuleSystem, I guess the >> preparation logic can call reload() and re-try IFF a suitable definition >> cannot be found. > > I see your point. The good news is that the updated specification does > not require "reload" to be invoked in your example because a module > definition would be available for subsequence searches after it has been > installed, see Section 6.2.4 in the specification. I hope this would > address your concern. > >>>> 6.5.1 >> >> Yes, but LocalRepository is a specific implementation, and one that >> requires an expansion directory for each instance, right? We should make >> that clear. > > No. If two LocalRepository instances load module definitions from the > same source location, the implementation may share the same expansion > directory, or maybe not. I think the specification should leave this up > to the implementors, and I don't see good reason to dictate one > implementation approach over the other. Ah. You are thinking that two different JVM vendors might implement this differently. Fair enough. But this means that the actual location of the expansion directory must also be defined by the JVM. But LocalRepository instances will likely need to be created by IDEs, and other environments. And they will want to control the location of the expansion directory. So I am suggesting that LocalRepository needs an appropriate ctor to enable the caller to specify the location of the expansion directory. I think it makes much more sense to assume that LocalRepository (and URLRepostory) has a constant, well-defined model. JVM vendors are free to create variant classes, and to instantiate those by default. > >>>> 7.1.1/7.1.2 >>>> >> Thinking about this a bit more, the idea of a "default" module system is >> a bit odd. Certainly there will be one module system used by the JRE >> itself, and this is what you were thinking about here. And clearly >> LocalRepository and URLRepository are hard-wired to this same module >> system, so it makes some sense to be able to say in the javadoc for >> these classes that they use the "default" module (which needs to be >> added, btw). Do you have any other uses in mind? Perhaps we should be >> more explicit: >> >> public static ModuleSystem getJREModuleSystem(); >> >> Regardless, what I was really thinking about before is ModuleSystem >> initialization. >> >> ModuleDefinition has a getModuleSystem() method, but how is it >> implemented? Our model so far appears to assume that ModuleSystem >> instances will be shared, which is certainly reasonable, but... how? One >> simple model is that it is module system dependent, e.g. each module >> system implementation provides a singleton or equivalent. But this just >> pushes the problem up a layer, since the runtime type of >> ModuleDefinition will vary based on module system. > > As I mentioned previously, the APIs are still evolving significantly, > and they do not fully reflect what I have in mind. More specifically, > I've been planning to make ModuleSystem a service so its providers can > be discovered through the java.util.ServiceLoader API. (Now the service > providers strawman is available, it should help our discussion.) This > will make the static method above go away, and the repository > implementations can simply make use of the ServiceLoader API to discover > the specific ModuleSystem provider they need. Perfect. > >> I see that you added a ctor arg/getModuleSystem() method to Repository >> as well. In this model, the type problem is pushed up one more layer. It >> is the Repository implementation that must know the ModuleDefinition >> type, and therefore the ModuleSystem type. >> >> But we need to specify exactly how the initial repositories are >> configured/initialized, using the correct types, so that the JRE can >> instantiate them. (I took a cut at this in the class loading doc, if you >> recall.) This should be added to Appendix E--even if some of the details >> are JVM vendor specific, the requirements must at least be defined. > > I expect the repository configuration in the default implementation will > be documented as part of the JDK documents, similar to what has been > done to the security policy and other configuration files in the JDK: > > http://java.sun.com/javase/6/docs/technotes/guides/security/PolicyFiles.html > > > This way, different JRE implementations are allowed to innovate and we > don't have to put these implementation-specific details into the > specification. Ok, but the spec should at least say that. And there are some assumptions we are making about this mechanism, particularly that it will be possible to specify the runtime type of the repository classes. Shouldn't we say something about that as well? > >> But... didn't we decide long ago that a repository should not be >> restricted to a single module system? With a multi-repository search >> model, this is less of an issue, but why impose this limit? Composite >> implementations might have a hard time implementing getModuleSystem(). > > Right. Keep in mind that the APIs are still evolving. I expect this will > be changed when the interop proposal is in place. > >> Perhaps we should consider a model in which we: >> >> 1. Require ModuleSystem implementations to have a globally unique name. >> (There aren't likely to be hundreds of them, so this shouldn't be much >> of a hardship!) >> >> 2. Have a persistent configuration mechanism for ModuleSystems, like the >> one for the initial Repository instances. In addition to the runtime >> types for ModuleSystem subclasses, the instances themselves will likely >> need configuration, just like Repositories (logging, security, >> import/visibility policies, etc.) >> >> 3. During startup, the JRE initializes the registered ModuleSystems, >> then the registered Repositories. >> >> 4. Add to ModuleSystem: >> >> public abstract String getName(); >> >> public static ModuleSystem getModuleSystem(String name); >> public static List getModuleSystems(); >> >> Now repository implementations are not involved in *instantiating* >> ModuleSystems; they look them up by name. >> >> (and having a centralized lookup mechanism may enable other interesting >> behaviors) > > If we make ModuleSystem a service type, you will be able to perform this > kind of discovery easily using the ServiceLoader API, without requiring > a different set of API to be created for similar purpose. There is also > no need to have persistent configuration mechanism for ModuleSystem; if > a custom module system implementation is deployed as a service-provider > module into the system repository, you would be able to use the API to > look it up. Great. > >>>> 7.3.2.2 >>> >>>> We should also say something about the protocol for resource URLs. >> >> Shouldn't we? Like whether there will be a new protocol for them, or not. > > My current thinking is that the protocol in the resource URLs would be > entirely up to each repository implementation. Since there will be > custom repository implementations, we can't foresee what their > requirements are, so it would be difficult to define a standard protocol > that is suitable across all possible repository implementations. If you > think otherwise, let me know. I will clarify this in the next revision > of the specification. I have two concerns: 1. I have seen a number of systems that *assume* a specific protocol (e.g. "jar"). They do this because they want to parse the url, either to construct new urls or to glean information from them (such as the jar path). This practice clearly limits implementors. The API contract of getResource/s does not say anything about the protocol, which makes perfect sense. But I think it should be even stronger, and say that callers *must not* assume anything about the protocol. 2. The "jar" protocol as implemented by the JRE includes a jar cache. This cache is problematic for two reasons: a. It forces a second ZipFile to be opened (the one managed by the cache). The repository will already have one open, so using this protocol is very inefficient. b. The lifecycle of the jars in the cache cannot be controlled by the repository implementor. On Windows, files cannot be deleted if they are open, so undeploy/uninstall can fail. So either we should clean up the jar protocol to eliminate these issues, or Repository implementations should be encouraged *not* to use it. Which makes #1 even more important. > > - Stanley > From Stanley.Ho at sun.com Wed May 23 10:30:03 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Wed, 23 May 2007 10:30:03 -0700 Subject: Strawman: Services and service-providers support In-Reply-To: References: Message-ID: <46547A1B.4060004@sun.com> Hi Glyn, This strawman was developed based on the updated JSR 277 specification which has undergone significant changes since the EDR, so ideally the updated specification is required in order for the readers to have the proper context to fully understand the strawman; otherwise, this could lead to misunderstanding or confusion. That said, I have already received requests from a few observers since yesterday, and I think it may indeed be helpful to have the strawman available alone. However, it is unclear what the JCP rules are for making material like this available outside the EG, and I have already contacted the JCP PMO for clarification. - Stanley Glyn Normington wrote: > > Hi Stanley > > Don't you think we should make the strawman available to observers of > this list so they have the context to follow subsequent discussion of > the strawman and send in any insights they may have? > > Glyn From glyn_normington at UK.IBM.COM Thu May 24 01:22:23 2007 From: glyn_normington at UK.IBM.COM (Glyn Normington) Date: Thu, 24 May 2007 09:22:23 +0100 Subject: Strawman: Services and service-providers support In-Reply-To: <46547A1B.4060004@sun.com> Message-ID: Hi Stanley "Stanley M. Ho" wrote on 23/05/2007 18:30:03: > Hi Glyn, > > This strawman was developed based on the updated JSR 277 specification > which has undergone significant changes since the EDR, so ideally the > updated specification is required in order for the readers to have the > proper context to fully understand the strawman; otherwise, this could > lead to misunderstanding or confusion. Agreed. It would be good to make both available to the general public. > > That said, I have already received requests from a few observers since > yesterday, and I think it may indeed be helpful to have the strawman > available alone. However, it is unclear what the JCP rules are for > making material like this available outside the EG, and I have already > contacted the JCP PMO for clarification. Fair enough, but I think this should be ok since the JCP rules seem to encourage transparency. For example, section 2.1.1 of JCP 2.6 ([1]) says: "While Spec Leads are free to operate Expert Groups in whatever style they feel is most appropriate, they are encouraged to choose a style that provides maximal transparency to the Expert Group, community, the EC members and the public. The PMO provides Spec Leads with tools and techniques for making the actions of their Expert Groups transparent, and the EC members expect Spec Leads to carefully choose which tools are best for their Expert Groups and commit to using them. Transparency is valuable to everyone in the community, especially the Expert Group, because it offers broader feedback to the group and helps build broader support for the final spec." So the JCP is, at least in spirit, fully supportive of the kind of transparency needed by JSR 277. It's just a shame we can't simply expose the Expert Group private page to the public, although this is something the JCP should really consider supporting soon. Section 2.3.1 of [1] says: "If the Expert Group makes major revisions to the draft during Early Draft Review, the Spec Lead should send the revised draft, along with a synopsis of the changes, to the PMO. The PMO will notify Members of any updated drafts and change synopses received and make them available for download by Members and the public." This allows for multiple early draft specifications. I guess the JCP's main concern here is appropriate notification to JCP Members that a draft has been issued. This is sensible as not everyone who is interested will have the time to follow the mailing list. I doubt that it would be appropriate to notify JCP Members of interim strawman documents though. Then there's the precedent of the JSR 294 strawman appearing in a blog even before the EG was formed. The JCP didn't object to that, AFAIK. Also, I forwarded at least one of the JSR 291 drafts to the publicly viewable mailing list and again the JCP didn't object. Anyway, let's hope the JCP clarify this quickly. :-) > > - Stanley > > > Glyn Normington wrote: > > > > Hi Stanley > > > > Don't you think we should make the strawman available to observers of > > this list so they have the context to follow subsequent discussion of > > the strawman and send in any insights they may have? > > > > Glyn Glyn [1] http://jcp.org/en/procedures/jcp2 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/attachments/20070524/ab660611/attachment.html From glyn_normington at UK.IBM.COM Thu May 24 02:34:26 2007 From: glyn_normington at UK.IBM.COM (Glyn Normington) Date: Thu, 24 May 2007 10:34:26 +0100 Subject: Query... In-Reply-To: <46548491.7040408@oracle.com> Message-ID: If Query is made Serializable, let's make sure it declares a serial version UID to enable serial compatibility of future versions. Glyn Bryan Atsatt wrote on 23/05/2007 19:14:41: > When I designed the Query/Repository model in the prototype, I had it > firmly in my mind that Query should be Serializable, so that it could be > sent over the wire to a remote repository. I've described it this way to > a number of people. But apparently I forgot to actually declare it. :^) > > And we haven't done so in the spec, either. I think we should. > > // Bryan Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/attachments/20070524/48d39c95/attachment.html From glyn_normington at UK.IBM.COM Thu May 24 03:00:32 2007 From: glyn_normington at UK.IBM.COM (Glyn Normington) Date: Thu, 24 May 2007 11:00:32 +0100 Subject: Relationship to JSR 291 [was: Re: Bryan's comments] In-Reply-To: <46549B8F.3090403@oracle.com> Message-ID: Bryan Atsatt wrote on 23/05/2007 20:52:47: > Stanley M. Ho wrote: > >> I think we need to think hard about this issue. The OSGi model of import > >> by *package name* decouples the importer from any explicit binding to a > >> bundle/module name. Refactoring under that model is *much* cleaner, and > >> far more natural. As is the usage model. After all, Foo.java import > >> statements contain package/class names, *not* module names. Programmers > >> think in terms of classes and packages. > >> > >> Peter makes this point pretty strongly, and I have to say I agree > >> wholeheartedly: > >> > >> http://www.aqute.biz/Blog/2006-04-29 > > > > I agreed that in some situations it is much better to have dependency > > that is loosely coupled. You may want to check out the service-providers > > strawman that I just sent out, and it deals with the exact issue around > > API vs implementation. > > That's great, but sort of beside the point :^). > > I am specifically referring to the syntax and semantics of "import" > declarations. At the moment, 277 supports only the "Reguire Bundle" > style semantics defined by OSGi. This model is is inherently > tightly-coupled, and its use is greatly discouraged in the OSGi world. > It was not even present in the initial releases. > > I strongly believe that it would be a huge mistake for 277 to support > only this model. If we want to simplify things and support only one > model, then we should choose import by package name/version, *not* > import by module name/version. Of course "import package" is superior to "require bundle" in many situations, but I think it would be a vast waste of time for JSR 277 to play "catch-up" with JSR 291. A better approach would be for the Java 7 platform to provide first class support for JSR 291. This boils down to standardising the experimental class loader deadlock fix ([1]) and enabling JSR 291 to exploit JSR 277's repositories and JSR 294's superpackages. The features in JSR 277 which are already provided by JSR 291 could be ditched (!) or could be retained for users wanting a module system "out of the box" or for exploitation by the JRE itself as a properly architected sequel to the consumer JRE. But if we retain these features, it is essential we provide first-class interoperation with JSR 291 as we have discussed many times in the past but which still looks pretty challenging (see [2]). Glyn [1] http://underlap.blogspot.com/2006/11/experimental-fix-for-sunbug-4670071.html [2] http://underlap.blogspot.com/2007/05/designing-module-system-interoperation.html Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/attachments/20070524/33d240bc/attachment.html From bryan.atsatt at oracle.com Thu May 24 11:45:55 2007 From: bryan.atsatt at oracle.com (Bryan Atsatt) Date: Thu, 24 May 2007 11:45:55 -0700 Subject: Relationship to JSR 291 [was: Re: Bryan's comments] In-Reply-To: References: Message-ID: <4655DD63.7020900@oracle.com> Let me clarify. It is my assumption that 277 will provide both an API/framework *and* a specific ModuleSystem implementation. And that OSGi will be able to supply another ModuleSystem implementation: 277 API/Framework / \ / \ 277ModuleSystem OSGiModuleSystem Therefore, the API's must *support* "import package", or it will not be usable by the OSGiModuleSystem. I agree that the 277ModuleSystem implementation does not need to support "import package" semantics. Which means that the 277 APIs must support *both* semantics. Otherwise, OSGi will not be able to operate as a first-class citizen in the 277 world. And, if 294 adds support for "import", it should at least do the natural thing of "import package". And it should also support "import module", but supporting both at the language level may be awkward. // Bryan Glyn Normington wrote: > > *Bryan Atsatt * wrote on 23/05/2007 20:52:47: > > > Stanley M. Ho wrote: > > >> I think we need to think hard about this issue. The OSGi model of > import > > >> by *package name* decouples the importer from any explicit binding > to a > > >> bundle/module name. Refactoring under that model is *much* > cleaner, and > > >> far more natural. As is the usage model. After all, Foo.java import > > >> statements contain package/class names, *not* module names. > Programmers > > >> think in terms of classes and packages. > > >> > > >> Peter makes this point pretty strongly, and I have to say I agree > > >> wholeheartedly: > > >> > > >> http://www.aqute.biz/Blog/2006-04-29 > > > > > > I agreed that in some situations it is much better to have dependency > > > that is loosely coupled. You may want to check out the > service-providers > > > strawman that I just sent out, and it deals with the exact issue around > > > API vs implementation. > > > > That's great, but sort of beside the point :^). > > > > I am specifically referring to the syntax and semantics of "import" > > declarations. At the moment, 277 supports only the "Reguire Bundle" > > style semantics defined by OSGi. This model is is inherently > > tightly-coupled, and its use is greatly discouraged in the OSGi world. > > It was not even present in the initial releases. > > > > I strongly believe that it would be a huge mistake for 277 to support > > only this model. If we want to simplify things and support only one > > model, then we should choose import by package name/version, *not* > > import by module name/version. > > Of course "import package" is superior to "require bundle" in many > situations, but I think it would be a vast waste of time for JSR 277 to > play "catch-up" with JSR 291. > > A better approach would be for the Java 7 platform to provide first > class support for JSR 291. This boils down to standardising the > experimental class loader deadlock fix ([1]) and enabling JSR 291 to > exploit JSR 277's repositories and JSR 294's superpackages. > > The features in JSR 277 which are already provided by JSR 291 could be > ditched (!) or could be retained for users wanting a module system "out > of the box" or for exploitation by the JRE itself as a properly > architected sequel to the consumer JRE. But if we retain these features, > it is essential we provide first-class interoperation with JSR 291 as we > have discussed many times in the past but which still looks pretty > challenging (see [2]). > > Glyn > [1] > http://underlap.blogspot.com/2006/11/experimental-fix-for-sunbug-4670071.html > > [2] > http://underlap.blogspot.com/2007/05/designing-module-system-interoperation.html > > > ------------------------------------------------------------------------ > > / > / > > /Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU/ > > > > > > From Stanley.Ho at sun.com Thu May 24 16:54:40 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Thu, 24 May 2007 16:54:40 -0700 Subject: Relationship to JSR 291 [was: Re: Bryan's comments] In-Reply-To: References: Message-ID: <465625C0.1050107@sun.com> Hi Glyn, Glyn Normington wrote: > ... > A better approach would be for the Java 7 platform to provide first > class support for JSR 291. This boils down to standardising the > experimental class loader deadlock fix ([1]) and enabling JSR 291 to > exploit JSR 277's repositories and JSR 294's superpackages. > > The features in JSR 277 which are already provided by JSR 291 could be > ditched (!) or could be retained for users wanting a module system "out > of the box" or for exploitation by the JRE itself as a properly > architected sequel to the consumer JRE. But if we retain these features, > it is essential we provide first-class interoperation with JSR 291 as we > have discussed many times in the past but which still looks pretty > challenging (see [2]). > > Glyn As we discussed before, the charter of JSR 277 is to develop a solution for the next generation Java SE platform to address the problems as described in the original JSR submission. What we want to provide in JSR 277 (and this EG also agreed) is first-class modularity, packaging, and deployment support in the SE platform, with integration with language/VM (through JSR 294), classes libraries and tools. This is the problem we are solving, and we must provide all the features required to solve this problem. Whether a feature already exists in other module systems is orthogonal to whether it should be supported in JSR 277. That said, we still want to learn from the experience of these existing systems as much as possible, so we can design JSR 277 in the best possible way. In terms of the relationship between JSR 277 and JSR 291, I think we all agreed in the EG that JSR 277 will provide the necessary framework for interoperation with other module systems, especially with JSR 291/OSGi. Following up the original email about import package, I don't think we should support it in JSR 277 for a different reason, and I will reply in another email. - Stanley From Stanley.Ho at sun.com Thu May 24 17:12:26 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Thu, 24 May 2007 17:12:26 -0700 Subject: Exported resources In-Reply-To: <46548736.7020507@oracle.com> References: <4654109D.40205@sun.com> <46548736.7020507@oracle.com> Message-ID: <465629EA.2050008@sun.com> Hi Bryan, Bryan Atsatt wrote: > 1. Definitely agree that resource search order should be identical to > class search order. Glad to hear! > 2. Using permissions to limit access to private resources seems like > overkill to me. The prototype implemented this in a very simple fashion: > > a. If resource is exported, return it, else > a. Get the caller's Module (get class from stack, get module from it) > b. If callerModule == this, return resource, else return null. The issue is that this approach still requires stack walking and there is no public API in the SE platform that let you implement this. If stack walking is required for the check anyway, I think the security permission approach is better that it is implementable with the existing API in the SE platform. - Stanley From Stanley.Ho at sun.com Thu May 24 17:51:04 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Thu, 24 May 2007 17:51:04 -0700 Subject: Relationship to JSR 291 [was: Re: Bryan's comments] In-Reply-To: <4655DD63.7020900@oracle.com> References: <4655DD63.7020900@oracle.com> Message-ID: <465632F8.3080900@sun.com> Hi Bryan, Bryan Atsatt wrote: > Let me clarify. It is my assumption that 277 will provide both an > API/framework *and* a specific ModuleSystem implementation. And that > OSGi will be able to supply another ModuleSystem implementation: > > 277 API/Framework > / \ > / \ > 277ModuleSystem OSGiModuleSystem Right. There are also other module systems that can be plugged into this framework, e.g. NetBeans. > Therefore, the API's must *support* "import package", or it will not be > usable by the OSGiModuleSystem. > > I agree that the 277ModuleSystem implementation does not need to support > "import package" semantics. Which means that the 277 APIs must support > *both* semantics. > > Otherwise, OSGi will not be able to operate as a first-class citizen in > the 277 world. > > And, if 294 adds support for "import", it should at least do the natural > thing of "import package". And it should also support "import module", > but supporting both at the language level may be awkward. > > // Bryan I think we all agreed that the module system defined by JSR 277 does not need to support import package. As far as I am aware (correct me if I'm wrong), all the existing systems support import at the module level, and OSGi is the only one that supports import at the package level. Thus, import by package should be a feature specific to the OSGi specific module system implementation (e.g. OSGiModuleDefinition), and I don't think it is necessary to push it into the base classes in JSR 277. This doesn't mean supporting the notion of import by package is impossible. What OSGi will expose from its repository is simply module definitions, and module definition is basically an abstraction. Therefore, it should be possible to treat an OSGi bundle as a module definition, as well as to treat each exported package in an OSGi bundle a module definition as well. The dependency supported in JSR 277 is import-by-module. In the JSR 277 module system, this is mapped to import-by-superpackage. In the OSGi module system, this can be mapped to import-by-OSGi-bundle or import-by-package. If other module systems want to support different import granularity or if OSGi evolves to add new import granularity in the future, this model should still work. Honestly, interoperability between multiple module systems with "import module" alone is already a hard enough problem for us to solve. I am concerned that we're making this problem towards insolvable if we keep adding complexity. ;) - Stanley From Stanley.Ho at sun.com Thu May 24 18:16:50 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Thu, 24 May 2007 18:16:50 -0700 Subject: Bryan's comments In-Reply-To: <46549B8F.3090403@oracle.com> References: <463A28ED.8080302@sun.com> <463A6323.1050307@oracle.com> <463F7C11.8090601@sun.com> <46524588.6070203@oracle.com> <46539DC7.7090300@sun.com> <46549B8F.3090403@oracle.com> Message-ID: <46563902.7060906@sun.com> Hi Bryan, Bryan Atsatt wrote: > Stanley M. Ho wrote: >> >> Based on the updated specification, the module system will provide a >> default version for the module definition which has no version declared >> in the metadata, so each module definition will always have a version. >> Is there any strong reason why we should reject all the module >> definitions which have no version declared in the metadata rather than >> giving them a default version number? > > It is more philosophical than technical: if developers are forced to > declare it, they must be conscious of the implications of that choice. I think that the barrier for developers to create a simple module should be as low as possible. Making the version mandatory will make this barrier higher. Also, many existing systems consider version to be optional in their modules, probably for the same reason. Do you think you can live with this if we continue to make version optional? >>>>> Refactoring in this way results in a perhaps unexpected runtime/memory >>>>> overhead. A pure wrapper module that re-exports must have its own >>>>> class >>>>> loader, and, even though it won't define any classes, the VM *cache* >>>>> will be updated as that loader is used. >> >> If there is no support for re-exporting subset of imported >> classes/resources, this becomes a non-issue. > > Sorry, but this doesn't follow. The VM class cache is updated (at least) > when: > > loader.defineClass() is called. > VM calls loader.internalLoadClass() > Class.forName() is called. > > The cache records that the initiating loader returned a specific class, > regardless of what loader defined it. This enables > loader.findLoadedClass() to work as expected. > > Given that: > > a. Module has-a single ClassLoader, and > b. A specific Module re-exports classes from an imported Module. > > we have two scenarios to consider: > > 1. Module has both member and re-exported classes. The loader will both > define member classes and the VM cache will be updated with mappings > from Module loader to imported-module-Class instances. > > 2. Module has no member classes, only re-exported classes. The cache > will be updated with mappings from Module loader to > imported-module-Class instances. > > In the latter case, if there is only a single imported module, an > optimization could be that this view module returns the imported > module's loader rather than having it's own. > > But this is not possible when there are imports from more than one > module: a unique loader instance must be created to satisfy the api > contract module.getClassLoader(). Even though it will never define any > classes, the VM will still update the cache. > > This latter case is what I am referring to as overhead. Of course, with > appropriate updates to the VM/caching behavior, this overhead could be > eliminated. But this would have to be addressed or the overhead *will* > exist. Thanks for the explanation; I now understand your concern. One of our strengths in this JSR is the ability to require deep changes in the platform if necessary. If this particular classloading issue has significant overhead, it should be addressed. I have forwarded your concern to the classloading team for them to evaluate. If you can provide any performance data related to this overhead to our classloading team, it will be great. > Ah. You are thinking that two different JVM vendors might implement this > differently. Fair enough. But this means that the actual location of the > expansion directory must also be defined by the JVM. This is not limited to two different JVM vendors - even the same JVM vendor can choose to use the same or different expansion directory for two instances of local repository in the same session. > But LocalRepository instances will likely need to be created by IDEs, > and other environments. And they will want to control the location of > the expansion directory. So I am suggesting that LocalRepository needs > an appropriate ctor to enable the caller to specify the location of the > expansion directory. > > I think it makes much more sense to assume that LocalRepository (and > URLRepostory) has a constant, well-defined model. JVM vendors are free > to create variant classes, and to instantiate those by default. It sounds like what you really want is a way to hint the local repository where it should expand its content if necessary. This seems reasonable to me, and I will incorporate it into the next revision of the specification. > Ok, but the spec should at least say that. And there are some > assumptions we are making about this mechanism, particularly that it > will be possible to specify the runtime type of the repository classes. > Shouldn't we say something about that as well? No. If this is something that implementors are free to implement, it does not need to be in the spec. I think describing it in the JDK documentation would be sufficient for this purpose. >>>>> 7.3.2.2 > ... > I have two concerns: > > 1. I have seen a number of systems that *assume* a specific protocol > (e.g. "jar"). They do this because they want to parse the url, either to > construct new urls or to glean information from them (such as the jar > path). This practice clearly limits implementors. The API contract of > getResource/s does not say anything about the protocol, which makes > perfect sense. But I think it should be even stronger, and say that > callers *must not* assume anything about the protocol. > > 2. The "jar" protocol as implemented by the JRE includes a jar cache. > This cache is problematic for two reasons: > > a. It forces a second ZipFile to be opened (the one managed by the > cache). The repository will already have one open, so using this > protocol is very inefficient. > > b. The lifecycle of the jars in the cache cannot be controlled by the > repository implementor. On Windows, files cannot be deleted if they are > open, so undeploy/uninstall can fail. > > So either we should clean up the jar protocol to eliminate these issues, > or Repository implementations should be encouraged *not* to use it. > Which makes #1 even more important. Okay, I think we are now on the same page that it's not feasible to define a common protocol for this in JSR 277. I see your point of #1, and I will mention this "no protocol assumption" issue in the next revision of the specification. #2 is a JDK implementation issue, and I will forward this issue to the appropriate team in Java SE for evaluation. - Stanley From Stanley.Ho at sun.com Thu May 24 18:28:38 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Thu, 24 May 2007 18:28:38 -0700 Subject: Email problem In-Reply-To: <464E283C.4050207@sun.com> References: <464E283C.4050207@sun.com> Message-ID: <46563BC6.70402@sun.com> Hi, This is just a head-up. There has been some massive networking changes going on since early this week and are expected to be finished over the Memorial Day (next Monday). It is unclear if these changes will impact this EG mailing list and the observer mailing list. If you experience any problem, please let me know. - Stanley Stanley M. Ho wrote: > Hi, > > Apparently, my email problem has just been fixed, but you will now > receive all the emails that I attempted to send/re-send in the last few > days - please ignore them. > > Sorry for any confusion. > > - Stanley From bryan.atsatt at oracle.com Thu May 24 20:32:30 2007 From: bryan.atsatt at oracle.com (Bryan Atsatt) Date: Thu, 24 May 2007 20:32:30 -0700 Subject: Exported resources In-Reply-To: <465629EA.2050008@sun.com> References: <4654109D.40205@sun.com> <46548736.7020507@oracle.com> <465629EA.2050008@sun.com> Message-ID: <465658CE.5020409@oracle.com> Both solutions require stack walking (unless there is some new implementation of the java security model I've not yet seen!). The permission check does much more work than is necessary here. Take a look at AccessController.checkPermission() to see what I mean. And actually there is a very simple API to get the stack, which I've used for years: private static class StackAccessor extends SecurityManager { public Class[] getStack() { return getClassContext(); } } private static final STACK_ACCESSOR = new StackAccessor(); Now the enclosing class can simply call STACK_ACCESSOR.getStack(). // Bryan Stanley M. Ho wrote: > Hi Bryan, > > Bryan Atsatt wrote: >> 1. Definitely agree that resource search order should be identical to >> class search order. > > Glad to hear! > >> 2. Using permissions to limit access to private resources seems like >> overkill to me. The prototype implemented this in a very simple fashion: >> >> a. If resource is exported, return it, else >> a. Get the caller's Module (get class from stack, get module from it) >> b. If callerModule == this, return resource, else return null. > > The issue is that this approach still requires stack walking and there > is no public API in the SE platform that let you implement this. > > If stack walking is required for the check anyway, I think the security > permission approach is better that it is implementable with the existing > API in the SE platform. > > - Stanley > From abrock at REDHAT.COM Fri May 25 04:44:54 2007 From: abrock at REDHAT.COM (Adrian) Date: Fri, 25 May 2007 13:44:54 +0200 Subject: ClassLoader Deadlock fix was: Re: Relationship to JSR 291 [was: Re: Bryan's comments] In-Reply-To: References: Message-ID: <1180093494.3531.61.camel@warjort> On Thu, 2007-05-24 at 11:00 +0100, Glyn Normington wrote: > A better approach would be for the Java 7 platform to provide first > class support for JSR 291. This boils down to standardising the > experimental class loader deadlock fix ([1]) Fixing the deadlock just moves the problem. You'll still get ClassCircularityErrors when competing threads try to load classes using locks other than the classloader synchronization or they don't synchronize on the loadClass() or they release the lock during the classloading request to let others have a go (again to avoid the deadlock). This is because of the way the dictionary class determines whether a circular load is occuring. Although I haven't tried it with OpenJDK so maybe the dictionary class contains some other fixes to workaround the problem? The simple form of the problem: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4699981 has been fixed in recent JDKs, but spurious CCEs still exist in other cases. e.g. the testAbstractFactoryConcurrent() here: http://viewvc.jboss.org/cgi-bin/viewvc.cgi/jbossas/projects/microcontainer/trunk/classloader/src/tests/org/jboss/test/classloader/delegate/test/DelegateUnitTestCase.java?revision=62792&view=markup will show CCEs in the log if you enable TRACE logging. 1445 TRACE [ClassLoaderManager] Run failed with exception java.lang.ClassCircularityError: org/jboss/test/classloader/delegate/support/b/TestFactoryImplementation at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:620) -- xxxxxxxxxxxxxxxxxxxxxxxxxxxx Adrian Brock Chief Scientist JBoss, a division of Red Hat xxxxxxxxxxxxxxxxxxxxxxxxxxxx From heavy at UNGOVERNED.ORG Fri May 25 08:01:03 2007 From: heavy at UNGOVERNED.ORG (Richard S. Hall) Date: Fri, 25 May 2007 11:01:03 -0400 Subject: Relationship to JSR 291 [was: Re: Bryan's comments] In-Reply-To: <465632F8.3080900@sun.com> References: <4655DD63.7020900@oracle.com> <465632F8.3080900@sun.com> Message-ID: <4656FA2F.7080506@ungoverned.org> Stanley M. Ho wrote: > Hi Bryan, > > Bryan Atsatt wrote: >> Let me clarify. It is my assumption that 277 will provide both an >> API/framework *and* a specific ModuleSystem implementation. And that >> OSGi will be able to supply another ModuleSystem implementation: >> >> 277 API/Framework >> / \ >> / \ >> 277ModuleSystem OSGiModuleSystem > > Right. There are also other module systems that can be plugged into this > framework, e.g. NetBeans. > >> Therefore, the API's must *support* "import package", or it will not be >> usable by the OSGiModuleSystem. >> >> I agree that the 277ModuleSystem implementation does not need to support >> "import package" semantics. Which means that the 277 APIs must support >> *both* semantics. >> >> Otherwise, OSGi will not be able to operate as a first-class citizen in >> the 277 world. >> >> And, if 294 adds support for "import", it should at least do the natural >> thing of "import package". And it should also support "import module", >> but supporting both at the language level may be awkward. >> >> // Bryan > > I think we all agreed that the module system defined by JSR 277 does not > need to support import package. As far as I am aware (correct me if I'm > wrong), all the existing systems support import at the module level, and > OSGi is the only one that supports import at the package level. Thus, > import by package should be a feature specific to the OSGi specific > module system implementation (e.g. OSGiModuleDefinition), and I don't > think it is necessary to push it into the base classes in JSR 277. Well, I must at least say that this is flawed logic at best. The fact that only the OSGi framework supports importing packages does not say anything about whether or not it should be supported by 277 or not. There any many valid arguments why the import package model is better due to the fact the is declares "what" is needed, not "who" contains what is needed. Using "who" instead of "what" generally leads to terrible dependency fanout due to coarse-grained module definitions and brittleness. I am fairly certain that I could conjure up some people who could tell you stories about the horrors of the brittleness of the module dependency approach when trying to re-use modules for different purposes. However, I guess this is beside the point... I think the real argument that should be made for not necessarily including import package support is along the lines of what I was saying in my other message, which is that we should keep 277 as simple as possible. > This doesn't mean supporting the notion of import by package is > impossible. What OSGi will expose from its repository is simply module > definitions, and module definition is basically an abstraction. > Therefore, it should be possible to treat an OSGi bundle as a module > definition, as well as to treat each exported package in an OSGi bundle > a module definition as well. The dependency supported in JSR 277 is > import-by-module. In the JSR 277 module system, this is mapped to > import-by-superpackage. In the OSGi module system, this can be mapped to > import-by-OSGi-bundle or import-by-package. If other module systems want > to support different import granularity or if OSGi evolves to add new > import granularity in the future, this model should still work. > > Honestly, interoperability between multiple module systems with "import > module" alone is already a hard enough problem for us to solve. I am > concerned that we're making this problem towards insolvable if we keep > adding complexity. ;) Agreed, but I think part of the reason that there is an urge to push such features into the core is because there is already feature creep in the core that gives people the impression that we are redefining OSGi modularity, NetBeans modularity, etc. Thus, everyone wants to make sure their pet feature is added. If 277 could make a stand and declare that it is at a layer beneath all of that (and live up to that declaration), then there would likely be less push down. -> richard From bryan.atsatt at oracle.com Fri May 25 08:35:55 2007 From: bryan.atsatt at oracle.com (Bryan Atsatt) Date: Fri, 25 May 2007 08:35:55 -0700 Subject: Relationship to JSR 291 [was: Re: Bryan's comments] In-Reply-To: <4656FA2F.7080506@ungoverned.org> References: <4655DD63.7020900@oracle.com> <465632F8.3080900@sun.com> <4656FA2F.7080506@ungoverned.org> Message-ID: <4657025B.4000506@oracle.com> Let me try to focus this discussion. What I am aiming for here is conceptually simple: Don't force a specific interpretation of the import String in the 277 *API*. So, for example, change: String ImportDependency.getModuleName() to String ImportDependency.getImportName(); Each ModuleSystem is then free to interpret that name as appropriate. Sure, this leaves room for *mis*interpretation. So let's figure out a way to avoid that (e.g. define a syntax such that the string is self-describing). (I'm off for till Tuesday, so I'll tune back in then :^) // Bryan Richard S. Hall wrote: > Stanley M. Ho wrote: >> Hi Bryan, >> >> Bryan Atsatt wrote: >>> Let me clarify. It is my assumption that 277 will provide both an >>> API/framework *and* a specific ModuleSystem implementation. And that >>> OSGi will be able to supply another ModuleSystem implementation: >>> >>> 277 API/Framework >>> / \ >>> / \ >>> 277ModuleSystem OSGiModuleSystem >> >> Right. There are also other module systems that can be plugged into this >> framework, e.g. NetBeans. >> >>> Therefore, the API's must *support* "import package", or it will not be >>> usable by the OSGiModuleSystem. >>> >>> I agree that the 277ModuleSystem implementation does not need to support >>> "import package" semantics. Which means that the 277 APIs must support >>> *both* semantics. >>> >>> Otherwise, OSGi will not be able to operate as a first-class citizen in >>> the 277 world. >>> >>> And, if 294 adds support for "import", it should at least do the natural >>> thing of "import package". And it should also support "import module", >>> but supporting both at the language level may be awkward. >>> >>> // Bryan >> >> I think we all agreed that the module system defined by JSR 277 does not >> need to support import package. As far as I am aware (correct me if I'm >> wrong), all the existing systems support import at the module level, and >> OSGi is the only one that supports import at the package level. Thus, >> import by package should be a feature specific to the OSGi specific >> module system implementation (e.g. OSGiModuleDefinition), and I don't >> think it is necessary to push it into the base classes in JSR 277. > > Well, I must at least say that this is flawed logic at best. The fact > that only the OSGi framework supports importing packages does not say > anything about whether or not it should be supported by 277 or not. > There any many valid arguments why the import package model is better > due to the fact the is declares "what" is needed, not "who" contains > what is needed. > > Using "who" instead of "what" generally leads to terrible dependency > fanout due to coarse-grained module definitions and brittleness. I am > fairly certain that I could conjure up some people who could tell you > stories about the horrors of the brittleness of the module dependency > approach when trying to re-use modules for different purposes. However, > I guess this is beside the point... > > I think the real argument that should be made for not necessarily > including import package support is along the lines of what I was saying > in my other message, which is that we should keep 277 as simple as > possible. > >> This doesn't mean supporting the notion of import by package is >> impossible. What OSGi will expose from its repository is simply module >> definitions, and module definition is basically an abstraction. >> Therefore, it should be possible to treat an OSGi bundle as a module >> definition, as well as to treat each exported package in an OSGi bundle >> a module definition as well. The dependency supported in JSR 277 is >> import-by-module. In the JSR 277 module system, this is mapped to >> import-by-superpackage. In the OSGi module system, this can be mapped to >> import-by-OSGi-bundle or import-by-package. If other module systems want >> to support different import granularity or if OSGi evolves to add new >> import granularity in the future, this model should still work. >> >> Honestly, interoperability between multiple module systems with "import >> module" alone is already a hard enough problem for us to solve. I am >> concerned that we're making this problem towards insolvable if we keep >> adding complexity. ;) > > Agreed, but I think part of the reason that there is an urge to push > such features into the core is because there is already feature creep in > the core that gives people the impression that we are redefining OSGi > modularity, NetBeans modularity, etc. Thus, everyone wants to make sure > their pet feature is added. If 277 could make a stand and declare that > it is at a layer beneath all of that (and live up to that declaration), > then there would likely be less push down. > > -> richard > From glyn_normington at UK.IBM.COM Fri May 25 09:06:08 2007 From: glyn_normington at UK.IBM.COM (Glyn Normington) Date: Fri, 25 May 2007 17:06:08 +0100 Subject: Relationship to JSR 291 [was: Re: Bryan's comments] In-Reply-To: <80099cb10705250825p89f33b2i9d9c6b40dc3420c9@mail.gmail.com> Message-ID: I haven't lost interest in this thread but I'm going on leave now returning on Wednesday, so please don't misinterpret my silence in the interim. :-) Glyn Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/attachments/20070525/60747c24/attachment.html From Stanley.Ho at sun.com Fri May 25 15:19:32 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Fri, 25 May 2007 15:19:32 -0700 Subject: Relationship to JSR 291 [was: Re: Bryan's comments] In-Reply-To: <4656FA2F.7080506@ungoverned.org> References: <4655DD63.7020900@oracle.com> <465632F8.3080900@sun.com> <4656FA2F.7080506@ungoverned.org> Message-ID: <465760F4.5070306@sun.com> Hi Richard, Richard S. Hall wrote: > Stanley M. Ho wrote: >> >> I think we all agreed that the module system defined by JSR 277 does not >> need to support import package. As far as I am aware (correct me if I'm >> wrong), all the existing systems support import at the module level, and >> OSGi is the only one that supports import at the package level. Thus, >> import by package should be a feature specific to the OSGi specific >> module system implementation (e.g. OSGiModuleDefinition), and I don't >> think it is necessary to push it into the base classes in JSR 277. > > Well, I must at least say that this is flawed logic at best. The fact > that only the OSGi framework supports importing packages does not say > anything about whether or not it should be supported by 277 or not. Right. Let me reiterate what I really meant. I think whether a feature is supported by JSR 277 should not be determined primarily by whether other module systems have already supported it. Rather, it should be determined by whether the feature is required to address the problems we are set out to solve (see my other email, I think we're on the same page here). In case if other module systems already support the feature, this is still great so we can learn from their experience to make sure we do the right thing. > There any many valid arguments why the import package model is better > due to the fact the is declares "what" is needed, not "who" contains > what is needed. > > Using "who" instead of "what" generally leads to terrible dependency > fanout due to coarse-grained module definitions and brittleness. I am > fairly certain that I could conjure up some people who could tell you > stories about the horrors of the brittleness of the module dependency > approach when trying to re-use modules for different purposes. However, > I guess this is beside the point... > > I think the real argument that should be made for not necessarily > including import package support is along the lines of what I was saying > in my other message, which is that we should keep 277 as simple as > possible. I have no doubt that the import package model has its own strength over other models. My own argument against adding the import package support is that the "import module" model is very simple to understand and is already sufficient for the set of problems we want to address. I don't think the added values of the import package model on top of what the import module model already provides would justify the technical complexity (along your lines) and developers' confusion (two related-but-yet-different import models to choose). Anyway, it seems the EG consensus so far is to not add import package support. If any EG member disagrees, please speak up. >> This doesn't mean supporting the notion of import by package is >> impossible. What OSGi will expose from its repository is simply module >> definitions, and module definition is basically an abstraction. >> Therefore, it should be possible to treat an OSGi bundle as a module >> definition, as well as to treat each exported package in an OSGi bundle >> a module definition as well. The dependency supported in JSR 277 is >> import-by-module. In the JSR 277 module system, this is mapped to >> import-by-superpackage. In the OSGi module system, this can be mapped to >> import-by-OSGi-bundle or import-by-package. If other module systems want >> to support different import granularity or if OSGi evolves to add new >> import granularity in the future, this model should still work. >> >> Honestly, interoperability between multiple module systems with "import >> module" alone is already a hard enough problem for us to solve. I am >> concerned that we're making this problem towards insolvable if we keep >> adding complexity. ;) > > Agreed, but I think part of the reason that there is an urge to push > such features into the core is because there is already feature creep in > the core that gives people the impression that we are redefining OSGi > modularity, NetBeans modularity, etc. Thus, everyone wants to make sure > their pet feature is added. If 277 could make a stand and declare that > it is at a layer beneath all of that (and live up to that declaration), > then there would likely be less push down. > > -> richard Richard, It is unclear to me what you meant by feature creep in the core because everything in the specification is essentially required to address the problems this JSR is set out to solve. We can argue that some features should not be in the base layer API but in the module system defined by JSR 277 instead or vice versa, but they still belong to the overall JSR 277 specification. To recap what we have been discussing, I think our general consensus is to define a module system in JSR 277 that would address all the problems this JSR is set out to solve, as well as to provide a high level framework for different module systems (including the module system defined by JSR 277) to layer on top for interoperation. We only described the former in the EDR, but the latter should become more evident in the next revision of the JSR 277 specification. - Stanley From Stanley.Ho at sun.com Fri May 25 15:29:20 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Fri, 25 May 2007 15:29:20 -0700 Subject: [Fwd: Re: Relationship to JSR 291 [was: Re: Bryan's comments]] Message-ID: <46576340.7030408@sun.com> This is a repost to sync up the EG observer mailing list. - Stanley -------- Original Message -------- Subject: Re: Relationship to JSR 291 [was: Re: Bryan's comments] Date: Fri, 25 May 2007 11:00:16 -0400 From: Richard S. Hall Reply-To: Java Community Process JSR #277 Expert List To: JSR-277-EG at JCP.ORG References: <465625C0.1050107 at sun.com> Stanley M. Ho wrote: > Hi Glyn, > > Glyn Normington wrote: >> ... >> A better approach would be for the Java 7 platform to provide first >> class support for JSR 291. This boils down to standardising the >> experimental class loader deadlock fix ([1]) and enabling JSR 291 to >> exploit JSR 277's repositories and JSR 294's superpackages. >> >> The features in JSR 277 which are already provided by JSR 291 could be >> ditched (!) or could be retained for users wanting a module system "out >> of the box" or for exploitation by the JRE itself as a properly >> architected sequel to the consumer JRE. But if we retain these features, >> it is essential we provide first-class interoperation with JSR 291 as we >> have discussed many times in the past but which still looks pretty >> challenging (see [2]). >> >> Glyn > > As we discussed before, the charter of JSR 277 is to develop a solution > for the next generation Java SE platform to address the problems as > described in the original JSR submission. What we want to provide in JSR > 277 (and this EG also agreed) is first-class modularity, packaging, and > deployment support in the SE platform, with integration with language/VM > (through JSR 294), classes libraries and tools. This is the problem we > are solving, and we must provide all the features required to solve this > problem. Whether a feature already exists in other module systems is > orthogonal to whether it should be supported in JSR 277. That said, we > still want to learn from the experience of these existing systems as > much as possible, so we can design JSR 277 in the best possible way. Are you really saying that 277 is mandated to create everything from scratch? If so, I wasn't aware that this was mandated and it seems somewhat odd to me, especially if there are possibilities to incorporate or leverage existing and proven solutions. This type of thinking seems like it will over time force us to push all of the features from other modules systems, such as the OSGi framework, into the core platform. The service/service-provider interop strawman is another step in the direction of pulling more and more into the core. While the strawman in and of itself sounds reasonable, it is proposing modifications to the module layer to introduce service-related concepts (e.g., ModuleDefinition.getServices() and ModuleDefinitions.getServiceProviders()), which seem woefully out of place to me. I would expect that service interoperability should be implementable completely on top of the module layer, without pushing service concepts down into the module layer. As an aside, I was slightly humored by parts of the strawman that implicitly describe what amounts to an OSGi service registry dealing with the same issues of service selection and wiring in the face of multiple providers and multiple versions... Since what goes into the core SE platform is going to stick around for a while and evolve very slowly, it seems to make the most sense to define the minimal set of features possible to accomplish what needs to be accomplished to provide core modularity support in the Java platform. Adding more fanciful features on top of this minimal core should be left to the OSGi frameworks and NetBeans frameworks of the world, which are able to evolve more rapidly on top of the core. Trying to do too much in the core is a recipe for disaster since we all know that there is no going backwards once it is released, whereas focusing on the smallest set of features necessary gives us a better chance at doing it right the first time. I think this is the point that Glyn is getting at too... -> richard > > In terms of the relationship between JSR 277 and JSR 291, I think we all > agreed in the EG that JSR 277 will provide the necessary framework for > interoperation with other module systems, especially with JSR 291/OSGi. > > Following up the original email about import package, I don't think we > should support it in JSR 277 for a different reason, and I will reply in > another email. > > - Stanley From Stanley.Ho at sun.com Fri May 25 15:30:04 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Fri, 25 May 2007 15:30:04 -0700 Subject: [Fwd: Re: Relationship to JSR 291 [was: Re: Bryan's comments]] Message-ID: <4657636C.4020202@sun.com> This is another repost to sync up the EG observer mailing list. - Stanley -------- Original Message -------- Subject: Re: Relationship to JSR 291 [was: Re: Bryan's comments] Date: Fri, 25 May 2007 13:45:57 -0700 From: Stanley M. Ho Reply-To: Java Community Process JSR #277 Expert List Organization: Sun Microsystems, Inc. To: JSR-277-EG at JCP.ORG References: <465625C0.1050107 at sun.com> <4656FA00.7000507 at ungoverned.org> Hi Richard, Richard S. Hall wrote: > ... > Are you really saying that 277 is mandated to create everything from > scratch? If so, I wasn't aware that this was mandated and it seems > somewhat odd to me, especially if there are possibilities to incorporate > or leverage existing and proven solutions. > > This type of thinking seems like it will over time force us to push all > of the features from other modules systems, such as the OSGi framework, > into the core platform. That's not exactly what I was saying. JSR 277 is set out to address a set of modularity, packaging and deployment problems that are fundamental in the Java SE platform, e.g. classpath hell, versioning hell, other issues related to jars and standard extensions, etc. The solution will be used in a variety of applications and environments, e.g. standalone applications, sandboxed applications (applets, JNLP), etc. It will also be used along with JSR 294 for modularizing the JRE in the future. Thus, the features and functionalities in JSR 277 must be sufficient to address this set of problems. One example is JRE modularization. If we need certain features in JSR 277 to make this possible, we'll create these features; leveraging features in other module systems to do JRE modularization would be completely missing the point. :-) I totally agreed with you that we don't want to push all the features from other module systems into the core. On the other hand, I don't think we can avoid bringing some features into the core if they are necessary to address some very fundamental or common problems in the SE platform, and hopefully everyone will be benefited as a result. > The service/service-provider interop strawman is another step in the > direction of pulling more and more into the core. While the strawman in > and of itself sounds reasonable, it is proposing modifications to the > module layer to introduce service-related concepts (e.g., > ModuleDefinition.getServices() and > ModuleDefinitions.getServiceProviders()), which seem woefully out of > place to me. I would expect that service interoperability should be > implementable completely on top of the module layer, without pushing > service concepts down into the module layer. As an aside, I was slightly > humored by parts of the strawman that implicitly describe what amounts > to an OSGi service registry dealing with the same issues of service > selection and wiring in the face of multiple providers and multiple > versions... The service/service-provider model has existed in the SE platform since JDK 1.3 (as far as I recalled), and this is what the JRE has been using internally to look up SPI implementations in various components. The JAR file format also has support for service-providers, but it has many problems related to versioning and others that are supposed to be addressed by JSR 277. Since this service/service-provider model is already used heavily with JARs in the core platform and we envision modules will be used in many deployment scenarios where JARs are used today, what the strawman describes is simply how this existing service/service-provider model would fit in the world of modules. At a minimal, I think this existing service/service-provider model should be supported in the module system defined by JSR 277. The strawman currently describes this service/service-provider model would be supported in the module layer (in the hope that all module systems can benefit from it so they don't have to keep working around the service-provider lookup problems themselves), but I think this is still up for debate in another thread. This is the whole point of having a strawman in the first place. ;-) > Since what goes into the core SE platform is going to stick around for a > while and evolve very slowly, it seems to make the most sense to define > the minimal set of features possible to accomplish what needs to be > accomplished to provide core modularity support in the Java platform. > Adding more fanciful features on top of this minimal core should be left > to the OSGi frameworks and NetBeans frameworks of the world, which are > able to evolve more rapidly on top of the core. Trying to do too much in > the core is a recipe for disaster since we all know that there is no > going backwards once it is released, whereas focusing on the smallest > set of features necessary gives us a better chance at doing it right the > first time. I think this is the point that Glyn is getting at too... > > -> richard I may have misunderstood the context behind Glyn's statement about "ditch every JSR 277 feature that is overlapped with JSR 291". :-) But I agreed with you completely that we should define the minimal set of features possible to accomplish what we need to be accomplished, and I think this is what we have been doing all along. - Stanley From Stanley.Ho at sun.com Fri May 25 15:30:32 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Fri, 25 May 2007 15:30:32 -0700 Subject: [Fwd: Re: Relationship to JSR 291 [was: Re: Bryan's comments]] Message-ID: <46576388.10302@sun.com> This is another repost to sync up the EG observer mailing list. - Stanley -------- Original Message -------- Subject: Re: Relationship to JSR 291 [was: Re: Bryan's comments] Date: Fri, 25 May 2007 11:01:03 -0400 From: Richard S. Hall Reply-To: Java Community Process JSR #277 Expert List To: JSR-277-EG at JCP.ORG References: <4655DD63.7020900 at oracle.com> <465632F8.3080900 at sun.com> Stanley M. Ho wrote: > Hi Bryan, > > Bryan Atsatt wrote: >> Let me clarify. It is my assumption that 277 will provide both an >> API/framework *and* a specific ModuleSystem implementation. And that >> OSGi will be able to supply another ModuleSystem implementation: >> >> 277 API/Framework >> / \ >> / \ >> 277ModuleSystem OSGiModuleSystem > > Right. There are also other module systems that can be plugged into this > framework, e.g. NetBeans. > >> Therefore, the API's must *support* "import package", or it will not be >> usable by the OSGiModuleSystem. >> >> I agree that the 277ModuleSystem implementation does not need to support >> "import package" semantics. Which means that the 277 APIs must support >> *both* semantics. >> >> Otherwise, OSGi will not be able to operate as a first-class citizen in >> the 277 world. >> >> And, if 294 adds support for "import", it should at least do the natural >> thing of "import package". And it should also support "import module", >> but supporting both at the language level may be awkward. >> >> // Bryan > > I think we all agreed that the module system defined by JSR 277 does not > need to support import package. As far as I am aware (correct me if I'm > wrong), all the existing systems support import at the module level, and > OSGi is the only one that supports import at the package level. Thus, > import by package should be a feature specific to the OSGi specific > module system implementation (e.g. OSGiModuleDefinition), and I don't > think it is necessary to push it into the base classes in JSR 277. Well, I must at least say that this is flawed logic at best. The fact that only the OSGi framework supports importing packages does not say anything about whether or not it should be supported by 277 or not. There any many valid arguments why the import package model is better due to the fact the is declares "what" is needed, not "who" contains what is needed. Using "who" instead of "what" generally leads to terrible dependency fanout due to coarse-grained module definitions and brittleness. I am fairly certain that I could conjure up some people who could tell you stories about the horrors of the brittleness of the module dependency approach when trying to re-use modules for different purposes. However, I guess this is beside the point... I think the real argument that should be made for not necessarily including import package support is along the lines of what I was saying in my other message, which is that we should keep 277 as simple as possible. > This doesn't mean supporting the notion of import by package is > impossible. What OSGi will expose from its repository is simply module > definitions, and module definition is basically an abstraction. > Therefore, it should be possible to treat an OSGi bundle as a module > definition, as well as to treat each exported package in an OSGi bundle > a module definition as well. The dependency supported in JSR 277 is > import-by-module. In the JSR 277 module system, this is mapped to > import-by-superpackage. In the OSGi module system, this can be mapped to > import-by-OSGi-bundle or import-by-package. If other module systems want > to support different import granularity or if OSGi evolves to add new > import granularity in the future, this model should still work. > > Honestly, interoperability between multiple module systems with "import > module" alone is already a hard enough problem for us to solve. I am > concerned that we're making this problem towards insolvable if we keep > adding complexity. ;) Agreed, but I think part of the reason that there is an urge to push such features into the core is because there is already feature creep in the core that gives people the impression that we are redefining OSGi modularity, NetBeans modularity, etc. Thus, everyone wants to make sure their pet feature is added. If 277 could make a stand and declare that it is at a layer beneath all of that (and live up to that declaration), then there would likely be less push down. -> richard From Stanley.Ho at sun.com Fri May 25 15:31:13 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Fri, 25 May 2007 15:31:13 -0700 Subject: [Fwd: Re: Relationship to JSR 291 [was: Re: Bryan's comments]] Message-ID: <465763B1.4040403@sun.com> This is another repost to sync up the EG observer mailing list. - Stanley -------- Original Message -------- Subject: Re: Relationship to JSR 291 [was: Re: Bryan's comments] Date: Sat, 26 May 2007 00:25:38 +0900 From: Daniel Leuck Reply-To: Java Community Process JSR #277 Expert List To: JSR-277-EG at JCP.ORG References: <465625C0.1050107 at sun.com> <4656FA00.7000507 at ungoverned.org> > Since what goes into the core SE platform is going to stick around for a > while and evolve very slowly, it seems to make the most sense to define > the minimal set of features possible to accomplish what needs to be > accomplished to provide core modularity support in the Java platform. > Adding more fanciful features on top of this minimal core should be left > to the OSGi frameworks and NetBeans frameworks of the world, which are > able to evolve more rapidly on top of the core. Trying to do too much in > the core is a recipe for disaster since we all know that there is no > going backwards once it is released, whereas focusing on the smallest > set of features necessary gives us a better chance at doing it right the > first time. I think this is the point that Glyn is getting at too... I agree. We need to be far more risk averse with the core. Lets get the basics right. From Stanley.Ho at sun.com Fri May 25 15:33:03 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Fri, 25 May 2007 15:33:03 -0700 Subject: [Fwd: Re: Relationship to JSR 291 [was: Re: Bryan's comments]] Message-ID: <4657641F.5010206@sun.com> This is another repost to sync up the EG observer mailing list. - Stanley -------- Original Message -------- Subject: Re: Relationship to JSR 291 [was: Re: Bryan's comments] Date: Fri, 25 May 2007 17:06:08 +0100 From: Glyn Normington Reply-To: Java Community Process JSR #277 Expert List To: JSR-277-EG at JCP.ORG I haven't lost interest in this thread but I'm going on leave now returning on Wednesday, so please don't misinterpret my silence in the interim. :-) Glyn ------------------------------------------------------------------------ / / /Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU/ From Stanley.Ho at Sun.COM Fri May 25 15:56:42 2007 From: Stanley.Ho at Sun.COM (Stanley M. Ho) Date: Fri, 25 May 2007 15:56:42 -0700 Subject: Email problem In-Reply-To: <46574CE5.4020301@sun.com> References: <464E283C.4050207@sun.com> <46563BC6.70402@sun.com> <46574CE5.4020301@sun.com> Message-ID: <465769AA.2000602@sun.com> Apparently, the problem has been identified - one of the public blacklist servers has the IP address of one of our servers being black listed, and it caused some emails which went through the EG mailing list successfully but bounced back from the observer mailing list on openjdk. This problem has now been fixed. I have just reposted the messages that were bounced back from the observer list so the observers will see them: http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/2007-May/thread.html Sorry for any confusion. - Stanley Stanley M. Ho wrote: > > I have received several bounced emails from the observer mailing list > since yesterday, but the EG mailing list seems to be fine. I'll try to > sync up the observer mailing list after all the networking changes are > settled early next week. > > - Stanley > > > Stanley M. Ho wrote: >> Hi, >> >> This is just a head-up. There has been some massive networking changes >> going on since early this week and are expected to be finished over the >> Memorial Day (next Monday). It is unclear if these changes will impact >> this EG mailing list and the observer mailing list. If you experience >> any problem, please let me know. >> >> - Stanley From andyp at bea.com Tue May 29 03:39:58 2007 From: andyp at bea.com (Andy Piper) Date: Tue, 29 May 2007 11:39:58 +0100 Subject: Relationship to JSR 291 [was: Re: Bryan's comments] In-Reply-To: <465760F4.5070306@sun.com> References: <4655DD63.7020900@oracle.com> <465632F8.3080900@sun.com> <4656FA2F.7080506@ungoverned.org> <465760F4.5070306@sun.com> Message-ID: <6.2.5.6.2.20070529113817.02acea70@bea.com> At 23:19 25/05/2007, Stanley M. Ho wrote: >Anyway, it seems the EG consensus so far is to not add import package >support. If any EG member disagrees, please speak up. Well, it depends on what the solution for enabling interoperation with JSR 291 is. Our requirement is that there must be a solution, if that requires import package, so be it. If not then not. andy Notice: This email message, together with any attachments, may contain information of BEA Systems, Inc., its subsidiaries and affiliated entities, that may be confidential, proprietary, copyrighted and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by email and then delete it. From heavy at UNGOVERNED.ORG Tue May 29 10:38:59 2007 From: heavy at UNGOVERNED.ORG (Richard S. Hall) Date: Tue, 29 May 2007 13:38:59 -0400 Subject: Relationship to JSR 291 [was: Re: Bryan's comments] In-Reply-To: <465760F4.5070306@sun.com> References: <4655DD63.7020900@oracle.com> <465632F8.3080900@sun.com> <4656FA2F.7080506@ungoverned.org> <465760F4.5070306@sun.com> Message-ID: <465C6533.5080200@ungoverned.org> Hello Stanley, Stanley M. Ho wrote: > Richard S. Hall wrote: >> Agreed, but I think part of the reason that there is an urge to push >> such features into the core is because there is already feature creep in >> the core that gives people the impression that we are redefining OSGi >> modularity, NetBeans modularity, etc. Thus, everyone wants to make sure >> their pet feature is added. If 277 could make a stand and declare that >> it is at a layer beneath all of that (and live up to that declaration), >> then there would likely be less push down. > > Richard, It is unclear to me what you meant by feature creep in the core > because everything in the specification is essentially required to > address the problems this JSR is set out to solve. We can argue that > some features should not be in the base layer API but in the module > system defined by JSR 277 instead or vice versa, but they still belong > to the overall JSR 277 specification. I guess the addition of service-related concepts into the module layer would be one example of feature creep for me. While I think it makes perfect sense to figure out how the Service Loader stuff will work on top of the module layer, pushing Service Loader concepts down into the module layer doesn't make any sense to me at all. It should be sufficient for the Service Loader to probe the installed modules and perhaps examine module metadata to determine if a module provides a service or not. From there, the Service Loader can create module instances and provider instances as necessary. If this is not possible in our current constructs, then we should address these shortcomings rather than adding higher layer concepts into the module layer. -> richard > To recap what we have been discussing, I think our general consensus is > to define a module system in JSR 277 that would address all the problems > this JSR is set out to solve, as well as to provide a high level > framework for different module systems (including the module system > defined by JSR 277) to layer on top for interoperation. We only > described the former in the EDR, but the latter should become more > evident in the next revision of the JSR 277 specification. > > - Stanley From cierniak at google.com Tue May 29 11:14:19 2007 From: cierniak at google.com (Michal Cierniak) Date: Tue, 29 May 2007 11:14:19 -0700 Subject: JSR-277 talk on Google Video Message-ID: <237294540705291114h43f8ec25l230eddb7f98d8fc6@mail.gmail.com> Hi all, Stanley and I gave a talk about JSR-277 at Google. The talk was videotaped and is now available on Google Video: http://video.google.com/videoplay?docid=-8224544168880829029 This is exactly the same talk as what we gave at JavaOne earlier this month ("JSR 277: Java Module System", http://developers.sun.com/learning/javaoneonline/j1sessn.jsp?sessn=TS-2318&yr=2007&track=5), except for the questions from the audience that are different. The talk doesn't present anything new to those who have been following the mailing list and read the latest draft. However I would like to remind that the latest draft is not yet available publicly, so if you are reading this email as an observer, you don't have access to it yet. A recent email from Stanley said that he is investigating the best way to make the draft available but for now, the comments on this mailing list and the previous draft are the best companion materials to the video of the talk. Michal From bryan.atsatt at oracle.com Tue May 29 23:57:59 2007 From: bryan.atsatt at oracle.com (Bryan Atsatt) Date: Tue, 29 May 2007 23:57:59 -0700 Subject: Relationship to JSR 291 [was: Re: Bryan's comments] In-Reply-To: <6.2.5.6.2.20070529113817.02acea70@bea.com> References: <4655DD63.7020900@oracle.com> <465632F8.3080900@sun.com> <4656FA2F.7080506@ungoverned.org> <465760F4.5070306@sun.com> <6.2.5.6.2.20070529113817.02acea70@bea.com> Message-ID: <465D2077.30106@oracle.com> Andy Piper wrote: > At 23:19 25/05/2007, Stanley M. Ho wrote: >> Anyway, it seems the EG consensus so far is to not add import package >> support. If any EG member disagrees, please speak up. > > Well, it depends on what the solution for enabling interoperation > with JSR 291 is. > Our requirement is that there must be a solution, if that requires > import package, so be it. If not then not. Exactly. I think we can all agree that, at minimum, interoperation means that classes and resources are sharable *across* ModuleSystems at runtime. Which implies that *import dependencies must be resolvable across multiple ModuleSystem instances*. (BTW, I think we should change the state name "PREPARING" to "RESOLVING" in 7.2.1.) So the open issue is the richness of the import "language": must we support only lowest-common-denominator, or can we do better without over-complicating the design? I for one would like to be able to have a single module express dependencies on modules from both the same and different ModuleSystems, *using the standard semantics of each*. This may be reaching too far, but we should at least explore it seriously while we figure out what interop means here... BASICS So far, we only know of two different import semantics: module-name, and package-name. For discussion, let's call these: a. import-module b. import-package So, to start, we could: 1. Support declaration of both import types. If 294 supports imports at all, it should be relatively easy to support both, since a superpackage name is a module name, and it contains member package names. (Compiler support is clearly the critical issue here, but it will obviously require use of the 277 runtime, so the import *type* should be transparent to it.) At worst, we'd need two new annotation types. 2. Provide API for both import types (e.g. ImportDependency has getModuleName() and getPackageName() methods, one of which will return null on a given instance). However, we know these are necessary but not sufficient. Leaving aside the resolution issue for a moment, support for import-package also suggests that we: 3. Enable a single module to declare different versions for each of its member packages. 4. Enable efficient Repository lookup by package name. I think these are relatively easy (but *could* be considered optional). We also need: 5. Standard Query types for lookup by module and package name. EXISTING DEPENDENCY RESOLUTION MODEL The more interesting issue is dependency resolution. But this hasn't been discussed in any real detail, so lets do so before talking further about import-package. To simplify this discussion, I'm ignoring bundled/custom import policies for now... Resolution in the current spec is delegated to the associated ModuleSystem instance (7.2.2 #8). While the details are not spelled out, the expectation appears to be that ModuleSystem.getModule(ModuleDefinition) must: - Select an initial repository. Call getRepository() on the input parameter. Then, for each ImportDependency of the definition: - Select a matching definition. Construct a Query from the ImportDependency and use Repository.find() to lookup a matching ModuleDefinition. - Get an instance. Use def.getModuleSystem().getModule(def). The ModuleSystem is expected to return a cached instance if available, or create/cache/return one if not. (TBD: The PlatformBinding must be taken into account somehow during selection. ModuleDefinition must include an accessor for it, and either Repository.find() should implicitly filter them, or the caller must construct a Query which will do so. I think we should add a CURRENT_PLATFORM constant to Query, which will evaluate to true if no binding is present in a definition.) (The spec also talks about Repository as the mechanism of isolation (6.4). This was the case in the prototype, where the repository itself provided caching. It doesn't appear to work with the current design. There is no need that I can see to isolate ModuleDefinition instances--it is Module instances with their associated loaders that may require isolation.) (Also note that if ImportDependency was itself a Query subclass, there would be no need to do any mapping. And since the ModuleDefinition subclass must produce ImportDependency instances, it can even produce more specialized Query instances if desired.) REFINEMENT I think we can improve on the existing model in several ways: A. Provide a model for Module isolation (e.g. for EE, Applets, etc). B. Encapsulate all selection logic in a single mechanism. C. Eliminate the overhead of the repository lookup when a cached instance exists. Let me propose a new class that encapsulates the caching logic, enables lookup using Query, and supports multiple instances for isolation: public abstract class ModuleContext { // Get the context used to define JRE modules. public static ModuleContext getBootstrapContext(){...}; // Get the context used to define the main module. public static ModuleContext getSystemContext(){...}; // Get all contexts. public static List getContexts() {...}; // Add a new context. public static void addContext(ModuleContext ctx) {...} // Remove a context (error if == default). public static boolean removeContext(ModuleContext ctx) {...} // Get the parent context (null if bootstrap). public ModuleContext getParentContext(){...} // Get the name of this context. public String getContextName() {...} // Create a new Module instance and store it in the cache. public abstract Module createModule(ModuleDefinition def); // Find cached Module instances. Must check parent first. public abstract List findModules(Query query); // Set the context used for JRE modules. static void setBootstrapContext(ModuleContext ctx){...} // Set the context used to define the main module. static void setSystemContext(ModuleContext ctx){...} } The JVM will create an appropriate subtype and call setBootstrapContext(). The launcher will create a child context and call setSystemContext(). An EE (or similar) environment can create/remove new contexts as needed for application isolation. And the resolution algorithm can now check the cache *first*, before doing a repository lookup, using the same mechanism in both. Query should be used to express *all* selection criteria (including attributes, which we have not yet directly supported). Caches are no longer tied to ModuleSystem instances. ModuleSystem.getModule() can become simply createModule(). The normal implementation of ModuleContext.createModule() just calls ModuleSystem.createModule() and caches/returns the result. This class could easily be made concrete, but it may be useful to support subtypes for specialization (e.g. event generation, lifecycle management, specialized diagnostics, etc). RESOLUTION MODELS The current design requires that each ModuleSystem provide its own resolution logic, and that each definition will be resolved by its owning ModuleSystem. This model appears to provide flexibility for significant differences in implementation, but we really don't know enough at this point. Perhaps only an actual second implementation will tell us if this provides useful flexibility. It wouldn't surprise me if we have to keep tightening the spec as we go, in order to remove inconsistencies that arise from the separate algorithms. And this may eliminate flexibility to the point where it is no longer useful. Much worse, we may not even discover this until after the spec and RI are released, if that second implementation (e.g. OSGi) is not completed beforehand. We should at least consider the obvious alternative: one algorithm (to rule them all :^). And I don't mean one hard-coded algorithm, I mean one replaceable, extensible class, such as: public abstract class ImportResolver { public abstract Module resolve(ModuleDefinition def); } And we add a method to ModuleContext to retrieve an instance: public abstract class ModuleContext { ... public abstract ImportResolver getImportResolver(); } (Note that ImportResolver is now in a position to subsume the functionality of both VisibilityPolicy and ImportOverridePolicy.) Repository usage is encapsulated within the implementation of the resolve() method. The full resolution algorithm becomes: context.getImportResolver().resolve(definition); The launcher uses the "system" context for this. EE, Applets, etc. make and use their own distinct, isolated instances. RESOLUTION WITH IMPORT-NAME AND IMPORT-PACKAGE With this scaffolding in place we can easily take a phased approach to supporting import-package: the initial implementation simply does not support it at runtime. A subsequent implementation may support import-package, but only within the boundaries of the same ModuleSystem. And a full blown implementation may support import-package across ModuleSystems. We can build in support for selecting/configuring the ImportResolver as a service, just as we plan to do with ModuleSystem (and Repository, I presume). And maybe, just maybe, we can find a way to abstract and re-use the mature resolution logic from the OSGi reference implementation *as* the one implementation to rule them all. // Bryan From glyn_normington at UK.IBM.COM Wed May 30 04:27:51 2007 From: glyn_normington at UK.IBM.COM (Glyn Normington) Date: Wed, 30 May 2007 12:27:51 +0100 Subject: ClassLoader Deadlock fix was: Re: Relationship to JSR 291 [was: Re: Bryan's comments] In-Reply-To: <1180093494.3531.61.camel@warjort> Message-ID: Adrian wrote on 25/05/2007 12:44:54: > On Thu, 2007-05-24 at 11:00 +0100, Glyn Normington wrote: > > A better approach would be for the Java 7 platform to provide first > > class support for JSR 291. This boils down to standardising the > > experimental class loader deadlock fix ([1]) > > Fixing the deadlock just moves the problem. > > You'll still get ClassCircularityErrors when competing threads > try to load classes using locks other than the classloader > synchronization or they don't synchronize on the loadClass() > or they release the lock during the classloading request > to let others have a go (again to avoid the deadlock). > > This is because of the way the dictionary class determines > whether a circular load is occuring. > > Although I haven't tried it with OpenJDK so maybe the > dictionary class contains some other fixes to workaround the > problem? > > The simple form of the problem: > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4699981 > has been fixed in recent JDKs, but spurious CCEs > still exist in other cases. > e.g. the testAbstractFactoryConcurrent() here: > http://viewvc.jboss.org/cgi-bin/viewvc. > cgi/jbossas/projects/microcontainer/trunk/classloader/src/tests/org/jboss/test/classloader/delegate/test/DelegateUnitTestCase. > java?revision=62792&view=markup > will show CCEs in the log if you enable TRACE logging. > > 1445 TRACE [ClassLoaderManager] Run failed with exception > java.lang.ClassCircularityError: > org/jboss/test/classloader/delegate/support/b/TestFactoryImplementation > at java.lang.ClassLoader.defineClass1(Native Method) > at java.lang.ClassLoader.defineClass(ClassLoader.java:620) Some spurious CCEs were indeed fixed before the experimental deadlock fix was introduced, but there may be more to do to make this fix robust and complete. If you raise a sunbug for the above testcase, best to report it here so that class loader rearchitecture folks can take it into consideration for Java 7. Glyn Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/attachments/20070530/6491adc6/attachment.html From glyn_normington at UK.IBM.COM Wed May 30 09:07:40 2007 From: glyn_normington at UK.IBM.COM (Glyn Normington) Date: Wed, 30 May 2007 17:07:40 +0100 Subject: Relationship to JSR 291 [was: Re: Bryan's comments] In-Reply-To: <465D2077.30106@oracle.com> Message-ID: Bryan Atsatt wrote on 30/05/2007 07:57:59: > Andy Piper wrote: > > At 23:19 25/05/2007, Stanley M. Ho wrote: > >> Anyway, it seems the EG consensus so far is to not add import package > >> support. If any EG member disagrees, please speak up. > > > > Well, it depends on what the solution for enabling interoperation > > with JSR 291 is. > > Our requirement is that there must be a solution, if that requires > > import package, so be it. If not then not. > > Exactly. > > I think we can all agree that, at minimum, interoperation means that > classes and resources are sharable *across* ModuleSystems at runtime. > > Which implies that *import dependencies must be resolvable across > multiple ModuleSystem instances*. (BTW, I think we should change the > state name "PREPARING" to "RESOLVING" in 7.2.1.) Agreed. We must avoid the trap of thinking that module system interop. can be achieved by exposing class loaders (as loadClass will happily load unexported classes). > > So the open issue is the richness of the import "language": must we > support only lowest-common-denominator, or can we do better without > over-complicating the design? > > I for one would like to be able to have a single module express > dependencies on modules from both the same and different ModuleSystems, > *using the standard semantics of each*. This may be reaching too far, > but we should at least explore it seriously while we figure out what > interop means here... At this point, I feel that is likely to be reaching too far, but I'm happy to play along and see what we can learn along the way. > > > BASICS > > So far, we only know of two different import semantics: module-name, and > package-name. For discussion, let's call these: > > a. import-module > b. import-package > > So, to start, we could: > > 1. Support declaration of both import types. If 294 supports imports at > all, it should be relatively easy to support both, since a superpackage > name is a module name, and it contains member package names. (Compiler > support is clearly the critical issue here, but it will obviously > require use of the 277 runtime, so the import *type* should be > transparent to it.) At worst, we'd need two new annotation types. A superpackage name is a deployment module name in the JSR 277 model of one superpackage per deployment module, but I don't see any reason why a JSR 291 deployment module should not contain more than one superpackage. So if 294 were to support import, then its import-module would really be a superpackage import rather than a development module import. > > 2. Provide API for both import types (e.g. ImportDependency has > getModuleName() and getPackageName() methods, one of which will return > null on a given instance). > > However, we know these are necessary but not sufficient. Leaving aside > the resolution issue for a moment, support for import-package also > suggests that we: > > 3. Enable a single module to declare different versions for each of its > member packages. > > 4. Enable efficient Repository lookup by package name. > > I think these are relatively easy (but *could* be considered optional). > > We also need: > > 5. Standard Query types for lookup by module and package name. > > > EXISTING DEPENDENCY RESOLUTION MODEL > > The more interesting issue is dependency resolution. But this hasn't > been discussed in any real detail, so lets do so before talking further > about import-package. To simplify this discussion, I'm ignoring > bundled/custom import policies for now... > > Resolution in the current spec is delegated to the associated > ModuleSystem instance (7.2.2 #8). While the details are not spelled out, > the expectation appears to be that > ModuleSystem.getModule(ModuleDefinition) must: > > - Select an initial repository. Call getRepository() on the input parameter. > > Then, for each ImportDependency of the definition: > > - Select a matching definition. Construct a Query from the > ImportDependency and use Repository.find() to lookup a matching > ModuleDefinition. > > - Get an instance. Use def.getModuleSystem().getModule(def). The > ModuleSystem is expected to return a cached instance if available, or > create/cache/return one if not. I think there also needs to be some 'resolution context' object which explicitly denotes a particular resolution so that each module system can keep track of the state of a resolution. This is required when two or more imports of a given module from another module system need to resolve to the same module instance. A resolution context may also be needed for back-tracking when a set of module instances created earlier in resolution turn out not to satisfy all the necessary constraints. > > > (TBD: The PlatformBinding must be taken into account somehow during > selection. ModuleDefinition must include an accessor for it, and either > Repository.find() should implicitly filter them, or the caller must > construct a Query which will do so. I think we should add a > CURRENT_PLATFORM constant to Query, which will evaluate to true if no > binding is present in a definition.) > > (The spec also talks about Repository as the mechanism of isolation > (6.4). This was the case in the prototype, where the repository itself > provided caching. It doesn't appear to work with the current design. > There is no need that I can see to isolate ModuleDefinition > instances--it is Module instances with their associated loaders that may > require isolation.) > > (Also note that if ImportDependency was itself a Query subclass, there > would be no need to do any mapping. And since the ModuleDefinition > subclass must produce ImportDependency instances, it can even produce > more specialized Query instances if desired.) > > > REFINEMENT > > I think we can improve on the existing model in several ways: > > A. Provide a model for Module isolation (e.g. for EE, Applets, etc). > > B. Encapsulate all selection logic in a single mechanism. > > C. Eliminate the overhead of the repository lookup when a cached > instance exists. > > Let me propose a new class that encapsulates the caching logic, enables > lookup using Query, and supports multiple instances for isolation: > > public abstract class ModuleContext { > > // Get the context used to define JRE modules. > public static ModuleContext getBootstrapContext(){...}; > > // Get the context used to define the main module. > public static ModuleContext getSystemContext(){...}; > > // Get all contexts. > public static List getContexts() {...}; > > // Add a new context. > public static void addContext(ModuleContext ctx) {...} > > // Remove a context (error if == default). > public static boolean removeContext(ModuleContext ctx) {...} > > // Get the parent context (null if bootstrap). > public ModuleContext getParentContext(){...} > > // Get the name of this context. > public String getContextName() {...} > > // Create a new Module instance and store it in the cache. > public abstract Module createModule(ModuleDefinition def); > > // Find cached Module instances. Must check parent first. > public abstract List findModules(Query query); > > // Set the context used for JRE modules. > static void setBootstrapContext(ModuleContext ctx){...} > > // Set the context used to define the main module. > static void setSystemContext(ModuleContext ctx){...} > } > > The JVM will create an appropriate subtype and call > setBootstrapContext(). The launcher will create a child context and call > setSystemContext(). An EE (or similar) environment can create/remove new > contexts as needed for application isolation. > > And the resolution algorithm can now check the cache *first*, before > doing a repository lookup, using the same mechanism in both. Query > should be used to express *all* selection criteria (including > attributes, which we have not yet directly supported). > > Caches are no longer tied to ModuleSystem instances. > ModuleSystem.getModule() can become simply createModule(). The normal > implementation of ModuleContext.createModule() just calls > ModuleSystem.createModule() and caches/returns the result. > > > This class could easily be made concrete, but it may be useful to > support subtypes for specialization (e.g. event generation, lifecycle > management, specialized diagnostics, etc). > > > RESOLUTION MODELS > > The current design requires that each ModuleSystem provide its own > resolution logic, and that each definition will be resolved by its > owning ModuleSystem. This model appears to provide flexibility for > significant differences in implementation, but we really don't know > enough at this point. Perhaps only an actual second implementation will > tell us if this provides useful flexibility. > > It wouldn't surprise me if we have to keep tightening the spec as we go, > in order to remove inconsistencies that arise from the separate > algorithms. And this may eliminate flexibility to the point where it is > no longer useful. Much worse, we may not even discover this until after > the spec and RI are released, if that second implementation (e.g. OSGi) > is not completed beforehand. > > We should at least consider the obvious alternative: one algorithm (to > rule them all :^). And I don't mean one hard-coded algorithm, I mean one > replaceable, extensible class, such as: > > public abstract class ImportResolver { > public abstract Module resolve(ModuleDefinition def); > } > > And we add a method to ModuleContext to retrieve an instance: > > public abstract class ModuleContext { > ... > public abstract ImportResolver getImportResolver(); > } > > (Note that ImportResolver is now in a position to subsume the > functionality of both VisibilityPolicy and ImportOverridePolicy.) > > Repository usage is encapsulated within the implementation of the > resolve() method. The full resolution algorithm becomes: > > context.getImportResolver().resolve(definition); > > The launcher uses the "system" context for this. EE, Applets, etc. make > and use their own distinct, isolated instances. > > > RESOLUTION WITH IMPORT-NAME AND IMPORT-PACKAGE > > With this scaffolding in place we can easily take a phased approach to > supporting import-package: the initial implementation simply does not > support it at runtime. A phased approach would be particularly beneficial if the initial phase could be delivered as part of Java 7 and subsequent phases implemented strictly on top of Java 7. But getting the API right up front might be tricky unless we can spot some really good abstractions or prototype the later phases sufficiently well. Is that the kind of phasing you had in mind? > > A subsequent implementation may support import-package, but only within > the boundaries of the same ModuleSystem. > > And a full blown implementation may support import-package across > ModuleSystems. > > We can build in support for selecting/configuring the ImportResolver as > a service, just as we plan to do with ModuleSystem (and Repository, I > presume). > > > And maybe, just maybe, we can find a way to abstract and re-use the > mature resolution logic from the OSGi reference implementation *as* the > one implementation to rule them all. > > // Bryan Glyn Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/attachments/20070530/ca7389a8/attachment.html From bryan.atsatt at oracle.com Wed May 30 11:11:02 2007 From: bryan.atsatt at oracle.com (Bryan Atsatt) Date: Wed, 30 May 2007 11:11:02 -0700 Subject: Relationship to JSR 291 [was: Re: Bryan's comments] In-Reply-To: References: Message-ID: <465DBE36.2020604@oracle.com> Responses inline, and a few clarifications here (I was a bit tired when I finished this last night :^)... The main point I was trying to make is that resolution must occur within a specific context, but I don't think my example APIs showed that well. I was assuming that ImportResolver had a ModuleContext as a field, but we can make this much cleaner and easier to understand by passing it as an argument: public abstract class ImportResolver { public abstract Module resolve(ModuleDefinition def, ModuleContext ctx); } And we can really tie this together by adding a convenience method to ModuleContext: public abstract class ModuleContext { ... public Module resolve(ModuleDefinition def) { return getImportResolver().resolve(def, this); } } Now resolution becomes simply: context.resolve(definition); (I also left out any use of the ImportPolicy, as it isn't yet clear to me that it should remain a separate type in this model.) // Bryan Glyn Normington wrote: > > *Bryan Atsatt * wrote on 30/05/2007 07:57:59: > > > Andy Piper wrote: > > > At 23:19 25/05/2007, Stanley M. Ho wrote: > > >> Anyway, it seems the EG consensus so far is to not add import package > > >> support. If any EG member disagrees, please speak up. > > > > > > Well, it depends on what the solution for enabling interoperation > > > with JSR 291 is. > > > Our requirement is that there must be a solution, if that requires > > > import package, so be it. If not then not. > > > > Exactly. > > > > I think we can all agree that, at minimum, interoperation means that > > classes and resources are sharable *across* ModuleSystems at runtime. > > > > Which implies that *import dependencies must be resolvable across > > multiple ModuleSystem instances*. (BTW, I think we should change the > > state name "PREPARING" to "RESOLVING" in 7.2.1.) > > Agreed. We must avoid the trap of thinking that module system interop. > can be achieved by exposing class loaders (as loadClass will happily > load unexported classes). > > > > > So the open issue is the richness of the import "language": must we > > support only lowest-common-denominator, or can we do better without > > over-complicating the design? > > > > I for one would like to be able to have a single module express > > dependencies on modules from both the same and different ModuleSystems, > > *using the standard semantics of each*. This may be reaching too far, > > but we should at least explore it seriously while we figure out what > > interop means here... > > At this point, I feel that is likely to be reaching too far, but I'm > happy to play along and see what we can learn along the way. > > > > > > > BASICS > > > > So far, we only know of two different import semantics: module-name, and > > package-name. For discussion, let's call these: > > > > a. import-module > > b. import-package > > > > So, to start, we could: > > > > 1. Support declaration of both import types. If 294 supports imports at > > all, it should be relatively easy to support both, since a superpackage > > name is a module name, and it contains member package names. (Compiler > > support is clearly the critical issue here, but it will obviously > > require use of the 277 runtime, so the import *type* should be > > transparent to it.) At worst, we'd need two new annotation types. > > A superpackage name is a deployment module name in the JSR 277 model of > one superpackage per deployment module, but I don't see any reason why a > JSR 291 deployment module should not contain more than one superpackage. > So if 294 were to support import, then its import-module would really be > a superpackage import rather than a development module import. If we end up with nested superpackages, might it make sense to model multiple superpackages by enclosing them in a single top-level one? > > > > > 2. Provide API for both import types (e.g. ImportDependency has > > getModuleName() and getPackageName() methods, one of which will return > > null on a given instance). > > > > However, we know these are necessary but not sufficient. Leaving aside > > the resolution issue for a moment, support for import-package also > > suggests that we: > > > > 3. Enable a single module to declare different versions for each of its > > member packages. > > > > 4. Enable efficient Repository lookup by package name. > > > > I think these are relatively easy (but *could* be considered optional). > > > > We also need: > > > > 5. Standard Query types for lookup by module and package name. > > > > > > EXISTING DEPENDENCY RESOLUTION MODEL > > > > The more interesting issue is dependency resolution. But this hasn't > > been discussed in any real detail, so lets do so before talking further > > about import-package. To simplify this discussion, I'm ignoring > > bundled/custom import policies for now... > > > > Resolution in the current spec is delegated to the associated > > ModuleSystem instance (7.2.2 #8). While the details are not spelled out, > > the expectation appears to be that > > ModuleSystem.getModule(ModuleDefinition) must: > > > > - Select an initial repository. Call getRepository() on the input > parameter. > > > > Then, for each ImportDependency of the definition: > > > > - Select a matching definition. Construct a Query from the > > ImportDependency and use Repository.find() to lookup a matching > > ModuleDefinition. > > > > - Get an instance. Use def.getModuleSystem().getModule(def). The > > ModuleSystem is expected to return a cached instance if available, or > > create/cache/return one if not. > > I think there also needs to be some 'resolution context' object which > explicitly denotes a particular resolution so that each module system > can keep track of the state of a resolution. This is required when two > or more imports of a given module from another module system need to > resolve to the same module instance. A resolution context may also be > needed for back-tracking when a set of module instances created earlier > in resolution turn out not to satisfy all the necessary constraints. Yes. I had (briefly :^) thought that we need only a List as field in the ImportResolver to hold the resolved modules as we go. But probably we need to keep some additional state along with each Module, so a new type may be required. Let's call it ResolutionContext, and create one at the start of each resolution: public abstract class ImportResolver { public Module resolve(ModuleDefinition def, ModuleContext modCtx) { ResolutionContext resCtx = createResolutionContext(); return resolve(def, modCtx, resCtx); } protected abstract ResolutionContext createResolutionContext(); protected abstract Module resolve(ModuleDefinition def, ModuleContext modCtx, ResolutionContext resCtx); } Does this approach make sense? Any thoughts on details for ResolutionContext? > > > > > > > (TBD: The PlatformBinding must be taken into account somehow during > > selection. ModuleDefinition must include an accessor for it, and either > > Repository.find() should implicitly filter them, or the caller must > > construct a Query which will do so. I think we should add a > > CURRENT_PLATFORM constant to Query, which will evaluate to true if no > > binding is present in a definition.) > > > > (The spec also talks about Repository as the mechanism of isolation > > (6.4). This was the case in the prototype, where the repository itself > > provided caching. It doesn't appear to work with the current design. > > There is no need that I can see to isolate ModuleDefinition > > instances--it is Module instances with their associated loaders that may > > require isolation.) > > > > (Also note that if ImportDependency was itself a Query subclass, there > > would be no need to do any mapping. And since the ModuleDefinition > > subclass must produce ImportDependency instances, it can even produce > > more specialized Query instances if desired.) > > > > > > REFINEMENT > > > > I think we can improve on the existing model in several ways: > > > > A. Provide a model for Module isolation (e.g. for EE, Applets, etc). > > > > B. Encapsulate all selection logic in a single mechanism. > > > > C. Eliminate the overhead of the repository lookup when a cached > > instance exists. > > > > Let me propose a new class that encapsulates the caching logic, enables > > lookup using Query, and supports multiple instances for isolation: > > > > public abstract class ModuleContext { > > > > // Get the context used to define JRE modules. > > public static ModuleContext getBootstrapContext(){...}; > > > > // Get the context used to define the main module. > > public static ModuleContext getSystemContext(){...}; > > > > // Get all contexts. > > public static List getContexts() {...}; > > > > // Add a new context. > > public static void addContext(ModuleContext ctx) {...} > > > > // Remove a context (error if == default). > > public static boolean removeContext(ModuleContext ctx) {...} > > > > // Get the parent context (null if bootstrap). > > public ModuleContext getParentContext(){...} > > > > // Get the name of this context. > > public String getContextName() {...} > > > > // Create a new Module instance and store it in the cache. > > public abstract Module createModule(ModuleDefinition def); > > > > // Find cached Module instances. Must check parent first. > > public abstract List findModules(Query query); > > > > // Set the context used for JRE modules. > > static void setBootstrapContext(ModuleContext ctx){...} > > > > // Set the context used to define the main module. > > static void setSystemContext(ModuleContext ctx){...} > > } > > > > The JVM will create an appropriate subtype and call > > setBootstrapContext(). The launcher will create a child context and call > > setSystemContext(). An EE (or similar) environment can create/remove new > > contexts as needed for application isolation. > > > > And the resolution algorithm can now check the cache *first*, before > > doing a repository lookup, using the same mechanism in both. Query > > should be used to express *all* selection criteria (including > > attributes, which we have not yet directly supported). > > > > Caches are no longer tied to ModuleSystem instances. > > ModuleSystem.getModule() can become simply createModule(). The normal > > implementation of ModuleContext.createModule() just calls > > ModuleSystem.createModule() and caches/returns the result. > > > > > > This class could easily be made concrete, but it may be useful to > > support subtypes for specialization (e.g. event generation, lifecycle > > management, specialized diagnostics, etc). > > > > > > RESOLUTION MODELS > > > > The current design requires that each ModuleSystem provide its own > > resolution logic, and that each definition will be resolved by its > > owning ModuleSystem. This model appears to provide flexibility for > > significant differences in implementation, but we really don't know > > enough at this point. Perhaps only an actual second implementation will > > tell us if this provides useful flexibility. > > > > It wouldn't surprise me if we have to keep tightening the spec as we go, > > in order to remove inconsistencies that arise from the separate > > algorithms. And this may eliminate flexibility to the point where it is > > no longer useful. Much worse, we may not even discover this until after > > the spec and RI are released, if that second implementation (e.g. OSGi) > > is not completed beforehand. > > > > We should at least consider the obvious alternative: one algorithm (to > > rule them all :^). And I don't mean one hard-coded algorithm, I mean one > > replaceable, extensible class, such as: > > > > public abstract class ImportResolver { > > public abstract Module resolve(ModuleDefinition def); > > } > > > > And we add a method to ModuleContext to retrieve an instance: > > > > public abstract class ModuleContext { > > ... > > public abstract ImportResolver getImportResolver(); > > } > > > > (Note that ImportResolver is now in a position to subsume the > > functionality of both VisibilityPolicy and ImportOverridePolicy.) > > > > Repository usage is encapsulated within the implementation of the > > resolve() method. The full resolution algorithm becomes: > > > > context.getImportResolver().resolve(definition); > > > > The launcher uses the "system" context for this. EE, Applets, etc. make > > and use their own distinct, isolated instances. > > > > > > RESOLUTION WITH IMPORT-NAME AND IMPORT-PACKAGE > > > > With this scaffolding in place we can easily take a phased approach to > > supporting import-package: the initial implementation simply does not > > support it at runtime. > > A phased approach would be particularly beneficial if the initial phase > could be delivered as part of Java 7 and subsequent phases implemented > strictly on top of Java 7. But getting the API right up front might be > tricky unless we can spot some really good abstractions or prototype the > later phases sufficiently well. Is that the kind of phasing you had in > mind? Yes. It would be *extremely* useful to have an OSGi implementation/prototype well under way before Java 7 is completed, so that we can fine tune the model as we learn. > > > > > A subsequent implementation may support import-package, but only within > > the boundaries of the same ModuleSystem. > > > > And a full blown implementation may support import-package across > > ModuleSystems. > > > > We can build in support for selecting/configuring the ImportResolver as > > a service, just as we plan to do with ModuleSystem (and Repository, I > > presume). > > > > > > And maybe, just maybe, we can find a way to abstract and re-use the > > mature resolution logic from the OSGi reference implementation *as* the > > one implementation to rule them all. > > > > // Bryan > > Glyn > > > ------------------------------------------------------------------------ > > / > / > > /Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU/ > > > > > > From Stanley.Ho at SUN.COM Wed May 30 11:40:57 2007 From: Stanley.Ho at SUN.COM (Stanley M. Ho) Date: Wed, 30 May 2007 11:40:57 -0700 Subject: ClassLoader Deadlock fix In-Reply-To: <1180093494.3531.61.camel@warjort> References: <1180093494.3531.61.camel@warjort> Message-ID: <465DC539.7050108@sun.com> Hi JSR 277 experts, Below are some comments from Jeff Nisewanger and Karen Kinnear who are working on the ClassLoader API improvements for Java SE 7. If you are interested in contributing to the ClassLoader API improvements as a reviewer, you may contact Jeff and Karen at jeff.nisewanger at sun.com and karen.kinnear at sun.com. - Stanley -------- Original Message -------- Hi, Sun is working on the ClassLoader Deadlock fix. Our goal is to work closely with the community, of which the 277 EG contains key members, in coming up with small modifications to the current ClassLoader APIs to resolve that issue. As we've mentioned before, and Glyn even points out in his referenced blog, we did add a temporary risky unstable workaround which only really handles a small special case for WebSphere 6.1 which should be supplanted by the new mechanism as that becomes available. One of the key issues here is that a prerequisite for avoiding the ClassLoader deadlocks is that the JVM must not hold the ClassLoader Object lock when it calls out to LoadClass[Internal]. And presumably the lock must not be held when the custom class loader's loadClass() is called. As anyone who has written a custom class loader would immediately recognize, changing that underlying assumption is inherently risky. It is all too easy to break existing custom class loaders that were perhaps unknowingly relying on that lock to protect critical data and operations. Those are the kind of breakages that are extremely timing dependent and can pass many test cases and then unexpectedly fail in the field. I'm sure you all also remember when Unix user level libraries were converted to being MT-safe. We didn't get them all right the first time around. We don't want to put our customers in that position. So a part of the ClassLoader API modifications would be for class loaders to explicitly declare that they are safe for parallel class loading, both for multiple classes in the same class loader, and for multiple instances of the same class in the same class loader. I have been assuming that this would be the default for JSR 277, but we can work that issue in detail when we have a strawman ClassLoader API proposal. In the meantime, thank you very much to Adrian Brock for his classcircularity example. I will be studying it in detail. If any other EG member has a test scenario they would like to describe to us or even better they have a test case for, we would very much appreciate it. And if you are interested in contributing to the ClassLoader API improvements as a reviewer, please let us know. Thank you, Jeff Nisewanger & Karen Kinnear From Stanley.Ho at sun.com Wed May 30 12:19:45 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Wed, 30 May 2007 12:19:45 -0700 Subject: Exported resources In-Reply-To: <465658CE.5020409@oracle.com> References: <4654109D.40205@sun.com> <46548736.7020507@oracle.com> <465629EA.2050008@sun.com> <465658CE.5020409@oracle.com> Message-ID: <465DCE51.4010304@sun.com> Hi Bryan, Those resource-related methods in ClassLoader can be called by anyone, including code that is part of the module, code that is from other modules, or code that is part of the platform libraries (e.g. ResourceBundle). The approach you described would require walking the stack to get the caller's Module, but the real issue is that it is difficult to determine who the actual caller is from the stack. Treating the immediate caller on the stack as the actual caller wouldn't be sufficient because the immediate caller could be called by someone else who is the one actually making the call. On the other hand, treating the originated caller on the stack as the actual caller would be the right semantic, but this is basically the same as the security permission approach. - Stanley Bryan Atsatt wrote: > Both solutions require stack walking (unless there is some new > implementation of the java security model I've not yet seen!). > > The permission check does much more work than is necessary here. Take a > look at AccessController.checkPermission() to see what I mean. > > And actually there is a very simple API to get the stack, which I've > used for years: > > private static class StackAccessor extends SecurityManager { > public Class[] getStack() { > return getClassContext(); > } > } > > private static final STACK_ACCESSOR = new StackAccessor(); > > Now the enclosing class can simply call STACK_ACCESSOR.getStack(). > > // Bryan > > > > Stanley M. Ho wrote: >> Hi Bryan, >> >> Bryan Atsatt wrote: >>> 1. Definitely agree that resource search order should be identical to >>> class search order. >> >> Glad to hear! >> >>> 2. Using permissions to limit access to private resources seems like >>> overkill to me. The prototype implemented this in a very simple fashion: >>> >>> a. If resource is exported, return it, else >>> a. Get the caller's Module (get class from stack, get module from it) >>> b. If callerModule == this, return resource, else return null. >> >> The issue is that this approach still requires stack walking and there >> is no public API in the SE platform that let you implement this. >> >> If stack walking is required for the check anyway, I think the security >> permission approach is better that it is implementable with the existing >> API in the SE platform. >> >> - Stanley >> From stanley.ho at Sun.COM Wed May 30 12:35:47 2007 From: stanley.ho at Sun.COM (Stanley M. Ho) Date: Wed, 30 May 2007 12:35:47 -0700 Subject: Service providers (was: Richard's comments) In-Reply-To: <465C6533.5080200@ungoverned.org> References: <4655DD63.7020900@oracle.com> <465632F8.3080900@sun.com> <4656FA2F.7080506@ungoverned.org> <465760F4.5070306@sun.com> <465C6533.5080200@ungoverned.org> Message-ID: <465DD213.4050309@sun.com> Hi Richard, I have renamed the subject so we can discuss issues around the service-provider strawman in its own thread. Richard S. Hall wrote: > Hello Stanley, > ... > I guess the addition of service-related concepts into the module layer > would be one example of feature creep for me. While I think it makes > perfect sense to figure out how the Service Loader stuff will work on > top of the module layer, pushing Service Loader concepts down into the > module layer doesn't make any sense to me at all. > > It should be sufficient for the Service Loader to probe the installed > modules and perhaps examine module metadata to determine if a module > provides a service or not. From there, the Service Loader can create > module instances and provider instances as necessary. > > If this is not possible in our current constructs, then we should > address these shortcomings rather than adding higher layer concepts into > the module layer. > > -> richard The current service provider strawman suggests a few changes in ModuleDefinition/Query classes, and you basically suggest that these changes could be eliminated if we have a more generic way to express services and service-providers in the module metadata and a generic way to examine the information from the module metadata, all without service-related APIs in the module layer. I think what you suggested makes sense, and I will look into this. - Stanley From heavy at UNGOVERNED.ORG Wed May 30 13:06:07 2007 From: heavy at UNGOVERNED.ORG (Richard S. Hall) Date: Wed, 30 May 2007 16:06:07 -0400 Subject: Service providers In-Reply-To: <465DD213.4050309@sun.com> References: <4655DD63.7020900@oracle.com> <465632F8.3080900@sun.com> <4656FA2F.7080506@ungoverned.org> <465760F4.5070306@sun.com> <465C6533.5080200@ungoverned.org> <465DD213.4050309@sun.com> Message-ID: <465DD92F.6010303@ungoverned.org> Hey Stanley, Stanley M. Ho wrote: > Richard S. Hall wrote: >> Hello Stanley, >> ... >> I guess the addition of service-related concepts into the module layer >> would be one example of feature creep for me. While I think it makes >> perfect sense to figure out how the Service Loader stuff will work on >> top of the module layer, pushing Service Loader concepts down into the >> module layer doesn't make any sense to me at all. >> >> It should be sufficient for the Service Loader to probe the installed >> modules and perhaps examine module metadata to determine if a module >> provides a service or not. From there, the Service Loader can create >> module instances and provider instances as necessary. >> >> If this is not possible in our current constructs, then we should >> address these shortcomings rather than adding higher layer concepts into >> the module layer. >> >> -> richard > > The current service provider strawman suggests a few changes in > ModuleDefinition/Query classes, and you basically suggest that these > changes could be eliminated if we have a more generic way to express > services and service-providers in the module metadata and a generic way > to examine the information from the module metadata, all without > service-related APIs in the module layer. > > I think what you suggested makes sense, and I will look into this. To me, this sounds like what we call the "extender model" for the OSGi framework. The way it works is that bundles that want to participate in certain scenarios simply include some metadata inside of themselves, then some infrastructure can probe for this metadata and automatically do work on their behalf. This is how Declarative Services works in the R4 spec and also how Spring-OSGi works...it is becoming a very common model. The benefit of this approach is that it doesn't push upper layer concepts down into the modularity layer and keeps it simple. Off the top of my head, I don't see a reason why the Service Loader couldn't use a similar approach. -> richard From bryan.atsatt at oracle.com Wed May 30 13:21:36 2007 From: bryan.atsatt at oracle.com (Bryan Atsatt) Date: Wed, 30 May 2007 13:21:36 -0700 Subject: Exported resources In-Reply-To: <465DCE51.4010304@sun.com> References: <4654109D.40205@sun.com> <46548736.7020507@oracle.com> <465629EA.2050008@sun.com> <465658CE.5020409@oracle.com> <465DCE51.4010304@sun.com> Message-ID: <465DDCD0.8090703@oracle.com> I've been assuming that Module private resources should not be visible to *any* class outside of the module. Including ResourceBundle, or any other existing framework classes that do resource lookups (e.g. ServiceLoader, JSF, etc). If resources need to be visible to these existing classes, they must be exported. The very simple check I proposed (immediate caller) is sufficient to make this assertion. I believe your point is that if we used the permission model instead, it would become possible for a module to invoke an external class (e.g. ResourceBundle.getBundle()) and enable *it* to successfully load a private resource from the module. Aside from the permission *grant* mechanism this model would rely on, it is an entirely different model than that used for classes! (Though we haven't explicitly defined this in 294, it seems extremely unlikely that we will rely on permissions--none of the other access modes do so.) Such asymmetry is very disconcerting to me, and, I believe, just plain wrong... Consider that you could grant the ServiceLoader, for example, access to a resource that names a class that it could not instantiate. That class would have to be exported. I believe the resource should be as well. // Bryan Stanley M. Ho wrote: > Hi Bryan, > > Those resource-related methods in ClassLoader can be called by anyone, > including code that is part of the module, code that is from other > modules, or code that is part of the platform libraries (e.g. > ResourceBundle). The approach you described would require walking the > stack to get the caller's Module, but the real issue is that it is > difficult to determine who the actual caller is from the stack. > > Treating the immediate caller on the stack as the actual caller wouldn't > be sufficient because the immediate caller could be called by someone > else who is the one actually making the call. On the other hand, > treating the originated caller on the stack as the actual caller would > be the right semantic, but this is basically the same as the security > permission approach. > > - Stanley > > > Bryan Atsatt wrote: >> Both solutions require stack walking (unless there is some new >> implementation of the java security model I've not yet seen!). >> >> The permission check does much more work than is necessary here. Take a >> look at AccessController.checkPermission() to see what I mean. >> >> And actually there is a very simple API to get the stack, which I've >> used for years: >> >> private static class StackAccessor extends SecurityManager { >> public Class[] getStack() { >> return getClassContext(); >> } >> } >> >> private static final STACK_ACCESSOR = new StackAccessor(); >> >> Now the enclosing class can simply call STACK_ACCESSOR.getStack(). >> >> // Bryan >> >> >> >> Stanley M. Ho wrote: >>> Hi Bryan, >>> >>> Bryan Atsatt wrote: >>>> 1. Definitely agree that resource search order should be identical to >>>> class search order. >>> >>> Glad to hear! >>> >>>> 2. Using permissions to limit access to private resources seems like >>>> overkill to me. The prototype implemented this in a very simple >>>> fashion: >>>> >>>> a. If resource is exported, return it, else >>>> a. Get the caller's Module (get class from stack, get module from it) >>>> b. If callerModule == this, return resource, else return null. >>> >>> The issue is that this approach still requires stack walking and there >>> is no public API in the SE platform that let you implement this. >>> >>> If stack walking is required for the check anyway, I think the security >>> permission approach is better that it is implementable with the existing >>> API in the SE platform. >>> >>> - Stanley >>> > From Stanley.Ho at sun.com Wed May 30 14:55:48 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Wed, 30 May 2007 14:55:48 -0700 Subject: Relationship to JSR 291 [was: Re: Bryan's comments] In-Reply-To: References: Message-ID: <465DF2E4.4090603@sun.com> Hi, Glyn Normington wrote: > > *Bryan Atsatt * wrote on 30/05/2007 07:57:59: > ... > > So the open issue is the richness of the import "language": must we > > support only lowest-common-denominator, or can we do better without > > over-complicating the design? > > > > I for one would like to be able to have a single module express > > dependencies on modules from both the same and different ModuleSystems, > > *using the standard semantics of each*. This may be reaching too far, > > but we should at least explore it seriously while we figure out what > > interop means here... > > At this point, I feel that is likely to be reaching too far, but I'm > happy to play along and see what we can learn along the way. I agreed with Glyn that this might be reaching too far. Before we dive too much into how to implement interoperability, I think one of the outstanding questions we should answer first is what degree of import interoperability we want to offer. There are four possibilities: 1. JSR 277 module imports OSGi module by module name 2. OSGi module imports JSR 277 module by module name 3. OSGi module imports JSR 277 module by package name 4. JSR 277 module imports OSGi module by package name Let's ignore the 294 issues and the module initialization issues for now to simplify this discussion. I think we all agreed #1 and #2 are important to support and the reflective APIs already enable these (of course, there are minor issues we still have to address as we evolve the APIs.) That said, it is unclear to me how important it is to support #3 and #4, so I think the first question for this EG is whether we want to support #3 and #4 at all. However, let's pretend #3 and #4 are important for this specific discussion. I think #3 is already possible since the OSGi framework could look up the appropriate module which exported the package in the repository, using the reflective APIs. Afterwards, the OSGi framework can then do the necessary wiring. Yes, the reflective APIs can be further refined to make it easier for the OSGi framework to query modules by exported package name from the repository, but this is a minor issue that we can address easily. #4 is a bit complicated, because there is no import-by-package semantic in the current module layer's APIs and I think we all agreed that we don't want to support this semantic directly in the module system defined by JSR 277. In this context, the question is really about what it would take to make #4 possible if we want to support it, and so far there are two different approaches we have discussed: a. Treat import-by-package separately from import-by-module in the reflective APIs and make the module layer fully aware of the import-by-package concept. b. Expose exported-package using the ModuleDefinition abstraction, and provide necessary hooks (e.g. consistency checking) for the module system to resolve this kind of dependency appropriately. The module layer is not aware of the import-by-package concept at all. New import dependency granularity could also be introduced in a similar manner in the future without requiring significant changes in the module layer. As Richard and I discussed in previous emails, it seems to make the most sense for JSR 277 to define the minimal set of features possible in the module layer to accomplish what needs to be accomplished to provide core modularity support in the Java platform. I think (b) follows this principle nicely while (a) does not. If we decide to support #3 and #4, then how we want to address #4 would impact our overall design significantly. Is there anyone in the EG provide good reasons why #3 and #4 are important to be supported? If so, is there anyone in the EG think (b) is not feasible at all and we should do (a) instead to support #4? Can you explain your rationale? > > RESOLUTION MODELS > > > > The current design requires that each ModuleSystem provide its own > > resolution logic, and that each definition will be resolved by its > > owning ModuleSystem. This model appears to provide flexibility for > > significant differences in implementation, but we really don't know > > enough at this point. Perhaps only an actual second implementation will > > tell us if this provides useful flexibility. In the existing module systems we are aware, e.g. OSGi and NetBeans, we already know their resolution logics are different from each other. Even if they are migrated/re-based to JSR 277 in the future, I think we should expect their resolution logic would remain the same to maintain backward compatibility with their existing modules/bundles. In other words, their resolution logic would still be different from each other. > A phased approach would be particularly beneficial if the initial phase > could be delivered as part of Java 7 and subsequent phases implemented > strictly on top of Java 7. But getting the API right up front might be > tricky unless we can spot some really good abstractions or prototype the > later phases sufficiently well. Is that the kind of phasing you had in > mind? Once an API is put into the SE platform, it will stick around almost forever. I think if we go with the phased approach on interoperability support (I believe this is what Bryan suggested), it would be much better to simply left out those features/APIs which do not have implementations to back them up from JDK 7, rather than putting the half-baked features/APIs into the release and regret later. ;) We can always add new APIs in subsequence releases if we decide to offer better degree of interoperability in the future. - Stanley From Stanley.Ho at sun.com Wed May 30 15:12:38 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Wed, 30 May 2007 15:12:38 -0700 Subject: Exported resources In-Reply-To: <465DDCD0.8090703@oracle.com> References: <4654109D.40205@sun.com> <46548736.7020507@oracle.com> <465629EA.2050008@sun.com> <465658CE.5020409@oracle.com> <465DCE51.4010304@sun.com> <465DDCD0.8090703@oracle.com> Message-ID: <465DF6D6.5050608@sun.com> Hi Bryan, A module can use the ResourceBundle API to retrieve resources from other resource modules, but it can also use the ResourceBundle API to retrieve resources from the (target) module itself. In the latter case, the resources are currently not required to be exported from the target module. I don't think we want to force a module to export its own private resources simply because it wants to use the ResourceBundle API in this case, would you agree? Regarding there are differences in the access models between classes and resources, I also prefer symmetry if possible. However, the access model for classes and resources have always been different, and there are built-in JVM support for classes while there is none for resources, so it is unclear if we can completely eliminate this asymmetry after all. - Stanley Bryan Atsatt wrote: > I've been assuming that Module private resources should not be visible > to *any* class outside of the module. Including ResourceBundle, or any > other existing framework classes that do resource lookups (e.g. > ServiceLoader, JSF, etc). If resources need to be visible to these > existing classes, they must be exported. The very simple check I > proposed (immediate caller) is sufficient to make this assertion. > > I believe your point is that if we used the permission model instead, it > would become possible for a module to invoke an external class (e.g. > ResourceBundle.getBundle()) and enable *it* to successfully load a > private resource from the module. > > Aside from the permission *grant* mechanism this model would rely on, it > is an entirely different model than that used for classes! (Though we > haven't explicitly defined this in 294, it seems extremely unlikely that > we will rely on permissions--none of the other access modes do so.) Such > asymmetry is very disconcerting to me, and, I believe, just plain wrong... > > Consider that you could grant the ServiceLoader, for example, access to > a resource that names a class that it could not instantiate. That class > would have to be exported. I believe the resource should be as well. > > // Bryan From Stanley.Ho at sun.com Wed May 30 15:16:34 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Wed, 30 May 2007 15:16:34 -0700 Subject: Service providers In-Reply-To: <465DD92F.6010303@ungoverned.org> References: <4655DD63.7020900@oracle.com> <465632F8.3080900@sun.com> <4656FA2F.7080506@ungoverned.org> <465760F4.5070306@sun.com> <465C6533.5080200@ungoverned.org> <465DD213.4050309@sun.com> <465DD92F.6010303@ungoverned.org> Message-ID: <465DF7C2.3090805@sun.com> Hi Richard, Richard S. Hall wrote: > Hey Stanley, >... > To me, this sounds like what we call the "extender model" for the OSGi > framework. The way it works is that bundles that want to participate in > certain scenarios simply include some metadata inside of themselves, > then some infrastructure can probe for this metadata and automatically > do work on their behalf. This is how Declarative Services works in the > R4 spec and also how Spring-OSGi works...it is becoming a very common > model. > > The benefit of this approach is that it doesn't push upper layer > concepts down into the modularity layer and keeps it simple. Off the top > of my head, I don't see a reason why the Service Loader couldn't use a > similar approach. > > -> richard Thanks. I will certainly check it out to see if this approach is feasible. - Stanley From bryan.atsatt at oracle.com Wed May 30 16:00:17 2007 From: bryan.atsatt at oracle.com (Bryan Atsatt) Date: Wed, 30 May 2007 16:00:17 -0700 Subject: Exported resources In-Reply-To: <465DF6D6.5050608@sun.com> References: <4654109D.40205@sun.com> <46548736.7020507@oracle.com> <465629EA.2050008@sun.com> <465658CE.5020409@oracle.com> <465DCE51.4010304@sun.com> <465DDCD0.8090703@oracle.com> <465DF6D6.5050608@sun.com> Message-ID: <465E0201.2030308@oracle.com> Hey Stanley, Sorry to be generating so much traffic while you're traveling! I'm not in any rush here, so feel free to take your time responding... Stanley M. Ho wrote: > Hi Bryan, > > A module can use the ResourceBundle API to retrieve resources from other > resource modules, but it can also use the ResourceBundle API to retrieve > resources from the (target) module itself. In the latter case, the > resources are currently not required to be exported from the target module. > > I don't think we want to force a module to export its own private > resources simply because it wants to use the ResourceBundle API in this > case, would you agree? No, I don't agree. That's what I said before :^). I *really* don't like the idea that I'm going to have to explicitly grant permission to some module to access my resource. This *significantly* complicates deployment. How will the grant occur, especially given that each environment can use an entirely different permission storage mechanism? Or are you thinking that the loaders in the module system will construct ProtectionDomains pre-wired with permissions? If so, exactly which entities will be blessed? Just the JRE code? What about all the existing non-JRE frameworks in use? > Regarding there are differences in the access models between classes and > resources, I also prefer symmetry if possible. However, the access model > for classes and resources have always been different, and there are > built-in JVM support for classes while there is none for resources, so > it is unclear if we can completely eliminate this asymmetry after all. Sure, resources have never had JVM access control applied to them. But I don't think we should go to the other extreme (permissions) just so that I can give "friend" access to private resources. I'd much prefer that such resources simply be exported. And I don't see this as a particular problem, either. We've gotten along just fine without private resources so far (or maybe there is some big demand for this that I'm missing?). And what about the mismatch issue I brought up? Lots of existing frameworks use the "services" pattern, in which: 1. A well known resource name is used in a getResource() call. 2. The resource contains a string naming a provider class. 3. The provider class is loaded using Class.forName(). This is a simple but very powerful pattern, but it requires that *both* the resource and the named class be accessible to the framework. If we require a permission grant for the resource, but not for the class, it will be very easy to get access to one and not the other. Is it really worth opening up this potential headache to enable a friend model for resources? // Bryan > > - Stanley > > > Bryan Atsatt wrote: >> I've been assuming that Module private resources should not be visible >> to *any* class outside of the module. Including ResourceBundle, or any >> other existing framework classes that do resource lookups (e.g. >> ServiceLoader, JSF, etc). If resources need to be visible to these >> existing classes, they must be exported. The very simple check I >> proposed (immediate caller) is sufficient to make this assertion. >> >> I believe your point is that if we used the permission model instead, it >> would become possible for a module to invoke an external class (e.g. >> ResourceBundle.getBundle()) and enable *it* to successfully load a >> private resource from the module. >> >> Aside from the permission *grant* mechanism this model would rely on, it >> is an entirely different model than that used for classes! (Though we >> haven't explicitly defined this in 294, it seems extremely unlikely that >> we will rely on permissions--none of the other access modes do so.) Such >> asymmetry is very disconcerting to me, and, I believe, just plain >> wrong... >> >> Consider that you could grant the ServiceLoader, for example, access to >> a resource that names a class that it could not instantiate. That class >> would have to be exported. I believe the resource should be as well. >> >> // Bryan > From heavy at UNGOVERNED.ORG Wed May 30 16:02:36 2007 From: heavy at UNGOVERNED.ORG (Richard S. Hall) Date: Wed, 30 May 2007 19:02:36 -0400 Subject: Relationship to JSR 291 [was: Re: Bryan's comments] In-Reply-To: <465DF2E4.4090603@sun.com> References: <465DF2E4.4090603@sun.com> Message-ID: <9E52FB97-F312-4A4F-A096-ECA0430F9D7C@ungoverned.org> On May 30, 2007, at 5:55 PM, Stanley M. Ho wrote: > Glyn Normington wrote: >> >> *Bryan Atsatt * wrote on 30/05/2007 >> 07:57:59: >> ... >> > So the open issue is the richness of the import "language": >> must we >> > support only lowest-common-denominator, or can we do better >> without >> > over-complicating the design? >> > >> > I for one would like to be able to have a single module express >> > dependencies on modules from both the same and different >> ModuleSystems, >> > *using the standard semantics of each*. This may be reaching >> too far, >> > but we should at least explore it seriously while we figure out >> what >> > interop means here... >> >> At this point, I feel that is likely to be reaching too far, but I'm >> happy to play along and see what we can learn along the way. > > I agreed with Glyn that this might be reaching too far. Before we dive > too much into how to implement interoperability, I think one of the > outstanding questions we should answer first is what degree of import > interoperability we want to offer. There are four possibilities: > > 1. JSR 277 module imports OSGi module by module name > 2. OSGi module imports JSR 277 module by module name > 3. OSGi module imports JSR 277 module by package name > 4. JSR 277 module imports OSGi module by package name > > Let's ignore the 294 issues and the module initialization issues > for now > to simplify this discussion. > > I think we all agreed #1 and #2 are important to support and the > reflective APIs already enable these (of course, there are minor > issues > we still have to address as we evolve the APIs.) > > That said, it is unclear to me how important it is to support #3 > and #4, > so I think the first question for this EG is whether we want to > support > #3 and #4 at all. However, let's pretend #3 and #4 are important for > this specific discussion. > > I think #3 is already possible since the OSGi framework could look up > the appropriate module which exported the package in the repository, > using the reflective APIs. Afterwards, the OSGi framework can then do > the necessary wiring. Yes, the reflective APIs can be further > refined to > make it easier for the OSGi framework to query modules by exported > package name from the repository, but this is a minor issue that we > can > address easily. I am not so sure supporting #3 is as straight forward as you suggest here. Specially, the OSGi modularity layer [for the most part] assumes that packages are atomic units; there is no way to specify that your export or import something smaller than a package. Since 277 modules declare exports in terms of classes, it is not clear how we would know if a 277 module was exporting an entire package. This would, at a minimum, require that we assume we can inspect the contents of the module and see that the contents of a contained package matches the module's export signature for that package. -> richard > > #4 is a bit complicated, because there is no import-by-package > semantic > in the current module layer's APIs and I think we all agreed that we > don't want to support this semantic directly in the module system > defined by JSR 277. In this context, the question is really about what > it would take to make #4 possible if we want to support it, and so far > there are two different approaches we have discussed: > > a. Treat import-by-package separately from import-by-module in the > reflective APIs and make the module layer fully aware of the > import-by-package concept. > > b. Expose exported-package using the ModuleDefinition abstraction, and > provide necessary hooks (e.g. consistency checking) for the module > system to resolve this kind of dependency appropriately. The module > layer is not aware of the import-by-package concept at all. New import > dependency granularity could also be introduced in a similar manner in > the future without requiring significant changes in the module layer. > > > As Richard and I discussed in previous emails, it seems to make the > most > sense for JSR 277 to define the minimal set of features possible in > the > module layer to accomplish what needs to be accomplished to provide > core > modularity support in the Java platform. I think (b) follows this > principle nicely while (a) does not. If we decide to support #3 and > #4, > then how we want to address #4 would impact our overall design > significantly. > > Is there anyone in the EG provide good reasons why #3 and #4 are > important to be supported? If so, is there anyone in the EG think > (b) is > not feasible at all and we should do (a) instead to support #4? Can > you > explain your rationale? > > >> > RESOLUTION MODELS >> > >> > The current design requires that each ModuleSystem provide its own >> > resolution logic, and that each definition will be resolved by its >> > owning ModuleSystem. This model appears to provide flexibility for >> > significant differences in implementation, but we really don't >> know >> > enough at this point. Perhaps only an actual second >> implementation will >> > tell us if this provides useful flexibility. > > In the existing module systems we are aware, e.g. OSGi and > NetBeans, we > already know their resolution logics are different from each other. > Even > if they are migrated/re-based to JSR 277 in the future, I think we > should expect their resolution logic would remain the same to maintain > backward compatibility with their existing modules/bundles. In other > words, their resolution logic would still be different from each > other. > >> A phased approach would be particularly beneficial if the initial >> phase >> could be delivered as part of Java 7 and subsequent phases >> implemented >> strictly on top of Java 7. But getting the API right up front >> might be >> tricky unless we can spot some really good abstractions or >> prototype the >> later phases sufficiently well. Is that the kind of phasing you >> had in >> mind? > > Once an API is put into the SE platform, it will stick around almost > forever. I think if we go with the phased approach on interoperability > support (I believe this is what Bryan suggested), it would be much > better to simply left out those features/APIs which do not have > implementations to back them up from JDK 7, rather than putting the > half-baked features/APIs into the release and regret later. ;) We can > always add new APIs in subsequence releases if we decide to offer > better > degree of interoperability in the future. > > - Stanley From bryan.atsatt at ORACLE.COM Wed May 30 18:09:37 2007 From: bryan.atsatt at ORACLE.COM (Bryan Atsatt) Date: Wed, 30 May 2007 18:09:37 -0700 Subject: Relationship to JSR 291 [was: Re: Bryan's comments] In-Reply-To: <465DF2E4.4090603@sun.com> References: <465DF2E4.4090603@sun.com> Message-ID: <465E2051.7020008@oracle.com> Stanley M. Ho wrote: > Hi, > > Glyn Normington wrote: >> >> *Bryan Atsatt * wrote on 30/05/2007 07:57:59: >> ... >> > So the open issue is the richness of the import "language": must we >> > support only lowest-common-denominator, or can we do better without >> > over-complicating the design? >> > >> > I for one would like to be able to have a single module express >> > dependencies on modules from both the same and different >> ModuleSystems, >> > *using the standard semantics of each*. This may be reaching too far, >> > but we should at least explore it seriously while we figure out what >> > interop means here... >> >> At this point, I feel that is likely to be reaching too far, but I'm >> happy to play along and see what we can learn along the way. > > I agreed with Glyn that this might be reaching too far. Before we dive > too much into how to implement interoperability, I think one of the > outstanding questions we should answer first is what degree of import > interoperability we want to offer. There are four possibilities: > > 1. JSR 277 module imports OSGi module by module name > 2. OSGi module imports JSR 277 module by module name > 3. OSGi module imports JSR 277 module by package name > 4. JSR 277 module imports OSGi module by package name > > Let's ignore the 294 issues and the module initialization issues for now > to simplify this discussion. > > I think we all agreed #1 and #2 are important to support and the > reflective APIs already enable these (of course, there are minor issues > we still have to address as we evolve the APIs.) > > That said, it is unclear to me how important it is to support #3 and #4, > so I think the first question for this EG is whether we want to support > #3 and #4 at all. However, let's pretend #3 and #4 are important for > this specific discussion. > > I think #3 is already possible since the OSGi framework could look up > the appropriate module which exported the package in the repository, > using the reflective APIs. Afterwards, the OSGi framework can then do > the necessary wiring. Yes, the reflective APIs can be further refined to > make it easier for the OSGi framework to query modules by exported > package name from the repository, but this is a minor issue that we can > address easily. > > #4 is a bit complicated, because there is no import-by-package semantic > in the current module layer's APIs and I think we all agreed that we > don't want to support this semantic directly in the module system > defined by JSR 277. Not me. I do understand your point that this is more difficult, but I don't really care--it is our job to do the Right Thing here. Sure, if it is not feasible in the time frame we have, then so be it. But we haven't even tried! If we're forced to make a choice between the two models, I'd pick import-by-package with zero hesitation. OSGi has a lot of experience here, and we shouldn't ignore it. Import-by-name was added late in the game, and is now seriously downplayed, for good reason. And Oracle has a lot of relevant experience as well, since the shared loader mechanism I created for our AS stack ("shared libraries") supports only the import-by-name model. We have *lots* of shared-libraries (I've seen 100+ running simultaneously), used by many different components in the stack, in addition to customer applications. It is a tremendous pain when refactoring or even simple renaming has to be done (I hope to shift this implementation onto 277 to solve this problem, rather than inventing an interim approach). And this kind of change happens far more frequently than you probably expect. I don't want to repeat the same mistake here. Nor do I think that leaving a long trail of extraneous "view" modules behind for compatibility is a good solution. Worse still, if the 277 APIs don't support import-by-package, then the compiler won't support it either. If the compiler doesn't support it, then we've lost a golden opportunity to move to a better world, one in which packaging is irrelevant. We're supposed to be eliminating "jar-hell" here. Let's not simply replace it with "module-hell". > In this context, the question is really about what > it would take to make #4 possible if we want to support it, and so far > there are two different approaches we have discussed: > > a. Treat import-by-package separately from import-by-module in the > reflective APIs and make the module layer fully aware of the > import-by-package concept. > > b. Expose exported-package using the ModuleDefinition abstraction, and > provide necessary hooks (e.g. consistency checking) for the module > system to resolve this kind of dependency appropriately. The module > layer is not aware of the import-by-package concept at all. New import > dependency granularity could also be introduced in a similar manner in > the future without requiring significant changes in the module layer. > > > As Richard and I discussed in previous emails, it seems to make the most > sense for JSR 277 to define the minimal set of features possible in the > module layer to accomplish what needs to be accomplished to provide core > modularity support in the Java platform. I think (b) follows this > principle nicely while (a) does not. If we decide to support #3 and #4, > then how we want to address #4 would impact our overall design > significantly. > > Is there anyone in the EG provide good reasons why #3 and #4 are > important to be supported? If so, is there anyone in the EG think (b) is > not feasible at all and we should do (a) instead to support #4? Can you > explain your rationale? Yes, and yes. See above for rationale (ease-of-refactoring and compiler support). > > >> > RESOLUTION MODELS >> > >> > The current design requires that each ModuleSystem provide its own >> > resolution logic, and that each definition will be resolved by its >> > owning ModuleSystem. This model appears to provide flexibility for >> > significant differences in implementation, but we really don't know >> > enough at this point. Perhaps only an actual second implementation >> will >> > tell us if this provides useful flexibility. > > In the existing module systems we are aware, e.g. OSGi and NetBeans, we > already know their resolution logics are different from each other. Even > if they are migrated/re-based to JSR 277 in the future, I think we > should expect their resolution logic would remain the same to maintain > backward compatibility with their existing modules/bundles. In other > words, their resolution logic would still be different from each other. And this is very likely to be the source of subtle but very painful interop issues. Which is why I also suggested that we should consider moving to a canonical, though evolvable, resolution mechanism. > >> A phased approach would be particularly beneficial if the initial phase >> could be delivered as part of Java 7 and subsequent phases implemented >> strictly on top of Java 7. But getting the API right up front might be >> tricky unless we can spot some really good abstractions or prototype the >> later phases sufficiently well. Is that the kind of phasing you had in >> mind? > > Once an API is put into the SE platform, it will stick around almost > forever. I think if we go with the phased approach on interoperability > support (I believe this is what Bryan suggested), it would be much > better to simply left out those features/APIs which do not have > implementations to back them up from JDK 7, rather than putting the > half-baked features/APIs into the release and regret later. ;) We can > always add new APIs in subsequence releases if we decide to offer better > degree of interoperability in the future. I couldn't agree more that we don't want to put half-baked APIs into SE! If we cannot expect significant validation of our interoperation model (whatever we settle on) from NetBeans and OSGi, then we shouldn't expose these APIs at all. Period. But this doesn't just apply to import-by-package. We haven't even defined how class and resource sharing will occur. I won't be a bit surprised if we want one or more new types here. And, last thoughts for the day... Even if you are adamantly opposed to import-by-package, please take a good look at the other ideas buried in my message from yesterday, particularly the ModuleContext proposal. It is completely independent of import-by-package, and seems to me at least to be a concise, flexible model for caching/isolating/finding Module instances. (The idea of exposing a resolve() method on it also makes a clean API, even if it delegates to ModuleSystem instead of a new resolver type.) There are other potentially interesting tidbits there as well (PlatformBinding exposure on definition, Query support for current platform, ImportDependency *as* Query). I'd hate for the baby to be thrown out with the bath water :^) // Bryan > > - Stanley > From glyn_normington at UK.IBM.COM Thu May 31 09:15:19 2007 From: glyn_normington at UK.IBM.COM (Glyn Normington) Date: Thu, 31 May 2007 17:15:19 +0100 Subject: Relationship to JSR 291 [was: Re: Bryan's comments] In-Reply-To: <465DBE36.2020604@oracle.com> Message-ID: Bryan Atsatt wrote on 30/05/2007 19:11:02: > Responses inline, and a few clarifications here (I was a bit tired when > I finished this last night :^)... > > The main point I was trying to make is that resolution must occur within > a specific context, but I don't think my example APIs showed that well. > I was assuming that ImportResolver had a ModuleContext as a field, but > we can make this much cleaner and easier to understand by passing it as > an argument: > > public abstract class ImportResolver { > public abstract Module resolve(ModuleDefinition def, > ModuleContext ctx); > } > > And we can really tie this together by adding a convenience method to > ModuleContext: > > public abstract class ModuleContext { > ... > public Module resolve(ModuleDefinition def) { > return getImportResolver().resolve(def, this); > } > } > > Now resolution becomes simply: > > context.resolve(definition); > > > (I also left out any use of the ImportPolicy, as it isn't yet clear to > me that it should remain a separate type in this model.) > > // Bryan > > Glyn Normington wrote: > > > > *Bryan Atsatt * wrote on 30/05/2007 07:57:59: > > > > > Andy Piper wrote: > > > > At 23:19 25/05/2007, Stanley M. Ho wrote: > > > >> Anyway, it seems the EG consensus so far is to not add import package > > > >> support. If any EG member disagrees, please speak up. > > > > > > > > Well, it depends on what the solution for enabling interoperation > > > > with JSR 291 is. > > > > Our requirement is that there must be a solution, if that requires > > > > import package, so be it. If not then not. > > > > > > Exactly. > > > > > > I think we can all agree that, at minimum, interoperation means that > > > classes and resources are sharable *across* ModuleSystems at runtime. > > > > > > Which implies that *import dependencies must be resolvable across > > > multiple ModuleSystem instances*. (BTW, I think we should change the > > > state name "PREPARING" to "RESOLVING" in 7.2.1.) > > > > Agreed. We must avoid the trap of thinking that module system interop. > > can be achieved by exposing class loaders (as loadClass will happily > > load unexported classes). > > > > > > > > So the open issue is the richness of the import "language": must we > > > support only lowest-common-denominator, or can we do better without > > > over-complicating the design? > > > > > > I for one would like to be able to have a single module express > > > dependencies on modules from both the same and different ModuleSystems, > > > *using the standard semantics of each*. This may be reaching too far, > > > but we should at least explore it seriously while we figure out what > > > interop means here... > > > > At this point, I feel that is likely to be reaching too far, but I'm > > happy to play along and see what we can learn along the way. > > > > > > > > > > > BASICS > > > > > > So far, we only know of two different import semantics: module-name, and > > > package-name. For discussion, let's call these: > > > > > > a. import-module > > > b. import-package > > > > > > So, to start, we could: > > > > > > 1. Support declaration of both import types. If 294 supports imports at > > > all, it should be relatively easy to support both, since a superpackage > > > name is a module name, and it contains member package names. (Compiler > > > support is clearly the critical issue here, but it will obviously > > > require use of the 277 runtime, so the import *type* should be > > > transparent to it.) At worst, we'd need two new annotation types. > > > > A superpackage name is a deployment module name in the JSR 277 model of > > one superpackage per deployment module, but I don't see any reason why a > > JSR 291 deployment module should not contain more than one superpackage. > > So if 294 were to support import, then its import-module would really be > > a superpackage import rather than a development module import. > > If we end up with nested superpackages, might it make sense to model > multiple superpackages by enclosing them in a single top-level one? That is an option, but of course each nested superpackage has to name its parent, so it wouldn't be possible to combine superpackages from independent groups or sources without either modifying their superpackage declarations or getting them to agree on the name of the parent superpackage and code it themselves. > > > > > > > > > 2. Provide API for both import types (e.g. ImportDependency has > > > getModuleName() and getPackageName() methods, one of which will return > > > null on a given instance). > > > > > > However, we know these are necessary but not sufficient. Leaving aside > > > the resolution issue for a moment, support for import-package also > > > suggests that we: > > > > > > 3. Enable a single module to declare different versions for each of its > > > member packages. > > > > > > 4. Enable efficient Repository lookup by package name. > > > > > > I think these are relatively easy (but *could* be considered optional). > > > > > > We also need: > > > > > > 5. Standard Query types for lookup by module and package name. > > > > > > > > > EXISTING DEPENDENCY RESOLUTION MODEL > > > > > > The more interesting issue is dependency resolution. But this hasn't > > > been discussed in any real detail, so lets do so before talking further > > > about import-package. To simplify this discussion, I'm ignoring > > > bundled/custom import policies for now... > > > > > > Resolution in the current spec is delegated to the associated > > > ModuleSystem instance (7.2.2 #8). While the details are not spelled out, > > > the expectation appears to be that > > > ModuleSystem.getModule(ModuleDefinition) must: > > > > > > - Select an initial repository. Call getRepository() on the input > > parameter. > > > > > > Then, for each ImportDependency of the definition: > > > > > > - Select a matching definition. Construct a Query from the > > > ImportDependency and use Repository.find() to lookup a matching > > > ModuleDefinition. > > > > > > - Get an instance. Use def.getModuleSystem().getModule(def). The > > > ModuleSystem is expected to return a cached instance if available, or > > > create/cache/return one if not. > > > > I think there also needs to be some 'resolution context' object which > > explicitly denotes a particular resolution so that each module system > > can keep track of the state of a resolution. This is required when two > > or more imports of a given module from another module system need to > > resolve to the same module instance. A resolution context may also be > > needed for back-tracking when a set of module instances created earlier > > in resolution turn out not to satisfy all the necessary constraints. > > Yes. I had (briefly :^) thought that we need only a List as > field in the ImportResolver to hold the resolved modules as we go. But > probably we need to keep some additional state along with each Module, > so a new type may be required. Let's call it ResolutionContext, and > create one at the start of each resolution: > > public abstract class ImportResolver { > > public Module resolve(ModuleDefinition def, ModuleContext modCtx) { > ResolutionContext resCtx = createResolutionContext(); > return resolve(def, modCtx, resCtx); > } > > protected abstract ResolutionContext createResolutionContext(); > > protected abstract Module resolve(ModuleDefinition def, > ModuleContext modCtx, > ResolutionContext resCtx); > } > > Does this approach make sense? Any thoughts on details for > ResolutionContext? Seems reasonable. I think ResolutionContext doesn't need any detail - it can simply be used as the key of a map recording the resolutions that a module system is currently aware of. > > > > > > > > > > > > (TBD: The PlatformBinding must be taken into account somehow during > > > selection. ModuleDefinition must include an accessor for it, and either > > > Repository.find() should implicitly filter them, or the caller must > > > construct a Query which will do so. I think we should add a > > > CURRENT_PLATFORM constant to Query, which will evaluate to true if no > > > binding is present in a definition.) > > > > > > (The spec also talks about Repository as the mechanism of isolation > > > (6.4). This was the case in the prototype, where the repository itself > > > provided caching. It doesn't appear to work with the current design. > > > There is no need that I can see to isolate ModuleDefinition > > > instances--it is Module instances with their associated loaders that may > > > require isolation.) > > > > > > (Also note that if ImportDependency was itself a Query subclass, there > > > would be no need to do any mapping. And since the ModuleDefinition > > > subclass must produce ImportDependency instances, it can even produce > > > more specialized Query instances if desired.) > > > > > > > > > REFINEMENT > > > > > > I think we can improve on the existing model in several ways: > > > > > > A. Provide a model for Module isolation (e.g. for EE, Applets, etc). > > > > > > B. Encapsulate all selection logic in a single mechanism. > > > > > > C. Eliminate the overhead of the repository lookup when a cached > > > instance exists. > > > > > > Let me propose a new class that encapsulates the caching logic, enables > > > lookup using Query, and supports multiple instances for isolation: > > > > > > public abstract class ModuleContext { > > > > > > // Get the context used to define JRE modules. > > > public static ModuleContext getBootstrapContext(){...}; > > > > > > // Get the context used to define the main module. > > > public static ModuleContext getSystemContext(){...}; > > > > > > // Get all contexts. > > > public static List getContexts() {...}; > > > > > > // Add a new context. > > > public static void addContext(ModuleContext ctx) {...} > > > > > > // Remove a context (error if == default). > > > public static boolean removeContext(ModuleContext ctx) {...} > > > > > > // Get the parent context (null if bootstrap). > > > public ModuleContext getParentContext(){...} > > > > > > // Get the name of this context. > > > public String getContextName() {...} > > > > > > // Create a new Module instance and store it in the cache. > > > public abstract Module createModule(ModuleDefinition def); > > > > > > // Find cached Module instances. Must check parent first. > > > public abstract List findModules(Query query); > > > > > > // Set the context used for JRE modules. > > > static void setBootstrapContext(ModuleContext ctx){...} > > > > > > // Set the context used to define the main module. > > > static void setSystemContext(ModuleContext ctx){...} > > > } > > > > > > The JVM will create an appropriate subtype and call > > > setBootstrapContext(). The launcher will create a child context and call > > > setSystemContext(). An EE (or similar) environment can create/remove new > > > contexts as needed for application isolation. > > > > > > And the resolution algorithm can now check the cache *first*, before > > > doing a repository lookup, using the same mechanism in both. Query > > > should be used to express *all* selection criteria (including > > > attributes, which we have not yet directly supported). > > > > > > Caches are no longer tied to ModuleSystem instances. > > > ModuleSystem.getModule() can become simply createModule(). The normal > > > implementation of ModuleContext.createModule() just calls > > > ModuleSystem.createModule() and caches/returns the result. > > > > > > > > > This class could easily be made concrete, but it may be useful to > > > support subtypes for specialization (e.g. event generation, lifecycle > > > management, specialized diagnostics, etc). > > > > > > > > > RESOLUTION MODELS > > > > > > The current design requires that each ModuleSystem provide its own > > > resolution logic, and that each definition will be resolved by its > > > owning ModuleSystem. This model appears to provide flexibility for > > > significant differences in implementation, but we really don't know > > > enough at this point. Perhaps only an actual second implementation will > > > tell us if this provides useful flexibility. > > > > > > It wouldn't surprise me if we have to keep tightening the spec as we go, > > > in order to remove inconsistencies that arise from the separate > > > algorithms. And this may eliminate flexibility to the point where it is > > > no longer useful. Much worse, we may not even discover this until after > > > the spec and RI are released, if that second implementation (e.g. OSGi) > > > is not completed beforehand. > > > > > > We should at least consider the obvious alternative: one algorithm (to > > > rule them all :^). And I don't mean one hard-coded algorithm, I mean one > > > replaceable, extensible class, such as: > > > > > > public abstract class ImportResolver { > > > public abstract Module resolve(ModuleDefinition def); > > > } > > > > > > And we add a method to ModuleContext to retrieve an instance: > > > > > > public abstract class ModuleContext { > > > ... > > > public abstract ImportResolver getImportResolver(); > > > } > > > > > > (Note that ImportResolver is now in a position to subsume the > > > functionality of both VisibilityPolicy and ImportOverridePolicy.) > > > > > > Repository usage is encapsulated within the implementation of the > > > resolve() method. The full resolution algorithm becomes: > > > > > > context.getImportResolver().resolve(definition); > > > > > > The launcher uses the "system" context for this. EE, Applets, etc. make > > > and use their own distinct, isolated instances. > > > > > > > > > RESOLUTION WITH IMPORT-NAME AND IMPORT-PACKAGE > > > > > > With this scaffolding in place we can easily take a phased approach to > > > supporting import-package: the initial implementation simply does not > > > support it at runtime. > > > > A phased approach would be particularly beneficial if the initial phase > > could be delivered as part of Java 7 and subsequent phases implemented > > strictly on top of Java 7. But getting the API right up front might be > > tricky unless we can spot some really good abstractions or prototype the > > later phases sufficiently well. Is that the kind of phasing you had in > > mind? > > Yes. It would be *extremely* useful to have an OSGi > implementation/prototype well under way before Java 7 is completed, so > that we can fine tune the model as we learn. > > > > > > > > > A subsequent implementation may support import-package, but only within > > > the boundaries of the same ModuleSystem. > > > > > > And a full blown implementation may support import-package across > > > ModuleSystems. > > > > > > We can build in support for selecting/configuring the ImportResolver as > > > a service, just as we plan to do with ModuleSystem (and Repository, I > > > presume). > > > > > > > > > And maybe, just maybe, we can find a way to abstract and re-use the > > > mature resolution logic from the OSGi reference implementation *as* the > > > one implementation to rule them all. > > > > > > // Bryan > > > > Glyn > > > > > > ------------------------------------------------------------------------ > > > > / > > / > > > > /Unless stated otherwise above: > > IBM United Kingdom Limited - Registered in England and Wales with number > > 741598. > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU/ > > > > > > > > > > > > Glyn Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/attachments/20070531/986d2a51/attachment.html From bryan.atsatt at oracle.com Thu May 31 11:01:48 2007 From: bryan.atsatt at oracle.com (Bryan Atsatt) Date: Thu, 31 May 2007 11:01:48 -0700 Subject: Relationship to JSR 291 [was: Re: Bryan's comments] In-Reply-To: <465EBE00.2060306@ungoverned.org> References: <465DF2E4.4090603@sun.com> <465E2051.7020008@oracle.com> <465EBE00.2060306@ungoverned.org> Message-ID: <465F0D8C.5010606@oracle.com> Richard S. Hall wrote: > Bryan Atsatt wrote: >> Stanley M. Ho wrote: >>> Glyn Normington wrote: >>>> >>>> *Bryan Atsatt * wrote on 30/05/2007 07:57:59: >>>> ... >>>> > So the open issue is the richness of the import "language": must we >>>> > support only lowest-common-denominator, or can we do better without >>>> > over-complicating the design? >>>> > >>>> > I for one would like to be able to have a single module express >>>> > dependencies on modules from both the same and different >>>> ModuleSystems, >>>> > *using the standard semantics of each*. This may be reaching too >>>> far, >>>> > but we should at least explore it seriously while we figure out what >>>> > interop means here... >>>> >>>> At this point, I feel that is likely to be reaching too far, but I'm >>>> happy to play along and see what we can learn along the way. >>> >>> I agreed with Glyn that this might be reaching too far. Before we dive >>> too much into how to implement interoperability, I think one of the >>> outstanding questions we should answer first is what degree of import >>> interoperability we want to offer. There are four possibilities: >>> >>> 1. JSR 277 module imports OSGi module by module name >>> 2. OSGi module imports JSR 277 module by module name >>> 3. OSGi module imports JSR 277 module by package name >>> 4. JSR 277 module imports OSGi module by package name >>> >>> Let's ignore the 294 issues and the module initialization issues for now >>> to simplify this discussion. >>> >>> I think we all agreed #1 and #2 are important to support and the >>> reflective APIs already enable these (of course, there are minor issues >>> we still have to address as we evolve the APIs.) >>> >>> That said, it is unclear to me how important it is to support #3 and #4, >>> so I think the first question for this EG is whether we want to support >>> #3 and #4 at all. However, let's pretend #3 and #4 are important for >>> this specific discussion. >>> >>> I think #3 is already possible since the OSGi framework could look up >>> the appropriate module which exported the package in the repository, >>> using the reflective APIs. Afterwards, the OSGi framework can then do >>> the necessary wiring. Yes, the reflective APIs can be further refined to >>> make it easier for the OSGi framework to query modules by exported >>> package name from the repository, but this is a minor issue that we can >>> address easily. >>> >>> #4 is a bit complicated, because there is no import-by-package semantic >>> in the current module layer's APIs and I think we all agreed that we >>> don't want to support this semantic directly in the module system >>> defined by JSR 277. >> >> Not me. >> >> I do understand your point that this is more difficult, but I don't >> really care--it is our job to do the Right Thing here. Sure, if it is >> not feasible in the time frame we have, then so be it. But we haven't >> even tried! >> >> If we're forced to make a choice between the two models, I'd pick >> import-by-package with zero hesitation. >> >> OSGi has a lot of experience here, and we shouldn't ignore it. >> Import-by-name was added late in the game, and is now seriously >> downplayed, for good reason. > > Just for information, import-by-name (or module dependencies) was not > added to OSGi R4 because they were thought to be widely needed or > useful, adding them was always somewhat contentious which is why the R4 > spec contained a section saying why you shouldn't use them (this section > has been further expanded in R4.1). The main motivation for adding them > was certainly legacy situation and for tightly coupled sub-systems. > > All in all, I agree with everything that Bryan is saying and I argued > these same points early on; however, I was under the impression that we > had long since lost that battle. > > Bryan, it is unclear to me whether your view is that we need to make > sure that 277 supports other modules systems the use import-by-package > or if you think that 277 should somehow directly support > import-by-package (i.e., explicitly expose such concepts). I want 277 to explicitly expose the concept of import-by-package through appropriate APIs. It would be very nice if the 277ModuleSystem implementation itself directly supports import-by-package (case #3). But this is less critical to me than case #4... I want a 277ModuleDefinition to be able to import-by-package an OSGiModuleDefinition (i.e. a bundle exposed via the 277 APIs). I believe this case will be a particularly important one. By the time SE 7 ships, there will be a very large number of OSGi bundles in the world, and near zero 277 modules. If 277 modules want to have a chance to survive in that world, they will *need* to rely on OSGi bundles. Could they rely only on import-by-name? Sure. But I believe natural selection would eventually result in the extinction of 277 modules, in favor of systems that are less brittle. I actually like the development/deployment/API model in 277, and want to use it. At the same time, I am acutely aware that OSGi is being widely adopted, at an accelerating rate. So I believe that it is incumbent on this JSR to support OSGi in a manner that is natural to it. // Bryan > > -> richard > >> >> And Oracle has a lot of relevant experience as well, since the shared >> loader mechanism I created for our AS stack ("shared libraries") >> supports only the import-by-name model. We have *lots* of >> shared-libraries (I've seen 100+ running simultaneously), used by many >> different components in the stack, in addition to customer applications. >> It is a tremendous pain when refactoring or even simple renaming has to >> be done (I hope to shift this implementation onto 277 to solve this >> problem, rather than inventing an interim approach). And this kind of >> change happens far more frequently than you probably expect. >> >> I don't want to repeat the same mistake here. Nor do I think that >> leaving a long trail of extraneous "view" modules behind for >> compatibility is a good solution. >> >> Worse still, if the 277 APIs don't support import-by-package, then the >> compiler won't support it either. If the compiler doesn't support it, >> then we've lost a golden opportunity to move to a better world, one in >> which packaging is irrelevant. >> >> We're supposed to be eliminating "jar-hell" here. Let's not simply >> replace it with "module-hell". >> >>> In this context, the question is really about what >>> it would take to make #4 possible if we want to support it, and so far >>> there are two different approaches we have discussed: >>> >>> a. Treat import-by-package separately from import-by-module in the >>> reflective APIs and make the module layer fully aware of the >>> import-by-package concept. >>> >>> b. Expose exported-package using the ModuleDefinition abstraction, and >>> provide necessary hooks (e.g. consistency checking) for the module >>> system to resolve this kind of dependency appropriately. The module >>> layer is not aware of the import-by-package concept at all. New import >>> dependency granularity could also be introduced in a similar manner in >>> the future without requiring significant changes in the module layer. >>> >>> >>> As Richard and I discussed in previous emails, it seems to make the most >>> sense for JSR 277 to define the minimal set of features possible in the >>> module layer to accomplish what needs to be accomplished to provide core >>> modularity support in the Java platform. I think (b) follows this >>> principle nicely while (a) does not. If we decide to support #3 and #4, >>> then how we want to address #4 would impact our overall design >>> significantly. >>> >>> Is there anyone in the EG provide good reasons why #3 and #4 are >>> important to be supported? If so, is there anyone in the EG think (b) is >>> not feasible at all and we should do (a) instead to support #4? Can you >>> explain your rationale? >> >> Yes, and yes. See above for rationale (ease-of-refactoring and compiler >> support). >> >>> >>> >>>> > RESOLUTION MODELS >>>> > >>>> > The current design requires that each ModuleSystem provide its own >>>> > resolution logic, and that each definition will be resolved by its >>>> > owning ModuleSystem. This model appears to provide flexibility for >>>> > significant differences in implementation, but we really don't know >>>> > enough at this point. Perhaps only an actual second implementation >>>> will >>>> > tell us if this provides useful flexibility. >>> >>> In the existing module systems we are aware, e.g. OSGi and NetBeans, we >>> already know their resolution logics are different from each other. Even >>> if they are migrated/re-based to JSR 277 in the future, I think we >>> should expect their resolution logic would remain the same to maintain >>> backward compatibility with their existing modules/bundles. In other >>> words, their resolution logic would still be different from each other. >> >> And this is very likely to be the source of subtle but very painful >> interop issues. Which is why I also suggested that we should consider >> moving to a canonical, though evolvable, resolution mechanism. >> >>> >>>> A phased approach would be particularly beneficial if the initial phase >>>> could be delivered as part of Java 7 and subsequent phases implemented >>>> strictly on top of Java 7. But getting the API right up front might be >>>> tricky unless we can spot some really good abstractions or prototype >>>> the >>>> later phases sufficiently well. Is that the kind of phasing you had in >>>> mind? >>> >>> Once an API is put into the SE platform, it will stick around almost >>> forever. I think if we go with the phased approach on interoperability >>> support (I believe this is what Bryan suggested), it would be much >>> better to simply left out those features/APIs which do not have >>> implementations to back them up from JDK 7, rather than putting the >>> half-baked features/APIs into the release and regret later. ;) We can >>> always add new APIs in subsequence releases if we decide to offer better >>> degree of interoperability in the future. >> >> I couldn't agree more that we don't want to put half-baked APIs into SE! >> If we cannot expect significant validation of our interoperation model >> (whatever we settle on) from NetBeans and OSGi, then we shouldn't expose >> these APIs at all. Period. >> >> But this doesn't just apply to import-by-package. We haven't even >> defined how class and resource sharing will occur. I won't be a bit >> surprised if we want one or more new types here. >> >> >> And, last thoughts for the day... >> >> Even if you are adamantly opposed to import-by-package, please take a >> good look at the other ideas buried in my message from yesterday, >> particularly the ModuleContext proposal. >> >> It is completely independent of import-by-package, and seems to me at >> least to be a concise, flexible model for caching/isolating/finding >> Module instances. (The idea of exposing a resolve() method on it also >> makes a clean API, even if it delegates to ModuleSystem instead of a new >> resolver type.) >> >> There are other potentially interesting tidbits there as well >> (PlatformBinding exposure on definition, Query support for current >> platform, ImportDependency *as* Query). >> >> I'd hate for the baby to be thrown out with the bath water :^) >> >> // Bryan >> >>> >>> - Stanley >>> > From glyn_normington at UK.IBM.COM Thu May 31 05:03:37 2007 From: glyn_normington at UK.IBM.COM (Glyn Normington) Date: Thu, 31 May 2007 13:03:37 +0100 Subject: Exported resources In-Reply-To: <465DDCD0.8090703@oracle.com> Message-ID: Some feedback from an observer which may help: > Bryan misses the need for extender model to load internal impl classes ala > Bundle.loadClass in OSGi. > > You do not want to have to export the service impl class from a module. > You want to hide the class from others' casual loading. However an > extender bundle (e.g. ServiceLoader) will need to be able to load that > class to make it instances of it available to others under the service > interface class. Glyn Bryan Atsatt wrote on 30/05/2007 21:21:36: > I've been assuming that Module private resources should not be visible > to *any* class outside of the module. Including ResourceBundle, or any > other existing framework classes that do resource lookups (e.g. > ServiceLoader, JSF, etc). If resources need to be visible to these > existing classes, they must be exported. The very simple check I > proposed (immediate caller) is sufficient to make this assertion. > > I believe your point is that if we used the permission model instead, it > would become possible for a module to invoke an external class (e.g. > ResourceBundle.getBundle()) and enable *it* to successfully load a > private resource from the module. > > Aside from the permission *grant* mechanism this model would rely on, it > is an entirely different model than that used for classes! (Though we > haven't explicitly defined this in 294, it seems extremely unlikely that > we will rely on permissions--none of the other access modes do so.) Such > asymmetry is very disconcerting to me, and, I believe, just plain wrong... > > Consider that you could grant the ServiceLoader, for example, access to > a resource that names a class that it could not instantiate. That class > would have to be exported. I believe the resource should be as well. > > // Bryan > > > > > Stanley M. Ho wrote: > > Hi Bryan, > > > > Those resource-related methods in ClassLoader can be called by anyone, > > including code that is part of the module, code that is from other > > modules, or code that is part of the platform libraries (e.g. > > ResourceBundle). The approach you described would require walking the > > stack to get the caller's Module, but the real issue is that it is > > difficult to determine who the actual caller is from the stack. > > > > Treating the immediate caller on the stack as the actual caller wouldn't > > be sufficient because the immediate caller could be called by someone > > else who is the one actually making the call. On the other hand, > > treating the originated caller on the stack as the actual caller would > > be the right semantic, but this is basically the same as the security > > permission approach. > > > > - Stanley > > > > > > Bryan Atsatt wrote: > >> Both solutions require stack walking (unless there is some new > >> implementation of the java security model I've not yet seen!). > >> > >> The permission check does much more work than is necessary here. Take a > >> look at AccessController.checkPermission() to see what I mean. > >> > >> And actually there is a very simple API to get the stack, which I've > >> used for years: > >> > >> private static class StackAccessor extends SecurityManager { > >> public Class[] getStack() { > >> return getClassContext(); > >> } > >> } > >> > >> private static final STACK_ACCESSOR = new StackAccessor(); > >> > >> Now the enclosing class can simply call STACK_ACCESSOR.getStack(). > >> > >> // Bryan > >> > >> > >> > >> Stanley M. Ho wrote: > >>> Hi Bryan, > >>> > >>> Bryan Atsatt wrote: > >>>> 1. Definitely agree that resource search order should be identical to > >>>> class search order. > >>> > >>> Glad to hear! > >>> > >>>> 2. Using permissions to limit access to private resources seems like > >>>> overkill to me. The prototype implemented this in a very simple > >>>> fashion: > >>>> > >>>> a. If resource is exported, return it, else > >>>> a. Get the caller's Module (get class from stack, get module from it) > >>>> b. If callerModule == this, return resource, else return null. > >>> > >>> The issue is that this approach still requires stack walking and there > >>> is no public API in the SE platform that let you implement this. > >>> > >>> If stack walking is required for the check anyway, I think the security > >>> permission approach is better that it is implementable with the existing > >>> API in the SE platform. > >>> > >>> - Stanley > >>> > > Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/attachments/20070531/0f8f8542/attachment.html From bryan.atsatt at ORACLE.COM Thu May 31 10:17:09 2007 From: bryan.atsatt at ORACLE.COM (Bryan Atsatt) Date: Thu, 31 May 2007 10:17:09 -0700 Subject: Exported resources In-Reply-To: References: Message-ID: <465F0315.6040300@oracle.com> I don't think I'm missing anything, just looking at it from perhaps a different perspective :^) First, 294 will determine accessibility for non-exported classes. The assumption at the moment is that they will not be accessible to *any* class outside of the module. I do understand the value of a "friend" semantic; it would certainly be nice to grant certain frameworks special access. But, so far, that is not on the table for classes. And if we don't have it for classes, I can't see why we should have it for resources. // Bryan Glyn Normington wrote: > > Some feedback from an observer which may help: > > > Bryan misses the need for extender model to load internal impl > classes ala > > Bundle.loadClass in OSGi. > > > > You do not want to have to export the service impl class from a module. > > You want to hide the class from others' casual loading. However an > > extender bundle (e.g. ServiceLoader) will need to be able to load that > > class to make it instances of it available to others under the service > > interface class. > > Glyn > > *Bryan Atsatt * wrote on 30/05/2007 21:21:36: > > > I've been assuming that Module private resources should not be visible > > to *any* class outside of the module. Including ResourceBundle, or any > > other existing framework classes that do resource lookups (e.g. > > ServiceLoader, JSF, etc). If resources need to be visible to these > > existing classes, they must be exported. The very simple check I > > proposed (immediate caller) is sufficient to make this assertion. > > > > I believe your point is that if we used the permission model instead, it > > would become possible for a module to invoke an external class (e.g. > > ResourceBundle.getBundle()) and enable *it* to successfully load a > > private resource from the module. > > > > Aside from the permission *grant* mechanism this model would rely on, it > > is an entirely different model than that used for classes! (Though we > > haven't explicitly defined this in 294, it seems extremely unlikely that > > we will rely on permissions--none of the other access modes do so.) Such > > asymmetry is very disconcerting to me, and, I believe, just plain > wrong... > > > > Consider that you could grant the ServiceLoader, for example, access to > > a resource that names a class that it could not instantiate. That class > > would have to be exported. I believe the resource should be as well. > > > > // Bryan > > > > > > > > > > Stanley M. Ho wrote: > > > Hi Bryan, > > > > > > Those resource-related methods in ClassLoader can be called by anyone, > > > including code that is part of the module, code that is from other > > > modules, or code that is part of the platform libraries (e.g. > > > ResourceBundle). The approach you described would require walking the > > > stack to get the caller's Module, but the real issue is that it is > > > difficult to determine who the actual caller is from the stack. > > > > > > Treating the immediate caller on the stack as the actual caller > wouldn't > > > be sufficient because the immediate caller could be called by someone > > > else who is the one actually making the call. On the other hand, > > > treating the originated caller on the stack as the actual caller would > > > be the right semantic, but this is basically the same as the security > > > permission approach. > > > > > > - Stanley > > > > > > > > > Bryan Atsatt wrote: > > >> Both solutions require stack walking (unless there is some new > > >> implementation of the java security model I've not yet seen!). > > >> > > >> The permission check does much more work than is necessary here. > Take a > > >> look at AccessController.checkPermission() to see what I mean. > > >> > > >> And actually there is a very simple API to get the stack, which I've > > >> used for years: > > >> > > >> private static class StackAccessor extends SecurityManager { > > >> public Class[] getStack() { > > >> return getClassContext(); > > >> } > > >> } > > >> > > >> private static final STACK_ACCESSOR = new StackAccessor(); > > >> > > >> Now the enclosing class can simply call STACK_ACCESSOR.getStack(). > > >> > > >> // Bryan > > >> > > >> > > >> > > >> Stanley M. Ho wrote: > > >>> Hi Bryan, > > >>> > > >>> Bryan Atsatt wrote: > > >>>> 1. Definitely agree that resource search order should be > identical to > > >>>> class search order. > > >>> > > >>> Glad to hear! > > >>> > > >>>> 2. Using permissions to limit access to private resources seems like > > >>>> overkill to me. The prototype implemented this in a very simple > > >>>> fashion: > > >>>> > > >>>> a. If resource is exported, return it, else > > >>>> a. Get the caller's Module (get class from stack, get module > from it) > > >>>> b. If callerModule == this, return resource, else return null. > > >>> > > >>> The issue is that this approach still requires stack walking and > there > > >>> is no public API in the SE platform that let you implement this. > > >>> > > >>> If stack walking is required for the check anyway, I think the > security > > >>> permission approach is better that it is implementable with the > existing > > >>> API in the SE platform. > > >>> > > >>> - Stanley > > >>> > > > > > > > ------------------------------------------------------------------------ > > / > / > > /Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU/ > > > > > > From heavy at UNGOVERNED.ORG Thu May 31 05:22:24 2007 From: heavy at UNGOVERNED.ORG (Richard S. Hall) Date: Thu, 31 May 2007 08:22:24 -0400 Subject: Relationship to JSR 291 [was: Re: Bryan's comments] In-Reply-To: <465E2051.7020008@oracle.com> References: <465DF2E4.4090603@sun.com> <465E2051.7020008@oracle.com> Message-ID: <465EBE00.2060306@ungoverned.org> Bryan Atsatt wrote: > Stanley M. Ho wrote: >> Glyn Normington wrote: >>> >>> *Bryan Atsatt * wrote on 30/05/2007 07:57:59: >>> ... >>> > So the open issue is the richness of the import "language": must we >>> > support only lowest-common-denominator, or can we do better without >>> > over-complicating the design? >>> > >>> > I for one would like to be able to have a single module express >>> > dependencies on modules from both the same and different >>> ModuleSystems, >>> > *using the standard semantics of each*. This may be reaching too >>> far, >>> > but we should at least explore it seriously while we figure out what >>> > interop means here... >>> >>> At this point, I feel that is likely to be reaching too far, but I'm >>> happy to play along and see what we can learn along the way. >> >> I agreed with Glyn that this might be reaching too far. Before we dive >> too much into how to implement interoperability, I think one of the >> outstanding questions we should answer first is what degree of import >> interoperability we want to offer. There are four possibilities: >> >> 1. JSR 277 module imports OSGi module by module name >> 2. OSGi module imports JSR 277 module by module name >> 3. OSGi module imports JSR 277 module by package name >> 4. JSR 277 module imports OSGi module by package name >> >> Let's ignore the 294 issues and the module initialization issues for now >> to simplify this discussion. >> >> I think we all agreed #1 and #2 are important to support and the >> reflective APIs already enable these (of course, there are minor issues >> we still have to address as we evolve the APIs.) >> >> That said, it is unclear to me how important it is to support #3 and #4, >> so I think the first question for this EG is whether we want to support >> #3 and #4 at all. However, let's pretend #3 and #4 are important for >> this specific discussion. >> >> I think #3 is already possible since the OSGi framework could look up >> the appropriate module which exported the package in the repository, >> using the reflective APIs. Afterwards, the OSGi framework can then do >> the necessary wiring. Yes, the reflective APIs can be further refined to >> make it easier for the OSGi framework to query modules by exported >> package name from the repository, but this is a minor issue that we can >> address easily. >> >> #4 is a bit complicated, because there is no import-by-package semantic >> in the current module layer's APIs and I think we all agreed that we >> don't want to support this semantic directly in the module system >> defined by JSR 277. > > Not me. > > I do understand your point that this is more difficult, but I don't > really care--it is our job to do the Right Thing here. Sure, if it is > not feasible in the time frame we have, then so be it. But we haven't > even tried! > > If we're forced to make a choice between the two models, I'd pick > import-by-package with zero hesitation. > > OSGi has a lot of experience here, and we shouldn't ignore it. > Import-by-name was added late in the game, and is now seriously > downplayed, for good reason. Just for information, import-by-name (or module dependencies) was not added to OSGi R4 because they were thought to be widely needed or useful, adding them was always somewhat contentious which is why the R4 spec contained a section saying why you shouldn't use them (this section has been further expanded in R4.1). The main motivation for adding them was certainly legacy situation and for tightly coupled sub-systems. All in all, I agree with everything that Bryan is saying and I argued these same points early on; however, I was under the impression that we had long since lost that battle. Bryan, it is unclear to me whether your view is that we need to make sure that 277 supports other modules systems the use import-by-package or if you think that 277 should somehow directly support import-by-package (i.e., explicitly expose such concepts). -> richard > > And Oracle has a lot of relevant experience as well, since the shared > loader mechanism I created for our AS stack ("shared libraries") > supports only the import-by-name model. We have *lots* of > shared-libraries (I've seen 100+ running simultaneously), used by many > different components in the stack, in addition to customer applications. > It is a tremendous pain when refactoring or even simple renaming has to > be done (I hope to shift this implementation onto 277 to solve this > problem, rather than inventing an interim approach). And this kind of > change happens far more frequently than you probably expect. > > I don't want to repeat the same mistake here. Nor do I think that > leaving a long trail of extraneous "view" modules behind for > compatibility is a good solution. > > Worse still, if the 277 APIs don't support import-by-package, then the > compiler won't support it either. If the compiler doesn't support it, > then we've lost a golden opportunity to move to a better world, one in > which packaging is irrelevant. > > We're supposed to be eliminating "jar-hell" here. Let's not simply > replace it with "module-hell". > >> In this context, the question is really about what >> it would take to make #4 possible if we want to support it, and so far >> there are two different approaches we have discussed: >> >> a. Treat import-by-package separately from import-by-module in the >> reflective APIs and make the module layer fully aware of the >> import-by-package concept. >> >> b. Expose exported-package using the ModuleDefinition abstraction, and >> provide necessary hooks (e.g. consistency checking) for the module >> system to resolve this kind of dependency appropriately. The module >> layer is not aware of the import-by-package concept at all. New import >> dependency granularity could also be introduced in a similar manner in >> the future without requiring significant changes in the module layer. >> >> >> As Richard and I discussed in previous emails, it seems to make the most >> sense for JSR 277 to define the minimal set of features possible in the >> module layer to accomplish what needs to be accomplished to provide core >> modularity support in the Java platform. I think (b) follows this >> principle nicely while (a) does not. If we decide to support #3 and #4, >> then how we want to address #4 would impact our overall design >> significantly. >> >> Is there anyone in the EG provide good reasons why #3 and #4 are >> important to be supported? If so, is there anyone in the EG think (b) is >> not feasible at all and we should do (a) instead to support #4? Can you >> explain your rationale? > > Yes, and yes. See above for rationale (ease-of-refactoring and compiler > support). > >> >> >>> > RESOLUTION MODELS >>> > >>> > The current design requires that each ModuleSystem provide its own >>> > resolution logic, and that each definition will be resolved by its >>> > owning ModuleSystem. This model appears to provide flexibility for >>> > significant differences in implementation, but we really don't know >>> > enough at this point. Perhaps only an actual second implementation >>> will >>> > tell us if this provides useful flexibility. >> >> In the existing module systems we are aware, e.g. OSGi and NetBeans, we >> already know their resolution logics are different from each other. Even >> if they are migrated/re-based to JSR 277 in the future, I think we >> should expect their resolution logic would remain the same to maintain >> backward compatibility with their existing modules/bundles. In other >> words, their resolution logic would still be different from each other. > > And this is very likely to be the source of subtle but very painful > interop issues. Which is why I also suggested that we should consider > moving to a canonical, though evolvable, resolution mechanism. > >> >>> A phased approach would be particularly beneficial if the initial phase >>> could be delivered as part of Java 7 and subsequent phases implemented >>> strictly on top of Java 7. But getting the API right up front might be >>> tricky unless we can spot some really good abstractions or prototype >>> the >>> later phases sufficiently well. Is that the kind of phasing you had in >>> mind? >> >> Once an API is put into the SE platform, it will stick around almost >> forever. I think if we go with the phased approach on interoperability >> support (I believe this is what Bryan suggested), it would be much >> better to simply left out those features/APIs which do not have >> implementations to back them up from JDK 7, rather than putting the >> half-baked features/APIs into the release and regret later. ;) We can >> always add new APIs in subsequence releases if we decide to offer better >> degree of interoperability in the future. > > I couldn't agree more that we don't want to put half-baked APIs into SE! > If we cannot expect significant validation of our interoperation model > (whatever we settle on) from NetBeans and OSGi, then we shouldn't expose > these APIs at all. Period. > > But this doesn't just apply to import-by-package. We haven't even > defined how class and resource sharing will occur. I won't be a bit > surprised if we want one or more new types here. > > > And, last thoughts for the day... > > Even if you are adamantly opposed to import-by-package, please take a > good look at the other ideas buried in my message from yesterday, > particularly the ModuleContext proposal. > > It is completely independent of import-by-package, and seems to me at > least to be a concise, flexible model for caching/isolating/finding > Module instances. (The idea of exposing a resolve() method on it also > makes a clean API, even if it delegates to ModuleSystem instead of a new > resolver type.) > > There are other potentially interesting tidbits there as well > (PlatformBinding exposure on definition, Query support for current > platform, ImportDependency *as* Query). > > I'd hate for the baby to be thrown out with the bath water :^) > > // Bryan > >> >> - Stanley >> From Stanley.Ho at sun.com Thu May 31 15:35:00 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Thu, 31 May 2007 15:35:00 -0700 Subject: Relationship to JSR 291 [was: Re: Bryan's comments] In-Reply-To: <9E52FB97-F312-4A4F-A096-ECA0430F9D7C@ungoverned.org> References: <465DF2E4.4090603@sun.com> <9E52FB97-F312-4A4F-A096-ECA0430F9D7C@ungoverned.org> Message-ID: <465F4D94.9060809@sun.com> Hi Richard, Richard S. Hall wrote: > ... > I am not so sure supporting #3 is as straight forward as you suggest > here. Specially, the OSGi modularity layer [for the most part] > assumes that packages are atomic units; there is no way to specify > that your export or import something smaller than a package. Since > 277 modules declare exports in terms of classes, it is not clear how > we would know if a 277 module was exporting an entire package. This > would, at a minimum, require that we assume we can inspect the > contents of the module and see that the contents of a contained > package matches the module's export signature for that package. My assumption for #3 is that if an OSGi bundle attempts to import package x.y.z and if the OSGi framework wants to resolve such dependency using 277 modules, it can search all the modules in the repositories to find out which module have exported classes that are in package x.y.z. After the appropriate 277 module has been identified, the OSGi framework can do whatever is needed to wire that 277 module. Obviously, searching modules in the repositories this way may not be very efficient, and it may make sense to have some convenience methods in this case to make it easier for querying modules by package name. I don't think I fully understand your concerns around exporting an entire package. As you mentioned, what's get exported in 277 module is in terms of classes and the information is in the module metadata, and I think we can probably determine the packages associated with these exported classes and treat these packages as "exported packages" from the OSGi perspective. There is no split package allowed in 277 module, so I think each of these exported packages would be an entire package. - Stanley From bryan.atsatt at oracle.com Thu May 31 15:52:08 2007 From: bryan.atsatt at oracle.com (Bryan Atsatt) Date: Thu, 31 May 2007 15:52:08 -0700 Subject: Module isolation Message-ID: <465F5198.2090805@oracle.com> I'd like to continue this discussion separately from the 291/import-by-package thread :^). To recap, the spec (sections 1.2, 6.1, 6.4) talks about using separate Repository instances as a means to provide isolation for environments like EE and Applets. I believe this is an oversight--actually, a holdover from the ideas I put forth in the prototype, where Repository cached Module instances. But in the current design, that is no longer the case. Repositories hold ModuleDefinitions, not Modules, so they are not a suitable isolation mechanism. It is *Module* instances with their associated loaders that may require isolation. The current design also assumes that ModuleSystems cache Module instances. This certainly makes more sense than Repository. However, it means that systems that require isolation must arrange for separate ModuleSystem instances. This seems quite awkward, and a poor abstraction. So I think we need to introduce an explicit notion of an isolation context that: 1. Acts as a cache for Module instances. 2. Enables multiple instances, each forming a separate Module namespace. 3. Provides out-of-the box instances for bootstrap and "system" modules. 4. Provides a delegation model to link instances. 5. Provides a lookup mechanism identical to that of Repository. Any given resolution would take place within a single context. That is, an initial context is selected, and while existing instances can be retrieved from delegated contexts, *new* instances must be created in the initial context. To enforce this model, and to simplify the api, I believe the context should directly expose a "resolve" convenience method (which simply delegates the actual implementation). Here is a strawman API, with a simple parent-first delegation model: public abstract class ModuleContext { // Get the context used to resolve JRE modules. public static ModuleContext getBootstrapContext(){...}; // Get the context used to resolve the main module. public static ModuleContext getSystemContext(){...}; // Get all contexts. public static List getContexts() {...}; // Add a new context. public static void addContext(ModuleContext ctx) {...} // Remove a context (error if == default). public static boolean removeContext(ModuleContext ctx) {...} // Get the parent context (null if bootstrap). public ModuleContext getParentContext(){...} // Get the name of this context. public String getContextName() {...} // Add a new Module instance to the cache. Throw an error // if an existing module contains the same definition. public abstract void addModule(Module module); // Find cached Module instances. Must check parent first. public abstract List findModules(Query query); // Remove a Module. public abstract boolean remove(Module module); // Remove all Modules created from the specified definition. public static void removeAll(ModuleDefinition def) {...} // Resolve a definition. public Module resolve(ModuleDefinition def) { ModuleSystem ms = def.getModuleSystem(); return ms.resolve(def, this); } // Set the context used for JRE modules. static void setBootstrapContext(ModuleContext ctx){...} // Set the context used to define the main module. static void setSystemContext(ModuleContext ctx){...} } The JVM would create an appropriate subtype and call setBootstrapContext(). JRE module instances would be cached here. The launcher would create a child context and call setSystemContext(). The main module and any of its dependencies would be resolved here. An EE/Applet (or similar) environment can create/remove/use new contexts as needed for application isolation. Note that the these new contexts need not have the system context in their parent chain. But they do need the bootstrap context so the code should enforce that. The resolve() method here assumes that ModuleSystem.getModule() is replaced by: ModuleSystem { ... public abstract Module resolve(ModuleDefinition def, ModuleContext ctx); } The ModuleSystem is no longer required to directly cache Modules, but to implement the resolution algorithm using the supplied context. ModuleSystem releaseModule() and disableModuleDefinition() methods just use remove() and removeAll(), respectively. This class could easily be made concrete, as the implementations are obviously simple. Regardless, for extensibility the actual types used at runtime should be under the control of the JVM/JRE and those systems that need to create new contexts. And, we might want to tie other existing ideas in here. For example, ModuleContext could be the home for "default" policies: public abstract ImportPolicy getImportPolicy(); public abstract ImportOverridePolicy getImportOverridePolicy(); Thoughts? // Bryan From heavy at UNGOVERNED.ORG Thu May 31 15:53:26 2007 From: heavy at UNGOVERNED.ORG (Richard S.Hall) Date: Thu, 31 May 2007 18:53:26 -0400 Subject: Relationship to JSR 291 [was: Re: Bryan's comments] In-Reply-To: <465F4D94.9060809@sun.com> References: <465DF2E4.4090603@sun.com> <9E52FB97-F312-4A4F-A096-ECA0430F9D7C@ungoverned.org> <465F4D94.9060809@sun.com> Message-ID: <0e79b722e610c3de839326fb68539389@ungoverned.org> Stanley, On May 31, 2007, at 6:35 PM, Stanley M. Ho wrote: > Richard S. Hall wrote: >> ... >> I am not so sure supporting #3 is as straight forward as you suggest >> here. Specially, the OSGi modularity layer [for the most part] >> assumes that packages are atomic units; there is no way to specify >> that your export or import something smaller than a package. Since >> 277 modules declare exports in terms of classes, it is not clear how >> we would know if a 277 module was exporting an entire package. This >> would, at a minimum, require that we assume we can inspect the >> contents of the module and see that the contents of a contained >> package matches the module's export signature for that package. > > My assumption for #3 is that if an OSGi bundle attempts to import > package x.y.z and if the OSGi framework wants to resolve such > dependency > using 277 modules, it can search all the modules in the repositories to > find out which module have exported classes that are in package x.y.z. > After the appropriate 277 module has been identified, the OSGi > framework > can do whatever is needed to wire that 277 module. Obviously, searching > modules in the repositories this way may not be very efficient, and it > may make sense to have some convenience methods in this case to make it > easier for querying modules by package name. > > I don't think I fully understand your concerns around exporting an > entire package. As you mentioned, what's get exported in 277 module is > in terms of classes and the information is in the module metadata, and > I > think we can probably determine the packages associated with these > exported classes and treat these packages as "exported packages" from > the OSGi perspective. There is no split package allowed in 277 module, > so I think each of these exported packages would be an entire package. The point is that an OSGi exported package FOO specifically means that all public classes in the FOO package are accessible to importers. Such an export is not the same as a 277 module that just happens to export a class in its export signature from the FOO package. Without cracking open the module, there is no way to tell if its export signature contains all public classes from the FOO package (i.e., the entire package). -> richard From Stanley.Ho at sun.com Thu May 31 16:59:19 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Thu, 31 May 2007 16:59:19 -0700 Subject: Relationship to JSR 291 [was: Re: Bryan's comments] In-Reply-To: <465F0D8C.5010606@oracle.com> References: <465DF2E4.4090603@sun.com> <465E2051.7020008@oracle.com> <465EBE00.2060306@ungoverned.org> <465F0D8C.5010606@oracle.com> Message-ID: <465F6157.7070706@sun.com> Hi Bryan, I don't think I will have time to response to all your comments this week, so my response would focus on the import-by-package discussion. First, the only import dependency we have consensus in this EG so far is import-by-module-name (or import-by-superpackage-name), and this was also what's in the EDR. That said, it is by no mean that we can't consider other import dependency granularities (e.g. import-by-package-name, or import-by-contract) in addition to import-by-module-name. I would like to focus this discussion more on whether it makes sense for JSR 277 to support additional import dependency granularity, mainly import-by-package-name, and if so, in what context. There are three cases we should consider: 1. 277 module imports another 277 module by package name I think our consensus so far is that this is nice-to-have or non-critical, given we already have support for import-by-module-name. To keep this discussion in focus, let's not dive into import-by-package-name v.s. import-by-module-name, and let's assume there is no support for #1 for now. 2. OSGi module imports 277 module by package name As I discussed this with Richard in previous emails, majority of the works for this fall under the OSGi framework if it wants to enable this kind of wiring, but we could probably make the implementor's life easier by provide the appropriate 277 reflective API if necessary. 3. 277 module imports OSGi module by package name If I understand you correctly, I think what you really want is the support for this case, so the existing OSGi bundles can be leveraged by 277 modules in a nature way. Let me say it up front that I don't oppose the idea of #3 in the context of interoperability. If #3 is what you really want, I think where we diverge is simply how to make it possible. Do you agree so far? Bryan Atsatt wrote: > ... > I want 277 to explicitly expose the concept of import-by-package through > appropriate APIs. > > It would be very nice if the 277ModuleSystem implementation itself > directly supports import-by-package (case #3). But this is less critical > to me than case #4... > > I want a 277ModuleDefinition to be able to import-by-package an > OSGiModuleDefinition (i.e. a bundle exposed via the 277 APIs). I believe > this case will be a particularly important one. By the time SE 7 ships, ... As I described several emails back: > This doesn't mean supporting the notion of import by package is > impossible. What OSGi will expose from its repository is simply module > definitions, and module definition is basically an abstraction. > Therefore, it should be possible to treat an OSGi bundle as a module > definition, as well as to treat each exported package in an OSGi > bundle a module definition as well. The dependency supported in JSR > 277 is import-by-module. In the JSR 277 module system, this is mapped > to import-by-superpackage. In the OSGi module system, this can be > mapped to import-by-OSGi-bundle or import-by-package. If other module > systems want to support different import granularity or if OSGi > evolves to add new import granularity in the future, this model should > still work. Rather than surfacing the import-by-package at the API level, I think we can solve #3 by generalizing the import dependency concept as import-by-name, and this name can be mapped to superpackage-name in JSR 277 and OSGi-bundle-name or OSGi-exported-package-name in OSGi (i.e. exposing OSGi bundles and exported packages through implementing OSGiBundleModuleDefinition and OSGiPackageModuleDefinition). I think this is also what you intended to say at one point (correct me if I misunderstand you) by suggesting to change ImportDependency.getModuleName() into something like ImportDependency.getName(). We expect the module-aware compiler would somehow recognize the import dependency defined in 277. In other words, if the OSGi framework exposes the module definitions appropriately, you'll get the compiler support. Since the import dependency is generic, the compiler and other tools ed to do extra work to support other forms of import dependency. In case if the 277 module would like to import these OSGi exported packages using an OSGi-like resolution algorithm, it should be possible for it to use a custom import policy (or import override policy or maybe some new override mechanisms) to get the behavior it wants. Do you think this is a workable approach to address #3? - Stanley From Stanley.Ho at sun.com Thu May 31 17:06:20 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Thu, 31 May 2007 17:06:20 -0700 Subject: Relationship to JSR 291 [was: Re: Bryan's comments] In-Reply-To: <0e79b722e610c3de839326fb68539389@ungoverned.org> References: <465DF2E4.4090603@sun.com> <9E52FB97-F312-4A4F-A096-ECA0430F9D7C@ungoverned.org> <465F4D94.9060809@sun.com> <0e79b722e610c3de839326fb68539389@ungoverned.org> Message-ID: <465F62FC.7070201@sun.com> Hi Richard, Richard S.Hall wrote: >> I don't think I fully understand your concerns around exporting an >> entire package. As you mentioned, what's get exported in 277 module is >> in terms of classes and the information is in the module metadata, and >> I >> think we can probably determine the packages associated with these >> exported classes and treat these packages as "exported packages" from >> the OSGi perspective. There is no split package allowed in 277 module, >> so I think each of these exported packages would be an entire package. > > The point is that an OSGi exported package FOO specifically means that > all public classes in the FOO package are accessible to importers. Such > an export is not the same as a 277 module that just happens to export a > class in its export signature from the FOO package. Without cracking > open the module, there is no way to tell if its export signature > contains all public classes from the FOO package (i.e., the entire > package). > > -> richard I think there is a misunderstanding here. Both JSR 277 and OSGi export all "accessible" classes in a package. JSR 294 changes what accessible means from "public" to "exported public", and this affects 277 and OSGi in the same way. I don't think there is a conflict here. - Stanley From heavy at UNGOVERNED.ORG Thu May 31 17:11:36 2007 From: heavy at UNGOVERNED.ORG (Richard S.Hall) Date: Thu, 31 May 2007 20:11:36 -0400 Subject: Relationship to JSR 291 [was: Re: Bryan's comments] In-Reply-To: <465F62FC.7070201@sun.com> References: <465DF2E4.4090603@sun.com> <9E52FB97-F312-4A4F-A096-ECA0430F9D7C@ungoverned.org> <465F4D94.9060809@sun.com> <0e79b722e610c3de839326fb68539389@ungoverned.org> <465F62FC.7070201@sun.com> Message-ID: <2071f55c84384fb5d141568f491677c9@ungoverned.org> On May 31, 2007, at 8:06 PM, Stanley M. Ho wrote: > Hi Richard, > > Richard S.Hall wrote: >>> I don't think I fully understand your concerns around exporting an >>> entire package. As you mentioned, what's get exported in 277 module >>> is >>> in terms of classes and the information is in the module metadata, >>> and >>> I >>> think we can probably determine the packages associated with these >>> exported classes and treat these packages as "exported packages" from >>> the OSGi perspective. There is no split package allowed in 277 >>> module, >>> so I think each of these exported packages would be an entire >>> package. >> >> The point is that an OSGi exported package FOO specifically means that >> all public classes in the FOO package are accessible to importers. >> Such >> an export is not the same as a 277 module that just happens to export >> a >> class in its export signature from the FOO package. Without cracking >> open the module, there is no way to tell if its export signature >> contains all public classes from the FOO package (i.e., the entire >> package). >> >> -> richard > > I think there is a misunderstanding here. Both JSR 277 and OSGi export > all "accessible" classes in a package. JSR 294 changes what accessible > means from "public" to "exported public", and this affects 277 and OSGi > in the same way. I don't think there is a conflict here. Perhaps so. I am operating off my [possibly dated] recollection that 277 modules explicitly list the classes that they export, which meant that they may expose arbitrary public classes from a package. If this is no longer the case, then this is fine. The point remains, all legacy OSGi dependencies on a package assume that all public classes are exported from that package. An OSGi dependency on a given package cannot be resolved to a 277 module whose "exported public" classes are not the same as all "public" classes in the specific package. -> richard From Stanley.Ho at sun.com Thu May 31 17:31:04 2007 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Thu, 31 May 2007 17:31:04 -0700 Subject: Exported resources In-Reply-To: <465E0201.2030308@oracle.com> References: <4654109D.40205@sun.com> <46548736.7020507@oracle.com> <465629EA.2050008@sun.com> <465658CE.5020409@oracle.com> <465DCE51.4010304@sun.com> <465DDCD0.8090703@oracle.com> <465DF6D6.5050608@sun.com> <465E0201.2030308@oracle.com> Message-ID: <465F68C8.4000103@sun.com> Hi Bryan, Bryan Atsatt wrote: > ... > No, I don't agree. That's what I said before :^). I *really* don't like > the idea that I'm going to have to explicitly grant permission to some > module to access my resource. This *significantly* complicates No, this was not what I suggested. If some other modules want to access your resource, your resources need to be exported. However, if you want to access your own private resource in your module (call ClassLoader.getResource*() directly or call ResourceBundle.getBundle(), etc.), then your code would require additional permissions. > deployment. How will the grant occur, especially given that each > environment can use an entirely different permission storage mechanism? > > Or are you thinking that the loaders in the module system will construct > ProtectionDomains pre-wired with permissions? If so, exactly which > entities will be blessed? Just the JRE code? What about all the existing > non-JRE frameworks in use? When your module instance is initialized, the module system can construct the ProtectionDomain with the appropriate permissions and associate that with the classes in your module, so your classes in your module will always have the necessary permissions to access your own resources. > Sure, resources have never had JVM access control applied to them. But I > don't think we should go to the other extreme (permissions) just so that > I can give "friend" access to private resources. I'd much prefer that > such resources simply be exported. > > And I don't see this as a particular problem, either. We've gotten along > just fine without private resources so far (or maybe there is some big > demand for this that I'm missing?). > > And what about the mismatch issue I brought up? Lots of existing > frameworks use the "services" pattern, in which: > > 1. A well known resource name is used in a getResource() call. > 2. The resource contains a string naming a provider class. > 3. The provider class is loaded using Class.forName(). > > This is a simple but very powerful pattern, but it requires that *both* > the resource and the named class be accessible to the framework. If we > require a permission grant for the resource, but not for the class, it > will be very easy to get access to one and not the other. Is it really > worth opening up this potential headache to enable a friend model for > resources? > > // Bryan Note that the security permission approach can be abused if a module has all permissions, it can access even the private resources in other modules. This is the friend access as you mentioned, but this is not my intention of using the security permission approach. Sorry, I have to go and I can't response in details here. Are you saying that we should probably give up the idea of exported/private resources and make all resources in a module to be always public and accessible? Or are you saying that you still like the idea of exported/private resources but you don't like to address it using the security permission approach? - Stanley