From Stanley.Ho at sun.com Wed Apr 2 22:27:27 2008 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Wed, 02 Apr 2008 22:27:27 -0700 Subject: Updates on Interoperability Message-ID: <47F46ABF.1020306@sun.com> Hi experts, As we discussed in the EG, we would like to make JSR 277 interoperable with other module systems (e.g. OSGi, NetBeans, etc.). We have been building prototypings to figure out how it should work and to also validate the overall approach. Since there are many module systems out there and they are all implemented very differently (there are also multiple OSGi implementations), it requires us to build more than a few prototypes and the whole process takes time. The good news is that we now have a prototype that allows JSR 277 modules to interoperate with OSGi bundles in certain degrees, and another prototype that enables interoperation with Maven. I'm in the process of sorting out the details with the development team so we could have a strawman for this EG to discuss soon. - Stanley From sam at SAMPULLARA.COM Wed Apr 2 22:31:52 2008 From: sam at SAMPULLARA.COM (Sam Pullara) Date: Wed, 02 Apr 2008 22:31:52 -0700 Subject: Updates on Interoperability In-Reply-To: <47F46ABF.1020306@sun.com> References: <47F46ABF.1020306@sun.com> Message-ID: <5B5BE0F4-9E59-484F-BC9F-5C1212AFE9F8@sampullara.com> Can you send out what our roadmap might look like so that we can understand what needs to be done to get this into JDK 7? Sam On Apr 2, 2008, at 10:27 PM, Stanley M. Ho wrote: > Hi experts, > > As we discussed in the EG, we would like to make JSR 277 > interoperable with other module systems (e.g. OSGi, NetBeans, etc.). > We have been building prototypings to figure out how it should work > and to also validate the overall approach. Since there are many > module systems out there and they are all implemented very > differently (there are also multiple OSGi implementations), it > requires us to build more than a few prototypes and the whole > process takes time. > > The good news is that we now have a prototype that allows JSR 277 > modules to interoperate with OSGi bundles in certain degrees, and > another prototype that enables interoperation with Maven. I'm in the > process of sorting out the details with the development team so we > could have a strawman for this EG to discuss soon. > > - Stanley From dan at IKAYZO.COM Thu Apr 3 03:10:00 2008 From: dan at IKAYZO.COM (Daniel Leuck) Date: Thu, 03 Apr 2008 00:10:00 -1000 Subject: Updates on Interoperability In-Reply-To: <5B5BE0F4-9E59-484F-BC9F-5C1212AFE9F8@sampullara.com> References: <47F46ABF.1020306@sun.com> <5B5BE0F4-9E59-484F-BC9F-5C1212AFE9F8@sampullara.com> Message-ID: <80099cb10804030310k555da1d9se3786553432e8974@mail.gmail.com> Hi Sam, My two cents: It seems like OSGi and Maven would be enough. I don't think any other non-IDE system is widely used. I assume Sun will have NetBeans working with JSR-277 modules in short order. I don't think its important to spend a lot of time making JSR-277 interoperate with a lot of other systems. It should be the other way around. Dan On Wed, Apr 2, 2008 at 7:31 PM, Sam Pullara wrote: > Can you send out what our roadmap might look like so that we can understand > what needs to be done to get this into JDK 7? > > Sam > > > > On Apr 2, 2008, at 10:27 PM, Stanley M. Ho wrote: > > > Hi experts, > > > > As we discussed in the EG, we would like to make JSR 277 interoperable > with other module systems (e.g. OSGi, NetBeans, etc.). We have been building > prototypings to figure out how it should work and to also validate the > overall approach. Since there are many module systems out there and they are > all implemented very differently (there are also multiple OSGi > implementations), it requires us to build more than a few prototypes and the > whole process takes time. > > > > The good news is that we now have a prototype that allows JSR 277 modules > to interoperate with OSGi bundles in certain degrees, and another prototype > that enables interoperation with Maven. I'm in the process of sorting out > the details with the development team so we could have a strawman for this > EG to discuss soon. > > > > - Stanley > > > -- Daniel Leuck President Ikayzo, inc. +1 (808) 539-3804 (US Direct) +81 03-3655-2829 (Japan Direct) +1 (808) 393-9119 (Mobile) +1 (808) 591-1496 (Fax) http://www.ikayzo.com http://www.javaui.net From andyp at bea.com Thu Apr 3 03:28:24 2008 From: andyp at bea.com (Andy Piper) Date: Thu, 03 Apr 2008 11:28:24 +0100 Subject: Updates on Interoperability In-Reply-To: <80099cb10804030310k555da1d9se3786553432e8974@mail.gmail.co m> References: <47F46ABF.1020306@sun.com> <5B5BE0F4-9E59-484F-BC9F-5C1212AFE9F8@sampullara.com> <80099cb10804030310k555da1d9se3786553432e8974@mail.gmail.com> Message-ID: <6.2.5.6.2.20080403112230.02fba618@bea.com> Yes, my sentiments exactly. I wouldn't want a focus on a *breadth* of systems to end up meaning that (a) nothing is done well or (b) this slips out of Java 7. As always OSGi is the most important to us and, I suspect, to the majority also. Its also the only one with an associated approved JCP standard, so it would be interesting to know what platform specific issues have been encountered. andy At 11:10 03/04/2008, Daniel Leuck wrote: >Hi Sam, > >My two cents: It seems like OSGi and Maven would be enough. I don't >think any other non-IDE system is widely used. I assume Sun will have >NetBeans working with JSR-277 modules in short order. I don't think >its important to spend a lot of time making JSR-277 interoperate with >a lot of other systems. It should be the other way around. > >Dan > >On Wed, Apr 2, 2008 at 7:31 PM, Sam Pullara wrote: > > Can you send out what our roadmap might look like so that we can understand > > what needs to be done to get this into JDK 7? > > > > Sam > > > > > > > > On Apr 2, 2008, at 10:27 PM, Stanley M. Ho wrote: > > > > > Hi experts, > > > > > > As we discussed in the EG, we would like to make JSR 277 interoperable > > with other module systems (e.g. OSGi, NetBeans, etc.). We have > been building > > prototypings to figure out how it should work and to also validate the > > overall approach. Since there are many module systems out there > and they are > > all implemented very differently (there are also multiple OSGi > > implementations), it requires us to build more than a few > prototypes and the > > whole process takes time. > > > > > > The good news is that we now have a prototype that allows JSR 277 modules > > to interoperate with OSGi bundles in certain degrees, and another prototype > > that enables interoperation with Maven. I'm in the process of sorting out > > the details with the development team so we could have a strawman for this > > EG to discuss soon. > > > > > > - Stanley > > > > > > > > >-- >Daniel Leuck >President >Ikayzo, inc. >+1 (808) 539-3804 (US Direct) >+81 03-3655-2829 (Japan Direct) >+1 (808) 393-9119 (Mobile) >+1 (808) 591-1496 (Fax) >http://www.ikayzo.com >http://www.javaui.net Notice: This email message, together with any attachments, may contain information of BEA Systems, Inc., its subsidiaries and affiliated entities, that may be confidential, proprietary, copyrighted and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by email and then delete it. From sam at SAMPULLARA.COM Thu Apr 3 07:59:13 2008 From: sam at SAMPULLARA.COM (Sam Pullara) Date: Thu, 03 Apr 2008 07:59:13 -0700 Subject: Updates on Interoperability In-Reply-To: <6.2.5.6.2.20080403112230.02fba618@bea.com> References: <47F46ABF.1020306@sun.com> <5B5BE0F4-9E59-484F-BC9F-5C1212AFE9F8@sampullara.com> <80099cb10804030310k555da1d9se3786553432e8974@mail.gmail.com> <6.2.5.6.2.20080403112230.02fba618@bea.com> Message-ID: I meant so that JSR-277 would get into JDK 7 regardless of the decisions we make around interoperability. What I would like from Stanley is a list of things that the expert group needs to do and dates by when they need to be done to ensure that whatever choices are made actually make it into JSR-277. There has been some question in the community as to whether or not JSR-277 would even make it into JDK 7 based on the public schedule of the project. Sorry about the confusion, Sam On Apr 3, 2008, at 3:28 AM, Andy Piper wrote: > Yes, my sentiments exactly. I wouldn't want a focus on a *breadth* of > systems to end up meaning that (a) nothing is done well or (b) this > slips out of Java 7. > > As always OSGi is the most important to us and, I suspect, to the > majority also. > Its also the only one with an associated approved JCP standard, so it > would be interesting to know what platform specific issues have been > encountered. > > andy > > At 11:10 03/04/2008, Daniel Leuck wrote: >> Hi Sam, >> >> My two cents: It seems like OSGi and Maven would be enough. I don't >> think any other non-IDE system is widely used. I assume Sun will >> have >> NetBeans working with JSR-277 modules in short order. I don't think >> its important to spend a lot of time making JSR-277 interoperate with >> a lot of other systems. It should be the other way around. >> >> Dan >> >> On Wed, Apr 2, 2008 at 7:31 PM, Sam Pullara >> wrote: >>> Can you send out what our roadmap might look like so that we can >>> understand >>> what needs to be done to get this into JDK 7? >>> >>> Sam >>> >>> >>> >>> On Apr 2, 2008, at 10:27 PM, Stanley M. Ho wrote: >>> >>>> Hi experts, >>>> >>>> As we discussed in the EG, we would like to make JSR 277 >>>> interoperable >>> with other module systems (e.g. OSGi, NetBeans, etc.). We have >> been building >>> prototypings to figure out how it should work and to also validate >>> the >>> overall approach. Since there are many module systems out there >> and they are >>> all implemented very differently (there are also multiple OSGi >>> implementations), it requires us to build more than a few >> prototypes and the >>> whole process takes time. >>>> >>>> The good news is that we now have a prototype that allows JSR 277 >>>> modules >>> to interoperate with OSGi bundles in certain degrees, and another >>> prototype >>> that enables interoperation with Maven. I'm in the process of >>> sorting out >>> the details with the development team so we could have a strawman >>> for this >>> EG to discuss soon. >>>> >>>> - Stanley >>>> >>> >> >> >> >> -- >> Daniel Leuck >> President >> Ikayzo, inc. >> +1 (808) 539-3804 (US Direct) >> +81 03-3655-2829 (Japan Direct) >> +1 (808) 393-9119 (Mobile) >> +1 (808) 591-1496 (Fax) >> http://www.ikayzo.com >> http://www.javaui.net > > > > Notice: This email message, together with any attachments, may > contain information of BEA Systems, Inc., its subsidiaries and > affiliated entities, that may be confidential, proprietary, > copyrighted and/or legally privileged, and is intended solely for > the use of the individual or entity named in this message. If you > are not the intended recipient, and have received this message in > error, please immediately return this by email and then delete it. From Stanley.Ho at sun.com Fri Apr 4 00:35:48 2008 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Fri, 04 Apr 2008 00:35:48 -0700 Subject: Updates on Interoperability In-Reply-To: References: <47F46ABF.1020306@sun.com> <5B5BE0F4-9E59-484F-BC9F-5C1212AFE9F8@sampullara.com> <80099cb10804030310k555da1d9se3786553432e8974@mail.gmail.com> <6.2.5.6.2.20080403112230.02fba618@bea.com> Message-ID: <47F5DA54.502@sun.com> Hi Sam, I am in the process of writing the EDR2 spec and it should be ready for the EG to review within the next few weeks. Once the spec is available, I hope it will give the EG a clearer overall picture in terms of the current JSR state. I will also provide a TODO list as you suggested. - Stanley Sam Pullara wrote: > I meant so that JSR-277 would get into JDK 7 regardless of the decisions > we make around interoperability. What I would like from Stanley is a > list of things that the expert group needs to do and dates by when they > need to be done to ensure that whatever choices are made actually make > it into JSR-277. There has been some question in the community as to > whether or not JSR-277 would even make it into JDK 7 based on the public > schedule of the project. > > Sorry about the confusion, > Sam From hargrave at US.IBM.COM Mon Apr 7 08:55:18 2008 From: hargrave at US.IBM.COM (BJ Hargrave) Date: Mon, 07 Apr 2008 11:55:18 -0400 Subject: Updates on Interoperability In-Reply-To: <47F46ABF.1020306@sun.com> References: <47F46ABF.1020306@sun.com> Message-ID: Java Community Process JSR #277 Expert List wrote on 2008/04/03 01:27:27 AM: > [image removed] > > Updates on Interoperability > > Stanley M. Ho > > to: > > JSR-277-EG > > 2008/04/03 01:30 AM > > Sent by: > > Java Community Process JSR #277 Expert List > > Please respond to Java Community Process JSR #277 Expert List > > > Hi experts, > > As we discussed in the EG, we would like to make JSR 277 interoperable > with other module systems (e.g. OSGi, NetBeans, etc.). We have been > building prototypings to figure out how it should work and to also > validate the overall approach. Since there are many module systems out > there and they are all implemented very differently (there are also > multiple OSGi implementations), it requires us to build more than a few > prototypes and the whole process takes time. This is good news. I realize there are many OSGi implementations, but they all implement the OSGi spec. So the goal of OSGi interoperation must be aimed at the OSGi spec and not the various idiosyncrasies of the various implementations. > > The good news is that we now have a prototype that allows JSR 277 > modules to interoperate with OSGi bundles in certain degrees, and > another prototype that enables interoperation with Maven. I'm in the > process of sorting out the details with the development team so we could > have a strawman for this EG to discuss soon. Looking forward to it. > > - Stanley -- BJ Hargrave Senior Technical Staff Member, IBM OSGi Fellow and CTO of the OSGi Alliance hargrave at us.ibm.com office: +1 386 848 1781 mobile: +1 386 848 3788 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/attachments/20080407/faff217c/attachment.html From bryan.atsatt at oracle.com Thu Apr 17 17:47:54 2008 From: bryan.atsatt at oracle.com (Bryan Atsatt) Date: Thu, 17 Apr 2008 17:47:54 -0700 Subject: Spec re-organization/update proposal... Message-ID: <4807EFBA.7000701@oracle.com> I propose that we re-organize and update the next draft of the spec to achieve the following: 1. Clearly identify and separate the two main elements of the spec: a. Framework (i.e. api/spi) b. Default implementation (i.e. .jam distribution format, tools, usage examples, etc.) 2. Ensure that the framework is the primary focus of the spec, identifying it as the means by which any implementation integrates with the SE compiler and runtime. 3. Clarify that any implementation will benefit from the integration story provided by the framework (I describe four concrete benefits in http://atsatt.blogspot.com/2008/04/jsr-277-could-be-great-for-osgi.html). 4. Clarify the rationale for the existence of a new implementation. 5. Recognize the importance of building a second implementation to validate the framework design. 6. Recognize OSGi as the right choice for that second implementation: a. A formal JCP adopted module standard. b. Wide market adoption. If the reasoning behind these proposals is not sufficiently clear to everyone, I'll be happy to elaborate. // Bryan From Stanley.Ho at sun.com Thu Apr 17 18:50:11 2008 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Thu, 17 Apr 2008 18:50:11 -0700 Subject: Spec re-organization/update proposal... In-Reply-To: <4807EFBA.7000701@oracle.com> References: <4807EFBA.7000701@oracle.com> Message-ID: <4807FE53.8080500@sun.com> Hi Bryan, Bryan Atsatt wrote: > I propose that we re-organize and update the next draft of the spec to > achieve the following: > > 1. Clearly identify and separate the two main elements of the spec: > > a. Framework (i.e. api/spi) > b. Default implementation (i.e. .jam distribution format, tools, > usage examples, etc.) > > 2. Ensure that the framework is the primary focus of the spec, > identifying it as the means by which any implementation integrates with > the SE compiler and runtime. > > 3. Clarify that any implementation will benefit from the integration > story provided by the framework (I describe four concrete benefits in > http://atsatt.blogspot.com/2008/04/jsr-277-could-be-great-for-osgi.html). > > 4. Clarify the rationale for the existence of a new implementation. Agreed on all four points. > 5. Recognize the importance of building a second implementation to > validate the framework design. > > 6. Recognize OSGi as the right choice for that second implementation: > > a. A formal JCP adopted module standard. > b. Wide market adoption. JSR 277 certainly wants to enable multiple implementations of the Repository and ModuleDefinition APIs, especially by OSGi containers. It would be unusual for a normative spec to mention particular implementations (except the default implementation). However, we are happy to mention OSGi in a non-normative section that discusses the benefits of integration between module systems. - Stanley From andyp at bea.com Fri Apr 18 03:19:04 2008 From: andyp at bea.com (Andy Piper) Date: Fri, 18 Apr 2008 11:19:04 +0100 Subject: Spec re-organization/update proposal... In-Reply-To: <4807EFBA.7000701@oracle.com> References: <4807EFBA.7000701@oracle.com> Message-ID: <6.2.5.6.2.20080418111859.02f1a250@bea.com> Agreed andy At 01:47 18/04/2008, Bryan Atsatt wrote: >I propose that we re-organize and update the next draft of the spec >to achieve the following: > >1. Clearly identify and separate the two main elements of the spec: > > a. Framework (i.e. api/spi) > b. Default implementation (i.e. .jam distribution format, tools, > usage examples, etc.) > >2. Ensure that the framework is the primary focus of the spec, >identifying it as the means by which any implementation integrates >with the SE compiler and runtime. > >3. Clarify that any implementation will benefit from the integration >story provided by the framework (I describe four concrete benefits >in http://atsatt.blogspot.com/2008/04/jsr-277-could-be-great-for-osgi.html). > >4. Clarify the rationale for the existence of a new implementation. > >5. Recognize the importance of building a second implementation to >validate the framework design. > >6. Recognize OSGi as the right choice for that second implementation: > > a. A formal JCP adopted module standard. > b. Wide market adoption. > > >If the reasoning behind these proposals is not sufficiently clear to >everyone, I'll be happy to elaborate. > >// Bryan Notice: This email message, together with any attachments, may contain information of BEA Systems, Inc., its subsidiaries and affiliated entities, that may be confidential, proprietary, copyrighted and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by email and then delete it. From andyp at bea.com Fri Apr 18 03:20:09 2008 From: andyp at bea.com (Andy Piper) Date: Fri, 18 Apr 2008 11:20:09 +0100 Subject: Spec re-organization/update proposal... In-Reply-To: <4807FE53.8080500@sun.com> References: <4807EFBA.7000701@oracle.com> <4807FE53.8080500@sun.com> Message-ID: <6.2.5.6.2.20080418111914.05207ca8@bea.com> At 02:50 18/04/2008, Stanley M. Ho wrote: >JSR 277 certainly wants to enable multiple implementations of the >Repository and ModuleDefinition APIs, especially by OSGi containers. > >It would be unusual for a normative spec to mention particular >implementations (except the default implementation). Well, except of course that OSGi 4.1 is already a JSR courtesy of JSR 291. That puts it in a class of its own IMO. andy Notice: This email message, together with any attachments, may contain information of BEA Systems, Inc., its subsidiaries and affiliated entities, that may be confidential, proprietary, copyrighted and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by email and then delete it. From heavy at UNGOVERNED.ORG Fri Apr 18 07:14:48 2008 From: heavy at UNGOVERNED.ORG (Richard S. Hall) Date: Fri, 18 Apr 2008 10:14:48 -0400 Subject: Spec re-organization/update proposal... In-Reply-To: <4807EFBA.7000701@oracle.com> References: <4807EFBA.7000701@oracle.com> Message-ID: <4808ACD8.5030301@ungoverned.org> Sounds good to me. -> richard Bryan Atsatt wrote: > I propose that we re-organize and update the next draft of the spec to > achieve the following: > > 1. Clearly identify and separate the two main elements of the spec: > > a. Framework (i.e. api/spi) > b. Default implementation (i.e. .jam distribution format, tools, > usage examples, etc.) > > 2. Ensure that the framework is the primary focus of the spec, > identifying it as the means by which any implementation integrates > with the SE compiler and runtime. > > 3. Clarify that any implementation will benefit from the integration > story provided by the framework (I describe four concrete benefits in > http://atsatt.blogspot.com/2008/04/jsr-277-could-be-great-for-osgi.html). > > 4. Clarify the rationale for the existence of a new implementation. > > 5. Recognize the importance of building a second implementation to > validate the framework design. > > 6. Recognize OSGi as the right choice for that second implementation: > > a. A formal JCP adopted module standard. > b. Wide market adoption. > > > If the reasoning behind these proposals is not sufficiently clear to > everyone, I'll be happy to elaborate. > > // Bryan From vladimir.a.strigun at INTEL.COM Fri Apr 18 07:32:20 2008 From: vladimir.a.strigun at INTEL.COM (Strigun, Vladimir A) Date: Fri, 18 Apr 2008 18:32:20 +0400 Subject: Spec re-organization/update proposal... In-Reply-To: A <4807EFBA.7000701@oracle.com> References: A <4807EFBA.7000701@oracle.com> Message-ID: <3D8E84095C6A524A985B787423094E4001F84A97@mssmsx411.ccr.corp.intel.com> agreed -----Original Message----- From: Java Community Process JSR #277 Expert List [mailto:JSR-277-EG at JCP.ORG] On Behalf Of Bryan Atsatt Sent: Friday, April 18, 2008 4:48 AM To: JSR-277-EG at JCP.ORG Subject: Spec re-organization/update proposal... I propose that we re-organize and update the next draft of the spec to achieve the following: 1. Clearly identify and separate the two main elements of the spec: a. Framework (i.e. api/spi) b. Default implementation (i.e. .jam distribution format, tools, usage examples, etc.) 2. Ensure that the framework is the primary focus of the spec, identifying it as the means by which any implementation integrates with the SE compiler and runtime. 3. Clarify that any implementation will benefit from the integration story provided by the framework (I describe four concrete benefits in http://atsatt.blogspot.com/2008/04/jsr-277-could-be-great-for-osgi.html) . 4. Clarify the rationale for the existence of a new implementation. 5. Recognize the importance of building a second implementation to validate the framework design. 6. Recognize OSGi as the right choice for that second implementation: a. A formal JCP adopted module standard. b. Wide market adoption. If the reasoning behind these proposals is not sufficiently clear to everyone, I'll be happy to elaborate. // Bryan -------------------------------------------------------------------- Closed Joint Stock Company Intel A/O Registered legal address: Krylatsky Hills Business Park, 17 Krylatskaya Str., Bldg 4, Moscow 121614, Russian Federation This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. From bryan.atsatt at oracle.com Fri Apr 18 11:13:59 2008 From: bryan.atsatt at oracle.com (Bryan Atsatt) Date: Fri, 18 Apr 2008 11:13:59 -0700 Subject: EG revitalization Message-ID: <4808E4E7.5060507@oracle.com> This EG has stagnated and needs to be brought back to life. I'd like to propose these concrete steps: 1. Add Peter Kriens to the EG. Peter has clearly demonstrated great expertise in this area, and I believe he would quickly become a valued, active contributor. 2. Bring the interop design/discussion/investigation process out of Sun offices and into the EG. 3. Give the EG read access to the tip of the source tree (or at least very current snapshots). // Bryan From heavy at UNGOVERNED.ORG Fri Apr 18 11:17:13 2008 From: heavy at UNGOVERNED.ORG (Richard S. Hall) Date: Fri, 18 Apr 2008 14:17:13 -0400 Subject: EG revitalization In-Reply-To: <4808E4E7.5060507@oracle.com> References: <4808E4E7.5060507@oracle.com> Message-ID: <4808E5A9.6070007@ungoverned.org> +3 -> richard Bryan Atsatt wrote: > This EG has stagnated and needs to be brought back to life. I'd like > to propose these concrete steps: > > 1. Add Peter Kriens to the EG. Peter has clearly demonstrated great > expertise in this area, and I believe he would quickly become a > valued, active contributor. > > 2. Bring the interop design/discussion/investigation process out of > Sun offices and into the EG. > > 3. Give the EG read access to the tip of the source tree (or at least > very current snapshots). > > // Bryan From Stanley.Ho at sun.com Fri Apr 18 18:10:08 2008 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Fri, 18 Apr 2008 18:10:08 -0700 Subject: Simplification of JSR 277 and JSR 294 Message-ID: <48094670.5070209@sun.com> Dear 277 and 294 experts, This is to announce that Alex Buckley and Stanley Ho will be co-spec leads for JSR 277 as soon as the JCP PMO processes the change. To ensure that all experts can follow both the development and deployment aspects of the Java Module System, the plan is for *all* Java modularity discussions to occur in the 277 EG from now on. The development aspects of modularity are specified in the Java Language Specification and Java Virtual Machine Specification regardless of which EG they are discussed in. The effects of this plan are most keenly felt by the two members of the 294 EG who are not already on the 277 EG. The members in question, Eugene Vigdorchik and Matthew Flatt, have kindly agreed to join the 277 observer list [1]. We would like to invite members of the 294 observer list [2] to subscribe to the 277 observer list too. Also, the jsr-277-comments at jcp.org alias is available for anyone to send private comments to the 277 spec leads at any time. We try to respond individually to such comments. Every JSR has a comments alias but their existence is not well known, so hopefully the use of jsr-277-comments at jcp.org will grow following this message. Thanks, Stanley and Alex [1] http://mail.openjdk.java.net/mailman/listinfo/jsr277-eg-observer [2] http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-observer From Stanley.Ho at sun.com Fri Apr 18 18:11:05 2008 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Fri, 18 Apr 2008 18:11:05 -0700 Subject: EG revitalization In-Reply-To: <4808E4E7.5060507@oracle.com> References: <4808E4E7.5060507@oracle.com> Message-ID: <480946A9.8050406@sun.com> Thanks Bryan. I will get in touch with Peter to gauge his interest level and availability. For the interop and the source tree, we are already planning to make them available to the EG in the next week or two. - Stanley From andyp at bea.com Wed Apr 23 06:06:08 2008 From: andyp at bea.com (Andy Piper) Date: Wed, 23 Apr 2008 14:06:08 +0100 Subject: Simplification of JSR 277 and JSR 294 In-Reply-To: <48094670.5070209@sun.com> References: <48094670.5070209@sun.com> Message-ID: <6.2.5.6.2.20080423140448.02f3b380@bea.com> This seems like an excellent development! This coupled with adoption of Bryan's proposal should breath new life back into this JSR. andy At 02:10 19/04/2008, Stanley M. Ho wrote: >Dear 277 and 294 experts, > >This is to announce that Alex Buckley and Stanley Ho will be co-spec >leads for JSR 277 as soon as the JCP PMO processes the change. > >To ensure that all experts can follow both the development and >deployment aspects of the Java Module System, the plan is for *all* >Java modularity discussions to occur in the 277 EG from now on. > >The development aspects of modularity are specified in the Java >Language Specification and Java Virtual Machine Specification >regardless of which EG they are discussed in. > >The effects of this plan are most keenly felt by the two members of >the 294 EG who are not already on the 277 EG. The members in >question, Eugene Vigdorchik and Matthew Flatt, have kindly agreed to >join the 277 observer list [1]. > >We would like to invite members of the 294 observer list [2] to >subscribe to the 277 observer list too. > >Also, the jsr-277-comments at jcp.org alias is available for anyone to >send private comments to the 277 spec leads at any time. We try to >respond individually to such comments. Every JSR has a comments >alias but their existence is not well known, so hopefully the use of >jsr-277-comments at jcp.org will grow following this message. > >Thanks, >Stanley and Alex > >[1] http://mail.openjdk.java.net/mailman/listinfo/jsr277-eg-observer >[2] http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-observer Notice: This email message, together with any attachments, may contain information of BEA Systems, Inc., its subsidiaries and affiliated entities, that may be confidential, proprietary, copyrighted and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by email and then delete it. From brett at apache.org Fri Apr 25 01:34:24 2008 From: brett at apache.org (Brett Porter) Date: Fri, 25 Apr 2008 16:34:24 +0800 Subject: Updates on Interoperability In-Reply-To: <47F46ABF.1020306@sun.com> References: <47F46ABF.1020306@sun.com> Message-ID: <17AF02C2-048E-4509-AD62-8FC0C2802C70@apache.org> On 03/04/2008, at 1:27 PM, Stanley M. Ho wrote: > The good news is that we now have a prototype that allows JSR 277 > modules to interoperate with OSGi bundles in certain degrees, and > another prototype that enables interoperation with Maven. I'm in the > process of sorting out the details with the development team so we > could have a strawman for this EG to discuss soon. With my Maven hat on, this sounds quite interesting. Is there any update on the availability of this for discussion? Thanks, Brett -- Brett Porter brett at apache.org http://blogs.exist.com/bporter/ From bryan.atsatt at oracle.com Fri Apr 25 10:57:54 2008 From: bryan.atsatt at oracle.com (Bryan Atsatt) Date: Fri, 25 Apr 2008 10:57:54 -0700 Subject: Updates on Interoperability In-Reply-To: <47F46ABF.1020306@sun.com> References: <47F46ABF.1020306@sun.com> Message-ID: <48121BA2.7060001@oracle.com> Nice to see some movement here Stanley! I've been thinking a bit about this issue, and tried to summarize that here: http://atsatt.blogspot.com/2008/04/jsr-277-interoperation.html Do folks generally agree with this characterization? Does anyone think I've grossly misstated the situation, or left out anything important? // Bryan Stanley M. Ho wrote: > Hi experts, > > As we discussed in the EG, we would like to make JSR 277 interoperable > with other module systems (e.g. OSGi, NetBeans, etc.). We have been > building prototypings to figure out how it should work and to also > validate the overall approach. Since there are many module systems out > there and they are all implemented very differently (there are also > multiple OSGi implementations), it requires us to build more than a > few prototypes and the whole process takes time. > > The good news is that we now have a prototype that allows JSR 277 > modules to interoperate with OSGi bundles in certain degrees, and > another prototype that enables interoperation with Maven. I'm in the > process of sorting out the details with the development team so we > could have a strawman for this EG to discuss soon. > > - Stanley > From gordon.hirsch at SAS.COM Fri Apr 25 12:33:55 2008 From: gordon.hirsch at SAS.COM (Gordon Hirsch) Date: Fri, 25 Apr 2008 15:33:55 -0400 Subject: Updates on Interoperability In-Reply-To: <48121BA2.7060001@oracle.com> References: <47F46ABF.1020306@sun.com> <48121BA2.7060001@oracle.com> Message-ID: <48123223.9070803@sas.com> Bryan Atsatt wrote: > I've been thinking a bit about > this issue, and tried to summarize that here: > > http://atsatt.blogspot.com/2008/04/jsr-277-interoperation.html > > Do folks generally agree with this characterization? Does anyone think > I've grossly misstated the situation, or left out anything important? Hi Bryan, Great summary. I generally agree with your characterization of interoperability. This degree of interoperability, at least with OSGi, is necessary to make JSR 277 a success. It's true, as you mention, that much of the "interoperability story is already present in the spec". However, I think there's good reason to wonder how achievable that story actually is. That's why it has been disappointing to wait so long for something concrete. It's encouraging to read that we'll see something soon. You mention that resolution requires each implementation to: > 1. Map its dependency declarations into a standard runtime representation. > 2. Support a standard search model over its stored modules, using the runtime dependency expressions. > 3. Map its stored module data into a standard runtime representation that can be returned by the search. One challenge lies in defining the "standard runtime representations" and "standard search model" in a universal enough way to encompass OSGi and other module systems. This implies embracing concepts (in these standard representations and search model) that were not universally liked by the EG early on. (Split packages and package-level import/export come to mind.) From heavy at UNGOVERNED.ORG Fri Apr 25 13:11:15 2008 From: heavy at UNGOVERNED.ORG (Richard S. Hall) Date: Fri, 25 Apr 2008 16:11:15 -0400 Subject: Updates on Interoperability In-Reply-To: <48123223.9070803@sas.com> References: <47F46ABF.1020306@sun.com> <48121BA2.7060001@oracle.com> <48123223.9070803@sas.com> Message-ID: <48123AE3.1090708@ungoverned.org> Gordon Hirsch wrote: >> 1. Map its dependency declarations into a standard runtime >> representation. >> 2. Support a standard search model over its stored modules, using the >> runtime dependency expressions. >> 3. Map its stored module data into a standard runtime representation >> that can be returned by the search. > > One challenge lies in defining the "standard runtime representations" > and "standard search model" in a universal enough way to encompass > OSGi and other module systems. This implies embracing concepts (in > these standard representations and search model) that were not > universally liked by the EG early on. (Split packages and > package-level import/export come to mind.) Package-level import/export seem like a must, but I still don't see split packages as a necessity. -> richard From bryan.atsatt at oracle.com Fri Apr 25 13:35:40 2008 From: bryan.atsatt at oracle.com (Bryan Atsatt) Date: Fri, 25 Apr 2008 13:35:40 -0700 Subject: Updates on Interoperability In-Reply-To: <48123AE3.1090708@ungoverned.org> References: <47F46ABF.1020306@sun.com> <48121BA2.7060001@oracle.com> <48123223.9070803@sas.com> <48123AE3.1090708@ungoverned.org> Message-ID: <4812409C.1030000@oracle.com> Richard S. Hall wrote: > Gordon Hirsch wrote: >>> 1. Map its dependency declarations into a standard runtime >>> representation. >>> 2. Support a standard search model over its stored modules, using >>> the runtime dependency expressions. >>> 3. Map its stored module data into a standard runtime representation >>> that can be returned by the search. >> >> One challenge lies in defining the "standard runtime representations" >> and "standard search model" in a universal enough way to encompass >> OSGi and other module systems. This implies embracing concepts (in >> these standard representations and search model) that were not >> universally liked by the EG early on. (Split packages and >> package-level import/export come to mind.) > > Package-level import/export seem like a must, but I still don't see > split packages as a necessity. Strongly agreed on import-by-package, and that ModuleDefinition requires a method to enumerate exported packages (and probably an exportsPackage(String packageName) method to enable an efficient Query). On the split package front, however, it seems a little fuzzy to me. If an OSGi implementation of 277 is also going to remain OSGi Rx compliant, it will still need to support split packages, right? If so, don't we need to surface this fact at the 277 level? Perhaps that is as simple as acknowledging that Module.getImportedModules() may return more than one instance that exports a given package. // Bryan From heavy at UNGOVERNED.ORG Fri Apr 25 13:57:02 2008 From: heavy at UNGOVERNED.ORG (Richard S. Hall) Date: Fri, 25 Apr 2008 16:57:02 -0400 Subject: Updates on Interoperability In-Reply-To: <4812409C.1030000@oracle.com> References: <47F46ABF.1020306@sun.com> <48121BA2.7060001@oracle.com> <48123223.9070803@sas.com> <48123AE3.1090708@ungoverned.org> <4812409C.1030000@oracle.com> Message-ID: <4812459E.2060407@ungoverned.org> Bryan Atsatt wrote: > Richard S. Hall wrote: >> Gordon Hirsch wrote: >>>> 1. Map its dependency declarations into a standard runtime >>>> representation. >>>> 2. Support a standard search model over its stored modules, using >>>> the runtime dependency expressions. >>>> 3. Map its stored module data into a standard runtime >>>> representation that can be returned by the search. >>> >>> One challenge lies in defining the "standard runtime >>> representations" and "standard search model" in a universal enough >>> way to encompass OSGi and other module systems. This implies >>> embracing concepts (in these standard representations and search >>> model) that were not universally liked by the EG early on. (Split >>> packages and package-level import/export come to mind.) >> >> Package-level import/export seem like a must, but I still don't see >> split packages as a necessity. > Strongly agreed on import-by-package, and that ModuleDefinition > requires a method to enumerate exported packages (and probably an > exportsPackage(String packageName) method to enable an efficient Query). > On the split package front, however, it seems a little fuzzy to me. If > an OSGi implementation of 277 is also going to remain OSGi Rx > compliant, it will still need to support split packages, right? If so, > don't we need to surface this fact at the 277 level? Perhaps that is > as simple as acknowledging that Module.getImportedModules() may return > more than one instance that exports a given package. That all depends on if the 277 API is a subset of OSGi functionality or equal to it, I suppose. -> richard > > // Bryan From gordon.hirsch at SAS.COM Fri Apr 25 14:03:05 2008 From: gordon.hirsch at SAS.COM (Gordon Hirsch) Date: Fri, 25 Apr 2008 17:03:05 -0400 Subject: Updates on Interoperability In-Reply-To: <4812409C.1030000@oracle.com> References: <47F46ABF.1020306@sun.com> <48121BA2.7060001@oracle.com> <48123223.9070803@sas.com> <48123AE3.1090708@ungoverned.org> <4812409C.1030000@oracle.com> Message-ID: <48124709.1000109@sas.com> Bryan Atsatt wrote: > > On the split package front, however, it seems a little fuzzy to me. If > an OSGi implementation of 277 is also going to remain OSGi Rx compliant, > it will still need to support split packages, right? Yes, and, like it or not, split packages are common enough that it may be hard to completely ignore them wrt to interoperability. If so, don't we > need to surface this fact at the 277 level? Perhaps that is as simple as > acknowledging that Module.getImportedModules() may return more than one > instance that exports a given package. This is the case I was thinking of. Also, IIRC, OSGi allows the specification of additional attributes on Import-Package statements which can be used to help disambiguate the import in the presence of split packages. Does this facility get surfaced to the standard search mechanism? If features at this level are involved, this is where I begin to become concerned about the complexity of our task :-). From sam at SAMPULLARA.COM Fri Apr 25 14:08:56 2008 From: sam at SAMPULLARA.COM (Sam Pullara) Date: Fri, 25 Apr 2008 14:08:56 -0700 Subject: Updates on Interoperability In-Reply-To: <4812459E.2060407@ungoverned.org> References: <47F46ABF.1020306@sun.com> <48121BA2.7060001@oracle.com> <48123223.9070803@sas.com> <48123AE3.1090708@ungoverned.org> <4812409C.1030000@oracle.com> <4812459E.2060407@ungoverned.org> Message-ID: <25680D44-7C07-4DC9-841F-12751540ECD4@sampullara.com> Did we decide which way it will be compatible? I'm a little concerned that implementing something that can use OSGi modules is a far bigger task that implementing something that OSGi can use. If we are going for total compatibility it seems like we should just use OSGi. Personally I wanted something in Java that was simpler and cleaner that OSGi-users could leverage if they wanted. But I'm coming at this from the we-need-something-like-maven/gem/ivy camp. The vast majority of people that are going to use this system will have never heard of OSGI nor care about any of their more subtle features and just want an easy way to leverage the vast amount of libraries available (virtually all of them are not OSGi modules today but are jar files in maven repositories). We shouldn't be specing out some sort of uber container but rather a simple way to tie code to its dependencies. I strongly suggest KISS applies to our effort. I write this knowing that the committee seems to be heavily in favor of this alternate view that we implement the kitchen sink. Sam On Apr 25, 2008, at 1:57 PM, Richard S. Hall wrote: > Bryan Atsatt wrote: >> Richard S. Hall wrote: >>> Gordon Hirsch wrote: >>>>> 1. Map its dependency declarations into a standard runtime >>>>> representation. >>>>> 2. Support a standard search model over its stored modules, >>>>> using the runtime dependency expressions. >>>>> 3. Map its stored module data into a standard runtime >>>>> representation that can be returned by the search. >>>> >>>> One challenge lies in defining the "standard runtime >>>> representations" and "standard search model" in a universal >>>> enough way to encompass OSGi and other module systems. This >>>> implies embracing concepts (in these standard representations and >>>> search model) that were not universally liked by the EG early on. >>>> (Split packages and package-level import/export come to mind.) >>> >>> Package-level import/export seem like a must, but I still don't >>> see split packages as a necessity. >> Strongly agreed on import-by-package, and that ModuleDefinition >> requires a method to enumerate exported packages (and probably an >> exportsPackage(String packageName) method to enable an efficient >> Query). >> On the split package front, however, it seems a little fuzzy to me. >> If an OSGi implementation of 277 is also going to remain OSGi Rx >> compliant, it will still need to support split packages, right? If >> so, don't we need to surface this fact at the 277 level? Perhaps >> that is as simple as acknowledging that Module.getImportedModules() >> may return more than one instance that exports a given package. > > That all depends on if the 277 API is a subset of OSGi functionality > or equal to it, I suppose. > > -> richard >> >> // Bryan From heavy at UNGOVERNED.ORG Fri Apr 25 14:08:59 2008 From: heavy at UNGOVERNED.ORG (Richard S. Hall) Date: Fri, 25 Apr 2008 17:08:59 -0400 Subject: Updates on Interoperability In-Reply-To: <48124709.1000109@sas.com> References: <47F46ABF.1020306@sun.com> <48121BA2.7060001@oracle.com> <48123223.9070803@sas.com> <48123AE3.1090708@ungoverned.org> <4812409C.1030000@oracle.com> <48124709.1000109@sas.com> Message-ID: <4812486B.5030301@ungoverned.org> Gordon Hirsch wrote: > Bryan Atsatt wrote: >> >> On the split package front, however, it seems a little fuzzy to me. If >> an OSGi implementation of 277 is also going to remain OSGi Rx compliant, >> it will still need to support split packages, right? > > Yes, and, like it or not, split packages are common enough that it may > be hard to completely ignore them wrt to interoperability. > > If so, don't we >> need to surface this fact at the 277 level? Perhaps that is as simple as >> acknowledging that Module.getImportedModules() may return more than one >> instance that exports a given package. > > This is the case I was thinking of. > > Also, IIRC, OSGi allows the specification of additional attributes on > Import-Package statements which can be used to help disambiguate the > import in the presence of split packages. Does this facility get > surfaced to the standard search mechanism? If features at this level > are involved, this is where I begin to become concerned about the > complexity of our task :-). Well, the attributes really have nothing to do with split packages, although the recommendation is to use mandatory attributes on split packages to avoid importers from accidentally getting wired to a partial package. We can go down this path, but I doubt it will be as simple as just changing getImportedModules() to return an array... -> richard From Stanley.Ho at sun.com Fri Apr 25 15:44:53 2008 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Fri, 25 Apr 2008 15:44:53 -0700 Subject: Updates on Interoperability In-Reply-To: <17AF02C2-048E-4509-AD62-8FC0C2802C70@apache.org> References: <47F46ABF.1020306@sun.com> <17AF02C2-048E-4509-AD62-8FC0C2802C70@apache.org> Message-ID: <48125EE5.2060006@sun.com> Hi Brett, We are in the process of finalizing the document, and hopefully we could provide more details to the EG next week. - Stanley Brett Porter wrote: > On 03/04/2008, at 1:27 PM, Stanley M. Ho wrote: > >> The good news is that we now have a prototype that allows JSR 277 >> modules to interoperate with OSGi bundles in certain degrees, and >> another prototype that enables interoperation with Maven. I'm in the >> process of sorting out the details with the development team so we >> could have a strawman for this EG to discuss soon. > > With my Maven hat on, this sounds quite interesting. Is there any update > on the availability of this for discussion? > > Thanks, > Brett > > -- > Brett Porter > brett at apache.org > http://blogs.exist.com/bporter/ From Stanley.Ho at sun.com Fri Apr 25 15:53:38 2008 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Fri, 25 Apr 2008 15:53:38 -0700 Subject: Updates on Interoperability In-Reply-To: <4812409C.1030000@oracle.com> References: <47F46ABF.1020306@sun.com> <48121BA2.7060001@oracle.com> <48123223.9070803@sas.com> <48123AE3.1090708@ungoverned.org> <4812409C.1030000@oracle.com> Message-ID: <481260F2.2040508@sun.com> The current APis do not prohibit module definition to have split packages. It is only when a Java module is instantiated from the module definition, the implementation of the default module system would perform shallow validation and consider split packages as an error. Other module system implementations can continue to have their own validation policies to resolve their modules. I hope this will become more clear when the details are available next week. - Stanley Bryan Atsatt wrote: > Richard S. Hall wrote: >> Gordon Hirsch wrote: >>>> 1. Map its dependency declarations into a standard runtime >>>> representation. >>>> 2. Support a standard search model over its stored modules, using >>>> the runtime dependency expressions. >>>> 3. Map its stored module data into a standard runtime representation >>>> that can be returned by the search. >>> >>> One challenge lies in defining the "standard runtime representations" >>> and "standard search model" in a universal enough way to encompass >>> OSGi and other module systems. This implies embracing concepts (in >>> these standard representations and search model) that were not >>> universally liked by the EG early on. (Split packages and >>> package-level import/export come to mind.) >> >> Package-level import/export seem like a must, but I still don't see >> split packages as a necessity. > Strongly agreed on import-by-package, and that ModuleDefinition requires > a method to enumerate exported packages (and probably an > exportsPackage(String packageName) method to enable an efficient Query). > On the split package front, however, it seems a little fuzzy to me. If > an OSGi implementation of 277 is also going to remain OSGi Rx compliant, > it will still need to support split packages, right? If so, don't we > need to surface this fact at the 277 level? Perhaps that is as simple as > acknowledging that Module.getImportedModules() may return more than one > instance that exports a given package. > > // Bryan From heavy at UNGOVERNED.ORG Fri Apr 25 16:46:50 2008 From: heavy at UNGOVERNED.ORG (Richard S. Hall) Date: Fri, 25 Apr 2008 19:46:50 -0400 Subject: Updates on Interoperability In-Reply-To: <25680D44-7C07-4DC9-841F-12751540ECD4@sampullara.com> References: <47F46ABF.1020306@sun.com> <48121BA2.7060001@oracle.com> <48123223.9070803@sas.com> <48123AE3.1090708@ungoverned.org> <4812409C.1030000@oracle.com> <4812459E.2060407@ungoverned.org> <25680D44-7C07-4DC9-841F-12751540ECD4@sampullara.com> Message-ID: <48126D6A.7000001@ungoverned.org> Sam Pullara wrote: > Did we decide which way it will be compatible? I'm a little concerned > that implementing something that can use OSGi modules is a far bigger > task that implementing something that OSGi can use. If we are going > for total compatibility it seems like we should just use OSGi. > Personally I wanted something in Java that was simpler and cleaner > that OSGi-users could leverage if they wanted. But I'm coming at this > from the we-need-something-like-maven/gem/ivy camp. > > The vast majority of people that are going to use this system will > have never heard of OSGI nor care about any of their more subtle > features and just want an easy way to leverage the vast amount of > libraries available (virtually all of them are not OSGi modules today > but are jar files in maven repositories). We shouldn't be specing out > some sort of uber container but rather a simple way to tie code to its > dependencies. If this is all that people wanted, then we could just define repositories and associated metadata and forget about all runtime modularity support, but I strongly disagree that this is all people want or else there wouldn't be such an upsurge in OSGi interest and adoption. > I strongly suggest KISS applies to our effort. I write this knowing > that the committee seems to be heavily in favor of this alternate view > that we implement the kitchen sink. I think our interest is similar, but perhaps our conclusions are different. Initially, I think my comments were construed as an argument against split packages in general, but I am really arguing the same as you (I think) that we don't really need split packages to be a concept in our "interoperability API", if that is how we are to view the 277 API. Of course, I am also willing to have the former argument too, but I will leave that for another time. :-) From my point of view, I do not think it is wholly reasonable to assume that we want to try to accomplish interoperability across module systems, where that includes arbitrarily sharing split packages across module systems. Let me try to accurately conjure the peculiarities of split packages in the OSGi module layer... We support split packages in two ways, via required bundles (i.e., module dependencies) and via bundle fragments. In general, we recommend that if you split your packages that you attach mandatory attributes on them so that importers do not get them by mistake. By requiring a bundle, you can ignore mandatory attributes and get everything a bundle has to offer and combine it with other required bundles that might be exporting other parts of the split packages. Given this, there is a semi-reasonable chance* that we can tell you which bundles are contributing to the "complete" packages, but we have no way of knowing if/when combined split packages are complete. Further, we have to assume that split package parts offered by different bundles do not overlap, because if we allowed them to overlap then we would have shadowing and ordering issues. Again we have no easy way to verify that they do not overlap, so we just assume they don't. Fragments on the other hand are a little trickier, since split packages from them are loaded from the same class loader to provide package private access (which is not afforded for split packages in the require bundle approach). The "host" bundle doesn't really know about fragments and it is possible for fragments to shadow or extend packages in the "host" bundle. Since fragments provide their split packages in an implicit way (i.e., they largely just become part of the host bundle's class path), there is no easy way to determine if a fragment is contributing to a given package or not. (* This feature of fragments also makes it so it is not really possible to know who is contributing classes to a split package in the require bundle case.) I think that is fairly complete description of the situation, sorry if it is somewhat terse. In summary, I wouldn't assume that we can get reasonable interoperability at this level. If you need these types of features, then you should stick to a single modularity framework...preferably OSGi. ;-) But my overall point is, I believe that if we can support interoperability at module- and package-level dependencies across module systems, then I think we would hit most use cases. -> richard > > Sam > > On Apr 25, 2008, at 1:57 PM, Richard S. Hall wrote: > >> Bryan Atsatt wrote: >>> Richard S. Hall wrote: >>>> Gordon Hirsch wrote: >>>>>> 1. Map its dependency declarations into a standard runtime >>>>>> representation. >>>>>> 2. Support a standard search model over its stored modules, using >>>>>> the runtime dependency expressions. >>>>>> 3. Map its stored module data into a standard runtime >>>>>> representation that can be returned by the search. >>>>> >>>>> One challenge lies in defining the "standard runtime >>>>> representations" and "standard search model" in a universal enough >>>>> way to encompass OSGi and other module systems. This implies >>>>> embracing concepts (in these standard representations and search >>>>> model) that were not universally liked by the EG early on. (Split >>>>> packages and package-level import/export come to mind.) >>>> >>>> Package-level import/export seem like a must, but I still don't see >>>> split packages as a necessity. >>> Strongly agreed on import-by-package, and that ModuleDefinition >>> requires a method to enumerate exported packages (and probably an >>> exportsPackage(String packageName) method to enable an efficient >>> Query). >>> On the split package front, however, it seems a little fuzzy to me. >>> If an OSGi implementation of 277 is also going to remain OSGi Rx >>> compliant, it will still need to support split packages, right? If >>> so, don't we need to surface this fact at the 277 level? Perhaps >>> that is as simple as acknowledging that Module.getImportedModules() >>> may return more than one instance that exports a given package. >> >> That all depends on if the 277 API is a subset of OSGi functionality >> or equal to it, I suppose. >> >> -> richard >>> >>> // Bryan From sam at SAMPULLARA.COM Fri Apr 25 17:05:29 2008 From: sam at SAMPULLARA.COM (Sam Pullara) Date: Fri, 25 Apr 2008 17:05:29 -0700 Subject: Updates on Interoperability In-Reply-To: <48126D6A.7000001@ungoverned.org> References: <47F46ABF.1020306@sun.com> <48121BA2.7060001@oracle.com> <48123223.9070803@sas.com> <48123AE3.1090708@ungoverned.org> <4812409C.1030000@oracle.com> <4812459E.2060407@ungoverned.org> <25680D44-7C07-4DC9-841F-12751540ECD4@sampullara.com> <48126D6A.7000001@ungoverned.org> Message-ID: On Apr 25, 2008, at 4:46 PM, Richard S. Hall wrote: > Sam Pullara wrote: >> The vast majority of people that are going to use this system will >> have never heard of OSGI nor care about any of their more subtle >> features and just want an easy way to leverage the vast amount of >> libraries available (virtually all of them are not OSGi modules >> today but are jar files in maven repositories). We shouldn't be >> specing out some sort of uber container but rather a simple way to >> tie code to its dependencies. > > If this is all that people wanted, then we could just define > repositories and associated metadata and forget about all runtime > modularity support, but I strongly disagree that this is all people > want or else there wouldn't be such an upsurge in OSGi interest and > adoption. I guess that is what I am arguing for. I don't see the critical need, outside the internals of application servers and a very few plugin based tools, for runtime modularity support. And if OSGi can load a JSR-277 module then I feel like the interoperability job is done. Sam From heavy at UNGOVERNED.ORG Fri Apr 25 17:19:38 2008 From: heavy at UNGOVERNED.ORG (Richard S. Hall) Date: Fri, 25 Apr 2008 20:19:38 -0400 Subject: Updates on Interoperability In-Reply-To: References: <47F46ABF.1020306@sun.com> <48121BA2.7060001@oracle.com> <48123223.9070803@sas.com> <48123AE3.1090708@ungoverned.org> <4812409C.1030000@oracle.com> <4812459E.2060407@ungoverned.org> <25680D44-7C07-4DC9-841F-12751540ECD4@sampullara.com> <48126D6A.7000001@ungoverned.org> Message-ID: <4812751A.9010809@ungoverned.org> Sam Pullara wrote: > On Apr 25, 2008, at 4:46 PM, Richard S. Hall wrote: >> Sam Pullara wrote: >>> The vast majority of people that are going to use this system will >>> have never heard of OSGI nor care about any of their more subtle >>> features and just want an easy way to leverage the vast amount of >>> libraries available (virtually all of them are not OSGi modules >>> today but are jar files in maven repositories). We shouldn't be >>> specing out some sort of uber container but rather a simple way to >>> tie code to its dependencies. >> >> If this is all that people wanted, then we could just define >> repositories and associated metadata and forget about all runtime >> modularity support, but I strongly disagree that this is all people >> want or else there wouldn't be such an upsurge in OSGi interest and >> adoption. > > I guess that is what I am arguing for. I don't see the critical need, > outside the internals of application servers and a very few plugin > based tools, for runtime modularity support. And if OSGi can load a > JSR-277 module then I feel like the interoperability job is done. I can understand your point of view, but I think you need to take a look around and see what kind of systems out there are creating either their own plugin mechanism or using an existing framework (like OSGi) to create a plugin mechanism. This is a huge area. It is actually one of the biggest impacts that Java made with its class loader approach, it brought code loading to the masses. I could give countless examples that fall outside the "few tools" above (heck, today I just ran across a gaming platform using OSGi), but I won't go into that. I might wholeheartedly accept your view if the powers to be decided to pare down 277 to be just a repository model for Java JAR files, because that could potentially be fairly easy for us to specify and get working as an OSGi module repository (which is currently lacking). I have said before that it makes sense to have 294 give us needed VM support for modules, 277 give us repository support for modules, and 291 give us run-time support for modules. However, that isn't the way we have been going, nor has it ever been as far as I am aware. -> richard > > Sam From sam at SAMPULLARA.COM Fri Apr 25 17:28:36 2008 From: sam at SAMPULLARA.COM (Sam Pullara) Date: Fri, 25 Apr 2008 17:28:36 -0700 Subject: Updates on Interoperability In-Reply-To: <4812751A.9010809@ungoverned.org> References: <47F46ABF.1020306@sun.com> <48121BA2.7060001@oracle.com> <48123223.9070803@sas.com> <48123AE3.1090708@ungoverned.org> <4812409C.1030000@oracle.com> <4812459E.2060407@ungoverned.org> <25680D44-7C07-4DC9-841F-12751540ECD4@sampullara.com> <48126D6A.7000001@ungoverned.org> <4812751A.9010809@ungoverned.org> Message-ID: <96D4A4E0-1770-4843-8B85-A5441364D220@sampullara.com> On Apr 25, 2008, at 5:19 PM, Richard S. Hall wrote: > Sam Pullara wrote: >> On Apr 25, 2008, at 4:46 PM, Richard S. Hall wrote: >>> Sam Pullara wrote: >>>> The vast majority of people that are going to use this system >>>> will have never heard of OSGI nor care about any of their more >>>> subtle features and just want an easy way to leverage the vast >>>> amount of libraries available (virtually all of them are not OSGi >>>> modules today but are jar files in maven repositories). We >>>> shouldn't be specing out some sort of uber container but rather a >>>> simple way to tie code to its dependencies. >>> >>> If this is all that people wanted, then we could just define >>> repositories and associated metadata and forget about all runtime >>> modularity support, but I strongly disagree that this is all >>> people want or else there wouldn't be such an upsurge in OSGi >>> interest and adoption. >> >> I guess that is what I am arguing for. I don't see the critical >> need, outside the internals of application servers and a very few >> plugin based tools, for runtime modularity support. And if OSGi >> can load a JSR-277 module then I feel like the interoperability job >> is done. > > I can understand your point of view, but I think you need to take a > look around and see what kind of systems out there are creating > either their own plugin mechanism or using an existing framework > (like OSGi) to create a plugin mechanism. This is a huge area. It is > actually one of the biggest impacts that Java made with its class > loader approach, it brought code loading to the masses. > I could give countless examples that fall outside the "few tools" > above (heck, today I just ran across a gaming platform using OSGi), > but I won't go into that. I have no problem with them using OSGi and I think it has a lot of value. It just doesn't seem like you need to make it built into Java but rather another container specification. > I might wholeheartedly accept your view if the powers to be decided > to pare down 277 to be just a repository model for Java JAR files, > because that could potentially be fairly easy for us to specify and > get working as an OSGi module repository (which is currently > lacking). I have said before that it makes sense to have 294 give us > needed VM support for modules, 277 give us repository support for > modules, and 291 give us run-time support for modules. This seems like a good plan. We'd have no problem wrapping this up quickly with this as the goal. > However, that isn't the way we have been going, nor has it ever been > as far as I am aware. I agree. I've been trying to push us this way since the beginning. Unfortunately, me: FAIL. Sam From heavy at UNGOVERNED.ORG Fri Apr 25 17:30:14 2008 From: heavy at UNGOVERNED.ORG (Richard S. Hall) Date: Fri, 25 Apr 2008 20:30:14 -0400 Subject: Updates on Interoperability In-Reply-To: <96D4A4E0-1770-4843-8B85-A5441364D220@sampullara.com> References: <47F46ABF.1020306@sun.com> <48121BA2.7060001@oracle.com> <48123223.9070803@sas.com> <48123AE3.1090708@ungoverned.org> <4812409C.1030000@oracle.com> <4812459E.2060407@ungoverned.org> <25680D44-7C07-4DC9-841F-12751540ECD4@sampullara.com> <48126D6A.7000001@ungoverned.org> <4812751A.9010809@ungoverned.org> <96D4A4E0-1770-4843-8B85-A5441364D220@sampullara.com> Message-ID: <48127796.8070908@ungoverned.org> Sam Pullara wrote: > On Apr 25, 2008, at 5:19 PM, Richard S. Hall wrote: >> Sam Pullara wrote: >>> On Apr 25, 2008, at 4:46 PM, Richard S. Hall wrote: >>>> Sam Pullara wrote: >>>>> The vast majority of people that are going to use this system will >>>>> have never heard of OSGI nor care about any of their more subtle >>>>> features and just want an easy way to leverage the vast amount of >>>>> libraries available (virtually all of them are not OSGi modules >>>>> today but are jar files in maven repositories). We shouldn't be >>>>> specing out some sort of uber container but rather a simple way to >>>>> tie code to its dependencies. >>>> >>>> If this is all that people wanted, then we could just define >>>> repositories and associated metadata and forget about all runtime >>>> modularity support, but I strongly disagree that this is all people >>>> want or else there wouldn't be such an upsurge in OSGi interest and >>>> adoption. >>> >>> I guess that is what I am arguing for. I don't see the critical >>> need, outside the internals of application servers and a very few >>> plugin based tools, for runtime modularity support. And if OSGi can >>> load a JSR-277 module then I feel like the interoperability job is >>> done. >> >> I can understand your point of view, but I think you need to take a >> look around and see what kind of systems out there are creating >> either their own plugin mechanism or using an existing framework >> (like OSGi) to create a plugin mechanism. This is a huge area. It is >> actually one of the biggest impacts that Java made with its class >> loader approach, it brought code loading to the masses. > >> I could give countless examples that fall outside the "few tools" >> above (heck, today I just ran across a gaming platform using OSGi), >> but I won't go into that. > > I have no problem with them using OSGi and I think it has a lot of > value. It just doesn't seem like you need to make it built into Java > but rather another container specification. > >> I might wholeheartedly accept your view if the powers to be decided >> to pare down 277 to be just a repository model for Java JAR files, >> because that could potentially be fairly easy for us to specify and >> get working as an OSGi module repository (which is currently >> lacking). I have said before that it makes sense to have 294 give us >> needed VM support for modules, 277 give us repository support for >> modules, and 291 give us run-time support for modules. > > This seems like a good plan. We'd have no problem wrapping this up > quickly with this as the goal. > >> However, that isn't the way we have been going, nor has it ever been >> as far as I am aware. > > I agree. I've been trying to push us this way since the beginning. > Unfortunately, me: FAIL. Join the club... -> richard > > Sam From andyp at bea.com Mon Apr 28 01:33:55 2008 From: andyp at bea.com (Andy Piper) Date: Mon, 28 Apr 2008 09:33:55 +0100 Subject: Updates on Interoperability In-Reply-To: References: <47F46ABF.1020306@sun.com> <48121BA2.7060001@oracle.com> <48123223.9070803@sas.com> <48123AE3.1090708@ungoverned.org> <4812409C.1030000@oracle.com> <4812459E.2060407@ungoverned.org> <25680D44-7C07-4DC9-841F-12751540ECD4@sampullara.com> <48126D6A.7000001@ungoverned.org> Message-ID: <6.2.5.6.2.20080428093224.02f32a20@bea.com> At 01:05 26/04/2008, Sam Pullara wrote: >I guess that is what I am arguing for. I don't see the critical need, >outside the internals of application servers and a very few plugin >based tools, for runtime modularity support. FWIW that's not the message I get when talking to customers. There is a lot of interest in runtime modularity and OSGi in particular. Now whether that's what they actually need is a different question :) andy Notice: This email message, together with any attachments, may contain information of BEA Systems, Inc., its subsidiaries and affiliated entities, that may be confidential, proprietary, copyrighted and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by email and then delete it. From sam at SAMPULLARA.COM Mon Apr 28 08:25:04 2008 From: sam at SAMPULLARA.COM (Sam Pullara) Date: Mon, 28 Apr 2008 08:25:04 -0700 Subject: Updates on Interoperability In-Reply-To: <6.2.5.6.2.20080428093224.02f32a20@bea.com> References: <47F46ABF.1020306@sun.com> <48121BA2.7060001@oracle.com> <48123223.9070803@sas.com> <48123AE3.1090708@ungoverned.org> <4812409C.1030000@oracle.com> <4812459E.2060407@ungoverned.org> <25680D44-7C07-4DC9-841F-12751540ECD4@sampullara.com> <48126D6A.7000001@ungoverned.org> <6.2.5.6.2.20080428093224.02f32a20@bea.com> Message-ID: <2A31947F-AAF0-460A-A25F-8B292B8CA4FB@sampullara.com> Yes, but do they want that from you, in your application server, or do they need to build independent applications that are themselves containers? Sam On Apr 28, 2008, at 1:33 AM, Andy Piper wrote: > At 01:05 26/04/2008, Sam Pullara wrote: >> I guess that is what I am arguing for. I don't see the critical >> need, >> outside the internals of application servers and a very few plugin >> based tools, for runtime modularity support. > > FWIW that's not the message I get when talking to customers. There is > a lot of interest in runtime modularity and OSGi in particular. Now > whether that's what they actually need is a different question :) > > andy > > > > Notice: This email message, together with any attachments, may > contain information of BEA Systems, Inc., its subsidiaries and > affiliated entities, that may be confidential, proprietary, > copyrighted and/or legally privileged, and is intended solely for > the use of the individual or entity named in this message. If you > are not the intended recipient, and have received this message in > error, please immediately return this by email and then delete it. From andyp at bea.com Mon Apr 28 09:49:05 2008 From: andyp at bea.com (Andy Piper) Date: Mon, 28 Apr 2008 17:49:05 +0100 Subject: Updates on Interoperability In-Reply-To: <2A31947F-AAF0-460A-A25F-8B292B8CA4FB@sampullara.com> References: <47F46ABF.1020306@sun.com> <48121BA2.7060001@oracle.com> <48123223.9070803@sas.com> <48123AE3.1090708@ungoverned.org> <4812409C.1030000@oracle.com> <4812459E.2060407@ungoverned.org> <25680D44-7C07-4DC9-841F-12751540ECD4@sampullara.com> <48126D6A.7000001@ungoverned.org> <6.2.5.6.2.20080428093224.02f32a20@bea.com> <2A31947F-AAF0-460A-A25F-8B292B8CA4FB@sampullara.com> Message-ID: <6.2.5.6.2.20080428174742.03074e70@bea.com> At 16:25 28/04/2008, Sam Pullara wrote: >Yes, but do they want that from you, in your application server, or do >they need to build independent applications that are themselves >containers? The latter. There seems to be a great emphasis on lightweight containers these days and a modularity solution plays well in that space. andy Notice: This email message, together with any attachments, may contain information of BEA Systems, Inc., its subsidiaries and affiliated entities, that may be confidential, proprietary, copyrighted and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by email and then delete it. From gordon.hirsch at SAS.COM Mon Apr 28 11:14:48 2008 From: gordon.hirsch at SAS.COM (Gordon Hirsch) Date: Mon, 28 Apr 2008 14:14:48 -0400 Subject: Updates on Interoperability In-Reply-To: <96D4A4E0-1770-4843-8B85-A5441364D220@sampullara.com> References: <47F46ABF.1020306@sun.com> <48121BA2.7060001@oracle.com> <48123223.9070803@sas.com> <48123AE3.1090708@ungoverned.org> <4812409C.1030000@oracle.com> <4812459E.2060407@ungoverned.org> <25680D44-7C07-4DC9-841F-12751540ECD4@sampullara.com> <48126D6A.7000001@ungoverned.org> <4812751A.9010809@ungoverned.org> <96D4A4E0-1770-4843-8B85-A5441364D220@sampullara.com> Message-ID: <48161418.1080400@sas.com> Sam Pullara wrote: > On Apr 25, 2008, at 5:19 PM, Richard S. Hall wrote: ... >> I might wholeheartedly accept your view if the powers to be decided >> to pare down 277 to be just a repository model for Java JAR files, >> because that could potentially be fairly easy for us to specify and >> get working as an OSGi module repository (which is currently >> lacking). I have said before that it makes sense to have 294 give us >> needed VM support for modules, 277 give us repository support for >> modules, and 291 give us run-time support for modules. > > This seems like a good plan. We'd have no problem wrapping this up > quickly with this as the goal. +1, for the record, at least. I too would be happy with the 277/291/294 breakdown described above (but which I have been assuming is a lost cause). If, instead, there is a separate 277 runtime, then that's when strong interoperability (as defined in Bryan's blog entry) becomes critical, simply because of the wide usage of at least one other module system runtime already today. From bryan.atsatt at oracle.com Mon Apr 28 14:13:26 2008 From: bryan.atsatt at oracle.com (Bryan Atsatt) Date: Mon, 28 Apr 2008 14:13:26 -0700 Subject: Updates on Interoperability In-Reply-To: <48126D6A.7000001@ungoverned.org> References: <47F46ABF.1020306@sun.com> <48121BA2.7060001@oracle.com> <48123223.9070803@sas.com> <48123AE3.1090708@ungoverned.org> <4812409C.1030000@oracle.com> <4812459E.2060407@ungoverned.org> <25680D44-7C07-4DC9-841F-12751540ECD4@sampullara.com> <48126D6A.7000001@ungoverned.org> Message-ID: <48163DF6.3010705@oracle.com> Richard S. Hall wrote: > Sam Pullara wrote: >> Did we decide which way it will be compatible? I'm a little >> concerned that implementing something that can use OSGi modules is a >> far bigger task that implementing something that OSGi can use. If we >> are going for total compatibility it seems like we should just use >> OSGi. Personally I wanted something in Java that was simpler and >> cleaner that OSGi-users could leverage if they wanted. But I'm >> coming at this from the we-need-something-like-maven/gem/ivy camp. >> >> The vast majority of people that are going to use this system will >> have never heard of OSGI nor care about any of their more subtle >> features and just want an easy way to leverage the vast amount of >> libraries available (virtually all of them are not OSGi modules today >> but are jar files in maven repositories). We shouldn't be specing >> out some sort of uber container but rather a simple way to tie code >> to its dependencies. > > If this is all that people wanted, then we could just define > repositories and associated metadata and forget about all runtime > modularity support, but I strongly disagree that this is all people > want or else there wouldn't be such an upsurge in OSGi interest and > adoption. > >> I strongly suggest KISS applies to our effort. I write this knowing >> that the committee seems to be heavily in favor of this alternate >> view that we implement the kitchen sink. > > I think our interest is similar, but perhaps our conclusions are > different. Initially, I think my comments were construed as an > argument against split packages in general, but I am really arguing > the same as you (I think) that we don't really need split packages to > be a concept in our "interoperability API", if that is how we are to > view the 277 API. Of course, I am also willing to have the former > argument too, but I will leave that for another time. :-) > > From my point of view, I do not think it is wholly reasonable to > assume that we want to try to accomplish interoperability across > module systems, where that includes arbitrarily sharing split packages > across module systems. Nor do I, but I'm trying to understand the implications if we assume that OSGi will continue to split packages. > > Let me try to accurately conjure the peculiarities of split packages > in the OSGi module layer... > > We support split packages in two ways, via required bundles (i.e., > module dependencies) and via bundle fragments. In general, we > recommend that if you split your packages that you attach mandatory > attributes on them so that importers do not get them by mistake. By > requiring a bundle, you can ignore mandatory attributes and get > everything a bundle has to offer and combine it with other required > bundles that might be exporting other parts of the split packages. > Given this, there is a semi-reasonable chance* that we can tell you > which bundles are contributing to the "complete" packages, but we have > no way of knowing if/when combined split packages are complete. > Further, we have to assume that split package parts offered by > different bundles do not overlap, because if we allowed them to > overlap then we would have shadowing and ordering issues. Again we > have no easy way to verify that they do not overlap, so we just assume > they don't. > > Fragments on the other hand are a little trickier, since split > packages from them are loaded from the same class loader to provide > package private access (which is not afforded for split packages in > the require bundle approach). The "host" bundle doesn't really know > about fragments and it is possible for fragments to shadow or extend > packages in the "host" bundle. Since fragments provide their split > packages in an implicit way (i.e., they largely just become part of > the host bundle's class path), there is no easy way to determine if a > fragment is contributing to a given package or not. (* This feature of > fragments also makes it so it is not really possible to know who is > contributing classes to a split package in the require bundle case.) Ok, so first I assume it would make sense for an OSGi Repository to simply never return a ModuleDefinition for a fragment. And the "you don't know what you've got till it's resolved" problem is going to be present in any module system for which package name is not a required component of a dependency expression. But as long as re-export is not the default behavior (it isn't), doesn't this really just boil down to the leakage problem? And this is addressed in 277 the same as it is in OSGi: you must declare such "leaks" as re-exports ("uses"). Not perfect, but probably good enough. So I think we're left with the issue of a single search visiting multiple loaders that can return the same package. This won't happen in the fragment case, since the "split" gets mended at runtime, but it will in the require-bundle case. And this will manifest as multiple entries in the import list which export the same package. But even this isn't an issue, as long as dependency resolution relies solely on declared exports and re-exports of a module, and not on what is available by browsing the imports of that module. So I'm back to my original statement that we just need to acknowledge that package duplication may exist in an import list. Or am I missing something here? > > I think that is fairly complete description of the situation, sorry if > it is somewhat terse. In summary, I wouldn't assume that we can get > reasonable interoperability at this level. If you need these types of > features, then you should stick to a single modularity > framework...preferably OSGi. ;-) > > But my overall point is, I believe that if we can support > interoperability at module- and package-level dependencies across > module systems, then I think we would hit most use cases. > > -> richard > >> >> Sam >> >> On Apr 25, 2008, at 1:57 PM, Richard S. Hall wrote: >> >>> Bryan Atsatt wrote: >>>> Richard S. Hall wrote: >>>>> Gordon Hirsch wrote: >>>>>>> 1. Map its dependency declarations into a standard runtime >>>>>>> representation. >>>>>>> 2. Support a standard search model over its stored modules, >>>>>>> using the runtime dependency expressions. >>>>>>> 3. Map its stored module data into a standard runtime >>>>>>> representation that can be returned by the search. >>>>>> >>>>>> One challenge lies in defining the "standard runtime >>>>>> representations" and "standard search model" in a universal >>>>>> enough way to encompass OSGi and other module systems. This >>>>>> implies embracing concepts (in these standard representations and >>>>>> search model) that were not universally liked by the EG early on. >>>>>> (Split packages and package-level import/export come to mind.) >>>>> >>>>> Package-level import/export seem like a must, but I still don't >>>>> see split packages as a necessity. >>>> Strongly agreed on import-by-package, and that ModuleDefinition >>>> requires a method to enumerate exported packages (and probably an >>>> exportsPackage(String packageName) method to enable an efficient >>>> Query). >>>> On the split package front, however, it seems a little fuzzy to me. >>>> If an OSGi implementation of 277 is also going to remain OSGi Rx >>>> compliant, it will still need to support split packages, right? If >>>> so, don't we need to surface this fact at the 277 level? Perhaps >>>> that is as simple as acknowledging that Module.getImportedModules() >>>> may return more than one instance that exports a given package. >>> >>> That all depends on if the 277 API is a subset of OSGi functionality >>> or equal to it, I suppose. >>> >>> -> richard >>>> >>>> // Bryan > From Stanley.Ho at Sun.COM Mon Apr 28 22:06:17 2008 From: Stanley.Ho at Sun.COM (Stanley M. Ho) Date: Mon, 28 Apr 2008 22:06:17 -0700 Subject: Supporting OSGi Bundles in the Java Module System Message-ID: <4816ACC9.1010803@sun.com> Dear 277 observers, The original message is too big for this observer list, so I forward the original message with first attachment only. If any of you are interested in getting the latest API javadoc, please send email to jsr-277-comments at jcp.org. - Stanley ---- Dear 277 experts, The first attachment is a draft spec for supporting OSGi bundles in the Java Module System. This is based on the past EG discussion in particular the proposals and inputs from Glyn, Richard, and Bryan. This is a work in progress and is expected to evolve based on further inputs from this EG. The second attachment is the latest API javadoc for your reference. We're currently updating the JSR 277 EDR2 specification and APIs to make the distinction between the framework/abstractions for the Java Module System more clear, and we expect they will be available for the EG to review in a few weeks after JavaOne. In the meantime, the draft is intended to provide enough information for the EG to review and provide inputs without the EDR2. Numerous EG members are preparing to attend and/or speak at JavaOne next week, so some of them may well not respond until afterwards. Finally, thanks to Bryan and his blog that serves really well to set up the context for this discussion. - Stanley and Alex -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/jsr277-eg-observer/attachments/20080428/177cb076/attachment.html From gordon.hirsch at SAS.COM Tue Apr 29 07:43:05 2008 From: gordon.hirsch at SAS.COM (Gordon Hirsch) Date: Tue, 29 Apr 2008 10:43:05 -0400 Subject: Supporting OSGi Bundles in the Java Module System In-Reply-To: <4816930D.4080106@sun.com> References: <4816930D.4080106@sun.com> Message-ID: <481733F9.1050902@sas.com> Stanley, The javadoc zip file was stripped by our firewall. Could you either resend with a different extension or post it to the EG homepage at jcp.org? Stanley M. Ho wrote: > Dear 277 experts, > > The first attachment is a draft spec for supporting OSGi bundles in the > Java Module System. This is based on the past EG discussion in > particular the proposals and inputs from Glyn, Richard, and Bryan. This > is a work in progress and is expected to evolve based on further inputs > from this EG. > > The second attachment is the latest API javadoc for your reference. > > We're currently updating the JSR 277 EDR2 specification and APIs to make > the distinction between the framework/abstractions for the Java Module > System more clear, and we expect they will be available for the EG to > review in a few weeks after JavaOne. In the meantime, the draft is > intended to provide enough information for the EG to review and provide > inputs without the EDR2. > > Numerous EG members are preparing to attend and/or speak at JavaOne next > week, so some of them may well not respond until afterwards. > > Finally, thanks to Bryan and his blog that serves really well to set up > the context for this discussion. > > - Stanley and Alex > > > ------------------------------------------------------------------------ > > > Supporting OSGi Bundles in the Java Module System > > * > > Mandy Chung and Stanley Ho > Version 0.1 > April 28, 2008 > Copyright 2008 Sun Microsystems, Inc. > > * JSR 277 Early Draft defines the framework for the Java Module System > and provides the API to access Java modules such as searching for > modules, loading and initializing a module, and also managing its life > time. JSR 277 enables one or more module systems run on a single Java > virtual machine. This document^1 defines how JSR 277 supports OSGi > bundles as defined in JSR 291. This document is a work in progress and > is expected to evolve based on more input from the Expert Group. > > Section 1 describes various definitions in the Java Module System. > Section 2 describes the requirements for supporting OSGi bundles in the > Java Module System and section 3 describes the proposed API changes. > Section 4 specifies how to map an OSGi bundle to a ModuleDefinition so > they can be consumed by other modules in the Java Module System. Section > 5 describes the repository mechanism that enables the OSGi module system > to be plugged into the Java Module System framework to make OSGi bundles > accessible by other module systems at runtime. Appendix A shows the code > example illustrating how a JAM module system implementation uses the JSR > 277 API to search and import OSGi bundles. > > ^1 This draft is built upon from the basis of the past EG discussions, > proposals and suggestions for the interoperability support especially > the ideas and invaluable inputs from Glyn Normington, Richard Hall and > Bryan Atsatt. > > > 0. Current Status > > The JSR 277 EDR2 is being updated to make the distinction between the > framework/abstractions for the Java Module System and the JAM module > system (See Section 1) clearer. > > This section highlights the main items that remain to be sorted out. > > 1. Versioning scheme (see Section 4.2) > 2. Security > 3. Java Module Events > > > 1. Definitions > > /Java module/ > > A development abstraction that exists at compile-time in the Java > programming language and is reified in the Java Virtual Machine. > > /Module Definition/ > > A deployment abstraction that contains metadata, classes and resources, > and is reified as ModuleDefinition as part of a Java Module System > implementation. > > /JAM module/ > > A concrete, portable deployment format which implements the Module > Definition. Amongst other details, it has metadata, classes, resources, > and can contain embedded JAR files and native libraries. The > distribution format is called JAM (JAva Module) that is based on the JAR > file format. > > /Java Module System/ > > A set of runtime abstractions that includes ModuleSystem, Repository, > ModuleDefinition (see "Module Definition" above), Module, > ImportDependency and others. > > /JAM Module System/ > > A concrete implementation of the Java Module System that supports JAM > modules. It is the default implementation of the Java Module System for > the Java SE platform. > > /OSGi Module System/ > > A concrete implementation of the Java Module System by an OSGi container. > > > 2. Requirements > > 1. It shall be possible for an OSGi container to implement the Java > Module System. > 2. It shall be possible for a JAM module to express an import > dependency on any Module Definition in any Java Module System. > > Below provides an example to illustrate how a JAM module imports OSGi > bundles. > > A Wombat application is a JAM module that processes shopping orders for > a website. It depends on the Apache Xerces XML parser and the Apache > Derby and both of them are available as OSGi bundles. The module > definition of the Wombat application looks like this: > > // > // com/wombat/app/module-info.java > // > @Version("1.0") > @ImportModules({ > @ImportModule(name="org.apache.xerces.parsers", version="2.6.6+") > @ImportModule(name="org.apache.derby", version="10.0+") > }) > module com.wombat.app; > > > This Wombat application is packaged as a JAM file named > "com.wombat.app-1.0.jam" and the OSGi bundles it depends on are packaged > as JAR files containing the following manifests,^2 per the OSGi > specifications: > > org.apache.xerces.parsers: > Bundle-SymbolicName: org.apache.xerces.parsers > Bundle-Version: 2.9.10 > > Export-Package: org.apache.xerces.parsers; version=2.6.6; uses="org.apache.commons.logging", > org.apache.xerces.jaxp; version=2.6.6, > org.apache.xerces.framework; version=2.6.6, > org.apache.xerces.readers; version=2.6.6, > org.apache.xerces.utils; version=2.6.6, > org.apache.commons.logging; version=1.0.4 > Import-Package: javax.xml.parsers; version=1.2.0, > org.w3c.dom; version=1.0.0, > org.xml.sax; version=2.0.1 > Require-Bundle: org.osgi.util.xml; version=1.0.0; visibility:=reexport; resolution:=optional > > org.apache.derby: > Bundle-SymbolicName: org.apache.derby > Bundle-Version: 10.0.2 > Export-Package: org.apache.derby.authentication, > org.apache.derby.database, > org.apache.derby.io, > org.apache.derby.jdbc > Import-Package: java.sql > > > ^2 These example manifests are for illustration purpose and they do not > represent the manifest in the actual Apache Xerces XML Parser and Apache > Derby bundles. > > > 3. Proposed API Changes > > This section describes the proposed API changes for supporting OSGi > bundles in the Java Module System. > > > 3.1 ModuleSystem class > > The ModuleSystem specification is updated to support multiple > implementations running in a single Java virtual machine. This update is > necessary to meet the requirement (1). See JSR 277 EDR2 for the full > specification. > > A ModuleSystem implementation is responsible for creation, management, > and release of Java Modules owned by this module system. The > ModuleSystem specification does not define the resolution algorithm and > the distribution format that a module system supports. Instead, the > resolution algorithm and the distribution format are specific to a > ModuleSystem implementation. > > > 3.1.1 Module Initialization > > This section is a clarification to the JSR 277 EDR about the > initialization of a Module instance. > > The ModuleSystem.getModule(ModuleDefinition) method returns a /fully > initialized/ Module instance^3 for a given ModuleDefinition. A Module > instance is fully initialized if the following conditions are all met: > > 1. Its import dependencies have been resolved. ModuleDefinitions for > the imported Java modules satisfying the import dependencies and > all its constraints are found using the algorithm defined by this > ModuleSystem. > 2. Module instances for its imports are successfully instantiated and > initialized by its owning ModuleSystem. > 3. This ModuleSystem has performed its own type consistency checking > successfully in the Module instance. > 4. If this ModuleSystem supports an initializer^4 to be invoked > before a Module instance is fully initialized, the initializer for > this Module instance is invoked. > > In addition, a ModuleSystem can support import constraints specific to > its algorithm to tailor the resolution of the imports for a > ModuleDefinition. The constraints specified in a ModuleDefinition is > only known to its owning ModuleSystem and other ModuleSystem > implementations are not required to understand them. > > ^3 Initializing an OSGi bundle is equivalent to resolving, wiring and > starting an OSGi bundle. > > ^4 The ModuleSystem specification does not define a generic initializer > mechanism for ModuleDefinitions. The JAM module system supports the > module initializer through the ModuleInitializer API and the > ModuleInitializerClass annotation (see JSR 277 EDR 2). The OSGi module > system supports the module initializer through the bundle activator. > > > 3.1.2 Constraint Checking > > A new method ModuleSystem.getModules is added to allow a ModuleSystem > implementation to instantiate Module instances for multiple > ModuleDefinitions in the same resolution and also to enforce constraints > specified in these Modules. > > ModuleSystem class: > /** > * Returns the list of Module instances for the imports > * in the same resolution for the specified importer. > * The returned Module instances are instantiated and initialized > * using the algorithm specific to this ModuleSystem. > * > * This method is called by a ModuleSystem when initializing > * a Module instance (importer) that imports Modules (imports) > * from this ModuleSystem. > */ > public List getModules(ModuleDefinition importer, > List imports) > throws ModuleInitializationException; > > > OSGi allows to put constraints on the importer through the metadata for > an exported package in another bundle. The OSGi module system can > enforce the constraints on the importer in the implementation of this > getModules method. The importer can be a ModuleDefinition from other > ModuleSystem. > > For example, the Apache Xerces XML parser bundle in Example 1 of Section > 1 exports the org.apache.xerces.parsers package whose export definition > has a "use" constraint. This export definition puts a constraint on the > importer of the org.apache.xerces.parsers package to use the > org.apache.commons.logging package wired to the > org.apache.xerces.parsers package in the same resolution. The 1.0 > version of the Wombat application imports the org.apache.xerces.parsers > bundle and the org.apache.derby bundle. The org.apache.xerces.parsers > package will get wired to the version 1.0.4 logging package and the > constraint is satisfied and thus it can be wired successfully. > > Let's say a new version of the Wombat application (say version 1.2) is > updated and it depends on an additional Apache Commons Logging utility > which is also an OSGi bundle. > > // > // com/wombat/app/module-info.java > // > @Version("1.2") > @ImportModules({ > @ImportModule(name="org.apache.xerces.parsers", version="2.6.6+") > @ImportModule(name="org.apache.derby", version="10.0+") > @ImportModule(name="org.apache.commons.logging", version="2.0+") > }) > module com.wombat.app; > > > The bundle manifest header for the Apache Commons Logging utility is: > > org.apache.commons.logging: > Bundle-SymbolicName: org.apache.commons.logging > Bundle-Version: 2.0 > Export-Package: org.apache.commons.logging; version=2.0 > > > The 1.2 version of the Wombat application imports the > org.apache.commons.logging bundle that violates the constraint if the > org.apache.xerces.parsers package is wired to the version 1.0.4 logging > package but the Wombat application requires the version 2.0 logging > package. The version 1.2 of the Wombat application should fail to > initialize due to this constraint. In other words, the getModules() > method of the OSGi module system should throw a > ModuleInitializationException when the OSGi module system determines > that the constraint is violated. > > > 3.2 ModuleDefinition class > > Two new methods are added in the ModuleDefinition class to return the > exported packages and member packages respectively. The export and > member definitions contained in the OSGi metadata are in package > granularity. In addition, a new PackageDefinition class is added to > allow an OSGi bundle to expose the metadata for an exported package. > This is required to meet the requirements (1) and (2). > > ModuleDefinition class: > public abstract Set getExportedPackageDefinitions(); > public abstract Set getMemberPackageDefinitions(); > > PackageDefinition class: > public abstract class PackageDefinition { > public String getName(); > public Version getVersion(); > public Set getAttributeNames(); > public String getAttribute(String name); > } > > > The version and attributes in the PackageDefinition are optional and for > information only and to aid diagnosis. A ModuleSystem importing a > ModuleDefinition from other ModuleSystem is not required to understand > the version and attributes of its exported PackageDefinitions. > > > 3.3 ImportDependency class > > The ImportDependency class is updated as follows. > > 1. An import type is added to indicate if the ImportDependency is for > module-level or package-level. "module" and "package" are two > defined import types. > 2. The ImportDependency class can have attributes for a module system > to include additional information about an import dependency. > > This is required to meet the requirement (1). > > ImportDependency class: > // static factory methods > public static ImportDependency > newImportModuleDependency(String moduleName, > VersionConstraint constraint, > boolean reexport, > boolean optional, > Map attributes); > public static ImportDependency > newImportPackageDependency(String packageName, > VersionConstraint constraint, > boolean optional, > Map attributes); > public String getType(); > public String getAttribute(String name); > public Set getAttributeNames(); > > > > 4. Mapping OSGi Bundles to ModuleDefinitions > > This section specifies how an OSGi bundle maps to a ModuleDefinition to > expose in the Java Module System so that other ModuleDefinitions can > import them. > > > 4.1 Bundle-SymbolicName > > The bundle symbolic name maps to a module name (i.e. > ModuleDefinition.getName()). The directives for the Bundle-SymbolicName > header maps to the module attributes. > > For example: > > Bundle-SymbolicName: com.acme.foo > > > The Java Module System and OSGi do not enforce any naming convention. It > is encouraged to use the reverse domain name convention to name OSGi > bundles and Java modules to avoid the name space conflict. > > > 4.2 Bundle-Version > > A bundle version maps to a module version^5 (i.e. > ModuleDefinition.getVersion()). Bundle-Version is an optional header and > the default value is 0.0.0. The bundle version format is: > > major[.minor[.micro]][.qualifier] > > > The module version format is: > > major[.minor[.micro[.update]]][-qualifier] > > > If the bundle version contains a qualifier, the delimiter prior to the > qualifier will need to be changed from a period ('.') to a dash ('-'). > For example, the bundle version 3.1.4.pi maps to the module version > 3.1.4-pi. > > ^5 *Difference in OSGi and JSR 277 versioning scheme* > The versioning scheme defined in the JSR 277 Early Draft is loosely > based on the existing versioning schemes that are widely used in the > Java platform today and for backward compatibility reason (see JSR 277 > EDR chapter 5). Many existing products including the JDK use the version > format with the micro, minor, micro, and update numbers. A version with > no qualifier is higher than the version with the same version number but > with a qualifier. This is more intuitive and has been the convention the > JDK has been using. The Expert Group has discussed the difference with > the OSGi versioning scheme and agreed with the JSR 277 versioning scheme > defined. > > *Open Issue:* > Need to investigate the version mapping due to the difference in the > comparison of two versions - one with a qualifier and the other without > a qualifier. > > When two bundle versions have the same major, minor, and micro numbers, > the bundle version that has a qualifier is lower than the bundle version > that has no qualifier. e.g. 7.8.9.pi < 7.8.9 > > When two module versions have the same major, minor, micro, and update > numbers, the module version that has a qualifier is higher than the > module version that has no qualifier. e.g. 7.8.9 < 7.8.9-b04-alpha > > One possible solution would be: > > OSGi version JSR 277 Version > > major.minor.micro -> major.minor.micro-0 > major.minor.micro.qualifier -> major.minor.micro-1-qualifier > > > > 4.3 Import-Package > > The Import-Package header maps to the import dependencies for a Java > module (i.e. ModuleDefinition.getImportDependencies()). > > The Import-Package header contains one or more import definitions, each > of which describes a single package import for a bundle. Each import > definition maps to an ImportDependency instance with the "package" type > as follows: > > * The import package name maps to ImportDependency.getName(). > * ImportDependency.getType() returns "package". > * The "resolution" directive maps to the "optional" input parameter > of the ImportDependency constructor; true if > "resolution:=optional" is specified and false otherwise. > * The "version" attribute maps to the VersionConstraint of the > ImportDependency as described in the version-range section below. > * All other attributes specified in the import definition including > the bundle-symbolic-name and bundle-version attributes map to the > attributes in the ImportDependency. > > Example, > > Import-Package: p; > version="[1.23, 1.24]"; > resolution:=optional > > > maps to the import dependencies equivalent to: > > ImportDependency importP = > ImportDependency.newImportPackageDependency("p", > VersionConstraint.valueOf("[1.23,1.24]", > true /* optional */); > > > > The version-range mapping: > > The OSGi version-range maps to a VersionConstraint as follows: > > * If a version has no qualifier, the mapping is exact. For example, > a bundle version range [1.1, 1.2) maps to a module version range > [1.1, 1.2). If there is a qualifier, then section 4.2 should be used. > * If a bundle version range is specified as a single version, it > will map to an open version range. For example, the bundle version > range "1.23" maps to the module version range "1.23+". > > > 4.4 Export-Package > > The Export-Package header maps to the exported package definitions for a > Java module (i.e. ModuleDefinition.getExportedPackageDefinitions()). > > The Export-Package header contains one or more export definitions, each > of which describes a single package export for a bundle. Each export > definition maps to a PackageDefinition instance as follows: > > * The package name maps to PackageDefinition.getName(). > * The "include" and "exclude" directive along with the classes in > the exported package are the input to determine the returned value > of the ModuleDefinition.isClassExported() method. > * The "version" attribute maps to PackageDefinition.getVersion(). > * Other attributes and directives including the "use" directive in > the export definition can map to the attributes in the > PackageDefinition. > * The exported package definition is also a member package > definition for the module. > > > 4.5 Require-Bundle > > The required bundles maps to the import dependencies for a Java module > (i.e. ModuleDefinition.getImportDependencies()). Each required bundle > maps to an ImportDependency instance with the "module" type: > > * The bundle symbolic name of the required bundle maps to > ImportDependency.getName(). > * ImportDependency.getType() returns "module". > * The "visibility" directive maps to > ImportDependency.isReexported(). The isReexported() method returns > true if "visibility:=reexport" is specified; false otherwise. > * The "resolution" directive maps to ImportDependency.isOptional(). > The isOptional() method returns true if "resolution:=optional" is > specified; false otherwise. > * The "bundle-version" attribute maps to the VersionConstraint of > the ImportDependency as described in the Version Range Mapping > section of section 4.4. > > Example, > > Require-Bundle: com.acme.facade;visibility:=reexport, > com.acme.bar;visibility:=reexport;resolution:=optional > > > maps to the import dependencies equivalent to: > > ImportDependency facade = > ImportDependency.newImportModuleDependency("com.acme.facade", > VersionConstraint.DEFAULT, > true /* reexport */, > false /* optional */); > > ImportDependency bar = > ImportDependency.newImportModuleDependency("com.acme.bar", > VersionConstraint.DEFAULT, > true /* reexport */, > true /* optional */); > > > > 4.6 Other Manifest Headers > > The above sections cover the manifest headers that provide the metadata > for the OSGi resolver (see Section 3.5 of the OSGi Service Platform Core > Specification Release 4, Version 4.1). The other bundle manifest > headers, including Bundle-Vendor, Bundle-Description and > DynamicImport-Package, do not affect the module resolution. This > specification does not need to define how to map them to > ModuleDefinition. However, implementations are encouraged to include > them as the module attributes (i.e. ModuleDefinition.getAttribute()) as > additional information to aid diagnosis. > > > 4.7 Fragment Bundles > > Fragment bundles are not exposed as ModuleDefinitions in the Java Module > System. Instead, they are exposed as part of the ModuleDefinition of its > host bundle to which they are attached to. > > > 4.8 Example > > The manifest in the org.apache.xerces.parsers bundle shown in Example 1 > of Section 2 is: > > org.apache.xerces.parsers: > Bundle-SymbolicName: org.apache.xerces.parsers > Bundle-Version: 2.9.10 > > Export-Package: org.apache.xerces.parsers; version=2.6.6; uses="org.apache.commons.logging", > org.apache.xerces.jaxp; version=2.6.6, > org.apache.xerces.framework; version=2.6.6, > org.apache.xerces.readers; version=2.6.6, > org.apache.xerces.utils; version=2.6.6, > org.apache.commons.logging; version=1.0.4 > Import-Package: javax.xml.parsers; version=1.2.0, > org.w3c.dom; version=1.0.0, > org.xml.sax; version=2.0.1 > Require-Bundle: org.osgi.util.xml; version=1.0.0; visibility:=reexport; resolution:=optional > > > Below shows the ModuleDefinition for this OSGi bundle when exposed in > the Java Module System. For clarity, we only show one Export-Package > entry and one Import-Package entry. > > Method of ModuleDefinition Returned Value Bundle Manifest Header > getName() "org.apache.xerces.parsers" Bundle-SymbolicName > getVersion() Version.valueOf("2.9.10") Bundle-Version > getImportDependencies() > > ImportDependency.newImportModuleDependency("org.osgi.util.xml", > VersionConstraint.valueOf("1.0.0+"), > true /* reexport */, > true /* optional */); > > Require-Bundle > > ImportDependency.newImportPackageDependency("javax.xml.parsers", > VersionConstraint.valueOf("1.2.0+"), > false /* optional */); > > Import-Package > getExportedPackageDefinition() PackageDefinition with: > > name="org.apache.xerces.parsers", > version="2.6.6" > attributes=(("uses", "org.apache.commons.logging")} > > Export-Package > > > 5. Enabling the OSGi Module System in the Framework > > To enable the OSGi module system in the framework, an OSGi Repository > implementation should be plugged into the runtime. The OSGi repository > is responsible for discovering OSGi bundles and exposing them as > ModuleDefinitions so that OSGi bundles are available for other module > systems to use. A ModuleSystem implementation finds OSGi bundles via the > repository delegation model and therefore the OSGi repository has to be > configured as an ancestor of the repository where the Java module > depending on OSGi bundles resides. Otherwise, it will fail to find the > importing OSGi bundles. > > One or more repositories can be created for the OSGi module system and > interoperate with other module systems via the repository delegation model. > > The following picture depicts the repository tree set up to run the > Wombat application described in Example 1 of Section 2. The OSGi > repository is configured as the parent of the application repository. > This particular OSGi repository implementation includes the Apache Felix > OSGi runtime for loading and resolving OSGi bundles. > > > ------------------------ > | Bootstrap Repository | > ------------------------ > | > | > -------------------- > | Global Repository | > -------------------- > | > | xxxxxxxxxxxxxxxx > ------------------- x Apache Felix x > | OSGi Repository | <====> x OSGi x ---> org.apache.xerces.parsers version 2.10 > ------------------- x Runtime x org.apache.derby version 10.0.2 > | xxxxxxxxxxxxxxxx org.apache.derby version 9.1 > | > ----------------- > | Application | > | Repository | > ----------------- > com.wombat.app-1.0.jam > > > For example, the following command will launch the Wombat application in > the "/wombat-application" directory. > > > > java -repository /wombat-application -module com.wombat.app > > > The com.wombat.app module is located in the application repository which > is a repository for the JAM module system. The JAM module system first > loads the com.wombat.app module. To initialize this JAM module, the JAM > module system looks at its import dependencies and performs a search of > two imported OSGi bundles through the repository delegation model from > the OSGi repository. The JAM module system then requests the OSGi module > system associated with the OSGi repository to get the Module instances > for the imports (see Section 3.1). Once the imported OSGi bundles are > loaded and started, the initialization process for the com.wombat.app > module continues. > > > 6. Delegation of Class and Resource Loading > > The class loader of a Module instance (i.e. returned by the > Module.getClassLoader() method) must be capable to load all classes and > resources in the module. > > As described in Section 5 above, when the com.wombat.app JAM module is > initialized, two other Module instances for its imports representing the > resolved OSGi bundles are created in the system. The class loader for > the Module instance for the org.apache.xerces.parsers bundle must be > capable to load all classes and resources in it. Similarly for the > org.apache.derby bundle. > > > 7. Security > > TBD. > > > 8. Implementation Notes > > The following are the notes for the implementation of the OSGi > repository and the implementation of the Java Module System. > > 1. The repository delegation hierarchy is a tree and thus cycles > involving multiple module systems are inherently unsupported. > 2. The repository delegation model is designed to offer isolation > between ModuleDefinitions in different repositories. Although a > module system could have access to multiple repositories, the > module system should adhere to the repository delegation module. > Otherwise, it would break the isolation model the repository > provides. > 3. Java SE 7 is expected to have parallel class loading support. All > module systems plugged in the framework are required have parallel > class loading enabled in order to avoid potential deadlocks. > 4. Split packages without shadowing are explicitly permitted in OSGi > whereas the JAM module system does not allow split packages. So > importing OSGi bundles with split packages in a JAM module will > result in module initialization failure. > 5. When a resolution involves multiple module systems, a module > system implementation should take the possible potential issues > (such as hanging) into account in their design to prevent a > foreign module system from bringing down the module system or the > entire JVM. A module system could implement time out policy to > prevent from hanging the module system. > > > 8. References > > * OSGi Service Platform Core Specification Release 4, Version 4.1 > April 2007 > * JSR 277 Interoperation with OSGi by Richard Hall and Glyn > Normington, Apr 24, 2006. > * Module System Interoperability by Richard Hall, May 11, 2006. > > > A. Appendix > > The following illustrates how the JAM module system implementation uses > the JSR 277 API to search and import OSGi bundles This example does not > cover the exact resolution algorithm and the implementation of the > module system runtime. > Example 1 of Section 1 has a com.wombat.app JAM module importing two > OSGi bundles. > > com.wombat.app ----> org.apache.xerces.parsers > | > |---> org.apache.derby > > // the Java runtime will first find the module com.wombat.app > ModuleDefinition wombatModDef = repository.find("com.wombat.app", "1.0+"); > // This call blocks until com.wombat.app is fully initialized > Module wombat = wombatModDef.getModuleInstance(); > > > /JAM Module System Runtime/ > > // find imports for wombat > List imports = ...; > Map> foreignImportMap = new HashMap....; > for (ModuleDefinition md : imports) { > if (md is from a foreign module system (ms)) { > // the case to add a new entry in foreignImportMap > // is not shown in this pseudo-code. > // > // org.apache.xerces.parsers and org.apache.derby will be added in this map > foreignImportMap.get(ms).add(md); > } else { > // JAM resolution algorithm > ... > } > } > > // Gets imports from foreign module systems > // Should do this in a separate thread since this is a synchronous call > for (ModuleSystem ms : foreignImportMap.keySet()) { > // The getModules() method will allow a module system to know if > // a set of ModuleDefinition are resolved in the same resolution. > List imports = ms.getModules(wombatModDef, foreignImportMap.get(ms)); > // interconnect the imports with wombat > ... > } > > // continue the module initialization process such as shadow validatation > // and execute initializers > ... > > // Module initialization completed > if (succeeded) { > com.wombat.app is now fully initialized > } else { > // signal the getModule method to throw ModuleInitializationException > ... > } > From Stanley.Ho at sun.com Tue Apr 29 08:49:55 2008 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Tue, 29 Apr 2008 08:49:55 -0700 Subject: Supporting OSGi Bundles in the Java Module System In-Reply-To: <481733F9.1050902@sas.com> References: <4816930D.4080106@sun.com> <481733F9.1050902@sas.com> Message-ID: <481743A3.5090305@sun.com> Gordon, I have posted the file to the EG home page [1]. - Stanley [1] http://www.jcp.org/en/eg/download/jsr-277-edr2-api-04282008.zip?id=277&fileId=3786 From heavy at UNGOVERNED.ORG Tue Apr 29 08:50:39 2008 From: heavy at UNGOVERNED.ORG (Richard S. Hall) Date: Tue, 29 Apr 2008 11:50:39 -0400 Subject: Updates on Interoperability In-Reply-To: <48163DF6.3010705@oracle.com> References: <47F46ABF.1020306@sun.com> <48121BA2.7060001@oracle.com> <48123223.9070803@sas.com> <48123AE3.1090708@ungoverned.org> <4812409C.1030000@oracle.com> <4812459E.2060407@ungoverned.org> <25680D44-7C07-4DC9-841F-12751540ECD4@sampullara.com> <48126D6A.7000001@ungoverned.org> <48163DF6.3010705@oracle.com> Message-ID: <481743CF.2020102@ungoverned.org> Bryan Atsatt wrote: > Richard S. Hall wrote: >> Sam Pullara wrote: >>> Did we decide which way it will be compatible? I'm a little >>> concerned that implementing something that can use OSGi modules is a >>> far bigger task that implementing something that OSGi can use. If >>> we are going for total compatibility it seems like we should just >>> use OSGi. Personally I wanted something in Java that was simpler >>> and cleaner that OSGi-users could leverage if they wanted. But I'm >>> coming at this from the we-need-something-like-maven/gem/ivy camp. >>> >>> The vast majority of people that are going to use this system will >>> have never heard of OSGI nor care about any of their more subtle >>> features and just want an easy way to leverage the vast amount of >>> libraries available (virtually all of them are not OSGi modules >>> today but are jar files in maven repositories). We shouldn't be >>> specing out some sort of uber container but rather a simple way to >>> tie code to its dependencies. >> >> If this is all that people wanted, then we could just define >> repositories and associated metadata and forget about all runtime >> modularity support, but I strongly disagree that this is all people >> want or else there wouldn't be such an upsurge in OSGi interest and >> adoption. >> >>> I strongly suggest KISS applies to our effort. I write this knowing >>> that the committee seems to be heavily in favor of this alternate >>> view that we implement the kitchen sink. >> >> I think our interest is similar, but perhaps our conclusions are >> different. Initially, I think my comments were construed as an >> argument against split packages in general, but I am really arguing >> the same as you (I think) that we don't really need split packages to >> be a concept in our "interoperability API", if that is how we are to >> view the 277 API. Of course, I am also willing to have the former >> argument too, but I will leave that for another time. :-) >> >> From my point of view, I do not think it is wholly reasonable to >> assume that we want to try to accomplish interoperability across >> module systems, where that includes arbitrarily sharing split >> packages across module systems. > Nor do I, but I'm trying to understand the implications if we assume > that OSGi will continue to split packages. >> >> Let me try to accurately conjure the peculiarities of split packages >> in the OSGi module layer... >> >> We support split packages in two ways, via required bundles (i.e., >> module dependencies) and via bundle fragments. In general, we >> recommend that if you split your packages that you attach mandatory >> attributes on them so that importers do not get them by mistake. By >> requiring a bundle, you can ignore mandatory attributes and get >> everything a bundle has to offer and combine it with other required >> bundles that might be exporting other parts of the split packages. >> Given this, there is a semi-reasonable chance* that we can tell you >> which bundles are contributing to the "complete" packages, but we >> have no way of knowing if/when combined split packages are complete. >> Further, we have to assume that split package parts offered by >> different bundles do not overlap, because if we allowed them to >> overlap then we would have shadowing and ordering issues. Again we >> have no easy way to verify that they do not overlap, so we just >> assume they don't. >> >> Fragments on the other hand are a little trickier, since split >> packages from them are loaded from the same class loader to provide >> package private access (which is not afforded for split packages in >> the require bundle approach). The "host" bundle doesn't really know >> about fragments and it is possible for fragments to shadow or extend >> packages in the "host" bundle. Since fragments provide their split >> packages in an implicit way (i.e., they largely just become part of >> the host bundle's class path), there is no easy way to determine if a >> fragment is contributing to a given package or not. (* This feature >> of fragments also makes it so it is not really possible to know who >> is contributing classes to a split package in the require bundle case.) > Ok, so first I assume it would make sense for an OSGi Repository to > simply never return a ModuleDefinition for a fragment. I am not sure I understand your point. Host bundles would be returned and during the resolve process, fragments might be attached to that host, thus modifying its content in ways that we cannot predict. > And the "you don't know what you've got till it's resolved" problem is > going to be present in any module system for which package name is not > a required component of a dependency expression. But as long as > re-export is not the default behavior (it isn't), doesn't this really > just boil down to the leakage problem? And this is addressed in 277 > the same as it is in OSGi: you must declare such "leaks" as re-exports > ("uses"). Not perfect, but probably good enough. > > So I think we're left with the issue of a single search visiting > multiple loaders that can return the same package. This won't happen > in the fragment case, since the "split" gets mended at runtime, but it > will in the require-bundle case. And this will manifest as multiple > entries in the import list which export the same package. > > But even this isn't an issue, as long as dependency resolution relies > solely on declared exports and re-exports of a module, and not on what > is available by browsing the imports of that module. > > So I'm back to my original statement that we just need to acknowledge > that package duplication may exist in an import list. > > Or am I missing something here? I am just not sure why we want to make the fact that some module systems support split packages a visible concept in our "interoperability API", when it would be difficult to get interoperability around split packages across module systems when we already have issues with split packages within a single module system. Especially, given my description above, since we would not easily know which modules were contributing to a split package in OSGi. -> richard >> >> I think that is fairly complete description of the situation, sorry >> if it is somewhat terse. In summary, I wouldn't assume that we can >> get reasonable interoperability at this level. If you need these >> types of features, then you should stick to a single modularity >> framework...preferably OSGi. ;-) >> >> But my overall point is, I believe that if we can support >> interoperability at module- and package-level dependencies across >> module systems, then I think we would hit most use cases. >> >> -> richard >> >>> >>> Sam >>> >>> On Apr 25, 2008, at 1:57 PM, Richard S. Hall wrote: >>> >>>> Bryan Atsatt wrote: >>>>> Richard S. Hall wrote: >>>>>> Gordon Hirsch wrote: >>>>>>>> 1. Map its dependency declarations into a standard runtime >>>>>>>> representation. >>>>>>>> 2. Support a standard search model over its stored modules, >>>>>>>> using the runtime dependency expressions. >>>>>>>> 3. Map its stored module data into a standard runtime >>>>>>>> representation that can be returned by the search. >>>>>>> >>>>>>> One challenge lies in defining the "standard runtime >>>>>>> representations" and "standard search model" in a universal >>>>>>> enough way to encompass OSGi and other module systems. This >>>>>>> implies embracing concepts (in these standard representations >>>>>>> and search model) that were not universally liked by the EG >>>>>>> early on. (Split packages and package-level import/export come >>>>>>> to mind.) >>>>>> >>>>>> Package-level import/export seem like a must, but I still don't >>>>>> see split packages as a necessity. >>>>> Strongly agreed on import-by-package, and that ModuleDefinition >>>>> requires a method to enumerate exported packages (and probably an >>>>> exportsPackage(String packageName) method to enable an efficient >>>>> Query). >>>>> On the split package front, however, it seems a little fuzzy to >>>>> me. If an OSGi implementation of 277 is also going to remain OSGi >>>>> Rx compliant, it will still need to support split packages, right? >>>>> If so, don't we need to surface this fact at the 277 level? >>>>> Perhaps that is as simple as acknowledging that >>>>> Module.getImportedModules() may return more than one instance that >>>>> exports a given package. >>>> >>>> That all depends on if the 277 API is a subset of OSGi >>>> functionality or equal to it, I suppose. >>>> >>>> -> richard >>>>> >>>>> // Bryan >> From bryan.atsatt at oracle.com Tue Apr 29 13:30:06 2008 From: bryan.atsatt at oracle.com (Bryan Atsatt) Date: Tue, 29 Apr 2008 13:30:06 -0700 Subject: Updates on Interoperability In-Reply-To: <481743CF.2020102@ungoverned.org> References: <47F46ABF.1020306@sun.com> <48121BA2.7060001@oracle.com> <48123223.9070803@sas.com> <48123AE3.1090708@ungoverned.org> <4812409C.1030000@oracle.com> <4812459E.2060407@ungoverned.org> <25680D44-7C07-4DC9-841F-12751540ECD4@sampullara.com> <48126D6A.7000001@ungoverned.org> <48163DF6.3010705@oracle.com> <481743CF.2020102@ungoverned.org> Message-ID: <4817854E.1010906@oracle.com> Richard S. Hall wrote: > Bryan Atsatt wrote: >> Richard S. Hall wrote: >>> Sam Pullara wrote: >>>> Did we decide which way it will be compatible? I'm a little >>>> concerned that implementing something that can use OSGi modules is >>>> a far bigger task that implementing something that OSGi can use. >>>> If we are going for total compatibility it seems like we should >>>> just use OSGi. Personally I wanted something in Java that was >>>> simpler and cleaner that OSGi-users could leverage if they wanted. >>>> But I'm coming at this from the >>>> we-need-something-like-maven/gem/ivy camp. >>>> >>>> The vast majority of people that are going to use this system will >>>> have never heard of OSGI nor care about any of their more subtle >>>> features and just want an easy way to leverage the vast amount of >>>> libraries available (virtually all of them are not OSGi modules >>>> today but are jar files in maven repositories). We shouldn't be >>>> specing out some sort of uber container but rather a simple way to >>>> tie code to its dependencies. >>> >>> If this is all that people wanted, then we could just define >>> repositories and associated metadata and forget about all runtime >>> modularity support, but I strongly disagree that this is all people >>> want or else there wouldn't be such an upsurge in OSGi interest and >>> adoption. >>> >>>> I strongly suggest KISS applies to our effort. I write this >>>> knowing that the committee seems to be heavily in favor of this >>>> alternate view that we implement the kitchen sink. >>> >>> I think our interest is similar, but perhaps our conclusions are >>> different. Initially, I think my comments were construed as an >>> argument against split packages in general, but I am really arguing >>> the same as you (I think) that we don't really need split packages >>> to be a concept in our "interoperability API", if that is how we are >>> to view the 277 API. Of course, I am also willing to have the former >>> argument too, but I will leave that for another time. :-) >>> >>> From my point of view, I do not think it is wholly reasonable to >>> assume that we want to try to accomplish interoperability across >>> module systems, where that includes arbitrarily sharing split >>> packages across module systems. >> Nor do I, but I'm trying to understand the implications if we assume >> that OSGi will continue to split packages. >>> >>> Let me try to accurately conjure the peculiarities of split packages >>> in the OSGi module layer... >>> >>> We support split packages in two ways, via required bundles (i.e., >>> module dependencies) and via bundle fragments. In general, we >>> recommend that if you split your packages that you attach mandatory >>> attributes on them so that importers do not get them by mistake. By >>> requiring a bundle, you can ignore mandatory attributes and get >>> everything a bundle has to offer and combine it with other required >>> bundles that might be exporting other parts of the split packages. >>> Given this, there is a semi-reasonable chance* that we can tell you >>> which bundles are contributing to the "complete" packages, but we >>> have no way of knowing if/when combined split packages are complete. >>> Further, we have to assume that split package parts offered by >>> different bundles do not overlap, because if we allowed them to >>> overlap then we would have shadowing and ordering issues. Again we >>> have no easy way to verify that they do not overlap, so we just >>> assume they don't. >>> >>> Fragments on the other hand are a little trickier, since split >>> packages from them are loaded from the same class loader to provide >>> package private access (which is not afforded for split packages in >>> the require bundle approach). The "host" bundle doesn't really know >>> about fragments and it is possible for fragments to shadow or extend >>> packages in the "host" bundle. Since fragments provide their split >>> packages in an implicit way (i.e., they largely just become part of >>> the host bundle's class path), there is no easy way to determine if >>> a fragment is contributing to a given package or not. (* This >>> feature of fragments also makes it so it is not really possible to >>> know who is contributing classes to a split package in the require >>> bundle case.) >> Ok, so first I assume it would make sense for an OSGi Repository to >> simply never return a ModuleDefinition for a fragment. > > I am not sure I understand your point. Host bundles would be returned > and during the resolve process, fragments might be attached to that > host, thus modifying its content in ways that we cannot predict. I simply meant that a fragment would never be returned by itself, as a separate ModuleDefinition. > >> And the "you don't know what you've got till it's resolved" problem >> is going to be present in any module system for which package name is >> not a required component of a dependency expression. But as long as >> re-export is not the default behavior (it isn't), doesn't this really >> just boil down to the leakage problem? And this is addressed in 277 >> the same as it is in OSGi: you must declare such "leaks" as >> re-exports ("uses"). Not perfect, but probably good enough. >> >> So I think we're left with the issue of a single search visiting >> multiple loaders that can return the same package. This won't happen >> in the fragment case, since the "split" gets mended at runtime, but >> it will in the require-bundle case. And this will manifest as >> multiple entries in the import list which export the same package. >> >> But even this isn't an issue, as long as dependency resolution relies >> solely on declared exports and re-exports of a module, and not on >> what is available by browsing the imports of that module. >> >> So I'm back to my original statement that we just need to acknowledge >> that package duplication may exist in an import list. >> >> Or am I missing something here? > > I am just not sure why we want to make the fact that some module > systems support split packages a visible concept in our > "interoperability API", when it would be difficult to get > interoperability around split packages across module systems when we > already have issues with split packages within a single module system. > Especially, given my description above, since we would not easily know > which modules were contributing to a split package in OSGi. I was just exploring whether or not we need to make it a visible concept in order for cross module system resolution to work correctly. You may note in Stanley's doc that he simply took the position that a JAM definition would fail to resolve against an OSGi definition that split packages. This may be an acceptable solution, but it still requires a means to *detect* that a split has occurred. > > -> richard > >>> >>> I think that is fairly complete description of the situation, sorry >>> if it is somewhat terse. In summary, I wouldn't assume that we can >>> get reasonable interoperability at this level. If you need these >>> types of features, then you should stick to a single modularity >>> framework...preferably OSGi. ;-) >>> >>> But my overall point is, I believe that if we can support >>> interoperability at module- and package-level dependencies across >>> module systems, then I think we would hit most use cases. >>> >>> -> richard >>> >>>> >>>> Sam >>>> >>>> On Apr 25, 2008, at 1:57 PM, Richard S. Hall wrote: >>>> >>>>> Bryan Atsatt wrote: >>>>>> Richard S. Hall wrote: >>>>>>> Gordon Hirsch wrote: >>>>>>>>> 1. Map its dependency declarations into a standard runtime >>>>>>>>> representation. >>>>>>>>> 2. Support a standard search model over its stored modules, >>>>>>>>> using the runtime dependency expressions. >>>>>>>>> 3. Map its stored module data into a standard runtime >>>>>>>>> representation that can be returned by the search. >>>>>>>> >>>>>>>> One challenge lies in defining the "standard runtime >>>>>>>> representations" and "standard search model" in a universal >>>>>>>> enough way to encompass OSGi and other module systems. This >>>>>>>> implies embracing concepts (in these standard representations >>>>>>>> and search model) that were not universally liked by the EG >>>>>>>> early on. (Split packages and package-level import/export come >>>>>>>> to mind.) >>>>>>> >>>>>>> Package-level import/export seem like a must, but I still don't >>>>>>> see split packages as a necessity. >>>>>> Strongly agreed on import-by-package, and that ModuleDefinition >>>>>> requires a method to enumerate exported packages (and probably an >>>>>> exportsPackage(String packageName) method to enable an efficient >>>>>> Query). >>>>>> On the split package front, however, it seems a little fuzzy to >>>>>> me. If an OSGi implementation of 277 is also going to remain OSGi >>>>>> Rx compliant, it will still need to support split packages, >>>>>> right? If so, don't we need to surface this fact at the 277 >>>>>> level? Perhaps that is as simple as acknowledging that >>>>>> Module.getImportedModules() may return more than one instance >>>>>> that exports a given package. >>>>> >>>>> That all depends on if the 277 API is a subset of OSGi >>>>> functionality or equal to it, I suppose. >>>>> >>>>> -> richard >>>>>> >>>>>> // Bryan >>> > From bryan.atsatt at oracle.com Tue Apr 29 13:32:03 2008 From: bryan.atsatt at oracle.com (Bryan Atsatt) Date: Tue, 29 Apr 2008 13:32:03 -0700 Subject: Supporting OSGi Bundles in the Java Module System In-Reply-To: <481733F9.1050902@sas.com> References: <4816930D.4080106@sun.com> <481733F9.1050902@sas.com> Message-ID: <481785C3.6070403@oracle.com> Somehow your original message didn't make it at all to our servers. Could you please resend it? // Bryan Gordon Hirsch wrote: > Stanley, > > The javadoc zip file was stripped by our firewall. Could you either > resend with a different extension or post it to the EG homepage at > jcp.org? > > Stanley M. Ho wrote: >> Dear 277 experts, >> >> The first attachment is a draft spec for supporting OSGi bundles in the >> Java Module System. This is based on the past EG discussion in >> particular the proposals and inputs from Glyn, Richard, and Bryan. This >> is a work in progress and is expected to evolve based on further inputs >> from this EG. >> >> The second attachment is the latest API javadoc for your reference. >> >> We're currently updating the JSR 277 EDR2 specification and APIs to make >> the distinction between the framework/abstractions for the Java Module >> System more clear, and we expect they will be available for the EG to >> review in a few weeks after JavaOne. In the meantime, the draft is >> intended to provide enough information for the EG to review and provide >> inputs without the EDR2. >> >> Numerous EG members are preparing to attend and/or speak at JavaOne next >> week, so some of them may well not respond until afterwards. >> >> Finally, thanks to Bryan and his blog that serves really well to set up >> the context for this discussion. >> >> - Stanley and Alex >> >> >> ------------------------------------------------------------------------ >> >> >> Supporting OSGi Bundles in the Java Module System >> >> * >> >> Mandy Chung and Stanley Ho >> Version 0.1 >> April 28, 2008 >> Copyright 2008 Sun Microsystems, Inc. >> >> * JSR 277 Early Draft defines the framework for the Java Module >> System and provides the API to access Java modules such as searching >> for modules, loading and initializing a module, and also managing its >> life time. JSR 277 enables one or more module systems run on a single >> Java virtual machine. This document^1 defines how JSR 277 supports >> OSGi bundles as defined in JSR 291. This document is a work in >> progress and is expected to evolve based on more input from the >> Expert Group. >> >> Section 1 describes various definitions in the Java Module System. >> Section 2 describes the requirements for supporting OSGi bundles in >> the Java Module System and section 3 describes the proposed API >> changes. Section 4 specifies how to map an OSGi bundle to a >> ModuleDefinition so they can be consumed by other modules in the Java >> Module System. Section 5 describes the repository mechanism that >> enables the OSGi module system to be plugged into the Java Module >> System framework to make OSGi bundles accessible by other module >> systems at runtime. Appendix A shows the code example illustrating >> how a JAM module system implementation uses the JSR 277 API to search >> and import OSGi bundles. >> >> ^1 This draft is built upon from the basis of the past EG >> discussions, proposals and suggestions for the interoperability >> support especially the ideas and invaluable inputs from Glyn >> Normington, Richard Hall and Bryan Atsatt. >> >> >> 0. Current Status >> >> The JSR 277 EDR2 is being updated to make the distinction between the >> framework/abstractions for the Java Module System and the JAM module >> system (See Section 1) clearer. >> >> This section highlights the main items that remain to be sorted out. >> >> 1. Versioning scheme (see Section 4.2) >> 2. Security >> 3. Java Module Events >> >> >> 1. Definitions >> >> /Java module/ >> >> A development abstraction that exists at compile-time in the Java >> programming language and is reified in the Java Virtual Machine. >> >> /Module Definition/ >> >> A deployment abstraction that contains metadata, classes and >> resources, and is reified as ModuleDefinition as part of a Java >> Module System implementation. >> >> /JAM module/ >> >> A concrete, portable deployment format which implements the Module >> Definition. Amongst other details, it has metadata, classes, >> resources, and can contain embedded JAR files and native libraries. >> The distribution format is called JAM (JAva Module) that is based on >> the JAR file format. >> >> /Java Module System/ >> >> A set of runtime abstractions that includes ModuleSystem, Repository, >> ModuleDefinition (see "Module Definition" above), Module, >> ImportDependency and others. >> >> /JAM Module System/ >> >> A concrete implementation of the Java Module System that supports JAM >> modules. It is the default implementation of the Java Module System >> for the Java SE platform. >> >> /OSGi Module System/ >> >> A concrete implementation of the Java Module System by an OSGi >> container. >> >> >> 2. Requirements >> >> 1. It shall be possible for an OSGi container to implement the Java >> Module System. >> 2. It shall be possible for a JAM module to express an import >> dependency on any Module Definition in any Java Module System. >> >> Below provides an example to illustrate how a JAM module imports OSGi >> bundles. >> >> A Wombat application is a JAM module that processes shopping orders >> for a website. It depends on the Apache Xerces XML parser and the >> Apache Derby and both of them are available as OSGi bundles. The >> module definition of the Wombat application looks like this: >> >> // >> // com/wombat/app/module-info.java >> // >> @Version("1.0") @ImportModules({ >> @ImportModule(name="org.apache.xerces.parsers", version="2.6.6+") >> @ImportModule(name="org.apache.derby", version="10.0+") >> }) >> module com.wombat.app; >> >> This Wombat application is packaged as a JAM file named >> "com.wombat.app-1.0.jam" and the OSGi bundles it depends on are >> packaged as JAR files containing the following manifests,^2 per the >> OSGi specifications: >> >> org.apache.xerces.parsers: >> Bundle-SymbolicName: org.apache.xerces.parsers >> Bundle-Version: 2.9.10 >> >> Export-Package: org.apache.xerces.parsers; version=2.6.6; >> uses="org.apache.commons.logging", >> org.apache.xerces.jaxp; version=2.6.6, >> org.apache.xerces.framework; version=2.6.6, >> org.apache.xerces.readers; version=2.6.6, >> org.apache.xerces.utils; version=2.6.6, >> org.apache.commons.logging; version=1.0.4 >> Import-Package: javax.xml.parsers; version=1.2.0, >> org.w3c.dom; version=1.0.0, >> org.xml.sax; version=2.0.1 >> Require-Bundle: org.osgi.util.xml; version=1.0.0; >> visibility:=reexport; resolution:=optional >> org.apache.derby: >> Bundle-SymbolicName: org.apache.derby >> Bundle-Version: 10.0.2 >> Export-Package: org.apache.derby.authentication, >> org.apache.derby.database, >> org.apache.derby.io, >> org.apache.derby.jdbc >> Import-Package: java.sql >> >> ^2 These example manifests are for illustration purpose and they do >> not represent the manifest in the actual Apache Xerces XML Parser and >> Apache Derby bundles. >> >> >> 3. Proposed API Changes >> >> This section describes the proposed API changes for supporting OSGi >> bundles in the Java Module System. >> >> >> 3.1 ModuleSystem class >> >> The ModuleSystem specification is updated to support multiple >> implementations running in a single Java virtual machine. This update >> is necessary to meet the requirement (1). See JSR 277 EDR2 for the >> full specification. >> >> A ModuleSystem implementation is responsible for creation, >> management, and release of Java Modules owned by this module system. >> The ModuleSystem specification does not define the resolution >> algorithm and the distribution format that a module system supports. >> Instead, the resolution algorithm and the distribution format are >> specific to a ModuleSystem implementation. >> >> >> 3.1.1 Module Initialization >> >> This section is a clarification to the JSR 277 EDR about the >> initialization of a Module instance. >> >> The ModuleSystem.getModule(ModuleDefinition) method returns a /fully >> initialized/ Module instance^3 for a given ModuleDefinition. A Module >> instance is fully initialized if the following conditions are all met: >> >> 1. Its import dependencies have been resolved. ModuleDefinitions for >> the imported Java modules satisfying the import dependencies and >> all its constraints are found using the algorithm defined by this >> ModuleSystem. >> 2. Module instances for its imports are successfully instantiated and >> initialized by its owning ModuleSystem. >> 3. This ModuleSystem has performed its own type consistency checking >> successfully in the Module instance. >> 4. If this ModuleSystem supports an initializer^4 to be invoked >> before a Module instance is fully initialized, the initializer for >> this Module instance is invoked. >> >> In addition, a ModuleSystem can support import constraints specific >> to its algorithm to tailor the resolution of the imports for a >> ModuleDefinition. The constraints specified in a ModuleDefinition is >> only known to its owning ModuleSystem and other ModuleSystem >> implementations are not required to understand them. >> >> ^3 Initializing an OSGi bundle is equivalent to resolving, wiring and >> starting an OSGi bundle. >> >> ^4 The ModuleSystem specification does not define a generic >> initializer mechanism for ModuleDefinitions. The JAM module system >> supports the module initializer through the ModuleInitializer API and >> the ModuleInitializerClass annotation (see JSR 277 EDR 2). The OSGi >> module system supports the module initializer through the bundle >> activator. >> >> >> 3.1.2 Constraint Checking >> >> A new method ModuleSystem.getModules is added to allow a ModuleSystem >> implementation to instantiate Module instances for multiple >> ModuleDefinitions in the same resolution and also to enforce >> constraints specified in these Modules. >> >> ModuleSystem class: >> /** >> * Returns the list of Module instances for the imports >> * in the same resolution for the specified importer. >> * The returned Module instances are instantiated and initialized >> * using the algorithm specific to this ModuleSystem. >> * >> * This method is called by a ModuleSystem when initializing >> * a Module instance (importer) that imports Modules (imports) >> * from this ModuleSystem. >> */ >> public List getModules(ModuleDefinition importer, >> List >> imports) >> throws ModuleInitializationException; >> >> OSGi allows to put constraints on the importer through the metadata >> for an exported package in another bundle. The OSGi module system can >> enforce the constraints on the importer in the implementation of this >> getModules method. The importer can be a ModuleDefinition from other >> ModuleSystem. >> >> For example, the Apache Xerces XML parser bundle in Example 1 of >> Section 1 exports the org.apache.xerces.parsers package whose export >> definition has a "use" constraint. This export definition puts a >> constraint on the importer of the org.apache.xerces.parsers package >> to use the org.apache.commons.logging package wired to the >> org.apache.xerces.parsers package in the same resolution. The 1.0 >> version of the Wombat application imports the >> org.apache.xerces.parsers bundle and the org.apache.derby bundle. The >> org.apache.xerces.parsers package will get wired to the version 1.0.4 >> logging package and the constraint is satisfied and thus it can be >> wired successfully. >> >> Let's say a new version of the Wombat application (say version 1.2) >> is updated and it depends on an additional Apache Commons Logging >> utility which is also an OSGi bundle. >> >> // >> // com/wombat/app/module-info.java >> // >> @Version("1.2") @ImportModules({ >> @ImportModule(name="org.apache.xerces.parsers", version="2.6.6+") >> @ImportModule(name="org.apache.derby", version="10.0+") >> @ImportModule(name="org.apache.commons.logging", version="2.0+") >> }) >> module com.wombat.app; >> >> The bundle manifest header for the Apache Commons Logging utility is: >> >> org.apache.commons.logging: >> Bundle-SymbolicName: org.apache.commons.logging >> Bundle-Version: 2.0 >> Export-Package: org.apache.commons.logging; version=2.0 >> >> The 1.2 version of the Wombat application imports the >> org.apache.commons.logging bundle that violates the constraint if the >> org.apache.xerces.parsers package is wired to the version 1.0.4 >> logging package but the Wombat application requires the version 2.0 >> logging package. The version 1.2 of the Wombat application should >> fail to initialize due to this constraint. In other words, the >> getModules() method of the OSGi module system should throw a >> ModuleInitializationException when the OSGi module system determines >> that the constraint is violated. >> >> >> 3.2 ModuleDefinition class >> >> Two new methods are added in the ModuleDefinition class to return the >> exported packages and member packages respectively. The export and >> member definitions contained in the OSGi metadata are in package >> granularity. In addition, a new PackageDefinition class is added to >> allow an OSGi bundle to expose the metadata for an exported package. >> This is required to meet the requirements (1) and (2). >> >> ModuleDefinition class: >> public abstract Set >> getExportedPackageDefinitions(); >> public abstract Set >> getMemberPackageDefinitions(); >> PackageDefinition class: >> public abstract class PackageDefinition { >> public String getName(); >> public Version getVersion(); >> public Set getAttributeNames(); >> public String getAttribute(String name); } >> >> The version and attributes in the PackageDefinition are optional and >> for information only and to aid diagnosis. A ModuleSystem importing a >> ModuleDefinition from other ModuleSystem is not required to >> understand the version and attributes of its exported >> PackageDefinitions. >> >> >> 3.3 ImportDependency class >> >> The ImportDependency class is updated as follows. >> >> 1. An import type is added to indicate if the ImportDependency is for >> module-level or package-level. "module" and "package" are two >> defined import types. >> 2. The ImportDependency class can have attributes for a module system >> to include additional information about an import dependency. >> >> This is required to meet the requirement (1). >> >> ImportDependency class: >> // static factory methods >> public static ImportDependency >> newImportModuleDependency(String moduleName, >> VersionConstraint constraint, >> boolean reexport, >> boolean optional, >> Map attributes); >> public static ImportDependency >> newImportPackageDependency(String packageName, >> VersionConstraint constraint, >> boolean optional, >> Map attributes); >> public String getType(); >> public String getAttribute(String name); >> public Set getAttributeNames(); >> >> >> 4. Mapping OSGi Bundles to ModuleDefinitions >> >> This section specifies how an OSGi bundle maps to a ModuleDefinition >> to expose in the Java Module System so that other ModuleDefinitions >> can import them. >> >> >> 4.1 Bundle-SymbolicName >> >> The bundle symbolic name maps to a module name (i.e. >> ModuleDefinition.getName()). The directives for the >> Bundle-SymbolicName header maps to the module attributes. >> >> For example: >> >> Bundle-SymbolicName: com.acme.foo >> >> The Java Module System and OSGi do not enforce any naming convention. >> It is encouraged to use the reverse domain name convention to name >> OSGi bundles and Java modules to avoid the name space conflict. >> >> >> 4.2 Bundle-Version >> >> A bundle version maps to a module version^5 (i.e. >> ModuleDefinition.getVersion()). Bundle-Version is an optional header >> and the default value is 0.0.0. The bundle version format is: >> >> major[.minor[.micro]][.qualifier] >> >> The module version format is: >> >> major[.minor[.micro[.update]]][-qualifier] >> >> If the bundle version contains a qualifier, the delimiter prior to >> the qualifier will need to be changed from a period ('.') to a dash >> ('-'). For example, the bundle version 3.1.4.pi maps to the module >> version 3.1.4-pi. >> >> ^5 *Difference in OSGi and JSR 277 versioning scheme* >> The versioning scheme defined in the JSR 277 Early Draft is loosely >> based on the existing versioning schemes that are widely used in the >> Java platform today and for backward compatibility reason (see JSR >> 277 EDR chapter 5). Many existing products including the JDK use the >> version format with the micro, minor, micro, and update numbers. A >> version with no qualifier is higher than the version with the same >> version number but with a qualifier. This is more intuitive and has >> been the convention the JDK has been using. The Expert Group has >> discussed the difference with the OSGi versioning scheme and agreed >> with the JSR 277 versioning scheme defined. >> >> *Open Issue:* >> Need to investigate the version mapping due to the difference in the >> comparison of two versions - one with a qualifier and the other >> without a qualifier. >> >> When two bundle versions have the same major, minor, and micro >> numbers, the bundle version that has a qualifier is lower than the >> bundle version that has no qualifier. e.g. 7.8.9.pi < 7.8.9 >> >> When two module versions have the same major, minor, micro, and >> update numbers, the module version that has a qualifier is higher >> than the module version that has no qualifier. e.g. 7.8.9 < >> 7.8.9-b04-alpha >> >> One possible solution would be: >> >> OSGi version JSR 277 Version >> major.minor.micro -> major.minor.micro-0 >> major.minor.micro.qualifier -> major.minor.micro-1-qualifier >> >> 4.3 Import-Package >> >> The Import-Package header maps to the import dependencies for a Java >> module (i.e. ModuleDefinition.getImportDependencies()). >> >> The Import-Package header contains one or more import definitions, >> each of which describes a single package import for a bundle. Each >> import definition maps to an ImportDependency instance with the >> "package" type as follows: >> >> * The import package name maps to ImportDependency.getName(). >> * ImportDependency.getType() returns "package". >> * The "resolution" directive maps to the "optional" input parameter >> of the ImportDependency constructor; true if >> "resolution:=optional" is specified and false otherwise. >> * The "version" attribute maps to the VersionConstraint of the >> ImportDependency as described in the version-range section below. >> * All other attributes specified in the import definition including >> the bundle-symbolic-name and bundle-version attributes map to the >> attributes in the ImportDependency. >> >> Example, >> >> Import-Package: p; >> version="[1.23, 1.24]"; >> resolution:=optional >> >> maps to the import dependencies equivalent to: >> >> ImportDependency importP = >> ImportDependency.newImportPackageDependency("p", >> >> VersionConstraint.valueOf("[1.23,1.24]", >> true /* >> optional */); >> >> >> The version-range mapping: >> >> The OSGi version-range maps to a VersionConstraint as follows: >> >> * If a version has no qualifier, the mapping is exact. For example, >> a bundle version range [1.1, 1.2) maps to a module version range >> [1.1, 1.2). If there is a qualifier, then section 4.2 should be >> used. >> * If a bundle version range is specified as a single version, it >> will map to an open version range. For example, the bundle version >> range "1.23" maps to the module version range "1.23+". >> >> >> 4.4 Export-Package >> >> The Export-Package header maps to the exported package definitions >> for a Java module (i.e. >> ModuleDefinition.getExportedPackageDefinitions()). >> >> The Export-Package header contains one or more export definitions, >> each of which describes a single package export for a bundle. Each >> export definition maps to a PackageDefinition instance as follows: >> >> * The package name maps to PackageDefinition.getName(). >> * The "include" and "exclude" directive along with the classes in >> the exported package are the input to determine the returned value >> of the ModuleDefinition.isClassExported() method. >> * The "version" attribute maps to PackageDefinition.getVersion(). >> * Other attributes and directives including the "use" directive in >> the export definition can map to the attributes in the >> PackageDefinition. >> * The exported package definition is also a member package >> definition for the module. >> >> >> 4.5 Require-Bundle >> >> The required bundles maps to the import dependencies for a Java >> module (i.e. ModuleDefinition.getImportDependencies()). Each required >> bundle maps to an ImportDependency instance with the "module" type: >> >> * The bundle symbolic name of the required bundle maps to >> ImportDependency.getName(). >> * ImportDependency.getType() returns "module". >> * The "visibility" directive maps to >> ImportDependency.isReexported(). The isReexported() method returns >> true if "visibility:=reexport" is specified; false otherwise. >> * The "resolution" directive maps to ImportDependency.isOptional(). >> The isOptional() method returns true if "resolution:=optional" is >> specified; false otherwise. >> * The "bundle-version" attribute maps to the VersionConstraint of >> the ImportDependency as described in the Version Range Mapping >> section of section 4.4. >> >> Example, >> >> Require-Bundle: com.acme.facade;visibility:=reexport, >> com.acme.bar;visibility:=reexport;resolution:=optional >> >> maps to the import dependencies equivalent to: >> >> ImportDependency facade = >> ImportDependency.newImportModuleDependency("com.acme.facade", >> >> VersionConstraint.DEFAULT, >> true /* reexport >> */, >> false /* >> optional */); >> >> ImportDependency bar = >> ImportDependency.newImportModuleDependency("com.acme.bar", >> >> VersionConstraint.DEFAULT, >> true /* reexport >> */, >> true /* optional >> */); >> >> >> 4.6 Other Manifest Headers >> >> The above sections cover the manifest headers that provide the >> metadata for the OSGi resolver (see Section 3.5 of the OSGi Service >> Platform Core Specification Release 4, Version 4.1). The other bundle >> manifest headers, including Bundle-Vendor, Bundle-Description and >> DynamicImport-Package, do not affect the module resolution. This >> specification does not need to define how to map them to >> ModuleDefinition. However, implementations are encouraged to include >> them as the module attributes (i.e. ModuleDefinition.getAttribute()) >> as additional information to aid diagnosis. >> >> >> 4.7 Fragment Bundles >> >> Fragment bundles are not exposed as ModuleDefinitions in the Java >> Module System. Instead, they are exposed as part of the >> ModuleDefinition of its host bundle to which they are attached to. >> >> >> 4.8 Example >> >> The manifest in the org.apache.xerces.parsers bundle shown in Example >> 1 of Section 2 is: >> >> org.apache.xerces.parsers: >> Bundle-SymbolicName: org.apache.xerces.parsers >> Bundle-Version: 2.9.10 >> >> Export-Package: org.apache.xerces.parsers; version=2.6.6; >> uses="org.apache.commons.logging", >> org.apache.xerces.jaxp; version=2.6.6, >> org.apache.xerces.framework; version=2.6.6, >> org.apache.xerces.readers; version=2.6.6, >> org.apache.xerces.utils; version=2.6.6, >> org.apache.commons.logging; version=1.0.4 >> Import-Package: javax.xml.parsers; version=1.2.0, >> org.w3c.dom; version=1.0.0, >> org.xml.sax; version=2.0.1 >> Require-Bundle: org.osgi.util.xml; version=1.0.0; >> visibility:=reexport; resolution:=optional >> >> Below shows the ModuleDefinition for this OSGi bundle when exposed in >> the Java Module System. For clarity, we only show one Export-Package >> entry and one Import-Package entry. >> >> Method of ModuleDefinition Returned Value Bundle Manifest Header >> getName() "org.apache.xerces.parsers" Bundle-SymbolicName >> getVersion() Version.valueOf("2.9.10") Bundle-Version >> getImportDependencies() >> >> ImportDependency.newImportModuleDependency("org.osgi.util.xml", >> >> VersionConstraint.valueOf("1.0.0+"), >> true /* reexport */, >> true /* optional */); >> >> Require-Bundle >> >> ImportDependency.newImportPackageDependency("javax.xml.parsers", >> >> VersionConstraint.valueOf("1.2.0+"), >> false /* optional >> */); >> Import-Package >> getExportedPackageDefinition() PackageDefinition with: >> >> name="org.apache.xerces.parsers", >> version="2.6.6" >> attributes=(("uses", "org.apache.commons.logging")} >> >> Export-Package >> >> >> 5. Enabling the OSGi Module System in the Framework >> >> To enable the OSGi module system in the framework, an OSGi Repository >> implementation should be plugged into the runtime. The OSGi >> repository is responsible for discovering OSGi bundles and exposing >> them as ModuleDefinitions so that OSGi bundles are available for >> other module systems to use. A ModuleSystem implementation finds OSGi >> bundles via the repository delegation model and therefore the OSGi >> repository has to be configured as an ancestor of the repository >> where the Java module depending on OSGi bundles resides. Otherwise, >> it will fail to find the importing OSGi bundles. >> >> One or more repositories can be created for the OSGi module system >> and interoperate with other module systems via the repository >> delegation model. >> >> The following picture depicts the repository tree set up to run the >> Wombat application described in Example 1 of Section 2. The OSGi >> repository is configured as the parent of the application repository. >> This particular OSGi repository implementation includes the Apache >> Felix OSGi runtime for loading and resolving OSGi bundles. >> >> >> ------------------------ >> | Bootstrap Repository | >> ------------------------ >> | >> | >> -------------------- >> | Global Repository | >> -------------------- >> | >> | xxxxxxxxxxxxxxxx >> ------------------- x Apache Felix x >> | OSGi Repository | <====> x OSGi x >> ---> org.apache.xerces.parsers version 2.10 >> ------------------- x Runtime >> x org.apache.derby version 10.0.2 >> | >> xxxxxxxxxxxxxxxx org.apache.derby version 9.1 >> | >> ----------------- | Application >> | | Repository | >> ----------------- >> com.wombat.app-1.0.jam >> >> For example, the following command will launch the Wombat application >> in the "/wombat-application" directory. >> >> > java -repository /wombat-application -module >> com.wombat.app >> The com.wombat.app module is located in the application repository >> which is a repository for the JAM module system. The JAM module >> system first loads the com.wombat.app module. To initialize this JAM >> module, the JAM module system looks at its import dependencies and >> performs a search of two imported OSGi bundles through the repository >> delegation model from the OSGi repository. The JAM module system then >> requests the OSGi module system associated with the OSGi repository >> to get the Module instances for the imports (see Section 3.1). Once >> the imported OSGi bundles are loaded and started, the initialization >> process for the com.wombat.app module continues. >> >> >> 6. Delegation of Class and Resource Loading >> >> The class loader of a Module instance (i.e. returned by the >> Module.getClassLoader() method) must be capable to load all classes >> and resources in the module. >> >> As described in Section 5 above, when the com.wombat.app JAM module >> is initialized, two other Module instances for its imports >> representing the resolved OSGi bundles are created in the system. The >> class loader for the Module instance for the >> org.apache.xerces.parsers bundle must be capable to load all classes >> and resources in it. Similarly for the org.apache.derby bundle. >> >> >> 7. Security >> >> TBD. >> >> >> 8. Implementation Notes >> >> The following are the notes for the implementation of the OSGi >> repository and the implementation of the Java Module System. >> >> 1. The repository delegation hierarchy is a tree and thus cycles >> involving multiple module systems are inherently unsupported. >> 2. The repository delegation model is designed to offer isolation >> between ModuleDefinitions in different repositories. Although a >> module system could have access to multiple repositories, the >> module system should adhere to the repository delegation module. >> Otherwise, it would break the isolation model the repository >> provides. >> 3. Java SE 7 is expected to have parallel class loading support. All >> module systems plugged in the framework are required have parallel >> class loading enabled in order to avoid potential deadlocks. >> 4. Split packages without shadowing are explicitly permitted in OSGi >> whereas the JAM module system does not allow split packages. So >> importing OSGi bundles with split packages in a JAM module will >> result in module initialization failure. >> 5. When a resolution involves multiple module systems, a module >> system implementation should take the possible potential issues >> (such as hanging) into account in their design to prevent a >> foreign module system from bringing down the module system or the >> entire JVM. A module system could implement time out policy to >> prevent from hanging the module system. >> >> >> 8. References >> >> * OSGi Service Platform Core Specification Release 4, Version 4.1 >> April 2007 >> * JSR 277 Interoperation with OSGi by Richard Hall and Glyn >> Normington, Apr 24, 2006. >> * Module System Interoperability by Richard Hall, May 11, 2006. >> >> >> A. Appendix >> >> The following illustrates how the JAM module system implementation >> uses the JSR 277 API to search and import OSGi bundles This example >> does not cover the exact resolution algorithm and the implementation >> of the module system runtime. >> Example 1 of Section 1 has a com.wombat.app JAM module importing two >> OSGi bundles. >> >> com.wombat.app ----> org.apache.xerces.parsers >> | >> |---> org.apache.derby >> >> // the Java runtime will first find the module com.wombat.app >> ModuleDefinition wombatModDef = repository.find("com.wombat.app", >> "1.0+"); >> // This call blocks until com.wombat.app is fully initialized >> Module wombat = wombatModDef.getModuleInstance(); >> >> >> /JAM Module System Runtime/ >> >> // find imports for wombat >> List imports = ...; >> Map> foreignImportMap = new >> HashMap....; >> for (ModuleDefinition md : imports) { >> if (md is from a foreign module system (ms)) { >> // the case to add a new entry in foreignImportMap // >> is not shown in this pseudo-code. >> // // org.apache.xerces.parsers and org.apache.derby >> will be added in this map >> foreignImportMap.get(ms).add(md); } else { >> // JAM resolution algorithm >> ... >> } >> } >> >> // Gets imports from foreign module systems >> // Should do this in a separate thread since this is a synchronous call >> for (ModuleSystem ms : foreignImportMap.keySet()) { >> // The getModules() method will allow a module system to know if >> // a set of ModuleDefinition are resolved in the same resolution. >> List imports = ms.getModules(wombatModDef, >> foreignImportMap.get(ms)); >> // interconnect the imports with wombat >> ... >> } >> >> // continue the module initialization process such as shadow >> validatation >> // and execute initializers >> ... >> >> // Module initialization completed >> if (succeeded) { >> com.wombat.app is now fully initialized >> } else { >> // signal the getModule method to throw ModuleInitializationException >> ... >> } >> > From Stanley.Ho at sun.com Tue Apr 29 13:46:03 2008 From: Stanley.Ho at sun.com (Stanley M. Ho) Date: Tue, 29 Apr 2008 13:46:03 -0700 Subject: Supporting OSGi Bundles in the Java Module System In-Reply-To: <481785C3.6070403@oracle.com> References: <4816930D.4080106@sun.com> <481733F9.1050902@sas.com> <481785C3.6070403@oracle.com> Message-ID: <4817890B.3090706@sun.com> Hi Bryan, This probably has to do with attachment filtering. I have posted both documents on the EG home page as separate downloads. - Stanley [1] Support OSGi bundles in Java Module System: http://jcp.org/en/eg/download/JSR277-OSGi-support.html?id=277&fileId=3785 [2] JSR 277 API Draft Spec (4/28/2008) http://www.jcp.org/en/eg/download/jsr-277-edr2-api-04282008.zip?id=277&fileId=3786 Bryan Atsatt wrote: > Somehow your original message didn't make it at all to our servers. > Could you please resend it? > > // Bryan From heavy at UNGOVERNED.ORG Tue Apr 29 14:10:52 2008 From: heavy at UNGOVERNED.ORG (Richard S. Hall) Date: Tue, 29 Apr 2008 17:10:52 -0400 Subject: Updates on Interoperability In-Reply-To: <4817854E.1010906@oracle.com> References: <47F46ABF.1020306@sun.com> <48121BA2.7060001@oracle.com> <48123223.9070803@sas.com> <48123AE3.1090708@ungoverned.org> <4812409C.1030000@oracle.com> <4812459E.2060407@ungoverned.org> <25680D44-7C07-4DC9-841F-12751540ECD4@sampullara.com> <48126D6A.7000001@ungoverned.org> <48163DF6.3010705@oracle.com> <481743CF.2020102@ungoverned.org> <4817854E.1010906@oracle.com> Message-ID: <48178EDC.2040706@ungoverned.org> Bryan Atsatt wrote: > Richard S. Hall wrote: >> Bryan Atsatt wrote: >>> Richard S. Hall wrote: >>>> Sam Pullara wrote: >>>>> Did we decide which way it will be compatible? I'm a little >>>>> concerned that implementing something that can use OSGi modules is >>>>> a far bigger task that implementing something that OSGi can use. >>>>> If we are going for total compatibility it seems like we should >>>>> just use OSGi. Personally I wanted something in Java that was >>>>> simpler and cleaner that OSGi-users could leverage if they >>>>> wanted. But I'm coming at this from the >>>>> we-need-something-like-maven/gem/ivy camp. >>>>> >>>>> The vast majority of people that are going to use this system will >>>>> have never heard of OSGI nor care about any of their more subtle >>>>> features and just want an easy way to leverage the vast amount of >>>>> libraries available (virtually all of them are not OSGi modules >>>>> today but are jar files in maven repositories). We shouldn't be >>>>> specing out some sort of uber container but rather a simple way to >>>>> tie code to its dependencies. >>>> >>>> If this is all that people wanted, then we could just define >>>> repositories and associated metadata and forget about all runtime >>>> modularity support, but I strongly disagree that this is all people >>>> want or else there wouldn't be such an upsurge in OSGi interest and >>>> adoption. >>>> >>>>> I strongly suggest KISS applies to our effort. I write this >>>>> knowing that the committee seems to be heavily in favor of this >>>>> alternate view that we implement the kitchen sink. >>>> >>>> I think our interest is similar, but perhaps our conclusions are >>>> different. Initially, I think my comments were construed as an >>>> argument against split packages in general, but I am really arguing >>>> the same as you (I think) that we don't really need split packages >>>> to be a concept in our "interoperability API", if that is how we >>>> are to view the 277 API. Of course, I am also willing to have the >>>> former argument too, but I will leave that for another time. :-) >>>> >>>> From my point of view, I do not think it is wholly reasonable to >>>> assume that we want to try to accomplish interoperability across >>>> module systems, where that includes arbitrarily sharing split >>>> packages across module systems. >>> Nor do I, but I'm trying to understand the implications if we assume >>> that OSGi will continue to split packages. >>>> >>>> Let me try to accurately conjure the peculiarities of split >>>> packages in the OSGi module layer... >>>> >>>> We support split packages in two ways, via required bundles (i.e., >>>> module dependencies) and via bundle fragments. In general, we >>>> recommend that if you split your packages that you attach mandatory >>>> attributes on them so that importers do not get them by mistake. By >>>> requiring a bundle, you can ignore mandatory attributes and get >>>> everything a bundle has to offer and combine it with other required >>>> bundles that might be exporting other parts of the split packages. >>>> Given this, there is a semi-reasonable chance* that we can tell you >>>> which bundles are contributing to the "complete" packages, but we >>>> have no way of knowing if/when combined split packages are >>>> complete. Further, we have to assume that split package parts >>>> offered by different bundles do not overlap, because if we allowed >>>> them to overlap then we would have shadowing and ordering issues. >>>> Again we have no easy way to verify that they do not overlap, so we >>>> just assume they don't. >>>> >>>> Fragments on the other hand are a little trickier, since split >>>> packages from them are loaded from the same class loader to provide >>>> package private access (which is not afforded for split packages in >>>> the require bundle approach). The "host" bundle doesn't really know >>>> about fragments and it is possible for fragments to shadow or >>>> extend packages in the "host" bundle. Since fragments provide their >>>> split packages in an implicit way (i.e., they largely just become >>>> part of the host bundle's class path), there is no easy way to >>>> determine if a fragment is contributing to a given package or not. >>>> (* This feature of fragments also makes it so it is not really >>>> possible to know who is contributing classes to a split package in >>>> the require bundle case.) >>> Ok, so first I assume it would make sense for an OSGi Repository to >>> simply never return a ModuleDefinition for a fragment. >> >> I am not sure I understand your point. Host bundles would be returned >> and during the resolve process, fragments might be attached to that >> host, thus modifying its content in ways that we cannot predict. > I simply meant that a fragment would never be returned by itself, as a > separate ModuleDefinition. >> >>> And the "you don't know what you've got till it's resolved" problem >>> is going to be present in any module system for which package name >>> is not a required component of a dependency expression. But as long >>> as re-export is not the default behavior (it isn't), doesn't this >>> really just boil down to the leakage problem? And this is addressed >>> in 277 the same as it is in OSGi: you must declare such "leaks" as >>> re-exports ("uses"). Not perfect, but probably good enough. >>> >>> So I think we're left with the issue of a single search visiting >>> multiple loaders that can return the same package. This won't happen >>> in the fragment case, since the "split" gets mended at runtime, but >>> it will in the require-bundle case. And this will manifest as >>> multiple entries in the import list which export the same package. >>> >>> But even this isn't an issue, as long as dependency resolution >>> relies solely on declared exports and re-exports of a module, and >>> not on what is available by browsing the imports of that module. >>> >>> So I'm back to my original statement that we just need to >>> acknowledge that package duplication may exist in an import list. >>> >>> Or am I missing something here? >> >> I am just not sure why we want to make the fact that some module >> systems support split packages a visible concept in our >> "interoperability API", when it would be difficult to get >> interoperability around split packages across module systems when we >> already have issues with split packages within a single module >> system. Especially, given my description above, since we would not >> easily know which modules were contributing to a split package in OSGi. > I was just exploring whether or not we need to make it a visible > concept in order for cross module system resolution to work correctly. > You may note in Stanley's doc that he simply took the position that a > JAM definition would fail to resolve against an OSGi definition that > split packages. This may be an acceptable solution, but it still > requires a means to *detect* that a split has occurred. Understood and that is precisely the point...it is difficult to determine that a split has occurred...perhaps not impossible, but the OSGi spec avoided dealing with it due to the inherent complexity. For the most part, if you start splitting packages, you are on your own and don't get much explicit support from the OSGi framework. -> richard >> >>>> >>>> I think that is fairly complete description of the situation, sorry >>>> if it is somewhat terse. In summary, I wouldn't assume that we can >>>> get reasonable interoperability at this level. If you need these >>>> types of features, then you should stick to a single modularity >>>> framework...preferably OSGi. ;-) >>>> >>>> But my overall point is, I believe that if we can support >>>> interoperability at module- and package-level dependencies across >>>> module systems, then I think we would hit most use cases. >>>> >>>> -> richard >>>> >>>>> >>>>> Sam >>>>> >>>>> On Apr 25, 2008, at 1:57 PM, Richard S. Hall wrote: >>>>> >>>>>> Bryan Atsatt wrote: >>>>>>> Richard S. Hall wrote: >>>>>>>> Gordon Hirsch wrote: >>>>>>>>>> 1. Map its dependency declarations into a standard runtime >>>>>>>>>> representation. >>>>>>>>>> 2. Support a standard search model over its stored modules, >>>>>>>>>> using the runtime dependency expressions. >>>>>>>>>> 3. Map its stored module data into a standard runtime >>>>>>>>>> representation that can be returned by the search. >>>>>>>>> >>>>>>>>> One challenge lies in defining the "standard runtime >>>>>>>>> representations" and "standard search model" in a universal >>>>>>>>> enough way to encompass OSGi and other module systems. This >>>>>>>>> implies embracing concepts (in these standard representations >>>>>>>>> and search model) that were not universally liked by the EG >>>>>>>>> early on. (Split packages and package-level import/export come >>>>>>>>> to mind.) >>>>>>>> >>>>>>>> Package-level import/export seem like a must, but I still don't >>>>>>>> see split packages as a necessity. >>>>>>> Strongly agreed on import-by-package, and that ModuleDefinition >>>>>>> requires a method to enumerate exported packages (and probably >>>>>>> an exportsPackage(String packageName) method to enable an >>>>>>> efficient Query). >>>>>>> On the split package front, however, it seems a little fuzzy to >>>>>>> me. If an OSGi implementation of 277 is also going to remain >>>>>>> OSGi Rx compliant, it will still need to support split packages, >>>>>>> right? If so, don't we need to surface this fact at the 277 >>>>>>> level? Perhaps that is as simple as acknowledging that >>>>>>> Module.getImportedModules() may return more than one instance >>>>>>> that exports a given package. >>>>>> >>>>>> That all depends on if the 277 API is a subset of OSGi >>>>>> functionality or equal to it, I suppose. >>>>>> >>>>>> -> richard >>>>>>> >>>>>>> // Bryan >>>> >>