From pavel.genevski at SAP.COM Wed Dec 5 04:48:46 2007 From: pavel.genevski at SAP.COM (Genevski, Pavel) Date: Wed, 05 Dec 2007 13:48:46 +0100 Subject: Thread context class loader... In-Reply-To: A References: <46EF3E2C.7090609@oracle.com> A <6.2.5.6.2.20070918103951.02c8a2f8@bea.com> A Message-ID: Hi, Do you have any comments on my suggestions below ? And in addition, can you point me to the source code locations of the reference implementation of this JSR ? Best Regards, Pavel Genevski Deployment Team SAP Labs Bulgaria Tel. +359 2 9157 133 -----Original Message----- From: Java Community Process JSR #277 Expert List [mailto:JSR-277-EG at JCP.ORG] On Behalf Of Genevski, Pavel Sent: Wednesday, 17. October 2007 11:41 To: JSR-277-EG at JCP.ORG Subject: Re: Thread context class loader... Hi, While we are talking about classloading I would like to add a few comments about the cyclic dependencies mentioned in section 8.3.4 of the spec. How do you imagine the classloading with cyclic dependencies? What are the benefits of allowing cyclic dependencies ? In my opinion cylcic dependencies occur mostly because of bad system architecture and are always hard to handle. The tough questions that emerge with them are for example: - what should be built first - which classloader should be created first. One obvious runtime solution for classloading of modules with cyclic dependencies is to have a common classloader for all the modules in the cycle, but are there enough benefits that come from cyclic dependencies that are worth doing this and breaking module isolation? Best Regards, Pavel Genevski Deployment Team SAP Labs Bulgaria Tel. +359 2 9157 133 -----Original Message----- From: Java Community Process JSR #277 Expert List [mailto:JSR-277-EG at JCP.ORG] On Behalf Of Andy Piper Sent: Tuesday, 18. September 2007 12:44 To: JSR-277-EG at JCP.ORG Subject: [LIKELY JUNK]Re: Thread context class loader... At 03:55 18/09/2007, Bryan Atsatt wrote: >Thoughts? Given our experience with OSGi I agree that this needs addressing. We have found, though, that the notion of a "thread of execution" is somewhat more amorphous in a non-JEE environment. In using Spring-OSGI we have found we often have to force the client classloader to be a delegate of the target bundle's, but there are issues with trying to actually make this switch happen. I'm wondering whether we actually need some VM-level support to make this work well. andy Notice: This email message, together with any attachments, may contain information of BEA Systems, Inc., its subsidiaries and affiliated entities, that may be confidential, proprietary, copyrighted and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by email and then delete it. From abrock at REDHAT.COM Wed Dec 5 04:53:28 2007 From: abrock at REDHAT.COM (Adrian Brock) Date: Wed, 05 Dec 2007 13:53:28 +0100 Subject: Thread context class loader... In-Reply-To: References: <46EF3E2C.7090609@oracle.com> A <6.2.5.6.2.20070918103951.02c8a2f8@bea.com> A Message-ID: <1196859208.3808.15.camel@warjort> We already discussed the cyclic dependencies issue. If I remember the summary correctly: * We decided it is a problem that always exists including in the JDK itself. * Not supporting it would be a barrier to migration for existing code. * It isn't really that difficult to support * We should discourage its use, in the spec but also an implementation could have something like a system property that disallows it. On the "RI" Stanley posted this back in June: " Hi experts and observers, We have just made an early snapshot of the JSR 277 and 294 implementation available through the Modules project on openjdk: http://openjdk.java.net/projects/modules/ This snapshot basically covers the basic features described in the updated specification in JSR 277, and the reflective APIs defined in JSR 294. You should expect the snapshot will be updated periodically going forwards, as we continue to evolve the implementation and adding many more functionalities in the next few months. We will continue to focus our discussions in this EG mailing list for the JSR 277 specification. For those of you interested in discussing the implementation in the Modules project, we have made two discussion lists available to the Java community: * modules-dev at openjdk.java.net Technical discussion about the implementation of the Modules project * modules-discuss at openjdk.java.net General discussion of the Modules project and how to use modules If you are interested in discussing the Modules project with its developers, or in discussing how to use JSR 277 and JSR 294, be a subscriber today! - Stanley " On Wed, 2007-12-05 at 13:48 +0100, Genevski, Pavel wrote: > Hi, > > Do you have any comments on my suggestions below ? > And in addition, can you point me to the source code locations of the > reference implementation of this JSR ? > > > Best Regards, > Pavel Genevski > Deployment Team > SAP Labs Bulgaria > Tel. +359 2 9157 133 > > -----Original Message----- > From: Java Community Process JSR #277 Expert List > [mailto:JSR-277-EG at JCP.ORG] On Behalf Of Genevski, Pavel > Sent: Wednesday, 17. October 2007 11:41 > To: JSR-277-EG at JCP.ORG > Subject: Re: Thread context class loader... > > Hi, > > While we are talking about classloading I would like to add a few > comments about the cyclic dependencies mentioned in section 8.3.4 of the > spec. How do you imagine the classloading with cyclic dependencies? What > are the benefits of allowing cyclic dependencies ? > > In my opinion cylcic dependencies occur mostly because of bad system > architecture and are always hard to handle. The tough questions that > emerge with them are for example: > > - what should be built first > - which classloader should be created first. > > One obvious runtime solution for classloading of modules with cyclic > dependencies is to have a common classloader for all the modules in the > cycle, but are there enough benefits that come from cyclic dependencies > that are worth doing this and breaking module isolation? > > > Best Regards, > Pavel Genevski > Deployment Team > SAP Labs Bulgaria > Tel. +359 2 9157 133 > > > > -----Original Message----- > From: Java Community Process JSR #277 Expert List > [mailto:JSR-277-EG at JCP.ORG] On Behalf Of Andy Piper > Sent: Tuesday, 18. September 2007 12:44 > To: JSR-277-EG at JCP.ORG > Subject: [LIKELY JUNK]Re: Thread context class loader... > > At 03:55 18/09/2007, Bryan Atsatt wrote: > >Thoughts? > > Given our experience with OSGi I agree that this needs addressing. We > have found, though, that the notion of a "thread of execution" is > somewhat more amorphous in a non-JEE environment. In using > Spring-OSGI we have found we often have to force the client > classloader to be a delegate of the target bundle's, but there are > issues with trying to actually make this switch happen. I'm wondering > whether we actually need some VM-level support to make this work well. > > andy > > > Notice: This email message, together with any attachments, may contain > information of BEA Systems, Inc., its subsidiaries and affiliated > entities, that may be confidential, proprietary, copyrighted and/or > legally privileged, and is intended solely for the use of the individual > or entity named in this message. If you are not the intended recipient, > and have received this message in error, please immediately return this > by email and then delete it. -- xxxxxxxxxxxxxxxxxxxxxxxxxxxx Adrian Brock Chief Scientist JBoss, a division of Red Hat xxxxxxxxxxxxxxxxxxxxxxxxxxxx From pavel.genevski at SAP.COM Wed Dec 5 07:48:25 2007 From: pavel.genevski at SAP.COM (Genevski, Pavel) Date: Wed, 05 Dec 2007 16:48:25 +0100 Subject: Thread context class loader... In-Reply-To: A <1196859208.3808.15.camel@warjort> References: <46EF3E2C.7090609@oracle.com> A <6.2.5.6.2.20070918103951.02c8a2f8@bea.com> A A <1196859208.3808.15.camel@warjort> Message-ID: Hi Adrian, I am particularily interested in this statement: " * It isn't really that difficult to support " What do you exactly mean ? What is the current mechanism for supporting it ? Also I would like to mention that even if we do not allow cyclic dependencies it will still be possible to migrate the old code. All the classes from a cycle should be contained in a single module. I am afraid that if we allow cyclic dependencies in the specification it will be widely abused and the outcome will be negative. We had such bad experience in the past. From my point of view the specification should encourage the people to create good designs not bad ones with the excuse for backward compatibility. Best Regards, Pavel Genevski Deployment Team SAP Labs Bulgaria Tel. +359 2 9157 133 -----Original Message----- From: Java Community Process JSR #277 Expert List [mailto:JSR-277-EG at JCP.ORG] On Behalf Of Adrian Brock Sent: Wednesday, 5. December 2007 14:53 To: JSR-277-EG at JCP.ORG Subject: Re: Thread context class loader... We already discussed the cyclic dependencies issue. If I remember the summary correctly: * We decided it is a problem that always exists including in the JDK itself. * Not supporting it would be a barrier to migration for existing code. * It isn't really that difficult to support * We should discourage its use, in the spec but also an implementation could have something like a system property that disallows it. On the "RI" Stanley posted this back in June: " Hi experts and observers, We have just made an early snapshot of the JSR 277 and 294 implementation available through the Modules project on openjdk: http://openjdk.java.net/projects/modules/ This snapshot basically covers the basic features described in the updated specification in JSR 277, and the reflective APIs defined in JSR 294. You should expect the snapshot will be updated periodically going forwards, as we continue to evolve the implementation and adding many more functionalities in the next few months. We will continue to focus our discussions in this EG mailing list for the JSR 277 specification. For those of you interested in discussing the implementation in the Modules project, we have made two discussion lists available to the Java community: * modules-dev at openjdk.java.net Technical discussion about the implementation of the Modules project * modules-discuss at openjdk.java.net General discussion of the Modules project and how to use modules If you are interested in discussing the Modules project with its developers, or in discussing how to use JSR 277 and JSR 294, be a subscriber today! - Stanley " On Wed, 2007-12-05 at 13:48 +0100, Genevski, Pavel wrote: > Hi, > > Do you have any comments on my suggestions below ? > And in addition, can you point me to the source code locations of the > reference implementation of this JSR ? > > > Best Regards, > Pavel Genevski > Deployment Team > SAP Labs Bulgaria > Tel. +359 2 9157 133 > > -----Original Message----- > From: Java Community Process JSR #277 Expert List > [mailto:JSR-277-EG at JCP.ORG] On Behalf Of Genevski, Pavel > Sent: Wednesday, 17. October 2007 11:41 > To: JSR-277-EG at JCP.ORG > Subject: Re: Thread context class loader... > > Hi, > > While we are talking about classloading I would like to add a few > comments about the cyclic dependencies mentioned in section 8.3.4 of the > spec. How do you imagine the classloading with cyclic dependencies? What > are the benefits of allowing cyclic dependencies ? > > In my opinion cylcic dependencies occur mostly because of bad system > architecture and are always hard to handle. The tough questions that > emerge with them are for example: > > - what should be built first > - which classloader should be created first. > > One obvious runtime solution for classloading of modules with cyclic > dependencies is to have a common classloader for all the modules in the > cycle, but are there enough benefits that come from cyclic dependencies > that are worth doing this and breaking module isolation? > > > Best Regards, > Pavel Genevski > Deployment Team > SAP Labs Bulgaria > Tel. +359 2 9157 133 > > > > -----Original Message----- > From: Java Community Process JSR #277 Expert List > [mailto:JSR-277-EG at JCP.ORG] On Behalf Of Andy Piper > Sent: Tuesday, 18. September 2007 12:44 > To: JSR-277-EG at JCP.ORG > Subject: [LIKELY JUNK]Re: Thread context class loader... > > At 03:55 18/09/2007, Bryan Atsatt wrote: > >Thoughts? > > Given our experience with OSGi I agree that this needs addressing. We > have found, though, that the notion of a "thread of execution" is > somewhat more amorphous in a non-JEE environment. In using > Spring-OSGI we have found we often have to force the client > classloader to be a delegate of the target bundle's, but there are > issues with trying to actually make this switch happen. I'm wondering > whether we actually need some VM-level support to make this work well. > > andy > > > Notice: This email message, together with any attachments, may contain > information of BEA Systems, Inc., its subsidiaries and affiliated > entities, that may be confidential, proprietary, copyrighted and/or > legally privileged, and is intended solely for the use of the individual > or entity named in this message. If you are not the intended recipient, > and have received this message in error, please immediately return this > by email and then delete it. -- xxxxxxxxxxxxxxxxxxxxxxxxxxxx Adrian Brock Chief Scientist JBoss, a division of Red Hat xxxxxxxxxxxxxxxxxxxxxxxxxxxx From abrock at REDHAT.COM Wed Dec 5 08:21:25 2007 From: abrock at REDHAT.COM (Adrian Brock) Date: Wed, 05 Dec 2007 17:21:25 +0100 Subject: Thread context class loader... In-Reply-To: References: <46EF3E2C.7090609@oracle.com> A <6.2.5.6.2.20070918103951.02c8a2f8@bea.com> A A <1196859208.3808.15.camel@warjort> Message-ID: <1196871685.3808.34.camel@warjort> On Wed, 2007-12-05 at 16:48 +0100, Genevski, Pavel wrote: > Hi Adrian, > > I am particularily interested in this statement: > > " * It isn't really that difficult to support " > > What do you exactly mean ? What is the current mechanism for supporting > it ? I haven't looked at how Stanley supports them. Here's a flavour of the previous discussions (selected to answer your question) on the topic from my mail archive. Georgi and Richard discussing arguments against cycles and how hard they are to implement: " Danov, Georgi wrote: > Hi, > I know it is closed now, however, cycles are something too serious (for me) to let it just go : > - I know very well the importance of backward compatibility and the pain of fixing your old mistakes that have become de-facto standards > - the fact that SE itself has cycles should make it's developers think if they really need them. It would be interesting for me to understand, why do we have cycles in SE and what imposed their existence. > - I am still behind the statement that cycles mean simply *bad* design, and I've witnessed how huge product was de-cycled successfully. > - why are cycles bad? Because they create clouds of components that easily interconnect and create even bigger clouds. At the end, in order to stop single component, you need to bring the whole program/stack down. > - in the scope of small to middle application, which does not need to be modular, cycles work and are not big problem. However, if you think about platforms, that provide runtime capabilities, cycles harm their operation. > I am not trying to say that cycles are good, but I don't think these arguments against them are compelling. For example, any coupling among components, whether a cycle or not, creates a "cloud of components". I agree that cycles should generally be avoided, but supporting cycles is not the same as encouraging cycles. > My proposal is that the runtime can run with cycle support, but by default this is off. This way legacy products will have alternative, plus affordable transition plan. Freshly developed components will not make use of cycles by default, so hopefully we see growing ecosystem of cycle-free components. > Another, little more radical proposal is to allow cycles only for legacy jars that are wrapped as modules. Cycles between 'normal' modules should not be allowed. > It seems like your fear is that by supporting cycles, then everyone will try to create cycles every chance they get. I don't think this is the case. OSGi has always supported cycles, but it is still exceptional to find the situation (even though it does happen). -> richard > Georgi Danov > NW JST Deployment > Java EE 5 at SAP " Stanley and Richard discussing the complexity issue raised by an early draft reviewer: " > I think we have briefly talked about cyclic dependency many months ago, > but I don't think we have fully discussed it. The early draft currently > assumes cyclic dependency is supported. > > NetBeans manages to work well without supporting cyclic dependencies, > and same as .NET (Visual Studio detects and prevents circular > dependencies to be created in assemblies). On the other hand, cyclic > dependency is supported in OSGi, Ivy, and Maven. > > I agreed with the reviewer that supporting cyclic dependencies adds > complexity to the module system, and many cyclic dependencies can be > reduced by proper modularization. The fact that NetBeans and .NET manage > to live without cyclic dependencies shows that it is possible to produce > a module system with no cyclic dependency support that can still satisfy > developers. On the other hand, while direct cyclic dependency can be > detected and avoided, it may be more difficult for developers to avoid > transitive cyclic dependency. I won't sit here and argue that cyclic dependencies are a must have feature, but I do think that they are not something that can always be avoided, especially in the transitive case and even in the case where you have closely related modules where you still want separation so that you can have alternate subsystem implementations and independent updatability. However, as someone who has implemented large portions of the OSGi R4 resolver algorithm, I can say that supporting cycles was not a significant source of complication in my code...that was actually the easy part. :-) -> richard " Glynn arguing for cycles such that modularization can be incremental: " A use case for cyclic dependencies is modularising a large, existing application. Cyclic dependency support makes it possible to adopt an incremental approach: 1. overlay existing components with modules and define or generate module dependencies based on the existing code dependencies 2. regression test to make sure nothing is broken 3. analyse module dependencies e.g. to spot opportunities for a more layered architecture 4. incrementally apply the improvements identified in #3, regression testing after each set of changes. Note that #4 may take more than one release to complete. Without cyclic dependency support, the approach tends to be to wrap the whole application in one big module and then gradually rework the code structure to permit modules to be broken out into an acyclic dependency graph. This approach feels quite different - there is a higher *perceived* cost to modularisation and a greater inertia inertia in making progress. I think this approach doesn't lend itself so well to parallel development. There are also some admittedly relatively rare but important cases of pairs (or groups) of modules in the same subsystem that actually need cyclic dependencies because of their close relationship but which benefit from being split into separate modules because of the way the responsibilities split between the modules. Glyn " Andy and Stanley discussing tools and warning about cycles: " First, I hope everyone had a wonderful holiday break, and have a great 2007! Andy Piper wrote: > I think tools provided in Java SE should support cycles and creation > of modules with cycles. If its optional it won't get done and you may > as well prohibit them. I agreed the tools provided in the JDK should support cycles and creation of modules with cycles. It's the tools outside the JDK that I think should have the choice to support cyclic dependency optionally. > In terms of defaults I think you should add options to -Xlint or > something equivalent. But if you want something stronger that's > probably ok as well. So your suggestion is to warn developers when cyclic dependency is detected, but not necessarily treat it as an error, right? - Stanley " -- xxxxxxxxxxxxxxxxxxxxxxxxxxxx Adrian Brock Chief Scientist JBoss, a division of Red Hat xxxxxxxxxxxxxxxxxxxxxxxxxxxx From heavy at UNGOVERNED.ORG Wed Dec 5 10:18:34 2007 From: heavy at UNGOVERNED.ORG (Richard S. Hall) Date: Wed, 05 Dec 2007 13:18:34 -0500 Subject: Thread context class loader... In-Reply-To: <1196871685.3808.34.camel@warjort> References: <46EF3E2C.7090609@oracle.com> A <6.2.5.6.2.20070918103951.02c8a2f8@bea.com> A A <1196859208.3808.15.camel@warjort> <1196871685.3808.34.camel@warjort> Message-ID: <4756EB7A.2000307@ungoverned.org> And my point remains, OSGi has always supported cycles and they have not been abused. -> richard Adrian Brock wrote: > On Wed, 2007-12-05 at 16:48 +0100, Genevski, Pavel wrote: > >> Hi Adrian, >> >> I am particularily interested in this statement: >> >> " * It isn't really that difficult to support " >> >> What do you exactly mean ? What is the current mechanism for supporting >> it ? >> > > I haven't looked at how Stanley supports them. > > Here's a flavour of the previous discussions (selected to answer your > question) on the topic from my mail archive. > > Georgi and Richard discussing arguments against cycles and > how hard they are to implement: > > " > Danov, Georgi wrote: > >> Hi, >> I know it is closed now, however, cycles are something too serious >> > (for me) to let it just go : > >> - I know very well the importance of backward compatibility >> > and the pain of fixing your old mistakes that have become de-facto > standards > >> - the fact that SE itself has cycles should make it's >> > developers think if they really need them. It would be interesting for > me to understand, why do we have cycles in SE and what imposed their > existence. > >> - I am still behind the statement that cycles mean simply >> > *bad* design, and I've witnessed how huge product was de-cycled > successfully. > >> - why are cycles bad? Because they create clouds of components >> > that easily interconnect and create even bigger clouds. At the end, in > order to stop single component, you need to bring the whole > program/stack down. > >> - in the scope of small to middle application, which does not >> > need to be modular, cycles work and are not big problem. However, if you > think about platforms, that provide runtime capabilities, cycles harm > their operation. > > > I am not trying to say that cycles are good, but I don't think these > arguments against them are compelling. > > For example, any coupling among components, whether a cycle or not, > creates a "cloud of components". > > I agree that cycles should generally be avoided, but supporting cycles > is not the same as encouraging cycles. > > >> My proposal is that the runtime can run with cycle support, but by >> > default this is off. This way legacy products will have alternative, > plus affordable transition plan. Freshly developed components will not > make use of cycles by default, so hopefully we see growing ecosystem of > cycle-free components. > >> Another, little more radical proposal is to allow cycles only for >> > legacy jars that are wrapped as modules. Cycles between 'normal' modules > should not be allowed. > > > It seems like your fear is that by supporting cycles, then everyone will > try to create cycles every chance they get. I don't think this is the > case. OSGi has always supported cycles, but it is still exceptional to > find the situation (even though it does happen). > > -> richard > > >> Georgi Danov >> NW JST Deployment >> Java EE 5 at SAP >> > " > > Stanley and Richard discussing the complexity issue > raised by an early draft reviewer: > > " > >> I think we have briefly talked about cyclic dependency many months >> > ago, > >> but I don't think we have fully discussed it. The early draft >> > currently > >> assumes cyclic dependency is supported. >> >> NetBeans manages to work well without supporting cyclic dependencies, >> and same as .NET (Visual Studio detects and prevents circular >> dependencies to be created in assemblies). On the other hand, cyclic >> dependency is supported in OSGi, Ivy, and Maven. >> >> I agreed with the reviewer that supporting cyclic dependencies adds >> complexity to the module system, and many cyclic dependencies can be >> reduced by proper modularization. The fact that NetBeans and .NET >> > manage > >> to live without cyclic dependencies shows that it is possible to >> > produce > >> a module system with no cyclic dependency support that can still >> > satisfy > >> developers. On the other hand, while direct cyclic dependency can be >> detected and avoided, it may be more difficult for developers to avoid >> transitive cyclic dependency. >> > > I won't sit here and argue that cyclic dependencies are a must have > feature, but I do think that they are not something that can always be > avoided, especially in the transitive case and even in the case where > you have closely related modules where you still want separation so that > you can have alternate subsystem implementations and independent > updatability. > > However, as someone who has implemented large portions of the OSGi R4 > resolver algorithm, I can say that supporting cycles was not a > significant source of complication in my code...that was actually the > easy part. :-) > > -> richard > " > > Glynn arguing for cycles such that modularization can > be incremental: > > " > A use case for cyclic dependencies is modularising a large, existing > application. Cyclic dependency support makes it possible to adopt an > incremental approach: > > 1. overlay existing components with modules and define or generate > module > dependencies based on the existing code dependencies > 2. regression test to make sure nothing is broken > 3. analyse module dependencies e.g. to spot opportunities for a more > layered architecture > 4. incrementally apply the improvements identified in #3, regression > testing after each set of changes. > > Note that #4 may take more than one release to complete. > > Without cyclic dependency support, the approach tends to be to wrap the > whole application in one big module and then gradually rework the code > structure to permit modules to be broken out into an acyclic dependency > graph. This approach feels quite different - there is a higher > *perceived* > cost to modularisation and a greater inertia inertia in making progress. > I > think this approach doesn't lend itself so well to parallel development. > > There are also some admittedly relatively rare but important cases of > pairs > (or groups) of modules in the same subsystem that actually need cyclic > dependencies because of their close relationship but which benefit from > being split into separate modules because of the way the > responsibilities > split between the modules. > > Glyn > " > > Andy and Stanley discussing tools and warning about cycles: > > " > First, I hope everyone had a wonderful holiday break, and have a great > 2007! > > Andy Piper wrote: > >> I think tools provided in Java SE should support cycles and creation >> of modules with cycles. If its optional it won't get done and you may >> as well prohibit them. >> > > I agreed the tools provided in the JDK should support cycles and > creation of modules with cycles. It's the tools outside the JDK that I > think should have the choice to support cyclic dependency optionally. > > >> In terms of defaults I think you should add options to -Xlint or >> something equivalent. But if you want something stronger that's >> probably ok as well. >> > > So your suggestion is to warn developers when cyclic dependency is > detected, but not necessarily treat it as an error, right? > > - Stanley > " >