From paul.sandoz at oracle.com Wed Aug 1 07:46:19 2012 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 1 Aug 2012 07:46:19 -0700 Subject: Service provider module resolution Re: Services/configuration/context webrevs In-Reply-To: <50187F70.5060805@oracle.com> References: <4FF5101B.3010907@oracle.com> <339D0144-4FBE-4557-AF2B-68B9CE38257C@oracle.com> <50187F70.5060805@oracle.com> Message-ID: <014C6BF2-1990-4AC6-8E8B-F5CD37864F55@oracle.com> On Jul 31, 2012, at 5:59 PM, Mandy Chung wrote: > On 7/9/2012 8:52 AM, Paul Sandoz wrote: >> Hi, >> >> The following webrev fixes the nits. Many service-based tests are added to _Configurator.java, which makes most of the service-based *.sh tests redundant so those redundant ones removed. I added a runtime service-based test [*] that ensures correct behavior for all three ServiceLoader methods. >> >> http://cr.openjdk.java.net/~psandoz/jigsaw/resolver-services/webrev.3/ > > Looks good. Thanks for looking at this. > It's good to see the service-based tests added to > _Configurator.java and cleanup the services/*.sh tests. Minor comment: > > test/org/openjdk/jigsaw/_Configurator.java > It might be worth adding a test to check that the service provider > module's exported package is not visible to the service consumer module > (e.g. x requires service S and the context("s at 1") has no remotePackage > to the package exported by y that provides S.) > > L2: typo "2021" -> "2012" > http://cr.openjdk.java.net/~psandoz/jigsaw/resolver-services/webrev.4/ > ServiceLoaderTest.java > nit: L116-118, L124-126, L132-134 - could be refactored out as a method. > I'm fine if you prefer to leave it as it is. > I will leave it as is. If i let rip with TestNG :-) this could change. >> >> >> I also added tests that show the permits behavior is general and not specific to services: >> >> ... > > Thanks for adding these tests. It's a general issue. As we discussed > it offline today, it'll be addressed in a separate changeset. I know > Alan is going to review it too. Once he reviews it, I can push this > changeset for you. > OK, thanks, Paul. From sean.mullan at oracle.com Wed Aug 1 11:10:01 2012 From: sean.mullan at oracle.com (Sean Mullan) Date: Wed, 01 Aug 2012 14:10:01 -0400 Subject: Review Request: loading JCE providers using ServiceLoader In-Reply-To: References: <50098E48.7010903@oracle.com> Message-ID: <501970F9.9050908@oracle.com> On 07/31/2012 12:26 AM, Paul Sandoz wrote: > Hi Sean, > > The patch looks OK to me. > > You have a redundant synchronised block in > Provider#doLoadProviderUsingServiceLoader you might want to remove. The lock is on the ServiceLoader and I think that's still necessary. There's a ProviderConfig created for each Provider. It's possible there may be more than one thread accessing the ServiceLoader which is currently shared by all Providers. > You might wanna put some comments in as to why you are caching the > ServiceLoader instance. IIRC you are trying to reduce the amount of > code changes to ProviderList while ensuring service instance > selection is efficient, is that so? Ok. I'm working on another webrev for review soon. --Sean > > Paul. > > On Jul 20, 2012, at 9:58 AM, Sean Mullan > wrote: > >> Please review my code changes to enable JCE providers to be >> registered and loaded as services using the ServiceLoader API: >> >> http://cr.openjdk.java.net/~mullan/jigsaw/webrevs/jce-service/webrev.00/ >> >> >> There is one outstanding issue that needs to be resolved, but I think we can try >> to resolve this later: >> >> You cannot load more than one PKCS11 provider (ex: with different >> configurations), as currently supported and documented in the >> PKCS11 guide [1]. >> >> Thanks, Sean >> >> [1] >> http://docs.oracle.com/javase/7/docs/technotes/guides/security/p11guide.html > From paul.sandoz at oracle.com Wed Aug 1 11:58:05 2012 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 1 Aug 2012 11:58:05 -0700 Subject: Review Request: loading JCE providers using ServiceLoader In-Reply-To: <501970F9.9050908@oracle.com> References: <50098E48.7010903@oracle.com> <501970F9.9050908@oracle.com> Message-ID: On Aug 1, 2012, at 11:10 AM, Sean Mullan wrote: > On 07/31/2012 12:26 AM, Paul Sandoz wrote: >> Hi Sean, >> >> The patch looks OK to me. >> >> You have a redundant synchronised block in >> Provider#doLoadProviderUsingServiceLoader you might want to remove. > > The lock is on the ServiceLoader and I think that's still necessary. There's a ProviderConfig created for each Provider. It's possible there may be more than one thread accessing the ServiceLoader which is currently shared by all Providers. Doh! yes, the reference is statically held. Paul. From sean.mullan at oracle.com Wed Aug 1 13:12:33 2012 From: sean.mullan at oracle.com (Sean Mullan) Date: Wed, 01 Aug 2012 16:12:33 -0400 Subject: Review Request: loading JCE providers using ServiceLoader In-Reply-To: References: <50098E48.7010903@oracle.com> <501970F9.9050908@oracle.com> Message-ID: <50198DB1.2060707@oracle.com> I discovered another issue with respect to using a ServiceLoader to load JCE providers. In our current JDK implementation, if you try to load a Provider using Security.getProvider(String) and the Provider's ctor throws a ProviderException, this gets passed through to the caller. This type of behavior is difficult to preserve when using a ServiceLoader. I can catch a ServiceConfigurationError and check the cause to see if is an instanceof ProviderException, but I don't necessarily know what Provider threw the exception. Do you see the problem? If the ServiceConfigurationError also captured the name of the class, then I could figure out which Provider was throwing the exception and pass it back through. Thanks, Sean From paul.sandoz at oracle.com Wed Aug 1 15:04:58 2012 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 1 Aug 2012 15:04:58 -0700 Subject: Review Request: loading JCE providers using ServiceLoader In-Reply-To: <50198DB1.2060707@oracle.com> References: <50098E48.7010903@oracle.com> <501970F9.9050908@oracle.com> <50198DB1.2060707@oracle.com> Message-ID: <761B3313-0C66-4AC3-B1E1-1F0735A50A74@oracle.com> Hi Sean, Unfortunately there is no ability to to ask ServiceLoader to "jump to" a specific service provider class and instantiate, if present, or just iterate through the class names, nor is there currently a way to get the offending class name from SCE. So you may have to resort to doing something unpleasant with the SCE message. However, i think there may be a more serious problem. You are caching a reference to ServiceLoader, and that will cache the service instances that were successfully instantiated via iteration. Basically you may get an SCE on the first iteration but you will not get an SCE on the second and subsequent iterations. Paul. On Aug 1, 2012, at 1:12 PM, Sean Mullan wrote: > I discovered another issue with respect to using a ServiceLoader to load JCE providers. In our current JDK implementation, if you try to load a Provider using Security.getProvider(String) and the Provider's ctor throws a ProviderException, this gets passed through to the caller. > > This type of behavior is difficult to preserve when using a ServiceLoader. I can catch a ServiceConfigurationError and check the cause to see if is an instanceof ProviderException, but I don't necessarily know what Provider threw the exception. Do you see the problem? If the ServiceConfigurationError also captured the name of the class, then I could figure out which Provider was throwing the exception and pass it back through. > > Thanks, > Sean > From Alan.Bateman at oracle.com Sun Aug 5 14:00:19 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 05 Aug 2012 22:00:19 +0100 Subject: Service provider module resolution Re: Services/configuration/context webrevs In-Reply-To: <50187F70.5060805@oracle.com> References: <4FF5101B.3010907@oracle.com> <339D0144-4FBE-4557-AF2B-68B9CE38257C@oracle.com> <50187F70.5060805@oracle.com> Message-ID: <501EDEE3.8010705@oracle.com> On 01/08/2012 01:59, Mandy Chung wrote: > > Thanks for adding these tests. It's a general issue. As we discussed > it offline today, it'll be addressed in a separate changeset. I know > Alan is going to review it too. Once he reviews it, I can push this > changeset for you. Sorry for the delay, but I have finally got time to go this (webrev.04). Overall I've come around to agreeing with approach. Clearly we could conjure up cases where the resolver would find a solution today but not find a solution with the changes. On the other hand it is very important that the solution (and selected versions) can be explained to the user and it isn't nice that the dependencies of a service module can change the outcome. So I think the approach makes things more predictable and should help the diagnostic output too. I went through the resolver changes in detail and don't see anything obvious wrong. Naming of variables is a bit more verbose than the existing code but that's no big deal. A minor point is that serviceInterfaceQueue is a LinkedList so the contains check will be O(n). No big deal now but worth adding a comment as such usages have a habit of biting us later. I also went through the test changes and the replacement of the shell tests with tests based _Configurator/ModuleInfoBuilder and I think it looks good (took me a few minutes to properly tune into this but overall I think the test coverage has improved quite a bit). I guess I probably would drop the "runtime" directory and move runtime.sh up one level as services.sh. So overall I think is a good improvement and good work. (Mandy - as I was going through the tests I notice we've got the GPL with Classpath exception header on the tests. Also did we discuss -esa and -ea recently? Usually they are passed through via the jtreg -vmoption rather than needing to be in each shell test). -Alan. From paul.sandoz at Oracle.com Mon Aug 6 03:41:42 2012 From: paul.sandoz at Oracle.com (Paul Sandoz) Date: Mon, 6 Aug 2012 12:41:42 +0200 Subject: Service provider module resolution Re: Services/configuration/context webrevs In-Reply-To: <501EDEE3.8010705@oracle.com> References: <4FF5101B.3010907@oracle.com> <339D0144-4FBE-4557-AF2B-68B9CE38257C@oracle.com> <50187F70.5060805@oracle.com> <501EDEE3.8010705@oracle.com> Message-ID: <0DB58D3D-A24F-4F32-BA24-C5F744CA9955@Oracle.com> On Aug 5, 2012, at 11:00 PM, Alan Bateman wrote: > On 01/08/2012 01:59, Mandy Chung wrote: >> >> Thanks for adding these tests. It's a general issue. As we discussed >> it offline today, it'll be addressed in a separate changeset. I know >> Alan is going to review it too. Once he reviews it, I can push this >> changeset for you. > Sorry for the delay, but I have finally got time to go this (webrev.04). > > Overall I've come around to agreeing with approach. Clearly we could conjure up cases where the resolver would find a solution today but not find a solution with the changes. On the other hand it is very important that the solution (and selected versions) can be explained to the user and it isn't nice that the dependencies of a service module can change the outcome. So I think the approach makes things more predictable and should help the diagnostic output too. > And i am gonna do the same for optional dependencies too, so this code is gonna change again. Thus the mandatory dependencies have to resolve and are not affected by any optional dependencies (service-based or otherwise). > I went through the resolver changes in detail and don't see anything obvious wrong. Naming of variables is a bit more verbose than the existing code but that's no big deal. A minor point is that serviceInterfaceQueue is a LinkedList so the contains check will be O(n). No big deal now but worth adding a comment as such usages have a habit of biting us later. > I was tempted to write a Deque implementation that was based on linked hash set :-) It's gonna change soon with support for optional dependencies, so there is no point adding such a comment. Basically there is no need to cache optional stuff (services or otherwise) while traversing the dependency graph. Instead linear traversal of the partial configuration can be performed i.e. iteration of Resolver.modules, which needs to change to preserve order (and in general we need to preserve the topological order of the modules through out the resolving and linking process). > I also went through the test changes and the replacement of the shell tests with tests based _Configurator/ModuleInfoBuilder and I think it looks good (took me a few minutes to properly tune into this but overall I think the test coverage has improved quite a bit). I guess I probably would drop the "runtime" directory and move runtime.sh up one level as services.sh. > No, it needs to be separated out as a project, there is exploded source, and i want to open it in a IDE (plus it will be updated when moving to TestNG, notice that there is a test dimension for testing the various class loaders, a TestNG data provider can be used to abstract this). We also may have other projects in the future. When this patch goes in i will be updating the runtime tests to include more test code related to service instance iteration failure. > So overall I think is a good improvement and good work. > Thanks. Paul. > (Mandy - as I was going through the tests I notice we've got the GPL with Classpath exception header on the tests. Also did we discuss -esa and -ea recently? Usually they are passed through via the jtreg -vmoption rather than needing to be in each shell test). > > -Alan. From mandy.chung at oracle.com Mon Aug 6 09:16:32 2012 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 06 Aug 2012 09:16:32 -0700 Subject: Service provider module resolution Re: Services/configuration/context webrevs In-Reply-To: <501EDEE3.8010705@oracle.com> References: <4FF5101B.3010907@oracle.com> <339D0144-4FBE-4557-AF2B-68B9CE38257C@oracle.com> <50187F70.5060805@oracle.com> <501EDEE3.8010705@oracle.com> Message-ID: <501FEDE0.3030805@oracle.com> On 8/5/2012 2:00 PM, Alan Bateman wrote: > (Mandy - as I was going through the tests I notice we've got the GPL > with Classpath exception header on the tests. We shall fix that. There should be a script to patch the copyright header/legal notice. Let me patch that. > Also did we discuss -esa and -ea recently? Usually they are passed > through via the jtreg -vmoption rather than needing to be in each > shell test). We discussed that [1] and it's handy to leave the -esa -ea option there during our prototype since we sometimes run the tests directly rather than via jtreg. Mandy [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2012-April/002391.html From Alan.Bateman at oracle.com Tue Aug 7 02:36:14 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 07 Aug 2012 10:36:14 +0100 Subject: Service provider module resolution Re: Services/configuration/context webrevs In-Reply-To: <0DB58D3D-A24F-4F32-BA24-C5F744CA9955@Oracle.com> References: <4FF5101B.3010907@oracle.com> <339D0144-4FBE-4557-AF2B-68B9CE38257C@oracle.com> <50187F70.5060805@oracle.com> <501EDEE3.8010705@oracle.com> <0DB58D3D-A24F-4F32-BA24-C5F744CA9955@Oracle.com> Message-ID: <5020E18E.5050607@oracle.com> On 06/08/2012 11:41, Paul Sandoz wrote: > : > And i am gonna do the same for optional dependencies too, so this code is gonna change again. Thus the mandatory dependencies have to resolve and are not affected by any optional dependencies (service-based or otherwise). This make sense. > : > I was tempted to write a Deque implementation that was based on linked hash set :-) It's gonna change soon with support for optional dependencies, so there is no point adding such a comment. > > Basically there is no need to cache optional stuff (services or otherwise) while traversing the dependency graph. Instead linear traversal of the partial configuration can be performed i.e. iteration of Resolver.modules, which needs to change to preserve order (and in general we need to preserve the topological order of the modules through out the resolving and linking process). Okay, just something I notice when going through the changes. > : > No, it needs to be separated out as a project, there is exploded source, and i want to open it in a IDE (plus it will be updated when moving to TestNG, notice that there is a test dimension for testing the various class loaders, a TestNG data provider can be used to abstract this). We also may have other projects in the future. > > When this patch goes in i will be updating the runtime tests to include more test code related to service instance iteration failure. > My comment was actually about the additional level rather the source tree. When open in an IDE or with TestNG then I assume that the shell test isn't interesting and could remain at the same level as the other shell scripts. If this is all going to be reorganized then it's no big deal but it is critical that all the infrastructure we use support TestNG before we can really move over. -Alan. From paul.sandoz at oracle.com Tue Aug 7 02:57:19 2012 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 7 Aug 2012 11:57:19 +0200 Subject: Service provider module resolution Re: Services/configuration/context webrevs In-Reply-To: <5020E18E.5050607@oracle.com> References: <4FF5101B.3010907@oracle.com> <339D0144-4FBE-4557-AF2B-68B9CE38257C@oracle.com> <50187F70.5060805@oracle.com> <501EDEE3.8010705@oracle.com> <0DB58D3D-A24F-4F32-BA24-C5F744CA9955@Oracle.com> <5020E18E.5050607@oracle.com> Message-ID: <4A77928E-D666-4CFB-AECD-021BA8D45BAD@oracle.com> On Aug 7, 2012, at 11:36 AM, Alan Bateman wrote: >> : >> No, it needs to be separated out as a project, there is exploded source, and i want to open it in a IDE (plus it will be updated when moving to TestNG, notice that there is a test dimension for testing the various class loaders, a TestNG data provider can be used to abstract this). We also may have other projects in the future. >> >> When this patch goes in i will be updating the runtime tests to include more test code related to service instance iteration failure. >> > My comment was actually about the additional level rather the source tree. When open in an IDE or with TestNG then I assume that the shell test isn't interesting and could remain at the same level as the other shell scripts. I consider the shell script to be part of the project, it's the ad-hoc project file that builds and executes, and could be the one kicks off TestNG (not sure yet). If the project directory is renamed or moved somewhere else we should not have to update or move that script. > If this is all going to be reorganized then it's no big deal but it is critical that all the infrastructure we use support TestNG before we can really move over. > Yes, the plan is to look closely at what lambda is doing and follow in its wake. Paul. From mandy.chung at oracle.com Tue Aug 7 10:47:35 2012 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 07 Aug 2012 10:47:35 -0700 Subject: Jigsaw classpath mode support Message-ID: <502154B7.7080804@oracle.com> I have implemented the "classpath mode" support running on a module image. Webrev: http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/classpath-mode.00/ JDK has been modularized in the jigsaw repo. There no longer has a rt.jar, tools.jar, or any jar file. The system classes are now installed in a system module library as modules. This change will allow the bootstrap class loader, extension class loader, and system class loader to search classes and resources in the system module library as the default. Some implementation details: 1. VM used to maintain the default bootclasspath list and search classes in it. Now it will call jigsaw native interface to find classes from the module library unless -Xbootclasspath is set to override the default bootclasspath. The -Xbootclasspath/p and -Xbootclasspath/a will continue to work as it is today. Karen will be sending out the webrev for the hotspot change separately. 2. tools.jar For compatibility, -classpath tools.jar should continue to work. There are existing applications using URLClassLoader that loads classes from tools.jar (e.g. test harness) that also should continue to work. To be able to determine if the path of the given tools.jar to either -classpath option or URL parameter to URLClassLoader is the same path as JAVA_HOME/lib/tools.jar in a lightweight, the module image for the full JDK will contain an empty tools.jar. This also allows applications that check for the existence of tools.jar to continue to work. sun.misc.URLClassPath is the main implementation class to support URLClassLoader to search a file from the given URLs. Both ExtClassLoader and AppClassLoader are URLClassLoader. URLClassPath is extended to find resource file from the appropriate system modules if the given URL is JAVA_HOME/lib/tools.jar or JAVA_HOME/lib/ext. 3. How to determine which modules belong to bootclasspath, ext directory, and tools.jar (and a few jar files in JDK/lib) at runtime A configuration for the classpath mode support is loaded at startup rather than generated to minimize the startup cost. The idea is to have one ClassPathContext for bootclasspath, one for extension and one for the jdk tools. In this prototype, I just define a new module "jdk.classpath" and install it in the module image. It hardcodes the list of modules for each ClassPathContext for now. One alternative is to extend jpkg to generate "jdk.classpath" config specifically for this classpath mode. We'll explore this later. But I think this is good enough for now. 4. I made simple change in StoredConfiguration to reduce the footprint of the configuration stored on disk. There were several regression tests setting -Xmx failed due to the increase of the startup footprint without this change. With storing the context names and package names only once, it reduces the size of the config with 35-40%. I suspect the fast configuration will make similar change. $ du -k `find . -name 'config'` 696 ./jdk.corba/8-ea/config 1304 ./jdk.classpath/8-ea/config 1232 ./jdk.tools/8-ea/config 604 ./jdk.rmi/8-ea/config 680 ./jdk.tools.base/8-ea/config 604 ./jdk.base/8-ea/config 1056 ./jdk.tools.jaxws/8-ea/config 676 ./jdk.devtools/8-ea/config 608 ./jdk.tools.jre/8-ea/config $ du -k `find . -name 'config'` 444 ./jdk.corba/8-ea/config 792 ./jdk.classpath/8-ea/config 748 ./jdk.tools/8-ea/config 392 ./jdk.rmi/8-ea/config 436 ./jdk.tools.base/8-ea/config 392 ./jdk.base/8-ea/config 636 ./jdk.tools.jaxws/8-ea/config 436 ./jdk.devtools/8-ea/config 396 ./jdk.tools.jre/8-ea/config 5. I ran the jtreg regression tests and JCK api& lang tests. The test results are looking real good. Compared with the jigsaw repo as the baseline, there are 2 new JCK test failures that I have to investigate (passed: 23,685; failed: 138; error: 28) There is a startup initialization issue due to the use of Files.isSameFile method and the default file system provider is replaced with user defined one. I'm currently looking into a fix for it. I'd like to get this webrev out for review early and I also expect any issue found will be minor that I will generate a separate patch on top of this as a separate review. Open Issues: 1. sun.boot.class.path no longer represents the search path for the bootstrap class loader. Any application depending on this system property may be impacted. rmic is one example that uses this property value to find system classes. There has been work to convert rmic to use the javax.tool APIs that will support a module image. Unless rmic becomes module-aware, it continues to use the launcher hack setting -Xbootclasspath/p as a workaround. Mandy From mandy.chung at oracle.com Tue Aug 7 17:55:36 2012 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 07 Aug 2012 17:55:36 -0700 Subject: Services/configuration/context webrevs In-Reply-To: References: Message-ID: <5021B908.6070009@oracle.com> Paul, On 7/1/2012 2:34 AM, Paul Sandoz wrote: > Service state on configuration > ---------------------------------------- > > http://cr.openjdk.java.net/~psandoz/jigsaw/config-services/webrev/ Sorry for the delay on getting to this one. This change keeps a global map of service interface to its service provider class names in the configuration for a more efficient service lookup and service instance creation at runtime which is good. On the other hand, I'm not sure about the new InstalledConfiguration class, an extension to Configuration. While service lookup is only needed at runtime, it seems to me that Configuration would be the subclass of Configuration rather than Configuration. With modulepath, there isn't any installed configuration that some refactoring might be needed. There will be some API refactoring/cleanup anyway (as you have already attempted). I wonder if this patch can simply define the Configuration.findServices method that might well be an abstract method implemented by the InstalledConfiguration class which will only be an implementation class used by StoredConfiguration. Mandy From paul.sandoz at oracle.com Wed Aug 8 05:02:22 2012 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 8 Aug 2012 14:02:22 +0200 Subject: Services/configuration/context webrevs In-Reply-To: <5021B908.6070009@oracle.com> References: <5021B908.6070009@oracle.com> Message-ID: Hi Mandy, On Aug 8, 2012, at 2:55 AM, Mandy Chung wrote: > Paul, > > On 7/1/2012 2:34 AM, Paul Sandoz wrote: > >> Service state on configuration >> ---------------------------------------- >> >> http://cr.openjdk.java.net/~psandoz/jigsaw/config-services/webrev/ > > Sorry for the delay on getting to this one. > > This change keeps a global map of service interface to its service > provider class names in the configuration for a more efficient service > lookup and service instance creation at runtime which is good. > Yes, plus it does not expose the Map, it only exposes the look up using Iterable given a service interface class, thus allowing more flexibility of the implementation. > On the other hand, I'm not sure about the new InstalledConfiguration > class, an extension to Configuration. While service lookup is only > needed at runtime, it seems to me that Configuration would > be the subclass of Configuration rather than Configuration. There is currently no additional functionality specific to Configuration that a subtype is required for that. > With modulepath, there isn't any installed configuration that some > refactoring might be needed. Are you referring to the potential execution from compiled exploded modules? If so, there are still services present, so it will still likely be the same as Configuration and using org.openjdk.jigsaw.Linker, where the storage is abstracted using a special Library implementation to handle the exploded layout of modules and class files [*]. AND, it does not negate caching/storing the configuration on disk if we know that nothing has changed for a subsequent execution. If not, InstalledConfiguration is never referenced for compile-time related functionality. PathContext holds no service-related information and thus PathLinker does not manipulate such information. -- Note that potentially we may in the future control the iteration order of service provider classes i.e. the order which they are stored in the configuration, so the InstalledConfiguration may require further functionality/persistence to adjust the order. > There will be some API refactoring/cleanup > anyway (as you have already attempted). And... :-) When should that occur, and who is gonna do it? > I wonder if this patch can simply > define the Configuration.findServices method that might well be an > abstract method implemented by the InstalledConfiguration class > which will only be an implementation class used by StoredConfiguration. > IMHO that is breaking the abstraction since service provider modules are not linked at compile time. What would be the implementation of Configuration.findServices for a type of Configuration? -- I did not want to mess around with StoredConfiguration knowing that it will change. Currently it needs to use the same type for reading/writing (as per the abstraction StoredConfiguration extends from). FWIW this was the "purest" abstraction i could think of without doing further refactoring [1], so i think we should reconsider abstracting run-time and install-time configuration, if it makes sense, as part of a larger refactoring effort, or as part of other patches if it makes sense within that context. Maybe renaming to RunAndInstallConfiguration would help? Paul. [*] Javac does a similar trick with Configuration and a special implementation of Catalog, com.sun.tools.javac.jigsaw.JavacCatalog. [1] I also wanted to avoid pushing this explicitly into the runtime, say in LoaderPool, as the implementation of the Iterable could change over time depending on how things are stored/ordered etc. From paul.sandoz at oracle.com Thu Aug 9 05:00:01 2012 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 9 Aug 2012 14:00:01 +0200 Subject: Require optional linking error Message-ID: <66A198C0-6C47-4365-B206-614743CC42D1@oracle.com> Hi, If i have the following modules: module x at 1.0 { requires y at 2.0; } module y at 2.0 { } module z at 1.0 { requires optional y at 1.0; requires x; } and javac processes module z first, then the resolver passes and module z at 1.0 will get linked to y at 2.0: [exec] context +y [exec] module y at 2.0 [exec] view y at 2.0 [exec] local [y at 2.0] [exec] remote [+jdk.auth+jdk.base+jdk.desktop+jdk.jndi+jdk.prefs+jdk.tls+sun.charsets+sun.localedata+sun.resources] ... [exec] context +z [exec] module z at 1.0 [exec] view z at 1.0 [exec] local [z at 1.0] [exec] remote [+jdk.auth+jdk.base+jdk.desktop+jdk.jndi+jdk.prefs+jdk.tls+sun.charsets+sun.localedata+sun.resources, +y, +x] ... [exec] context +x [exec] module x at 1.0 [exec] view x at 1.0 [exec] local [x at 1.0] [exec] remote [+jdk.auth+jdk.base+jdk.desktop+jdk.jndi+jdk.prefs+jdk.tls+sun.charsets+sun.localedata+sun.resources, +y] This is similar to the permits issues that i previously highlighted. It's easy to catch this error (and the one for permits) if optional dependency resolution is augmented on to result of the mandatory dependency resolution. Paul. From Alan.Bateman at oracle.com Thu Aug 9 06:01:52 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 09 Aug 2012 14:01:52 +0100 Subject: Require optional linking error In-Reply-To: <66A198C0-6C47-4365-B206-614743CC42D1@oracle.com> References: <66A198C0-6C47-4365-B206-614743CC42D1@oracle.com> Message-ID: <5023B4C0.9040600@oracle.com> On 09/08/2012 13:00, Paul Sandoz wrote: > Hi, > > If i have the following modules: > > module x at 1.0 { > requires y at 2.0; > } > module y at 2.0 { > } > module z at 1.0 { > requires optional y at 1.0; > requires x; > } > > and javac processes module z first, then the resolver passes and module z at 1.0 will get linked to y at 2.0: > > [exec] context +y > [exec] module y at 2.0 > [exec] view y at 2.0 > [exec] local [y at 2.0] > [exec] remote [+jdk.auth+jdk.base+jdk.desktop+jdk.jndi+jdk.prefs+jdk.tls+sun.charsets+sun.localedata+sun.resources] > ... > [exec] context +z > [exec] module z at 1.0 > [exec] view z at 1.0 > [exec] local [z at 1.0] > [exec] remote [+jdk.auth+jdk.base+jdk.desktop+jdk.jndi+jdk.prefs+jdk.tls+sun.charsets+sun.localedata+sun.resources, +y, +x] > ... > [exec] context +x > [exec] module x at 1.0 > [exec] view x at 1.0 > [exec] local [x at 1.0] > [exec] remote [+jdk.auth+jdk.base+jdk.desktop+jdk.jndi+jdk.prefs+jdk.tls+sun.charsets+sun.localedata+sun.resources, +y] > > This is similar to the permits issues that i previously highlighted. > > It's easy to catch this error (and the one for permits) if optional dependency resolution is augmented on to result of the mandatory dependency resolution. > > Paul. There are anomalies with the current implementation that arise due to order. In this case I think it should be an error due to the conflict (which I think is what you are saying too). One could argue that it could succeed with z just not linked to y but that might be problematic if something in z were to meet up with a type from y at runtime. -Alan. From paul.sandoz at oracle.com Thu Aug 9 06:20:03 2012 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 9 Aug 2012 15:20:03 +0200 Subject: Require optional linking error In-Reply-To: <5023B4C0.9040600@oracle.com> References: <66A198C0-6C47-4365-B206-614743CC42D1@oracle.com> <5023B4C0.9040600@oracle.com> Message-ID: <14CE7C7E-D39F-4F66-936C-031527FCEC5F@oracle.com> On Aug 9, 2012, at 3:01 PM, Alan Bateman wrote > There are anomalies with the current implementation that arise due to order. Yep. > In this case I think it should be an error due to the conflict (which I think is what you are saying too). Yes. > One could argue that it could succeed with z just not linked to y but that might be problematic if something in z were to meet up with a type from y at runtime. > Exactly, errors would be pushed to compiler or even worse the runtime making it hard to diagnose the underlying cause. Paul. From Alan.Bateman at oracle.com Thu Aug 9 06:26:27 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 09 Aug 2012 14:26:27 +0100 Subject: Require optional linking error In-Reply-To: <14CE7C7E-D39F-4F66-936C-031527FCEC5F@oracle.com> References: <66A198C0-6C47-4365-B206-614743CC42D1@oracle.com> <5023B4C0.9040600@oracle.com> <14CE7C7E-D39F-4F66-936C-031527FCEC5F@oracle.com> Message-ID: <5023BA83.8070307@oracle.com> On 09/08/2012 14:20, Paul Sandoz wrote: > : > Exactly, errors would be pushed to compiler or even worse the runtime making it hard to diagnose the underlying cause. > Yes, compile time if possible but very likely at install/configure time too when the resolver might see a bigger picture. -Alan. From sean.mullan at oracle.com Thu Aug 9 08:14:02 2012 From: sean.mullan at oracle.com (Sean Mullan) Date: Thu, 09 Aug 2012 11:14:02 -0400 Subject: Review Request: loading JCE providers using ServiceLoader Message-ID: <5023D3BA.8030407@oracle.com> Please review the 2nd webrev for my code changes to enable JCE providers to be registered and loaded as services using the ServiceLoader API: http://cr.openjdk.java.net/~mullan/jigsaw/webrevs/jce-service/webrev.01/ See [1] for previous discussion. The main change is that in module-mode all available Providers are now agressively loaded and cached. The code for loading providers has also been restructured. There is a new Providers.ProviderLoader interface and subclass implementations for module and classpath mode. There are a few outstanding issues that should be looked at later. I will create a new bug so that we don't lose track of them, but briefly: 1. ServiceLoaders don't support a configured preference order for providers 2. Cannot pass an optional argument (ex: a config file) to a provider via ServiceLoader 3. Cannot have more than one instance of a provider (ex: the SunPKCS11 Provider allows multiple instances with different config files) 4. ServiceLoaderConfigurationError does not retain which provider generated the error I've worked around some of these in the code, but none is a perfect solution. However, in the interests of moving forward, I would like to leave these as open issues for now. Thanks, Sean [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2012-July/003053.html From paul.sandoz at oracle.com Fri Aug 10 03:10:20 2012 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 10 Aug 2012 12:10:20 +0200 Subject: Jigsaw classpath mode support In-Reply-To: <502154B7.7080804@oracle.com> References: <502154B7.7080804@oracle.com> Message-ID: Hi Mandy, This looks good to me, although i admit to not fully understanding the interactions between the Java layer and the VM layer. IMHO, given that the test results are so good, this patch should be pushed early and iterated on to refine/improve. That way we all get exposure to it. Paul. -- Looks like the Library interface may need to be updated so that: public abstract URI findLocalResource(ModuleId mid, String rn) returns a URI plus some meta-data e.g. size. -- Definite room for optimisation in ClassPathContext, for example if we know the resource is a class or using some reverse lookup of resource path name. -- FWIW in sun.misc.Launcher you can replace lines 215-224 with the two following lines: prepaths = urls.subList(0, index).toArray(new URL[index]); postpaths = urls.subList(index, urls.size()).toArray(new URL[urls.size() - index]); Plus Vector could probably be replaced with List. 189 // keep prepaths and postpaths for native library search 190 private static URL[] prepaths; 191 private static URL[] postpaths; 192 static private URL[] getExtURLs(File[] dirs) throws IOException { 193 File defaultExtDir = EXTENSION.defaultPath(); 194 Vector urls = new Vector(); 195 int index = 0; 196 for (int i = 0; i < dirs.length; i++) { 197 File dir = dirs[i]; 198 String[] files = dir.list(); 199 if (files != null) { 200 for (int j = 0; j < files.length; j++) { 201 if (!files[j].equals("meta-index")) { 202 File f = new File(dir, files[j]); 203 urls.add(getFileURL(f)); 204 } 205 } 206 } 207 if (dir.exists() && defaultExtDir != null && 208 Files.isSameFile(defaultExtDir.toPath(), dir.toPath())) { 209 // set index to urls that dirs[i] is the default ext dir 210 // so that prepaths and postpaths can be determined. 211 index = urls.size(); 212 urls.add(getFileURL(dir)); 213 } 214 } 215 URL[] ua = new URL[urls.size()]; 216 urls.copyInto(ua); 217 prepaths = new URL[index]; 218 postpaths = new URL[urls.size() - index]; 219 for (int i=0, j=0; i < ua.length; i++) { 220 if (i < index) 221 prepaths[i] = ua[i]; 222 else 223 postpaths[j++] = ua[i]; 224 } 225 return ua; 226 } On Aug 7, 2012, at 7:47 PM, Mandy Chung wrote: > I have implemented the "classpath mode" support running on > a module image. > > Webrev: > http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/classpath-mode.00/ > > JDK has been modularized in the jigsaw repo. There no longer > has a rt.jar, tools.jar, or any jar file. The system classes > are now installed in a system module library as modules. This > change will allow the bootstrap class loader, extension class > loader, and system class loader to search classes and resources > in the system module library as the default. Some implementation > details: > > 1. VM used to maintain the default bootclasspath list and search > classes in it. Now it will call jigsaw native interface to find > classes from the module library unless -Xbootclasspath is set > to override the default bootclasspath. The -Xbootclasspath/p > and -Xbootclasspath/a will continue to work as it is today. > Karen will be sending out the webrev for the hotspot change > separately. > > 2. tools.jar > > For compatibility, -classpath tools.jar should continue > to work. There are existing applications using URLClassLoader > that loads classes from tools.jar (e.g. test harness) that > also should continue to work. > > To be able to determine if the path of the given tools.jar > to either -classpath option or URL parameter to URLClassLoader > is the same path as JAVA_HOME/lib/tools.jar in a lightweight, > the module image for the full JDK will contain an empty tools.jar. > This also allows applications that check for the existence of > tools.jar to continue to work. > > sun.misc.URLClassPath is the main implementation class to > support URLClassLoader to search a file from the given URLs. > Both ExtClassLoader and AppClassLoader are URLClassLoader. > URLClassPath is extended to find resource file from the > appropriate system modules if the given URL is > JAVA_HOME/lib/tools.jar or JAVA_HOME/lib/ext. > > 3. How to determine which modules belong to bootclasspath, > ext directory, and tools.jar (and a few jar files in JDK/lib) > at runtime > > A configuration for the classpath mode support is loaded at > startup rather than generated to minimize the startup cost. > The idea is to have one ClassPathContext for bootclasspath, > one for extension and one for the jdk tools. > > In this prototype, I just define a new module "jdk.classpath" > and install it in the module image. It hardcodes the list > of modules for each ClassPathContext for now. One alternative > is to extend jpkg to generate "jdk.classpath" config specifically > for this classpath mode. We'll explore this later. But I > think this is good enough for now. > > 4. I made simple change in StoredConfiguration to reduce > the footprint of the configuration stored on disk. There > were several regression tests setting -Xmx failed due to > the increase of the startup footprint without this change. > > With storing the context names and package names only once, > it reduces the size of the config with 35-40%. I suspect > the fast configuration will make similar change. > > $ du -k `find . -name 'config'` > 696 ./jdk.corba/8-ea/config > 1304 ./jdk.classpath/8-ea/config > 1232 ./jdk.tools/8-ea/config > 604 ./jdk.rmi/8-ea/config > 680 ./jdk.tools.base/8-ea/config > 604 ./jdk.base/8-ea/config > 1056 ./jdk.tools.jaxws/8-ea/config > 676 ./jdk.devtools/8-ea/config > 608 ./jdk.tools.jre/8-ea/config > > $ du -k `find . -name 'config'` > 444 ./jdk.corba/8-ea/config > 792 ./jdk.classpath/8-ea/config > 748 ./jdk.tools/8-ea/config > 392 ./jdk.rmi/8-ea/config > 436 ./jdk.tools.base/8-ea/config > 392 ./jdk.base/8-ea/config > 636 ./jdk.tools.jaxws/8-ea/config > 436 ./jdk.devtools/8-ea/config > 396 ./jdk.tools.jre/8-ea/config > > > 5. I ran the jtreg regression tests and JCK api& lang tests. > The test results are looking real good. Compared with the > jigsaw repo as the baseline, there are 2 new JCK test failures > that I have to investigate (passed: 23,685; failed: 138; error: 28) > > There is a startup initialization issue due to the use of > Files.isSameFile method and the default file system provider > is replaced with user defined one. I'm currently looking into > a fix for it. > > I'd like to get this webrev out for review early and I also > expect any issue found will be minor that I will generate a > separate patch on top of this as a separate review. > > Open Issues: > 1. sun.boot.class.path no longer represents the search path > for the bootstrap class loader. Any application depending > on this system property may be impacted. > > rmic is one example that uses this property value to find > system classes. There has been work to convert rmic to > use the javax.tool APIs that will support a module image. > Unless rmic becomes module-aware, it continues to use > the launcher hack setting -Xbootclasspath/p as a workaround. > > Mandy > From mandy.chung at oracle.com Fri Aug 10 09:29:44 2012 From: mandy.chung at oracle.com (Mandy Chung) Date: Fri, 10 Aug 2012 09:29:44 -0700 Subject: Jigsaw classpath mode support In-Reply-To: References: <502154B7.7080804@oracle.com> Message-ID: <502536F8.4000506@oracle.com> Hi Paul, Thanks for the review. On 8/10/2012 3:10 AM, Paul Sandoz wrote: > Hi Mandy, > > This looks good to me, although i admit to not fully understanding the interactions between the Java layer and the VM layer. I'm not sure if there is any document describing the existing VM and libs interactions. Karen and I will write up a document describing such interaction and startup sequence and post it to the project page. > IMHO, given that the test results are so good, this patch should be pushed early and iterated on to refine/improve. That way we all get exposure to it. Yes and especially to find out any compatibility issue to be addressed. > > Paul. > > -- > > Looks like the Library interface may need to be updated so that: > > public abstract URI findLocalResource(ModuleId mid, String rn) > > returns a URI plus some meta-data e.g. size. That's one option I consider. I prefer to leave it as it is for now and do this with other optimization. > -- > > Definite room for optimisation in ClassPathContext, for example if we know the resource is a class or using some reverse lookup of resource path name. Right - that's one thing and I also consider returning the size of the resource data is another. I should add a comment about the possible optimization. > > -- > > FWIW in sun.misc.Launcher you can replace lines 215-224 with the two following lines: > > prepaths = urls.subList(0, index).toArray(new URL[index]); > postpaths = urls.subList(index, urls.size()).toArray(new URL[urls.size() - index]); Fixed. Thanks for catching this. > Plus Vector could probably be replaced with List. FWIW. Vector is also used in ClassLoader and other jdk classes. It has been on the list of technical debts to clean up but it has to be done carefully since they are used during startup. As in sun.misc.Launcher, I change it to use ArrayList since I don't see any problem with it. Thanks Mandy > 189 // keep prepaths and postpaths for native library search > 190 private static URL[] prepaths; > 191 private static URL[] postpaths; > 192 static private URL[] getExtURLs(File[] dirs) throws IOException { > 193 File defaultExtDir = EXTENSION.defaultPath(); > 194 Vector urls = new Vector(); > 195 int index = 0; > 196 for (int i = 0; i< dirs.length; i++) { > 197 File dir = dirs[i]; > 198 String[] files = dir.list(); > 199 if (files != null) { > 200 for (int j = 0; j< files.length; j++) { > 201 if (!files[j].equals("meta-index")) { > 202 File f = new File(dir, files[j]); > 203 urls.add(getFileURL(f)); > 204 } > 205 } > 206 } > 207 if (dir.exists()&& defaultExtDir != null&& > 208 Files.isSameFile(defaultExtDir.toPath(), dir.toPath())) { > 209 // set index to urls that dirs[i] is the default ext dir > 210 // so that prepaths and postpaths can be determined. > 211 index = urls.size(); > 212 urls.add(getFileURL(dir)); > 213 } > 214 } > 215 URL[] ua = new URL[urls.size()]; > 216 urls.copyInto(ua); > 217 prepaths = new URL[index]; > 218 postpaths = new URL[urls.size() - index]; > 219 for (int i=0, j=0; i< ua.length; i++) { > 220 if (i< index) > 221 prepaths[i] = ua[i]; > 222 else > 223 postpaths[j++] = ua[i]; > 224 } > 225 return ua; > 226 } > > On Aug 7, 2012, at 7:47 PM, Mandy Chung wrote: > >> I have implemented the "classpath mode" support running on >> a module image. >> >> Webrev: >> http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/classpath-mode.00/ >> >> JDK has been modularized in the jigsaw repo. There no longer >> has a rt.jar, tools.jar, or any jar file. The system classes >> are now installed in a system module library as modules. This >> change will allow the bootstrap class loader, extension class >> loader, and system class loader to search classes and resources >> in the system module library as the default. Some implementation >> details: >> >> 1. VM used to maintain the default bootclasspath list and search >> classes in it. Now it will call jigsaw native interface to find >> classes from the module library unless -Xbootclasspath is set >> to override the default bootclasspath. The -Xbootclasspath/p >> and -Xbootclasspath/a will continue to work as it is today. >> Karen will be sending out the webrev for the hotspot change >> separately. >> >> 2. tools.jar >> >> For compatibility, -classpath tools.jar should continue >> to work. There are existing applications using URLClassLoader >> that loads classes from tools.jar (e.g. test harness) that >> also should continue to work. >> >> To be able to determine if the path of the given tools.jar >> to either -classpath option or URL parameter to URLClassLoader >> is the same path as JAVA_HOME/lib/tools.jar in a lightweight, >> the module image for the full JDK will contain an empty tools.jar. >> This also allows applications that check for the existence of >> tools.jar to continue to work. >> >> sun.misc.URLClassPath is the main implementation class to >> support URLClassLoader to search a file from the given URLs. >> Both ExtClassLoader and AppClassLoader are URLClassLoader. >> URLClassPath is extended to find resource file from the >> appropriate system modules if the given URL is >> JAVA_HOME/lib/tools.jar or JAVA_HOME/lib/ext. >> >> 3. How to determine which modules belong to bootclasspath, >> ext directory, and tools.jar (and a few jar files in JDK/lib) >> at runtime >> >> A configuration for the classpath mode support is loaded at >> startup rather than generated to minimize the startup cost. >> The idea is to have one ClassPathContext for bootclasspath, >> one for extension and one for the jdk tools. >> >> In this prototype, I just define a new module "jdk.classpath" >> and install it in the module image. It hardcodes the list >> of modules for each ClassPathContext for now. One alternative >> is to extend jpkg to generate "jdk.classpath" config specifically >> for this classpath mode. We'll explore this later. But I >> think this is good enough for now. >> >> 4. I made simple change in StoredConfiguration to reduce >> the footprint of the configuration stored on disk. There >> were several regression tests setting -Xmx failed due to >> the increase of the startup footprint without this change. >> >> With storing the context names and package names only once, >> it reduces the size of the config with 35-40%. I suspect >> the fast configuration will make similar change. >> >> $ du -k `find . -name 'config'` >> 696 ./jdk.corba/8-ea/config >> 1304 ./jdk.classpath/8-ea/config >> 1232 ./jdk.tools/8-ea/config >> 604 ./jdk.rmi/8-ea/config >> 680 ./jdk.tools.base/8-ea/config >> 604 ./jdk.base/8-ea/config >> 1056 ./jdk.tools.jaxws/8-ea/config >> 676 ./jdk.devtools/8-ea/config >> 608 ./jdk.tools.jre/8-ea/config >> >> $ du -k `find . -name 'config'` >> 444 ./jdk.corba/8-ea/config >> 792 ./jdk.classpath/8-ea/config >> 748 ./jdk.tools/8-ea/config >> 392 ./jdk.rmi/8-ea/config >> 436 ./jdk.tools.base/8-ea/config >> 392 ./jdk.base/8-ea/config >> 636 ./jdk.tools.jaxws/8-ea/config >> 436 ./jdk.devtools/8-ea/config >> 396 ./jdk.tools.jre/8-ea/config >> >> >> 5. I ran the jtreg regression tests and JCK api& lang tests. >> The test results are looking real good. Compared with the >> jigsaw repo as the baseline, there are 2 new JCK test failures >> that I have to investigate (passed: 23,685; failed: 138; error: 28) >> >> There is a startup initialization issue due to the use of >> Files.isSameFile method and the default file system provider >> is replaced with user defined one. I'm currently looking into >> a fix for it. >> >> I'd like to get this webrev out for review early and I also >> expect any issue found will be minor that I will generate a >> separate patch on top of this as a separate review. >> >> Open Issues: >> 1. sun.boot.class.path no longer represents the search path >> for the bootstrap class loader. Any application depending >> on this system property may be impacted. >> >> rmic is one example that uses this property value to find >> system classes. There has been work to convert rmic to >> use the javax.tool APIs that will support a module image. >> Unless rmic becomes module-aware, it continues to use >> the launcher hack setting -Xbootclasspath/p as a workaround. >> >> Mandy >> From mandy.chung at oracle.com Fri Aug 10 16:46:27 2012 From: mandy.chung at oracle.com (Mandy Chung) Date: Fri, 10 Aug 2012 16:46:27 -0700 Subject: Review Request: loading JCE providers using ServiceLoader In-Reply-To: <5023D3BA.8030407@oracle.com> References: <5023D3BA.8030407@oracle.com> Message-ID: <50259D53.4010102@oracle.com> Hi Sean, On 8/9/2012 8:14 AM, Sean Mullan wrote: > Please review the 2nd webrev for my code changes to enable JCE providers to be > registered and loaded as services using the ServiceLoader API: > > http://cr.openjdk.java.net/~mullan/jigsaw/webrevs/jce-service/webrev.01/ This looks good and cleaner with the refactoring you have done. I agree we should move forward with this and address the open issues next. One issue to add to the list is about the "java.security" config file that lists the security provider classname. As Alan points out, they are (likely) not exported in modules. Also all the Providers available as services should be automatically added to the list of registered providers (I think your patch only registers the ones listed in java.security). These are related to the configured preference order. make/modules/modules.group jdk.depconfig is intended for auto-generation of these service dependency and implementation providing services but META-INF/services/java.security.Provider doesn't exist. That's why you need to declare these "provides service" explicitly. Since we're going to check in module-info.java in the source tree, I'm okay with this workaround. It might worth adding a comment for that. Most of the changes except Providers.ModuleProviderLoader are refactoring. It's good to get that non-module-specific change into jdk8 to reduce the number of changes in jigsaw/jigsaw to carry that would help avoid merge issues. It's fine to push that to jdk8 after this gets into jigsaw/jigsaw - whichever is convenient. Thanks Mandy From Alan.Bateman at oracle.com Sun Aug 12 13:10:48 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 12 Aug 2012 21:10:48 +0100 Subject: Review Request: loading JCE providers using ServiceLoader In-Reply-To: <5023D3BA.8030407@oracle.com> References: <5023D3BA.8030407@oracle.com> Message-ID: <50280DC8.9060801@oracle.com> On 09/08/2012 16:14, Sean Mullan wrote: > Please review the 2nd webrev for my code changes to enable JCE providers to be > registered and loaded as services using the ServiceLoader API: > > http://cr.openjdk.java.net/~mullan/jigsaw/webrevs/jce-service/webrev.01/ > > See [1] for previous discussion. > > The main change is that in module-mode all available Providers are now > agressively loaded and cached. The code for loading providers has also been > restructured. There is a new Providers.ProviderLoader interface and subclass > implementations for module and classpath mode. > > There are a few outstanding issues that should be looked at later. I will create > a new bug so that we don't lose track of them, but briefly: > > 1. ServiceLoaders don't support a configured preference order for providers > 2. Cannot pass an optional argument (ex: a config file) to a provider via > ServiceLoader > 3. Cannot have more than one instance of a provider (ex: the SunPKCS11 Provider > allows multiple instances with different config files) > 4. ServiceLoaderConfigurationError does not retain which provider generated the > error > > I've worked around some of these in the code, but none is a perfect solution. > However, in the interests of moving forward, I would like to leave these as open > issues for now. > > Thanks, > Sean > > [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2012-July/003053.html I think this looks much cleaner. One thing that isn't completely clear to me is what happens to security providers that are installed but not registered in the java.security file. If I read the code correctly then they will be loaded but just ignored because the lookup is keyed on class name. I looked at the exception handling (which is a hack, as you noted in the comments). I'm not so sure that having SCE define a method to get the class name of the service is the right thing to do. I think the real issue is that the listing the provider class names in the java.security file is getting in the way, forcing you to eagerly load all the providers so that you can match up the class name and saving any exceptions thrown during the loading so that you can throw them later. So I think what you have is fine for now but it will need to be re-examined, maybe after there is a way to configure the order of the service provider modules. If that was a way to do this and we could somehow migrate away listing the providers by class name in the java.security file then it would be a lot cleaner. Just in passing, is UOE documented anywhere? From the javadoc it looks like ProviderException is the only exception that may be thrown. -Alan. From sean.mullan at oracle.com Mon Aug 13 12:17:14 2012 From: sean.mullan at oracle.com (Sean Mullan) Date: Mon, 13 Aug 2012 15:17:14 -0400 Subject: Review Request: loading JCE providers using ServiceLoader In-Reply-To: <50259D53.4010102@oracle.com> References: <5023D3BA.8030407@oracle.com> <50259D53.4010102@oracle.com> Message-ID: <502952BA.9080702@oracle.com> On 08/10/2012 07:46 PM, Mandy Chung wrote: > Hi Sean, > > On 8/9/2012 8:14 AM, Sean Mullan wrote: > >> Please review the 2nd webrev for my code changes to enable JCE >> providers to be >> registered and loaded as services using the ServiceLoader API: >> >> http://cr.openjdk.java.net/~mullan/jigsaw/webrevs/jce-service/webrev.01/ > This looks good and cleaner with the refactoring you have done. > I agree we should move forward with this and address the open issues > next. One issue to add to the list is about the "java.security" > config file that lists the security provider classname. As Alan points > out, they are (likely) not exported in modules. Right, but just listing the classnames in the java.security file doesn't cause any problems. It is only a problem if an application instantiates them directly, ex: MessageDigest md = MessageDigest.getInstance("SHA-256", new sun.security.provider.SUN()); There are probably some applications doing that, although it is discouraged in our docs, and we advise using the provider's String name instead if you want to use a specific provider: MessageDigest md = MessageDigest.getInstance("SHA-256", "SUN"); > Also all the Providers > available as services should be automatically added to the list of > registered providers (I think your patch only registers the ones > listed in java.security). These are related to the configured preference > order. I thought about doing that, just adding them on to the end. But there are a few other subtle issues. Right now, if you want to use a JCE provider that isn't registered, you instantiate it directly (as I showed above). Or you could also instantiate it and use the Security.addProvider or insertProviderAt methods to add it to the list of registered providers. There are apps doing that today. That code may break (addProvider will return -1) if we automatically registered all JCE providers that were found by ServiceLoader. So I would rather hold off on this and address it later with the ordering of providers. We may need to adjust what it actually means for a JCE provider to be installed: http://docs.oracle.com/javase/7/docs/api/java/security/Security.html#getProviders%28%29 Right now "installed" means as configured in the java.security file. > make/modules/modules.group > jdk.depconfig is intended for auto-generation of these > service dependency and implementation providing services > but META-INF/services/java.security.Provider doesn't exist. > That's why you need to declare these "provides service" > explicitly. Since we're going to check in module-info.java > in the source tree, I'm okay with this workaround. > It might worth adding a comment for that. > > Most of the changes except Providers.ModuleProviderLoader are > refactoring. It's good to get that non-module-specific change > into jdk8 to reduce the number of changes in jigsaw/jigsaw to > carry that would help avoid merge issues. It's fine to push > that to jdk8 after this gets into jigsaw/jigsaw - whichever > is convenient. Ok, I will do that after I push it into jigsaw. Thanks, Sean From sean.mullan at oracle.com Mon Aug 13 13:14:02 2012 From: sean.mullan at oracle.com (Sean Mullan) Date: Mon, 13 Aug 2012 16:14:02 -0400 Subject: Review Request: loading JCE providers using ServiceLoader In-Reply-To: <50280DC8.9060801@oracle.com> References: <5023D3BA.8030407@oracle.com> <50280DC8.9060801@oracle.com> Message-ID: <5029600A.5040404@oracle.com> On 08/12/2012 04:10 PM, Alan Bateman wrote: > On 09/08/2012 16:14, Sean Mullan wrote: >> Please review the 2nd webrev for my code changes to enable JCE >> providers to be >> registered and loaded as services using the ServiceLoader API: >> >> http://cr.openjdk.java.net/~mullan/jigsaw/webrevs/jce-service/webrev.01/ > I think this looks much cleaner. One thing that isn't completely clear > to me is what happens to security providers that are installed but not > registered in the java.security file. If I read the code correctly then > they will be loaded but just ignored because the lookup is keyed on > class name. Yes, see my reply to Mandy for more info. > I looked at the exception handling (which is a hack, as you noted in the > comments). I'm not so sure that having SCE define a method to get the > class name of the service is the right thing to do. I think the real > issue is that the listing the provider class names in the java.security > file is getting in the way, forcing you to eagerly load all the > providers so that you can match up the class name and saving any > exceptions thrown during the loading so that you can throw them later. > So I think what you have is fine for now but it will need to be > re-examined, maybe after there is a way to configure the order of the > service provider modules. If that was a way to do this and we could > somehow migrate away listing the providers by class name in the > java.security file then it would be a lot cleaner. I agree. I think that would also address the SCE hack. > Just in passing, is UOE documented anywhere? From the javadoc it looks > like ProviderException is the only exception that may be thrown. UOE is not documented AFAIK. If a provider's ctor throws an UOE, the exception is swallowed, the provider is disabled, and we will not attempt to load the provider again, whereas with a ProviderException, that exception usually will pass through to the caller (as it may be a configuration issue they should be made aware of and can be fixed). UOE is thrown by the SunPKCS11 provider in a few cases. It seems like something that we should potentially add to the APIs. --Sean From Alan.Bateman at oracle.com Tue Aug 14 02:30:55 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 14 Aug 2012 10:30:55 +0100 Subject: Review Request: loading JCE providers using ServiceLoader In-Reply-To: <502952BA.9080702@oracle.com> References: <5023D3BA.8030407@oracle.com> <50259D53.4010102@oracle.com> <502952BA.9080702@oracle.com> Message-ID: <502A1ACF.8040902@oracle.com> On 13/08/2012 20:17, Sean Mullan wrote: > > I thought about doing that, just adding them on to the end. But there > are a few other subtle issues. Right now, if you want to use a JCE > provider that isn't registered, you instantiate it directly (as I > showed above). Or you could also instantiate it and use the > Security.addProvider or insertProviderAt methods to add it to the list > of registered providers. There are apps doing that today. That code > may break (addProvider will return -1) if we automatically registered > all JCE providers that were found by ServiceLoader. So I would rather > hold off on this and address it later with the ordering of providers. > We may need to adjust what it actually means for a JCE provider to be > installed: I think it would make sense to add them at the end. One could imagine calling Signature.getInstance(algorithm) and the implementation would iterate over the configured providers and if none of them support the algorithm then it will move onto the other installed providers looking for one that supports this algorithm. On the dynamic registration via Security.addProvider then I would think it should be an error for someone to install the provider as a service provider module and also attempts to register it themselves (and to register it themselves would require they have some means to instantiate it, suggesting it exports an API for that). It may be that bundled providers intended to be dynamically registered just don't declare themselves as services. So I think it would be good to re-visit this, not critical for the initial push of course. -Alan. From sean.mullan at oracle.com Tue Aug 14 07:30:15 2012 From: sean.mullan at oracle.com (Sean Mullan) Date: Tue, 14 Aug 2012 10:30:15 -0400 Subject: Review Request: loading JCE providers using ServiceLoader In-Reply-To: <502A1ACF.8040902@oracle.com> References: <5023D3BA.8030407@oracle.com> <50259D53.4010102@oracle.com> <502952BA.9080702@oracle.com> <502A1ACF.8040902@oracle.com> Message-ID: <502A60F7.3050909@oracle.com> On 8/14/12 5:30 AM, Alan Bateman wrote: > On 13/08/2012 20:17, Sean Mullan wrote: >> >> I thought about doing that, just adding them on to the end. But there >> are a few other subtle issues. Right now, if you want to use a JCE >> provider that isn't registered, you instantiate it directly (as I >> showed above). Or you could also instantiate it and use the >> Security.addProvider or insertProviderAt methods to add it to the list >> of registered providers. There are apps doing that today. That code >> may break (addProvider will return -1) if we automatically registered >> all JCE providers that were found by ServiceLoader. So I would rather >> hold off on this and address it later with the ordering of providers. >> We may need to adjust what it actually means for a JCE provider to be >> installed: > I think it would make sense to add them at the end. One could imagine > calling Signature.getInstance(algorithm) and the implementation would > iterate over the configured providers and if none of them support the > algorithm then it will move onto the other installed providers looking > for one that supports this algorithm. > > On the dynamic registration via Security.addProvider then I would think > it should be an error for someone to install the provider as a service > provider module and also attempts to register it themselves (and to > register it themselves would require they have some means to instantiate > it, suggesting it exports an API for that). It may be that bundled > providers intended to be dynamically registered just don't declare > themselves as services. > > So I think it would be good to re-visit this, not critical for the > initial push of course. Ok, that's good. It's a bit more work to make providers not configured in java.security available by default -- it will require more surgery to the sun.security.jca classes, so I'd rather do this later. I've created a new CR to track the issues we should look at later: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7191104 --Sean From sean.mullan at oracle.com Tue Aug 14 14:27:08 2012 From: sean.mullan at oracle.com (sean.mullan at oracle.com) Date: Tue, 14 Aug 2012 21:27:08 +0000 Subject: hg: jigsaw/jigsaw/jdk: 2 new changesets Message-ID: <20120814212744.B25C54751C@hg.openjdk.java.net> Changeset: bf6c3ddca9f9 Author: mullan Date: 2012-08-14 17:09 -0400 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/bf6c3ddca9f9 Load JCE Providers in module mode using ServiceLoader Reviewed-by: alanb, mchung, psandoz ! make/modules/jdk.depconfig ! make/modules/modules.group ! src/share/classes/com/sun/crypto/provider/AESKeyGenerator.java ! src/share/classes/com/sun/crypto/provider/BlowfishKeyGenerator.java ! src/share/classes/com/sun/crypto/provider/CipherCore.java ! src/share/classes/com/sun/crypto/provider/DESKeyGenerator.java ! src/share/classes/com/sun/crypto/provider/DESedeKeyGenerator.java ! src/share/classes/com/sun/crypto/provider/DESedeWrapCipher.java ! src/share/classes/com/sun/crypto/provider/DHKeyPairGenerator.java ! src/share/classes/com/sun/crypto/provider/DHParameterGenerator.java ! src/share/classes/com/sun/crypto/provider/HmacMD5KeyGenerator.java ! src/share/classes/com/sun/crypto/provider/HmacPKCS12PBESHA1.java ! src/share/classes/com/sun/crypto/provider/HmacSHA1KeyGenerator.java ! src/share/classes/com/sun/crypto/provider/ISO10126Padding.java ! src/share/classes/com/sun/crypto/provider/KeyGeneratorCore.java ! src/share/classes/com/sun/crypto/provider/KeyProtector.java ! src/share/classes/com/sun/crypto/provider/PBECipherCore.java ! src/share/classes/com/sun/crypto/provider/PKCS12PBECipherCore.java ! src/share/classes/com/sun/crypto/provider/SunJCE.java ! src/share/classes/org/openjdk/jigsaw/cli/Packager.java ! src/share/classes/sun/security/jca/ProviderConfig.java ! src/share/classes/sun/security/jca/ProviderList.java ! src/share/classes/sun/security/jca/Providers.java ! src/share/classes/sun/security/pkcs11/SunPKCS11.java + test/org/openjdk/jigsaw/services/jce.sh Changeset: ba3e77271be3 Author: mullan Date: 2012-08-14 17:17 -0400 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/ba3e77271be3 Merge ! make/modules/jdk.depconfig ! make/modules/modules.group - makefiles/LegacyMakefiles.gmk - makefiles/altclasses/Makefile - makefiles/apple/Makefile - makefiles/apple/applescript/Makefile - makefiles/com/Makefile - makefiles/com/apple/Makefile - makefiles/com/apple/osx/Makefile - makefiles/com/apple/osxui/Makefile - makefiles/com/oracle/Makefile - makefiles/com/oracle/jfr/Makefile - makefiles/com/oracle/security/ucrypto/FILES_c.gmk - makefiles/com/oracle/security/ucrypto/Makefile - makefiles/com/oracle/security/ucrypto/mapfile-vers - makefiles/com/sun/Makefile - makefiles/com/sun/crypto/provider/Makefile - makefiles/common/Classes.gmk - makefiles/common/Cscope.gmk - makefiles/common/Defs-embedded.gmk - makefiles/common/Defs-linux.gmk - makefiles/common/Defs-macosx.gmk - makefiles/common/Defs-solaris.gmk - makefiles/common/Defs-windows.gmk - makefiles/common/Defs.gmk - makefiles/common/Demo.gmk - makefiles/common/Library.gmk - makefiles/common/Mapfile-vers.gmk - makefiles/common/Modules.gmk - makefiles/common/Program.gmk - makefiles/common/Release-embedded.gmk - makefiles/common/Release-macosx.gmk - makefiles/common/Release.gmk - makefiles/common/Rules.gmk - makefiles/common/Subdirs.gmk - makefiles/common/internal/Defs-corba.gmk - makefiles/common/internal/Defs-jaxp.gmk - makefiles/common/internal/Defs-jaxws.gmk - makefiles/common/internal/Defs-langtools.gmk - makefiles/common/internal/ImportComponents.gmk - makefiles/common/internal/NativeCompileRules.gmk - makefiles/common/internal/Resources.gmk - makefiles/common/shared/Compiler-gcc.gmk - makefiles/common/shared/Compiler-llvm.gmk - makefiles/common/shared/Compiler-msvc.gmk - makefiles/common/shared/Compiler-sun.gmk - makefiles/common/shared/Defs-control.gmk - makefiles/common/shared/Defs-java.gmk - makefiles/common/shared/Defs-javadoc.gmk - makefiles/common/shared/Defs-linux.gmk - makefiles/common/shared/Defs-macosx.gmk - makefiles/common/shared/Defs-solaris.gmk - makefiles/common/shared/Defs-utils.gmk - makefiles/common/shared/Defs-versions.gmk - makefiles/common/shared/Defs-windows.gmk - makefiles/common/shared/Defs.gmk - makefiles/common/shared/Platform.gmk - makefiles/common/shared/PrivateDefs.gmk-example - makefiles/common/shared/Sanity-Settings.gmk - makefiles/java/Makefile - makefiles/java/fdlibm/FILES_c.gmk - makefiles/java/fdlibm/Makefile - makefiles/java/instrument/Makefile - makefiles/java/instrument/mapfile-vers - makefiles/java/invoke/Makefile - makefiles/java/java/Exportedfiles.gmk - makefiles/java/java/FILES_c.gmk - makefiles/java/java/FILES_java.gmk - makefiles/java/java/Makefile - makefiles/java/java/localelist.sh - makefiles/java/java/mapfile-vers - makefiles/java/java/reflect/Makefile - makefiles/java/java/reorder-i586 - makefiles/java/java/reorder-sparc - makefiles/java/java/reorder-sparcv9 - makefiles/java/java_crw_demo/Makefile - makefiles/java/java_crw_demo/mapfile-vers - makefiles/java/java_hprof_demo/Makefile - makefiles/java/java_hprof_demo/mapfile-vers - makefiles/java/jexec/Makefile - makefiles/java/jli/Makefile - makefiles/java/jli/mapfile-vers - makefiles/java/jobjc/Makefile - makefiles/java/jvm/Makefile - makefiles/java/main/Makefile - makefiles/java/main/java/Makefile - makefiles/java/main/java/mapfile-amd64 - makefiles/java/main/java/mapfile-i586 - makefiles/java/main/java/mapfile-sparc - makefiles/java/main/java/mapfile-sparcv9 - makefiles/java/main/javaw/Makefile - makefiles/java/management/Exportedfiles.gmk - makefiles/java/management/FILES_c.gmk - makefiles/java/management/Makefile - makefiles/java/management/mapfile-vers - makefiles/java/net/FILES_c.gmk - makefiles/java/net/Makefile - makefiles/java/net/mapfile-vers - makefiles/java/nio/Exportedfiles.gmk - makefiles/java/nio/FILES_c.gmk - makefiles/java/nio/FILES_java.gmk - makefiles/java/nio/Makefile - makefiles/java/nio/addNotices.sh - makefiles/java/nio/genBuffer.sh - makefiles/java/nio/genCharsetProvider.sh - makefiles/java/nio/genCoder.sh - makefiles/java/nio/genExceptions.sh - makefiles/java/nio/mapfile-bsd - makefiles/java/nio/mapfile-linux - makefiles/java/nio/mapfile-solaris - makefiles/java/nio/reorder-i586 - makefiles/java/nio/reorder-sparc - makefiles/java/nio/reorder-sparcv9 - makefiles/java/npt/Makefile - makefiles/java/npt/mapfile-vers - makefiles/java/redist/Makefile - makefiles/java/redist/fonts/Makefile - makefiles/java/redist/sajdi/Makefile - makefiles/java/security/Makefile - makefiles/java/sun_nio/FILES_java.gmk - makefiles/java/sun_nio/Makefile - makefiles/java/util/FILES_java.gmk - makefiles/java/util/FILES_properties.gmk - makefiles/java/util/Makefile - makefiles/java/verify/Makefile - makefiles/java/verify/mapfile-vers - makefiles/java/verify/reorder-i586 - makefiles/java/verify/reorder-sparc - makefiles/java/verify/reorder-sparcv9 - makefiles/javax/Makefile - makefiles/javax/crypto/Defs-jce.gmk - makefiles/javax/crypto/Makefile - makefiles/javax/crypto/policy/limited/LIMITED - makefiles/javax/crypto/policy/limited/default_local.policy - makefiles/javax/crypto/policy/limited/exempt_local.policy - makefiles/javax/crypto/policy/unlimited/UNLIMITED - makefiles/javax/crypto/policy/unlimited/default_US_export.policy - makefiles/javax/crypto/policy/unlimited/default_local.policy - makefiles/javax/imageio/Makefile - makefiles/javax/management/Makefile - makefiles/javax/sound/FILES_c.gmk - makefiles/javax/sound/Makefile - makefiles/javax/sound/SoundDefs.gmk - makefiles/javax/sound/jsoundalsa/Makefile - makefiles/javax/sound/jsoundalsa/mapfile-vers - makefiles/javax/sound/jsoundds/Makefile - makefiles/javax/sound/mapfile-vers - makefiles/javax/sql/Makefile - makefiles/javax/swing/FILES.gmk - makefiles/javax/swing/Makefile - makefiles/javax/swing/beaninfo/FILES.gmk - makefiles/javax/swing/beaninfo/Makefile - makefiles/javax/swing/beaninfo/SwingBeans.gmk - makefiles/javax/swing/beaninfo/manifest - makefiles/javax/swing/html32dtd/Makefile - makefiles/javax/swing/plaf/FILES.gmk - makefiles/javax/swing/plaf/Makefile - makefiles/mkdemo/Makefile - makefiles/mkdemo/jni/Makefile - makefiles/mkdemo/jni/Poller/Makefile - makefiles/mkdemo/jvmti/Makefile - makefiles/mkdemo/jvmti/README.txt - makefiles/mkdemo/jvmti/hprof/Makefile - makefiles/mkdemo/jvmti/mapfile-vers - makefiles/mkdemo/management/README.txt - makefiles/sun/Makefile - makefiles/sun/awt/CondenseRules.awk - makefiles/sun/awt/Depend.mak - makefiles/sun/awt/Depend.sed - makefiles/sun/awt/FILES_c_macosx.gmk - makefiles/sun/awt/FILES_c_unix.gmk - makefiles/sun/awt/FILES_c_windows.gmk - makefiles/sun/awt/FILES_export_macosx.gmk - makefiles/sun/awt/FILES_export_unix.gmk - makefiles/sun/awt/FILES_export_windows.gmk - makefiles/sun/awt/Makefile - makefiles/sun/awt/README - makefiles/sun/awt/make.depend - makefiles/sun/awt/mapfile-mawt-vers - makefiles/sun/awt/mapfile-vers - makefiles/sun/awt/mapfile-vers-linux - makefiles/sun/awt/mawt.gmk - makefiles/sun/cmm/Makefile - makefiles/sun/cmm/kcms/FILES_c_unix.gmk - makefiles/sun/cmm/kcms/FILES_c_windows.gmk - makefiles/sun/cmm/kcms/Makefile - makefiles/sun/cmm/kcms/mapfile-vers - makefiles/sun/dcpr/FILES_c.gmk - makefiles/sun/dcpr/Makefile - makefiles/sun/dcpr/mapfile-vers - makefiles/sun/headless/Makefile - makefiles/sun/headless/mapfile-vers - makefiles/sun/headless/reorder-i586 - makefiles/sun/headless/reorder-sparc - makefiles/sun/headless/reorder-sparcv9 - makefiles/sun/image/Makefile - makefiles/sun/image/generic/FILES_c.gmk - makefiles/sun/image/generic/Makefile - makefiles/sun/image/generic/mapfile-vers - makefiles/sun/image/vis/FILES_c.gmk - makefiles/sun/image/vis/Makefile - makefiles/sun/javazic/Makefile - makefiles/sun/jdbc/Makefile - makefiles/sun/jdga/Makefile - makefiles/sun/jdga/mapfile-vers - makefiles/sun/jkernel/Makefile - makefiles/sun/lwawt/FILES_c_macosx.gmk - makefiles/sun/lwawt/FILES_export_macosx.gmk - makefiles/sun/lwawt/Makefile - makefiles/sun/nio/Makefile - makefiles/sun/nio/cs/FILES_java.gmk - makefiles/sun/nio/cs/Makefile - makefiles/sun/org/Makefile - makefiles/sun/org/mozilla/Makefile - makefiles/sun/org/mozilla/javascript/Makefile - makefiles/sun/osxapp/Makefile - makefiles/sun/security/Makefile - makefiles/sun/security/ec/FILES_c.gmk - makefiles/sun/security/ec/Makefile - makefiles/sun/security/ec/mapfile-vers - makefiles/sun/security/jgss/Makefile - makefiles/sun/security/jgss/wrapper/FILES_c.gmk - makefiles/sun/security/jgss/wrapper/Makefile - makefiles/sun/security/jgss/wrapper/mapfile-vers - makefiles/sun/security/krb5/FILES_c_windows.gmk - makefiles/sun/security/krb5/Makefile - makefiles/sun/security/mscapi/FILES_cpp.gmk - makefiles/sun/security/mscapi/Makefile - makefiles/sun/security/other/Makefile - makefiles/sun/security/pkcs11/FILES_c.gmk - makefiles/sun/security/pkcs11/Makefile - makefiles/sun/security/pkcs11/mapfile-vers - makefiles/sun/security/smartcardio/FILES_c.gmk - makefiles/sun/security/smartcardio/Makefile - makefiles/sun/security/smartcardio/mapfile-vers - makefiles/sun/security/tools/Makefile - makefiles/sun/security/util/Makefile - makefiles/sun/splashscreen/FILES_c.gmk - makefiles/sun/splashscreen/Makefile - makefiles/sun/splashscreen/mapfile-vers - makefiles/sun/xawt/FILES_c_unix.gmk - makefiles/sun/xawt/FILES_export_unix.gmk - makefiles/sun/xawt/Makefile - makefiles/sun/xawt/mapfile-vers - src/share/classes/sun/nio/ch/DevPollSelectorProvider.java - src/share/classes/sun/security/provider/certpath/CrlRevocationChecker.java - src/share/classes/sun/security/provider/certpath/OCSPChecker.java - src/share/classes/sun/tools/jconsole/resources/JConsoleResources.java - src/share/classes/sun/tools/jconsole/resources/JConsoleResources_ja.java - src/share/classes/sun/tools/jconsole/resources/JConsoleResources_zh_CN.java - src/share/native/java/sql/DriverManager.c - test/sun/security/krb5/auto/ok-as-delegate-xrealm.sh - test/sun/security/krb5/auto/ok-as-delegate.sh From Alan.Bateman at oracle.com Wed Aug 15 11:05:12 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 15 Aug 2012 19:05:12 +0100 Subject: Jigsaw classpath mode support In-Reply-To: <502154B7.7080804@oracle.com> References: <502154B7.7080804@oracle.com> Message-ID: <502BE4D8.8010402@oracle.com> On 07/08/2012 18:47, Mandy Chung wrote: > I have implemented the "classpath mode" support running on > a module image. > > Webrev: > http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/classpath-mode.00/ I've finally got through all of this. Overall this is really good work. A couple of comments/questions: In ClassPathContext when you distribute the contexts to each loader then it may be more efficient to iterate over each loader's module list and invoke the configuration's getContextForModuleName to get the Context directly. Then distribute the remaining to the loader for the boot class path. I suspect that could be more efficient than all the nested contains. One thing that I can't tell from the changes is the value of the sun.boot.module.classpath property. That might be clear once Karen publishes the hotspot changes. I ask because I can't tell if Package.SystemPackage.isPackageFromModule is right, meaning whether JVM_GetSystemPackage returns the value of sun.boot.module.classpath for packages in the system modules. In Launcher. ExtClassLoader there is code to split the extension directory into the paths that come before and after the default location. I think I need help in understand why someof this is needed because if someone sets java.ext.dirs then it should be a complete override? I agree with your idea in the mail to get this pushed and deal with any clean-up/comments in later changes, especially as the test results are good. The only thing is I assume it will need to go in with the HotSpot changes. In terms of clean-up then I think CleanPathContext should be more efficient. There are also a few things in the sun.misc.Launcher code but they aren't urgent for now. -Alan. From mandy.chung at oracle.com Wed Aug 15 15:04:05 2012 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 15 Aug 2012 15:04:05 -0700 Subject: Jigsaw classpath mode support In-Reply-To: <502BE4D8.8010402@oracle.com> References: <502154B7.7080804@oracle.com> <502BE4D8.8010402@oracle.com> Message-ID: <502C1CD5.5000102@oracle.com> On 8/15/2012 11:05 AM, Alan Bateman wrote: > On 07/08/2012 18:47, Mandy Chung wrote: >> I have implemented the "classpath mode" support running on >> a module image. >> >> Webrev: >> http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/classpath-mode.00/ > I've finally got through all of this. Overall this is really good work. > Thanks for the review. > A couple of comments/questions: > > In ClassPathContext when you distribute the contexts to each loader > then it may be more efficient to iterate over each loader's module > list and invoke the configuration's getContextForModuleName to get the > Context directly. Then distribute the remaining to the loader for the > boot class path. I suspect that could be more efficient than all the > nested contains. > You are referring to L209-222, right? This is a good suggestion. I'll revise it. > One thing that I can't tell from the changes is the value of the > sun.boot.module.classpath property. That might be clear once Karen > publishes the hotspot changes. I ask because I can't tell if > Package.SystemPackage.isPackageFromModule is right, meaning whether > JVM_GetSystemPackage returns the value of sun.boot.module.classpath > for packages in the system modules. > Yes, JVM_GetSystemPackage will return the value of sun.boot.module.classpath for packages in the system modules. It's a simple way to tag a package in the system module. > In Launcher. ExtClassLoader there is code to split the extension > directory into the paths that come before and after the default > location. I think I need help in understand why someof this is needed > because if someone sets java.ext.dirs then it should be a complete > override? > It's for searching native libraries from the module image or user-specified ext dir. The ExtClassLoader overrides the findLibrary method. The java.ext.dirs can be overridden in the command line; if not set, the VM will set its value to the default ext dirs. In addition, someone can set java.ext.dirs to include JAVA_HOME/lib/ext plus other additional directory. If the default ext dir is inclued in the java.ext.dirs, ExtClassLoader needs to locate the native library from the module image and loads a native library from each directory specified in the property value before and after the default. > I agree with your idea in the mail to get this pushed and deal with > any clean-up/comments in later changes, especially as the test results > are good. The only thing is I assume it will need to go in with the > HotSpot changes. > > In terms of clean-up then I think CleanPathContext should be more > efficient. There are also a few things in the sun.misc.Launcher code > but they aren't urgent for now. > I'll fix the ClassPathContext how it distributes the modules/contexts to each loader. Let me know what other things you have in mind and I will see if I should fix them together. Thanks Mandy From jonathan.gibbons at oracle.com Wed Aug 15 17:16:18 2012 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Wed, 15 Aug 2012 17:16:18 -0700 Subject: v51 class files Message-ID: <502C3BD2.7060004@oracle.com> Arguably this is a question for a wider audience, but I'll ask here first. javac is still generating v51 class files for -target 8. When should we cut over to generating v52 class files? -- Jon From Alan.Bateman at oracle.com Thu Aug 16 08:05:16 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 16 Aug 2012 16:05:16 +0100 Subject: Jigsaw classpath mode support In-Reply-To: <502C1CD5.5000102@oracle.com> References: <502154B7.7080804@oracle.com> <502BE4D8.8010402@oracle.com> <502C1CD5.5000102@oracle.com> Message-ID: <502D0C2C.5080707@oracle.com> On 15/08/2012 23:04, Mandy Chung wrote: > : > >> One thing that I can't tell from the changes is the value of the >> sun.boot.module.classpath property. That might be clear once Karen >> publishes the hotspot changes. I ask because I can't tell if >> Package.SystemPackage.isPackageFromModule is right, meaning whether >> JVM_GetSystemPackage returns the value of sun.boot.module.classpath >> for packages in the system modules. >> > Yes, JVM_GetSystemPackage will return the value of > sun.boot.module.classpath for packages in the system modules. It's a > simple way to tag a package in the system module. Okay, that's what I thought (it caught my eye because I thought that JVM_GetSystemPackage was expected to return a specific path or filename rather than a classpath). > >> In Launcher. ExtClassLoader there is code to split the extension >> directory into the paths that come before and after the default >> location. I think I need help in understand why someof this is needed >> because if someone sets java.ext.dirs then it should be a complete >> override? >> > > It's for searching native libraries from the module image or > user-specified ext dir. The ExtClassLoader overrides the findLibrary > method. The java.ext.dirs can be overridden in the command line; if > not set, the VM will set its value to the default ext dirs. In > addition, someone can set java.ext.dirs to include JAVA_HOME/lib/ext > plus other additional directory. If the default ext dir is inclued > in the java.ext.dirs, ExtClassLoader needs to locate the native > library from the module image and loads a native library from each > directory specified in the property value before and after the default. Thanks, the bit I was missing here was the possibility that someone might run with java.ext.dirs set to include the legacy ext directory. In that case, shouldn't the check on L220 be <= index so that the extensions directory ends up in prepaths rather than postpaths. That way it will search the legacy location before looking in the module directory. -Alan. From alex.buckley at oracle.com Thu Aug 16 09:36:10 2012 From: alex.buckley at oracle.com (Alex Buckley) Date: Thu, 16 Aug 2012 09:36:10 -0700 Subject: v51 class files In-Reply-To: <502C3BD2.7060004@oracle.com> References: <502C3BD2.7060004@oracle.com> Message-ID: <502D217A.3010005@oracle.com> Any javac that emits ClassFiles beyond those specified in JVMS7 - such as with new attributes (Jigsaw, Type Annotations) or new flags (Lambda) - should be using version 52.0. Alex On 8/15/2012 5:16 PM, Jonathan Gibbons wrote: > Arguably this is a question for a wider audience, but I'll ask here first. > > javac is still generating v51 class files for -target 8. > > When should we cut over to generating v52 class files? > > -- Jon From jonathan.gibbons at oracle.com Thu Aug 16 11:18:10 2012 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 16 Aug 2012 11:18:10 -0700 Subject: v51 class files In-Reply-To: <502D217A.3010005@oracle.com> References: <502C3BD2.7060004@oracle.com> <502D217A.3010005@oracle.com> Message-ID: <502D3962.8010602@oracle.com> I agree, but it has been convenient to allow a grace period for Jigsaw development to allow the rest of the tools to catch up with support for v52. So, the question is, are we ready to end the grace period and convert to using v52 class files in normal use? Who gets to make that decision, and who gets to coordinate the change? And, since Jigsaw is now targeted for JDK 9, and defines attributes that will presumably not be in Java 8, should Jigsaw start moving towards v53 class files? -- Jon On 08/16/2012 09:36 AM, Alex Buckley wrote: > Any javac that emits ClassFiles beyond those specified in JVMS7 - such > as with new attributes (Jigsaw, Type Annotations) or new flags > (Lambda) - should be using version 52.0. > > Alex > > On 8/15/2012 5:16 PM, Jonathan Gibbons wrote: >> Arguably this is a question for a wider audience, but I'll ask here >> first. >> >> javac is still generating v51 class files for -target 8. >> >> When should we cut over to generating v52 class files? >> >> -- Jon From mandy.chung at oracle.com Thu Aug 16 17:01:11 2012 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 16 Aug 2012 17:01:11 -0700 Subject: Jigsaw classpath mode support In-Reply-To: <502D0C2C.5080707@oracle.com> References: <502154B7.7080804@oracle.com> <502BE4D8.8010402@oracle.com> <502C1CD5.5000102@oracle.com> <502D0C2C.5080707@oracle.com> Message-ID: <502D89C7.8050203@oracle.com> On 8/16/2012 8:05 AM, Alan Bateman wrote: > On 15/08/2012 23:04, Mandy Chung wrote: >> : >> >>> One thing that I can't tell from the changes is the value of the >>> sun.boot.module.classpath property. That might be clear once Karen >>> publishes the hotspot changes. I ask because I can't tell if >>> Package.SystemPackage.isPackageFromModule is right, meaning whether >>> JVM_GetSystemPackage returns the value of sun.boot.module.classpath >>> for packages in the system modules. >>> >> Yes, JVM_GetSystemPackage will return the value of >> sun.boot.module.classpath for packages in the system modules. It's a >> simple way to tag a package in the system module. > Okay, that's what I thought (it caught my eye because I thought that > JVM_GetSystemPackage was expected to return a specific path or > filename rather than a classpath). > > Ah the property name causes the confusion. Would it help if it's renamed to "sun.boot.base.module.path"? Right now its value is set to "/jdk.base". It's a unique value that the libs can use to determine if it's from the module image. We might reconsider what value makes sense here when doing the modulepath support. > > In that case, shouldn't the check on L220 be <= index so that the > extensions directory ends up in prepaths rather than postpaths. That > way it will search the legacy location before looking in the module > directory. > Good catch and will fix it. Mandy From alex.buckley at oracle.com Fri Aug 17 12:23:03 2012 From: alex.buckley at oracle.com (Alex Buckley) Date: Fri, 17 Aug 2012 12:23:03 -0700 Subject: Scopes in the Jigsaw module system Message-ID: <502E9A17.8010608@oracle.com> I have added a page to the Jigsaw project which explains our recent thinking on "scopes", a mechanism to control the phases in which module dependencies must be resolved. http://openjdk.java.net/projects/jigsaw/doc/topics/scopes.html Comments welcome. Alex From jaroslav.tulach at oracle.com Fri Aug 17 23:05:24 2012 From: jaroslav.tulach at oracle.com (Jaroslav Tulach) Date: Sat, 18 Aug 2012 08:05:24 +0200 Subject: Scopes in the Jigsaw module system In-Reply-To: <502E9A17.8010608@oracle.com> References: <502E9A17.8010608@oracle.com> Message-ID: <1866048.muKhuYOYOD@logoutik> Dne P? 17. srpna 2012 12:23:03, Alex Buckley napsal(a): > I have added a page to the Jigsaw project which explains our recent > thinking on "scopes", a mechanism to control the phases in which module > dependencies must be resolved. > > http://openjdk.java.net/projects/jigsaw/doc/topics/scopes.html > > Comments welcome. Hello Alex, nice. This addresses majority of the issues we run into recently. I like the "reflection" scope - it directly names the intent of what people are about to do with such dependency most often - e.g. use Class.forName. As an unimportant contribution to the terminology dispute, here is my 2K? vote: I like "reflection" more than "optional execution", for the reason stated in previous paragraph. Moreover I have a feeling that a more direct description for "execution" scope would be "linkage" - e.g. directly express that the intention of this scope is to prevent LinkageErrors. It is true that "requires M for execution" is sort of weird (this kind of dependency should rather be handled by "requires service" imho), but for people who want to do Class.forName(...) and be sure it always succeeds it may be useful. Btw. it is nice how symmetrical the scope system is! I am glad there is a paragraph about Annotation processors, but I will need some clarifications. I see following sentence: > That said, the proper way to view an annotation processor is as a standalone > module which has a) a main class that implements > javax.annotation.processing.Processor and b) its own dependencies, some of > which may be optional. If I interpret it correctly, you suggest to package the processor into independent module, separated from the module hosting the annotation. Good, that is how it should be! Annotation processors usually have much wider set of dependencies (including I/O, javax.processing, etc.) and mixing them with the module that provides the API with source time annotation would needlessly increase the weight of such module (weight being a number of outgoing transitive dependencies; even conditional). However how do we guarantee that a source only annotation is properly processed whenever somebody compiles against the API? Imagine that java.beans module provides @GenerateBeanInfo annotation to be placed on any JavaBean class and generate appropriate BeanInfo class automatically rather than requiring it to be code manually. If I correctly understood, the associated GenerateBeanInfoProcessor should be placed in a separate java.beans.processor module. How will the javac know that whenever somebody uses @GenerateBeanInfo annotation in own code the java.beans.processor module must be consulted during compilation? -jt From daniel.latremoliere at gmail.com Sat Aug 18 03:41:36 2012 From: daniel.latremoliere at gmail.com (=?UTF-8?B?RGFuaWVsIExhdHLDqW1vbGnDqHJl?=) Date: Sat, 18 Aug 2012 12:41:36 +0200 Subject: Scopes in the Jigsaw module system In-Reply-To: <1866048.muKhuYOYOD@logoutik> References: <502E9A17.8010608@oracle.com> <1866048.muKhuYOYOD@logoutik> Message-ID: <502F7160.7040405@gmail.com> > Imagine that java.beans module provides @GenerateBeanInfo annotation > to be placed on any JavaBean class and generate appropriate BeanInfo > class automatically rather than requiring it to be code manually. If I > correctly understood, the associated GenerateBeanInfoProcessor should > be placed in a separate java.beans.processor module. How will the > javac know that whenever somebody uses @GenerateBeanInfo annotation in > own code the java.beans.processor module must be consulted during > compilation? Wouldn't it be possible to use custom scopes: an annotation processor will provide scopes with some mapped to "annotation-processor" and "execution", to implement his contract of annotation processor. How many scopes for generated bytecode? By example an XSLT stylesheet loaded with xsltc (like in OpenJDK) will be transformed dynamically in bytecode without compilation (javac) step. How many scopes for weaved modules, by example with AspectJ or OpenJPA? Will weaver change the module-info.class to add the dependency to his runtime library in execution scope? I can understand one scope (execution), many scopes (the custom steps, and that is what I am using with Ivy), but I have a problem with two and only two scopes. cf. section "Different concepts" on Ivy configurations against Maven scopes: http://ant.apache.org/ivy/m2comparison.html Sorry for being boring, Daniel. From jaroslav.tulach at oracle.com Sat Aug 18 07:08:39 2012 From: jaroslav.tulach at oracle.com (Jaroslav Tulach) Date: Sat, 18 Aug 2012 16:08:39 +0200 Subject: Different concepts was: Scopes In-Reply-To: <502F7160.7040405@gmail.com> References: <502E9A17.8010608@oracle.com> <1866048.muKhuYOYOD@logoutik> <502F7160.7040405@gmail.com> Message-ID: <5311175.KqOESHy1Cl@logoutik> Hello Daniel, > Sorry for being boring, your post is not boring, it is interesting. It shows that there is a problem to solve. It also shows that there are various ideas to deal with the problem and it helps me see drawbacks of various solutions. A quote from the Ivy documentation follows: > The problem is that hibernate can be used with several cache > implementations, several connection pool implementation, ... And this can't > be managed with scopes, wheres Apache Ivy configurations offers an elegant > solution to this kind of problem. For instance, assuming hibernate has an > ivy file like this one, then you can declare a dependency like that: The problem that Ivy configurations try to solve (e.g. different cache implementations for Hibernate) is not at all addressed by Maven. No suprise Ivy guys feel they have something good to offer. On the other hand runtime containers like NetBeans, OSGi 4.3 as well as Jigsaw offer (in my opinion more flexible) solution to the problem. All of them provide a concept of "requires a service". With a concept like that the hibernate at 1.0 module can say it: requires service hibernate.Cache and possibly also provide a hint offering the default for the service to come from a module for JBoss: with default hibernate.jbosscache at 7.8 As soon as anyone depends on hibernate at 1.0 module the system knows it has to select at least one provider of the Cache service. It also knows what is the preferred one (the JBoss probably), but can of course be configured to select any other provider of the hibernate.Cache service[1]. This means the system never carries "several cache implementations" - some of which would have to be excluded. One only depends on the hibernate API module and it is left up to the Jigsaw resolver[2] to provide at least one workable cache. -jt [1] Actual style of the configuration in Jigsaw to be specified yet; but the need is clear, imho. [2] Configured by the overall application deployer, probably[1]. Dne So 18. srpna 2012 12:41:36, Daniel Latr?moli?re napsal(a): > > Imagine that java.beans module provides @GenerateBeanInfo annotation > > to be placed on any JavaBean class and generate appropriate BeanInfo > > class automatically rather than requiring it to be code manually. If I > > correctly understood, the associated GenerateBeanInfoProcessor should > > be placed in a separate java.beans.processor module. How will the > > javac know that whenever somebody uses @GenerateBeanInfo annotation in > > own code the java.beans.processor module must be consulted during > > compilation? > > Wouldn't it be possible to use custom scopes: an annotation processor > will provide scopes with some mapped to "annotation-processor" and > "execution", to implement his contract of annotation processor. > > How many scopes for generated bytecode? By example an XSLT stylesheet > loaded with xsltc (like in OpenJDK) will be transformed dynamically in > bytecode without compilation (javac) step. > > How many scopes for weaved modules, by example with AspectJ or OpenJPA? > Will weaver change the module-info.class to add the dependency to his > runtime library in execution scope? > > I can understand one scope (execution), many scopes (the custom steps, > and that is what I am using with Ivy), but I have a problem with two and > only two scopes. > > cf. section "Different concepts" on Ivy configurations against Maven scopes: > http://ant.apache.org/ivy/m2comparison.html > > Sorry for being boring, > Daniel. From sean.mullan at oracle.com Sat Aug 18 09:02:11 2012 From: sean.mullan at oracle.com (sean.mullan at oracle.com) Date: Sat, 18 Aug 2012 16:02:11 +0000 Subject: hg: jigsaw/jigsaw/jdk: 7191717: jigsaw solaris-i586 and solaris-sparc build failure with SKIP_BOOT_CYCLE=false Message-ID: <20120818160236.EF8F3475C4@hg.openjdk.java.net> Changeset: 710aafca95a1 Author: smullan Date: 2012-08-18 08:52 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/710aafca95a1 7191717: jigsaw solaris-i586 and solaris-sparc build failure with SKIP_BOOT_CYCLE=false Reviewed-by: mchung ! make/com/oracle/security/ucrypto/Makefile ! make/com/sun/crypto/provider/Makefile ! make/javax/crypto/Defs-jce.gmk ! make/javax/crypto/Makefile ! make/sun/security/ec/Makefile ! make/sun/security/mscapi/Makefile ! make/sun/security/pkcs11/Makefile From daniel.latremoliere at gmail.com Sat Aug 18 11:48:01 2012 From: daniel.latremoliere at gmail.com (=?ISO-8859-1?Q?Daniel_Latr=E9moli=E8re?=) Date: Sat, 18 Aug 2012 20:48:01 +0200 Subject: Different concepts was: Scopes In-Reply-To: <5311175.KqOESHy1Cl@logoutik> References: <502E9A17.8010608@oracle.com> <1866048.muKhuYOYOD@logoutik> <502F7160.7040405@gmail.com> <5311175.KqOESHy1Cl@logoutik> Message-ID: <502FE361.2060008@gmail.com> > On the other hand runtime containers like NetBeans, OSGi 4.3 as well as Jigsaw > offer (in my opinion more flexible) solution to the problem. All of them provide > a concept of "requires a service".: On container (sharply for simplicity): * If Jigsaw is a runtime container, it does not know compile time, then does not need this scope in its descriptor, then it will have only one scope (good for me at runtime, cf. end). * If it is a aware of lifecycle of code, like compiling, it will need to accept all processes building bytecode, like direct bytecode generation (xsltc), or weaving (sometimes statically with JPA), source code generation and compilation (JAXB/XJC) then it will integrate with build process. Ant script for building can be complex and very customized then need sometimes multiple custom scopes, sometimes does not use even one standard javac compilation (xsltc). On services vs. Ivy configurations: Shortly, for me, services and Ivy configurations are not exclusive, they are only the two parts of solution (dynamic and static). At runtime, you can ask for implementations allowing dynamic cyclic dependencies with IoC. At build time, you avoid cyclic dependencies by having an explicit list of features needed and disallowing IoC. More precisely, think services vs. Ivy configurations is a dynamic vs. static problem (slightly added with an Inversion of Control problem). * A service is mostly an execution problem (obtaining an instance of the service) with IoC (I name the service API and ask for implementations). His only static part is common API visible for producers and consumers. * An Ivy configuration is a build problem (providing a specific optional feature) directly (caller will list all features asked; there is no IoC). Service is clearly more flexible with IoC than asking specific instance but can lead to code difficult to understand (binding provider-consumer difficult to understand/manage). It has much more module explosion (service API need his own small module at runtime, and possibly at build time to avoid cyclic dependencies when building). For me, services in static modules would be good if they are used for a fully common contract defined externally between many sides using it without knowing the other parts. By example JavaEE is a contract with many providers (JavaEE servers) and many consumers (Webapps). You can deploy any JavaEE webapp on any JavaEE server (with hope and very good code). In this case, I clearly see JavaEE API as one module by itself, at build time (statically) and at execution time (dynamically), used by the two sides for communicating between them. But many plugins systems are dissymetric: one product on one side, many products on the other side. By example, Eclipse or NetBeans have plugins, but only one Eclipse or one NetBeans (and no compatibility between Eclipse plugins and NetBeans plugins). In this case, I see the plugin API as an Ivy configuration of Eclipse/Netbeans plugin manager. A plugin always know its container (Netbeans or Eclipse respectively): it is not a common contract between code not knowing each other. The plugin API is only one artefact of module at build time (statically) but one module at execution time (dynamically). Each plugin have registered a static dependency to plugin API scope of Eclipse/NetBeans plugin manager. In the two case, services can be used dynamically for finding instances implementing (by examples all Eclipse/NetBeans plugins). On JavaEE, services are not applicable in current JavaEE-specific deployment. > This means the system never carries "several cache implementations" - some of > which would have to be excluded. One only depends on the hibernate API module > and it is left up to the Jigsaw resolver[2] to provide at least one workable > cache. > [1] Actual style of the configuration in Jigsaw to be specified yet; but the > need is clear, imho. > [2] Configured by the overall application deployer, probably[1]. Exactly, Ivy configurations are useful when used at deployment time, not at execution time (it is only one classpath of artefacts). Yes, the need will be clear: how can I configure Jigsaw resolver if I want to deploy two webapps on my JavaEE server with different pluggable implementations (like strategies of cache, filesystems supported by VFS, etc.)? But it is important to see that this need is mostly defined at deployment time (not compilation, not execution). NB: I think, that sometimes it will probably be simpler to write a simple deployment module with all deployment dependencies and some small code for explicitly binding providers and consumers (like Guava binder), registering all plugins allowed in their manager (like FileSystems). It can avoid nasty bugs created by not understood behaviour of automatic binding, automatic dependencies resolving (by example, when a new service provider come in module repository by a side effect of something unrelated specifically to the service, and introduce a hole in security with his new feature added in current environment). --- Given I think this is important, I will add precision to something (with some refining of my problem). When I have written that I understand one scope or multiple custom scopes but not two scopes (compilation and execution), you can see this as a semantic problem of "module" definition. * If you call module something with only one artefact (JAR by example), I will see only one scope for it: visible (or invisible) to module requiring it. This is typically the case for a runtime modularity container. * If you call module something with multiple artefacts (source, binary, etc.), multiples scopes/configurations, like Ivy, I will see various/custom scopes depending of its build process (source compiled, generated bytecode, weaved bytecode). This is typically the case of Ivy static modularity. Fundamentally, this is only a problem of naming "module" each artefact in repository (dynamically) or each directory of repository containing artefacts (statically). The two definitions seems good for me, but one is for dynamic modularity, the other for static modularity, in my opinion. Globally, my problem with compilation scope is probably in not understanding reasons of merging all facets of modularity in Jigsaw: * static part of modularity (repository of artefacts used/populated by build/deployment process, with multiple artefacts by static module) [Ivy-like]. * dynamic part of modularity (ClassLoader used at runtime for loading code, with one artefact by dynamic module) [classpath on steroids]. Dynamic modularity is only one user of static modularity (with javac, javadoc, IDE, etc.). By example a java compiler need only signature of methods, etc. in dependencies for making his job: it can obtain this without loading bytecode in JVM; by example, ASM is sufficient for visiting classes. Finally, for me a dynamic module can simply be an artefact of a static module. A static module is a directory containing many artefacts, including some dynamic modules, but potentially also source code or many other published files (XSD schema, etc.), sometimes deployed and used on a JVM (by example a schema used for validating XML), sometimes not deployed by administrator/creator of application. Static and Dynamic modularity need, for me, to be seen as separate problems (even with different definitions of "module"). Thank you, if you attained this line! Daniel. From alex.buckley at oracle.com Mon Aug 20 10:39:25 2012 From: alex.buckley at oracle.com (Alex Buckley) Date: Mon, 20 Aug 2012 10:39:25 -0700 Subject: Scopes in the Jigsaw module system In-Reply-To: <1866048.muKhuYOYOD@logoutik> References: <502E9A17.8010608@oracle.com> <1866048.muKhuYOYOD@logoutik> Message-ID: <5032764D.9050509@oracle.com> On 8/17/2012 11:05 PM, Jaroslav Tulach wrote: > If I interpret it correctly, you suggest to package the processor into > independent module, separated from the module hosting the annotation. Good, > that is how it should be! Agreed. Caveat: the Jigsaw team hasn't discussed annotation processors much, so people shouldn't assume much about how other use cases will be handled. > However how do we guarantee that a source only annotation is properly > processed whenever somebody compiles against the API? > > Imagine that java.beans module provides @GenerateBeanInfo annotation to be > placed on any JavaBean class and generate appropriate BeanInfo class > automatically rather than requiring it to be code manually. If I correctly > understood, the associated GenerateBeanInfoProcessor should be placed in a > separate java.beans.processor module. How will the javac know that whenever > somebody uses @GenerateBeanInfo annotation in own code the > java.beans.processor module must be consulted during compilation? Good question. Again, the Jigsaw team hasn't discussed how to identify, package, or execute annotation processors yet. JSR 269 defines the environment that an annotation processor can see (e.g. ProcessingEnvironment, Filer) but the identification and execution of annotation processors is not standardized. How much can easily be standardized in a module system is an open question. Alex From jaroslav.tulach at oracle.com Mon Aug 20 15:26:47 2012 From: jaroslav.tulach at oracle.com (Jaroslav Tulach) Date: Tue, 21 Aug 2012 00:26:47 +0200 Subject: Modular annotation processor registration was: Scopes in the Jigsaw module system In-Reply-To: <5032764D.9050509@oracle.com> References: <502E9A17.8010608@oracle.com> <1866048.muKhuYOYOD@logoutik> <5032764D.9050509@oracle.com> Message-ID: <1604925.KoJNYENX2H@logoutik> Dne Po 20. srpna 2012 10:39:25, Alex Buckley napsal(a): > On 8/17/2012 11:05 PM, Jaroslav Tulach wrote: > > If I interpret it correctly, you suggest to package the processor into > > independent module, separated from the module hosting the annotation. > > Good, > > that is how it should be! > > Agreed. Caveat: the Jigsaw team hasn't discussed annotation processors > much, so people shouldn't assume much about how other use cases will be > handled. OK. I keep that in mind. > > However how do we guarantee that a source only annotation is properly > > processed whenever somebody compiles against the API? > > > > Imagine that java.beans module provides @GenerateBeanInfo annotation to be > > placed on any JavaBean class and generate appropriate BeanInfo class > > automatically rather than requiring it to be code manually. If I correctly > > understood, the associated GenerateBeanInfoProcessor should be placed in a > > separate java.beans.processor module. How will the javac know that > > whenever > > somebody uses @GenerateBeanInfo annotation in own code the > > java.beans.processor module must be consulted during compilation? > > Good question. Again, the Jigsaw team hasn't discussed how to identify, > package, or execute annotation processors yet. JSR 269 defines the > environment that an annotation processor can see (e.g. > ProcessingEnvironment, Filer) but the identification and execution of > annotation processors is not standardized. How much can easily be > standardized in a module system is an open question. I have an idea I'd like to share. Right now the Processor subclass is registered as "provides service" and then it identifies which annotations it processes. There is a pointer from the processor to the annotation. If we want to have processors separated from annotations, then we also need a link from the annotation to its processor. I guess a meta-annotation like: @Target(ElementType.ANNOTATION_TYPE) @Retention(RetentionType.CLASS) @interface WhichModuleProvidesMyProcessor { String value(); } which I could use to define my source annotation @Retention(SOURCE) @WhichModuleProvidesMyProcessor("my.processor.module at 2.7") @interface MyAnnotation { } would be enough for javac to know that it has to find and turn on my.processor.module in order to compile sources refering to MyAnnotation. If the module my.processor.module at 2.7 is not found, the compilation fails with error. Just my 2K? to discussion about annotation processors (when you get to it). -jt From alex.buckley at oracle.com Mon Aug 20 15:37:15 2012 From: alex.buckley at oracle.com (Alex Buckley) Date: Mon, 20 Aug 2012 15:37:15 -0700 Subject: Modular annotation processor registration was: Scopes in the Jigsaw module system In-Reply-To: <1604925.KoJNYENX2H@logoutik> References: <502E9A17.8010608@oracle.com> <1866048.muKhuYOYOD@logoutik> <5032764D.9050509@oracle.com> <1604925.KoJNYENX2H@logoutik> Message-ID: <5032BC1B.70407@oracle.com> On 8/20/2012 3:26 PM, Jaroslav Tulach wrote: > If we want to have processors separated from annotations, then we also need a > link from the annotation to its processor. I guess a meta-annotation like: > > @Target(ElementType.ANNOTATION_TYPE) > @Retention(RetentionType.CLASS) > @interface WhichModuleProvidesMyProcessor { > String value(); > } > > which I could use to define my source annotation > > @Retention(SOURCE) > @WhichModuleProvidesMyProcessor("my.processor.module at 2.7") > @interface MyAnnotation { > } > > would be enough for javac to know that it has to find and turn on > my.processor.module in order to compile sources refering to MyAnnotation. If > the module my.processor.module at 2.7 is not found, the compilation fails with > error. > > Just my 2K? to discussion about annotation processors (when you get to it). > -jt Thanks. While "annotation processing dependencies" are an interesting special case, we should stick to the common case of module dependencies for now. I'll just note that binding an annotation type to a processor only works when there's an annotation type involved - in general, you can write processors which have no interest in annotations whatsoever. Alex From jonathan.gibbons at oracle.com Mon Aug 20 15:46:41 2012 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Mon, 20 Aug 2012 15:46:41 -0700 Subject: Modular annotation processor registration was: Scopes in the Jigsaw module system In-Reply-To: <5032BC1B.70407@oracle.com> References: <502E9A17.8010608@oracle.com> <1866048.muKhuYOYOD@logoutik> <5032764D.9050509@oracle.com> <1604925.KoJNYENX2H@logoutik> <5032BC1B.70407@oracle.com> Message-ID: <5032BE51.9040608@oracle.com> On 08/20/2012 03:37 PM, Alex Buckley wrote: > On 8/20/2012 3:26 PM, Jaroslav Tulach wrote: >> If we want to have processors separated from annotations, then we >> also need a >> link from the annotation to its processor. I guess a meta-annotation >> like: >> >> @Target(ElementType.ANNOTATION_TYPE) >> @Retention(RetentionType.CLASS) >> @interface WhichModuleProvidesMyProcessor { >> String value(); >> } >> >> which I could use to define my source annotation >> >> @Retention(SOURCE) >> @WhichModuleProvidesMyProcessor("my.processor.module at 2.7") >> @interface MyAnnotation { >> } >> >> would be enough for javac to know that it has to find and turn on >> my.processor.module in order to compile sources refering to >> MyAnnotation. If >> the module my.processor.module at 2.7 is not found, the compilation >> fails with >> error. >> >> Just my 2K? to discussion about annotation processors (when you get >> to it). >> -jt > > Thanks. While "annotation processing dependencies" are an interesting > special case, we should stick to the common case of module > dependencies for now. > > I'll just note that binding an annotation type to a processor only > works when there's an annotation type involved - in general, you can > write processors which have no interest in annotations whatsoever. > > Alex Jaroslav, ... and you are assuming that there is a single processor to be used to process any particular annotation, which may not be the case, and that such a processor should be specified in the source file. -- Jon From jaroslav.tulach at oracle.com Mon Aug 20 16:10:05 2012 From: jaroslav.tulach at oracle.com (Jaroslav Tulach) Date: Tue, 21 Aug 2012 01:10:05 +0200 Subject: Modular annotation processor registration In-Reply-To: <5032BC1B.70407@oracle.com> References: <502E9A17.8010608@oracle.com> <1604925.KoJNYENX2H@logoutik> <5032BC1B.70407@oracle.com> Message-ID: <1689273.2fG99e1lOb@logoutik> Dne Po 20. srpna 2012 15:37:15, Alex Buckley napsal(a): > On 8/20/2012 3:26 PM, Jaroslav Tulach wrote: > > If we want to have processors separated from annotations, then we also > > need a link from the annotation to its processor. I guess a > > meta-annotation like: > > > > @Target(ElementType.ANNOTATION_TYPE) > > @Retention(RetentionType.CLASS) > > @interface WhichModuleProvidesMyProcessor { > > > > String value(); > > > > } > > > > which I could use to define my source annotation > > > > @Retention(SOURCE) > > @WhichModuleProvidesMyProcessor("my.processor.module at 2.7") > > @interface MyAnnotation { > > } > > > > would be enough for javac to know that it has to find and turn on > > my.processor.module in order to compile sources refering to MyAnnotation. > > If the module my.processor.module at 2.7 is not found, the compilation fails > > with error. > > > > Just my 2K? to discussion about annotation processors (when you get to > > it). > > -jt > > Thanks. While "annotation processing dependencies" are an interesting > special case, we should stick to the common case of module dependencies > for now. > > I'll just note that binding an annotation type to a processor only works > when there's an annotation type involved - in general, you can write > processors which have no interest in annotations whatsoever. True. But those who wish to use such processors need to be aware of their existence and need to include their modules on compilation path. That is not much different from what people need to do these days and good enough for a common clueless programmer to handle. The case of a separate annotation (like @GenerateBeanInfo discussed previously) and its processor is new. These days people usually package the processor next to the annotation. Then it is guaranteed javac will find the processor and use it by default. It is desirable (from a modular point of view) to support annotations separated from their processors. The major thing to be aware of is the danger of producing different result depending whether end users forget/don't forget to include also dependency on the module with the processor. If we allow people to use @GenerateBeanInfo without java.beans.impl module with appropriate processor, there will be surprises. I certainly don't want to be on the hotline to explain why something compiles, but don't not produce the desirable results... A hint from the annotation al? @WhichModuleProvidesMyProcessor to javac would guarantee reliable compilation results. -jt From mandy.chung at oracle.com Mon Aug 20 19:52:50 2012 From: mandy.chung at oracle.com (mandy.chung at oracle.com) Date: Tue, 21 Aug 2012 02:52:50 +0000 Subject: hg: jigsaw/jigsaw/jdk: Fix jps argument parsing and jstatd runs in legacy mode Message-ID: <20120821025327.6D87F47653@hg.openjdk.java.net> Changeset: 51e9f5c0fa56 Author: mchung Date: 2012-08-20 19:52 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/51e9f5c0fa56 Fix jps argument parsing and jstatd runs in legacy mode ! make/launchers/Makefile ! src/share/classes/sun/jvmstat/monitor/MonitoredVmUtil.java ! test/ModulesProblemList.txt From mandy.chung at oracle.com Tue Aug 21 10:09:42 2012 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 21 Aug 2012 10:09:42 -0700 Subject: Jigsaw classpath mode support In-Reply-To: <502154B7.7080804@oracle.com> References: <502154B7.7080804@oracle.com> Message-ID: <5033C0D6.5080409@oracle.com> Updated webrev at: http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/classpath-mode.01/ I have modified org.openjdk.jigsaw.ClassPathContext and sun.misc.Launcher classes to incorporate the comments Alan, Paul and Karen have. Also updated the ModulesProblemList.txt to include a few known failures that will be looked into next. Mandy On 8/7/2012 10:47 AM, Mandy Chung wrote: > I have implemented the "classpath mode" support running on > a module image. > > Webrev: > http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/classpath-mode.00/ > > JDK has been modularized in the jigsaw repo. There no longer > has a rt.jar, tools.jar, or any jar file. The system classes > are now installed in a system module library as modules. This > change will allow the bootstrap class loader, extension class > loader, and system class loader to search classes and resources > in the system module library as the default. Some implementation > details: > > 1. VM used to maintain the default bootclasspath list and search > classes in it. Now it will call jigsaw native interface to find > classes from the module library unless -Xbootclasspath is set > to override the default bootclasspath. The -Xbootclasspath/p > and -Xbootclasspath/a will continue to work as it is today. > > Karen will be sending out the webrev for the hotspot change > separately. > > 2. tools.jar > > For compatibility, -classpath tools.jar should continue > to work. There are existing applications using URLClassLoader > that loads classes from tools.jar (e.g. test harness) that > also should continue to work. > > To be able to determine if the path of the given tools.jar > to either -classpath option or URL parameter to URLClassLoader > is the same path as JAVA_HOME/lib/tools.jar in a lightweight, > the module image for the full JDK will contain an empty tools.jar. > This also allows applications that check for the existence of > tools.jar to continue to work. > > sun.misc.URLClassPath is the main implementation class to > support URLClassLoader to search a file from the given URLs. > Both ExtClassLoader and AppClassLoader are URLClassLoader. > URLClassPath is extended to find resource file from the > appropriate system modules if the given URL is > JAVA_HOME/lib/tools.jar or JAVA_HOME/lib/ext. > > 3. How to determine which modules belong to bootclasspath, > ext directory, and tools.jar (and a few jar files in JDK/lib) > at runtime > > A configuration for the classpath mode support is loaded at > startup rather than generated to minimize the startup cost. > The idea is to have one ClassPathContext for bootclasspath, > one for extension and one for the jdk tools. > > In this prototype, I just define a new module "jdk.classpath" > and install it in the module image. It hardcodes the list > of modules for each ClassPathContext for now. One alternative > is to extend jpkg to generate "jdk.classpath" config specifically > for this classpath mode. We'll explore this later. But I > think this is good enough for now. > > 4. I made simple change in StoredConfiguration to reduce > the footprint of the configuration stored on disk. There > were several regression tests setting -Xmx failed due to > the increase of the startup footprint without this change. > > With storing the context names and package names only once, > it reduces the size of the config with 35-40%. I suspect > the fast configuration will make similar change. > > $ du -k `find . -name 'config'` > 696 ./jdk.corba/8-ea/config > 1304 ./jdk.classpath/8-ea/config > 1232 ./jdk.tools/8-ea/config > 604 ./jdk.rmi/8-ea/config > 680 ./jdk.tools.base/8-ea/config > 604 ./jdk.base/8-ea/config > 1056 ./jdk.tools.jaxws/8-ea/config > 676 ./jdk.devtools/8-ea/config > 608 ./jdk.tools.jre/8-ea/config > > $ du -k `find . -name 'config'` > 444 ./jdk.corba/8-ea/config > 792 ./jdk.classpath/8-ea/config > 748 ./jdk.tools/8-ea/config > 392 ./jdk.rmi/8-ea/config > 436 ./jdk.tools.base/8-ea/config > 392 ./jdk.base/8-ea/config > 636 ./jdk.tools.jaxws/8-ea/config > 436 ./jdk.devtools/8-ea/config > 396 ./jdk.tools.jre/8-ea/config > > > 5. I ran the jtreg regression tests and JCK api& lang tests. > The test results are looking real good. Compared with the > jigsaw repo as the baseline, there are 2 new JCK test failures > that I have to investigate (passed: 23,685; failed: 138; error: 28) > > There is a startup initialization issue due to the use of > Files.isSameFile method and the default file system provider > is replaced with user defined one. I'm currently looking into > a fix for it. > > I'd like to get this webrev out for review early and I also > expect any issue found will be minor that I will generate a > separate patch on top of this as a separate review. > > Open Issues: > 1. sun.boot.class.path no longer represents the search path > for the bootstrap class loader. Any application depending > on this system property may be impacted. > > rmic is one example that uses this property value to find > system classes. There has been work to convert rmic to > use the javax.tool APIs that will support a module image. > Unless rmic becomes module-aware, it continues to use > the launcher hack setting -Xbootclasspath/p as a workaround. > > Mandy > From Alan.Bateman at oracle.com Wed Aug 22 07:04:37 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 22 Aug 2012 15:04:37 +0100 Subject: Jigsaw classpath mode support In-Reply-To: <5033C0D6.5080409@oracle.com> References: <502154B7.7080804@oracle.com> <5033C0D6.5080409@oracle.com> Message-ID: <5034E6F5.8060406@oracle.com> On 21/08/2012 18:09, Mandy Chung wrote: > Updated webrev at: > http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/classpath-mode.01/ > > I have modified org.openjdk.jigsaw.ClassPathContext and > sun.misc.Launcher classes to incorporate the comments Alan, Paul and > Karen have. Also updated the ModulesProblemList.txt to include a few > known failures that will be looked into next. > > Mandy > Thanks for the updates, I think you should just push this once the VM changes are ready to go. -Alan From karen.kinnear at oracle.com Thu Aug 23 13:14:28 2012 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Thu, 23 Aug 2012 16:14:28 -0400 Subject: Please review: VM side of Jigsaw classpath mode support In-Reply-To: <5033C0D6.5080409@oracle.com> References: <502154B7.7080804@oracle.com> <5033C0D6.5080409@oracle.com> Message-ID: The VM side of this webrev is at: http://cr.openjdk.java.net/~acorn/vm.classpath/webrev/ Note that this has to go back at the same time as the jdk changes, so if you pick up this patch you need the patch Mandy sent out below. thanks, Karen On Aug 21, 2012, at 1:09 PM, Mandy Chung wrote: > Updated webrev at: > http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/classpath-mode.01/ > > I have modified org.openjdk.jigsaw.ClassPathContext and sun.misc.Launcher classes to incorporate the comments Alan, Paul and Karen have. Also updated the ModulesProblemList.txt to include a few known failures that will be looked into next. > > Mandy > > On 8/7/2012 10:47 AM, Mandy Chung wrote: >> I have implemented the "classpath mode" support running on >> a module image. >> >> Webrev: >> http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/classpath-mode.00/ >> >> JDK has been modularized in the jigsaw repo. There no longer >> has a rt.jar, tools.jar, or any jar file. The system classes >> are now installed in a system module library as modules. This >> change will allow the bootstrap class loader, extension class >> loader, and system class loader to search classes and resources >> in the system module library as the default. Some implementation >> details: >> >> 1. VM used to maintain the default bootclasspath list and search >> classes in it. Now it will call jigsaw native interface to find >> classes from the module library unless -Xbootclasspath is set >> to override the default bootclasspath. The -Xbootclasspath/p >> and -Xbootclasspath/a will continue to work as it is today. >> >> Karen will be sending out the webrev for the hotspot change >> separately. >> >> 2. tools.jar >> >> For compatibility, -classpath tools.jar should continue >> to work. There are existing applications using URLClassLoader >> that loads classes from tools.jar (e.g. test harness) that >> also should continue to work. >> >> To be able to determine if the path of the given tools.jar >> to either -classpath option or URL parameter to URLClassLoader >> is the same path as JAVA_HOME/lib/tools.jar in a lightweight, >> the module image for the full JDK will contain an empty tools.jar. >> This also allows applications that check for the existence of >> tools.jar to continue to work. >> >> sun.misc.URLClassPath is the main implementation class to >> support URLClassLoader to search a file from the given URLs. >> Both ExtClassLoader and AppClassLoader are URLClassLoader. >> URLClassPath is extended to find resource file from the >> appropriate system modules if the given URL is >> JAVA_HOME/lib/tools.jar or JAVA_HOME/lib/ext. >> >> 3. How to determine which modules belong to bootclasspath, >> ext directory, and tools.jar (and a few jar files in JDK/lib) >> at runtime >> >> A configuration for the classpath mode support is loaded at >> startup rather than generated to minimize the startup cost. >> The idea is to have one ClassPathContext for bootclasspath, >> one for extension and one for the jdk tools. >> >> In this prototype, I just define a new module "jdk.classpath" >> and install it in the module image. It hardcodes the list >> of modules for each ClassPathContext for now. One alternative >> is to extend jpkg to generate "jdk.classpath" config specifically >> for this classpath mode. We'll explore this later. But I >> think this is good enough for now. >> >> 4. I made simple change in StoredConfiguration to reduce >> the footprint of the configuration stored on disk. There >> were several regression tests setting -Xmx failed due to >> the increase of the startup footprint without this change. >> >> With storing the context names and package names only once, >> it reduces the size of the config with 35-40%. I suspect >> the fast configuration will make similar change. >> >> $ du -k `find . -name 'config'` >> 696 ./jdk.corba/8-ea/config >> 1304 ./jdk.classpath/8-ea/config >> 1232 ./jdk.tools/8-ea/config >> 604 ./jdk.rmi/8-ea/config >> 680 ./jdk.tools.base/8-ea/config >> 604 ./jdk.base/8-ea/config >> 1056 ./jdk.tools.jaxws/8-ea/config >> 676 ./jdk.devtools/8-ea/config >> 608 ./jdk.tools.jre/8-ea/config >> >> $ du -k `find . -name 'config'` >> 444 ./jdk.corba/8-ea/config >> 792 ./jdk.classpath/8-ea/config >> 748 ./jdk.tools/8-ea/config >> 392 ./jdk.rmi/8-ea/config >> 436 ./jdk.tools.base/8-ea/config >> 392 ./jdk.base/8-ea/config >> 636 ./jdk.tools.jaxws/8-ea/config >> 436 ./jdk.devtools/8-ea/config >> 396 ./jdk.tools.jre/8-ea/config >> >> >> 5. I ran the jtreg regression tests and JCK api& lang tests. >> The test results are looking real good. Compared with the >> jigsaw repo as the baseline, there are 2 new JCK test failures >> that I have to investigate (passed: 23,685; failed: 138; error: 28) >> >> There is a startup initialization issue due to the use of >> Files.isSameFile method and the default file system provider >> is replaced with user defined one. I'm currently looking into >> a fix for it. >> >> I'd like to get this webrev out for review early and I also >> expect any issue found will be minor that I will generate a >> separate patch on top of this as a separate review. >> >> Open Issues: >> 1. sun.boot.class.path no longer represents the search path >> for the bootstrap class loader. Any application depending >> on this system property may be impacted. >> >> rmic is one example that uses this property value to find >> system classes. There has been work to convert rmic to >> use the javax.tool APIs that will support a module image. >> Unless rmic becomes module-aware, it continues to use >> the launcher hack setting -Xbootclasspath/p as a workaround. >> >> Mandy >> From mark.reinhold at oracle.com Thu Aug 23 15:27:16 2012 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Thu, 23 Aug 2012 22:27:16 +0000 Subject: hg: jigsaw/jigsaw/jdk: 2 new changesets Message-ID: <20120823222750.33101476BD@hg.openjdk.java.net> Changeset: 19af224f7f08 Author: mr Date: 2012-08-23 15:24 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/19af224f7f08 Remove carriage returns from ModuleFileParserImpl.java ! src/share/classes/org/openjdk/jigsaw/ModuleFileParserImpl.java Changeset: fb3cd8b4b19f Author: mr Date: 2012-08-23 15:24 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/fb3cd8b4b19f Fill in some obvious missing doc comments, and fix up others ! src/share/classes/org/openjdk/jigsaw/ClassInfo.java ! src/share/classes/org/openjdk/jigsaw/Configuration.java ! src/share/classes/org/openjdk/jigsaw/ConfigurationException.java ! src/share/classes/org/openjdk/jigsaw/FileConstants.java ! src/share/classes/org/openjdk/jigsaw/Files.java ! src/share/classes/org/openjdk/jigsaw/Hi.java ! src/share/classes/org/openjdk/jigsaw/JigsawModuleSystem.java ! src/share/classes/org/openjdk/jigsaw/Launcher.java ! src/share/classes/org/openjdk/jigsaw/Library.java ! src/share/classes/org/openjdk/jigsaw/Loader.java ! src/share/classes/org/openjdk/jigsaw/LoaderPool.java ! src/share/classes/org/openjdk/jigsaw/LocatableCatalog.java ! src/share/classes/org/openjdk/jigsaw/Manifest.java ! src/share/classes/org/openjdk/jigsaw/ModuleFile.java ! src/share/classes/org/openjdk/jigsaw/ModuleFileParser.java ! src/share/classes/org/openjdk/jigsaw/ModuleFileParserDelegate.java ! src/share/classes/org/openjdk/jigsaw/ModuleFileParserException.java ! src/share/classes/org/openjdk/jigsaw/ModuleFileWriter.java ! src/share/classes/org/openjdk/jigsaw/PersistentTreeMap.java ! src/share/classes/org/openjdk/jigsaw/Platform.java ! src/share/classes/org/openjdk/jigsaw/RemoteRepositoryList.java ! src/share/classes/org/openjdk/jigsaw/Repository.java ! src/share/classes/org/openjdk/jigsaw/RepositoryCatalog.java ! src/share/classes/org/openjdk/jigsaw/Resolution.java ! src/share/classes/org/openjdk/jigsaw/SignedModule.java ! src/share/classes/org/openjdk/jigsaw/SimpleLibrary.java ! src/share/classes/org/openjdk/jigsaw/ValidatingModuleFileParser.java From mark.reinhold at oracle.com Thu Aug 23 15:30:13 2012 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Thu, 23 Aug 2012 15:30:13 -0700 Subject: Javadoc update Message-ID: <20120823223013.443C0DBE@eggemoggin.niobe.net> After filling in a bunch of missing doc comments, and editing some existing ones, I've pushed the changes and updated the published API: http://cr.openjdk.java.net/~mr/jigsaw/api/ Nothing dramatic here, just some progress. More to come. - Mark From david.holmes at oracle.com Thu Aug 23 17:47:00 2012 From: david.holmes at oracle.com (David Holmes) Date: Fri, 24 Aug 2012 10:47:00 +1000 Subject: Please review: VM side of Jigsaw classpath mode support In-Reply-To: References: <502154B7.7080804@oracle.com> <5033C0D6.5080409@oracle.com> Message-ID: <5036CF04.2060701@oracle.com> Hi Karen, On 24/08/2012 6:14 AM, Karen Kinnear wrote: > The VM side of this webrev is at: > http://cr.openjdk.java.net/~acorn/vm.classpath/webrev/ I gave this a quick look over for curiosity. I didn't see anything obviously "strange". One query though. This path format: "%/lib/modules/jdk.base"; is used to detect a modular runtime but I thought that the actual names of the directories used for a module library were configurable when jmod is used to create it? David ----- > Note that this has to go back at the same time as the jdk changes, so if you pick up this patch you need the > patch Mandy sent out below. > > thanks, > Karen > > > > On Aug 21, 2012, at 1:09 PM, Mandy Chung wrote: > >> Updated webrev at: >> http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/classpath-mode.01/ >> >> I have modified org.openjdk.jigsaw.ClassPathContext and sun.misc.Launcher classes to incorporate the comments Alan, Paul and Karen have. Also updated the ModulesProblemList.txt to include a few known failures that will be looked into next. >> >> Mandy >> >> On 8/7/2012 10:47 AM, Mandy Chung wrote: >>> I have implemented the "classpath mode" support running on >>> a module image. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/classpath-mode.00/ >>> >>> JDK has been modularized in the jigsaw repo. There no longer >>> has a rt.jar, tools.jar, or any jar file. The system classes >>> are now installed in a system module library as modules. This >>> change will allow the bootstrap class loader, extension class >>> loader, and system class loader to search classes and resources >>> in the system module library as the default. Some implementation >>> details: >>> >>> 1. VM used to maintain the default bootclasspath list and search >>> classes in it. Now it will call jigsaw native interface to find >>> classes from the module library unless -Xbootclasspath is set >>> to override the default bootclasspath. The -Xbootclasspath/p >>> and -Xbootclasspath/a will continue to work as it is today. >>> >>> Karen will be sending out the webrev for the hotspot change >>> separately. >>> >>> 2. tools.jar >>> >>> For compatibility, -classpath tools.jar should continue >>> to work. There are existing applications using URLClassLoader >>> that loads classes from tools.jar (e.g. test harness) that >>> also should continue to work. >>> >>> To be able to determine if the path of the given tools.jar >>> to either -classpath option or URL parameter to URLClassLoader >>> is the same path as JAVA_HOME/lib/tools.jar in a lightweight, >>> the module image for the full JDK will contain an empty tools.jar. >>> This also allows applications that check for the existence of >>> tools.jar to continue to work. >>> >>> sun.misc.URLClassPath is the main implementation class to >>> support URLClassLoader to search a file from the given URLs. >>> Both ExtClassLoader and AppClassLoader are URLClassLoader. >>> URLClassPath is extended to find resource file from the >>> appropriate system modules if the given URL is >>> JAVA_HOME/lib/tools.jar or JAVA_HOME/lib/ext. >>> >>> 3. How to determine which modules belong to bootclasspath, >>> ext directory, and tools.jar (and a few jar files in JDK/lib) >>> at runtime >>> >>> A configuration for the classpath mode support is loaded at >>> startup rather than generated to minimize the startup cost. >>> The idea is to have one ClassPathContext for bootclasspath, >>> one for extension and one for the jdk tools. >>> >>> In this prototype, I just define a new module "jdk.classpath" >>> and install it in the module image. It hardcodes the list >>> of modules for each ClassPathContext for now. One alternative >>> is to extend jpkg to generate "jdk.classpath" config specifically >>> for this classpath mode. We'll explore this later. But I >>> think this is good enough for now. >>> >>> 4. I made simple change in StoredConfiguration to reduce >>> the footprint of the configuration stored on disk. There >>> were several regression tests setting -Xmx failed due to >>> the increase of the startup footprint without this change. >>> >>> With storing the context names and package names only once, >>> it reduces the size of the config with 35-40%. I suspect >>> the fast configuration will make similar change. >>> >>> $ du -k `find . -name 'config'` >>> 696 ./jdk.corba/8-ea/config >>> 1304 ./jdk.classpath/8-ea/config >>> 1232 ./jdk.tools/8-ea/config >>> 604 ./jdk.rmi/8-ea/config >>> 680 ./jdk.tools.base/8-ea/config >>> 604 ./jdk.base/8-ea/config >>> 1056 ./jdk.tools.jaxws/8-ea/config >>> 676 ./jdk.devtools/8-ea/config >>> 608 ./jdk.tools.jre/8-ea/config >>> >>> $ du -k `find . -name 'config'` >>> 444 ./jdk.corba/8-ea/config >>> 792 ./jdk.classpath/8-ea/config >>> 748 ./jdk.tools/8-ea/config >>> 392 ./jdk.rmi/8-ea/config >>> 436 ./jdk.tools.base/8-ea/config >>> 392 ./jdk.base/8-ea/config >>> 636 ./jdk.tools.jaxws/8-ea/config >>> 436 ./jdk.devtools/8-ea/config >>> 396 ./jdk.tools.jre/8-ea/config >>> >>> >>> 5. I ran the jtreg regression tests and JCK api& lang tests. >>> The test results are looking real good. Compared with the >>> jigsaw repo as the baseline, there are 2 new JCK test failures >>> that I have to investigate (passed: 23,685; failed: 138; error: 28) >>> >>> There is a startup initialization issue due to the use of >>> Files.isSameFile method and the default file system provider >>> is replaced with user defined one. I'm currently looking into >>> a fix for it. >>> >>> I'd like to get this webrev out for review early and I also >>> expect any issue found will be minor that I will generate a >>> separate patch on top of this as a separate review. >>> >>> Open Issues: >>> 1. sun.boot.class.path no longer represents the search path >>> for the bootstrap class loader. Any application depending >>> on this system property may be impacted. >>> >>> rmic is one example that uses this property value to find >>> system classes. There has been work to convert rmic to >>> use the javax.tool APIs that will support a module image. >>> Unless rmic becomes module-aware, it continues to use >>> the launcher hack setting -Xbootclasspath/p as a workaround. >>> >>> Mandy >>> > From paul.sandoz at oracle.com Fri Aug 24 03:29:43 2012 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 24 Aug 2012 12:29:43 +0200 Subject: Please review: VM side of Jigsaw classpath mode support In-Reply-To: <5036CF04.2060701@oracle.com> References: <502154B7.7080804@oracle.com> <5033C0D6.5080409@oracle.com> <5036CF04.2060701@oracle.com> Message-ID: Hi David, On Aug 24, 2012, at 2:47 AM, David Holmes wrote: > Hi Karen, > > On 24/08/2012 6:14 AM, Karen Kinnear wrote: >> The VM side of this webrev is at: >> http://cr.openjdk.java.net/~acorn/vm.classpath/webrev/ > > I gave this a quick look over for curiosity. I didn't see anything obviously "strange". > > One query though. This path format: > > "%/lib/modules/jdk.base"; > > is used to detect a modular runtime but I thought that the actual names of the directories used for a module library were configurable when jmod is used to create it? > I presume it is assuming there is an installation of a Java runtime, which i guess there has to be, where "%/lib/modules" is the location of the module library of that installation, and the directory underneath, "jdk.base", is an internal detail to the module library implementation. Paul. From david.holmes at oracle.com Fri Aug 24 04:29:49 2012 From: david.holmes at oracle.com (David Holmes) Date: Fri, 24 Aug 2012 21:29:49 +1000 Subject: Please review: VM side of Jigsaw classpath mode support In-Reply-To: References: <502154B7.7080804@oracle.com> <5033C0D6.5080409@oracle.com> <5036CF04.2060701@oracle.com> Message-ID: <503765AD.3090205@oracle.com> Hi Paul, On 24/08/2012 8:29 PM, Paul Sandoz wrote: > On Aug 24, 2012, at 2:47 AM, David Holmes wrote: >> On 24/08/2012 6:14 AM, Karen Kinnear wrote: >>> The VM side of this webrev is at: >>> http://cr.openjdk.java.net/~acorn/vm.classpath/webrev/ >> >> I gave this a quick look over for curiosity. I didn't see anything obviously "strange". >> >> One query though. This path format: >> >> "%/lib/modules/jdk.base"; >> >> is used to detect a modular runtime but I thought that the actual names of the directories used for a module library were configurable when jmod is used to create it? >> > > I presume it is assuming there is an installation of a Java runtime, which i guess there has to be, where "%/lib/modules" is the location of the module library of that installation, and the directory underneath, "jdk.base", is an internal detail to the module library implementation. Not quite an internal detail. I can do: jmod create -L myjre jmod -L myjre install jdk.base at 8-ea.jmod to get a JRE - albeit non-functional. If there is a required layout then the tools need to enforce that. David From Alan.Bateman at oracle.com Fri Aug 24 04:48:04 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 24 Aug 2012 12:48:04 +0100 Subject: Please review: VM side of Jigsaw classpath mode support In-Reply-To: <503765AD.3090205@oracle.com> References: <502154B7.7080804@oracle.com> <5033C0D6.5080409@oracle.com> <5036CF04.2060701@oracle.com> <503765AD.3090205@oracle.com> Message-ID: <503769F4.6080906@oracle.com> On 24/08/2012 12:29, David Holmes wrote: > > Not quite an internal detail. I can do: > > jmod create -L myjre > jmod -L myjre install jdk.base at 8-ea.jmod > > to get a JRE - albeit non-functional. If there is a required layout > then the tools need to enforce that. > > David The system module library is special and needs to be created in a way that takes account that it part of an image. For compatibility (and usability reasons) this means that the commands (javac, java, etc.) need to be in the image's bin directory rather than a module-private location. Native libraries are complicated by dependencies and also the need to have libjvm in a known location so that custom launchers can dlopen it. So yes, definitely not functional if you do the above. In any case, I think the VM has to special case the base module and has to have code to detect that it's running with module image. -Alan. From karen.kinnear at oracle.com Fri Aug 24 06:07:08 2012 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Fri, 24 Aug 2012 09:07:08 -0400 Subject: Please review: VM side of Jigsaw classpath mode support In-Reply-To: <503769F4.6080906@oracle.com> References: <502154B7.7080804@oracle.com> <5033C0D6.5080409@oracle.com> <5036CF04.2060701@oracle.com> <503765AD.3090205@oracle.com> <503769F4.6080906@oracle.com> Message-ID: David, Thank you for looking at this. The jvm is trying to detect whether it is running with rt.jar or a module image - so it is looking for jdk.base relative to where the jre is. Given we are looking at multiple profiles including some minimal ones, it is not yet clear what other modules would be required for the system module library. I could see value in tools checking for profile completeness. thanks, Karen On Aug 24, 2012, at 7:48 AM, Alan Bateman wrote: > On 24/08/2012 12:29, David Holmes wrote: >> >> Not quite an internal detail. I can do: >> >> jmod create -L myjre >> jmod -L myjre install jdk.base at 8-ea.jmod >> >> to get a JRE - albeit non-functional. If there is a required layout then the tools need to enforce that. >> >> David > The system module library is special and needs to be created in a way that takes account that it part of an image. For compatibility (and usability reasons) this means that the commands (javac, java, etc.) need to be in the image's bin directory rather than a module-private location. Native libraries are complicated by dependencies and also the need to have libjvm in a known location so that custom launchers can dlopen it. So yes, definitely not functional if you do the above. In any case, I think the VM has to special case the base module and has to have code to detect that it's running with module image. > > -Alan. From mandy.chung at oracle.com Fri Aug 24 13:59:19 2012 From: mandy.chung at oracle.com (Mandy Chung) Date: Fri, 24 Aug 2012 13:59:19 -0700 Subject: Please review: VM side of Jigsaw classpath mode support In-Reply-To: References: <502154B7.7080804@oracle.com> <5033C0D6.5080409@oracle.com> Message-ID: <5037EB27.2080509@oracle.com> On 8/23/2012 1:14 PM, Karen Kinnear wrote: > The VM side of this webrev is at: > http://cr.openjdk.java.net/~acorn/vm.classpath/webrev/ Looks good, Karen. I should take this opportunity to describe the new properties VM sets for JDK to use but they are removed from System properties at startup time sun.boot.class.prepend.path - paths set by -Xbootclasspath/p sun.boot.class.append.path - paths set by -Xbootclasspath/a and added by JVMTI or java.lang.instrument. The above 2 are used for searching system resources. It used to search system resources from "sun.boot.class.path" but in a modular JDK, we need to separate the prepend and append paths. We take this simple approach to pass these 2 properties to the JDK properties since the VM already has the separated search paths. sun.boot.module.classpath - Class.getPackage() calls JVM_GetSystemPackage to obtain the source where a system class is loaded (it may be a JAR file or a directory). If a system class is loaded from a module image in classpath mode, it can be loaded from any module for the runtime. The current VM implementation only supports an entry to the bootclasspath be appended. In addition, there is no manifest for modules anyway. All we need is to tell the JDK that this system class is loaded from modules; then it will construct a Package instance based on the system properties. So sun.boot.module.classpath serves as a unique tag for JDK to compare with the returned value from JVM_GetSystemPackage. I have a few minor comments and you can be addressed in future revisions. There will be another update for modulepath support next. In os.cpp, does it need to reset UseModuleNativeLibs if there is no base module? You already check for the flag and has_module_image(): if (UseModuleNativeLibs&& Arguments::has_module_image()) os.cpp L1140-1142 - I thought this is no longer needed and can be removed. classLoader.hpp looks like "_boot_module_source" is not used. BTW, should Arguments::boot_module_index() belong to classLoader.hpp? It's an open issue what "sun.boot.class.path" should be. I believe your current patch includes the legacy rt.jar and other paths in sun.boot.class.path because you want the VM to be able to run on a legacy image (e.g. JDK8). Perhaps you can detect from JDK_Version if it's a jigsaw build, not to include the legacy paths in sun.boot.class.path. I have been considering adding some interface to indicate it's a modular image (since the JDK release is still 8 in our build). I can add a bit in the jdk_version_info in our next revision. Thanks Mandy From karen.kinnear at oracle.com Fri Aug 24 14:31:36 2012 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Fri, 24 Aug 2012 17:31:36 -0400 Subject: Please review: VM side of Jigsaw classpath mode support In-Reply-To: <5037EB27.2080509@oracle.com> References: <502154B7.7080804@oracle.com> <5033C0D6.5080409@oracle.com> <5037EB27.2080509@oracle.com> Message-ID: Mandy, Thank you for the review and for describing the internal system properties as used so far. On Aug 24, 2012, at 4:59 PM, Mandy Chung wrote: > On 8/23/2012 1:14 PM, Karen Kinnear wrote: >> The VM side of this webrev is at: >> http://cr.openjdk.java.net/~acorn/vm.classpath/webrev/ > Looks good, Karen. I should take this opportunity to describe > the new properties VM sets for JDK to use but they are removed > from System properties at startup time > > sun.boot.class.prepend.path - paths set by -Xbootclasspath/p > sun.boot.class.append.path - paths set by -Xbootclasspath/a and > added by JVMTI or java.lang.instrument. > The above 2 are used for searching system resources. > It used to search system resources from "sun.boot.class.path" > but in a modular JDK, we need to separate the prepend and > append paths. We take this simple approach to pass these 2 > properties to the JDK properties since the VM already has > the separated search paths. > > sun.boot.module.classpath - Class.getPackage() calls JVM_GetSystemPackage > to obtain the source where a system class is loaded (it may > be a JAR file or a directory). If a system class is loaded > from a module image in classpath mode, it can be loaded from > any module for the runtime. The current VM implementation > only supports an entry to the bootclasspath be appended. > In addition, there is no manifest for modules anyway. All > we need is to tell the JDK that this system class is loaded > from modules; then it will construct a Package instance > based on the system properties. So sun.boot.module.classpath > serves as a unique tag for JDK to compare with the returned > value from JVM_GetSystemPackage. > > I have a few minor comments and you can be addressed in > future revisions. There will be another update for modulepath > support next. > > In os.cpp, does it need to reset UseModuleNativeLibs if there > is no base module? > You already check for the flag and > has_module_image(): > if (UseModuleNativeLibs&& Arguments::has_module_image()) > > os.cpp L1140-1142 - I thought this is no longer needed and can be > removed. Due to the incremental bootstrapping steps, I would like to revisit the exact order in which each of these decisions is made and try to make this a bit cleaner. I would like to do that in the context of the upcoming modulepath changes since that may modify some of this approach. So I will note your suggestions and revisit them. > > classLoader.hpp looks like "_boot_module_source" is not used. Good catch. Thank you. > BTW, should Arguments::boot_module_index() belong to classLoader.hpp? I am anticipating that that is temporary and may change due to modulepath support to support a list of modules. I expect that will need to be shared between the launcher, jvm and jdk in some form. > > It's an open issue what "sun.boot.class.path" should be. > I believe your current patch includes the legacy rt.jar and other > paths in sun.boot.class.path because you want the VM to be able > to run on a legacy image (e.g. JDK8). Perhaps you can detect from > JDK_Version if it's a jigsaw build, not to include the legacy paths > in sun.boot.class.path. I have been considering adding some interface > to indicate it's a modular image (since the JDK release is still 8 > in our build). I can add a bit in the jdk_version_info in our next > revision. Yes, the jvm needs to be able to run with rt.jar or with the module image for now. We can discuss possible approaches. > > Thanks > Mandy thanks, Karen > > From mark.reinhold at oracle.com Fri Aug 24 15:23:57 2012 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Fri, 24 Aug 2012 15:23:57 -0700 Subject: Service provider module resolution Re: Services/configuration/context webrevs In-Reply-To: paul.sandoz@oracle.com; Wed, 01 Aug 2012 07:46:19 PDT; <014C6BF2-1990-4AC6-8E8B-F5CD37864F55@oracle.com> Message-ID: <20120824222357.ECF7E2A53@eggemoggin.niobe.net> 2012/8/1 7:46 -0700, paul.sandoz at oracle.com: > http://cr.openjdk.java.net/~psandoz/jigsaw/resolver-services/webrev.4/ Overall this looks very good, and the resulting behavior will definitely be more intuitive. Resolver.java [84-94] Please follow the local convention of using "//" to introduce comments on implementation internals. This makes it easy to distinguish them from actual doc comments. [88] s/synthesizedServiceDependencies/synthesizedServiceDependences/ Dependencies and dependences are different things. You're talking about the latter here. [432-444] // [445-473] dependencies -> dependences [455,460] s/: / : / [514-556] This code is crying out to be placed in its own method, say `resolveServices`. As to the tests, I'm not comfortable with splitting what's logically a single test, which happens to involve a bunch of small files, into a bunch of actual small files. That just makes the test harder to read and harder to maintain. I agree that the current shell hackery has issues, particularly around portability, but we can address that in other ways. Please use existing mechanisms and conventions for now, until we have something better. (One idea (just discussed with Jon and Mandy here in the hallway) is to write a test library that understands a simple "snippet" file format. When run it would extract each snippet into a separate source file, possibly doing some template expansion along the way, and then perform the requested actions on them. This is, however, for another day.) - Mark From david.holmes at oracle.com Fri Aug 24 15:53:06 2012 From: david.holmes at oracle.com (David Holmes) Date: Sat, 25 Aug 2012 08:53:06 +1000 Subject: Please review: VM side of Jigsaw classpath mode support In-Reply-To: <503769F4.6080906@oracle.com> References: <502154B7.7080804@oracle.com> <5033C0D6.5080409@oracle.com> <5036CF04.2060701@oracle.com> <503765AD.3090205@oracle.com> <503769F4.6080906@oracle.com> Message-ID: <503805D2.7090205@oracle.com> On 24/08/2012 9:48 PM, Alan Bateman wrote: > On 24/08/2012 12:29, David Holmes wrote: >> >> Not quite an internal detail. I can do: >> >> jmod create -L myjre >> jmod -L myjre install jdk.base at 8-ea.jmod >> >> to get a JRE - albeit non-functional. If there is a required layout >> then the tools need to enforce that. >> >> David > The system module library is special and needs to be created in a way > that takes account that it part of an image. For compatibility (and > usability reasons) this means that the commands (javac, java, etc.) need > to be in the image's bin directory rather than a module-private > location. Native libraries are complicated by dependencies and also the > need to have libjvm in a known location so that custom launchers can > dlopen it. So yes, definitely not functional if you do the above. In any > case, I think the VM has to special case the base module and has to have > code to detect that it's running with module image. But I think we have a disconnect on what you are permitted to do via jmod and what is required for a functional JRE. Using --natcmd and --natlib and "jmod create" you can end up with the JRE binaries & libraries in a completely different location to everything actually associated with the base module. At a minimum this required layout must be documented. But preferably jmod would enforce it. David > -Alan. From Alan.Bateman at oracle.com Sat Aug 25 04:31:46 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 25 Aug 2012 12:31:46 +0100 Subject: Please review: VM side of Jigsaw classpath mode support In-Reply-To: <503805D2.7090205@oracle.com> References: <502154B7.7080804@oracle.com> <5033C0D6.5080409@oracle.com> <5036CF04.2060701@oracle.com> <503765AD.3090205@oracle.com> <503769F4.6080906@oracle.com> <503805D2.7090205@oracle.com> Message-ID: <5038B7A2.4090907@oracle.com> On 24/08/2012 23:53, David Holmes wrote: > > But I think we have a disconnect on what you are permitted to do via > jmod and what is required for a functional JRE. Using --natcmd and > --natlib and "jmod create" you can end up with the JRE binaries & > libraries in a completely different location to everything actually > associated with the base module. > > At a minimum this required layout must be documented. But preferably > jmod would enforce it. > > David I have a write-up of the modules image structure that I should probably add to the project page. Note that this does not cover the layout of the module library as that it implementation specific and will likely change significantly over time. In any case, the system module is mostly created by the build and it creates this in the image with appropriate options so that the commands and other files are in the expected locations. As I mentioned in the previous mail we have some constraints and can't be too radical in changing the image structure. So yes, this means the jmod command in the build needs to get the options right. Same thing for other tools that that create JRE images. In any case, I don't see this as a concern for most developers, at least not unless they are creating custom runtimes that don't have the base image as their starting point. -Alan From paul.sandoz at oracle.com Mon Aug 27 09:19:57 2012 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 27 Aug 2012 18:19:57 +0200 Subject: Service provider module resolution Re: Services/configuration/context webrevs In-Reply-To: <20120824222357.ECF7E2A53@eggemoggin.niobe.net> References: <20120824222357.ECF7E2A53@eggemoggin.niobe.net> Message-ID: <75C2580F-E937-4BB8-B5AC-8C00DB62B272@oracle.com> On Aug 25, 2012, at 12:23 AM, mark.reinhold at oracle.com wrote: > 2012/8/1 7:46 -0700, paul.sandoz at oracle.com: >> http://cr.openjdk.java.net/~psandoz/jigsaw/resolver-services/webrev.4/ > > Overall this looks very good, and the resulting behavior will definitely > be more intuitive. > Thanks. I will send another webrev tomorrow. > Resolver.java > > [84-94] Please follow the local convention of using "//" to introduce > comments on implementation internals. This makes it easy to > distinguish them from actual doc comments. > > [88] s/synthesizedServiceDependencies/synthesizedServiceDependences/ > Dependencies and dependences are different things. You're talking > about the latter here. > > [432-444] // > > [445-473] dependencies -> dependences > > [455,460] s/: / : / > > [514-556] This code is crying out to be placed in its own method, say > `resolveServices`. > OK. > As to the tests, I'm not comfortable with splitting what's logically a > single test, which happens to involve a bunch of small files, into a > bunch of actual small files. That just makes the test harder to read and > harder to maintain. Some intents: - make it easier to edit the tests in an IDE. I actually have the service runtime loaded as a project in NetBeans. From that perspective I find it much easier to read and maintain. - add other runtime-based tests to this project e.g. i wanted to add one that iterated when there was an exception on construction of a service provider class, but this does require more service interface/provider modules which really should be derived from a template, like those other ones. - i marked the this out as one to update to using TestNG, for example i could define one test that uses a data provider supplying the 5 class loader values. > I agree that the current shell hackery has issues, > particularly around portability, but we can address that in other ways. > Please use existing mechanisms and conventions for now, until we have > something better. > OK, i will sully myself with the shell hackery :-) > (One idea (just discussed with Jon and Mandy here in the hallway) is to > write a test library that understands a simple "snippet" file format. > When run it would extract each snippet into a separate source file, > possibly doing some template expansion along the way, and then perform > the requested actions on them. This is, however, for another day.) > Thoughts: - perhaps we could use annotation processors to generate modules/code. - there are some compiler tricks performed by the lambda TestNG suite, dunno if they are relevant. - it would be nice if the code that performed the tests/assertions could be edited in an IDE. For example, we may be able to declare from a unified source base how it is split into modules e.g. the test module, a service interface module, and generated service provider modules from templates. Paul. From paul.sandoz at oracle.com Tue Aug 28 03:35:16 2012 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 28 Aug 2012 12:35:16 +0200 Subject: Service provider module resolution Re: Services/configuration/context webrevs In-Reply-To: <75C2580F-E937-4BB8-B5AC-8C00DB62B272@oracle.com> References: <20120824222357.ECF7E2A53@eggemoggin.niobe.net> <75C2580F-E937-4BB8-B5AC-8C00DB62B272@oracle.com> Message-ID: <9C87ACFC-F891-483F-BCD5-6A0D84D4230A@oracle.com> Hi, On Aug 27, 2012, at 6:19 PM, Paul Sandoz wrote: > > On Aug 25, 2012, at 12:23 AM, mark.reinhold at oracle.com wrote: > >> 2012/8/1 7:46 -0700, paul.sandoz at oracle.com: >>> http://cr.openjdk.java.net/~psandoz/jigsaw/resolver-services/webrev.4/ >> >> Overall this looks very good, and the resulting behavior will definitely >> be more intuitive. >> > > Thanks. I will send another webrev tomorrow. > http://cr.openjdk.java.net/~psandoz/jigsaw/resolver-services/webrev.5/ Paul. From chris.hegarty at oracle.com Tue Aug 28 05:55:22 2012 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 28 Aug 2012 13:55:22 +0100 Subject: Update for modules Problem List Message-ID: <503CBFBA.5000908@oracle.com> Here is a small update to the ModulesProblemList.txt. The failing tests are mostly related to the Solaris 64 bit layout/launcher issue, this will be addressed separately. I just intent to add these additional tests to similar ones already excluded. Then there is a jigsaw specific test, hello-native. I actually like this test, but it depends on native code and a compile environment. For now, we should just exclude it and defer making a decision on it's future. : hg diff test/ModulesProblemList.txt diff -r fb3cd8b4b19f test/ModulesProblemList.txt --- a/test/ModulesProblemList.txt Thu Aug 23 15:24:45 2012 -0700 +++ b/test/ModulesProblemList.txt Tue Aug 28 13:49:38 2012 +0100 @@ -121,6 +121,10 @@ java/nio/channels/SocketChannel/AdaptSoc java/nio/channels/SocketChannel/AdaptSocket.java linux-x64 java/nio/channels/SocketChannel/Connect.java linux-x64 java/nio/MappedByteBuffer/Force.java windows-i586 + +# Tests rely on bin/sparcv9|amd64/java launcher +java/nio/channels/spi/SelectorProvider/inheritedChannel/run_tests.sh solaris-sparcv9 +java/nio/channels/spi/SelectorProvider/inheritedChannel/run_tests.sh solaris-x64 # This test sets to Turkish language without resetting it when exit @@ -249,12 +253,16 @@ com/sun/jdi/DoubleAgentTest.java com/sun/jdi/DoubleAgentTest.java solaris-x64 tools/launcher/Test7029048.java solaris-x64 sun/tools/jhat/HatHeapDump1Test.java solaris-x64 +com/sun/jdi/PrivateTransportTest.sh solaris-x64 +com/sun/jdi/Solaris32AndSolaris64Test.sh solaris-x64 com/sun/jdi/BadHandshakeTest.java solaris-sparcv9 com/sun/jdi/ExclusiveBind.java solaris-sparcv9 com/sun/jdi/RunToExit.java solaris-sparcv9 com/sun/jdi/DoubleAgentTest.java solaris-sparcv9 tools/launcher/Test7029048.java solaris-sparcv9 sun/tools/jhat/HatHeapDump1Test.java solaris-sparcv9 +com/sun/jdi/PrivateTransportTest.sh solaris-sparcv9 +com/sun/jdi/Solaris32AndSolaris64Test.sh solaris-sparcv9 # Test expects tools.jar and rt.jar. Needs to be update. tools/pack200/Pack200Simple.sh generic-all @@ -299,6 +307,12 @@ java/util/concurrent/locks/Lock/TimedAcq ############################################################################ +# jdk_jigsaw + +org/openjdk/jigsaw/hello-native.sh generic-all + +############################################################################ + # langtools_jtreg known failures - included it here for reference # these tests are not excluded in the langtools jtreg run From Alan.Bateman at oracle.com Tue Aug 28 06:17:40 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 28 Aug 2012 14:17:40 +0100 Subject: Update for modules Problem List In-Reply-To: <503CBFBA.5000908@oracle.com> References: <503CBFBA.5000908@oracle.com> Message-ID: <503CC4F4.1090204@oracle.com> No objection from me. One thing is that Mandy's classpath mode changes also exclude hello-native.sh, see: http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/classpath-mode.01/test/ModulesProblemList.txt.udiff.html -Alan On 28/08/2012 13:55, Chris Hegarty wrote: > Here is a small update to the ModulesProblemList.txt. > > The failing tests are mostly related to the Solaris 64 bit > layout/launcher issue, this will be addressed separately. I just > intent to add these additional tests to similar ones already excluded. > Then there is a jigsaw specific test, hello-native. I actually like > this test, but it depends on native code and a compile environment. > For now, we should just exclude it and defer making a decision on it's > future. > > : hg diff test/ModulesProblemList.txt > diff -r fb3cd8b4b19f test/ModulesProblemList.txt > --- a/test/ModulesProblemList.txt Thu Aug 23 15:24:45 2012 -0700 > +++ b/test/ModulesProblemList.txt Tue Aug 28 13:49:38 2012 +0100 > @@ -121,6 +121,10 @@ java/nio/channels/SocketChannel/AdaptSoc > java/nio/channels/SocketChannel/AdaptSocket.java linux-x64 > java/nio/channels/SocketChannel/Connect.java linux-x64 > java/nio/MappedByteBuffer/Force.java windows-i586 > + > +# Tests rely on bin/sparcv9|amd64/java launcher > +java/nio/channels/spi/SelectorProvider/inheritedChannel/run_tests.sh > solaris-sparcv9 > +java/nio/channels/spi/SelectorProvider/inheritedChannel/run_tests.sh > solaris-x64 > > > # This test sets to Turkish language without resetting it when exit > @@ -249,12 +253,16 @@ com/sun/jdi/DoubleAgentTest.java > com/sun/jdi/DoubleAgentTest.java solaris-x64 > tools/launcher/Test7029048.java solaris-x64 > sun/tools/jhat/HatHeapDump1Test.java solaris-x64 > +com/sun/jdi/PrivateTransportTest.sh solaris-x64 > +com/sun/jdi/Solaris32AndSolaris64Test.sh solaris-x64 > com/sun/jdi/BadHandshakeTest.java solaris-sparcv9 > com/sun/jdi/ExclusiveBind.java solaris-sparcv9 > com/sun/jdi/RunToExit.java solaris-sparcv9 > com/sun/jdi/DoubleAgentTest.java solaris-sparcv9 > tools/launcher/Test7029048.java solaris-sparcv9 > sun/tools/jhat/HatHeapDump1Test.java solaris-sparcv9 > +com/sun/jdi/PrivateTransportTest.sh solaris-sparcv9 > +com/sun/jdi/Solaris32AndSolaris64Test.sh solaris-sparcv9 > > # Test expects tools.jar and rt.jar. Needs to be update. > tools/pack200/Pack200Simple.sh generic-all > @@ -299,6 +307,12 @@ java/util/concurrent/locks/Lock/TimedAcq > > > ############################################################################ > > > +# jdk_jigsaw > + > +org/openjdk/jigsaw/hello-native.sh generic-all > + > +############################################################################ > > + > # langtools_jtreg known failures - included it here for reference > # these tests are not excluded in the langtools jtreg run From chris.hegarty at oracle.com Tue Aug 28 06:19:53 2012 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 28 Aug 2012 14:19:53 +0100 Subject: Update for modules Problem List In-Reply-To: <503CC4F4.1090204@oracle.com> References: <503CBFBA.5000908@oracle.com> <503CC4F4.1090204@oracle.com> Message-ID: <503CC579.7080306@oracle.com> On 28/08/12 14:17, Alan Bateman wrote: > > No objection from me. One thing is that Mandy's classpath mode changes > also exclude hello-native.sh, see: > > http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/classpath-mode.01/test/ModulesProblemList.txt.udiff.html D'oh, ok I can drop hello-native and push the remaining ones. -Chris. > > > -Alan > > > On 28/08/2012 13:55, Chris Hegarty wrote: >> Here is a small update to the ModulesProblemList.txt. >> >> The failing tests are mostly related to the Solaris 64 bit >> layout/launcher issue, this will be addressed separately. I just >> intent to add these additional tests to similar ones already excluded. >> Then there is a jigsaw specific test, hello-native. I actually like >> this test, but it depends on native code and a compile environment. >> For now, we should just exclude it and defer making a decision on it's >> future. >> >> : hg diff test/ModulesProblemList.txt >> diff -r fb3cd8b4b19f test/ModulesProblemList.txt >> --- a/test/ModulesProblemList.txt Thu Aug 23 15:24:45 2012 -0700 >> +++ b/test/ModulesProblemList.txt Tue Aug 28 13:49:38 2012 +0100 >> @@ -121,6 +121,10 @@ java/nio/channels/SocketChannel/AdaptSoc >> java/nio/channels/SocketChannel/AdaptSocket.java linux-x64 >> java/nio/channels/SocketChannel/Connect.java linux-x64 >> java/nio/MappedByteBuffer/Force.java windows-i586 >> + >> +# Tests rely on bin/sparcv9|amd64/java launcher >> +java/nio/channels/spi/SelectorProvider/inheritedChannel/run_tests.sh >> solaris-sparcv9 >> +java/nio/channels/spi/SelectorProvider/inheritedChannel/run_tests.sh >> solaris-x64 >> >> >> # This test sets to Turkish language without resetting it when exit >> @@ -249,12 +253,16 @@ com/sun/jdi/DoubleAgentTest.java >> com/sun/jdi/DoubleAgentTest.java solaris-x64 >> tools/launcher/Test7029048.java solaris-x64 >> sun/tools/jhat/HatHeapDump1Test.java solaris-x64 >> +com/sun/jdi/PrivateTransportTest.sh solaris-x64 >> +com/sun/jdi/Solaris32AndSolaris64Test.sh solaris-x64 >> com/sun/jdi/BadHandshakeTest.java solaris-sparcv9 >> com/sun/jdi/ExclusiveBind.java solaris-sparcv9 >> com/sun/jdi/RunToExit.java solaris-sparcv9 >> com/sun/jdi/DoubleAgentTest.java solaris-sparcv9 >> tools/launcher/Test7029048.java solaris-sparcv9 >> sun/tools/jhat/HatHeapDump1Test.java solaris-sparcv9 >> +com/sun/jdi/PrivateTransportTest.sh solaris-sparcv9 >> +com/sun/jdi/Solaris32AndSolaris64Test.sh solaris-sparcv9 >> >> # Test expects tools.jar and rt.jar. Needs to be update. >> tools/pack200/Pack200Simple.sh generic-all >> @@ -299,6 +307,12 @@ java/util/concurrent/locks/Lock/TimedAcq >> >> >> ############################################################################ >> >> >> +# jdk_jigsaw >> + >> +org/openjdk/jigsaw/hello-native.sh generic-all >> + >> +############################################################################ >> >> + >> # langtools_jtreg known failures - included it here for reference >> # these tests are not excluded in the langtools jtreg run > From mandy.chung at oracle.com Tue Aug 28 07:59:34 2012 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 28 Aug 2012 07:59:34 -0700 Subject: Update for modules Problem List In-Reply-To: <503CC579.7080306@oracle.com> References: <503CBFBA.5000908@oracle.com> <503CC4F4.1090204@oracle.com> <503CC579.7080306@oracle.com> Message-ID: <503CDCD6.5090704@oracle.com> On 8/28/2012 6:19 AM, Chris Hegarty wrote: > On 28/08/12 14:17, Alan Bateman wrote: >> >> No objection from me. One thing is that Mandy's classpath mode changes >> also exclude hello-native.sh, see: >> >> http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/classpath-mode.01/test/ModulesProblemList.txt.udiff.html >> > > D'oh, ok I can drop hello-native and push the remaining ones. Chris - you can exclude hello-native.sh in your patch since it has been failing due to a bug in javac and also compiler not present on test machines. I will do the merge when I push the classpath mode changeset. Thanks Mandy From chris.hegarty at oracle.com Tue Aug 28 08:05:15 2012 From: chris.hegarty at oracle.com (chris.hegarty at oracle.com) Date: Tue, 28 Aug 2012 15:05:15 +0000 Subject: hg: jigsaw/jigsaw/jdk: Update modules Problem List Message-ID: <20120828150605.A846D47775@hg.openjdk.java.net> Changeset: 32f9cef809bb Author: chegar Date: 2012-08-28 16:04 +0100 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/32f9cef809bb Update modules Problem List Reviewed-by: alanb, mchung ! test/ModulesProblemList.txt From alan.bateman at oracle.com Wed Aug 29 05:54:41 2012 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Wed, 29 Aug 2012 12:54:41 +0000 Subject: hg: jigsaw/jigsaw/jdk: Updates to service provider module resolution Message-ID: <20120829125501.E51F3477B5@hg.openjdk.java.net> Changeset: 0ffdcf18b3ac Author: psandoz Date: 2012-08-29 13:51 +0100 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/0ffdcf18b3ac Updates to service provider module resolution - "permits" clauses do not affect service provider module resolution - service provider module resolution occurs after "application" resolution Reviewed-by: mchung, alanb, mr ! src/share/classes/org/openjdk/jigsaw/Context.java ! src/share/classes/org/openjdk/jigsaw/Resolver.java ! test/org/openjdk/jigsaw/ContextBuilder.java ! test/org/openjdk/jigsaw/MockLibrary.java ! test/org/openjdk/jigsaw/ModuleInfoBuilder.java ! test/org/openjdk/jigsaw/_Configurator.java - test/org/openjdk/jigsaw/services/classloader.sh - test/org/openjdk/jigsaw/services/hello.sh - test/org/openjdk/jigsaw/services/jce.sh + test/org/openjdk/jigsaw/services/loading.sh - test/org/openjdk/jigsaw/services/many.sh - test/org/openjdk/jigsaw/services/optional.sh - test/org/openjdk/jigsaw/services/services-jar.sh - test/org/openjdk/jigsaw/services/view.sh From alan.bateman at oracle.com Wed Aug 29 08:21:32 2012 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Wed, 29 Aug 2012 15:21:32 +0000 Subject: hg: jigsaw/jigsaw/jdk: Re-add services tests (accidently removed by previous change-set) Message-ID: <20120829152145.6241B477B9@hg.openjdk.java.net> Changeset: 6961f936a0b3 Author: alanb Date: 2012-08-29 16:20 +0100 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/6961f936a0b3 Re-add services tests (accidently removed by previous change-set) + test/org/openjdk/jigsaw/services/jce.sh + test/org/openjdk/jigsaw/services/services-jar.sh From paul.sandoz at oracle.com Wed Aug 29 09:11:00 2012 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 29 Aug 2012 18:11:00 +0200 Subject: Test for service iteration when SCE occurs Message-ID: <62EF692C-2CAE-4BCF-B61E-BCF4A0AABC19@oracle.com> Hi, http://cr.openjdk.java.net/~psandoz/jigsaw/iteration-error-test/webrev/jdk.patch Since the iteration of service instances from modules is separated out j.u.SL we need to ensure that behaviour is preserved when an SCE occurs. Paul. From mandy.chung at oracle.com Wed Aug 29 09:49:19 2012 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 29 Aug 2012 09:49:19 -0700 Subject: Test for service iteration when SCE occurs In-Reply-To: <62EF692C-2CAE-4BCF-B61E-BCF4A0AABC19@oracle.com> References: <62EF692C-2CAE-4BCF-B61E-BCF4A0AABC19@oracle.com> Message-ID: <503E480F.4050402@oracle.com> On 8/29/2012 9:11 AM, Paul Sandoz wrote: > Hi, > > http://cr.openjdk.java.net/~psandoz/jigsaw/iteration-error-test/webrev/jdk.patch > > Since the iteration of service instances from modules is separated out j.u.SL we need to ensure that behaviour is preserved when an SCE occurs. Paul - I think this test can use test/org/openjdk/jigsaw/tester.sh that will parse the given source and compile/install/run the main module. test/org/openjdk/jigsaw/hello.sh is a simple example. tester.sh currently doesn't allow blank lines though. Mandy From paul.sandoz at oracle.com Thu Aug 30 03:02:51 2012 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 30 Aug 2012 12:02:51 +0200 Subject: Test for service iteration when SCE occurs In-Reply-To: <503E480F.4050402@oracle.com> References: <62EF692C-2CAE-4BCF-B61E-BCF4A0AABC19@oracle.com> <503E480F.4050402@oracle.com> Message-ID: <114A9F6C-3891-4EF7-BC6B-9E83C1691859@oracle.com> On Aug 29, 2012, at 6:49 PM, Mandy Chung wrote: > On 8/29/2012 9:11 AM, Paul Sandoz wrote: >> Hi, >> >> http://cr.openjdk.java.net/~psandoz/jigsaw/iteration-error-test/webrev/jdk.patch >> >> Since the iteration of service instances from modules is separated out j.u.SL we need to ensure that behaviour is preserved when an SCE occurs. > > Paul - I think this test can use test/org/openjdk/jigsaw/tester.sh that will parse the given source and compile/install/run the main module. test/org/openjdk/jigsaw/hello.sh is a simple example. tester.sh currently doesn't allow blank lines though. > Thanks, i did not know about that. I did some hacking (even modified tester.sh to support different separators) and tried to apply this to both iteration-error.sh and loader.sh but after a bit i stopped because it is proving extremely fragile. That script does not process interfaces correctly, nor classes that implement or extend, nor interfaces/classes with generic type signatures. So i think we should update tester.sh separately to support alternative separators and be more robust processing classes/interface names, then update other tests as required. -- I updated the webrev to include another service provider class that throws an exception to guarantee that iteration continues, as we currently make no guarantees on the order of iteration. I also removed the unused type parameter from ServiceLoaderTest. Paul. From mandy.chung at oracle.com Thu Aug 30 13:49:56 2012 From: mandy.chung at oracle.com (mandy.chung at oracle.com) Date: Thu, 30 Aug 2012 20:49:56 +0000 Subject: hg: jigsaw/jigsaw/jdk: Add a new test for service iteration when SCE occurs Message-ID: <20120830205021.40BE847801@hg.openjdk.java.net> Changeset: 0639c2b20219 Author: psandoz Date: 2012-08-30 13:49 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/0639c2b20219 Add a new test for service iteration when SCE occurs + test/org/openjdk/jigsaw/services/iteration-error.sh From mandy.chung at oracle.com Thu Aug 30 14:03:09 2012 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 30 Aug 2012 14:03:09 -0700 Subject: Test for service iteration when SCE occurs In-Reply-To: <114A9F6C-3891-4EF7-BC6B-9E83C1691859@oracle.com> References: <62EF692C-2CAE-4BCF-B61E-BCF4A0AABC19@oracle.com> <503E480F.4050402@oracle.com> <114A9F6C-3891-4EF7-BC6B-9E83C1691859@oracle.com> Message-ID: <503FD50D.3000907@oracle.com> On 8/30/2012 3:02 AM, Paul Sandoz wrote: > I did some hacking (even modified tester.sh to support different separators) and tried to apply this to both iteration-error.sh and loader.sh but after a bit i stopped because it is proving extremely fragile. > > That script does not process interfaces correctly, nor classes that implement or extend, nor interfaces/classes with generic type signatures. So i think we should update tester.sh separately to support alternative separators and be more robust processing classes/interface names, then update other tests as required. Sorry for taking you to the wrong path. Rather than updating tester.sh, it's probably time to write a test library to replace it, as Mark mentions earlier [1]. > -- > > I updated the webrev to include another service provider class that throws an exception to guarantee that iteration continues, as we currently make no guarantees on the order of iteration. > > I also removed the unused type parameter from ServiceLoaderTest. I have pushed the changeset for you. Mandy [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2012-August/003142.html From mandy.chung at oracle.com Thu Aug 30 14:51:35 2012 From: mandy.chung at oracle.com (mandy.chung at oracle.com) Date: Thu, 30 Aug 2012 21:51:35 +0000 Subject: hg: jigsaw/jigsaw/jdk: Fix tests copyright notice with GPL Message-ID: <20120830215202.8B66F47803@hg.openjdk.java.net> Changeset: 3dd93a4d44fb Author: mchung Date: 2012-08-30 14:51 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/3dd93a4d44fb Fix tests copyright notice with GPL ! test/java/lang/module/_ModuleId.java ! test/java/lang/module/_ModuleInfoReader.java ! test/org/openjdk/jigsaw/ConfigurationBuilder.java ! test/org/openjdk/jigsaw/ContextBuilder.java ! test/org/openjdk/jigsaw/InstallFromRepo.java ! test/org/openjdk/jigsaw/MockLibrary.java ! test/org/openjdk/jigsaw/ModuleInfoBuilder.java ! test/org/openjdk/jigsaw/TrivialWebServer.java ! test/org/openjdk/jigsaw/_Configurator.java ! test/org/openjdk/jigsaw/_JigsawVersion.java ! test/org/openjdk/jigsaw/_JigsawVersionQuery.java ! test/org/openjdk/jigsaw/_Library.java ! test/org/openjdk/jigsaw/_PublishedRepository.java ! test/org/openjdk/jigsaw/_RemoteRepository.java ! test/org/openjdk/jigsaw/_RemoteRepositoryList.java ! test/org/openjdk/jigsaw/_RepositoryCatalog.java ! test/org/openjdk/jigsaw/circular-deps.sh ! test/org/openjdk/jigsaw/cli/SignedModuleFileTest.sh ! test/org/openjdk/jigsaw/cli/jmod-basic.sh ! test/org/openjdk/jigsaw/cli/jmod-signed.sh ! test/org/openjdk/jigsaw/cli/signed-modular-jar.sh ! test/org/openjdk/jigsaw/cli/signed-module.sh ! test/org/openjdk/jigsaw/cli/timestamp-test.sh ! test/org/openjdk/jigsaw/install-repo.sh ! test/org/openjdk/jigsaw/library.sh ! test/org/openjdk/jigsaw/optional-base.sh ! test/org/openjdk/jigsaw/optional-deps.sh ! test/org/openjdk/jigsaw/optional-jaxp.sh ! test/org/openjdk/jigsaw/optional-reexport.sh ! test/org/openjdk/jigsaw/pubrepo.sh ! test/org/openjdk/jigsaw/remrepo.sh ! test/org/openjdk/jigsaw/repocat.sh ! test/org/openjdk/jigsaw/repolist-jars.sh ! test/org/openjdk/jigsaw/repolist.sh ! test/org/openjdk/jigsaw/services/iteration-error.sh ! test/org/openjdk/jigsaw/services/jce.sh ! test/org/openjdk/jigsaw/services/loading.sh ! test/org/openjdk/jigsaw/services/services-jar.sh From paul.sandoz at oracle.com Fri Aug 31 01:41:47 2012 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 31 Aug 2012 10:41:47 +0200 Subject: Test for service iteration when SCE occurs In-Reply-To: <503FD50D.3000907@oracle.com> References: <62EF692C-2CAE-4BCF-B61E-BCF4A0AABC19@oracle.com> <503E480F.4050402@oracle.com> <114A9F6C-3891-4EF7-BC6B-9E83C1691859@oracle.com> <503FD50D.3000907@oracle.com> Message-ID: <18333F16-A08D-4094-B4C9-AFAE819A5FBD@oracle.com> On Aug 30, 2012, at 11:03 PM, Mandy Chung wrote: > On 8/30/2012 3:02 AM, Paul Sandoz wrote: >> I did some hacking (even modified tester.sh to support different separators) and tried to apply this to both iteration-error.sh and loader.sh but after a bit i stopped because it is proving extremely fragile. >> >> That script does not process interfaces correctly, nor classes that implement or extend, nor interfaces/classes with generic type signatures. So i think we should update tester.sh separately to support alternative separators and be more robust processing classes/interface names, then update other tests as required. > > Sorry for taking you to the wrong path. Rather than updating tester.sh, it's probably time to write a test library to replace it, as Mark mentions earlier [1]. > +1. >> -- >> >> I updated the webrev to include another service provider class that throws an exception to guarantee that iteration continues, as we currently make no guarantees on the order of iteration. >> >> I also removed the unused type parameter from ServiceLoaderTest. > > I have pushed the changeset for you. > Thanks! Paul. > Mandy > [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2012-August/003142.html > From paul.sandoz at oracle.com Fri Aug 31 01:45:08 2012 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 31 Aug 2012 10:45:08 +0200 Subject: Jigsaw-dev archived on MarkMail Message-ID: <4CEF4863-726B-48A0-919C-1757D963A76D@oracle.com> See: http://markmail.org/search/?q=jigsaw-dev#query:jigsaw-dev%20list%3Anet.java.openjdk.jigsaw-dev%20order%3Adate-backward+page:1+state:facets Some may find that useful for searching as it is easier than: http://mail.openjdk.java.net/pipermail/jigsaw-dev/ Mandy, you are the top poster :-) with Alan close behind. Paul. From Alan.Bateman at oracle.com Fri Aug 31 02:36:56 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 31 Aug 2012 10:36:56 +0100 Subject: Jigsaw-dev archived on MarkMail In-Reply-To: <4CEF4863-726B-48A0-919C-1757D963A76D@oracle.com> References: <4CEF4863-726B-48A0-919C-1757D963A76D@oracle.com> Message-ID: <504085B8.5010004@oracle.com> On 31/08/2012 09:45, Paul Sandoz wrote: > See: > > http://markmail.org/search/?q=jigsaw-dev#query:jigsaw-dev%20list%3Anet.java.openjdk.jigsaw-dev%20order%3Adate-backward+page:1+state:facets > > Some may find that useful for searching as it is easier than: > > http://mail.openjdk.java.net/pipermail/jigsaw-dev/ > I think someone must have added a bunch of OpenJDK lists to markmail recently as it used to only index the lists that existed a few year ago. -Alan.