From sean.mullan at oracle.com Thu Sep 1 08:28:43 2011 From: sean.mullan at oracle.com (Sean Mullan) Date: Thu, 01 Sep 2011 11:28:43 -0400 Subject: keytool is not in jdk-base-image In-Reply-To: <4E5ED5E6.8040103@oracle.com> References: <4E5EA8DA.7080607@oracle.com> <4E5EAD4E.4090504@oracle.com> <4E5ED5E6.8040103@oracle.com> Message-ID: <4E5FA4AB.7000907@oracle.com> On 8/31/11 8:46 PM, Weijun Wang wrote: > > > On 09/01/2011 05:53 AM, Mandy Chung wrote: >> On 8/31/11 2:34 PM, Sean Mullan wrote: >>> It will be difficult to sign modules without keytool. (It's possible, >>> but you >>> would have to have a keystore already setup with your keys and >>> certificates and >>> know the alias name, etc). >> >> But keytool depends on jsse and jndi. Can these dependencies be >> eliminated or as optional? > > The jndi dependence can be optional, but keytool needs the jsse module > to generate certs. See below: > >> >> sun.security.tools.JarSigner -> sun.security.util.PathList (sun.jsse) > > JarSigner? > >> sun.security.tools.KeyTool -> javax.net.ssl.HttpsURLConnection (sun.jsse) >> sun.security.tools.KeyTool -> javax.net.ssl.SSLContext (sun.jsse) >> sun.security.tools.KeyTool -> javax.net.ssl.TrustManager (sun.jsse) > > sun.security.tools.KeyTool$2 -> javax.net.ssl.X509TrustManager (sun.jsse) > > sun.security.tools.KeyTool$3 -> javax.net.ssl.HostnameVerifier (sun.jsse) > > sun.security.tools.KeyTool$3 -> javax.net.ssl.SSLSession (sun.jsse) > > for keytool -printcert -sslserver This option is very specific to SSL. Can we make this optional? The option would fail if sun.jsse is not installed. >> sun.security.tools.KeyTool -> sun.security.pkcs.PKCS10 (sun.jsse) >> sun.security.tools.KeyTool -> sun.security.pkcs.PKCS10Attribute (sun.jsse) >> sun.security.tools.KeyTool -> sun.security.pkcs.PKCS10Attributes (sun.jsse) Move these into jdk.tools.base. I don't think anything else uses PKCS10. > > for keytool -certreq > >> sun.security.tools.KeyTool -> >> sun.security.provider.certpath.ldap.LDAPCertStoreHelper (sun.jndi) > > for keytool -printcrl -file ldap://.... Ok, we should make that optional so jndi is not required. > >> sun.security.tools.KeyTool -> sun.security.util.PathList (sun.jsse) > > PathList is a utility class to create "a:b:c" on Unix and "a;b;c" on > Windows. Is there a similar tool inside base? This class is simple enough that we can just copy the functionality into keytool. >> sun.security.tools.KeyTool -> sun.security.x509.CertAndKeyGen (sun.jsse) > > for keytool -genkeypair and keytool -gencert Move CertAndKeyGen into jdk.tools.base. I don't think anything else uses it. --Sean From ahughes at redhat.com Thu Sep 1 19:39:50 2011 From: ahughes at redhat.com (Dr Andrew John Hughes) Date: Fri, 2 Sep 2011 03:39:50 +0100 Subject: keytool is not in jdk-base-image In-Reply-To: <4E5FA4AB.7000907@oracle.com> References: <4E5EA8DA.7080607@oracle.com> <4E5EAD4E.4090504@oracle.com> <4E5ED5E6.8040103@oracle.com> <4E5FA4AB.7000907@oracle.com> Message-ID: <20110902023950.GA19081@rivendell.middle-earth.co.uk> On 11:28 Thu 01 Sep , Sean Mullan wrote: > On 8/31/11 8:46 PM, Weijun Wang wrote: snip... > > > > >> sun.security.tools.KeyTool -> sun.security.util.PathList (sun.jsse) > > > > PathList is a utility class to create "a:b:c" on Unix and "a;b;c" on > > Windows. Is there a similar tool inside base? > > This class is simple enough that we can just copy the functionality into keytool. > Might java.nio.file.Path be useful here? Or does that make the situation worse? -- Andrew :) Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) Support Free Java! Contribute to GNU Classpath and IcedTea http://www.gnu.org/software/classpath http://icedtea.classpath.org PGP Key: F5862A37 (https://keys.indymedia.org/) Fingerprint = EA30 D855 D50F 90CD F54D 0698 0713 C3ED F586 2A37 From lvjing at linux.vnet.ibm.com Fri Sep 2 02:47:01 2011 From: lvjing at linux.vnet.ibm.com (Jing LV) Date: Fri, 02 Sep 2011 17:47:01 +0800 Subject: Build issue : missing classanalyzer? In-Reply-To: <4E5E446C.30701@oracle.com> References: <4E5C9749.5050807@linux.vnet.ibm.com> <4E5CEFA7.9000309@oracle.com> <4E5E1197.9010809@linux.vnet.ibm.com> <4E5E1520.6010509@oracle.com> <4E5E446C.30701@oracle.com> Message-ID: <4E60A615.6020202@linux.vnet.ibm.com> Hi Alan, Mandy, Struggled but still failed. Yes I update all source (using "get_source.sh"), and build all langtools/hotpot/jdk all together (make under the root directory of jigsaw, which contains sub-dir corba, hotspot, jaxp, jaxws, jdk, langtools etc). I find an existing classananlyzer.jar from a win32 build and copy it to the dir "btjars" and it seems it will replace during the build. It's a bit wired, not sure it is due to windows2008 platform. ? 2011-8-31 22:25, Mandy Chung ??: > > On 8/31/11 4:04 AM, Alan Bateman wrote: >> Jing LV wrote: >>> Hi Alan, >>> >>> Thank you for reply. I've checked with the log (it's of ~300K in >>> size so unable to upload to the list) but do not have something >>> related to "HOST_JAVA_CMD" - yes anyway the build has pass nearly >>> all modules. Only blocked by the classanalyzer. >>> >>> I take a little more search in the log, it seems the problem is due >>> to the class "CONSTANT_ModuleId_info": >>> src\com\sun\classanalyzer\ >>> ConstantPoolParser.java:270: error: cannot find symbol >>> public String visitModuleId(CONSTANT_ModuleId_info info, Void p) { >>> ^ >>> symbol: class CONSTANT_ModuleId_info >>> location: class ConstantPoolParser.StringValueVisitor >>> >>> Search in the full openjdk directory for "CONSTANT_ModuleId_info" >>> but find nothing - am I miss somthing? (I see in google that some >>> archive mail discussed about this variable, anyway have some trouble >>> to open the link due to the network problem). >> Is your langtools repository up to date? > > In case you are just building the jigsaw/jdk repository, for building > jigsaw, you need to build jigsaw/langtools, jigsaw/hotspot, and > jigsaw/jdk repository together as there are jigsaw changes in the VM > and javac. The simplest way is to build the entire forest from the top > repo and do "make modules". > > Mandy -- Best Regards, Jimmy, Jing LV From Alan.Bateman at oracle.com Fri Sep 2 04:55:29 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 02 Sep 2011 12:55:29 +0100 Subject: Build issue : missing classanalyzer? In-Reply-To: <4E60A615.6020202@linux.vnet.ibm.com> References: <4E5C9749.5050807@linux.vnet.ibm.com> <4E5CEFA7.9000309@oracle.com> <4E5E1197.9010809@linux.vnet.ibm.com> <4E5E1520.6010509@oracle.com> <4E5E446C.30701@oracle.com> <4E60A615.6020202@linux.vnet.ibm.com> Message-ID: <4E60C431.70801@oracle.com> Jing LV wrote: > Hi Alan, Mandy, > > Struggled but still failed. Yes I update all source (using > "get_source.sh"), and build all langtools/hotpot/jdk all together > (make under the root directory of jigsaw, which contains sub-dir > corba, hotspot, jaxp, jaxws, jdk, langtools etc). I find an existing > classananlyzer.jar from a win32 build and copy it to the dir "btjars" > and it seems it will replace during the build. > It's a bit wired, not sure it is due to windows2008 platform. Can you put the build log on a http server? I can only guess that something went wrong building the class analyzer but the build continues. There may be a warning in the build log that gives a clue. -Alan From sean.mullan at oracle.com Fri Sep 2 09:49:03 2011 From: sean.mullan at oracle.com (Sean Mullan) Date: Fri, 02 Sep 2011 12:49:03 -0400 Subject: keytool is not in jdk-base-image In-Reply-To: <20110902023950.GA19081@rivendell.middle-earth.co.uk> References: <4E5EA8DA.7080607@oracle.com> <4E5EAD4E.4090504@oracle.com> <4E5ED5E6.8040103@oracle.com> <4E5FA4AB.7000907@oracle.com> <20110902023950.GA19081@rivendell.middle-earth.co.uk> Message-ID: <4E6108FF.6090603@oracle.com> On 9/1/11 10:39 PM, Dr Andrew John Hughes wrote: > On 11:28 Thu 01 Sep , Sean Mullan wrote: >> On 8/31/11 8:46 PM, Weijun Wang wrote: > > snip... > >> >>> >>>> sun.security.tools.KeyTool -> sun.security.util.PathList (sun.jsse) >>> >>> PathList is a utility class to create "a:b:c" on Unix and "a;b;c" on >>> Windows. Is there a similar tool inside base? >> >> This class is simple enough that we can just copy the functionality into keytool. >> > > Might java.nio.file.Path be useful here? I don't think so, at least not directly - PathList contains utility methods that operate on a list of paths (ex: classpath). --Sean From kelly.ohair at oracle.com Fri Sep 2 12:09:23 2011 From: kelly.ohair at oracle.com (Kelly O'Hair) Date: Fri, 2 Sep 2011 12:09:23 -0700 Subject: Build issue : missing classanalyzer? In-Reply-To: <4E60A615.6020202@linux.vnet.ibm.com> References: <4E5C9749.5050807@linux.vnet.ibm.com> <4E5CEFA7.9000309@oracle.com> <4E5E1197.9010809@linux.vnet.ibm.com> <4E5E1520.6010509@oracle.com> <4E5E446C.30701@oracle.com> <4E60A615.6020202@linux.vnet.ibm.com> Message-ID: Make sure the clock on the machine is correct. Clock skew can cause lots of strange build issues. -kto On Sep 2, 2011, at 2:47 AM, Jing LV wrote: > Hi Alan, Mandy, > > Struggled but still failed. Yes I update all source (using "get_source.sh"), and build all langtools/hotpot/jdk all together (make under the root directory of jigsaw, which contains sub-dir corba, hotspot, jaxp, jaxws, jdk, langtools etc). I find an existing classananlyzer.jar from a win32 build and copy it to the dir "btjars" and it seems it will replace during the build. > It's a bit wired, not sure it is due to windows2008 platform. > > ? 2011-8-31 22:25, Mandy Chung ??: >> >> On 8/31/11 4:04 AM, Alan Bateman wrote: >>> Jing LV wrote: >>>> Hi Alan, >>>> >>>> Thank you for reply. I've checked with the log (it's of ~300K in size so unable to upload to the list) but do not have something related to "HOST_JAVA_CMD" - yes anyway the build has pass nearly all modules. Only blocked by the classanalyzer. >>>> >>>> I take a little more search in the log, it seems the problem is due to the class "CONSTANT_ModuleId_info": >>>> src\com\sun\classanalyzer\ >>>> ConstantPoolParser.java:270: error: cannot find symbol >>>> public String visitModuleId(CONSTANT_ModuleId_info info, Void p) { >>>> ^ >>>> symbol: class CONSTANT_ModuleId_info >>>> location: class ConstantPoolParser.StringValueVisitor >>>> >>>> Search in the full openjdk directory for "CONSTANT_ModuleId_info" but find nothing - am I miss somthing? (I see in google that some archive mail discussed about this variable, anyway have some trouble to open the link due to the network problem). >>> Is your langtools repository up to date? >> >> In case you are just building the jigsaw/jdk repository, for building jigsaw, you need to build jigsaw/langtools, jigsaw/hotspot, and jigsaw/jdk repository together as there are jigsaw changes in the VM and javac. The simplest way is to build the entire forest from the top repo and do "make modules". >> >> Mandy > > > -- > Best Regards, > Jimmy, Jing LV > > From weijun.wang at oracle.com Sun Sep 4 17:46:10 2011 From: weijun.wang at oracle.com (Weijun Wang) Date: Mon, 05 Sep 2011 08:46:10 +0800 Subject: keytool is not in jdk-base-image In-Reply-To: <4E5FA4AB.7000907@oracle.com> References: <4E5EA8DA.7080607@oracle.com> <4E5EAD4E.4090504@oracle.com> <4E5ED5E6.8040103@oracle.com> <4E5FA4AB.7000907@oracle.com> Message-ID: <4E641BD2.50601@oracle.com> I see three kinds of solutions in your reply: 1. Move something to a module 2. Make a function optional 3. Re-write PathList inside keytool I understand #3 is something I need to do and #1 is configured somewhere in class classification. What about #2? To make a function optional, what will the user experience be? Does it mean when a module is not installed, the function just fails with a ClassNotFoundException? Or I can write something like Class.forName() inside to show user a warning message? Thanks Max On 09/01/2011 11:28 PM, Sean Mullan wrote: > On 8/31/11 8:46 PM, Weijun Wang wrote: >> >> >> On 09/01/2011 05:53 AM, Mandy Chung wrote: >>> On 8/31/11 2:34 PM, Sean Mullan wrote: >>>> It will be difficult to sign modules without keytool. (It's possible, >>>> but you >>>> would have to have a keystore already setup with your keys and >>>> certificates and >>>> know the alias name, etc). >>> >>> But keytool depends on jsse and jndi. Can these dependencies be >>> eliminated or as optional? >> >> The jndi dependence can be optional, but keytool needs the jsse module >> to generate certs. See below: >> >>> >>> sun.security.tools.JarSigner -> sun.security.util.PathList (sun.jsse) >> >> JarSigner? >> >>> sun.security.tools.KeyTool -> javax.net.ssl.HttpsURLConnection (sun.jsse) >>> sun.security.tools.KeyTool -> javax.net.ssl.SSLContext (sun.jsse) >>> sun.security.tools.KeyTool -> javax.net.ssl.TrustManager (sun.jsse) >> > sun.security.tools.KeyTool$2 -> javax.net.ssl.X509TrustManager (sun.jsse) >> > sun.security.tools.KeyTool$3 -> javax.net.ssl.HostnameVerifier (sun.jsse) >> > sun.security.tools.KeyTool$3 -> javax.net.ssl.SSLSession (sun.jsse) >> >> for keytool -printcert -sslserver > > This option is very specific to SSL. Can we make this optional? The option would > fail if sun.jsse is not installed. > >>> sun.security.tools.KeyTool -> sun.security.pkcs.PKCS10 (sun.jsse) >>> sun.security.tools.KeyTool -> sun.security.pkcs.PKCS10Attribute (sun.jsse) >>> sun.security.tools.KeyTool -> sun.security.pkcs.PKCS10Attributes (sun.jsse) > > Move these into jdk.tools.base. I don't think anything else uses PKCS10. > >> >> for keytool -certreq >> >>> sun.security.tools.KeyTool -> >>> sun.security.provider.certpath.ldap.LDAPCertStoreHelper (sun.jndi) >> >> for keytool -printcrl -file ldap://.... > > Ok, we should make that optional so jndi is not required. > >> >>> sun.security.tools.KeyTool -> sun.security.util.PathList (sun.jsse) >> >> PathList is a utility class to create "a:b:c" on Unix and "a;b;c" on >> Windows. Is there a similar tool inside base? > > This class is simple enough that we can just copy the functionality into keytool. > >>> sun.security.tools.KeyTool -> sun.security.x509.CertAndKeyGen (sun.jsse) >> >> for keytool -genkeypair and keytool -gencert > > Move CertAndKeyGen into jdk.tools.base. I don't think anything else uses it. > > --Sean From sean.mullan at oracle.com Tue Sep 6 10:13:02 2011 From: sean.mullan at oracle.com (Sean Mullan) Date: Tue, 06 Sep 2011 13:13:02 -0400 Subject: keytool is not in jdk-base-image In-Reply-To: <4E641BD2.50601@oracle.com> References: <4E5EA8DA.7080607@oracle.com> <4E5EAD4E.4090504@oracle.com> <4E5ED5E6.8040103@oracle.com> <4E5FA4AB.7000907@oracle.com> <4E641BD2.50601@oracle.com> Message-ID: <4E66549E.3010105@oracle.com> On 9/4/11 8:46 PM, Weijun Wang wrote: > I see three kinds of solutions in your reply: > > 1. Move something to a module > 2. Make a function optional > 3. Re-write PathList inside keytool > > I understand #3 is something I need to do and #1 is configured somewhere > in class classification. What about #2? To make a function optional, > what will the user experience be? > > Does it mean when a module is not installed, the function just fails > with a ClassNotFoundException? Or I can write something like > Class.forName() inside to show user a warning message? Yes, I think so. Please hold off on making any changes for now. I'm doing a bit of experimentation on my own. I want to try moving things around and it will also give me some more experience with how the modules are built. --Sean From mark.reinhold at oracle.com Wed Sep 7 14:58:41 2011 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Wed, 07 Sep 2011 14:58:41 -0700 Subject: Review request on the optional modules support In-Reply-To: mandy.chung@oracle.com; Thu, 04 Aug 2011 12:35:11 PDT; <4E3AF46F.1050801@oracle.com> Message-ID: <20110907215841.87EE12C40@eggemoggin.niobe.net> 2011/8/4 12:35 -0700, mandy.chung at oracle.com: > An update on the optional module support [1]. > > Webrev: > http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/optional-modules.00/ I looked at the newer version, as you suggested: http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/optional-modules.01/ > The note has been updated to include the open questions raised in > the discussion. > http://openjdk.java.net/projects/jigsaw/doc/topics/optional.html > > ... This mostly looks good but I have a few suggestions and questions. Line numbers are in [brackets]. org/openjdk/jigsaw/Context.java [158] Why a TreeSet? Is order important? If not, use a HashSet. [166] s/reexporting/re-exporting/ org/openjdk/jigsaw/Loader.java [131..132] s/mname/mn/g [158..164] Suggest reversing the order of these two statements. If you find the library first then you can pass that to the findModule method, avoiding a second lookup in that method. [182] s/kernel/boot/ [189] Brace should be at end of previous line. [190] s/this class's/this/ [196] If m != null then you can just return it here and remove the test on [204]. [382..388] The isModulePresent method should be defined earlier, before toString() [256]. org/openjdk/jigsaw/BootLoader.java [55] BASE_MODULE should be defined in Platform.java; it doesn't belong here. [58] If you're going to provide a factory method to enforce a singleton instance [118] then declare the constructor to be private. [94..115] I don't understand why this override is necessary. The boot and base modules will be in the same context, so the definition in the Loader superclass should work correctly. [131] Brace should be at end of previous line. org/openjdk/jigsaw/Linker.java [246] This comment should now mention that the supplier-name maps are being updated too. java/lang/reflect/Module.java I think the isModulePresent and requireModulePresent methods really belong in java.lang.module.ModuleClassLoader. These methods query a module context, not a module, and a ModuleClassLoader holds the run-time representation of a module context. This makes the usage idioms a little clunkier since now you have to cast the result of Class.getClassLoader(): ((ModuleClassLoader)this.class.getClassLoader()).isModulePresent(mn) but that could be ameliorated by defining a convenience method in java.lang.Class which does the cast for you (and maybe returns the boot module loader rather than null when in legacy mode, as we've discussed previously). java/lang/module/RequireOptionalModule.java Shouldn't this be named "RequiresOptionalModule", to match the fact that we use "requires" rather than "require" in module declarations? [47] If you rename this to "value" then annotations of the form @RequireOptionalModule(modules={"jdk.jaxp"}) can be shortened to @RequireOptionalModule("jdk.jaxp") - Mark From mandy.chung at oracle.com Wed Sep 7 18:37:38 2011 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 07 Sep 2011 18:37:38 -0700 Subject: Review request on the optional modules support In-Reply-To: <20110907215841.87EE12C40@eggemoggin.niobe.net> References: <20110907215841.87EE12C40@eggemoggin.niobe.net> Message-ID: <4E681C62.1080400@oracle.com> On 9/7/11 2:58 PM, mark.reinhold at oracle.com wrote: > This mostly looks good but I have a few suggestions and questions. Thanks for the review. I'll make the change per your suggestions. My comments inlined below. > Line numbers are in [brackets]. > > org/openjdk/jigsaw/Context.java > > [158] Why a TreeSet? Is order important? If not, use a HashSet. Order is not important and will change it to HashSet. > > > org/openjdk/jigsaw/BootLoader.java > > > [94..115] I don't understand why this override is necessary. The boot > and base modules will be in the same context, so the definition in the > Loader superclass should work correctly. The base and boot modules were made to be in different contexts to implement the exports work [1]. In the prototype I built for exports, if they are in the same context, the internal types exported from the boot module will be visible to the applications requiring jdk.base. As we have to explore further what the best model for exports, I should keep them in the same context in this patch. I do think this override is not necessary if they are in the same context. But I'm afraid split package could be one potential issue. I'll make the change and double check to confirm. One issue I ran into is that: java.util.XMLUtils is currently in sun.jaxp module due to its dependency on jaxp API. java.util package is split between jdk.boot and sun.jaxp and thus they are in the same context. In this case, jdk.base can't require optional jdk.jaxp. Otherwise it will fail to install with: org.openjdk.jigsaw.ConfigurationException: Package javax.crypto.interfaces defined in +jdk.base+jdk.boot+sun.charsets+sun.jaxp but exported by supplier +jdk.jaxp This is not a good example since java.util.XMLUtils should be renamed to another package to avoid the split package. It's on the todo list and I'll do the rename in this fix and see if there is any other issue. > > java/lang/reflect/Module.java > > I think the isModulePresent and requireModulePresent methods really > belong in java.lang.module.ModuleClassLoader. These methods query > a module context, not a module, and a ModuleClassLoader holds the > run-time representation of a module context. > > This makes the usage idioms a little clunkier since now you have to > cast the result of Class.getClassLoader(): > > ((ModuleClassLoader)this.class.getClassLoader()).isModulePresent(mn) > > but that could be ameliorated by defining a convenience method in > java.lang.Class which does the cast for you (and maybe returns the boot > module loader rather than null when in legacy mode, as we've discussed > previously). I also considered defining these methods in ModuleClassLoader and add a convenience method in java.lang.Class that would simplify the call to these methods after posting the webrev. I thought I added that question to in optional.html to look for feedback but apparently it's still in my private copy :( That's good and I'll make the change. > > java/lang/module/RequireOptionalModule.java > > Shouldn't this be named "RequiresOptionalModule", to match the fact > that we use "requires" rather than "require" in module declarations? I thought the new proposed grammar [2] is "require" rather than "requires". The current prototype is yet to be updated to implement the new proposal and thus we still use "requires". > [47] If you rename this to "value" then annotations of the form > > @RequireOptionalModule(modules={"jdk.jaxp"}) > > can be shortened to > > @RequireOptionalModule("jdk.jaxp") That looks more concise. Thanks Mandy [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2011-June/001333.html [2] http://openjdk.java.net/projects/jigsaw/doc/topics/grammar.html From mr at sun.com Fri Sep 9 09:20:37 2011 From: mr at sun.com (mr at sun.com) Date: Fri, 09 Sep 2011 16:20:37 +0000 Subject: hg: jigsaw/jigsaw/jdk: Fix repolist test to work when the local hostname != \"localhost\" Message-ID: <20110909162106.2EF0F474E9@hg.openjdk.java.net> Changeset: 8b8a71667912 Author: mr Date: 2011-09-09 09:20 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/8b8a71667912 Fix repolist test to work when the local hostname != \"localhost\" ! test/org/openjdk/jigsaw/_RemoteRepositoryList.java From forax at univ-mlv.fr Sun Sep 11 07:03:58 2011 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Sun, 11 Sep 2011 16:03:58 +0200 Subject: Legacy mode Message-ID: <4E6CBFCE.1030608@univ-mlv.fr> Hi everybody, I've found a document describing how the legacy mode works so I'm able to understand if this I have two libraries lib1and lib2 and an application app. The application module declare a dependency to the lib1 module and another to the lib2 module, lib2 also declares a dependency to lib1. Now, a new incompatible version of lib1 is created, and the app is updated to use this new version. lib2 is not updated and still use the old version. So in the module repository, there are two versions of lib1. Is there a problem if I run my application in legacy mode ? R?mi From Alan.Bateman at oracle.com Sun Sep 11 10:38:23 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 11 Sep 2011 18:38:23 +0100 Subject: Legacy mode In-Reply-To: <4E6CBFCE.1030608@univ-mlv.fr> References: <4E6CBFCE.1030608@univ-mlv.fr> Message-ID: <4E6CF20F.8070906@oracle.com> R?mi Forax wrote: > Hi everybody, > I've found a document describing how the legacy mode works > so I'm able to understand if this > > I have two libraries lib1and lib2 and an application app. > The application module declare a dependency to > the lib1 module and another to the lib2 module, > lib2 also declares a dependency to lib1. > > Now, a new incompatible version of lib1 is created, > and the app is updated to use this new version. > lib2 is not updated and still use the old version. > > So in the module repository, there are two versions > of lib1. > > Is there a problem if I run my application in legacy mode ? > > R?mi > I can't tell which document you are reading but legacy mode means running applications in the same way that they are run today (meaning classpaths, JAR files, etc.). There isn't any way to run applications installed as modules to run in legacy mode, at least not with making use of implementation knowledge of the module library layout. At some point, and to aid migration, then we'll probably have to add support to allow JAR files depend on modules. I don't think anything has been done on that. Hopefully this helps, -Alan. From forax at univ-mlv.fr Sun Sep 11 11:39:17 2011 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Sun, 11 Sep 2011 20:39:17 +0200 Subject: Legacy mode In-Reply-To: <4E6CF20F.8070906@oracle.com> References: <4E6CBFCE.1030608@univ-mlv.fr> <4E6CF20F.8070906@oracle.com> Message-ID: <4E6D0055.3010009@univ-mlv.fr> On 09/11/2011 07:38 PM, Alan Bateman wrote: > R?mi Forax wrote: >> Hi everybody, >> I've found a document describing how the legacy mode works >> so I'm able to understand if this >> >> I have two libraries lib1and lib2 and an application app. >> The application module declare a dependency to >> the lib1 module and another to the lib2 module, >> lib2 also declares a dependency to lib1. >> >> Now, a new incompatible version of lib1 is created, >> and the app is updated to use this new version. >> lib2 is not updated and still use the old version. >> >> So in the module repository, there are two versions >> of lib1. >> >> Is there a problem if I run my application in legacy mode ? >> >> R?mi >> > I can't tell which document you are reading oops sorry, I should have written "I've found no document" ... > but legacy mode means running applications in the same way that they > are run today (meaning classpaths, JAR files, etc.). There isn't any > way to run applications installed as modules to run in legacy mode, at > least not with making use of implementation knowledge of the module > library layout. At some point, and to aid migration, then we'll > probably have to add support to allow JAR files depend on modules. I > don't think anything has been done on that. The JDK8 will provide a legacy mode and will have private dependencies on several libraries that are currently repackaged by adding a package prefix (com.sun.). Now suppose that these libraries are installed as modules, how the JDK will avoid conflicts with versions of these libraries provided by the application in the classpath ? > > Hopefully this helps, > > -Alan. > R?mi From Alan.Bateman at oracle.com Sun Sep 11 12:41:55 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 11 Sep 2011 20:41:55 +0100 Subject: Legacy mode In-Reply-To: <4E6D0055.3010009@univ-mlv.fr> References: <4E6CBFCE.1030608@univ-mlv.fr> <4E6CF20F.8070906@oracle.com> <4E6D0055.3010009@univ-mlv.fr> Message-ID: <4E6D0F03.6040902@oracle.com> R?mi Forax wrote: > > The JDK8 will provide a legacy mode and will have private dependencies > on several libraries > that are currently repackaged by adding a package prefix (com.sun.). > Now suppose that these libraries are installed as modules, > how the JDK will avoid conflicts with versions of these libraries > provided by the application > in the classpath ? It shouldn't be any different to today. If the JDK has re-packaged an API org.foo into com.sun.internal.org.foo and is using it internally then it's just not going to use the org.foo API that you put on the class path. Or maybe you are trying to put a newer version of com.sun.internal.org.foo on the class path? That isn't going to be used either as delegation should mean it finds the classes on the boot class path. The usual way to override classes in rt.jar is via -Xbootclasspath/p and that should work exactly the same in jdk8 when running in legacy mode (-Xbootclasspath doesn't of course make sense in modules mode and so won't be allowed). -Alan. From forax at univ-mlv.fr Sun Sep 11 13:14:12 2011 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Sun, 11 Sep 2011 22:14:12 +0200 Subject: Legacy mode In-Reply-To: <4E6D0F03.6040902@oracle.com> References: <4E6CBFCE.1030608@univ-mlv.fr> <4E6CF20F.8070906@oracle.com> <4E6D0055.3010009@univ-mlv.fr> <4E6D0F03.6040902@oracle.com> Message-ID: <4E6D1694.30909@univ-mlv.fr> On 09/11/2011 09:41 PM, Alan Bateman wrote: > R?mi Forax wrote: >> >> The JDK8 will provide a legacy mode and will have private >> dependencies on several libraries >> that are currently repackaged by adding a package prefix (com.sun.). >> Now suppose that these libraries are installed as modules, >> how the JDK will avoid conflicts with versions of these libraries >> provided by the application >> in the classpath ? > It shouldn't be any different to today. If the JDK has re-packaged an > API org.foo into com.sun.internal.org.foo and is using it internally > then it's just not going to use the org.foo API that you put on the > class path. I was hoping that modules was a solution to avoid re-packaging. > Or maybe you are trying to put a newer version of > com.sun.internal.org.foo on the class path? That isn't going to be > used either as delegation should mean it finds the classes on the boot > class path. The usual way to override classes in rt.jar is via > -Xbootclasspath/p and that should work exactly the same in jdk8 when > running in legacy mode (-Xbootclasspath doesn't of course make sense > in modules mode and so won't be allowed). Why -Xbootclasspath doesn't make sense ? It's a fast way to patch what you want. > > -Alan. R?mi From Alan.Bateman at oracle.com Sun Sep 11 13:54:17 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 11 Sep 2011 21:54:17 +0100 Subject: Legacy mode In-Reply-To: <4E6D1694.30909@univ-mlv.fr> References: <4E6CBFCE.1030608@univ-mlv.fr> <4E6CF20F.8070906@oracle.com> <4E6D0055.3010009@univ-mlv.fr> <4E6D0F03.6040902@oracle.com> <4E6D1694.30909@univ-mlv.fr> Message-ID: <4E6D1FF9.7020806@oracle.com> R?mi Forax wrote: > > I was hoping that modules was a solution to avoid re-packaging. I don't think we want to get into mapping package names. Instead the right thing would be to re-visit those places where it was previously necessary to incorporate third party libraries to see if they make sense as modules now. > > Why -Xbootclasspath doesn't make sense ? > It's a fast way to patch what you want. It's doesn't make sense in module mode because the boot class path goes away. Once we have the modulepath support in then it should make it easy without needing to installing a module. -Alan. From forax at univ-mlv.fr Sun Sep 11 14:27:23 2011 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Sun, 11 Sep 2011 23:27:23 +0200 Subject: Legacy mode In-Reply-To: <4E6D1FF9.7020806@oracle.com> References: <4E6CBFCE.1030608@univ-mlv.fr> <4E6CF20F.8070906@oracle.com> <4E6D0055.3010009@univ-mlv.fr> <4E6D0F03.6040902@oracle.com> <4E6D1694.30909@univ-mlv.fr> <4E6D1FF9.7020806@oracle.com> Message-ID: <4E6D27BB.2060705@univ-mlv.fr> On 09/11/2011 10:54 PM, Alan Bateman wrote: > R?mi Forax wrote: >> >> I was hoping that modules was a solution to avoid re-packaging. > I don't think we want to get into mapping package names. Instead the > right thing would be to re-visit those places where it was previously > necessary to incorporate third party libraries to see if they make > sense as modules now. but you can't convert them as module because you need the re-packaging to support legacy mode. > >> >> Why -Xbootclasspath doesn't make sense ? >> It's a fast way to patch what you want. > It's doesn't make sense in module mode because the boot class path > goes away. it's a tautology. You don't need it because you don't need it :) > Once we have the modulepath support in then it should make it easy > without needing to installing a module. The module path is the way to find a module before looking it in the module system. Patching just one file should not require to declare a module or the whole module you perhaps need also the module-info but not more. > > -Alan. R?mi From David.Holmes at oracle.com Sun Sep 11 20:47:46 2011 From: David.Holmes at oracle.com (David Holmes) Date: Mon, 12 Sep 2011 13:47:46 +1000 Subject: Legacy mode In-Reply-To: <4E6D27BB.2060705@univ-mlv.fr> References: <4E6CBFCE.1030608@univ-mlv.fr> <4E6CF20F.8070906@oracle.com> <4E6D0055.3010009@univ-mlv.fr> <4E6D0F03.6040902@oracle.com> <4E6D1694.30909@univ-mlv.fr> <4E6D1FF9.7020806@oracle.com> <4E6D27BB.2060705@univ-mlv.fr> Message-ID: <4E6D80E2.4070803@oracle.com> On 12/09/2011 7:27 AM, R?mi Forax wrote: > On 09/11/2011 10:54 PM, Alan Bateman wrote: >> Once we have the modulepath support in then it should make it easy without >> needing to installing a module. > > The module path is the way to find a module before looking it in the module > system. > Patching just one file should not require to declare a module or the whole > module you perhaps need also the module-info but not more. I was just thinking the same thing. It is extremely convenient to be able to "override" individual class implementations using the bootclasspath. It would have been far less convenient if -Xbootclasspath had required you to use a jar rather than just a class. The same will go for modulepath if we can only "override" modules not individual classes within them. David > >> >> -Alan. > > R?mi > From Alan.Bateman at oracle.com Mon Sep 12 06:06:34 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 12 Sep 2011 14:06:34 +0100 Subject: Review request: jar tool to transform a plain jar file to a modular jar In-Reply-To: <4E5D724D.5050606@oracle.com> References: <4E56F4FB.6060205@oracle.com> <4E580D72.70605@oracle.com> <4E5D724D.5050606@oracle.com> Message-ID: <4E6E03DA.1080408@oracle.com> Mandy Chung wrote: > Updated webrev: > http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/modularize-jar.01/ I see Sherman has done a detailed review on the sun.tools.jar changes. I'm looking at additions to java.util.jar.JarFile and I'm wondering if the class or method descriptions should having wording to explain that a JAR file can have a legacy manifest and also a module-info. I also wonder if we will need to update the JAR specification too. Minor comment is that the new methods should have @since 1.8. A passing comment on SimpleLibrary lines 980-988 is that they could be replaced with Files.write(is, md.toPath().resolve("info")); The sun.tools.jar.ModuleInfo class looks very useful and I'm sure we will have other places that need to generate module-info classes. Would it make sense to put it in org.openjdk.jigsaw? If so then it would be good to have it support requires optional too. -Alan. From mbien at fh-landshut.de Mon Sep 12 07:15:46 2011 From: mbien at fh-landshut.de (Michael Bien) Date: Mon, 12 Sep 2011 16:15:46 +0200 Subject: Review request: jar tool to transform a plain jar file to a modularjar In-Reply-To: <4E56F4FB.6060205@oracle.com> References: <4E56F4FB.6060205@oracle.com> Message-ID: <4E6E1412.7090706@fh-landshut.de> any chance to add shortcuts to those common execution commands? $ java mlib com.greetings as shortcut for $ java -L mlib -m com.greetings or even $ java mlib if there is such thing like a default main method. modules have great potential to make java commandline friendly. regards, -michael On 08/26/2011 03:20 AM, Mandy Chung wrote: > I implemented the support in the jar tool to transform a plain JAR file > to a modular JAR file [1]: > http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/modularize-jar/ > > Simple example: > To modularize an existing jar file (two ways - see below): > $ jar uf astro.jar -module org.astro at 2.0 > $ jar uf greetings.jar -C modules/com.greetings module-info.class > $ java -jar greetings.jar # launch in legacy mode > > Installing a modular JAR file in the module library > $ jmod install greetings.jar astro.jar -L mlib > > Run in module mode > $ java -L mlib -m com.greetings > > Details: > The jar tool provides a new option to transform a plain old JAR file > into a modular JAR: > -module {module-name at version} > Generate META-INF/module-info.class entry in the JAR file [a] > > It is designed to handle simple cases that: > * requires the default Java platform module [b] > * exports all public types > * main entry point as specified in the manifest or in the 'e' > flag, if any > * infers the dependencies from 'Class-Path' attribute if it is > a modular JAR file, and the specific module name of > that version will be added in the module dependency [c]. > > The jar tool will determine if the inputfiles include a > "module-info.class" entry that indicates that it is a modular > application. If exists, it will create a modular JAR by copying > the module-info.class file in the META-INF directory [d]. > The "-module" option is not required in this case. > > Initially I propose to use the GNU-style option (--module) but > it should really be a separate project to have the existing > command-line tools to support GNU-style options that deserves > its own JEP [2]. I like the option be a long-name option rather > than the traditional single letter option which I don't see such > inconsistency is an issue or find a good letter for it :) > > Since the generated module-info.class is typically fairly simple, > it doesn't seem really necessary to output the module-info.java > and so didn't implement that. > > Implementation: I use the classfile library that Jon Gibbons wrote > to generate the module-info.class. Since Jon will be updating > javac and classfile per the module-info spec [3], I modified > ModuleExport_attribute and ModuleRequires_attribute classes in > a simple way but the right change will come when they are updated > per the proposed spec. > > Footnotes: > [a] jarfile name tends to be unnamed or contains '-' character. > The module name will not be inferred from the jarfile name. > [b] Private and internal APIs in the jdk are not exported. > If the JAR file depends on any of these APIs and running > in module mode, this will result in a runtime error. > [c] If the path listed in the 'Class-Path' attribute is not a > modular JAR, it will simply issue a warning and continue. > Module dependencies may be incomplete in this case. > [d] If an application has been modularized, module-info.java that > describes the module definition is placed in the root of the > source tree and compiled in the same destination directory as > its classes. > > BTW, I'm working on adding a new regression test for this new > functionality. It looks like I have to write a better framework > to cover different test cases and do the validation that may take > some time. > > Thanks > Mandy > > [1] > http://mail.openjdk.java.net/pipermail/jigsaw-dev/2011-April/001251.html > [2] http://openjdk.java.net/jeps > [3] > http://openjdk.java.net/projects/jigsaw/doc/topics/modules-classfile.html From Alan.Bateman at oracle.com Mon Sep 12 07:19:04 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 12 Sep 2011 15:19:04 +0100 Subject: Legacy mode In-Reply-To: <4E6D80E2.4070803@oracle.com> References: <4E6CBFCE.1030608@univ-mlv.fr> <4E6CF20F.8070906@oracle.com> <4E6D0055.3010009@univ-mlv.fr> <4E6D0F03.6040902@oracle.com> <4E6D1694.30909@univ-mlv.fr> <4E6D1FF9.7020806@oracle.com> <4E6D27BB.2060705@univ-mlv.fr> <4E6D80E2.4070803@oracle.com> Message-ID: <4E6E14D8.6040501@oracle.com> David Holmes wrote: > > I was just thinking the same thing. It is extremely convenient to be > able to "override" individual class implementations using the > bootclasspath. It would have been far less convenient if > -Xbootclasspath had required you to use a jar rather than just a > class. The same will go for modulepath if we can only "override" > modules not individual classes within them. -Xbootclasspath/p will continue to work in legacy mode. When you don't have a JDK build environment then I agree it's convenient for testing. We don't have an exact equivalent in module mode. In module mode then arguably the module version should be rev'ed. Also which module is being patched (there may be multiple versions installed)? With modulepath then we will at least have a way to override a module without needing to install it. It could be hacked to augment the classes that are in the module library but I think the module-info would be required as otherwise you wouldn't know which module they belong to. -Alan. From mandy.chung at oracle.com Mon Sep 12 15:34:01 2011 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 12 Sep 2011 15:34:01 -0700 Subject: Review request: jar tool to transform a plain jar file to a modular jar In-Reply-To: <4E5E8D7E.1010906@oracle.com> References: <4E56F4FB.6060205@oracle.com> <4E580D72.70605@oracle.com> <4E5D724D.5050606@oracle.com> <4E5E8D7E.1010906@oracle.com> Message-ID: <4E6E88D9.6090702@oracle.com> Sherman, Alan, Thanks for the review. I did some further cleanup and here is an updated webrev: http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/modularize-jar.02/ My comments inlined below. On 09/12/11 6:06, Alan Bateman wrote: > > I'm looking at additions to java.util.jar.JarFile and I'm wondering if > the class or method descriptions should having wording to explain that > a JAR file can have a legacy manifest and also a module-info. I also > wonder if we will need to update the JAR specification too. Yes, the JAR spec needs to be updated but the JAR spec is in the pubs repo. I prefer to defer to update the docs later before we integrate to JDK 8 and track it in the bug database for the time being. > Minor comment is that the new methods should have @since 1.8. Fixed. > > A passing comment on SimpleLibrary lines 980-988 is that they could be > replaced with Files.write(is, md.toPath().resolve("info")); > Thanks for the suggestion. I think you meant Files.copy. > The sun.tools.jar.ModuleInfo class looks very useful and I'm sure we > will have other places that need to generate module-info classes. > Would it make sense to put it in org.openjdk.jigsaw? If so then it > would be good to have it support requires optional too. org.openjdk.jigsaw.** is included in the base module while sun.tools.jar.ModuleInfo is for tools to use and it depends on com.sun.tools.classfile library. Sorry I initially also thought that org.openjdk.jigsaw might be an appropriate package to put this class but it's not. I propose to leave it in sun.tools.jar package for now and move it to an appropriate package later. I have modified this class to support require optional and other modifiers. On 08/31/11 12:37, Xueming Shen wrote: > Comments on the sun.tools.jar.Main > > (1) #197-#201 > Shouldn't the "equals" be the "ignore case equals"? module-info.class is a jar entry whose name is case-sensitive. On the other hand, there is a bug in L628 that should use equals instead of equalsIgnoreCase method. I fixed that in the new webrev. > > (2) #202-#214 > The code might be better(?) organized as > if (moduled != null) { > if (mif != nuill) { > ... > return false; > } > .... > } > Did some additional cleanup too (see the webrev). > (3) It might be better to move the addModuleRequires() functionality into > ModuleInfo implementation? in which it can simply take a > Manifest object > and take whatever info it might need from the manifest to create > the > new ModuleInfo object. The sun.tools.jar.Main might not want to > know the > details of how to transfer the necessary from a Manifest to a > ModuleInfo. > For example, later you might find you would need an additional > attribute > from a existing Manifest, ideally you should be able to simply > update the > ModuleInfo implementation, without touching the > sun.tools.jar.Main (the > minfo.write(zos) will take care of the writing...) > In fact, I'd like to keep sun.tools.jar.ModuleInfo not to couple with Manifest. It's a helper class to generate module-info.class with the given module metadata. > (4) With the migration of new try-resource in getJarPath(), the code > might look > much better by simply do > > if (man == null) > return files; > ... > if (value == null) > return files > I tried to retain the existing structure except those which bug me :) I folded the three if-statements into 1. > And we probably don't need check if (rt != null) > new JarFile() should never return null, it will through an > IOException if it > fails. > Removed the check. Thanks Mandy From mandy.chung at oracle.com Mon Sep 12 15:44:57 2011 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 12 Sep 2011 15:44:57 -0700 Subject: Review request: jar tool to transform a plain jar file to a modularjar In-Reply-To: <4E6E1412.7090706@fh-landshut.de> References: <4E56F4FB.6060205@oracle.com> <4E6E1412.7090706@fh-landshut.de> Message-ID: <4E6E8B69.8020201@oracle.com> On 09/12/11 07:15, Michael Bien wrote: > any chance to add shortcuts to those common execution commands? > > $ java mlib com.greetings > as shortcut for > $ java -L mlib -m com.greetings > BTW, you don't need the -L option when the module is in the system module library. IMO, I don't think "java -m com.greetings" is not command-line friendly. "-m" option is a way to tell the launcher to run the application in module mode. With "-m" option, it'll launch the application in legacy mode as today. > or even > $ java mlib > if there is such thing like a default main method. > I think you meant the default module. This only saves to type the module name for one single module in a module library. > modules have great potential to make java commandline friendly. > It already does as you no longer need the classpath in modules :) Mandy From kasperni at gmail.com Tue Sep 13 04:27:09 2011 From: kasperni at gmail.com (Kasper Nielsen) Date: Tue, 13 Sep 2011 13:27:09 +0200 Subject: source bundles Message-ID: <4E6F3E0D.8010905@gmail.com> Hi, I couldn't find anything about in the documentation. But I was wondering if Jigsaw supports any kind of packaging of source code. Or will the JDK still have a single src.zip file? cheers Kasper From Alan.Bateman at oracle.com Tue Sep 13 10:32:42 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 13 Sep 2011 18:32:42 +0100 Subject: Review request: jar tool to transform a plain jar file to a modular jar In-Reply-To: <4E6E88D9.6090702@oracle.com> References: <4E56F4FB.6060205@oracle.com> <4E580D72.70605@oracle.com> <4E5D724D.5050606@oracle.com> <4E5E8D7E.1010906@oracle.com> <4E6E88D9.6090702@oracle.com> Message-ID: <4E6F93BA.1060307@oracle.com> Mandy Chung wrote: > > Yes, the JAR spec needs to be updated but the JAR spec is in the pubs > repo. I prefer to defer to update the docs later before we integrate > to JDK 8 and track it in the bug database for the time being. You're right, we just need to remember to go back to it later. > >> Minor comment is that the new methods should have @since 1.8. > > Fixed. MODULEINFO_NAME will need this too. >> >> A passing comment on SimpleLibrary lines 980-988 is that they could >> be replaced with Files.write(is, md.toPath().resolve("info")); >> > > Thanks for the suggestion. I think you meant Files.copy. Sorry yes, I meant Files.copy. > >> The sun.tools.jar.ModuleInfo class looks very useful and I'm sure we >> will have other places that need to generate module-info classes. >> Would it make sense to put it in org.openjdk.jigsaw? If so then it >> would be good to have it support requires optional too. > > org.openjdk.jigsaw.** is included in the base module while > sun.tools.jar.ModuleInfo is for tools to use and it depends on > com.sun.tools.classfile library. Sorry I initially also thought that > org.openjdk.jigsaw might be an appropriate package to put this class > but it's not. Okay for now but I think we're going to need this in other places soon to avoid duplicating the code. -Alan. From mandy.chung at oracle.com Tue Sep 13 15:16:01 2011 From: mandy.chung at oracle.com (mandy.chung at oracle.com) Date: Tue, 13 Sep 2011 22:16:01 +0000 Subject: hg: jigsaw/jigsaw/langtools: Tool support to modularize plain jar file Message-ID: <20110913221607.738364764F@hg.openjdk.java.net> Changeset: e4648740bad5 Author: mchung Date: 2011-09-13 15:13 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/e4648740bad5 Tool support to modularize plain jar file Summary: Add jar -I option to generate module-info.class entry Reviewed-by: alanb, sherman ! src/share/classes/com/sun/tools/classfile/ModuleExport_attribute.java ! src/share/classes/com/sun/tools/classfile/ModuleRequires_attribute.java From mandy.chung at oracle.com Tue Sep 13 15:16:31 2011 From: mandy.chung at oracle.com (mandy.chung at oracle.com) Date: Tue, 13 Sep 2011 22:16:31 +0000 Subject: hg: jigsaw/jigsaw/jdk: Tool support to modularize plain jar file Message-ID: <20110913221658.8A4EA47650@hg.openjdk.java.net> Changeset: bb6f08a993fa Author: mchung Date: 2011-09-13 15:14 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/bb6f08a993fa Tool support to modularize plain jar file Summary: Add jar -I option to generate module-info.class entry Reviewed-by: alanb, sherman ! src/share/classes/java/util/jar/JarFile.java ! src/share/classes/org/openjdk/jigsaw/SimpleLibrary.java ! src/share/classes/sun/tools/jar/Main.java + src/share/classes/sun/tools/jar/ModuleInfo.java ! src/share/classes/sun/tools/jar/resources/jar.properties ! test/org/openjdk/jigsaw/cli/signed-modular-jar.sh ! test/org/openjdk/jigsaw/hello-jar.sh From sean.mullan at oracle.com Wed Sep 14 12:08:00 2011 From: sean.mullan at oracle.com (Sean Mullan) Date: Wed, 14 Sep 2011 15:08:00 -0400 Subject: Code Review Request for 6953295 and related changes to add keytool to jdk.tools.base Message-ID: <4E70FB90.7070702@oracle.com> Requesting review from Mandy and Max, others welcome. CR: http://monaco.us.oracle.com/detail.jsf?cr=6953295 Webrev: http://cr.openjdk.java.net/~mullan/jigsaw/webrevs/6953295/webrev.00/ Additional notes to reviewers: * I will push the non-module related changes to JDK 8 first. I wanted to first make sure everything worked with the module changes. * The following classes have been moved to the sun.security.pkcs10 package to remove dependencies on the sun.jsse module: sun.security.pkcs.PKCS10 sun.security.pkcs.PKCS10Attribute sun.security.pkcs.PKCS10Attributes and these classes have been moved to the sun.security.tools package: sun.security.x509.CertAndKeyGen sun.security.util.PathList The code is the same other than changes due to the package rename. * The following classes have been removed since nothing in the JDK uses them: sun.security.pkcs.EncodingException sun.security.util.BigInt The test/sun/security/util/BigInt/BigIntEqualsHashCode.java was also removed as it is now N/A. * Should I also include jarsigner in jdk.tools.base? I think it would be useful for creating signed modular jars. Thanks, Sean From sean.mullan at oracle.com Wed Sep 14 12:16:12 2011 From: sean.mullan at oracle.com (Sean Mullan) Date: Wed, 14 Sep 2011 15:16:12 -0400 Subject: Code Review Request for 6953295 and related changes to add keytool to jdk.tools.base In-Reply-To: <4E70FB90.7070702@oracle.com> References: <4E70FB90.7070702@oracle.com> Message-ID: <4E70FD7C.1030000@oracle.com> On 9/14/11 3:08 PM, Sean Mullan wrote: > Requesting review from Mandy and Max, others welcome. > > CR: http://monaco.us.oracle.com/detail.jsf?cr=6953295 Oops: should be http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6953295 > Webrev: http://cr.openjdk.java.net/~mullan/jigsaw/webrevs/6953295/webrev.00/ > > Additional notes to reviewers: > > * I will push the non-module related changes to JDK 8 first. I wanted to first > make sure everything worked with the module changes. > > * The following classes have been moved to the sun.security.pkcs10 package to > remove dependencies on the sun.jsse module: > > sun.security.pkcs.PKCS10 > sun.security.pkcs.PKCS10Attribute > sun.security.pkcs.PKCS10Attributes > > and these classes have been moved to the sun.security.tools package: > > sun.security.x509.CertAndKeyGen > sun.security.util.PathList > > The code is the same other than changes due to the package rename. > > * The following classes have been removed since nothing in the JDK uses them: > > sun.security.pkcs.EncodingException > sun.security.util.BigInt > > The test/sun/security/util/BigInt/BigIntEqualsHashCode.java was also removed as > it is now N/A. > > * Should I also include jarsigner in jdk.tools.base? I think it would be useful > for creating signed modular jars. > > Thanks, > Sean > From mandy.chung at oracle.com Wed Sep 14 16:14:05 2011 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 14 Sep 2011 16:14:05 -0700 Subject: Code Review Request for 6953295 and related changes to add keytool to jdk.tools.base In-Reply-To: <4E70FB90.7070702@oracle.com> References: <4E70FB90.7070702@oracle.com> Message-ID: <4E71353D.7070807@oracle.com> On 09/14/11 12:08, Sean Mullan wrote: > Requesting review from Mandy and Max, others welcome. > > CR: http://monaco.us.oracle.com/detail.jsf?cr=6953295 > Webrev: http://cr.openjdk.java.net/~mullan/jigsaw/webrevs/6953295/webrev.00/ > KeyTool.java L2079, 2336: keytool -sslserver option and -file ldap://.... option are now optional. Should keytool output an error if these options are used but class not found? Such detection can be done at startup. L2073, 2330: Besides these places, I also found that sun.security.provider.certpath.URICertStore.LDAP does similar work. Would it be better to add a factory class that returns the CertStoreHelper of a given type if present or null if not (e.g. sun.security.provider.certpath.CertStoreHelperFactory?) ? This would avoid hardcoding the implementation class name (as a string) in many places. L2064-2323: there are a couple of places that can be updated with try-with-resource. Since you're in this file, it might be good to modernize part of the existing code (nice to see the change to use the foreach) :) L2344: CertStore.getCertifications never returns null. Looks like L2355-2362 is to handle the case at L2347. If that's the case, can you move L2356-2361 to L2347? > Additional notes to reviewers: > > * I will push the non-module related changes to JDK 8 first. I wanted to first > make sure everything worked with the module changes. > Thanks. Any change for jdk modularization not depending on jigsaw will be in jdk 8 repos where other jdk8 development works are based on. > * The following classes have been moved to the sun.security.pkcs10 package to > remove dependencies on the sun.jsse module: > > sun.security.pkcs.PKCS10 > sun.security.pkcs.PKCS10Attribute > sun.security.pkcs.PKCS10Attributes > > and these classes have been moved to the sun.security.tools package: > > sun.security.x509.CertAndKeyGen > sun.security.util.PathList > > The code is the same other than changes due to the package rename. > > * The following classes have been removed since nothing in the JDK uses them: > > sun.security.pkcs.EncodingException > sun.security.util.BigInt > We all like removing dead code! > The test/sun/security/util/BigInt/BigIntEqualsHashCode.java was also removed as > it is now N/A. > > * Should I also include jarsigner in jdk.tools.base? I think it would be useful > for creating signed modular jars. > I think it's useful too. With this fix, it should be fine including jarsigner in the jdk.tools.base module. It was not included previously because of its dependency on KeyTool that in turn depends on jsse and jndi. jdk.tools.base is also intended to aggregate the developer tools that are needed for module development with only the base module so that it can be installed at once rather than installing each one individually. Mandy From mandy.chung at oracle.com Thu Sep 15 08:17:27 2011 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 15 Sep 2011 08:17:27 -0700 Subject: Review request on the optional modules support In-Reply-To: <4E681C62.1080400@oracle.com> References: <20110907215841.87EE12C40@eggemoggin.niobe.net> <4E681C62.1080400@oracle.com> Message-ID: <4E721707.5050208@oracle.com> I modified the code per Mark's review comments. Updated webrev at: http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/optional-modules.02/ * Move isModulePresent and requireModulePresent methods to ModuleClassLoader. Also add the convenience methods in java.lang.Class. * Class.getClassLoader for system classes returns the boot loader in module mode and returns null in legacy mode. * Removed BootLoader.isModulePresent override. The base and boot modules are in the same context and such override is not needed. Mandy From mandy.chung at oracle.com Thu Sep 15 13:32:29 2011 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 15 Sep 2011 13:32:29 -0700 Subject: Review request on the optional modules support In-Reply-To: <4E721707.5050208@oracle.com> References: <20110907215841.87EE12C40@eggemoggin.niobe.net> <4E681C62.1080400@oracle.com> <4E721707.5050208@oracle.com> Message-ID: <4E7260DD.6050205@oracle.com> On 09/15/11 08:17, Mandy Chung wrote: > > * Class.getClassLoader for system classes returns the boot > loader in module mode and returns null in legacy mode. > I would have to back out this Class.getClassLoader change because this requires more analysis and modification to the implementation. I'll look into this together with the legacy mode support. With more testing and investigation, there are places in the JDK that rely on null class loader for its implementation details. Also there is bootstrapping issue that we have to determine when the boot loader can be initialized. Final webrev: http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/optional-modules.03/ Mandy From mandy.chung at oracle.com Thu Sep 15 13:48:27 2011 From: mandy.chung at oracle.com (mandy.chung at oracle.com) Date: Thu, 15 Sep 2011 20:48:27 +0000 Subject: hg: jigsaw/jigsaw/jdk: 3 new changesets Message-ID: <20110915204927.016AC476FC@hg.openjdk.java.net> Changeset: 3a2fddb305f1 Author: mchung Date: 2011-09-13 17:28 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/3a2fddb305f1 Skip if it's same context when resolving remote suppliers ! src/share/classes/org/openjdk/jigsaw/Linker.java ! src/share/classes/org/openjdk/jigsaw/PathLinker.java ! test/org/openjdk/jigsaw/_Configurator.java Changeset: 27b52974710b Author: mchung Date: 2011-09-13 17:40 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/27b52974710b Minor update to modules.config and update ClassAnalyzer for application to use Reviewed-by: alanb ! make/modules/Makefile ! make/modules/modules.config ! make/modules/modules.group ! make/modules/modules.properties + make/tools/classanalyzer/classanalyzer.html ! make/tools/classanalyzer/src/com/sun/classanalyzer/AnnotationParser.java ! make/tools/classanalyzer/src/com/sun/classanalyzer/BootAnalyzer.java ! make/tools/classanalyzer/src/com/sun/classanalyzer/ClassAnalyzer.java ! make/tools/classanalyzer/src/com/sun/classanalyzer/ClassFileParser.java ! make/tools/classanalyzer/src/com/sun/classanalyzer/ClassListReader.java ! make/tools/classanalyzer/src/com/sun/classanalyzer/ClassPath.java ! make/tools/classanalyzer/src/com/sun/classanalyzer/Files.java + make/tools/classanalyzer/src/com/sun/classanalyzer/JigsawModuleBuilder.java ! make/tools/classanalyzer/src/com/sun/classanalyzer/Modularizer.java ! make/tools/classanalyzer/src/com/sun/classanalyzer/Module.java ! make/tools/classanalyzer/src/com/sun/classanalyzer/ModuleBuilder.java ! make/tools/classanalyzer/src/com/sun/classanalyzer/ModuleConfig.java ! make/tools/classanalyzer/src/com/sun/classanalyzer/ModuleInfo.java + make/tools/classanalyzer/src/com/sun/classanalyzer/PackageInfo.java ! make/tools/classanalyzer/src/com/sun/classanalyzer/PlatformModuleBuilder.java ! make/tools/classanalyzer/src/com/sun/classanalyzer/ResourceFile.java Changeset: 50f24359f688 Author: mchung Date: 2011-09-15 13:46 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/50f24359f688 Optional modules support Reviewed-by: alanb, mr * API to test if an optional module is present * RequireOptionalModule annotation to mark optional method ! make/common/Defs-modules.gmk ! make/java/java/FILES_java.gmk ! make/modules/modules.group ! make/modules/modules.properties ! make/tools/classanalyzer/src/com/sun/classanalyzer/BootAnalyzer.java ! make/tools/classanalyzer/src/com/sun/classanalyzer/ClassAnalyzer.java ! make/tools/classanalyzer/src/com/sun/classanalyzer/JigsawModuleBuilder.java ! make/tools/classanalyzer/src/com/sun/classanalyzer/Module.java ! make/tools/classanalyzer/src/com/sun/classanalyzer/ModuleBuilder.java ! make/tools/classanalyzer/src/com/sun/classanalyzer/ModuleInfo.java ! make/tools/classanalyzer/src/com/sun/classanalyzer/PackageInfo.java ! make/tools/classanalyzer/src/com/sun/classanalyzer/PlatformModuleBuilder.java ! make/tools/classanalyzer/src/com/sun/classanalyzer/ResourceFile.java ! src/share/classes/java/lang/Class.java ! src/share/classes/java/lang/SecurityManager.java ! src/share/classes/java/lang/module/ModuleClassLoader.java + src/share/classes/java/lang/module/ModuleNotPresentException.java + src/share/classes/java/lang/module/RequireOptionalModule.java ! src/share/classes/java/lang/module/UnsupportedElementTypeException.java ! src/share/classes/java/util/Properties.java ! src/share/classes/java/util/prefs/AbstractPreferences.java ! src/share/classes/java/util/prefs/Preferences.java ! src/share/classes/org/openjdk/jigsaw/BootLoader.java ! src/share/classes/org/openjdk/jigsaw/Context.java ! src/share/classes/org/openjdk/jigsaw/Launcher.java ! src/share/classes/org/openjdk/jigsaw/Linker.java ! src/share/classes/org/openjdk/jigsaw/Loader.java ! src/share/classes/org/openjdk/jigsaw/LoaderPool.java ! src/share/classes/org/openjdk/jigsaw/SimpleLibrary.java + test/org/openjdk/jigsaw/hello-optional.sh + test/org/openjdk/jigsaw/optional-base.sh ! test/org/openjdk/jigsaw/optional-deps.sh + test/org/openjdk/jigsaw/optional-jaxp.sh + test/org/openjdk/jigsaw/optional-reexport.sh + test/org/openjdk/jigsaw/properties.xml ! test/org/openjdk/jigsaw/tester.sh From Alan.Bateman at oracle.com Mon Sep 19 05:09:36 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 19 Sep 2011 13:09:36 +0100 Subject: Downloading and installing maven artifacts as modules Message-ID: <4E773100.9080109@oracle.com> The migration and interop section of the requirements document has a requirement to be able to download, install and resolve Maven artifacts as if they were modules. I've hacked up a prototype, as a demo module, that downloads artifacts and their transitive dependencies and installs them as modules. The idea is to make it easy to use popular libraries as modules and to tease out the issues that such interop would bring. The webrev is here: http://cr.openjdk.java.net/~alanb/MavenDemo/webrev.01/ The README.txt file has the details on how it used and lists some of the issues. For now, the code that we use in the jar tool to generate module-info classes is duplicated until we figure out whether it would be desirable to have a supported API for generating module-info classes (Mandy and I have been chatting off-line about the number of places in the JDK where we generate class files). As with generating a module-info for a JAR file then it is has to assume that all public types are exported. -Alan. From sean.mullan at oracle.com Tue Sep 20 14:44:18 2011 From: sean.mullan at oracle.com (Sean Mullan) Date: Tue, 20 Sep 2011 17:44:18 -0400 Subject: Code Review Request for 6953295 and related changes to add keytool to jdk.tools.base In-Reply-To: <4E71353D.7070807@oracle.com> References: <4E70FB90.7070702@oracle.com> <4E71353D.7070807@oracle.com> Message-ID: <4E790932.4000203@oracle.com> New webrev: http://cr.openjdk.java.net/~mullan/jigsaw/webrevs/6953295/webrev.01/ Comments on your comments below - On 9/14/11 7:14 PM, Mandy Chung wrote: > KeyTool.java > L2079, 2336: keytool -sslserver option and -file ldap://.... option > are now optional. Should keytool output an error if these options are > used but class not found? Such detection can be done at startup. It will output an error if the classes are not found indicating that SSL or LDAP was not available. I could probably move the check earlier but I'm not sure that is going to make much of a noticeable difference. > L2073, 2330: Besides these places, I also found that > sun.security.provider.certpath.URICertStore.LDAP does similar work. > Would it be better to add a factory class that returns the > CertStoreHelper of a given type if present or null if not (e.g. > sun.security.provider.certpath.CertStoreHelperFactory?) ? This would > avoid hardcoding the implementation class name (as a string) in many places. Good point. I have added a getInstance method to CertStoreHelper and it will load the implementations on demand and store them as SoftReferences. > L2064-2323: there are a couple of places that can be updated with > try-with-resource. Since you're in this file, it might be good to > modernize part of the existing code (nice to see the change to use the > foreach) :) Yes, I converted as many as I could to try-with-resource although there were a few with some complicated logic that I left alone for now. I also converted a few others in URICertStore and also changed some others to use multi-catch. > L2344: CertStore.getCertifications never returns null. Looks like > L2355-2362 is to handle the case at L2347. If that's the case, can you > move L2356-2361 to L2347? Good catch - you caught a bug in my code. Actually, an SSL server may return a null or empty chain, and I want to preserve the current behavior of KeyTool which treats this as an error. So I've modified the SSLServerCertStore to return an empty chain in this case. >> * The following classes have been moved to the sun.security.pkcs10 package to >> remove dependencies on the sun.jsse module: >> >> sun.security.pkcs.PKCS10 >> sun.security.pkcs.PKCS10Attribute >> sun.security.pkcs.PKCS10Attributes >> >> and these classes have been moved to the sun.security.tools package: >> >> sun.security.x509.CertAndKeyGen >> sun.security.util.PathList In the previous webrev, I used "hg remove" and "hg add" to move these files. I reverted those changes and used "hg rename" instead, so the diffs for these files should be much easier to review. >> * Should I also include jarsigner in jdk.tools.base? I think it would be useful >> for creating signed modular jars. >> > > I think it's useful too. With this fix, it should be fine including > jarsigner in the jdk.tools.base module. It was not included previously > because of its dependency on KeyTool that in turn depends on jsse and jndi. I have modified the module config files to include jarsigner in jdk.tools.base. --Sean From jesse.glick at oracle.com Thu Sep 22 08:04:55 2011 From: jesse.glick at oracle.com (Jesse Glick) Date: Thu, 22 Sep 2011 11:04:55 -0400 Subject: Legacy mode In-Reply-To: <4E6E14D8.6040501@oracle.com> References: <4E6CBFCE.1030608@univ-mlv.fr> <4E6CF20F.8070906@oracle.com> <4E6D0055.3010009@univ-mlv.fr> <4E6D0F03.6040902@oracle.com> <4E6D1694.30909@univ-mlv.fr> <4E6D1FF9.7020806@oracle.com> <4E6D27BB.2060705@univ-mlv.fr> <4E6D80E2.4070803@oracle.com> <4E6E14D8.6040501@oracle.com> Message-ID: <4E7B4E97.9060609@oracle.com> On 09/12/2011 10:19 AM, Alan Bateman wrote: > We don't have an exact equivalent [to -Xbootclasspath/p] in module mode. FWIW - in the NetBeans module system there is a convention that for a module $dir/$symbname.jar you can create $dir/patches/$symbname/*.jar whose contents can override the classes in the module JAR. This makes it straightforward to temporarily patch a class or two, for testing purposes or a "hot fix", without rebuilding the module. The (immediate) parent directory of the patch JAR indicates which module is being patched; the patch JAR basename can be anything you like, making it easy to create multiple patches for different purposes (e.g. two different hot fixes). Obviously some details would have to be different for use with JDK modules. You could imagine -Xmodulepatches:/tmp/patches where /tmp/patches/jdk.nio/1.0/java/nio/Buffer.class would override java.nio.Buffer in jdk.nio at 1.0. (This is assuming that the module system forbids >1 copy of the same version of the same module to be loaded at once, unlike the upcoming OSGi spec if I understand correctly; or at least that such a situation is rare and -Xmodulepatches need not support it.) From mandy.chung at oracle.com Mon Sep 26 19:35:56 2011 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 26 Sep 2011 19:35:56 -0700 Subject: Downloading and installing maven artifacts as modules In-Reply-To: <4E773100.9080109@oracle.com> References: <4E773100.9080109@oracle.com> Message-ID: <4E81368C.40009@oracle.com> On 09/19/11 05:09, Alan Bateman wrote: > > The migration and interop section of the requirements document has a > requirement to be able to download, install and resolve Maven > artifacts as if they were modules. I've hacked up a prototype, as a > demo module, that downloads artifacts and their transitive > dependencies and installs them as modules. The idea is to make it easy > to use popular libraries as modules and to tease out the issues that > such interop would bring. The webrev is here: > > http://cr.openjdk.java.net/~alanb/MavenDemo/webrev.01/ > I went through the webrev which looks good as a demo. It's okay for now to install this module demo as part of make/modules in the post-processing step. The modules build will be clean up at some point in time and include to build the demo/sample modules in the proper place. Also this demo might be just temporary and could be replaced by the future work. Some comments below. README.txt [-L library] option is an option to the demo.MavenImport. Since demo.MavenImport a module and it can be installed in a module library, it would be confusing: $ java -L mlib -m demo.MavenImport -L maven-module-lib where mlib is the library where demo.MavenImport is installed and maven-module-lib is the library to install the modules imported from maven into. Main.java L48: typo "comamnd-line" should be "command-line" L140: You can call Library.systemLibraryPath() or Library.openSystemLibrary()| to get the system module library rather than hardcoding the pathname. | |Pom.java| L190: I think it's a typo {@code jar}? Mandy From mandy.chung at oracle.com Mon Sep 26 19:49:29 2011 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 26 Sep 2011 19:49:29 -0700 Subject: Simple patch to fix building debian package Message-ID: <4E8139B9.5090400@oracle.com> jpkg fails to create the debian package for the JDK modules because it uses '_' which is no longer a valid character in the debian version. This patch simply fixes to return the jigsaw version and also checks for valid characters and throws exception if the jigsaw version was changed to use any invalid character. In addition, only include the mandatory dependencies in the Depend statements for the debian package. Webrev: http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/jigsaw-deb-pkgs/ Mandy From jesse.glick at oracle.com Tue Sep 27 05:12:48 2011 From: jesse.glick at oracle.com (Jesse Glick) Date: Tue, 27 Sep 2011 08:12:48 -0400 Subject: Downloading and installing maven artifacts as modules In-Reply-To: <4E81368C.40009@oracle.com> References: <4E773100.9080109@oracle.com> <4E81368C.40009@oracle.com> Message-ID: <4E81BDC0.5080400@oracle.com> On 09/19/11 05:09, Alan Bateman wrote: > http://cr.openjdk.java.net/~alanb/MavenDemo/webrev.01/ If this were to be more than a demo, you would not want to recreate the logic for parsing Maven metadata, but rather call into the official embedding APIs. I do not think you want to interpret zip; does such a packaging even exist? (Various named packagings produce *.zip artifacts which are usually software distributions, i.e. contain JARs and other files as entries.) From Alan.Bateman at oracle.com Tue Sep 27 05:47:26 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 27 Sep 2011 13:47:26 +0100 Subject: Downloading and installing maven artifacts as modules In-Reply-To: <4E81BDC0.5080400@oracle.com> References: <4E773100.9080109@oracle.com> <4E81368C.40009@oracle.com> <4E81BDC0.5080400@oracle.com> Message-ID: <4E81C5DE.6030908@oracle.com> Jesse Glick wrote: > > If this were to be more than a demo, you would not want to recreate > the logic for parsing Maven metadata, but rather call into the > official embedding APIs. > > I do not think you want to interpret zip; does > such a packaging even exist? (Various named packagings produce *.zip > artifacts which are usually software distributions, i.e. contain JARs > and other files as entries.) This is just a demo for now, it's purpose being to tease out the issues that we would have if we were to extend the repository support to allow a module library be associated with a maven repository. Clearly we don't want to be parsing maven metadata but it's done this way for now to avoid the dependency. -Alan From brian at pontarelli.com Tue Sep 27 08:06:20 2011 From: brian at pontarelli.com (Brian Pontarelli) Date: Tue, 27 Sep 2011 09:06:20 -0600 Subject: Downloading and installing maven artifacts as modules In-Reply-To: <4E81C5DE.6030908@oracle.com> References: <4E773100.9080109@oracle.com> <4E81368C.40009@oracle.com> <4E81BDC0.5080400@oracle.com> <4E81C5DE.6030908@oracle.com> Message-ID: On Sep 27, 2011, at 6:47 AM, Alan Bateman wrote: > Jesse Glick wrote: >> >> If this were to be more than a demo, you would not want to recreate the logic for parsing Maven metadata, but rather call into the official embedding APIs. >> >> I do not think you want to interpret zip; does such a packaging even exist? (Various named packagings produce *.zip artifacts which are usually software distributions, i.e. contain JARs and other files as entries.) > This is just a demo for now, it's purpose being to tease out the issues that we would have if we were to extend the repository support to allow a module library be associated with a maven repository. Clearly we don't want to be parsing maven metadata but it's done this way for now to avoid the dependency. I've been sporadically following Jigsaw but I saw this thread and have recently been discussing a similar topic with some folks. The topic we have been discussing is the meta-data needed for pure dependency management compared to the meta-data provided by Maven, Jigsaw, Ivy, Savant and others. I wanted to bounce this idea off of this list as well as other Maven, Ivy and others. The idea would be to create an open standard format that defines the meta-data needed to do pure dependency management. This has a number of key benefits including: - Standardize dependency repositories - Clean dependency definitions - Simple way for tool providers to work with dependencies separate from the build and runtime systems - Easier migration between build tools (pick the right tool for the job not the repository or dependency management system) There are probably other benefits as well. I wanted to get your thoughts this concept. It looks like Jigsaw intends to support POMs and potentially support some of OSGi, but at a baseline, there are a number of concepts that can be defined and shared across all these projects. -bp From david.lloyd at redhat.com Tue Sep 27 09:14:12 2011 From: david.lloyd at redhat.com (David M. Lloyd) Date: Tue, 27 Sep 2011 11:14:12 -0500 Subject: Downloading and installing maven artifacts as modules In-Reply-To: References: <4E773100.9080109@oracle.com> <4E81368C.40009@oracle.com> <4E81BDC0.5080400@oracle.com> <4E81C5DE.6030908@oracle.com> Message-ID: <4E81F654.9080906@redhat.com> On 09/27/2011 10:06 AM, Brian Pontarelli wrote: > > On Sep 27, 2011, at 6:47 AM, Alan Bateman wrote: > >> Jesse Glick wrote: >>> >>> If this were to be more than a demo, you would not want to recreate the logic for parsing Maven metadata, but rather call into the official embedding APIs. >>> >>> I do not think you want to interpretzip; does such a packaging even exist? (Various named packagings produce *.zip artifacts which are usually software distributions, i.e. contain JARs and other files as entries.) >> This is just a demo for now, it's purpose being to tease out the issues that we would have if we were to extend the repository support to allow a module library be associated with a maven repository. Clearly we don't want to be parsing maven metadata but it's done this way for now to avoid the dependency. > > I've been sporadically following Jigsaw but I saw this thread and have recently been discussing a similar topic with some folks. The topic we have been discussing is the meta-data needed for pure dependency management compared to the meta-data provided by Maven, Jigsaw, Ivy, Savant and others. > > I wanted to bounce this idea off of this list as well as other Maven, Ivy and others. The idea would be to create an open standard format that defines the meta-data needed to do pure dependency management. This has a number of key benefits including: > > - Standardize dependency repositories > - Clean dependency definitions > - Simple way for tool providers to work with dependencies separate from the build and runtime systems > - Easier migration between build tools (pick the right tool for the job not the repository or dependency management system) > > There are probably other benefits as well. I wanted to get your thoughts this concept. It looks like Jigsaw intends to support POMs and potentially support some of OSGi, but at a baseline, there are a number of concepts that can be defined and shared across all these projects. Not only this, but the Maven POM/repository format has many known inadequacies as the requirements for build dependencies are substantially different from the requirements for a runtime module environment. Establishing a base set of requirements with community input is a key step to doing this right. -- - DML From Alan.Bateman at oracle.com Tue Sep 27 10:03:41 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 27 Sep 2011 18:03:41 +0100 Subject: Simple patch to fix building debian package In-Reply-To: <4E8139B9.5090400@oracle.com> References: <4E8139B9.5090400@oracle.com> Message-ID: <4E8201ED.7050904@oracle.com> Mandy Chung wrote: > jpkg fails to create the debian package for the JDK modules > because it uses '_' which is no longer a valid character > in the debian version. > > This patch simply fixes to return the jigsaw version and > also checks for valid characters and throws exception if > the jigsaw version was changed to use any invalid character. > In addition, only include the mandatory dependencies in > the Depend statements for the debian package. > > Webrev: > http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/jigsaw-deb-pkgs/ > > Mandy > Thanks for fixing this, I've been building lately with BUILD_PACKAGES=false because of this. The changes look okay to me. I think computeProvides can also be changed to deps.substring(2). -Alan From brian at pontarelli.com Tue Sep 27 12:29:05 2011 From: brian at pontarelli.com (Brian Pontarelli) Date: Tue, 27 Sep 2011 13:29:05 -0600 Subject: Downloading and installing maven artifacts as modules In-Reply-To: <4E81F654.9080906@redhat.com> References: <4E773100.9080109@oracle.com> <4E81368C.40009@oracle.com> <4E81BDC0.5080400@oracle.com> <4E81C5DE.6030908@oracle.com> <4E81F654.9080906@redhat.com> Message-ID: <9F1AA9AA-13EC-48B7-A2BF-9623F27BE0A3@pontarelli.com> On Sep 27, 2011, at 10:14 AM, David M. Lloyd wrote: > On 09/27/2011 10:06 AM, Brian Pontarelli wrote: >> >> On Sep 27, 2011, at 6:47 AM, Alan Bateman wrote: >> >>> Jesse Glick wrote: >>>> >>>> If this were to be more than a demo, you would not want to recreate the logic for parsing Maven metadata, but rather call into the official embedding APIs. >>>> >>>> I do not think you want to interpretzip; does such a packaging even exist? (Various named packagings produce *.zip artifacts which are usually software distributions, i.e. contain JARs and other files as entries.) >>> This is just a demo for now, it's purpose being to tease out the issues that we would have if we were to extend the repository support to allow a module library be associated with a maven repository. Clearly we don't want to be parsing maven metadata but it's done this way for now to avoid the dependency. >> >> I've been sporadically following Jigsaw but I saw this thread and have recently been discussing a similar topic with some folks. The topic we have been discussing is the meta-data needed for pure dependency management compared to the meta-data provided by Maven, Jigsaw, Ivy, Savant and others. >> >> I wanted to bounce this idea off of this list as well as other Maven, Ivy and others. The idea would be to create an open standard format that defines the meta-data needed to do pure dependency management. This has a number of key benefits including: >> >> - Standardize dependency repositories >> - Clean dependency definitions >> - Simple way for tool providers to work with dependencies separate from the build and runtime systems >> - Easier migration between build tools (pick the right tool for the job not the repository or dependency management system) >> >> There are probably other benefits as well. I wanted to get your thoughts this concept. It looks like Jigsaw intends to support POMs and potentially support some of OSGi, but at a baseline, there are a number of concepts that can be defined and shared across all these projects. > > Not only this, but the Maven POM/repository format has many known inadequacies as the requirements for build dependencies are substantially different from the requirements for a runtime module environment. Establishing a base set of requirements with community input is a key step to doing this right. I agree. There are a number of concepts that some systems lack and others have, both around compile-time and runtime dependency management. A few ones that come to mind are: - Version compile-time compatibility - Version runtime compatibility - Version ranges - Code meta-data changes (group-id, package, ownership, etc) - Repository changes (re-location, etc) - Public and private code Depending on the level of interest, I'd be willing to reach out to the Maven and Ivy developers and work on putting together an expert group for this. -bp From eric at tibco.com Tue Sep 27 12:36:07 2011 From: eric at tibco.com (Eric Johnson) Date: Tue, 27 Sep 2011 21:36:07 +0200 Subject: Downloading and installing maven artifacts as modules In-Reply-To: References: <4E773100.9080109@oracle.com> <4E81368C.40009@oracle.com> <4E81BDC0.5080400@oracle.com> <4E81C5DE.6030908@oracle.com> Message-ID: <4E8225A7.3090408@tibco.com> Hi Brian, On 9/27/11 5:06 PM, Brian Pontarelli wrote: > I've been sporadically following Jigsaw but I saw this thread and have recently been discussing a similar topic with some folks. The topic we have been discussing is the meta-data needed for pure dependency management compared to the meta-data provided by Maven, Jigsaw, Ivy, Savant and others. > > I wanted to bounce this idea off of this list as well as other Maven, Ivy and others. The idea would be to create an open standard format that defines the meta-data needed to do pure dependency management. This has a number of key benefits including: > > - Standardize dependency repositories > - Clean dependency definitions > - Simple way for tool providers to work with dependencies separate from the build and runtime systems > - Easier migration between build tools (pick the right tool for the job not the repository or dependency management system) > > There are probably other benefits as well. I wanted to get your thoughts this concept. It looks like Jigsaw intends to support POMs and potentially support some of OSGi, but at a baseline, there are a number of concepts that can be defined and shared across all these projects. If that is something that gets taken up by this group, I feel compelled to throw in my several cents worth. a) Redundant metadata - extremely bad. Ideally, as much of the run-time dependency information should be used for build-time dependency resolution as possible. Only introduce build-time dependency metadata as needed, when it goes above and beyond what the run time data says. At my company, we've struggled with extra meta data driven by the use of an Eclipse-like notion of a feature, and the OSGi-level meta-data as well. This means that they can get out of sync, and do, and that can lead to nasty, annoying and subtle issues. b) Build against the supported minimum. I think of this as the JDK 1.4 problem. JDK 1.4 introduced "StringBuffer.append(StringBuffer sb);" You could take the same code and compile against both JDK 1.4 & 1.3. However, the version compiled against 1.4 would only run against 1.4, whereas the JDK 1.3 compiled version would run against both. What with auto-boxing, and newer language features, the opportunities for such mistakes are legion. So, if you want to assert that you *run* against version 2.3.0 and later of a dependency (be it a package, bundle, or module), you better make sure you build against that version. c) Follow-on to (b) - sometimes the dependencies you state simply aren't consistent. If A depends on B & C, and B also depends on C, you might be in a spot where B requires a newer version of C than A requires. The correct thing to do here is flag a warning that the dependency that A states on C is actually a lie, and its lower end must be bumped. d) What you need to build != what you need to run. For example, in OSGi terms, my bundle may require, at runtime, the ability to log. The API for the logging service may be fixed, but an implementation of that API must also be available. At build time, I don't need the implementation. In fact, if my build process does require an implementation, the one it grabs may trigger the problem indicated by (c), thus either incorrectly building against a newer version of some other API, or suggesting my bundle must raise its minimum dependencies unnecessarily. e) What you need to run is a subset of what you need to test. Once we start specifying ranges for dependencies, we really should be making sure that at runtime we test against the lowest version of that dependency, the highest stable version, and any future versions in development. And that's just a minimum. In practice I suspect most people frequently shortcut this notion by continually raising dependency minimums, and forgoing testing of future versions until those future versions are just about declared stable. Circling back to the notion that "there are a number of concepts that can be shared across all of these projects," I approach such an assertion with considerable trepidation. At some point you can boil this all down to a simple modeling notion of capabilities and requirements (as in code X provides capability, code Y requires some capability). However, we need to add modifiers to the predicates here. For example: * Version ranges * Security assertions (FIPS certified) * # of suppliers (I need > 1 implementation of service foo, I need exactly one supplier of service foo) * vendor * License (unless you intend to leave the open-source world out) So, now, what you've got is a system that involves an arbitrary set of subjects and objects, and an arbitrary set of predicates that combine those two sets. That's right, you've got RDF. To the extent that you standardize an RDF-like approach, it only works if everyone agrees on the meanings of some core set of subjects, predicates, and objects. And keep in mind, the more subjects and predicates you introduce, the more complexity you get. Having said all that, yes, please, let's have at it, and finally try to establish what dependencies mean in the Java space, so that the platform can further thrive. -Eric. From brian at pontarelli.com Tue Sep 27 13:23:25 2011 From: brian at pontarelli.com (Brian Pontarelli) Date: Tue, 27 Sep 2011 14:23:25 -0600 Subject: Downloading and installing maven artifacts as modules In-Reply-To: <4E8225A7.3090408@tibco.com> References: <4E773100.9080109@oracle.com> <4E81368C.40009@oracle.com> <4E81BDC0.5080400@oracle.com> <4E81C5DE.6030908@oracle.com> <4E8225A7.3090408@tibco.com> Message-ID: <90EB4482-AE0B-4752-A986-0BEAEFEE70F2@pontarelli.com> Thanks for the feedback Eric. I also have a ton of ideas and thoughts as well. I think we need to first get buy-in from the community and the numerous projects that exist and then there will certainly be a lengthy discussion to iron out a lot of these concepts. I take it you are on board and that you want to help push this forward as you can. If Jigsaw members are on board and willing then perhaps it would be time to open a discussion with other in the community and start an email list to start discussions. Not knowing the inner workings of the Jigsaw project and team, I have to ask, are we there yet? -bp On Sep 27, 2011, at 1:36 PM, Eric Johnson wrote: > Hi Brian, > > On 9/27/11 5:06 PM, Brian Pontarelli wrote: >> I've been sporadically following Jigsaw but I saw this thread and have recently been discussing a similar topic with some folks. The topic we have been discussing is the meta-data needed for pure dependency management compared to the meta-data provided by Maven, Jigsaw, Ivy, Savant and others. >> >> I wanted to bounce this idea off of this list as well as other Maven, Ivy and others. The idea would be to create an open standard format that defines the meta-data needed to do pure dependency management. This has a number of key benefits including: >> >> - Standardize dependency repositories >> - Clean dependency definitions >> - Simple way for tool providers to work with dependencies separate from the build and runtime systems >> - Easier migration between build tools (pick the right tool for the job not the repository or dependency management system) >> >> There are probably other benefits as well. I wanted to get your thoughts this concept. It looks like Jigsaw intends to support POMs and potentially support some of OSGi, but at a baseline, there are a number of concepts that can be defined and shared across all these projects. > > If that is something that gets taken up by this group, I feel compelled to throw in my several cents worth. > > a) Redundant metadata - extremely bad. Ideally, as much of the run-time dependency information should be used for build-time dependency resolution as possible. Only introduce build-time dependency metadata as needed, when it goes above and beyond what the run time data says. At my company, we've struggled with extra meta data driven by the use of an Eclipse-like notion of a feature, and the OSGi-level meta-data as well. This means that they can get out of sync, and do, and that can lead to nasty, annoying and subtle issues. > > b) Build against the supported minimum. I think of this as the JDK 1.4 problem. JDK 1.4 introduced "StringBuffer.append(StringBuffer sb);" You could take the same code and compile against both JDK 1.4 & 1.3. However, the version compiled against 1.4 would only run against 1.4, whereas the JDK 1.3 compiled version would run against both. What with auto-boxing, and newer language features, the opportunities for such mistakes are legion. So, if you want to assert that you *run* against version 2.3.0 and later of a dependency (be it a package, bundle, or module), you better make sure you build against that version. > > c) Follow-on to (b) - sometimes the dependencies you state simply aren't consistent. If A depends on B & C, and B also depends on C, you might be in a spot where B requires a newer version of C than A requires. The correct thing to do here is flag a warning that the dependency that A states on C is actually a lie, and its lower end must be bumped. > > d) What you need to build != what you need to run. For example, in OSGi terms, my bundle may require, at runtime, the ability to log. The API for the logging service may be fixed, but an implementation of that API must also be available. At build time, I don't need the implementation. In fact, if my build process does require an implementation, the one it grabs may trigger the problem indicated by (c), thus either incorrectly building against a newer version of some other API, or suggesting my bundle must raise its minimum dependencies unnecessarily. > > e) What you need to run is a subset of what you need to test. Once we start specifying ranges for dependencies, we really should be making sure that at runtime we test against the lowest version of that dependency, the highest stable version, and any future versions in development. And that's just a minimum. In practice I suspect most people frequently shortcut this notion by continually raising dependency minimums, and forgoing testing of future versions until those future versions are just about declared stable. > > Circling back to the notion that "there are a number of concepts that can be shared across all of these projects," I approach such an assertion with considerable trepidation. At some point you can boil this all down to a simple modeling notion of capabilities and requirements (as in code X provides capability, code Y requires some capability). However, we need to add modifiers to the predicates here. For example: > Version ranges > Security assertions (FIPS certified) > # of suppliers (I need > 1 implementation of service foo, I need exactly one supplier of service foo) > vendor > License (unless you intend to leave the open-source world out) > So, now, what you've got is a system that involves an arbitrary set of subjects and objects, and an arbitrary set of predicates that combine those two sets. That's right, you've got RDF. To the extent that you standardize an RDF-like approach, it only works if everyone agrees on the meanings of some core set of subjects, predicates, and objects. > > And keep in mind, the more subjects and predicates you introduce, the more complexity you get. > > Having said all that, yes, please, let's have at it, and finally try to establish what dependencies mean in the Java space, so that the platform can further thrive. > > -Eric. > > > From mandy.chung at oracle.com Tue Sep 27 15:29:02 2011 From: mandy.chung at oracle.com (mandy.chung at oracle.com) Date: Tue, 27 Sep 2011 22:29:02 +0000 Subject: hg: jigsaw/jigsaw/jdk: Fix jpkg to generate the proper debian version Message-ID: <20110927222919.BE85D47A28@hg.openjdk.java.net> Changeset: 90808f8d462f Author: mchung Date: 2011-09-27 13:31 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/90808f8d462f Fix jpkg to generate the proper debian version Reviewed-by: alanb ! src/share/classes/org/openjdk/jigsaw/cli/Packager.java