From Alan.Bateman at oracle.com Thu Mar 1 01:45:55 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 01 Mar 2012 09:45:55 +0000 Subject: ObjectInputStream In-Reply-To: <4F4EBE3D.6020704@oracle.com> References: <4EEB6998.6070005@oracle.com> <201202201049.42860.jaroslav.tulach@oracle.com> <4F42535B.7050800@redhat.com> <201202221814.23826.jaroslav.tulach@oracle.com> <4F45284D.2070104@redhat.com> <4F454D30.2080900@oracle.com> <4F4551D4.9050407@redhat.com> <4F462E0E.50009@oracle.com> <4F47C84C.9010405@redhat.com> <4F4EBE3D.6020704@oracle.com> Message-ID: <4F4F4553.4070100@oracle.com> On 01/03/2012 00:09, Jesse Glick wrote: > On 02/24/2012 12:26 PM, David M. Lloyd wrote: >> look at how ObjectInputStream resolves classes by default > > ...bizarrely, in fact. In NetBeans we created NbObjectInputStream [1] > in part to solve this exact problem. (The getNBClassLoader method > returns a "master" loader, also the default TCCL, which can load from > any enabled module when unambiguous.) > > While NbOIS is a helpful replacement for OIS in module code, if a > module calls a preexisting library using plain OIS, and this library > tries to deserialize an instance of a class defined in some unrelated > module, you are out of luck. Perhaps Jigsaw's ObjectStreamClass should > be modified to store the module ID where available, which would be > easier to use and also handle cases where multiple modules define a > class of the same name? > > [1] > https://hg.netbeans.org/core-main/raw-file/default/openide.util/src/org/openide/util/io/NbObjectInputStream.java We know that serialization will require work. Probably RMI too. There will be other areas in the JDK but we don't have a complete list. Jon has changes to jtreg in the works that will allow us to run most of the existing tests with modules and that should give us more complete picture. As always then contributions are welcome. For serialization then it'scritical that any changes have as many eyes on the changes as possible, particularly security eyes. -Alan From david.lloyd at redhat.com Thu Mar 1 07:05:34 2012 From: david.lloyd at redhat.com (David M. Lloyd) Date: Thu, 01 Mar 2012 09:05:34 -0600 Subject: ObjectInputStream In-Reply-To: <4F4EBE3D.6020704@oracle.com> References: <4EEB6998.6070005@oracle.com> <201202201049.42860.jaroslav.tulach@oracle.com> <4F42535B.7050800@redhat.com> <201202221814.23826.jaroslav.tulach@oracle.com> <4F45284D.2070104@redhat.com> <4F454D30.2080900@oracle.com> <4F4551D4.9050407@redhat.com> <4F462E0E.50009@oracle.com> <4F47C84C.9010405@redhat.com> <4F4EBE3D.6020704@oracle.com> Message-ID: <4F4F903E.6060908@redhat.com> On 02/29/2012 06:09 PM, Jesse Glick wrote: > On 02/24/2012 12:26 PM, David M. Lloyd wrote: >> look at how ObjectInputStream resolves classes by default > > ...bizarrely, in fact. In NetBeans we created NbObjectInputStream [1] in > part to solve this exact problem. (The getNBClassLoader method returns a > "master" loader, also the default TCCL, which can load from any enabled > module when unambiguous.) > > While NbOIS is a helpful replacement for OIS in module code, if a module > calls a preexisting library using plain OIS, and this library tries to > deserialize an instance of a class defined in some unrelated module, you > are out of luck. Perhaps Jigsaw's ObjectStreamClass should be modified > to store the module ID where available, which would be easier to use and > also handle cases where multiple modules define a class of the same name? Yeah this is exactly what we do in our modular serialization code, though the exact moral equivalent to our solution is to subclass ObjectInputStream and replace its class resolve methods. Since we have a concept of a module loader, we resolve classes based on their module ID with respect to a particular module loader (in our app server we typically have two: one for the "static" modules and one which covers deployments but also "sees" the static module loader). So far the results have been quite positive - better than non-modular serialization in fact. Users seem to encounter far fewer CNFE/NCDFE as a result of deserialization. This type of solution should provide for better security as well - a new permission type could be introduced which grants deserialize permission based upon module ID. -- - DML From jonathan.gibbons at oracle.com Thu Mar 1 15:16:44 2012 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 01 Mar 2012 15:16:44 -0800 Subject: jmod issue Message-ID: <4F50035C.5040308@oracle.com> jmod --help includes the following text: jmod install [--noverify] ... but if I specify a filename, I get the following issue: %/opt/jdk/jigsaw/bin/jmod install -L mlib modules/hello.jmod java.lang.IllegalArgumentException: modules/hello.jmod: Illegal module-name character at index 7 org.openjdk.jigsaw.cli.Command$Exception: java.lang.IllegalArgumentException: modules/hello.jmod: Illegal module-name character at index 7 at org.openjdk.jigsaw.cli.Librarian$Install.go(Librarian.java:242) at org.openjdk.jigsaw.cli.Librarian$Install.go(Librarian.java:186) at org.openjdk.jigsaw.cli.Command.run(Command.java:97) How should I install a .jmod file? -- Jon From jonathan.gibbons at oracle.com Thu Mar 1 15:22:28 2012 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 01 Mar 2012 15:22:28 -0800 Subject: jmod issue In-Reply-To: <4F50035C.5040308@oracle.com> References: <4F50035C.5040308@oracle.com> Message-ID: <4F5004B4.9050804@oracle.com> OK -- turns out " Illegal module-name character at index 7" is a synonym for "file not found" -- Jon On 03/01/2012 03:16 PM, Jonathan Gibbons wrote: > jmod --help includes the following text: > > > jmod install [--noverify] ... > > but if I specify a filename, I get the following issue: > > %/opt/jdk/jigsaw/bin/jmod install -L mlib modules/hello.jmod > java.lang.IllegalArgumentException: modules/hello.jmod: Illegal > module-name character at index 7 > org.openjdk.jigsaw.cli.Command$Exception: > java.lang.IllegalArgumentException: modules/hello.jmod: Illegal > module-name character at index 7 > at org.openjdk.jigsaw.cli.Librarian$Install.go(Librarian.java:242) > at org.openjdk.jigsaw.cli.Librarian$Install.go(Librarian.java:186) > at org.openjdk.jigsaw.cli.Command.run(Command.java:97) > > How should I install a .jmod file? > > -- Jon From david.holmes at oracle.com Thu Mar 1 16:33:53 2012 From: david.holmes at oracle.com (David Holmes) Date: Fri, 02 Mar 2012 10:33:53 +1000 Subject: jmod issue In-Reply-To: <4F5004B4.9050804@oracle.com> References: <4F50035C.5040308@oracle.com> <4F5004B4.9050804@oracle.com> Message-ID: <4F501571.4080805@oracle.com> On 2/03/2012 9:22 AM, Jonathan Gibbons wrote: > OK -- turns out " Illegal module-name character at index 7" is a synonym > for "file not found" I think it may expect a simple filename foo.jmod with no path component. David > -- Jon > > On 03/01/2012 03:16 PM, Jonathan Gibbons wrote: >> jmod --help includes the following text: >> >> >> jmod install [--noverify] ... >> >> but if I specify a filename, I get the following issue: >> >> %/opt/jdk/jigsaw/bin/jmod install -L mlib modules/hello.jmod >> java.lang.IllegalArgumentException: modules/hello.jmod: Illegal >> module-name character at index 7 >> org.openjdk.jigsaw.cli.Command$Exception: >> java.lang.IllegalArgumentException: modules/hello.jmod: Illegal >> module-name character at index 7 >> at org.openjdk.jigsaw.cli.Librarian$Install.go(Librarian.java:242) >> at org.openjdk.jigsaw.cli.Librarian$Install.go(Librarian.java:186) >> at org.openjdk.jigsaw.cli.Command.run(Command.java:97) >> >> How should I install a .jmod file? >> >> -- Jon > From jonathan.gibbons at oracle.com Thu Mar 1 16:46:41 2012 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 01 Mar 2012 16:46:41 -0800 Subject: jmod issue In-Reply-To: <4F501571.4080805@oracle.com> References: <4F50035C.5040308@oracle.com> <4F5004B4.9050804@oracle.com> <4F501571.4080805@oracle.com> Message-ID: <4F501871.207@oracle.com> On 03/01/2012 04:33 PM, David Holmes wrote: > On 2/03/2012 9:22 AM, Jonathan Gibbons wrote: >> OK -- turns out " Illegal module-name character at index 7" is a synonym >> for "file not found" > > I think it may expect a simple filename foo.jmod with no path component. > > David Tried that. No, it really was a somewhat misleading "file not found" issue. I presume the code sees if the item on the command line exists as a file, and treats it as such if so, but otherwise presumptuously treats it as a module name. -- Jon From mandy.chung at oracle.com Thu Mar 1 22:48:44 2012 From: mandy.chung at oracle.com (mandy.chung at oracle.com) Date: Fri, 02 Mar 2012 06:48:44 +0000 Subject: hg: jigsaw/jigsaw/jdk: Exports com.sun.tools.javac in the jdk.devtools default view Message-ID: <20120302064929.ABB3147786@hg.openjdk.java.net> Changeset: 02b685958d9e Author: mchung Date: 2012-03-01 22:43 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/02b685958d9e Exports com.sun.tools.javac in the jdk.devtools default view ! make/modules/Makefile ! make/modules/modules.group ! make/modules/modules.properties ! make/tools/classanalyzer/src/com/sun/classanalyzer/ModuleConfig.java From Alan.Bateman at oracle.com Fri Mar 2 03:11:48 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 02 Mar 2012 11:11:48 +0000 Subject: jmod issue In-Reply-To: <4F501871.207@oracle.com> References: <4F50035C.5040308@oracle.com> <4F5004B4.9050804@oracle.com> <4F501571.4080805@oracle.com> <4F501871.207@oracle.com> Message-ID: <4F50AAF4.5050505@oracle.com> On 02/03/2012 00:46, Jonathan Gibbons wrote: > > Tried that. No, it really was a somewhat misleading "file not found" > issue. > I presume the code sees if the item on the command line exists as a > file, and > treats it as such if so, but otherwise presumptuously treats it as a > module name. > > -- Jon I think the options to jmod install needs to be re-examined (more generally the CLIs need a bit of an overhaul, jpkg more so). As it stands the install command can't always figure out what you want to do. If you give it a path to a jmod file or a directory of classes then it gets it right. However it also allows a module query and this isn't easy to disambiguate from the case where you give it a path to something that doesn't exist. I'll create a bug for this so that it's not forgotten but I think it's something that will resolve itself anyway once someone gets around to cleaning up the CLIs. -Alan From david.lloyd at redhat.com Mon Mar 5 14:44:42 2012 From: david.lloyd at redhat.com (David M. Lloyd) Date: Mon, 05 Mar 2012 16:44:42 -0600 Subject: ClassLoader.getPackages() Message-ID: <4F5541DA.5090503@redhat.com> The JavaDoc for ClassLoader.getPackages() says: protected Package[] getPackages() Returns all of the Packages defined by this class loader and its ancestors. Returns: The array of Package objects defined by this ClassLoader The intention appears to be that the class loader returns all the packages visible to it. However, it also assumes a strict parent/child hierarchy of class loaders. My question is, how does the Jigsaw project interpret this method? Does it return only Packages known to it and the parent class loader(s) (which presumably includes only the "null" class loader)? Or does it also include the Packages from its dependencies (i.e. the complete set of visible Packages)? -- - DML From zhangshj at linux.vnet.ibm.com Mon Mar 5 19:36:38 2012 From: zhangshj at linux.vnet.ibm.com (Shi Jun Zhang) Date: Tue, 06 Mar 2012 11:36:38 +0800 Subject: provides doesn't work? Message-ID: <4F558646.6020704@linux.vnet.ibm.com> Hi, Has provides been implemented? I'm trying to require java.base in my module but it has compile error. So i write the following test case: foo/module-info.java module foo { provides bar; } baz/module-info.java module baz { requires bar; } I get the error message: error: Cannot resolve module dependencies using Jigsaw module resolver [baz, foo]: Cannot resolve 1 error If module baz requires foo, the compilation succeeds. Does i misunderstand the provides keyword? -- Regards, Shi Jun Zhang From mandy.chung at oracle.com Mon Mar 5 21:37:23 2012 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 05 Mar 2012 21:37:23 -0800 Subject: provides doesn't work? In-Reply-To: <4F558646.6020704@linux.vnet.ibm.com> References: <4F558646.6020704@linux.vnet.ibm.com> Message-ID: <4F55A293.8080606@oracle.com> The aliases support ("provides") is not yet implemented. I'm working on a prototype and hopefully will have it ready in a couple of weeks. Mandy On 3/5/2012 7:36 PM, Shi Jun Zhang wrote: > Hi, > > Has provides been implemented? I'm trying to require java.base in my > module but it has compile error. So i write the following test case: > > foo/module-info.java > module foo { > provides bar; > } > > baz/module-info.java > module baz { > requires bar; > } > > I get the error message: > > error: Cannot resolve module dependencies using Jigsaw module resolver > [baz, foo]: Cannot resolve > 1 error > > If module baz requires foo, the compilation succeeds. Does i > misunderstand the provides keyword? > From alan.bateman at oracle.com Tue Mar 6 04:53:10 2012 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Tue, 06 Mar 2012 12:53:10 +0000 Subject: hg: jigsaw/jigsaw/jdk: Ensure that native libs and commands are in the right section of jmod file. Message-ID: <20120306125341.A63C8477F8@hg.openjdk.java.net> Changeset: 8927224bf054 Author: chegar Date: 2012-03-06 12:45 +0000 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/8927224bf054 Ensure that native libs and commands are in the right section of jmod file. Reviewed-by: mchung ! make/com/sun/java/pack/Makefile ! make/common/Defs-modules.gmk ! make/common/Defs.gmk ! make/common/Modules.gmk ! make/common/Program.gmk ! make/modules/Makefile From mandy.chung at oracle.com Tue Mar 6 16:33:16 2012 From: mandy.chung at oracle.com (mandy.chung at oracle.com) Date: Wed, 07 Mar 2012 00:33:16 +0000 Subject: hg: jigsaw/jigsaw/jdk: Fix modules.config to exclude some sun.misc classes from the base module Message-ID: <20120307003342.84C7B4781A@hg.openjdk.java.net> Changeset: 55c21bd558ac Author: mchung Date: 2012-03-06 16:33 -0800 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/55c21bd558ac Fix modules.config to exclude some sun.misc classes from the base module ! make/modules/modules.config From fredrik.ohrstrom at oracle.com Sat Mar 10 10:32:44 2012 From: fredrik.ohrstrom at oracle.com (=?UTF-8?B?RnJlZHJpayDDlmhyc3Ryw7Zt?=) Date: Sat, 10 Mar 2012 10:32:44 -0800 (PST) Subject: Question about FileManagers in Jigsaw. Message-ID: If I create my own Forwarding file manager in Jigsaw, how do I detect which module a getFileForOutput is meant for? //Fredrik From jonathan.gibbons at oracle.com Sat Mar 10 10:49:07 2012 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Sat, 10 Mar 2012 10:49:07 -0800 Subject: Question about FileManagers in Jigsaw. In-Reply-To: References: Message-ID: <4F5BA223.6040502@oracle.com> On 03/10/2012 10:32 AM, Fredrik ?hrstr?m wrote: > If I create my own Forwarding file manager in Jigsaw, > how do I detect which module a getFileForOutput is meant for? > > //Fredrik You'll need to start getting your head around jigsaw compilation modes. There are essentially three possibilities: 1. no modules -- think of this as a legacy mode, similar to jdk 7 and earlier 2. a single module -- you can refer to other modules, but you're only compiling one -- in that case, all files written will be for the one module 3. multiple modules -- in that case, javac will use methods on the new ModuleFileManager interface, such that it dynamically creates a Location for each module. These locations are derived from the standard output location and an identifier for the module. Note this identifier is /not/ necessarily the same as the module ID. The identifier will be the same subdirectory named in which the sources for the module were found. It is expected that it will typically be very similar to the module name, but that is not a requirement. The rationale here is to /not/ have version numbers in the source directory naming hierarchy (because version numbers change over time), and to have 1-1 correspondence between the hierarchy under the input source and output class directories -- for the benefit of "simple" dependency analysis, such as found in Ant. -- Jon From fredrik.ohrstrom at oracle.com Mon Mar 12 02:59:49 2012 From: fredrik.ohrstrom at oracle.com (=?UTF-8?B?RnJlZHJpayDDlmhyc3Ryw7Zt?=) Date: Mon, 12 Mar 2012 10:59:49 +0100 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F5BA223.6040502@oracle.com> References: <4F5BA223.6040502@oracle.com> Message-ID: <4F5DC915.90706@oracle.com> 2012-03-10 19:49, Jonathan Gibbons skrev: > 1. no modules -- think of this as a legacy mode, similar to jdk 7 and > earlier > > 2. a single module -- you can refer to other modules, but you're only > compiling one -- in that case, all files written will be for the one module > > 3. multiple modules -- in that case, javac will use methods on the new > ModuleFileManager interface, such that it dynamically creates a Location > for each module. Thanks Jon! When in multiple module mode, is it the intention that module names look dns names in reverse? Like packages, if company Foo that owns www.foo.com the it will distribute their app under the module name com.foo? (The jdk is of course special with its jdk.base, jdk.corba etc etc) Is there a safety check for the views declared inside a module, so that the views have to be module+"."+viewname? Thus the only views allowed for com.foo are com.foo.bar com.foo.internal etc etc. Can a single javac invocation handle both legacy non-module sources and sources put into a module. I.e. javac -d bin src/com/foo/LegacyApp.java src/com.foo/com/foo/NewApp.java src/com.foo/module-info.java And can NewApp refer to/use LegacyApp? I assume that implicit compilation works as before? Ie. even though there are circular dependencies between different modules, as long as you supply the sourcepath to the entire source, javac will resolve and compile the dependencies automatically? Can -Xprefer:source be made smarter than it is today? We use -Xprefer:source when compiling some of the repositories in the OpenJDK because we need to link to the new classes declared in source, not the classes in the bootclasspath. However, when doing incremental compiles, we really would like to reuse the generated classes during the previous compile, not recompile from source again. Thus the option -Xprefer:nobootclasspath would be more valid? //Fredrik From jesse.glick at oracle.com Mon Mar 12 07:04:37 2012 From: jesse.glick at oracle.com (Jesse Glick) Date: Mon, 12 Mar 2012 10:04:37 -0400 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F5DC915.90706@oracle.com> References: <4F5BA223.6040502@oracle.com> <4F5DC915.90706@oracle.com> Message-ID: <4F5E0275.7020303@oracle.com> On 03/12/2012 05:59 AM, Fredrik ?hrstr?m wrote: > Can a single javac invocation handle both legacy non-module sources and > sources put into a module. A related question, which may be rather important: Can a javac invocation for single-module mode accept module-info.java from one location in -sourcepath and regular compilation units from another? From Alan.Bateman at oracle.com Mon Mar 12 07:09:49 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 12 Mar 2012 14:09:49 +0000 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F5DC915.90706@oracle.com> References: <4F5BA223.6040502@oracle.com> <4F5DC915.90706@oracle.com> Message-ID: <4F5E03AD.2050204@oracle.com> On 12/03/2012 09:59, Fredrik ?hrstr?m wrote: > : > Thanks Jon! > > When in multiple module mode, is it the intention that module names look > dns names in reverse? > Like packages, if company Foo that owns www.foo.com the it will > distribute their app under the > module name com.foo? (The jdk is of course special with its jdk.base, > jdk.corba etc etc) Not necessarily, reserve DNS is a possible convention but it's likely there will be others too, maybe . or names similar to Maven artifact ids for example. In the current prototype then JDK modules are prefixed with "jdk." or "sun." but that will likely change. As the module graph and the modules that comprise Java SE will be something defined by Java SE 8 then it's very likely they will include "java" in the name. > > Is there a safety check for the views declared inside a module, so that > the views have to be > module+"."+viewname? Thus the only views allowed for com.foo are com.foo.bar > com.foo.internal etc etc. There isn't requirement that the view names be prefixed with the module name. We have examples in the current prototype where this isn't the case - for example the jdk.devtools module has the views jdk.javac and jdk.javah. > > Can a single javac invocation handle both legacy non-module sources and > sources put into a module. > > I.e. > javac -d bin src/com/foo/LegacyApp.java src/com.foo/com/foo/NewApp.java > src/com.foo/module-info.java > > And can NewApp refer to/use LegacyApp? Jon is best one to answer this but it should work for a single module, but not multiple modules of course. -Alan. From fredrik.ohrstrom at oracle.com Mon Mar 12 07:23:07 2012 From: fredrik.ohrstrom at oracle.com (=?UTF-8?B?RnJlZHJpayDDlmhyc3Ryw7Zt?=) Date: Mon, 12 Mar 2012 15:23:07 +0100 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F5E03AD.2050204@oracle.com> References: <4F5BA223.6040502@oracle.com> <4F5DC915.90706@oracle.com> <4F5E03AD.2050204@oracle.com> Message-ID: <4F5E06CB.3080403@oracle.com> Thanks Alan! 2012-03-12 15:09, Alan Bateman skrev: > Not necessarily, reserve DNS is a possible convention but it's likely > there will be others too, maybe . or names > similar to Maven artifact ids for example. Are you going to allow slashes in the module name? > In the current prototype then JDK modules are prefixed with "jdk." or > "sun." but that will likely change. As the module graph and the modules > that comprise Java SE will be something defined by Java SE 8 then it's > very likely they will include "java" in the name. ok! > There isn't requirement that the view names be prefixed with the module > name. We have examples in the current prototype where this isn't the > case - for example the jdk.devtools module has the views jdk.javac and > jdk.javah. So, its just global module names and an error/exception is thrown if two modules views have the same name? Is it the -modulepath option that enables executing raw classes from disk, no jmod files, no installing of jmods into jvm? //Fredrik From jonathan.gibbons at oracle.com Mon Mar 12 07:54:47 2012 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Mon, 12 Mar 2012 07:54:47 -0700 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F5DC915.90706@oracle.com> References: <4F5BA223.6040502@oracle.com> <4F5DC915.90706@oracle.com> Message-ID: <4F5E0E37.7010402@oracle.com> On 03/12/2012 02:59 AM, Fredrik ?hrstr?m wrote: > 2012-03-10 19:49, Jonathan Gibbons skrev: >> 1. no modules -- think of this as a legacy mode, similar to jdk 7 and >> earlier >> >> 2. a single module -- you can refer to other modules, but you're only >> compiling one -- in that case, all files written will be for the one module >> >> 3. multiple modules -- in that case, javac will use methods on the new >> ModuleFileManager interface, such that it dynamically creates a Location >> for each module. > Thanks Jon! > > When in multiple module mode, is it the intention that module names look > dns names in reverse? > Like packages, if company Foo that owns www.foo.com the it will > distribute their app under the > module name com.foo? (The jdk is of course special with its jdk.base, > jdk.corba etc etc) No. No such recommendation has been made. I think the expectation is that the first component will be product-related, such as "foo", "jdk", "jtreg" etc. > Is there a safety check for the views declared inside a module, so that > the views have to be > module+"."+viewname? Thus the only views allowed for com.foo are com.foo.bar > com.foo.internal etc etc. There is currently no such requirement. > Can a single javac invocation handle both legacy non-module sources and > sources put into a module. No. The three modes I described earlier (no modules, one module, many modules) are mutually exclusive. > I.e. > javac -d bin src/com/foo/LegacyApp.java src/com.foo/com/foo/NewApp.java > src/com.foo/module-info.java > > And can NewApp refer to/use LegacyApp? > > I assume that implicit compilation works as before? Ie. even though > there are circular dependencies > between different modules, as long as you supply the sourcepath to the > entire source, javac will > resolve and compile the dependencies automatically? Yes. > Can -Xprefer:source be made smarter than it is today? We use > -Xprefer:source when compiling > some of the repositories in the OpenJDK because we need to link to the > new classes declared in source, > not the classes in the bootclasspath. However, when doing incremental > compiles, we really would > like to reuse the generated classes during the previous compile, not > recompile from source again. > Thus the option -Xprefer:nobootclasspath would be more valid? It sounds like you could fix that by changing the way your compiler options are set up. Move what you currently have on the bootclasspath to the classpath, and set the bootclasspath to empty. Then -Xprefer:source will have the effect you desire. > //Fredrik > From jonathan.gibbons at oracle.com Mon Mar 12 08:01:52 2012 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Mon, 12 Mar 2012 08:01:52 -0700 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F5E0275.7020303@oracle.com> References: <4F5BA223.6040502@oracle.com> <4F5DC915.90706@oracle.com> <4F5E0275.7020303@oracle.com> Message-ID: <4F5E0FE0.5050109@oracle.com> On 03/12/2012 07:04 AM, Jesse Glick wrote: > On 03/12/2012 05:59 AM, Fredrik ?hrstr?m wrote: >> Can a single javac invocation handle both legacy non-module sources and >> sources put into a module. > > A related question, which may be rather important: > > Can a javac invocation for single-module mode accept module-info.java > from one location in -sourcepath and regular compilation units from > another? Yes. In both single-module mode and multiple-module mode, -sourcepath and -modulepath retain the concept of a path being a series of places in which to search for desired content. The "classic" use case we have always had in mind for the JDK build has been the combination of source directories, such as "src/share/classes", "src/$PLATFORM/classes", and "build/$PLATFORM/gensrc". -- Jon From jesse.glick at oracle.com Mon Mar 12 08:08:55 2012 From: jesse.glick at oracle.com (Jesse Glick) Date: Mon, 12 Mar 2012 11:08:55 -0400 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F5BA223.6040502@oracle.com> References: <4F5BA223.6040502@oracle.com> Message-ID: <4F5E1187.9020905@oracle.com> On 03/10/2012 01:49 PM, Jonathan Gibbons wrote: > 1-1 correspondence between the hierarchy under the input source and output class directories -- for the benefit of "simple" dependency analysis, such as found in Ant. To be precise, such as found in Ant's . Ant's does something much more complex, which would probably not work in multimodule mode. From jonathan.gibbons at oracle.com Mon Mar 12 08:22:57 2012 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Mon, 12 Mar 2012 08:22:57 -0700 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F5E03AD.2050204@oracle.com> References: <4F5BA223.6040502@oracle.com> <4F5DC915.90706@oracle.com> <4F5E03AD.2050204@oracle.com> Message-ID: <4F5E14D1.2050602@oracle.com> On 03/12/2012 07:09 AM, Alan Bateman wrote: > >> >> Can a single javac invocation handle both legacy non-module sources and >> sources put into a module. >> >> I.e. >> javac -d bin src/com/foo/LegacyApp.java src/com.foo/com/foo/NewApp.java >> src/com.foo/module-info.java >> >> And can NewApp refer to/use LegacyApp? > Jon is best one to answer this but it should work for a single module, > but not multiple modules of course. In single module mode, the sourcepath, classpath and output directory should be organized as a single package hierarchy, /without any module-name component/, such as "com.foo". Thus, module-info.java should normally be placed in top level directory on the sourcepath or classpath. [Advanced note: technically module-info.java can be anywhere on the file system if it is always present on the command line, as is the case for all java source given on the command line today, subject to the rules about the base name of the file. But, in this case, you will effectively defeat tools like the Ant task that assume a 1-1 mapping between the input source hierarchy and the output class hierarchy.] The intent is to minimize the need to reorganize the source code for projects that create a single jar file (meaning no module mode) or a single jmod file (meaning single module mode). Note that even in single module module, the "single" refers to the sources being compiled. Even in single module mode, you can refer to other modules which have already been compiled are are available on either the module path or in a module library. If you need to compile the sources for multiple modules in the same compilation, then you must use multiple module mode, which means you will need to reorganize your sources so that the sourcepath and module path should be organized in "module path layout", meaning that there is an extra filename component to separate the package hierarchies for the various modules being compiled. It is expected, but not required, that there is a relationship between the name of that filename component and the module name. -- Jon From jonathan.gibbons at oracle.com Mon Mar 12 08:28:52 2012 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Mon, 12 Mar 2012 08:28:52 -0700 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F5E1187.9020905@oracle.com> References: <4F5BA223.6040502@oracle.com> <4F5E1187.9020905@oracle.com> Message-ID: <4F5E1634.5030207@oracle.com> On 03/12/2012 08:08 AM, Jesse Glick wrote: > On 03/10/2012 01:49 PM, Jonathan Gibbons wrote: >> 1-1 correspondence between the hierarchy under the input source and >> output class directories -- for the benefit of "simple" dependency >> analysis, such as found in Ant. > > To be precise, such as found in Ant's . Ant's does > something much more complex, which would probably not work in > multimodule mode. What does it do? It was an important design constraint that javac should continue to work as much as possible with . I know has some special magic for package-info.java, but that should not be required for jdk7 or later, given the -Xpkginfo option. -- Jon From fredrik.ohrstrom at oracle.com Mon Mar 12 09:08:38 2012 From: fredrik.ohrstrom at oracle.com (=?UTF-8?B?RnJlZHJpayDDlmhyc3Ryw7Zt?=) Date: Mon, 12 Mar 2012 17:08:38 +0100 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F5E14D1.2050602@oracle.com> References: <4F5BA223.6040502@oracle.com> <4F5DC915.90706@oracle.com> <4F5E03AD.2050204@oracle.com> <4F5E14D1.2050602@oracle.com> Message-ID: <4F5E1F86.2000503@oracle.com> 2012-03-12 16:22, Jonathan Gibbons skrev: > Note that even in single module module, the "single" refers to the > sources being compiled. > Even in single module mode, you can refer to other modules which have > already been compiled > are are available on either the module path or in a module library. > > If you need to compile the sources for multiple modules in the same > compilation, then you must > use multiple module mode, which means you will need to reorganize your > sources so that the > sourcepath and module path should be organized in "module path layout", > meaning that there > is an extra filename component to separate the package hierarchies for > the various modules > being compiled. It is expected, but not required, that there is a > relationship between the > name of that filename component and the module name. > So to be sure about where a module is and what its name is, tools like ant, have to open the module-info.java file and read it? If ant looks at a directory and finds both module-info.java and java sources, then it probably is a single module compile. If it only finds a module-info.java file and the module_name inside it, sort of corresponds to the directory (or is the directory name encoded into the module-info.java file) then it is multiple module setup? I was hoping that when I found a module-info.java file, the directory in which it resides is the name of the module. Fast and convenient..... //Fredrik From jonathan.gibbons at oracle.com Mon Mar 12 09:27:00 2012 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Mon, 12 Mar 2012 09:27:00 -0700 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F5E1F86.2000503@oracle.com> References: <4F5BA223.6040502@oracle.com> <4F5DC915.90706@oracle.com> <4F5E03AD.2050204@oracle.com> <4F5E14D1.2050602@oracle.com> <4F5E1F86.2000503@oracle.com> Message-ID: <4F5E23D4.7020304@oracle.com> On 03/12/2012 09:08 AM, Fredrik ?hrstr?m wrote: > 2012-03-12 16:22, Jonathan Gibbons skrev: >> Note that even in single module module, the "single" refers to the >> sources being compiled. >> Even in single module mode, you can refer to other modules which have >> already been compiled >> are are available on either the module path or in a module library. >> >> If you need to compile the sources for multiple modules in the same >> compilation, then you must >> use multiple module mode, which means you will need to reorganize your >> sources so that the >> sourcepath and module path should be organized in "module path layout", >> meaning that there >> is an extra filename component to separate the package hierarchies for >> the various modules >> being compiled. It is expected, but not required, that there is a >> relationship between the >> name of that filename component and the module name. >> > So to be sure about where a module is and what its name is, tools like > ant, have > to open the module-info.java file and read it? If ant looks at a > directory and finds > both module-info.java and java sources, then it probably is a single > module compile. > > If it only finds a module-info.java file and the module_name inside it, > sort of corresponds > to the directory (or is the directory name encoded into the > module-info.java file) > then it is multiple module setup? > > I was hoping that when I found a module-info.java file, the directory in > which it resides > is the name of the module. Fast and convenient..... > > //Fredrik > If -modulepath is set but not -classpath, then that implies multi-module mode. Otherwise, if there is a module-info on the command line or source path or class path, then it is single module mode, else zero module mode. The rules were designed for ease of migration of existing code. The truth about the name of the module lies solely in the module-info file, as will eventually be defined in JLS. JLS does not in general impose any requirements on the file system layout, leaving that up to "the host system". Just as the classes for a package do not have to live in a directory structure matching the package hierarchy, so too for modules. But in both cases, convention and convenience will likely imply a strong correspondence. -- Jon From jesse.glick at oracle.com Mon Mar 12 09:29:35 2012 From: jesse.glick at oracle.com (Jesse Glick) Date: Mon, 12 Mar 2012 12:29:35 -0400 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F5E1634.5030207@oracle.com> References: <4F5BA223.6040502@oracle.com> <4F5E1187.9020905@oracle.com> <4F5E1634.5030207@oracle.com> Message-ID: <4F5E246F.7050301@oracle.com> On 03/12/2012 11:28 AM, Jonathan Gibbons wrote: >> Ant's does something much more complex > > What does it do? Tracks binary dependencies (*) among compiled classes to support more precise incremental compilation. [1] Normally run immediately before . > I know has some special magic for package-info.java, > but that should not be required for jdk7 or later, given the -Xpkginfo option. Well, for now it is still required since it is too hard to know whether a version of the compiler supporting this flag is being used. [2] (*) Broken by compile-time constants [3]. [1] http://ant.apache.org/manual/Tasks/depend.html [2] https://issues.apache.org/bugzilla/show_bug.cgi?id=52096 [3] https://issues.apache.org/bugzilla/show_bug.cgi?id=2811 From Alan.Bateman at oracle.com Mon Mar 12 13:12:58 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 12 Mar 2012 20:12:58 +0000 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F5E06CB.3080403@oracle.com> References: <4F5BA223.6040502@oracle.com> <4F5DC915.90706@oracle.com> <4F5E03AD.2050204@oracle.com> <4F5E06CB.3080403@oracle.com> Message-ID: <4F5E58CA.5080406@oracle.com> On 12/03/2012 14:23, Fredrik ?hrstr?m wrote: > Is it the -modulepath option that enables executing raw classes from > disk, no jmod files, no installing of jmods into jvm? //Fredrik Yes, but it's not in yet. One important thing is that it should allow the base and other platform modules to be loaded from the module path and that would provide a possible option for bootstrapping the build. -Alan. From mandy.chung at oracle.com Mon Mar 12 13:33:16 2012 From: mandy.chung at oracle.com (mandy.chung at oracle.com) Date: Mon, 12 Mar 2012 20:33:16 +0000 Subject: hg: jigsaw/jigsaw/jdk: Exports sun.reflect in jdk.base as a workaround for reflection Message-ID: <20120312203343.072B047917@hg.openjdk.java.net> Changeset: 04e27358d0dc Author: mchung Date: 2012-03-12 13:33 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/04e27358d0dc Exports sun.reflect in jdk.base as a workaround for reflection ! make/modules/modules.group ! make/tools/classanalyzer/src/com/sun/classanalyzer/Module.java From fredrik.ohrstrom at oracle.com Tue Mar 13 05:35:55 2012 From: fredrik.ohrstrom at oracle.com (=?UTF-8?B?RnJlZHJpayDDlmhyc3Ryw7Zt?=) Date: Tue, 13 Mar 2012 13:35:55 +0100 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F5E58CA.5080406@oracle.com> References: <4F5BA223.6040502@oracle.com> <4F5DC915.90706@oracle.com> <4F5E03AD.2050204@oracle.com> <4F5E06CB.3080403@oracle.com> <4F5E58CA.5080406@oracle.com> Message-ID: <4F5F3F2B.1040301@oracle.com> 2012-03-12 21:12, Alan Bateman skrev: > On 12/03/2012 14:23, Fredrik ?hrstr?m wrote: >> > Is it the -modulepath option that enables executing raw classes from >> > disk, no jmod files, no installing of jmods into jvm? //Fredrik > Yes, but it's not in yet. One important thing is that it should allow > the base and other platform modules to be loaded from the module path > and that would provide a possible option for bootstrapping the build. 2012-03-12 17:27, Jonathan Gibbons skrev: >If -modulepath is set but not -classpath, then that implies multi-module > mode. It seems like -modulepath is needed to do multiple module mode compilation in javac. However -modulepath is not yet implemented according to Alan. The obvious way to setup a compilation of the shuffled source code tree is to use multiple mode compilation. In particular if we are going to deal to circular dependencies between the modules. To the extreme, it should be possible to compile the entire jdk from a single batch javac call, the same way it can be done with the build-infra repos. For a more modular design of the makefiles (no pun intended) we would like to split the modules into several small makefiles, that share the same javac server, to avoid redundant implicit compilations. But, since there is no -modulepath, is it possible to compile multiple module mode anyway? Or does single module mode, allow for a multiple module source path? If these questions are both answered with nay, then it is impossible to convert any makefiles at this time, since the compiler is not done yet. //Fredrik From Alan.Bateman at oracle.com Tue Mar 13 05:44:50 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 13 Mar 2012 12:44:50 +0000 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F5F3F2B.1040301@oracle.com> References: <4F5BA223.6040502@oracle.com> <4F5DC915.90706@oracle.com> <4F5E03AD.2050204@oracle.com> <4F5E06CB.3080403@oracle.com> <4F5E58CA.5080406@oracle.com> <4F5F3F2B.1040301@oracle.com> Message-ID: <4F5F4142.20106@oracle.com> On 13/03/2012 12:35, Fredrik ?hrstr?m wrote: > : > It seems like -modulepath is needed to do multiple module mode > compilation in javac. > However -modulepath is not yet implemented according to Alan. > javac has -modulepath for a long time, it's just that we don't have modulepath support in the runtime yet. It's coming (one of these days). -Alan. From fredrik.ohrstrom at oracle.com Tue Mar 13 05:49:59 2012 From: fredrik.ohrstrom at oracle.com (=?UTF-8?B?RnJlZHJpayDDlmhyc3Ryw7Zt?=) Date: Tue, 13 Mar 2012 13:49:59 +0100 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F5F4142.20106@oracle.com> References: <4F5BA223.6040502@oracle.com> <4F5DC915.90706@oracle.com> <4F5E03AD.2050204@oracle.com> <4F5E06CB.3080403@oracle.com> <4F5E58CA.5080406@oracle.com> <4F5F3F2B.1040301@oracle.com> <4F5F4142.20106@oracle.com> Message-ID: <4F5F4277.6040305@oracle.com> 2012-03-13 13:44, Alan Bateman skrev: > On 13/03/2012 12:35, Fredrik ?hrstr?m wrote: >> : >> It seems like -modulepath is needed to do multiple module mode >> compilation in javac. >> However -modulepath is not yet implemented according to Alan. >> > javac has -modulepath for a long time, it's just that we don't have > modulepath support in the runtime yet. It's coming (one of these days). Good! How do you compile and run the bootstrap javac? //Fredrik From Alan.Bateman at oracle.com Tue Mar 13 07:06:14 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 13 Mar 2012 14:06:14 +0000 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F5F4277.6040305@oracle.com> References: <4F5BA223.6040502@oracle.com> <4F5DC915.90706@oracle.com> <4F5E03AD.2050204@oracle.com> <4F5E06CB.3080403@oracle.com> <4F5E58CA.5080406@oracle.com> <4F5F3F2B.1040301@oracle.com> <4F5F4142.20106@oracle.com> <4F5F4277.6040305@oracle.com> Message-ID: <4F5F5456.3050604@oracle.com> On 13/03/2012 12:49, Fredrik ?hrstr?m wrote: > : > Good! > How do you compile and run the bootstrap javac? > > //Fredrik I think you are asking how should we bootstrap the Jigsaw build. The current build, as you know, is essentially a post-processing step so the classes in the jdk repository are compiled as they have always done using the bootstrap JDK + langtools. The launchers for the module-aware tools (including jmod) that are used in the post-processing step are built with -Xmode:legacy so they can be used in the build to create the module library and compile the module-info sources and get us to a module image. With a new build, where we compile the platform as a set of modules, then we have to work out the steps to get us to the point where we have langtools and a base module. Whether that is in module path layout or a base image isn't clear to me at this point. I think, and Jon is really the right person to answer this, that we have to be able to compile the base module with the bootstrap JDK + langtools that has some minimal emulation of Jigsaw (or javac with ZeroMod that we have today). There are details to work through. Another thing about the new build, and something that Jon mentioned recently, is that the jaxp, jaxws and corba repos will need to be compiled much later than they are done now (they are compiled with the bootstrap JDK/javac today). -Alan. From fredrik.ohrstrom at oracle.com Tue Mar 13 07:52:55 2012 From: fredrik.ohrstrom at oracle.com (=?UTF-8?B?RnJlZHJpayDDlmhyc3Ryw7Zt?=) Date: Tue, 13 Mar 2012 15:52:55 +0100 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F5F5456.3050604@oracle.com> References: <4F5BA223.6040502@oracle.com> <4F5DC915.90706@oracle.com> <4F5E03AD.2050204@oracle.com> <4F5E06CB.3080403@oracle.com> <4F5E58CA.5080406@oracle.com> <4F5F3F2B.1040301@oracle.com> <4F5F4142.20106@oracle.com> <4F5F4277.6040305@oracle.com> <4F5F5456.3050604@oracle.com> Message-ID: <4F5F5F47.7020106@oracle.com> 2012-03-13 15:06, Alan Bateman skrev: > The current build, as you know, is essentially a post-processing step so > the classes in the jdk repository are compiled as they have always done > using the bootstrap JDK + langtools. The launchers for the module-aware > tools (including jmod) that are used in the post-processing step are > built with -Xmode:legacy so they can be used in the build to create the > module library and compile the module-info sources and get us to a > module image. Ok, see if I got this right: We need a boot-jdk compatible: 2xboostrap_javac, bootstrap_jpkg (jpkg is used to create jmod files, right?) (but not bootstrap_jmod? (jmod is used to install jmod files into the jvm, right?) We do not really need to build a full launcher for the bootstrap jpkg do we? We can just run the classes directly? The bootstrap_javac will build a second bootstrap_javac with a built in zeromod imported from the jdk sources (potentially stripped with help of genstubs). We could skip the second bootstrap_javac if the zeromod code and other imported jdk sources can be compiled using boot_jdk supported syntax, or if Genstubs could strip them from jigsaw features? But since they most likely contain jigsaw features and/or need module resoultion, when compiled, and we can't have Genstubs strip out jigsaw features, we need the second bootstrap_javac, right? > With a new build, where we compile the platform as a set of modules, > then we have to work out the steps to get us to the point where we have > langtools and a base module. In the future when we have shuffled langtools as module(s), we could detect that the boot jdk is pre jigsaw and run a makefile that backshuffles the source to legacy structuring? It could perhaps run Genstubs and inject the cleaned zero mod into the backshuffled code? Thus we could get away with a single bootstrap? > Whether that is in module path layout or a > base image isn't clear to me at this point. I think, and Jon is really > the right person to answer this, that we have to be able to compile the > base module with the bootstrap JDK + langtools that has some minimal > emulation of Jigsaw (or javac with ZeroMod that we have today). Then we generate Java sources for Charsets et al. Next step is to build the base module from shuffled (multi module) sourcepath using the second bootstrap_javac? Or could we just compile the whole jdk using multi module compile? It seems like that should be possible? > There are details to work through. Another thing about the new build, and > something that Jon mentioned recently, is that the jaxp, jaxws and corba > repos will need to be compiled much later than they are done now (they > are compiled with the bootstrap JDK/javac today). Moving the jaxp, jaxws and corba into the jdk build makes sense even without jigsaw. //Fredrik From Alan.Bateman at oracle.com Tue Mar 13 08:47:21 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 13 Mar 2012 15:47:21 +0000 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F5F5F47.7020106@oracle.com> References: <4F5BA223.6040502@oracle.com> <4F5DC915.90706@oracle.com> <4F5E03AD.2050204@oracle.com> <4F5E06CB.3080403@oracle.com> <4F5E58CA.5080406@oracle.com> <4F5F3F2B.1040301@oracle.com> <4F5F4142.20106@oracle.com> <4F5F4277.6040305@oracle.com> <4F5F5456.3050604@oracle.com> <4F5F5F47.7020106@oracle.com> Message-ID: <4F5F6C09.8070009@oracle.com> On 13/03/2012 14:52, Fredrik ?hrstr?m wrote: > : > Ok, see if I got this right: > > We need a boot-jdk compatible: 2xboostrap_javac, bootstrap_jpkg (jpkg is > used to create jmod files, right?) > (but not bootstrap_jmod? (jmod is used to install jmod files into the > jvm, right?) > > We do not really need to build a full launcher for the bootstrap jpkg do > we? We can just run the classes directly? jmod is the tool that creates the module library so it's needed to create an image, even a minimal image that only contains the base module. It can also install modules where the classes are in module path layout, which is likely to be useful. I suspect that jpkg may not be critical for the bootstrapping, but clearly important for later when the build creates the native and jmod packages. Jon is the best person to consult with on the sequencing. I think that one possible approach is to use the bootstrap JDK + javac (with some Jigsaw emulation) to compile the base module and then run in module mode to get us to an initial module image. Once we have an image with the base + langtools modules installed then it should be easier sailing. > : > > Then we generate Java sources for Charsets et al. > Next step is to build the base module from shuffled (multi module) > sourcepath using the second > bootstrap_javac? Or could we just compile the whole jdk using multi > module compile? > It seems like that should be possible? I think this would be desirable, assuming that shuffled means the module path layout that javac understands. > : > > Moving the jaxp, jaxws and corba into the jdk build makes sense even > without jigsaw. > I don't know the history as to why as to why the bootstrap JDK is used for these repos. -Alan From jonathan.gibbons at oracle.com Tue Mar 13 09:10:42 2012 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Tue, 13 Mar 2012 09:10:42 -0700 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F5F6C09.8070009@oracle.com> References: <4F5BA223.6040502@oracle.com> <4F5DC915.90706@oracle.com> <4F5E03AD.2050204@oracle.com> <4F5E06CB.3080403@oracle.com> <4F5E58CA.5080406@oracle.com> <4F5F3F2B.1040301@oracle.com> <4F5F4142.20106@oracle.com> <4F5F4277.6040305@oracle.com> <4F5F5456.3050604@oracle.com> <4F5F5F47.7020106@oracle.com> <4F5F6C09.8070009@oracle.com> Message-ID: <4F5F7182.1050205@oracle.com> On 03/13/2012 08:47 AM, Alan Bateman wrote: > > > Jon is the best person to consult with on the sequencing. I think that > one possible approach is to use the bootstrap JDK + javac (with some > Jigsaw emulation) to compile the base module and then run in module > mode to get us to an initial module image. Once we have an image with > the base + langtools modules installed then it should be easier sailing. You need what I normally call the "hybrid javac", what you call "bootstrap javac" which is latest code compiled to run on boot jdk. That is enough to compile the base module, with a ZeroMod module resolver. I would think you would run these classes in /legacy/ mode to get to an initial module image. Hopefully, javac is the only part of the system that needs to be built twice, i.e. the same as is the case now. -- Jon From kelly.ohair at oracle.com Tue Mar 13 09:27:04 2012 From: kelly.ohair at oracle.com (Kelly O'Hair) Date: Tue, 13 Mar 2012 09:27:04 -0700 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F5F5F47.7020106@oracle.com> References: <4F5BA223.6040502@oracle.com> <4F5DC915.90706@oracle.com> <4F5E03AD.2050204@oracle.com> <4F5E06CB.3080403@oracle.com> <4F5E58CA.5080406@oracle.com> <4F5F3F2B.1040301@oracle.com> <4F5F4142.20106@oracle.com> <4F5F4277.6040305@oracle.com> <4F5F5456.3050604@oracle.com> <4F5F5F47.7020106@oracle.com> Message-ID: On Mar 13, 2012, at 7:52 AM, Fredrik ?hrstr?m wrote: >> >> There are details to work through. Another thing about the new build, and >> something that Jon mentioned recently, is that the jaxp, jaxws and corba >> repos will need to be compiled much later than they are done now (they >> are compiled with the bootstrap JDK/javac today). > Moving the jaxp, jaxws and corba into the jdk build makes sense even > without jigsaw. > > //Fredrik Not sure we want to fold corba, jaxp, and jaxws sources back into jdk repository. We worked hard to untangle them. I'm not completely against the idea, but we need to be very careful here and think this through. My preferences at this point is to leave them as separate repos and try and work with them that way. -kto From jonathan.gibbons at oracle.com Tue Mar 13 09:32:26 2012 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Tue, 13 Mar 2012 09:32:26 -0700 Subject: Question about FileManagers in Jigsaw. In-Reply-To: References: <4F5BA223.6040502@oracle.com> <4F5DC915.90706@oracle.com> <4F5E03AD.2050204@oracle.com> <4F5E06CB.3080403@oracle.com> <4F5E58CA.5080406@oracle.com> <4F5F3F2B.1040301@oracle.com> <4F5F4142.20106@oracle.com> <4F5F4277.6040305@oracle.com> <4F5F5456.3050604@oracle.com> <4F5F5F47.7020106@oracle.com> Message-ID: <4F5F769A.5070301@oracle.com> On 03/13/2012 09:27 AM, Kelly O'Hair wrote: > On Mar 13, 2012, at 7:52 AM, Fredrik ?hrstr?m wrote: > >>> There are details to work through. Another thing about the new build, and >>> something that Jon mentioned recently, is that the jaxp, jaxws and corba >>> repos will need to be compiled much later than they are done now (they >>> are compiled with the bootstrap JDK/javac today). >> Moving the jaxp, jaxws and corba into the jdk build makes sense even >> without jigsaw. >> >> //Fredrik > Not sure we want to fold corba, jaxp, and jaxws sources back into jdk repository. We worked hard to untangle them. > I'm not completely against the idea, but we need to be very careful here and think this through. > My preferences at this point is to leave them as separate repos and try and work with them that way. > > -kto I completely agree that we do not want to reorganize the repos here, and we have never considered that. However, it would be best if these repos could be built after the base module has been built, so we may have to go back and forth between the repos to get the right build sequencing. While that would be a pain with the existing SUBDIRS-style build, I was hoping/assuming it would be easier with the flat-Makefile-style build, in which make figures out the proper build order based on the dependencies -- i.e. the way make is supposed to be used. -- Jon From kelly.ohair at oracle.com Tue Mar 13 09:50:57 2012 From: kelly.ohair at oracle.com (Kelly O'Hair) Date: Tue, 13 Mar 2012 09:50:57 -0700 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F5F769A.5070301@oracle.com> References: <4F5BA223.6040502@oracle.com> <4F5DC915.90706@oracle.com> <4F5E03AD.2050204@oracle.com> <4F5E06CB.3080403@oracle.com> <4F5E58CA.5080406@oracle.com> <4F5F3F2B.1040301@oracle.com> <4F5F4142.20106@oracle.com> <4F5F4277.6040305@oracle.com> <4F5F5456.3050604@oracle.com> <4F5F5F47.7020106@oracle.com> <4F5F769A.5070301@oracle.com> Message-ID: On Mar 13, 2012, at 9:32 AM, Jonathan Gibbons wrote: > On 03/13/2012 09:27 AM, Kelly O'Hair wrote: >> On Mar 13, 2012, at 7:52 AM, Fredrik ?hrstr?m wrote: >> >>>> There are details to work through. Another thing about the new build, and >>>> something that Jon mentioned recently, is that the jaxp, jaxws and corba >>>> repos will need to be compiled much later than they are done now (they >>>> are compiled with the bootstrap JDK/javac today). >>> Moving the jaxp, jaxws and corba into the jdk build makes sense even >>> without jigsaw. >>> >>> //Fredrik >> Not sure we want to fold corba, jaxp, and jaxws sources back into jdk repository. We worked hard to untangle them. >> I'm not completely against the idea, but we need to be very careful here and think this through. >> My preferences at this point is to leave them as separate repos and try and work with them that way. >> >> -kto > > I completely agree that we do not want to reorganize the repos here, and we have never considered that. > > However, it would be best if these repos could be built after the base module has been built, so we may have to go back and forth between the repos to get the right build sequencing. While that would be a pain with the existing SUBDIRS-style build, I was hoping/assuming it would be easier with the flat-Makefile-style build, in which make figures out the proper build order based on the dependencies -- i.e. the way make is supposed to be used. My feelings too. I see no choice but a bootstrap sequence (or graph of dependencies really) that involves multiple visits to the repositories. It deserves some thought and probably a great deal of experimentation. Although jigsaw might be triggering the need for an overall 'build graph' re-design, the build-infra-dev project should probably be a major driving force, once we have all been jigsawized. ;^) The name jigsaw had nothing to do with that SAW movie did it? That's a serious disturbing movie, hope they aren't related. ;^) -kto > > -- Jon From aleksey.shipilev at oracle.com Tue Mar 13 10:26:56 2012 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Tue, 13 Mar 2012 21:26:56 +0400 Subject: Compressing SimpleLibrary with GZIP Message-ID: <4F5F8360.3020402@oracle.com> Hi guys, I'm keeping an eye on footprint of Jigsaw library. I understand the code we have not by any means complete, and one should anticipate changes to binary format for modules. However, can we compress it with GZIP at this point? That will require just a few one-liner changes [1], but the benefit is obvious: a. jre-module-image size is down from 161Mb to 158Mb b. dummy multi-module projects (which have plenty of metadata) are consuming 5-6 times less space -Aleksey. [1] diff -r 55c21bd558ac src/share/classes/org/openjdk/jigsaw/SimpleLibrary.java --- a/src/share/classes/org/openjdk/jigsaw/SimpleLibrary.java Tue Mar 06 16:33:00 2012 -0800 +++ b/src/share/classes/org/openjdk/jigsaw/SimpleLibrary.java Tue Mar 13 16:47:34 2012 +0400 @@ -89,8 +89,8 @@ void store() throws IOException { try (OutputStream fos = new FileOutputStream(file); - BufferedOutputStream bos = new BufferedOutputStream(fos); - DataOutputStream out = new DataOutputStream(bos)) { + GZIPOutputStream gos = new GZIPOutputStream(fos, 64*1024); + DataOutputStream out = new DataOutputStream(gos)) { out.writeInt(FileConstants.MAGIC); out.writeShort(type.value()); out.writeShort(majorVersion); @@ -104,8 +104,8 @@ protected void load() throws IOException { try (InputStream fis = new FileInputStream(file); - BufferedInputStream bis = new BufferedInputStream(fis); - DataInputStream in = new DataInputStream(bis)) { + GZIPInputStream gis = new GZIPInputStream(fis, 64*1024); + DataInputStream in = new DataInputStream(gis)) { if (in.readInt() != FileConstants.MAGIC) throw new IOException(file + ": Invalid magic number"); if (in.readShort() != type.value()) From Alan.Bateman at oracle.com Tue Mar 13 12:55:04 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 13 Mar 2012 19:55:04 +0000 Subject: Question about FileManagers in Jigsaw. In-Reply-To: References: <4F5BA223.6040502@oracle.com> <4F5DC915.90706@oracle.com> <4F5E03AD.2050204@oracle.com> <4F5E06CB.3080403@oracle.com> <4F5E58CA.5080406@oracle.com> <4F5F3F2B.1040301@oracle.com> <4F5F4142.20106@oracle.com> <4F5F4277.6040305@oracle.com> <4F5F5456.3050604@oracle.com> <4F5F5F47.7020106@oracle.com> Message-ID: <4F5FA618.5010907@oracle.com> On 13/03/2012 16:27, Kelly O'Hair wrote: > > Not sure we want to fold corba, jaxp, and jaxws sources back into jdk > repository. We worked hard to untangle them. > I'm not completely against the idea, but we need to be very careful > here and think this through. > My preferences at this point is to leave them as separate repos and > try and work with them that way. > One thing about separate repos is that partial builds (if they are supported) would need be able to reconstitute the modules that would otherwise be built if these repos were present. Either that or the jmod packages must be present in the import JDK. -Alan From jonathan.gibbons at oracle.com Tue Mar 13 13:02:11 2012 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Tue, 13 Mar 2012 13:02:11 -0700 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F5FA618.5010907@oracle.com> References: <4F5BA223.6040502@oracle.com> <4F5DC915.90706@oracle.com> <4F5E03AD.2050204@oracle.com> <4F5E06CB.3080403@oracle.com> <4F5E58CA.5080406@oracle.com> <4F5F3F2B.1040301@oracle.com> <4F5F4142.20106@oracle.com> <4F5F4277.6040305@oracle.com> <4F5F5456.3050604@oracle.com> <4F5F5F47.7020106@oracle.com> <4F5FA618.5010907@oracle.com> Message-ID: <4F5FA7C3.8080202@oracle.com> On 03/13/2012 12:55 PM, Alan Bateman wrote: > On 13/03/2012 16:27, Kelly O'Hair wrote: >> >> Not sure we want to fold corba, jaxp, and jaxws sources back into jdk >> repository. We worked hard to untangle them. >> I'm not completely against the idea, but we need to be very careful >> here and think this through. >> My preferences at this point is to leave them as separate repos and >> try and work with them that way. >> > One thing about separate repos is that partial builds (if they are > supported) would need be able to reconstitute the modules that would > otherwise be built if these repos were present. Either that or the > jmod packages must be present in the import JDK. > > -Alan It was my understanding from build-infra that partial builds would be supported, differently. You would always need access to a full forest, but you would be able to do a build of the form, "build here that forest over there, substituting my repo here" -- Jon From Alan.Bateman at oracle.com Tue Mar 13 13:07:38 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 13 Mar 2012 20:07:38 +0000 Subject: Compressing SimpleLibrary with GZIP In-Reply-To: <4F5F8360.3020402@oracle.com> References: <4F5F8360.3020402@oracle.com> Message-ID: <4F5FA90A.3000706@oracle.com> On 13/03/2012 17:26, Aleksey Shipilev wrote: > Hi guys, > > I'm keeping an eye on footprint of Jigsaw library. I understand the code > we have not by any means complete, and one should anticipate changes to > binary format > for modules. > > However, can we compress it with GZIP at this point? That will require > just a few one-liner changes [1], but the benefit is obvious: > > a. jre-module-image size is down from 161Mb to 158Mb > b. dummy multi-module projects (which have plenty of metadata) are > consuming 5-6 times less space > > -Aleksey. > The jmod create command has the -z (or --enable-compression) option to enable compression on the module contents. At this point it's for the module classes only. It's also possible to strip the debug attributes from classes during install via the -G (or --strip-debug option). So it is possible to reduce the disk usage now if it's a problem. Clearly if we were to continue with the existing configuration file then compression should be considered too but that all should change soon. The loading of the config file is a problem for startup performance too. -Alan From fredrik.ohrstrom at oracle.com Tue Mar 13 13:22:17 2012 From: fredrik.ohrstrom at oracle.com (=?UTF-8?B?RnJlZHJpayDDlmhyc3Ryw7Zt?=) Date: Tue, 13 Mar 2012 13:22:17 -0700 (PDT) Subject: Question about FileManagers in Jigsaw. Message-ID: ----- Alan.Bateman at oracle.com skrev: > On 13/03/2012 16:27, Kelly O'Hair wrote: > > > > Not sure we want to fold corba, jaxp, and jaxws sources back into > jdk > > repository. We worked hard to untangle them. > > I'm not completely against the idea, but we need to be very careful > > > here and think this through. > > My preferences at this point is to leave them as separate repos and > > > try and work with them that way. > > > One thing about separate repos is that partial builds (if they are > supported) would need be able to reconstitute the modules that would > otherwise be built if these repos were present. Either that or the > jmod > packages must be present in the import JDK. > How(if) the full product is under version control (repos) and how the parts are built are two orthogonal problems. I was not distinct enough in my statement. Clearly the jaxp,jaxws and corba repos should be built as part of the jdk makefiles, not before. In particular since currently corba/jaxp/jaxws cannot be updated to use a new jdk feature until the feature has propagated to the default bootstrap jdk, which can take quite a while. In Jigsaw, they are just three modules out of many built when building the jdk. Their makefiles should reflect this and be based on the same technology as the other module makefiles. In build-infra/jdk8 we still build corba/jaxp/jaxws before doing the jdk repo. This is probably rather easy to fix, eg ditch the imports and call a submake on those repoes from within the jdk and have OUTPUTDIR point to the correct destination. This would in fact speed up the build, since we would not have zip/unzip sources and jars. I just have not had time to look at it. //Fredrik From fredrik.ohrstrom at oracle.com Tue Mar 13 13:27:31 2012 From: fredrik.ohrstrom at oracle.com (=?UTF-8?B?RnJlZHJpayDDlmhyc3Ryw7Zt?=) Date: Tue, 13 Mar 2012 13:27:31 -0700 (PDT) Subject: Question about FileManagers in Jigsaw. Message-ID: ----- jonathan.gibbons at oracle.com skrev: > You need what I normally call the "hybrid javac", what you call > "bootstrap javac" which is latest code compiled to run on boot jdk. > > That is enough to compile the base module, with a ZeroMod module > resolver. Is the ZeroMod located inside the langtools repo or in the jdk repo? Where do Genstubs come in? > I would think you would run these classes in /legacy/ mode to get to > an initial module image. Initial? Do you mean a bootstrap base module image, needed for the bootstrap javac to compile the real base module? > Hopefully, javac is the only part of the system that needs to be built > twice, i.e. the same as is the case now. Good, I thought that perhaps javac (when introducing jigsaw) would be required to be built thrice. //Fredrik From jonathan.gibbons at oracle.com Tue Mar 13 13:36:20 2012 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Tue, 13 Mar 2012 13:36:20 -0700 Subject: Question about FileManagers in Jigsaw. In-Reply-To: References: Message-ID: <4F5FAFC4.9030506@oracle.com> On 03/13/2012 01:27 PM, Fredrik ?hrstr?m wrote: > ----- jonathan.gibbons at oracle.com skrev: > >> You need what I normally call the "hybrid javac", what you call >> "bootstrap javac" which is latest code compiled to run on boot jdk. >> >> That is enough to compile the base module, with a ZeroMod module >> resolver. > Is the ZeroMod located inside the langtools repo or in the jdk repo? > Where do Genstubs come in? ZeroMod is currently an inner class inside com.sun.tools.javac.comp.Modules. GenStubs is currently used to build the hybrid/bootstrap javac, and the final product javac. >> I would think you would run these classes in /legacy/ mode to get to >> an initial module image. > Initial? Do you mean a bootstrap base module image, needed for the bootstrap javac > to compile the real base module? The hybrid/bootstrap javac initially runs on the BOOTDIR java. The intent is that hybrid javac is good enough to create enough JDK 8 classes which can be run on the new JDK 8 JVM in legacy/classpath mode to create a basic module library containing a real base module. That should be good enough to run hybrid/bootstrap javac such that it now has access to the real Jigsaw library API, and so does not need to use ZeroMod for compiling the rest of JDK. > >> Hopefully, javac is the only part of the system that needs to be built >> twice, i.e. the same as is the case now. > Good, I thought that perhaps javac (when introducing jigsaw) would be > required to be built thrice. well, add SKIP_BOOT_CYCLE into the mix and we'll be up to four times (but not six!) :-) > //Fredrik > > > From fredrik.ohrstrom at oracle.com Tue Mar 13 13:39:56 2012 From: fredrik.ohrstrom at oracle.com (=?UTF-8?B?RnJlZHJpayDDlmhyc3Ryw7Zt?=) Date: Tue, 13 Mar 2012 13:39:56 -0700 (PDT) Subject: Question about FileManagers in Jigsaw. Message-ID: <6b19b1fe-dbfb-4ad1-ae6b-dac498551d42@default> ----- jonathan.gibbons at oracle.com skrev: > It was my understanding from build-infra that partial builds would be > supported, differently. You would always need access to a full forest, > but you would be able to do a build of the form, "build here that > forest over there, substituting my repo here" That is the intention. You have one full checkout, and then for example 50 langtools repoes (langtools1 - langtools50). You create 50 build directories build1 - build50, and type ....fullforest/configure --with-override-source-root=..../langtoolsXX in each of them. Now you can build any configuration simply (cd .../build42 ; make) Or if you put all (build1-build50) below jdk8/build you can type "make ALL" and it will build all 50 at once. The reason for this requirement is that we have to share the common makefiles and setup code between the repoes. It cannot be duplicated in each repo. And secondly it is a good idea to be able to always build the full product. //Fredrik From kelly.ohair at oracle.com Tue Mar 13 13:53:23 2012 From: kelly.ohair at oracle.com (Kelly O'Hair) Date: Tue, 13 Mar 2012 13:53:23 -0700 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F5FA618.5010907@oracle.com> References: <4F5BA223.6040502@oracle.com> <4F5DC915.90706@oracle.com> <4F5E03AD.2050204@oracle.com> <4F5E06CB.3080403@oracle.com> <4F5E58CA.5080406@oracle.com> <4F5F3F2B.1040301@oracle.com> <4F5F4142.20106@oracle.com> <4F5F4277.6040305@oracle.com> <4F5F5456.3050604@oracle.com> <4F5F5F47.7020106@oracle.com> <4F5FA618.5010907@oracle.com> Message-ID: <2434F755-6525-4A50-96A9-D3A4597B9EE6@oracle.com> On Mar 13, 2012, at 12:55 PM, Alan Bateman wrote: > On 13/03/2012 16:27, Kelly O'Hair wrote: >> >> Not sure we want to fold corba, jaxp, and jaxws sources back into jdk repository. We worked hard to untangle them. >> I'm not completely against the idea, but we need to be very careful here and think this through. >> My preferences at this point is to leave them as separate repos and try and work with them that way. >> > One thing about separate repos is that partial builds (if they are supported) would need be able to reconstitute the modules that would otherwise be built if these repos were present. Either that or the jmod packages must be present in the import JDK. I was actually thinking that the import jdk would have the jmod packages. But I'm a jigsaw-newbie at this point. I was assuming that corba, jaxp, and jaxws would deliver a distinct set of modules as their deliverables, and if those repos were not present at build time, the modules could be gotten from the import jdk. HOWEVER, I could also see us just requiring those repositories to always exist and be built when the jdk repository was being built. The assumption was that the build would be so fast that this would never be a speed issue. It might then eventually make sense then that corba, jaxp, and jaxws sit inside the jdk repository, either as nested repos or imported into it, but that's a decision for later. The partial builds are a maintenance burden from my point of view, and there have been no end of problems people have sent me when their partial build did not work. When it was just j2se and hotspot, that was one thing, but the complexity of 7 repos and all combinations of them, well, it's a bit out of control. If we can limit the variations on partial builds, that would be a good thing in my opinion. I had a plan of creating some top level Makefile logic that would enforce what sets of repositories were valid to build in a partial build, e.g. top+hotspot top+langtools+corba+jaxp+jaxws+jdk etc. And consider a stop of the build when certain repositories were missing, or foreign ones were present, or they were located in the wrong place or with the wrong names. But that was a selfish act meant to stop some of the emails I get about failed builds. :^( -kto > > -Alan From aleksey.shipilev at oracle.com Tue Mar 13 14:03:00 2012 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 14 Mar 2012 01:03:00 +0400 Subject: CNFE when resolving "required" dependency Message-ID: <4F5FB604.1050206@oracle.com> Hi, I'm refreshing my performance tests for Jigsaw, and realized my November-old scenarios are broken on current Jigsaw builds. They used to work just fine in November. The basic idea was to explore how multimodule library performs. For that, my home-brew generator can generate chains of modules, which require each other in DAG-like way. However, even the reduced scenario of sample.root -> sample.module0 dependency appears not to be working. This is what I have: $ cat library-c1-d1/modules/sample.root/module-info.java module sample.root at 1.0 { requires sample.module0 at 1.0; class sample.root.Entry; } $ cat library-c1-d1/modules/sample.root/sample/root/Entry.java package sample.root; public class Entry { public static void sayHello() { System.err.println("Hello from module sample.root"); sample.module0.Entry.sayHello(); } public static void main(String[] args) { sayHello(); } } $ cat library-c1-d1/modules/sample.module0/module-info.java module sample.module0 at 1.0 { class sample.module0.Entry; } $ cat library-c1-d1/modules/sample.module0/sample/module0/Entry.java package sample.module0; public class Entry { public static void sayHello() { System.err.println("Hello from module sample.module0"); } public static void main(String[] args) { sayHello(); } } Now the fun part. After compiling it, creating library, etc., this is what I got: $ jdk-module-image/bin/java -L library-c1-d1 -m sample.root Hello from module sample.root Exception in thread "main" java.lang.NoClassDefFoundError: sample/module0/Entry at sample.root.Entry.sayHello(Entry.java:6) at sample.root.Entry.main(Entry.java:9) Caused by: java.lang.ClassNotFoundException: sample.module0.Entry : requested by +sample.root at org.openjdk.jigsaw.Loader.loadClass(Loader.java:113) at java.lang.ClassLoader.loadClass(ClassLoader.java:356) ... 2 more ...then one could argue sample.module0 is indeed missing. However, doing this on the same library succeeds: $ jdk-module-image/bin/java -L library-c1-d1 -m sample.module0 Hello from module sample.module0 Is it something dumb I'm missing here? Or is it a bug? The library itself is somewhat heavy (~100 Kb compressed), so I can send it out off list. Thanks, Aleksey. From fredrik.ohrstrom at oracle.com Tue Mar 13 14:11:10 2012 From: fredrik.ohrstrom at oracle.com (=?UTF-8?B?RnJlZHJpayDDlmhyc3Ryw7Zt?=) Date: Tue, 13 Mar 2012 14:11:10 -0700 (PDT) Subject: Question about FileManagers in Jigsaw. Message-ID: <638148fe-c7e4-4cdd-8c65-e80476de675f@default> ----- jonathan.gibbons at oracle.com skrev: > The hybrid/bootstrap javac initially runs on the BOOTDIR java. The > intent is that hybrid javac is good enough to create enough JDK 8 > classes which can be run on the new JDK 8 JVM in legacy/classpath mode > to create a basic module library containing a real base module. That > should be good enough to run hybrid/bootstrap javac such that it now > has access to the real Jigsaw library API, and so does not need to use > ZeroMod for compiling the rest of JDK. So we bootstrap the entire jdk? Something like this: 1) Build hybrid/bootstrap javac (includes zeromod + fake linking to jdk) 2) Build entire jdk (includes langtools) and jvm using this bootstrap javac. this jdk runs in legacy/classpath mode. But has enough modules (a single base module) to understand proper modules. 3) Build hybrid/bootstrap javac again! Now it does not need zeromod, it still needs genstubs fake linking though. 4) Build the entire jdk (includes langtools) and jvm using this second bootstrap javac. this jdk will have all modules! Is this correct? //Fredrik From aleksey.shipilev at oracle.com Tue Mar 13 14:14:39 2012 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 14 Mar 2012 01:14:39 +0400 Subject: Compressing SimpleLibrary with GZIP In-Reply-To: <4F5FA90A.3000706@oracle.com> References: <4F5F8360.3020402@oracle.com> <4F5FA90A.3000706@oracle.com> Message-ID: <4F5FB8BF.5040505@oracle.com> On 03/14/2012 12:07 AM, Alan Bateman wrote: > On 13/03/2012 17:26, Aleksey Shipilev wrote: > The jmod create command has the -z (or --enable-compression) option to > enable compression on the module contents. At this point it's for the > module classes only. Yes, I noticed that option. It has no effects on my large multimodule tests which have hundreds of modules, metadata dominates there. For the reference, this is the library with "just" 500 modules, and 1 simple class per module: $ du -sk library-c500-d500* 1266020 library-c500-d500.BASELINE 107152 library-c500-d500.GZIP How about that! Is SimpleLibrary considered to be prototype? If so, then enabling compression aids me to do more experiments with it, when I don't have to burn my disks reading and writing gigabytes of metadata. Yes, I've been considering using squashfs to fit all these libraries on my machine, but it seems less appealing than changing a few lines in Jigsaw code. > Clearly if we were to continue with the existing configuration file then > compression should be considered too but that all should change soon. > The loading of the config file is a problem for startup performance too. Sure, that opens up a number of questions. Is there timeline for these changes? Is the binary format for config settled on? Is the internal API for Library more or less stable? If so, we can embark on implementing low-footprint high-performance Library for Jigsaw at this point, leaving SimpleLibrary alone as prototype. -Aleksey. From Alan.Bateman at oracle.com Tue Mar 13 15:02:46 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 13 Mar 2012 22:02:46 +0000 Subject: CNFE when resolving "required" dependency In-Reply-To: <4F5FB604.1050206@oracle.com> References: <4F5FB604.1050206@oracle.com> Message-ID: <4F5FC406.2090900@oracle.com> On 13/03/2012 21:03, Aleksey Shipilev wrote: > Hi, > > I'm refreshing my performance tests for Jigsaw, and realized my > November-old scenarios are broken on current Jigsaw builds. They used to > work just fine in November. The exports work in since then so add "exports sample.module0;" to sample.module's module-info and I assume it will work. This should be caught at compile time, just not yet. -Alan From aleksey.shipilev at oracle.com Tue Mar 13 15:09:41 2012 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 14 Mar 2012 02:09:41 +0400 Subject: CNFE when resolving "required" dependency In-Reply-To: <4F5FC406.2090900@oracle.com> References: <4F5FB604.1050206@oracle.com> <4F5FC406.2090900@oracle.com> Message-ID: <4F5FC5A5.2070707@oracle.com> On 03/14/2012 02:02 AM, Alan Bateman wrote: > On 13/03/2012 21:03, Aleksey Shipilev wrote: >> Hi, >> >> I'm refreshing my performance tests for Jigsaw, and realized my >> November-old scenarios are broken on current Jigsaw builds. They used to >> work just fine in November. > The exports work in since then so add "exports sample.module0;" to > sample.module's module-info and I assume it will work. This should be > caught at compile time, just not yet. Oh yes! That made the trick. Thanks, Aleksey. From jonathan.gibbons at oracle.com Tue Mar 13 15:26:06 2012 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Tue, 13 Mar 2012 15:26:06 -0700 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <6b19b1fe-dbfb-4ad1-ae6b-dac498551d42@default> References: <6b19b1fe-dbfb-4ad1-ae6b-dac498551d42@default> Message-ID: <4F5FC97E.1030008@oracle.com> On 03/13/2012 02:11 PM, Fredrik ?hrstr?m wrote: > ----- jonathan.gibbons at oracle.com skrev: > >> The hybrid/bootstrap javac initially runs on the BOOTDIR java. The >> intent is that hybrid javac is good enough to create enough JDK 8 >> classes which can be run on the new JDK 8 JVM in legacy/classpath mode >> to create a basic module library containing a real base module. That >> should be good enough to run hybrid/bootstrap javac such that it now >> has access to the real Jigsaw library API, and so does not need to use >> ZeroMod for compiling the rest of JDK. > So we bootstrap the entire jdk? Something like this: > 1) Build hybrid/bootstrap javac (includes zeromod + fake linking to jdk) > 2) Build entire jdk (includes langtools) and jvm using this bootstrap > javac. > this jdk runs in legacy/classpath mode. But has enough modules (a > single base module) > to understand proper modules. > 3) Build hybrid/bootstrap javac again! Now it does not need zeromod, > it still needs genstubs fake linking though. > 4) Build the entire jdk (includes langtools) and jvm using this second > bootstrap javac. > this jdk will have all modules! > > Is this correct? > > //Fredrik Well, not quite. If I understand correctly what you have written, you're still building javac 3 times, in 1, 3, 4, and you're building JDK twice. Step 3 is redundant and equivalent to 1. ZeroMod is always present: it is currently a small nested class in javac.comp.Modules. Whether or not to use it is a runtime decision largely based on the availability of the jigsaw library API. ZeroMod is substantially less functional than the full Jigsaw module resolver, and so can only be used in limited situations, such as the base module. Step 4 should be unnecessary. And, you need to build Hotspot and all the native libraries. So, here's what Mandy and I think we need: # start bootstrap 1) Build hybrid/bootstrap javac using BOOTDIR javac on BOOTDIR java (includes zeromod + fake linking to jdk) # build minimal new JDK runtime 2a) Compile Hotspot 2b) Compile base module using hybrid/bootstrap javac on BOOTDIR java 2c) Compile native libs for base module # build rest of JDK modules 3a) Compile all other modules, including langtools, using hybrid/bootstrap javac on new JDK runtime in classpath mode 3b) Compile native libs for each module # Dev usage 4) Devs can work with and run the modules compiled in 2, 3 using modulepath # Product bits 5) For final image, modules compiled in 2, 3 get converted to .jmod files and installed in library, using new jmod and jpkg on new JDK runtime -- Jon From Alan.Bateman at oracle.com Tue Mar 13 15:43:06 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 13 Mar 2012 22:43:06 +0000 Subject: Compressing SimpleLibrary with GZIP In-Reply-To: <4F5FB8BF.5040505@oracle.com> References: <4F5F8360.3020402@oracle.com> <4F5FA90A.3000706@oracle.com> <4F5FB8BF.5040505@oracle.com> Message-ID: <4F5FCD7A.2080108@oracle.com> On 13/03/2012 21:14, Aleksey Shipilev wrote: > : > Yes, I noticed that option. It has no effects on my large multimodule > tests which have hundreds of modules, metadata dominates there. For the > reference, this is the library with "just" 500 modules, and 1 simple > class per module: > > $ du -sk library-c500-d500* > 1266020 library-c500-d500.BASELINE > 107152 library-c500-d500.GZIP > > How about that! > > Is SimpleLibrary considered to be prototype? If so, then enabling > compression aids me to do more experiments with it, when I don't have to > burn my disks reading and writing gigabytes of metadata. Yes, I've been > considering using squashfs to fit all these libraries on my machine, but > it seems less appealing than changing a few lines in Jigsaw code. Sorry you're running into the issue with size of the meta data, that does need to be fixed, hopefully over the next few months. I wouldn't object to compressing the config file in the short term. As regards SimpleLibrary then it will likely evolve, maybe in the future there will be other Library implementations too. -Alan From aleksey.shipilev at oracle.com Tue Mar 13 15:50:33 2012 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 14 Mar 2012 02:50:33 +0400 Subject: Compressing SimpleLibrary with GZIP In-Reply-To: <4F5FCD7A.2080108@oracle.com> References: <4F5F8360.3020402@oracle.com> <4F5FA90A.3000706@oracle.com> <4F5FB8BF.5040505@oracle.com> <4F5FCD7A.2080108@oracle.com> Message-ID: <4F5FCF39.9070900@oracle.com> On 03/14/2012 02:43 AM, Alan Bateman wrote: > On 13/03/2012 21:14, Aleksey Shipilev wrote: > Sorry you're running into the issue with size of the meta data, that > does need to be fixed, hopefully over the next few months. That's not a problem, that's actually fun :) > I wouldn't > object to compressing the config file in the short term. As regards > SimpleLibrary then it will likely evolve, maybe in the future there will > be other Library implementations too. As my multimodule stuff is finally working, I can do startup tests on refreshed Jigsaw bundle. Some interesting results are showing up w.r.t. compression, and reading the config. But all-in-all, Jigsaw seems to be in *much* better shape performance-wise comparing to what it was in November. Will update you tomorrow. Am I reading your comment right that we should tune up the existing SimpleLibrary code instead of considering alternative implementations? It wouldn't make sense if there's a plan to trash SimpleLibrary. -Aleksey. From kelly.ohair at oracle.com Tue Mar 13 16:30:05 2012 From: kelly.ohair at oracle.com (Kelly O'Hair) Date: Tue, 13 Mar 2012 16:30:05 -0700 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F5FC97E.1030008@oracle.com> References: <6b19b1fe-dbfb-4ad1-ae6b-dac498551d42@default> <4F5FC97E.1030008@oracle.com> Message-ID: On Mar 13, 2012, at 3:26 PM, Jonathan Gibbons wrote: > On 03/13/2012 02:11 PM, Fredrik ?hrstr?m wrote: >> ----- jonathan.gibbons at oracle.com skrev: >> >>> The hybrid/bootstrap javac initially runs on the BOOTDIR java. The >>> intent is that hybrid javac is good enough to create enough JDK 8 >>> classes which can be run on the new JDK 8 JVM in legacy/classpath mode >>> to create a basic module library containing a real base module. That >>> should be good enough to run hybrid/bootstrap javac such that it now >>> has access to the real Jigsaw library API, and so does not need to use >>> ZeroMod for compiling the rest of JDK. >> So we bootstrap the entire jdk? Something like this: >> 1) Build hybrid/bootstrap javac (includes zeromod + fake linking to jdk) >> 2) Build entire jdk (includes langtools) and jvm using this bootstrap javac. >> this jdk runs in legacy/classpath mode. But has enough modules (a single base module) >> to understand proper modules. >> 3) Build hybrid/bootstrap javac again! Now it does not need zeromod, it still needs genstubs fake linking though. >> 4) Build the entire jdk (includes langtools) and jvm using this second bootstrap javac. >> this jdk will have all modules! >> >> Is this correct? >> >> //Fredrik > > Well, not quite. If I understand correctly what you have written, you're still building javac 3 times, in 1, 3, 4, and you're building JDK twice. > > Step 3 is redundant and equivalent to 1. ZeroMod is always present: it is currently a small nested class in javac.comp.Modules. Whether or not to use it is a runtime decision largely based on the availability of the jigsaw library API. ZeroMod is substantially less functional than the full Jigsaw module resolver, and so can only be used in limited situations, such as the base module. > > Step 4 should be unnecessary. > > And, you need to build Hotspot and all the native libraries. > > > So, here's what Mandy and I think we need: > > # start bootstrap > 1) Build hybrid/bootstrap javac using BOOTDIR javac on BOOTDIR java (includes zeromod + fake linking to jdk) > > # build minimal new JDK runtime > 2a) Compile Hotspot Just the C++ code of Hotspot, probably not the Java code. (Currently the hotspot Java code is built by the BOOT javac, which I think is wrong). And I think 2a 2b and 2c can be done in parallel? > 2b) Compile base module using hybrid/bootstrap javac on BOOTDIR java > 2c) Compile native libs for base module > > # build rest of JDK modules > 3a) Compile all other modules, including langtools, using hybrid/bootstrap javac on new JDK runtime in classpath mode > 3b) Compile native libs for each module Step 3 is where I think we may need to visit lots of repos, right? This is where parallel building is a big win I think. -kto > > # Dev usage > 4) Devs can work with and run the modules compiled in 2, 3 using modulepath > > # Product bits > 5) For final image, modules compiled in 2, 3 get converted to .jmod files and installed in library, using new jmod and jpkg on new JDK runtime > > > -- Jon From jonathan.gibbons at oracle.com Tue Mar 13 16:51:53 2012 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Tue, 13 Mar 2012 16:51:53 -0700 Subject: Question about FileManagers in Jigsaw. In-Reply-To: References: <6b19b1fe-dbfb-4ad1-ae6b-dac498551d42@default> <4F5FC97E.1030008@oracle.com> Message-ID: <4F5FDD99.9040009@oracle.com> On 03/13/2012 04:30 PM, Kelly O'Hair wrote: >> > >> > # build minimal new JDK runtime >> > 2a) Compile Hotspot > Just the C++ code of Hotspot, probably not the Java code. (Currently the hotspot Java code is built by the BOOT javac, which I think is wrong). > > And I think 2a 2b and 2c can be done in parallel? > >> > 2b) Compile base module using hybrid/bootstrap javac on BOOTDIR java >> > 2c) Compile native libs for base module The goal of 2{a,b,c} is to get a minimal runtime environment up and running. So do what you need and exclude the rest. Native libs (in 2c) require native headers, so you need to choose between use traditional javah, or use the newly added "javac -h" option. See links below. "javac -h" provides a simpler way to get the minimal accurate set of native header files, and avoids lots of custom make macros specifying args for javah, but comes at the expense of waiting for javac to complete. http://monaco.us.oracle.com/detail.jsf?cr=7150368 http://hg.openjdk.java.net/jdk8/tl/langtools/rev/b14d9583ce92 -- Jon From mandy.chung at oracle.com Tue Mar 13 17:32:32 2012 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 13 Mar 2012 17:32:32 -0700 Subject: Question about FileManagers in Jigsaw. In-Reply-To: References: <6b19b1fe-dbfb-4ad1-ae6b-dac498551d42@default> <4F5FC97E.1030008@oracle.com> Message-ID: <4F5FE720.3080706@oracle.com> On 3/13/2012 4:30 PM, Kelly O'Hair wrote: >> # build minimal new JDK runtime >> > 2a) Compile Hotspot > Just the C++ code of Hotspot, probably not the Java code. (Currently the hotspot Java code is built by the BOOT javac, which I think is wrong). The base module only needs the VM (C++ code). The hotspot Java code (SA) will be compiled in step 3 as they are in other modules. > > And I think 2a 2b and 2c can be done in parallel? It would depend if the native libraries for the base module have any static dependencies on the "new" symbols exported in the VM. We have been doing this kind of synchronized hotspot/jdk change by integrating the hotspot change first and then integrate the jdk change in the next promoted build and the jdk native libs can link against the new hotspot VM. Mandy From Alan.Bateman at oracle.com Wed Mar 14 08:28:17 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 14 Mar 2012 15:28:17 +0000 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F5FC97E.1030008@oracle.com> References: <6b19b1fe-dbfb-4ad1-ae6b-dac498551d42@default> <4F5FC97E.1030008@oracle.com> Message-ID: <4F60B911.3050705@oracle.com> On 13/03/2012 22:26, Jonathan Gibbons wrote: > : > > > So, here's what Mandy and I think we need: > > # start bootstrap > 1) Build hybrid/bootstrap javac using BOOTDIR javac on BOOTDIR java > (includes zeromod + fake linking to jdk) > > # build minimal new JDK runtime > 2a) Compile Hotspot > 2b) Compile base module using hybrid/bootstrap javac on BOOTDIR java > 2c) Compile native libs for base module > > # build rest of JDK modules > 3a) Compile all other modules, including langtools, using > hybrid/bootstrap javac on new JDK runtime in classpath mode > 3b) Compile native libs for each module > > # Dev usage > 4) Devs can work with and run the modules compiled in 2, 3 using > modulepath > > # Product bits > 5) For final image, modules compiled in 2, 3 get converted to .jmod > files and installed in library, using new jmod and jpkg on new JDK > runtime > > > -- Jon Just catching up from yesterday. For the minimal runtime at step 2 then do you see this as a minimal legacy or module runtime? Just thinking about legacy mode being implemented with a jdk.legacy module under the covers. Also at 3a then I'm curious about compiling in classpath mode rather than modulepath. -Alan. From jonathan.gibbons at oracle.com Wed Mar 14 08:50:00 2012 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Wed, 14 Mar 2012 08:50:00 -0700 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F60B911.3050705@oracle.com> References: <6b19b1fe-dbfb-4ad1-ae6b-dac498551d42@default> <4F5FC97E.1030008@oracle.com> <4F60B911.3050705@oracle.com> Message-ID: <4F60BE28.4020702@oracle.com> On 03/14/2012 08:28 AM, Alan Bateman wrote: > On 13/03/2012 22:26, Jonathan Gibbons wrote: >> : >> >> >> So, here's what Mandy and I think we need: >> >> # start bootstrap >> 1) Build hybrid/bootstrap javac using BOOTDIR javac on BOOTDIR java >> (includes zeromod + fake linking to jdk) >> >> # build minimal new JDK runtime >> 2a) Compile Hotspot >> 2b) Compile base module using hybrid/bootstrap javac on BOOTDIR java >> 2c) Compile native libs for base module >> >> # build rest of JDK modules >> 3a) Compile all other modules, including langtools, using >> hybrid/bootstrap javac on new JDK runtime in classpath mode >> 3b) Compile native libs for each module >> >> # Dev usage >> 4) Devs can work with and run the modules compiled in 2, 3 using >> modulepath >> >> # Product bits >> 5) For final image, modules compiled in 2, 3 get converted to .jmod >> files and installed in library, using new jmod and jpkg on new JDK >> runtime >> >> >> -- Jon > Just catching up from yesterday. > > For the minimal runtime at step 2 then do you see this as a minimal > legacy or module runtime? Just thinking about legacy mode being > implemented with a jdk.legacy module under the covers. Whatever necessary to support hybrid/bootstrap javac being able to run and have access to the Jigsaw module resolver. > > Also at 3a then I'm curious about compiling in classpath mode rather > than modulepath. We cannot compile a module-info.java for hybrid/bootstrap javac using BOOTDIR javac. If we were to use modulepath, we would need to add a step to use hybrid/bootstrap javac to compile its own module-info.java using either (BOOTDIR java + ZeroMod) or (minimal new runtime in classpath mode). > > -Alan. > > From Alan.Bateman at oracle.com Wed Mar 14 08:52:46 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 14 Mar 2012 15:52:46 +0000 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <2434F755-6525-4A50-96A9-D3A4597B9EE6@oracle.com> References: <4F5BA223.6040502@oracle.com> <4F5DC915.90706@oracle.com> <4F5E03AD.2050204@oracle.com> <4F5E06CB.3080403@oracle.com> <4F5E58CA.5080406@oracle.com> <4F5F3F2B.1040301@oracle.com> <4F5F4142.20106@oracle.com> <4F5F4277.6040305@oracle.com> <4F5F5456.3050604@oracle.com> <4F5F5F47.7020106@oracle.com> <4F5FA618.5010907@oracle.com> <2434F755-6525-4A50-96A9-D3A4597B9EE6@oracle.com> Message-ID: <4F60BECE.8040102@oracle.com> On 13/03/2012 20:53, Kelly O'Hair wrote: > : > I was actually thinking that the import jdk would have the jmod packages. > But I'm a jigsaw-newbie at this point. > > I was assuming that corba, jaxp, and jaxws would deliver a distinct set of modules as their deliverables, > and if those repos were not present at build time, the modules could be gotten from the import jdk. > HOWEVER, I could also see us just requiring those repositories to always exist and be built when the > jdk repository was being built. The assumption was that the build would be so fast that this would never be > a speed issue. It might then eventually make sense then that corba, jaxp, and jaxws sit inside the jdk repository, > either as nested repos or imported into it, but that's a decision for later. > > The partial builds are a maintenance burden from my point of view, and there have been no end of > problems people have sent me when their partial build did not work. > When it was just j2se and hotspot, that was one thing, but the complexity of 7 repos and all combinations > of them, well, it's a bit out of control. If we can limit the variations on partial builds, that would be a good thing > in my opinion. > > I had a plan of creating some top level Makefile logic that would enforce what sets of repositories were > valid to build in a partial build, e.g. top+hotspot top+langtools+corba+jaxp+jaxws+jdk etc. > And consider a stop of the build when certain repositories were missing, or foreign ones were present, or > they were located in the wrong place or with the wrong names. > But that was a selfish act meant to stop some of the emails I get about failed builds. :^( > > -kto > Yep, partial builds can be a pain, folks are always mixing up the bootstrap and import JDKs. So if certain sets of repos are required to be present then, and the build is speedy as expected, then it means that jaxp + jaxws+ corba (+ maybe others) shouldn't be a major issue but they may require separate compilations due to different source code organization. I suspect that hotspot might be tricky as the SA classes are needed for the jdk.tools module (in the current prototype) and also complicated by the fact that they implement JDI interfaces. It might be interesting to put jvmti.h on the table too to avoid having to check in the output of the build into the jdk repo. -Alan. From forax at univ-mlv.fr Wed Mar 14 08:54:50 2012 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Wed, 14 Mar 2012 16:54:50 +0100 Subject: JEP 153: Launch JavaFX Applications In-Reply-To: <20120314154622.3C43D371@eggemoggin.niobe.net> References: <20120314154622.3C43D371@eggemoggin.niobe.net> Message-ID: <4F60BF4A.40202@univ-mlv.fr> On 03/14/2012 04:46 PM, mark.reinhold at oracle.com wrote: > Posted: http://openjdk.java.net/jeps/153 > > - Mark How this will be integrated with module main class ? R?mi From Alan.Bateman at oracle.com Wed Mar 14 09:01:15 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 14 Mar 2012 16:01:15 +0000 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F60BE28.4020702@oracle.com> References: <6b19b1fe-dbfb-4ad1-ae6b-dac498551d42@default> <4F5FC97E.1030008@oracle.com> <4F60B911.3050705@oracle.com> <4F60BE28.4020702@oracle.com> Message-ID: <4F60C0CB.5000301@oracle.com> On 14/03/2012 15:50, Jonathan Gibbons wrote: > : > >> >> Also at 3a then I'm curious about compiling in classpath mode rather >> than modulepath. > > We cannot compile a module-info.java for hybrid/bootstrap javac using > BOOTDIR javac. If we were to use modulepath, we would need to add a > step to use hybrid/bootstrap javac to compile its own module-info.java > using either (BOOTDIR java + ZeroMod) or (minimal new runtime in > classpath mode). Okay, you're the expert. I kinda assumed that you would want to switch from the hybrid/bootstrap javac to the javac module running on the base module so that subsequent compilation is done using module path mode. -Alan From jonathan.gibbons at oracle.com Wed Mar 14 09:07:25 2012 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Wed, 14 Mar 2012 09:07:25 -0700 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F60C0CB.5000301@oracle.com> References: <6b19b1fe-dbfb-4ad1-ae6b-dac498551d42@default> <4F5FC97E.1030008@oracle.com> <4F60B911.3050705@oracle.com> <4F60BE28.4020702@oracle.com> <4F60C0CB.5000301@oracle.com> Message-ID: <4F60C23D.4050409@oracle.com> On 03/14/2012 09:01 AM, Alan Bateman wrote: > On 14/03/2012 15:50, Jonathan Gibbons wrote: >> : >> >>> >>> Also at 3a then I'm curious about compiling in classpath mode rather >>> than modulepath. >> >> We cannot compile a module-info.java for hybrid/bootstrap javac using >> BOOTDIR javac. If we were to use modulepath, we would need to add a >> step to use hybrid/bootstrap javac to compile its own >> module-info.java using either (BOOTDIR java + ZeroMod) or (minimal >> new runtime in classpath mode). > Okay, you're the expert. I kinda assumed that you would want to switch > from the hybrid/bootstrap javac to the javac module running on the > base module so that subsequent compilation is done using module path > mode. > > -Alan The main thing is to get the build completed as quickly as possible, with as few hacks as possible. With that in mind: we'll do whatever works best ;-) I guess I have in mind that we'll start building the "new jigsaw build" from the bottom up, leveraging all the work and experience from the existing build-infra project. -- Jon From Alan.Bateman at oracle.com Wed Mar 14 09:09:08 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 14 Mar 2012 16:09:08 +0000 Subject: JEP 153: Launch JavaFX Applications In-Reply-To: <4F60BF4A.40202@univ-mlv.fr> References: <20120314154622.3C43D371@eggemoggin.niobe.net> <4F60BF4A.40202@univ-mlv.fr> Message-ID: <4F60C2A4.3050109@oracle.com> On 14/03/2012 15:54, R?mi Forax wrote: > On 03/14/2012 04:46 PM, mark.reinhold at oracle.com wrote: >> Posted: http://openjdk.java.net/jeps/153 >> >> - Mark > > How this will be integrated with module main class ? > > R?mi It probably should be looked at again in the context of modules. FWIW, Mandy had the FX runtime and a few FX demos installed as modules at last year's JavaOne. -Alan From mandy.chung at oracle.com Wed Mar 14 09:11:35 2012 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 14 Mar 2012 09:11:35 -0700 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F60BE28.4020702@oracle.com> References: <6b19b1fe-dbfb-4ad1-ae6b-dac498551d42@default> <4F5FC97E.1030008@oracle.com> <4F60B911.3050705@oracle.com> <4F60BE28.4020702@oracle.com> Message-ID: <4F60C337.2070604@oracle.com> On 3/14/2012 8:50 AM, Jonathan Gibbons wrote: > On 03/14/2012 08:28 AM, Alan Bateman wrote: >> >> For the minimal runtime at step 2 then do you see this as a minimal >> legacy or module runtime? Just thinking about legacy mode being >> implemented with a jdk.legacy module under the covers. > > Whatever necessary to support hybrid/bootstrap javac being able to run > and have access to the Jigsaw module resolver. I was thinking that the minimal runtime at step 2 would support both legacy and module mode. The legacy mode in this minimal runtime will probably be a special case that won't require jdk.legacy to be present like the modulepath runtime support for only jdk.base be present. Mandy From kelly.ohair at oracle.com Wed Mar 14 09:58:10 2012 From: kelly.ohair at oracle.com (Kelly O'Hair) Date: Wed, 14 Mar 2012 09:58:10 -0700 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <4F60BECE.8040102@oracle.com> References: <4F5BA223.6040502@oracle.com> <4F5DC915.90706@oracle.com> <4F5E03AD.2050204@oracle.com> <4F5E06CB.3080403@oracle.com> <4F5E58CA.5080406@oracle.com> <4F5F3F2B.1040301@oracle.com> <4F5F4142.20106@oracle.com> <4F5F4277.6040305@oracle.com> <4F5F5456.3050604@oracle.com> <4F5F5F47.7020106@oracle.com> <4F5FA618.5010907@oracle.com> <2434F755-6525-4A50-96A9-D3A4597B9EE6@oracle.com> <4F60BECE.8040102@oracle.com> Message-ID: <24535E1D-7FBE-4833-9675-B01288DAF2B1@oracle.com> On Mar 14, 2012, at 8:52 AM, Alan Bateman wrote: > I suspect that hotspot might be tricky as the SA classes are needed for the jdk.tools module (in the current prototype) and also complicated by the fact that they implement JDI interfaces. It might be interesting to put jvmti.h on the table too to avoid having to check in the output of the build into the jdk repo. > > -Alan. > That's CR 6223012: Sync process for Hotspot/J2SE shared definitions (header files, etc...) --- For the sake of others... certain files have been copied between repositories, things like jni.h etc. should live in and be owned by the hotspot repo or VM team, but we have a separate version of jni.h in the jdk repository. The jvmti.h file is actually generated by the hotspot build, but we have a copy of a past generated jvmti.h stored in the jdk repository. This is an old issue which has been tricky to resolve. --- In my view the hotspot deliverables should be a bundle that include the shared libraries (libjvm.so etc.) and the include files needed to use them (jni.h, jvmti.h, etc.), and anything else that it delivers into the jdk image (SA jar files, JVM TI technical documentation that is generated, etc.). But do we really want to drag that issue in and toss it on the camel too? :^( I'd like to get a fix too, but no promises. -kto From Alan.Bateman at oracle.com Wed Mar 14 10:10:37 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 14 Mar 2012 17:10:37 +0000 Subject: Question about FileManagers in Jigsaw. In-Reply-To: <24535E1D-7FBE-4833-9675-B01288DAF2B1@oracle.com> References: <4F5BA223.6040502@oracle.com> <4F5DC915.90706@oracle.com> <4F5E03AD.2050204@oracle.com> <4F5E06CB.3080403@oracle.com> <4F5E58CA.5080406@oracle.com> <4F5F3F2B.1040301@oracle.com> <4F5F4142.20106@oracle.com> <4F5F4277.6040305@oracle.com> <4F5F5456.3050604@oracle.com> <4F5F5F47.7020106@oracle.com> <4F5FA618.5010907@oracle.com> <2434F755-6525-4A50-96A9-D3A4597B9EE6@oracle.com> <4F60BECE.8040102@oracle.com> <24535E1D-7FBE-4833-9675-B01288DAF2B1@oracle.com> Message-ID: <4F60D10D.5000905@oracle.com> On 14/03/2012 16:58, Kelly O'Hair wrote: > : > > That's CR 6223012: Sync process for Hotspot/J2SE shared definitions > (header files, etc...) > > --- > For the sake of others... certain files have been copied between > repositories, things like jni.h etc. should live > in and be owned by the hotspot repo or VM team, but we have a separate > version of jni.h in the jdk repository. > The jvmti.h file is actually generated by the hotspot build, but we > have a copy of a past generated jvmti.h > stored in the jdk repository. This is an old issue which has been > tricky to resolve. > --- > > In my view the hotspot deliverables should be a bundle that include > the shared libraries (libjvm.so etc.) > and the include files needed to use them (jni.h, jvmti.h, etc.), and > anything else that it delivers into the > jdk image (SA jar files, JVM TI technical documentation that is > generated, etc.). > > But do we really want to drag that issue in and toss it on the camel > too? :^( > I'd like to get a fix too, but no promises. > > -kto You're right, the header and other files can wait until the camel comes back for another load :-) But the SA classes do need consideration. If the hotspot repo is being built for jdk7 then it will need to emit sa-jdi.jar as always. For a modules build then there are probably choices but ultimately they will need be packaged in some module, currently the jdk.tools module. -Alan From aleksey.shipilev at oracle.com Wed Mar 14 11:59:02 2012 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 14 Mar 2012 22:59:02 +0400 Subject: Instrumented profiling for multimodule Jigsaw startup Message-ID: <4F60EA76.5070305@oracle.com> Ok, here's the first bunch of data from me. The setup is: - Intel i5 520M, 8 Gb DDR3, running Ubuntu 11.10 i686 - personal Jigsaw build, as of 2012-03-14; this build is also instrumented with multiple logging messages all around Java launcher, JDK, and Hotspot. - simple Perl script annotates output with timestamps; that is better than synchronizing timers between JDK and JVM code; the tracing code incurs non-negligible overhead, but one can see *relative* times spent in each section. - multimodule library with 500 modules, all in single dependency chain; each module references one other module, and invokes the method from there, forcing Jigsaw to actually load the modules at runtime - two measurement modes: a) hot startup, when file caches are warmed up, and b) cold startup, where drop_caches = 3 was set before the run, effectively forcing kernel to drop the read cache I had also run this setup for two cases: invoking from the dependency root (i.e. forcing loading all the modules) and from the dependency leaf (i.e. loading only one user module). The results are attached. In short, without tracing: 23.4 secs to start up in cold mode, loading 500 modules 1.7 secs to start up in warm mode, loading 500 modules 2.7 secs to start up in cold mode, loading single module 0.6 secs to start up in warm mode, loading single module Note that last two scenarios are effectively simple "HelloWorld" scenarios. Some observations on these results: A. Current Jigsaw build is actually starting up pretty fast. It was was somewhat 20s to load 500 modules in *warm* mode in November; now it's only 1.5s, which is a nice improvement. B. The major performance hog is reading module configuration. Even in warm mode, it takes around 500ms to load local class map. It seems to be connected with the size of it. I'm not sure if compressing will help here, will explore a bit more. C. After major init phase is over, the overhead of loading the class from other module seems to be OK, at least in warm mode; it takes some 1ms to load one up. It gets much worse in cold mode, where the cost of doing real I/O boosts that time up to 30 ms. (I would think it would be better if I have an SSD, but anyway). Thanks, Aleksey. From aleksey.shipilev at oracle.com Wed Mar 14 12:02:07 2012 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 14 Mar 2012 23:02:07 +0400 Subject: Instrumented profiling for multimodule Jigsaw startup In-Reply-To: <4F60EA76.5070305@oracle.com> References: <4F60EA76.5070305@oracle.com> Message-ID: <4F60EB2F.9060909@oracle.com> (Trying again with attachments) -Aleksey. On 03/14/2012 10:59 PM, Aleksey Shipilev wrote: > Ok, here's the first bunch of data from me. > > The setup is: > - Intel i5 520M, 8 Gb DDR3, running Ubuntu 11.10 i686 > - personal Jigsaw build, as of 2012-03-14; this build is also > instrumented with multiple logging messages all around Java launcher, > JDK, and Hotspot. > - simple Perl script annotates output with timestamps; that is better > than synchronizing timers between JDK and JVM code; the tracing code > incurs non-negligible overhead, but one can see *relative* times spent > in each section. > - multimodule library with 500 modules, all in single dependency > chain; each module references one other module, and invokes the method > from there, forcing Jigsaw to actually load the modules at runtime > - two measurement modes: a) hot startup, when file caches are warmed > up, and b) cold startup, where drop_caches = 3 was set before the run, > effectively forcing kernel to drop the read cache > > I had also run this setup for two cases: invoking from the dependency > root (i.e. forcing loading all the modules) and from the dependency leaf > (i.e. loading only one user module). > > The results are attached. > > In short, without tracing: > 23.4 secs to start up in cold mode, loading 500 modules > 1.7 secs to start up in warm mode, loading 500 modules > 2.7 secs to start up in cold mode, loading single module > 0.6 secs to start up in warm mode, loading single module > > Note that last two scenarios are effectively simple "HelloWorld" scenarios. > > Some observations on these results: > A. Current Jigsaw build is actually starting up pretty fast. It was > was somewhat 20s to load 500 modules in *warm* mode in November; now > it's only 1.5s, which is a nice improvement. > > B. The major performance hog is reading module configuration. Even in > warm mode, it takes around 500ms to load local class map. It seems to be > connected with the size of it. I'm not sure if compressing will help > here, will explore a bit more. > > C. After major init phase is over, the overhead of loading the class > from other module seems to be OK, at least in warm mode; it takes some > 1ms to load one up. It gets much worse in cold mode, where the cost of > doing real I/O boosts that time up to 30 ms. (I would think it would be > better if I have an SSD, but anyway). > > Thanks, > Aleksey. From jesse.glick at oracle.com Wed Mar 14 12:53:49 2012 From: jesse.glick at oracle.com (Jesse Glick) Date: Wed, 14 Mar 2012 15:53:49 -0400 Subject: Instrumented profiling for multimodule Jigsaw startup In-Reply-To: <4F60EB2F.9060909@oracle.com> References: <4F60EA76.5070305@oracle.com> <4F60EB2F.9060909@oracle.com> Message-ID: <4F60F74D.6080100@oracle.com> On 03/14/2012 03:02 PM, Aleksey Shipilev wrote: > (Trying again with attachments) They get rejected from this list, I am afraid - you have to inline everything. From aleksey.shipilev at gmail.com Wed Mar 14 13:15:28 2012 From: aleksey.shipilev at gmail.com (Aleksey Shipilev) Date: Thu, 15 Mar 2012 00:15:28 +0400 Subject: Instrumented profiling for multimodule Jigsaw startup In-Reply-To: <4F60F74D.6080100@oracle.com> References: <4F60EA76.5070305@oracle.com> <4F60EB2F.9060909@oracle.com> <4F60F74D.6080100@oracle.com> Message-ID: On Wed, Mar 14, 2012 at 11:53 PM, Jesse Glick wrote: > On 03/14/2012 03:02 PM, Aleksey Shipilev wrote: >> >> (Trying again with attachments) > > They get rejected from this list, I am afraid - you have to inline > everything. That's actually quite inconvenient. Inlining will cause more trouble with both formatting and consuming more space (since I can't inline gzip archive without ASCII-armoring it). Anyway, I had uploaded these two attachments to: http://people.apache.org/~shade/you_should_really_allow_sending_small_gzipped_attachments_to_the_lists/jigsaw-startup-500-root.log http://people.apache.org/~shade/you_should_really_allow_sending_small_gzipped_attachments_to_the_lists/jigsaw-startup-500-leaf.log ;) Thanks, Aleksey. From Alan.Bateman at oracle.com Wed Mar 14 14:50:57 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 14 Mar 2012 21:50:57 +0000 Subject: Instrumented profiling for multimodule Jigsaw startup In-Reply-To: <4F60EA76.5070305@oracle.com> References: <4F60EA76.5070305@oracle.com> Message-ID: <4F6112C1.2050701@oracle.com> On 14/03/2012 18:59, Aleksey Shipilev wrote: > : > > B. The major performance hog is reading module configuration. Even in > warm mode, it takes around 500ms to load local class map. It seems to be > connected with the size of it. I'm not sure if compressing will help > here, will explore a bit more. > Thanks for the data. Yes, the configuration loading is a bit embarrassing and hopefully should be addressed soon. -Alan. From aleksey.shipilev at gmail.com Wed Mar 14 15:16:11 2012 From: aleksey.shipilev at gmail.com (Aleksey Shipilev) Date: Thu, 15 Mar 2012 02:16:11 +0400 Subject: Instrumented profiling for multimodule Jigsaw startup In-Reply-To: <4F6112C1.2050701@oracle.com> References: <4F60EA76.5070305@oracle.com> <4F6112C1.2050701@oracle.com> Message-ID: On Thu, Mar 15, 2012 at 1:50 AM, Alan Bateman wrote: > On 14/03/2012 18:59, Aleksey Shipilev wrote: >> >> : >> >> ? B. The major performance hog is reading module configuration. Even in >> warm mode, it takes around 500ms to load local class map. It seems to be >> connected with the size of it. I'm not sure if compressing will help >> here, will explore a bit more. >> > Thanks for the data. Yes, the configuration loading is a bit embarrassing > and hopefully should be addressed soon. The thing I missed from explanation there: if you churn those 500ms local class map loading out of 600ms of total startup time, modularized Jigsaw startup is on-par with "legacy" JDK8 startup for HelloWorld tests. Do you want me to see what could be addressed in configuration, or is it someone else's part already? -Aleksey. From aleksey.shipilev at oracle.com Thu Mar 15 05:03:16 2012 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Thu, 15 Mar 2012 16:03:16 +0400 Subject: [performance] Re-arranging maps in config metadata improves startup time Message-ID: <4F61DA84.6000804@oracle.com> Hi, This is rather simple and straight-forward optimization: http://people.apache.org/~shade/jigsaw/webrev-rearrange-1/ Please take a look. I'd be happy to see this patch landing in Jigsaw code (are there any other steps besides preparing webrev?). The change rearranges the hugest map from Map to Map when writing to config datafile. This helps because module ID is de-duplicated, so it both occupies less space, and module IDs are not resolved multiple times during load. The effect of this is clearly visible on "Hello World" applications: BASELINE: 570 +- 20 ms PATCHED: 240 +- 20 ms ...which converts to 2.3x improvement. After this patch, loading config file takes ~30ms. I'm looking if this could be trimmed down further. -Aleksey. From Alan.Bateman at oracle.com Thu Mar 15 05:36:59 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 15 Mar 2012 12:36:59 +0000 Subject: Instrumented profiling for multimodule Jigsaw startup In-Reply-To: References: <4F60EA76.5070305@oracle.com> <4F6112C1.2050701@oracle.com> Message-ID: <4F61E26B.4050209@oracle.com> On 14/03/2012 22:16, Aleksey Shipilev wrote: > : > The thing I missed from explanation there: if you churn those 500ms > local class map loading out of 600ms of total startup time, > modularized Jigsaw startup is on-par with "legacy" JDK8 startup for > HelloWorld tests. Do you want me to see what could be addressed in > configuration, or is it someone else's part already? > > -Aleksey. A while back (perhaps >1 year ago) then Mark had a prototype of a fast configuration that used a mini database. That work needs to be re-based and pushed to jigsaw/jigsaw. I see your other mail with a patch to improve the existing format/code, much appreciated and we can look at that for the short term but eventually this will be replaced. -Alan From aleksey.shipilev at oracle.com Thu Mar 15 05:52:52 2012 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Thu, 15 Mar 2012 16:52:52 +0400 Subject: Instrumented profiling for multimodule Jigsaw startup In-Reply-To: <4F61E26B.4050209@oracle.com> References: <4F60EA76.5070305@oracle.com> <4F6112C1.2050701@oracle.com> <4F61E26B.4050209@oracle.com> Message-ID: <4F61E624.7070508@oracle.com> On 03/15/2012 04:36 PM, Alan Bateman wrote: > On 14/03/2012 22:16, Aleksey Shipilev wrote: >> : >> The thing I missed from explanation there: if you churn those 500ms >> local class map loading out of 600ms of total startup time, >> modularized Jigsaw startup is on-par with "legacy" JDK8 startup for >> HelloWorld tests. Do you want me to see what could be addressed in >> configuration, or is it someone else's part already? >> >> -Aleksey. > A while back (perhaps >1 year ago) then Mark had a prototype of a fast > configuration that used a mini database. That work needs to be re-based > and pushed to jigsaw/jigsaw. I see your other mail with a patch to > improve the existing format/code, much appreciated and we can look at > that for the short term but eventually this will be replaced. Is this the part of that mini-database stuff? http://cr.openjdk.java.net/~mr/jigsaw/notes/module-file-format/ Are we sure mini database (whatever that means) would be faster than carefully-tuned in-memory Java collections and efficient marshallers/demarshallers, tied to hand-tuned on-disk form? I have a reason to believe it would not make any difference at this point. We are talking about ~30ms time spent in configuration loader after my "reverse map" patch, in absence of pre-compiled classes, running in interpreted mode, experiencing huge I/O, etc. -Aleksey. From Alan.Bateman at oracle.com Thu Mar 15 06:12:03 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 15 Mar 2012 13:12:03 +0000 Subject: Instrumented profiling for multimodule Jigsaw startup In-Reply-To: <4F61E624.7070508@oracle.com> References: <4F60EA76.5070305@oracle.com> <4F6112C1.2050701@oracle.com> <4F61E26B.4050209@oracle.com> <4F61E624.7070508@oracle.com> Message-ID: <4F61EAA3.3050000@oracle.com> On 15/03/2012 12:52, Aleksey Shipilev wrote: > : > Is this the part of that mini-database stuff? > http://cr.openjdk.java.net/~mr/jigsaw/notes/module-file-format/ No, that's the module file format for transporting/distributing modules. The installed format is different (and library implementation specific). > > Are we sure mini database (whatever that means) would be faster than > carefully-tuned in-memory Java collections and efficient > marshallers/demarshallers, tied to hand-tuned on-disk form? > > I have a reason to believe it would not make any difference at this > point. We are talking about ~30ms time spent in configuration loader > after my "reverse map" patch, in absence of pre-compiled classes, > running in interpreted mode, experiencing huge I/O, etc. > I'll leave Mark to comment on the results that he observed with his prototype. -Alan. From pascal at rapicault.net Thu Mar 15 09:00:53 2012 From: pascal at rapicault.net (Pascal Rapicault) Date: Thu, 15 Mar 2012 12:00:53 -0400 Subject: Dependency resolver Message-ID: <1618E78D-C43F-4A5A-8DB3-B15F15E958FF@rapicault.net> Hi, I'm new to Jigsaw (just joined the mailing list today) and wanted to take a look at the dependency resolver. Could someone indicate in which SCM I could find it and a few entry points Thx Pascal From mark.reinhold at oracle.com Thu Mar 15 09:13:16 2012 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Thu, 15 Mar 2012 09:13:16 -0700 Subject: Dependency resolver In-Reply-To: pascal@rapicault.net; Thu, 15 Mar 2012 12:00:53 EDT; <1618E78D-C43F-4A5A-8DB3-B15F15E958FF@rapicault.net> Message-ID: <20120315161316.CA0ADAB5@eggemoggin.niobe.net> 2012/3/15 9:00 -0700, pascal at rapicault.net: > I'm new to Jigsaw (just joined the mailing list today) and wanted to > take a look at the dependency resolver. Could someone indicate in > which SCM I could find it and a few entry points http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/file/04e27358d0dc/src/share/classes/org/openjdk/jigsaw/Resolver.java - Mark From david.bosschaert at gmail.com Thu Mar 15 14:48:34 2012 From: david.bosschaert at gmail.com (David Bosschaert) Date: Thu, 15 Mar 2012 21:48:34 +0000 Subject: Debugging Jigsaw test cases in an IDE In-Reply-To: <4F4EB7AD.8090502@oracle.com> References: <4F4EB7AD.8090502@oracle.com> Message-ID: Thanks for the help. I managed to get this to work. BTW I also managed to get debugging set up. Run jtreg once from the command line, then running the following manually from .../jdk/test/JTwork/scratch does the trick: .../jdk/build/linux-i586/bin/java -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=localhost:7777 -cp z.classes/ someTestClass The simply remote attach debugging from Netbeans works a charm. Cheers, David On 29 February 2012 23:41, Jesse Glick wrote: > On 02/29/2012 05:14 PM, David Bosschaert wrote: >> >> I looked at the Netbeans >> setup instructions in /jdk/make/netbeans/j2se and set up the IDE that >> way, but when I do 'Debug File' on the _ModuleId.java test above (or >> any other test) >> It fails > > > Not surprisingly, because jtreg is not apparently bound to anything in > project.xml, and the generic targets that might get set up if you accept the > IDE's offer to generate them have no chance of working for this specialized > test harness. > > I took a stab at writing jtreg NB bindings for this project and am including > the patch for you to try. Run File seems to work (opens a browser on the > results if there were failures). Note that Debug File does not work for me > because of a "transport error"; not sure if this is just a case of debugging > being broken in the current VM, or if something in shared.xml is still > wrong. > > Cannot create attachments so just pasting inline - hope you can apply it. > This really ought to be applied against OpenJDK 7 and merged into 8 and then > Jigsaw, of course. > > -------%<-------- > > # HG changeset patch > # Parent 115b3bb6de96bbd0ce2203a272b80ef4a5b36d97 > # User Jesse Glick > Fix up jtreg actions, and bind them to run.single and debug.single in > NetBeans. > > > diff --git a/make/netbeans/common/shared.xml > b/make/netbeans/common/shared.xml > --- a/make/netbeans/common/shared.xml > +++ b/make/netbeans/common/shared.xml > @@ -75,7 +75,7 @@ > ? ? ? ? file="../common/architectures/arch-${os.arch}.properties"/> > ? ? ? ? > ? ? ? ? System configuration claims architecture is > ${platform}-${arch} > - ? ? ? ? location="${root}/build/${platform}-${arch}"/> > + ? ? ? ? location="${root}/../build/${platform}-${arch}"/> > ? ? ? ? > ? ? ? ? > ? ? ? ? > @@ -277,7 +277,7 @@ > ? ? ? ? > ? ? > ? ? depends="-init,-pre-jtreg,-taskdef-jtreg,-check-tests-defined,-jtreg-setup,-jtreg-make,-jtreg-ant,-post-jtreg"> > - ? ? ? ? location="${jtreg.dir}/JTreport/report.html"/> > + ? ? ? ? location="${jtreg.dir}/JTreport/html/report.html"/> > ? ? ? ? > ? ? ? ? ? ? > ? ? ? ? > diff --git a/make/netbeans/j2se/nbproject/project.xml > b/make/netbeans/j2se/nbproject/project.xml > --- a/make/netbeans/j2se/nbproject/project.xml > +++ b/make/netbeans/j2se/nbproject/project.xml > @@ -79,6 +79,30 @@ > ? ? ? ? ? ? ? ? ? ? javadoc-nb > ? ? ? ? ? ? ? ? > ? ? ? ? ? ? ? ? --> > + ? ? ? ? ? ? ? ? > + ? ? ? ? ? ? ? ? ? ?jtreg-nb > + ? ? ? ? ? ? ? ? ? ? > + ? ? ? ? ? ? ? ? ? ? ? ?jtreg.tests > + ? ? ? ? ? ? ? ? ? ? ? ?${root}/test > + ? ? ? ? ? ? ? ? ? ? ? ?\.java$ > + ? ? ? ? ? ? ? ? ? ? ? ?relative-path > + ? ? ? ? ? ? ? ? ? ? ? ? > + ? ? ? ? ? ? ? ? ? ? ? ? ? ? > + ? ? ? ? ? ? ? ? ? ? ? ? > + ? ? ? ? ? ? ? ? ? ? > + ? ? ? ? ? ? ? ? > + ? ? ? ? ? ? ? ? > + ? ? ? ? ? ? ? ? ? ?jtreg-debug-nb > + ? ? ? ? ? ? ? ? ? ? > + ? ? ? ? ? ? ? ? ? ? ? ?jtreg.tests > + ? ? ? ? ? ? ? ? ? ? ? ?${root}/test > + ? ? ? ? ? ? ? ? ? ? ? ?\.java$ > + ? ? ? ? ? ? ? ? ? ? ? ?relative-path > + ? ? ? ? ? ? ? ? ? ? ? ? > + ? ? ? ? ? ? ? ? ? ? ? ? ? ? > + ? ? ? ? ? ? ? ? ? ? ? ? > + ? ? ? ? ? ? ? ? ? ? > + ? ? ? ? ? ? ? ? > ? ? ? ? ? ? > ? ? ? ? ? ? > ? ? ? ? ? ? ? ? From forax at univ-mlv.fr Sat Mar 17 07:40:18 2012 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Sat, 17 Mar 2012 15:40:18 +0100 Subject: Annotation and module Message-ID: <4F64A252.4090801@univ-mlv.fr> The current spec disallows to put annotations on a module, while I understand why nodody want to have an artifact descriptor defined using annotations, I think this is too restrictive. Eclipse, by example, has an annotation named @NonNullByDefault that you can put on a package to improve the null-analysis. I think this kind of annotations should be allowed on a module. Perhaps, only annotations visible at runtime should be disallowed. cheers, R?mi From sanjeeb.sahoo at oracle.com Sun Mar 18 20:45:34 2012 From: sanjeeb.sahoo at oracle.com (Sahoo) Date: Mon, 19 Mar 2012 09:15:34 +0530 Subject: Lanching off modulepath directory? Message-ID: <4F66ABDE.6000703@oracle.com> As per [1], one should be able to launch VM pointing to a modulepath populated by javac: In order to support interactive development, the Java launcher can run a modular application directly from a module-path directory. When doing so it performs resolution before invoking the application?s entry point, although the resulting configuration is not stored for future use. But, I don't see any option in java launcher to specify modulepath. This is how my directory structure looks like: ./src ./src/m1 ./src/m1/p1 ./src/m1/p1/Foo.java ./src/m1/p2 ./src/m1/p2/Bar.java ./src/m1/module-info.java ./src/m2 ./src/m2/module-info.java ./modules ./modules/m1 ./modules/m1/p1 ./modules/m1/p1/Foo.class ./modules/m1/p2 ./modules/m1/p2/Bar.class ./modules/m1/module-info.class ./modules/m2 ./modules/m2/module-info.class `java -L modules -m m1` fails with following exception: Caused by: java.io.FileNotFoundException: modules/%jigsaw-library (No such file or directory) Is this a feature yet to be implemented in java launcher? Thanks, Sahoo [1] http://cr.openjdk.java.net/~mr/jigsaw/notes/jigsaw-big-picture-01 From sanjeeb.sahoo at oracle.com Sun Mar 18 21:05:04 2012 From: sanjeeb.sahoo at oracle.com (Sahoo) Date: Mon, 19 Mar 2012 09:35:04 +0530 Subject: modulepath layout and version number in output directories Message-ID: <4F66B070.8000104@oracle.com> According to [1], for an input source structure like: src/m1/module-info.jar #let's say m1's version is 1.0 in module-info.java src/m1/... src/m2/module-info.java #let's m2's version is 2.0 in module-info.java src/m2/... javac -d modules -modulepath modules `find src -name "*.java"` produces following output directory structure: modules/m1/module-info.class modules/m1/... modules/m2/module-info.class modules/m2/... Since every module has always got a version (if not explictly specified, there is always a default version), is there not a benefit in encoding the version number in output directory structure? e.g., how about the output directory structure looking like: modules/m1/1.0/module-info.class modules/m1/1.0/... modules/m2/2.0/module-info.class modules/m2/2.0/... The example in [1] that uses multiple module versions recommends encoding version number in src directory structure (e.g., src/m1-1.0, src/m2-2.0, etc), but I am not so sure how practical it is. How often do we find version number encoded in svn or hg or cvs directory name? Thanks, Sahoo [1] http://openjdk.java.net/projects/jigsaw/doc/ModulesAndJavac.pdf From Alan.Bateman at oracle.com Mon Mar 19 04:30:35 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 19 Mar 2012 11:30:35 +0000 Subject: Lanching off modulepath directory? In-Reply-To: <4F66ABDE.6000703@oracle.com> References: <4F66ABDE.6000703@oracle.com> Message-ID: <4F6718DB.5000307@oracle.com> On 19/03/2012 03:45, Sahoo wrote: > As per [1], one should be able to launch VM pointing to a modulepath > populated by javac: > > In order to support interactive development, the Java launcher can run > a modular application directly from a module-path directory. When > doing so it performs resolution before invoking the application?s > entry point, although the resulting configuration is not stored for > future use. > > But, I don't see any option in java launcher to specify modulepath. > This is how my directory structure looks like: > > ./src > ./src/m1 > ./src/m1/p1 > ./src/m1/p1/Foo.java > ./src/m1/p2 > ./src/m1/p2/Bar.java > ./src/m1/module-info.java > ./src/m2 > ./src/m2/module-info.java > ./modules > ./modules/m1 > ./modules/m1/p1 > ./modules/m1/p1/Foo.class > ./modules/m1/p2 > ./modules/m1/p2/Bar.class > ./modules/m1/module-info.class > ./modules/m2 > ./modules/m2/module-info.class > > `java -L modules -m m1` fails with following exception: > Caused by: java.io.FileNotFoundException: modules/%jigsaw-library (No > such file or directory) > > Is this a feature yet to be implemented in java launcher? It's not in jigsaw/jigsaw yet but when it is then you should be able to do "java -mp modules -m m1". Note the option is -modulepath or -mp, not -L as that is used to specify a module library. The jmod command is used to create and manage a module library where you may have multiple versions of m1 and m2 installed (which may help with your other question). -Alan From sanjeeb.sahoo at oracle.com Mon Mar 19 07:58:36 2012 From: sanjeeb.sahoo at oracle.com (Sahoo) Date: Mon, 19 Mar 2012 20:28:36 +0530 Subject: Lanching off modulepath directory? In-Reply-To: <4F6718DB.5000307@oracle.com> References: <4F66ABDE.6000703@oracle.com> <4F6718DB.5000307@oracle.com> Message-ID: <4F67499C.7090404@oracle.com> On Monday 19 March 2012 05:00 PM, Alan Bateman wrote: > It's not in jigsaw/jigsaw yet but when it is then you should be able > to do "java -mp modules -m m1". Note the option is -modulepath or -mp, > not -L as that is used to specify a module library. The jmod command > is used to create and manage a module library where you may have > multiple versions of m1 and m2 installed (which may help with your > other question). Thanks, Alan. Initially I tried java -modulepath, but it failed, so I tried with -L. Now I understand why I didn't work. Yes, I am aware of jmod command and used it in my other samples. In this particular case, I was exploring the iterative development use case. I feel I have raised a different issue in my other email and it will be responded separately. Sahoo From sanjeeb.sahoo at oracle.com Mon Mar 19 21:19:40 2012 From: sanjeeb.sahoo at oracle.com (Sahoo) Date: Tue, 20 Mar 2012 09:49:40 +0530 Subject: modulepath with multiple modules and inability to install more than one module using jmod Message-ID: <4F68055C.7000508@oracle.com> I have the following source layout: ./src/m1-1.0/m1/p1/Foo.java ./src/m1-1.0/m1/p2/Bar.java ./src/m1-1.0/module-info.java ./src/m2-1.0/module-info.java I have followed the recommendation of encoding version number in source directory name as mentioned in the modulepath/javac pdf, although I am convinced about it and hence have a separate email about it. This is how I invoke javac: javac -d modules/ -modulepath src/:modules/ `find src/ -name "*.java"` I am slightly confused as to whether I should use -modulepath src/ or -modulepath modules/ or -modulepath src/:modules/. I guess what I have used covers the case where module/ already contains some compiled modules that are required by the current compilation. Someone can help me understand this better. This generates classes like this: ./modules/m1-1.0/m1/p1/Foo.class ./modules/m1-1.0/m1/p2/Bar.class ./modules/m1-1.0/module-info.class ./modules/m2-1.0/module-info.class Because the classes are generated under modules/m1-1.0 and modules/m2-1.0 instead of modules/, I can't add more than one module at a time. e.g., I can't do: jmod -L mlib install modules/ m1 m2 This fails with: I/O error: modules/m1/module-info.class (No such file or directory) I have to do: jmod -L mlib install modules/m1-1.0 m1 jmod -L mlib install modules/m2-1.0 m2 It's a bit inconvenient as you can see. Thanks, Sahoo From mandy.chung at oracle.com Tue Mar 20 21:06:19 2012 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 20 Mar 2012 21:06:19 -0700 Subject: ShowDeps / ClassAnalyzer tool update Message-ID: <4F6953BB.2030509@oracle.com> Alan, Chris, I have updated the ShowDeps tool to show the dependencies of one or more given classes, directories or jar files. It's a simple tool that can help the process of modularizing existing applications. A brief description of this tool is at: http://cr.openjdk.java.net/~mchung/jigsaw/showdeps-webrev/raw_files/new/make/tools/classanalyzer/classanalyzer.html By default it will print the package dependencies; -v option will list the class dependencies. If a module ID is specified in the -id option, it will print the module declaration (i.e. module-info). Webrev at: http://cr.openjdk.java.net/~mchung/jigsaw/showdeps-webrev/ Mandy From Alan.Bateman at oracle.com Tue Mar 20 21:48:35 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 21 Mar 2012 04:48:35 +0000 Subject: ShowDeps / ClassAnalyzer tool update In-Reply-To: <4F6953BB.2030509@oracle.com> References: <4F6953BB.2030509@oracle.com> Message-ID: <4F695DA3.3060306@oracle.com> On 21/03/2012 04:06, Mandy Chung wrote: > Alan, Chris, > > I have updated the ShowDeps tool to show the dependencies of one or > more given classes, directories or jar files. It's a simple tool that > can help the process of modularizing existing applications. A brief > description of this tool is at: > > http://cr.openjdk.java.net/~mchung/jigsaw/showdeps-webrev/raw_files/new/make/tools/classanalyzer/classanalyzer.html > > By default it will print the package dependencies; -v option will list > the class dependencies. If a module ID is specified in the -id > option, it will print the module declaration (i.e. module-info). > > Webrev at: > http://cr.openjdk.java.net/~mchung/jigsaw/showdeps-webrev/ > > Mandy This is a nice improvement to the tool, the changes looks fine to me. -Alan From mandy.chung at oracle.com Tue Mar 20 22:10:05 2012 From: mandy.chung at oracle.com (mandy.chung at oracle.com) Date: Wed, 21 Mar 2012 05:10:05 +0000 Subject: hg: jigsaw/jigsaw/jdk: ShowDeps and ClassAnalyzer tool update to work with module library/views Message-ID: <20120321051022.C71A847A70@hg.openjdk.java.net> Changeset: a8c80b17128e Author: mchung Date: 2012-03-20 22:07 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/a8c80b17128e ShowDeps and ClassAnalyzer tool update to work with module library/views Reviewed-by: alanb ! make/tools/classanalyzer/classanalyzer.html ! make/tools/classanalyzer/src/com/sun/classanalyzer/JigsawModuleBuilder.java ! make/tools/classanalyzer/src/com/sun/classanalyzer/Module.java ! make/tools/classanalyzer/src/com/sun/classanalyzer/ModuleConfig.java ! make/tools/classanalyzer/src/com/sun/classanalyzer/ShowDeps.java From aleksey.shipilev at oracle.com Wed Mar 21 02:14:06 2012 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 21 Mar 2012 13:14:06 +0400 Subject: [performance] Re-arranging maps in config metadata improves startup time In-Reply-To: <4F61DA84.6000804@oracle.com> References: <4F61DA84.6000804@oracle.com> Message-ID: <4F699BDE.4060506@oracle.com> On 03/15/2012 04:03 PM, Aleksey Shipilev wrote: > This is rather simple and straight-forward optimization: > http://people.apache.org/~shade/jigsaw/webrev-rearrange-1/ > > Please take a look. I'd be happy to see this patch landing in Jigsaw > code (are there any other steps besides preparing webrev?). Um. Anyone? -Aleksey. From jesse.glick at oracle.com Wed Mar 21 07:37:57 2012 From: jesse.glick at oracle.com (Jesse Glick) Date: Wed, 21 Mar 2012 10:37:57 -0400 Subject: ShowDeps / ClassAnalyzer tool update In-Reply-To: <4F6953BB.2030509@oracle.com> References: <4F6953BB.2030509@oracle.com> Message-ID: <4F69E7C5.9070605@oracle.com> On 03/21/2012 12:06 AM, Mandy Chung wrote: > http://cr.openjdk.java.net/~mchung/jigsaw/showdeps-webrev/raw_files/new/make/tools/classanalyzer/classanalyzer.html "where is a class" shows as "where is a class" in the HTML. Need <. From Alan.Bateman at oracle.com Wed Mar 21 07:47:27 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 21 Mar 2012 14:47:27 +0000 Subject: [performance] Re-arranging maps in config metadata improves startup time In-Reply-To: <4F699BDE.4060506@oracle.com> References: <4F61DA84.6000804@oracle.com> <4F699BDE.4060506@oracle.com> Message-ID: <4F69E9FF.6070504@oracle.com> On 21/03/2012 09:14, Aleksey Shipilev wrote: > On 03/15/2012 04:03 PM, Aleksey Shipilev wrote: >> This is rather simple and straight-forward optimization: >> http://people.apache.org/~shade/jigsaw/webrev-rearrange-1/ >> >> Please take a look. I'd be happy to see this patch landing in Jigsaw >> code (are there any other steps besides preparing webrev?). > Um. Anyone? > > -Aleksey. We can certainly include this patch to help for the short term, but ultimately this code needs to be replaced with a b-tree implementation. -Alan From mark.reinhold at oracle.com Wed Mar 21 08:13:32 2012 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Wed, 21 Mar 2012 08:13:32 -0700 Subject: [performance] Re-arranging maps in config metadata improves startup time In-Reply-To: alan.bateman@oracle.com; Wed, 21 Mar 2012 14:47:27 -0000; <4F69E9FF.6070504@oracle.com> Message-ID: <20120321151332.A6CDD866@eggemoggin.niobe.net> 2012/3/21 7:47 -0700, alan.bateman at oracle.com: > On 21/03/2012 09:14, Aleksey Shipilev wrote: >> On 03/15/2012 04:03 PM, Aleksey Shipilev wrote: >>> This is rather simple and straight-forward optimization: >>> http://people.apache.org/~shade/jigsaw/webrev-rearrange-1/ >>> >>> Please take a look. I'd be happy to see this patch landing in Jigsaw >>> code (are there any other steps besides preparing webrev?). > > We can certainly include this patch to help for the short term, but ultimately > this code needs to be replaced with a b-tree implementation. Sorry, but I'd rather not see this go in. All of this code will soon be replaced with the b-tree implementation. Changing it now just makes it more tedious to rebase the evolving b-tree patch. - Mark From aleksey.shipilev at oracle.com Wed Mar 21 09:24:31 2012 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 21 Mar 2012 20:24:31 +0400 Subject: [performance] Re-arranging maps in config metadata improves startup time In-Reply-To: <20120321151332.A6CDD866@eggemoggin.niobe.net> References: <20120321151332.A6CDD866@eggemoggin.niobe.net> Message-ID: <4F6A00BF.7050808@oracle.com> On 03/21/2012 07:13 PM, mark.reinhold at oracle.com wrote: > 2012/3/21 7:47 -0700, alan.bateman at oracle.com: >> On 21/03/2012 09:14, Aleksey Shipilev wrote: >>> On 03/15/2012 04:03 PM, Aleksey Shipilev wrote: >>>> This is rather simple and straight-forward optimization: >>>> http://people.apache.org/~shade/jigsaw/webrev-rearrange-1/ >>>> >>>> Please take a look. I'd be happy to see this patch landing in Jigsaw >>>> code (are there any other steps besides preparing webrev?). >> >> We can certainly include this patch to help for the short term, but ultimately >> this code needs to be replaced with a b-tree implementation. > > Sorry, but I'd rather not see this go in. All of this code will soon be > replaced with the b-tree implementation. Changing it now just makes it > more tedious to rebase the evolving b-tree patch. It's definitely your call. However I'd like to point out that "evolving b-tree patch" is still in limbo. My patch on the other hand, is ready, is simple, is local, and is helping the problem now. The better is the enemy of the good. -Aleksey. From mandy.chung at oracle.com Wed Mar 21 15:36:16 2012 From: mandy.chung at oracle.com (mandy.chung at oracle.com) Date: Wed, 21 Mar 2012 22:36:16 +0000 Subject: hg: jigsaw/jigsaw/jdk: Fix '<' and '>' use in classanalyzer.html and fix the unknown module name Message-ID: <20120321223633.ED86247AAC@hg.openjdk.java.net> Changeset: 543b0d24a920 Author: mchung Date: 2012-03-21 15:36 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/543b0d24a920 Fix '<' and '>' use in classanalyzer.html and fix the unknown module name ! make/tools/classanalyzer/classanalyzer.html ! make/tools/classanalyzer/src/com/sun/classanalyzer/ModuleConfig.java From mandy.chung at oracle.com Wed Mar 21 16:11:32 2012 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 21 Mar 2012 16:11:32 -0700 Subject: ShowDeps / ClassAnalyzer tool update In-Reply-To: <4F69E7C5.9070605@oracle.com> References: <4F6953BB.2030509@oracle.com> <4F69E7C5.9070605@oracle.com> Message-ID: <4F6A6024.7070401@oracle.com> On 3/21/2012 7:37 AM, Jesse Glick wrote: > On 03/21/2012 12:06 AM, Mandy Chung wrote: >> http://cr.openjdk.java.net/~mchung/jigsaw/showdeps-webrev/raw_files/new/make/tools/classanalyzer/classanalyzer.html >> > > "where is a class" shows as "where is a class" in the HTML. > Need <. Fixed. Mandy From sanjeeb.sahoo at oracle.com Wed Mar 21 22:33:46 2012 From: sanjeeb.sahoo at oracle.com (Sahoo) Date: Thu, 22 Mar 2012 11:03:46 +0530 Subject: modulepath layout and version number in output directories In-Reply-To: <4F66B070.8000104@oracle.com> References: <4F66B070.8000104@oracle.com> Message-ID: <4F6AB9BA.2050307@oracle.com> Hope to see some response from relevant developers... On Monday 19 March 2012 09:35 AM, Sahoo wrote: > According to [1], for an input source structure like: > src/m1/module-info.jar #let's say m1's version is 1.0 in module-info.java > src/m1/... > src/m2/module-info.java #let's m2's version is 2.0 in module-info.java > src/m2/... > > javac -d modules -modulepath modules `find src -name "*.java"` > produces following output directory structure: > > modules/m1/module-info.class > modules/m1/... > modules/m2/module-info.class > modules/m2/... > > Since every module has always got a version (if not explictly > specified, there is always a default version), is there not a benefit > in encoding the version number in output directory structure? e.g., > how about the output directory structure looking like: > modules/m1/1.0/module-info.class > modules/m1/1.0/... > modules/m2/2.0/module-info.class > modules/m2/2.0/... > > The example in [1] that uses multiple module versions recommends > encoding version number in src directory structure (e.g., src/m1-1.0, > src/m2-2.0, etc), but I am not so sure how practical it is. How often > do we find version number encoded in svn or hg or cvs directory name? > > Thanks, > Sahoo > > [1] http://openjdk.java.net/projects/jigsaw/doc/ModulesAndJavac.pdf From sanjeeb.sahoo at oracle.com Wed Mar 21 22:34:40 2012 From: sanjeeb.sahoo at oracle.com (Sahoo) Date: Thu, 22 Mar 2012 11:04:40 +0530 Subject: modulepath with multiple modules and inability to install more than one module using jmod In-Reply-To: <4F68055C.7000508@oracle.com> References: <4F68055C.7000508@oracle.com> Message-ID: <4F6AB9F0.1090004@oracle.com> Will I assume it to be a limitation then? On Tuesday 20 March 2012 09:49 AM, Sahoo wrote: > I have the following source layout: > ./src/m1-1.0/m1/p1/Foo.java > ./src/m1-1.0/m1/p2/Bar.java > ./src/m1-1.0/module-info.java > ./src/m2-1.0/module-info.java > > I have followed the recommendation of encoding version number in > source directory name as mentioned in the modulepath/javac pdf, > although I am convinced about it and hence have a separate email about > it. > > This is how I invoke javac: > > javac -d modules/ -modulepath src/:modules/ `find src/ -name "*.java"` > > I am slightly confused as to whether I should use -modulepath src/ or > -modulepath modules/ or -modulepath src/:modules/. I guess what I have > used covers the case where module/ already contains some compiled > modules that are required by the current compilation. Someone can help me > understand this better. > > This generates classes like this: > ./modules/m1-1.0/m1/p1/Foo.class > ./modules/m1-1.0/m1/p2/Bar.class > ./modules/m1-1.0/module-info.class > ./modules/m2-1.0/module-info.class > > Because the classes are generated under modules/m1-1.0 and > modules/m2-1.0 instead of modules/, I can't add more than one module at > a time. e.g., I can't do: > > jmod -L mlib install modules/ m1 m2 > > This fails with: > I/O error: modules/m1/module-info.class (No such file or directory) > > I have to do: > jmod -L mlib install modules/m1-1.0 m1 > jmod -L mlib install modules/m2-1.0 m2 > > It's a bit inconvenient as you can see. > > Thanks, > Sahoo From jonathan.gibbons at oracle.com Thu Mar 22 15:31:44 2012 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 22 Mar 2012 15:31:44 -0700 Subject: modulepath layout and version number in output directories In-Reply-To: <4F6AB9BA.2050307@oracle.com> References: <4F66B070.8000104@oracle.com> <4F6AB9BA.2050307@oracle.com> Message-ID: <4F6BA850.5050607@oracle.com> Sahoo, There is a very strong desire to have the structure of the javac outpuit directory mirror the structure of the source directory. This is to support build tools like Ant that rely on a relatively simple time stamp comparison between source and derived file. So, while it may be attractive to have the version number in the output directory structure, it is not desirable to do so in the source directory structure, where revving the version would mean renaming source files, causing unnecessary headaches for source code management systems. -- Jon On 03/21/2012 10:33 PM, Sahoo wrote: > Hope to see some response from relevant developers... > > On Monday 19 March 2012 09:35 AM, Sahoo wrote: >> According to [1], for an input source structure like: >> src/m1/module-info.jar #let's say m1's version is 1.0 in >> module-info.java >> src/m1/... >> src/m2/module-info.java #let's m2's version is 2.0 in module-info.java >> src/m2/... >> >> javac -d modules -modulepath modules `find src -name "*.java"` >> produces following output directory structure: >> >> modules/m1/module-info.class >> modules/m1/... >> modules/m2/module-info.class >> modules/m2/... >> >> Since every module has always got a version (if not explictly >> specified, there is always a default version), is there not a benefit >> in encoding the version number in output directory structure? e.g., >> how about the output directory structure looking like: >> modules/m1/1.0/module-info.class >> modules/m1/1.0/... >> modules/m2/2.0/module-info.class >> modules/m2/2.0/... >> >> The example in [1] that uses multiple module versions recommends >> encoding version number in src directory structure (e.g., src/m1-1.0, >> src/m2-2.0, etc), but I am not so sure how practical it is. How often >> do we find version number encoded in svn or hg or cvs directory name? >> >> Thanks, >> Sahoo >> >> [1] http://openjdk.java.net/projects/jigsaw/doc/ModulesAndJavac.pdf > From david.bosschaert at gmail.com Fri Mar 23 04:42:09 2012 From: david.bosschaert at gmail.com (David Bosschaert) Date: Fri, 23 Mar 2012 11:42:09 +0000 Subject: What is a language construct? (was Re: Comment on state of Project Jigsaw) In-Reply-To: <4F05CEE0.7060906@tibco.com> References: <20111221223027.33AD123A6@eggemoggin.niobe.net> <4EF26029.1050700@univ-mlv.fr> <4EF2679F.9050408@oracle.com> <4EF27623.2050501@oracle.com> <4EF2889C.5070707@oracle.com> <4EF3D3D0.50704@oracle.com> <4F046650.3030103@univ-mlv.fr> <4F047807.7090507@univ-mlv.fr> <4F04BB6E.4040309@tibco.com> <4F05B2DF.7090907@univ-mlv.fr> <4F05CEE0.7060906@tibco.com> Message-ID: On 5 January 2012 16:25, Eric Johnson wrote: > Hi R?mi, > > > On 1/5/12 3:25 PM, R?mi Forax wrote: >> >> On 01/04/2012 09:49 PM, Eric Johnson wrote: >>> >>> Comment below... >>> >>> On 1/4/12 5:02 PM, R?mi Forax wrote: >>>> >>>> On 01/04/2012 04:23 PM, David Bosschaert wrote: >>>>> >>>>> On 4 January 2012 14:46, R?mi Forax ?wrote: >>>>>> >>>>>> The module descriptor is not the .java but the .class, >>>>>> the .java is a convenient way to specify the module descriptor. >>>>> >>>>> I have nothing against a convenient way of specifying a module >>>>> description, I actually see that as a requirement :) but that >>>>> requirement doesn't imply that it needs to be shoe-horned into .java >>>>> and .class files! >>>> >>>> >>>> The .class format is already an extensible versioned container format >>>> with a dictionnary which is able to store string and class name. >>>> No shoehorn here. >>> >>> >>> OK, perhaps the format is reusable, but how will existing tools fare with >>> it? >> >> >>> Will existing IDEs choke on the file unless modified because it has >>> unexpected content? Will code obfuscation tools obfuscate it by default? >>> Will Ant need to be updated to treat these files specially? Will any ASM >>> client ever written fall over because the module definition file doesn't >>> actually include byte codes? What about tools like "bnd" that introspect >>> ".class" files looking for referenced symbols, so that they can generate >>> OSGi information? >> >> >> The classfile format is versioned, each time a new version is created, >> like 1.5 and 1.7 did, >> you have to update your tools. most of the tools, first check the version >> and just exit >> if the version number is unknown. > > > I understood that. I think you're overlooking my point about how many > additional changes tools might need to make. Think of the universe of > existing tools that crawl Java .class files for whatever reason. How many of > those existing tools need to care about module information (answer 0%)? How > many of those tools will need to be updated - beyond accepting the newer > version - if the module information is loaded into an existing .class file > format, with a .class extension. > > If I were drawing a picture, I'd have a circle with three pie slices: > > ?* tools that need to change merely to accept the new version of the > ? class file format > ?* tools that don't care about the module information but will need to > ? change to work around/ignore said data > ?* tools that actually care about the new module information. > > Question is, how big are you making that middle slice? If you put the module > information into an alternate format, I *know* that the second slice is an > empty set. > >> >>> >>> I think the notion of reusing a format might work, but I suspect reusing >>> the extension ".class" will cause a lot of churn in downstream tooling, and >>> it doesn't sound like that's been considered here. (And I worry that I've >>> written code that would succumb to that churn.) >> >> >> >> >>> >>> If you're arguing that it is a useful container format, though, there are >>> plenty of other "container formats" - JSON and XML jump to mind - that can >>> equally well satisfy the requirements that have been stated so far. >> >> >> If you use a text format, you introduce several stupid problems like the >> charset encoding >> and the fact that a user can update the module descriptor without >> re-validating it. > > > XML deals quite nicely with character encodings. > > As for updating, it is merely that the tooling is simpler. Re-validating > might be required in either case. > > >> >>> Since modules must be "installed" (if I've understood what I've read >>> correctly), then any notions of pre-compiling them are likely not essential, >>> because the installation process can do what it wishes, and the files >>> themselves have no "external" dependencies that need to be controlled in a >>> compilation process (unlike, say, the existing Java classes that we >>> compile.) >> >> >> I don't agree. >> You still need to validate the module descriptor before distributing it so >> a long time before installing it. >> And the module descriptor contains references to packages and types that >> need to be checked. > > > My point was that any "compilation" process you can do on a module > descriptor can be done on the file itself without reference to those > external entities. /Validation/ is a separate problem. > > For an analogy, a Python file can be compiled to a ".pyc" file without any > knowledge of external dependencies. However, it cannot be run unless those > external dependencies are satisfied. > > -Eric. > Just FYI I started modifying OpenJDK/Jigsaw in the context of the Penrose project to see if something like module-info.xml instead of module-info.class/.java would work. From the prototype that I made it seems to make a lot of sense. See here: http://mail.openjdk.java.net/pipermail/penrose-dev/2012-March/000023.html Best regards, David From eric at tibco.com Fri Mar 23 05:07:16 2012 From: eric at tibco.com (Eric Johnson) Date: Fri, 23 Mar 2012 13:07:16 +0100 Subject: XML for module descriptor (was Re: What is a language construct? (was Re: Comment on state of Project Jigsaw)) In-Reply-To: References: <20111221223027.33AD123A6@eggemoggin.niobe.net> <4EF26029.1050700@univ-mlv.fr> <4EF2679F.9050408@oracle.com> <4EF27623.2050501@oracle.com> <4EF2889C.5070707@oracle.com> <4EF3D3D0.50704@oracle.com> <4F046650.3030103@univ-mlv.fr> <4F047807.7090507@univ-mlv.fr> <4F04BB6E.4040309@tibco.com> <4F05B2DF.7090907@univ-mlv.fr> <4F05CEE0.7060906@tibco.com> Message-ID: <4F6C6774.2050609@tibco.com> Hi David, On 3/23/12 12:42 PM, David Bosschaert wrote: > Just FYI I started modifying OpenJDK/Jigsaw in the context of the > Penrose project to see if something like module-info.xml instead of > module-info.class/.java would work. From the prototype that I made it > seems to make a lot of sense. See here: > http://mail.openjdk.java.net/pipermail/penrose-dev/2012-March/000023.html > > Best regards, > > David Thanks for exploring this question. I'm excited to see that someone has taken up this work. For the trivial case you note, the XML, of course, looks mildly more complicated, if only because the namespace declarations add to the verbosity (and I note that you left off the prologue). However, I think that is vastly outweighed by the significant increase in the number of tools that can immediately process the data - for example, I can write a script that extracts the module file from every module in a set, and then walks through the XML to generate an HTML report of the same. All without having to know a thing about Java class file formats. For another example, I can edit the XML in Eclipse today, and apply the schema so that I know I am doing it correctly. And I can do that without having to wait for the Eclipse Java toolkit to improve to handle the new module-info.java syntax. Of course, as noted in previous discussion, this won't encompass the runtime validity of the data (can the required modules be resolved?), but it will ease the development of the contents of said files. As I know developers who still insist on using Emacs and its derivatives, I think this is enormously useful. For yet another scenario, when a Linux distribution needs to patch upstream module metadata for whatever reason (for example, changing a choice of security module dependency), it will be far easier to alter an XML file (treated as text, most likely), than to patch a module-info.java file, then recompiling said file. And, as you note, extending the data you can put into the module file is trivial, in the sense that you can simply add new namespace declarations. In other words, I see many reasons for a hearty +1 to your prototype! -Eric. From jaroslav.tulach at oracle.com Sun Mar 25 19:50:30 2012 From: jaroslav.tulach at oracle.com (Jaroslav Tulach) Date: Sun, 25 Mar 2012 22:50:30 -0400 Subject: (Complete) range dependencies back in the game? In-Reply-To: <4F1032DF.20402@oracle.com> References: <20111223202734.4467CF7F@eggemoggin.niobe.net> <201201081744.57845.jaroslav.tulach@oracle.com> <4F1032DF.20402@oracle.com> Message-ID: <201203252250.30516.jaroslav.tulach@oracle.com> Hello guys, I am not real fan of separating SPI and API. Either the designer knows what is the goal or not. Either the class should belong to API or SPI. Such distinction is possible even now in Java. Whether it needs some special constructs in the future depends on our understanding how frequent such usage case is. We may expect it is frequent because almost nobody understands what SPI vs. API is. Rather than changing the language, rather than changing the modularity, it would be better to restrict and define what makes sense. >## 13.?1.?2012 08:34:23 ##< > On 01/08/2012 11:44 AM, Jaroslav Tulach wrote: > > http://wiki.apidesign.org/wiki/RangeDependenciesAnalysed > > "It does not make much sense to have higher upper bound of dependency on C > higher than any existing upper bound on C among reused libraries (thus s > <= q)" - I disagree that this does not make sense. The basic problem is not about disagreeing, however about providing flexibility while keeping the system powerful enough. > Let us say that C has both an API and an SPI; the API is kept quite stable > but the SPI is frequently rewritten (and the handful of implementations > are expected to keep up). So pretend C is DOM, and API clients normally > use ranges like [1.2,2) ("DOM 2 or any foreseeable compatible release") > whereas SPI implementations normally use ranges like [1.2,1.3) ("DOM 2 but > not 3"). Jesse, I know you are obsessed by different stability levels of API and SPIs. True, you are not the only one. I am just saying we can do much better. Those who assumed certain classes will never be implemented by NetBeans IDE were wrong! We have implemented almost everything! If it was just an attempt to intercept the calls - if there is an interface, people will implement it. So just stop your "friendly implementor vision". It hurts everyone. Try to behave like real "distributed fan" - allow people to evolve on their own, just keep backward compatibility. -jt > Now pretend B is a library for XML persistence of Java objects, and during > one serialization mode it creates flyweight org.w3c.dom.Document's. > Versions 1.0 through 1.3, written back when DOM 2 ruled the land, declares > [1.2,1.3); version 1.4 and later implement DOM 3 so declare [1.3,1.4) but > are otherwise fully compatible for callers. > > Finally say A is an application which uses XML serialization for some > preferences storage, and also uses DOM quite independently for dealing > with REST clients. So it would like to declare B@[1.0,2) and C@[1.2,2): it > uses only the basic initial serialization API, and can work with any known > DOM release. > > The fact that B at 1.0 is incompatible with C at 1.3 does not trouble the author > of A; the app would work fine with B at 1.5 and C at 1.3 as well. If the > compiler silently converted the dep on C@[1.2,2) to C@[1.2,1.3), it would > effectively mean that B at 1.4 could not be used even if it had numerous bug > fixes, which is generally not what you want. It could instead just reject > A's attempted expression of what its actual dependency ranges are, with > some sort of error message TBD, and the author of A would (after some > trial and error) eventually give up on supporting older libraries and > request B@[1.4,2) and C[1.3,2). > > This switch would be rather artificial: the originally attempted dependency > declaration was logically sound, and a module system using a SAT solver > would not have difficulty linking A with either newer or older libraries > according to what is available. (An organization which decided it hated > the DOM 3 license might forbid it from being installed, in which case you > would like to be able to run A with B at 1.3 and C at 1.2.) > > Indeed if A does not use XML namespaces in its REST API, it could as well > declare C@[1.1,2) even knowing that current versions of B require at least > C at 1.2; a future version of B might stop needing to implement DOM and (if > it did not use namespaces either) might declare C@[1.1,2), in which case > B at 1.7 C at 1.1 would be a valid configuration too. Obviously this is a less > likely scenario, but it shows that "[t]his definition seems to make sense > from the point of lower bound" is not so obvious either. > > > On this topic, I think the definition of complete repository is > unnecessarily strict; there is no reason apparent to me why [r,s) must be > a subrange of [p,q), though it does at least need to overlap (and probably > a version of C in that intersection must actually be present in the repo). > Where in the proof is the assumption of subranges used? You only check > whether cr is empty, which it would be in the attempted conversion to > 3-SAT. It seems that A could express any dependency on C it likes, and > trying to compile A would either work or not work, according to whether > its dep on C has at least some overlap with that of the earliest versions > of its other dependencies. > > > By the way, you do not really discuss selecting a configuration at install > time in a complete repository. [1] does not seem to apply to arbitrary > range dependencies. Probably you can start with a configuration using the > lower bounds of all dependencies as determined during compilation, then > iteratively try raising the version of each module individually so long as > doing so continues to result in a valid configuration - i.e. pick a > locally maximal configuration. This is polynomial. > > > [1] > http://wiki.apidesign.org/wiki/LibraryWithoutImplicitExportIsPolynomial#Me > rge_of_Configurations From jaroslav.tulach at oracle.com Mon Mar 26 13:51:22 2012 From: jaroslav.tulach at oracle.com (Jaroslav Tulach) Date: Mon, 26 Mar 2012 16:51:22 -0400 Subject: XML for module descriptor In-Reply-To: <4F6C6774.2050609@tibco.com> References: <20111221223027.33AD123A6@eggemoggin.niobe.net> <4F6C6774.2050609@tibco.com> Message-ID: <201203261651.22636.jaroslav.tulach@oracle.com> >## 23.?3.?2012 08:07:16 ##< > Hi David, > > On 3/23/12 12:42 PM, David Bosschaert wrote: > > Just FYI I started modifying OpenJDK/Jigsaw in the context of the > > Penrose project to see if something like module-info.xml instead of > > module-info.class/.java would work. From the prototype that I made it > > seems to make a lot of sense. See here: > > http://mail.openjdk.java.net/pipermail/penrose-dev/2012-March/000023.html I cannot speak for the Jigsaw team, but (while my organization also needs concurrently extensible module-info[1]) there may be a reason why not use XML instead of module-info.class[2]. It is my understanding that one goal of Jigsaw is to make core Java really small and allow it to execute on very limited devices. The core needs to understand module-info. Right now the core contains ~3000 classes. If you bring full XML parsing infrastructure in, then you increase the size by ~4000 new classes (that was my observation when I worked on splitting JDK last time[3]). You could argue, that XML need not be fully featured one, but some simplified variant. Something that can be parsed by 30KB parser and not bloated Xerces. But that is not much different from using binary module-info.class[4]. -jt [1] http://wiki.netbeans.org/Jigsaw [2] please note that I am not talking about module-info.java [3] http://wiki.apidesign.org/wiki/Modular_Java_SE [4] If module-info.class format was made truly extensible > For yet another scenario, when a Linux distribution needs to patch > upstream module metadata for whatever reason (for example, changing a > choice of security module dependency), it will be far easier to alter an > XML file (treated as text, most likely), than to patch a > module-info.java file, then recompiling said file. PS: Linux distributions always recompile upstream projects from sources, so they would be patching module-info.java - which would be simple textual diff. From alexey.x.fedorov at oracle.com Mon Mar 26 15:41:45 2012 From: alexey.x.fedorov at oracle.com (Alexey Fedorov) Date: Tue, 27 Mar 2012 02:41:45 +0400 Subject: Small questions about @RequireOptionalModule Message-ID: <4F70F0A9.4000100@oracle.com> Hello, I have small question about current state of Jigsaw implementation: Suppose we have the following code fragment: > class A { > @RequireOptionalModule("absent.module") > public void foo(){ > //do nothing > } > } > > new A().foo(); @RequireOptionalModule javadoc alludes that if module "absent.module" is absent, this code throws ModuleNotPresentException, but it doesn't. Is it a bug? -- Thank you, Alexey From mandy.chung at oracle.com Mon Mar 26 16:24:37 2012 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 26 Mar 2012 16:24:37 -0700 Subject: Small questions about @RequireOptionalModule In-Reply-To: <4F70F0A9.4000100@oracle.com> References: <4F70F0A9.4000100@oracle.com> Message-ID: <4F70FAB5.6080206@oracle.com> On 3/26/2012 3:41 PM, Alexey Fedorov wrote: > Hello, > > I have small question about current state of Jigsaw implementation: > > Suppose we have the following code fragment: > >> class A { >> @RequireOptionalModule("absent.module") >> public void foo(){ >> //do nothing >> } >> } >> >> new A().foo(); > > @RequireOptionalModule javadoc alludes that if module "absent.module" > is absent, this code throws ModuleNotPresentException, but it doesn't. > Is it a bug? The annotation does not imply any runtime check. Instead the annotated method is responsible for testing if the "absent.module" is present before calling any API exported by that module [1] - i.e. by calling A.requriesModulePresent("absent.module") @RequireOptionalModule annotation is intended for developer tools such as javadoc and javac to determine the sources of optional dependencies and it's not used at runtime to check if the named module is present or not. This annotation can be used to annotate functionality that requires an optional module (e.g. Properties.loadFromXML) and this would assist developer tools for example to detect if the caller of such functionality is missing the optional module dependency. Mandy [1] http://openjdk.java.net/projects/jigsaw/doc/topics/optional.html From david.bosschaert at gmail.com Mon Mar 26 17:21:59 2012 From: david.bosschaert at gmail.com (David Bosschaert) Date: Mon, 26 Mar 2012 20:21:59 -0400 Subject: XML for module descriptor In-Reply-To: <201203261651.22636.jaroslav.tulach@oracle.com> References: <20111221223027.33AD123A6@eggemoggin.niobe.net> <4F6C6774.2050609@tibco.com> <201203261651.22636.jaroslav.tulach@oracle.com> Message-ID: Hi Jaroslav, On 26 March 2012 16:51, Jaroslav Tulach wrote: >>## 23.?3.?2012 08:07:16 ##< >> Hi David, >> >> On 3/23/12 12:42 PM, David Bosschaert wrote: >> > Just FYI I started modifying OpenJDK/Jigsaw in the context of the >> > Penrose project to see if something like module-info.xml instead of >> > module-info.class/.java would work. From the prototype that I made it >> > seems to make a lot of sense. See here: >> > http://mail.openjdk.java.net/pipermail/penrose-dev/2012-March/000023.html > > I cannot speak for the Jigsaw team, but (while my organization also needs > concurrently extensible module-info[1]) there may be a reason why not use XML > instead of module-info.class[2]. > > It is my understanding that one goal of Jigsaw is to make core Java really > small and allow it to execute on very limited devices. The core needs to > understand module-info. Right now the core contains ~3000 classes. If you > bring full XML parsing infrastructure in, then you increase the size by ~4000 > new classes (that was my observation when I worked on splitting JDK last > time[3]). You certainly don't need a full blown XML parser. I think a restricted one is fine. In my prototype I modified the (ASL2-licensed) Apache Piccolo project to get things to work. I didn't optimize at all. Piccolo.jar is about 160KB and around 50 classes in its original form but that includes all the SAX and JAXP interfaces that go with it. > You could argue, that XML need not be fully featured one, but some simplified > variant. Something that can be parsed by 30KB parser and not bloated Xerces. > But that is not much different from using binary module-info.class[4]. Right, I have seen XML parsers implemented in jars < 15Kb so if we decide to go this route a dedicated parser could be implemented in such a small footprint. I think it would be *very* different from parsing the module-info.class because * XML can be authored by a myriad of existing tools out there * XML can be read by humans * XML provides a well-defined extensibility model (using custom name-spaces) * XML can be parsed easily by other module systems (as they most likely have an XML parser at their disposal) * XML can be parsed easily by other tools that wish to operate on module information On the other hand, I'm not married on XML - another textual format might also do the job. I'm open to ideas here. Cheers, David From njbartlett at gmail.com Mon Mar 26 19:27:18 2012 From: njbartlett at gmail.com (Neil Bartlett) Date: Tue, 27 Mar 2012 03:27:18 +0100 Subject: XML for module descriptor In-Reply-To: <201203261651.22636.jaroslav.tulach@oracle.com> References: <20111221223027.33AD123A6@eggemoggin.niobe.net> <4F6C6774.2050609@tibco.com> <201203261651.22636.jaroslav.tulach@oracle.com> Message-ID: <776EA71639194190A326D87453D99F03@gmail.com> On Monday, 26 March 2012 at 21:51, Jaroslav Tulach wrote: > > ## 23.?3.?2012 08:07:16 ##< > > Hi David, > > > > On 3/23/12 12:42 PM, David Bosschaert wrote: > > > Just FYI I started modifying OpenJDK/Jigsaw in the context of the > > > Penrose project to see if something like module-info.xml instead of > > > module-info.class/.java would work. From the prototype that I made it > > > seems to make a lot of sense. See here: > > > http://mail.openjdk.java.net/pipermail/penrose-dev/2012-March/000023.html > > > > > > > > > > I cannot speak for the Jigsaw team, but (while my organization also needs > concurrently extensible module-info[1]) there may be a reason why not use XML > instead of module-info.class[2]. > > It is my understanding that one goal of Jigsaw is to make core Java really > small and allow it to execute on very limited devices. The core needs to > understand module-info. Right now the core contains ~3000 classes. If you > bring full XML parsing infrastructure in, then you increase the size by ~4000 > new classes (that was my observation when I worked on splitting JDK last > time[3]). > > I agree that bloating the core with an XML parser would be undesirable, but at the same time support David's desire for an extensible, textual format. If there were a textual format with a parser already supported in the core then perhaps this could be used. For example the MANIFEST.MF parsing in java.util.jar is currently part of core, I believe. Regards, Neil > You could argue, that XML need not be fully featured one, but some simplified > variant. Something that can be parsed by 30KB parser and not bloated Xerces. > But that is not much different from using binary module-info.class[4]. > > -jt > > [1] http://wiki.netbeans.org/Jigsaw > [2] please note that I am not talking about module-info.java > [3] http://wiki.apidesign.org/wiki/Modular_Java_SE > [4] If module-info.class format was made truly extensible > > > > For yet another scenario, when a Linux distribution needs to patch > > upstream module metadata for whatever reason (for example, changing a > > choice of security module dependency), it will be far easier to alter an > > XML file (treated as text, most likely), than to patch a > > module-info.java file, then recompiling said file. > > > > > PS: Linux distributions always recompile upstream projects from sources, so > they would be patching module-info.java - which would be simple textual diff. > > From jaroslav.tulach at oracle.com Mon Mar 26 20:25:57 2012 From: jaroslav.tulach at oracle.com (Jaroslav Tulach) Date: Mon, 26 Mar 2012 23:25:57 -0400 Subject: XML for module descriptor In-Reply-To: <776EA71639194190A326D87453D99F03@gmail.com> References: <20111221223027.33AD123A6@eggemoggin.niobe.net> <201203261651.22636.jaroslav.tulach@oracle.com> <776EA71639194190A326D87453D99F03@gmail.com> Message-ID: <201203262325.57089.jaroslav.tulach@oracle.com> Hello David, Neil, if I was supposed to marry the benefits of XML (tooling, extensibility, encoding) with ability to parse the info, I would claim that module-info.xml is a source format which gets compiled by javac into module-info.class. module-info.class would contain all the information needed by Jigsaw runtime (so there is no XML processing in the core) and somehow stored the rest of the information in the XML for other systems to process later (most trivially as whole XML kept in an attribute of module-info.class, but possibly in better way). If there is a utility to read MANIFEST.MF in core, it is not bad idea to reuse it. Properties like format (which manifests are using) is almost as extensible as XML, although validation tools are hard to find[1]. Anyway I seem to like the approach of having two extensible formats. One which is easily editable and audit-able (in spite of requiring sophisticated tools with high memory and CPU consumption) and one that is easy to process with minimal CPU cycles (generated during compilation). -jt [1] This would probably not happen in XML: http://wiki.apidesign.org/wiki/PropertyFiles >## 26.?3.?2012 22:27:18 ##< > On Monday, 26 March 2012 at 21:51, Jaroslav Tulach wrote: > > > ## 23.?3.?2012 08:07:16 ##< > > > Hi David, > > > > > > On 3/23/12 12:42 PM, David Bosschaert wrote: > > > > Just FYI I started modifying OpenJDK/Jigsaw in the context of the > > > > Penrose project to see if something like module-info.xml instead of > > > > module-info.class/.java would work. From the prototype that I made it > > > > seems to make a lot of sense. See here: > > > > http://mail.openjdk.java.net/pipermail/penrose-dev/2012-March/000023. > > > > html > > > > I cannot speak for the Jigsaw team, but (while my organization also needs > > concurrently extensible module-info[1]) there may be a reason why not use > > XML instead of module-info.class[2]. > > > > It is my understanding that one goal of Jigsaw is to make core Java > > really small and allow it to execute on very limited devices. The core > > needs to understand module-info. Right now the core contains ~3000 > > classes. If you bring full XML parsing infrastructure in, then you > > increase the size by ~4000 new classes (that was my observation when I > > worked on splitting JDK last time[3]). > > I agree that bloating the core with an XML parser would be undesirable, but > at the same time support David's desire for an extensible, textual format. > If there were a textual format with a parser already supported in the core > then perhaps this could be used. For example the MANIFEST.MF parsing in > java.util.jar is currently part of core, I believe. > > Regards, > Neil > > > You could argue, that XML need not be fully featured one, but some > > simplified variant. Something that can be parsed by 30KB parser and not > > bloated Xerces. But that is not much different from using binary > > module-info.class[4]. > > > > -jt > > > > [1] http://wiki.netbeans.org/Jigsaw > > [2] please note that I am not talking about module-info.java > > [3] http://wiki.apidesign.org/wiki/Modular_Java_SE > > [4] If module-info.class format was made truly extensible > > > > > For yet another scenario, when a Linux distribution needs to patch > > > upstream module metadata for whatever reason (for example, changing a > > > choice of security module dependency), it will be far easier to alter > > > an XML file (treated as text, most likely), than to patch a > > > module-info.java file, then recompiling said file. > > > > PS: Linux distributions always recompile upstream projects from sources, > > so they would be patching module-info.java - which would be simple > > textual diff. From alexey.x.fedorov at oracle.com Tue Mar 27 07:24:38 2012 From: alexey.x.fedorov at oracle.com (Alexey Fedorov) Date: Tue, 27 Mar 2012 18:24:38 +0400 Subject: Case - class from required module is not available Message-ID: <4F71CDA6.7090506@oracle.com> Hello! Suppose we have simple construction with 4 modules: **************************************** module a @ 1.0 { } **************************************** module b @ 1.0 { requires optional a@<1.0; exports b; } package b; public class B { } **************************************** module c @ 1.0 { requires optional a; requires b; exports c; class c.C; } package c; public class C { public static void main(String[] args) { try { Class bClazz = Class.forName("b.B"); System.out.println(bClazz + " is AVAILABLE from c.C!"); } catch (ClassNotFoundException e) { System.out.println("b.B is NOT AVAILABLE from c.C!"); } } } **************************************** module d @ 1.0 { requires optional a; requires optional b; exports d; class d.D; } package d; public class D { public static void main(String[] args) { try { Class bClazz = Class.forName("b.B"); System.out.println(bClazz + " is AVAILABLE from d.D!"); } catch (ClassNotFoundException e) { System.out.println("b.B is NOT AVAILABLE from d.D!"); } } } **************************************** Running c.C and d.D in sequence shows: > class b.B is AVAILABLE from c.C! > class b.B is NOT AVAILABLE from d.D! and it seems very strange for me. Could you please clarify, why class b.B is not available from d.D? -- Alexey From mandy.chung at oracle.com Tue Mar 27 10:53:23 2012 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 27 Mar 2012 10:53:23 -0700 Subject: Case - class from required module is not available In-Reply-To: <4F71CDA6.7090506@oracle.com> References: <4F71CDA6.7090506@oracle.com> Message-ID: <4F71FE93.2060404@oracle.com> This is an interesting scenario - module c is linked with module b only whereas module d is linked with module a only. $ jmod -L mlib dump-config c configuration roots = [c at 1.0] context +c module c at 1.0 view c at 1.0 local (1) c.C:c at 1.0 remote (46) b=+b context +b module b at 1.0 view b at 1.0 local (1) b.B:b at 1.0 remote (45) .... $ jmod -L mlib dump-config d configuration roots = [d at 1.0] context +d module d at 1.0 view d at 1.0 local (1) d.D:d at 1.0 remote (45) context +a module a at 1.0 view a at 1.0 remote (45) .... module c requires module b and an optional dependence on module c. module b has an optional dependence on module a@<1.0 which does not exist. Thus module c is not linked with module a at 1.0 as it doesn't satisfy the dependence constraint. module d has an optional dependence on both module a and b with no version specified. The resolver attempts to resolve module d with module a at 1.0 and b at 1.0. Since there is no module satisfying the dependence from module d *and* from module b on a - i.e. "requires optional a" and "requires optional a@<1.0", module a at 1.0 is not resolved and linked in module d's configuration. Mandy On 3/27/2012 7:24 AM, Alexey Fedorov wrote: > Hello! > > Suppose we have simple construction with 4 modules: > > **************************************** > > module a @ 1.0 { > } > > **************************************** > > module b @ 1.0 { > requires optional a@<1.0; > exports b; > } > > package b; > public class B { > } > > **************************************** > > module c @ 1.0 { > requires optional a; > requires b; > > exports c; > class c.C; > } > > package c; > public class C { > public static void main(String[] args) { > try { > Class bClazz = Class.forName("b.B"); > System.out.println(bClazz + " is AVAILABLE from c.C!"); > } catch (ClassNotFoundException e) { > System.out.println("b.B is NOT AVAILABLE from c.C!"); > } > } > } > > **************************************** > > module d @ 1.0 { > requires optional a; > requires optional b; > > exports d; > class d.D; > } > > package d; > public class D { > public static void main(String[] args) { > try { > Class bClazz = Class.forName("b.B"); > System.out.println(bClazz + " is AVAILABLE from d.D!"); > } catch (ClassNotFoundException e) { > System.out.println("b.B is NOT AVAILABLE from d.D!"); > } > } > } > > **************************************** > > Running c.C and d.D in sequence shows: > >> class b.B is AVAILABLE from c.C! >> class b.B is NOT AVAILABLE from d.D! > > and it seems very strange for me. Could you please clarify, why class > b.B is not available from d.D? > > -- > Alexey > > From david.holmes at oracle.com Tue Mar 27 18:48:47 2012 From: david.holmes at oracle.com (David Holmes) Date: Wed, 28 Mar 2012 11:48:47 +1000 Subject: Case - class from required module is not available In-Reply-To: <4F71FE93.2060404@oracle.com> References: <4F71CDA6.7090506@oracle.com> <4F71FE93.2060404@oracle.com> Message-ID: <4F726DFF.1030805@oracle.com> On 28/03/2012 3:53 AM, Mandy Chung wrote: > This is an interesting scenario - module c is linked with module b only > whereas module d is linked with module a only. So is that a bug or a feature? Given b will not be able to find a<1.0, but a1.0 is already loaded, is it then necessary that b not be linked to d? David ----- > $ jmod -L mlib dump-config c > configuration roots = [c at 1.0] > context +c > module c at 1.0 > view c at 1.0 > local (1) > c.C:c at 1.0 > remote (46) > b=+b > context +b > module b at 1.0 > view b at 1.0 > local (1) > b.B:b at 1.0 > remote (45) > .... > > $ jmod -L mlib dump-config d > configuration roots = [d at 1.0] > context +d > module d at 1.0 > view d at 1.0 > local (1) > d.D:d at 1.0 > remote (45) > context +a > module a at 1.0 > view a at 1.0 > remote (45) > .... > > module c requires module b and an optional dependence on module c. > module b has an optional dependence on module a@<1.0 which does not > exist. Thus module c is not linked with module a at 1.0 as it doesn't > satisfy the dependence constraint. > > module d has an optional dependence on both module a and b with no > version specified. The resolver attempts to resolve module d with module > a at 1.0 and b at 1.0. Since there is no module satisfying the dependence from > module d *and* from module b on a - i.e. "requires optional a" and > "requires optional a@<1.0", module a at 1.0 is not resolved and linked in > module d's configuration. > > Mandy > > On 3/27/2012 7:24 AM, Alexey Fedorov wrote: >> Hello! >> >> Suppose we have simple construction with 4 modules: >> >> **************************************** >> >> module a @ 1.0 { >> } >> >> **************************************** >> >> module b @ 1.0 { >> requires optional a@<1.0; >> exports b; >> } >> >> package b; >> public class B { >> } >> >> **************************************** >> >> module c @ 1.0 { >> requires optional a; >> requires b; >> >> exports c; >> class c.C; >> } >> >> package c; >> public class C { >> public static void main(String[] args) { >> try { >> Class bClazz = Class.forName("b.B"); >> System.out.println(bClazz + " is AVAILABLE from c.C!"); >> } catch (ClassNotFoundException e) { >> System.out.println("b.B is NOT AVAILABLE from c.C!"); >> } >> } >> } >> >> **************************************** >> >> module d @ 1.0 { >> requires optional a; >> requires optional b; >> >> exports d; >> class d.D; >> } >> >> package d; >> public class D { >> public static void main(String[] args) { >> try { >> Class bClazz = Class.forName("b.B"); >> System.out.println(bClazz + " is AVAILABLE from d.D!"); >> } catch (ClassNotFoundException e) { >> System.out.println("b.B is NOT AVAILABLE from d.D!"); >> } >> } >> } >> >> **************************************** >> >> Running c.C and d.D in sequence shows: >> >>> class b.B is AVAILABLE from c.C! >>> class b.B is NOT AVAILABLE from d.D! >> >> and it seems very strange for me. Could you please clarify, why class >> b.B is not available from d.D? >> >> -- >> Alexey >> >> From mandy.chung at oracle.com Tue Mar 27 22:57:12 2012 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 27 Mar 2012 22:57:12 -0700 Subject: Case - class from required module is not available In-Reply-To: <4F726DFF.1030805@oracle.com> References: <4F71CDA6.7090506@oracle.com> <4F71FE93.2060404@oracle.com> <4F726DFF.1030805@oracle.com> Message-ID: <4F72A838.2070608@oracle.com> On 3/27/2012 6:48 PM, David Holmes wrote: >> This is an interesting scenario - module c is linked with module b only >> whereas module d is linked with module a only. > > So is that a bug or a feature? Given b will not be able to find a<1.0, > but a1.0 is already loaded, is it then necessary that b not be linked > to d? I consider it's not a bug as a module be resolved should have all of its dependences satisfied with all the modules in the resolution graph (i.e. module a is present in the graph and b -> a has to be satisfied for b to be resolved). Mark, what do you think? Mandy From Alan.Bateman at oracle.com Wed Mar 28 01:31:22 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 28 Mar 2012 09:31:22 +0100 Subject: Case - class from required module is not available In-Reply-To: <4F726DFF.1030805@oracle.com> References: <4F71CDA6.7090506@oracle.com> <4F71FE93.2060404@oracle.com> <4F726DFF.1030805@oracle.com> Message-ID: <4F72CC5A.1040903@oracle.com> On 28/03/2012 02:48, David Holmes wrote: > On 28/03/2012 3:53 AM, Mandy Chung wrote: >> This is an interesting scenario - module c is linked with module b only >> whereas module d is linked with module a only. > > So is that a bug or a feature? Given b will not be able to find a<1.0, > but a1.0 is already loaded, is it then necessary that b not be linked > to d? Right, a at 1.0 and b at 1.0 can't be the same configuration because of the constraint so the answer for d has to be {d at 1.0, a at 1.0} or {d at 1.0, b at 1.0}. There is code in the resolver that ensures that the dependencies are examined in order so I assume this is why {d at 1.0, a at 1.0} is chosen. If d's dependencies are reversed then it will generate the other answer. With c then the answer has to be {c at 1.0, b at 1.0}. -Alan. From david.holmes at oracle.com Wed Mar 28 03:34:32 2012 From: david.holmes at oracle.com (David Holmes) Date: Wed, 28 Mar 2012 20:34:32 +1000 Subject: Case - class from required module is not available In-Reply-To: <4F72CC5A.1040903@oracle.com> References: <4F71CDA6.7090506@oracle.com> <4F71FE93.2060404@oracle.com> <4F726DFF.1030805@oracle.com> <4F72CC5A.1040903@oracle.com> Message-ID: <4F72E938.7090506@oracle.com> On 28/03/2012 6:31 PM, Alan Bateman wrote: > On 28/03/2012 02:48, David Holmes wrote: >> On 28/03/2012 3:53 AM, Mandy Chung wrote: >>> This is an interesting scenario - module c is linked with module b only >>> whereas module d is linked with module a only. >> >> So is that a bug or a feature? Given b will not be able to find a<1.0, >> but a1.0 is already loaded, is it then necessary that b not be linked >> to d? > Right, a at 1.0 and b at 1.0 can't be the same configuration because of the > constraint so the answer for d has to be {d at 1.0, a at 1.0} or {d at 1.0, > b at 1.0}. There is code in the resolver that ensures that the dependencies > are examined in order so I assume this is why {d at 1.0, a at 1.0} is chosen. > If d's dependencies are reversed then it will generate the other answer. > With c then the answer has to be {c at 1.0, b at 1.0}. Obviously there's a lot of detailed semantics here that I'm not clear on. Is this just an artifact of the current implementation strategy or a hard limitation in the overall model. My thinking is that b could still be made available to d as long as b's attempts to access anything in "a" give a "module not found" exception. But that all depends on how things are actually implemented. Just curious. Thanks, David > -Alan. > > From mcconnell at dpml.net Wed Mar 28 06:06:31 2012 From: mcconnell at dpml.net (Stephen McConnell) Date: Wed, 28 Mar 2012 23:36:31 +1030 Subject: (Complete) range dependencies back in the game? In-Reply-To: <201203252250.30516.jaroslav.tulach@oracle.com> References: <20111223202734.4467CF7F@eggemoggin.niobe.net><201201081744.57845.jaroslav.tulach@oracle.com><4F1032DF.20402@oracle.com> <201203252250.30516.jaroslav.tulach@oracle.com> Message-ID: Jaroslav said: I am not real fan of separating SPI and API. I recognise your position but question your judgement. almost nobody understands what SPI vs. API is Which is basically saying that within your circle of influence - the distinction is not present? I get that. It doesn't mean that we have to sink to the lowest common denominator. I need better than that - I need the solutions that will take my best and brightest to the next step. Frankly I don't care too much about the 'almost nobody' spectrum. I do care intensely about that selective set - those people your refer to as "almost nobody" - the ones that get this stuff and do it day in and day out and expect the next generation of tooling to meet their professional aspirations. Cheers, Steve. -----Original Message----- From: Jaroslav Tulach Sent: Monday, March 26, 2012 1:20 PM To: jigsaw-dev at openjdk.java.net Subject: Re: (Complete) range dependencies back in the game? Hello guys, I am not real fan of separating SPI and API. Either the designer knows what is the goal or not. Either the class should belong to API or SPI. Such distinction is possible even now in Java. Whether it needs some special constructs in the future depends on our understanding how frequent such usage case is. We may expect it is frequent because almost nobody understands what SPI vs. API is. Rather than changing the language, rather than changing the modularity, it would be better to restrict and define what makes sense. >## 13. 1. 2012 08:34:23 ##< > On 01/08/2012 11:44 AM, Jaroslav Tulach wrote: > > http://wiki.apidesign.org/wiki/RangeDependenciesAnalysed > > "It does not make much sense to have higher upper bound of dependency on C > higher than any existing upper bound on C among reused libraries (thus s > <= q)" - I disagree that this does not make sense. The basic problem is not about disagreeing, however about providing flexibility while keeping the system powerful enough. > Let us say that C has both an API and an SPI; the API is kept quite stable > but the SPI is frequently rewritten (and the handful of implementations > are expected to keep up). So pretend C is DOM, and API clients normally > use ranges like [1.2,2) ("DOM 2 or any foreseeable compatible release") > whereas SPI implementations normally use ranges like [1.2,1.3) ("DOM 2 but > not 3"). Jesse, I know you are obsessed by different stability levels of API and SPIs. True, you are not the only one. I am just saying we can do much better. Those who assumed certain classes will never be implemented by NetBeans IDE were wrong! We have implemented almost everything! If it was just an attempt to intercept the calls - if there is an interface, people will implement it. So just stop your "friendly implementor vision". It hurts everyone. Try to behave like real "distributed fan" - allow people to evolve on their own, just keep backward compatibility. -jt > Now pretend B is a library for XML persistence of Java objects, and during > one serialization mode it creates flyweight org.w3c.dom.Document's. > Versions 1.0 through 1.3, written back when DOM 2 ruled the land, declares > [1.2,1.3); version 1.4 and later implement DOM 3 so declare [1.3,1.4) but > are otherwise fully compatible for callers. > > Finally say A is an application which uses XML serialization for some > preferences storage, and also uses DOM quite independently for dealing > with REST clients. So it would like to declare B@[1.0,2) and C@[1.2,2): it > uses only the basic initial serialization API, and can work with any known > DOM release. > > The fact that B at 1.0 is incompatible with C at 1.3 does not trouble the author > of A; the app would work fine with B at 1.5 and C at 1.3 as well. If the > compiler silently converted the dep on C@[1.2,2) to C@[1.2,1.3), it would > effectively mean that B at 1.4 could not be used even if it had numerous bug > fixes, which is generally not what you want. It could instead just reject > A's attempted expression of what its actual dependency ranges are, with > some sort of error message TBD, and the author of A would (after some > trial and error) eventually give up on supporting older libraries and > request B@[1.4,2) and C[1.3,2). > > This switch would be rather artificial: the originally attempted > dependency > declaration was logically sound, and a module system using a SAT solver > would not have difficulty linking A with either newer or older libraries > according to what is available. (An organization which decided it hated > the DOM 3 license might forbid it from being installed, in which case you > would like to be able to run A with B at 1.3 and C at 1.2.) > > Indeed if A does not use XML namespaces in its REST API, it could as well > declare C@[1.1,2) even knowing that current versions of B require at least > C at 1.2; a future version of B might stop needing to implement DOM and (if > it did not use namespaces either) might declare C@[1.1,2), in which case > B at 1.7 C at 1.1 would be a valid configuration too. Obviously this is a less > likely scenario, but it shows that "[t]his definition seems to make sense > from the point of lower bound" is not so obvious either. > > > On this topic, I think the definition of complete repository is > unnecessarily strict; there is no reason apparent to me why [r,s) must be > a subrange of [p,q), though it does at least need to overlap (and probably > a version of C in that intersection must actually be present in the repo). > Where in the proof is the assumption of subranges used? You only check > whether cr is empty, which it would be in the attempted conversion to > 3-SAT. It seems that A could express any dependency on C it likes, and > trying to compile A would either work or not work, according to whether > its dep on C has at least some overlap with that of the earliest versions > of its other dependencies. > > > By the way, you do not really discuss selecting a configuration at install > time in a complete repository. [1] does not seem to apply to arbitrary > range dependencies. Probably you can start with a configuration using the > lower bounds of all dependencies as determined during compilation, then > iteratively try raising the version of each module individually so long as > doing so continues to result in a valid configuration - i.e. pick a > locally maximal configuration. This is polynomial. > > > [1] > http://wiki.apidesign.org/wiki/LibraryWithoutImplicitExportIsPolynomial#Me > rge_of_Configurations From richard.s.hall at oracle.com Wed Mar 28 10:24:27 2012 From: richard.s.hall at oracle.com (Richard S. Hall) Date: Wed, 28 Mar 2012 13:24:27 -0400 Subject: Case - class from required module is not available In-Reply-To: <4F72E938.7090506@oracle.com> References: <4F71CDA6.7090506@oracle.com> <4F71FE93.2060404@oracle.com> <4F726DFF.1030805@oracle.com> <4F72CC5A.1040903@oracle.com> <4F72E938.7090506@oracle.com> Message-ID: <4F73494B.70801@oracle.com> On 3/28/12 6:34, David Holmes wrote: > On 28/03/2012 6:31 PM, Alan Bateman wrote: >> On 28/03/2012 02:48, David Holmes wrote: >>> On 28/03/2012 3:53 AM, Mandy Chung wrote: >>>> This is an interesting scenario - module c is linked with module b >>>> only >>>> whereas module d is linked with module a only. >>> >>> So is that a bug or a feature? Given b will not be able to find a<1.0, >>> but a1.0 is already loaded, is it then necessary that b not be linked >>> to d? >> Right, a at 1.0 and b at 1.0 can't be the same configuration because of the >> constraint so the answer for d has to be {d at 1.0, a at 1.0} or {d at 1.0, >> b at 1.0}. There is code in the resolver that ensures that the dependencies >> are examined in order so I assume this is why {d at 1.0, a at 1.0} is chosen. >> If d's dependencies are reversed then it will generate the other answer. >> With c then the answer has to be {c at 1.0, b at 1.0}. > > Obviously there's a lot of detailed semantics here that I'm not clear > on. Is this just an artifact of the current implementation strategy or > a hard limitation in the overall model. My thinking is that b could > still be made available to d as long as b's attempts to access > anything in "a" give a "module not found" exception. But that all > depends on how things are actually implemented. I would guess it is a hard limitation on the model. This is related to "uses" constraints in OSGi. In OSGi, "uses" constraints are expressed on exported packages and they specifically describe which other imported/exported packages are used by a given exported package (i.e., they are intra-module dependencies as opposed to the normal inter-module dependencies). This allows OSGi to partition resolution graphs in more fine-grained ways to support side-by-side types without inconsistency issues. Since Jigsaw doesn't model "uses" constraints at all, the only thing its resolver can do is assume that all required/exported types of a given module are used by all its exported types, which means it has to make coarser-grained decisions. So, in your case, it has to assume that your D would be expecting to get A types from B if it were resolved to both A and B. Thus, it cannot allow that since A isn't satisfied for B and thus it can only allow D to resolve to A. -> richard > > Just curious. > > Thanks, > David > >> -Alan. >> >> From david.bosschaert at gmail.com Wed Mar 28 11:09:50 2012 From: david.bosschaert at gmail.com (David Bosschaert) Date: Wed, 28 Mar 2012 14:09:50 -0400 Subject: XML for module descriptor In-Reply-To: <201203262325.57089.jaroslav.tulach@oracle.com> References: <20111221223027.33AD123A6@eggemoggin.niobe.net> <201203261651.22636.jaroslav.tulach@oracle.com> <776EA71639194190A326D87453D99F03@gmail.com> <201203262325.57089.jaroslav.tulach@oracle.com> Message-ID: Hi Jaroslav, I think combining module-info.class with an XML appendix really is the worst of both worlds, because it doesn't allow you to use XML tools to edit/view, as you end up with a combination binary/XML file. I would be much more in favour of staying within a textual format (XML, MANIFEST as Neil proposed, or something else) in the ultimate module where the actual JRE can cache a binary representation of this internally if it wishes to. WRT to he MANIFEST.MF, yes I think the core contains a parser for that, so it could be used. OSGi has leveraged the Manifest for about 14 years now without problems. I used XML in my prototype as it seems like a more versatile format than MANIFEST.MF and since we're designing something new here it might make sense to move to a more generally understood format. I don't think the implementation size of an XML parser which is only focused on parsing this file should be a big issue. I have to say I don't like having 2 different official representations of the module info at all (sorry ;) as it carries all kinds of synchronization issues and possibly doubles the burden of supporting it. If you want a binary format I think the way to go is let the JRE implementation generate that under the covers and cache it to optimize speed etc. Best regards, David On 26 March 2012 23:25, Jaroslav Tulach wrote: > Hello David, Neil, > if I was supposed to marry the benefits of XML (tooling, extensibility, > encoding) with ability to parse the info, I would claim that module-info.xml > is a source format which gets compiled by javac into module-info.class. > module-info.class would contain all the information needed by Jigsaw runtime > (so there is no XML processing in the core) and somehow stored the rest of the > information in the XML for other systems to process later (most trivially as > whole XML kept in an attribute of module-info.class, but possibly in better > way). > > If there is a utility to read MANIFEST.MF in core, it is not bad idea to reuse > it. Properties like format (which manifests are using) is almost as extensible > as XML, although validation tools are hard to find[1]. > > Anyway I seem to like the approach of having two extensible formats. One which > is easily editable and audit-able (in spite of requiring sophisticated tools > with high memory and CPU consumption) and one that is easy to process with > minimal CPU cycles (generated during compilation). > -jt > > [1] This would probably not happen in XML: > http://wiki.apidesign.org/wiki/PropertyFiles > > >>## 26.?3.?2012 22:27:18 ##< >> On Monday, 26 March 2012 at 21:51, Jaroslav Tulach wrote: >> > > ## 23.?3.?2012 08:07:16 ##< >> > > Hi David, >> > > >> > > On 3/23/12 12:42 PM, David Bosschaert wrote: >> > > > Just FYI I started modifying OpenJDK/Jigsaw in the context of the >> > > > Penrose project to see if something like module-info.xml instead of >> > > > module-info.class/.java would work. From the prototype that I made it >> > > > seems to make a lot of sense. See here: >> > > > http://mail.openjdk.java.net/pipermail/penrose-dev/2012-March/000023. >> > > > html >> > >> > I cannot speak for the Jigsaw team, but (while my organization also needs >> > concurrently extensible module-info[1]) there may be a reason why not use >> > XML instead of module-info.class[2]. >> > >> > It is my understanding that one goal of Jigsaw is to make core Java >> > really small and allow it to execute on very limited devices. The core >> > needs to understand module-info. Right now the core contains ~3000 >> > classes. If you bring full XML parsing infrastructure in, then you >> > increase the size by ~4000 new classes (that was my observation when I >> > worked on splitting JDK last time[3]). >> >> I agree that bloating the core with an XML parser would be undesirable, but >> at the same time support David's desire for an extensible, textual format. >> If there were a textual format with a parser already supported in the core >> then perhaps this could be used. For example the MANIFEST.MF parsing in >> java.util.jar is currently part of core, I believe. >> >> Regards, >> Neil >> >> > You could argue, that XML need not be fully featured one, but some >> > simplified variant. Something that can be parsed by 30KB parser and not >> > bloated Xerces. But that is not much different from using binary >> > module-info.class[4]. >> > >> > -jt >> > >> > [1] http://wiki.netbeans.org/Jigsaw >> > [2] please note that I am not talking about module-info.java >> > [3] http://wiki.apidesign.org/wiki/Modular_Java_SE >> > [4] If module-info.class format was made truly extensible >> > >> > > For yet another scenario, when a Linux distribution needs to patch >> > > upstream module metadata for whatever reason (for example, changing a >> > > choice of security module dependency), it will be far easier to alter >> > > an XML file (treated as text, most likely), than to patch a >> > > module-info.java file, then recompiling said file. >> > >> > PS: Linux distributions always recompile upstream projects from sources, >> > so they would be patching module-info.java - which would be simple >> > textual diff. From forax at univ-mlv.fr Wed Mar 28 11:52:31 2012 From: forax at univ-mlv.fr (=?UTF-8?B?UsOpbWkgRm9yYXg=?=) Date: Wed, 28 Mar 2012 20:52:31 +0200 Subject: XML for module descriptor In-Reply-To: References: <20111221223027.33AD123A6@eggemoggin.niobe.net> <201203261651.22636.jaroslav.tulach@oracle.com> <776EA71639194190A326D87453D99F03@gmail.com> <201203262325.57089.jaroslav.tulach@oracle.com> Message-ID: <4F735DEF.5000101@univ-mlv.fr> On 03/28/2012 08:09 PM, David Bosschaert wrote: > Hi Jaroslav, > > I think combining module-info.class with an XML appendix really is the > worst of both worlds, because it doesn't allow you to use XML tools to > edit/view, as you end up with a combination binary/XML file. I would > be much more in favour of staying within a textual format (XML, > MANIFEST as Neil proposed, or something else) in the ultimate module > where the actual JRE can cache a binary representation of this > internally if it wishes to. > > WRT to he MANIFEST.MF, yes I think the core contains a parser for > that, so it could be used. OSGi has leveraged the Manifest for about > 14 years now without problems. I used XML in my prototype as it seems > like a more versatile format than MANIFEST.MF and since we're > designing something new here it might make sense to move to a more > generally understood format. I don't think the implementation size of > an XML parser which is only focused on parsing this file should be a > big issue. There are issues with manifest, the two main points for my students are that they publish manifest that don't even parse and the second one is that they keep in the manifest generated information that are not true anymore. > > I have to say I don't like having 2 different official representations > of the module info at all (sorry ;) as it carries all kinds of > synchronization issues and possibly doubles the burden of supporting > it. If you want a binary format I think the way to go is let the JRE > implementation generate that under the covers and cache it to optimize > speed etc. As I already said, the best is to develop a small parser that is able to parse a module-info.class and provide XML events or properties. > > Best regards, > > David best regards, R?mi From richard.s.hall at oracle.com Wed Mar 28 12:11:11 2012 From: richard.s.hall at oracle.com (Richard S. Hall) Date: Wed, 28 Mar 2012 15:11:11 -0400 Subject: Case - class from required module is not available In-Reply-To: <4F73494B.70801@oracle.com> References: <4F71CDA6.7090506@oracle.com> <4F71FE93.2060404@oracle.com> <4F726DFF.1030805@oracle.com> <4F72CC5A.1040903@oracle.com> <4F72E938.7090506@oracle.com> <4F73494B.70801@oracle.com> Message-ID: <4F73624F.6040706@oracle.com> On 3/28/12 13:24, Richard S. Hall wrote: > On 3/28/12 6:34, David Holmes wrote: >> On 28/03/2012 6:31 PM, Alan Bateman wrote: >>> On 28/03/2012 02:48, David Holmes wrote: >>>> On 28/03/2012 3:53 AM, Mandy Chung wrote: >>>>> This is an interesting scenario - module c is linked with module b >>>>> only >>>>> whereas module d is linked with module a only. >>>> >>>> So is that a bug or a feature? Given b will not be able to find a<1.0, >>>> but a1.0 is already loaded, is it then necessary that b not be linked >>>> to d? >>> Right, a at 1.0 and b at 1.0 can't be the same configuration because of the >>> constraint so the answer for d has to be {d at 1.0, a at 1.0} or {d at 1.0, >>> b at 1.0}. There is code in the resolver that ensures that the >>> dependencies >>> are examined in order so I assume this is why {d at 1.0, a at 1.0} is chosen. >>> If d's dependencies are reversed then it will generate the other >>> answer. >>> With c then the answer has to be {c at 1.0, b at 1.0}. >> >> Obviously there's a lot of detailed semantics here that I'm not clear >> on. Is this just an artifact of the current implementation strategy >> or a hard limitation in the overall model. My thinking is that b >> could still be made available to d as long as b's attempts to access >> anything in "a" give a "module not found" exception. But that all >> depends on how things are actually implemented. > > I would guess it is a hard limitation on the model. This is related to > "uses" constraints in OSGi. In OSGi, "uses" constraints are expressed > on exported packages and they specifically describe which other > imported/exported packages are used by a given exported package (i.e., > they are intra-module dependencies as opposed to the normal > inter-module dependencies). This allows OSGi to partition resolution > graphs in more fine-grained ways to support side-by-side types without > inconsistency issues. Should have said, "...side-by-side versions..." -> richard > > Since Jigsaw doesn't model "uses" constraints at all, the only thing > its resolver can do is assume that all required/exported types of a > given module are used by all its exported types, which means it has to > make coarser-grained decisions. So, in your case, it has to assume > that your D would be expecting to get A types from B if it were > resolved to both A and B. Thus, it cannot allow that since A isn't > satisfied for B and thus it can only allow D to resolve to A. > > -> richard > >> >> Just curious. >> >> Thanks, >> David >> >>> -Alan. >>> >>> From neal at gafter.com Wed Mar 28 17:34:05 2012 From: neal at gafter.com (Neal Gafter) Date: Wed, 28 Mar 2012 17:34:05 -0700 Subject: hg: jigsaw/jigsaw/jdk: Remove module annotation support In-Reply-To: <4F231EF7.1070002@oracle.com> References: <20120127041553.5F87A471FD@hg.openjdk.java.net> <4F22BDBF.1090908@oracle.com> <4F22C073.5000006@oracle.com> <4F22CCEB.6060300@oracle.com> <4F231EF7.1070002@oracle.com> Message-ID: On Fri, Jan 27, 2012 at 2:02 PM, Alex Buckley wrote: > On 1/27/2012 8:12 AM, Jesse Glick wrote: > >> On 01/27/2012 10:19 AM, Alan Bateman wrote: >> >>> Anything after the module declaration is passed through by the >>> compiler into the ModuleData attribute. >>> >> >> lang-vm.html [1] does not seem to mention this but perhaps it is too >> old. >> > > It doesn't mention it because it doesn't need to. The limit of the Java > language is the CompilationUnit production. A compiler is free to parse > text outside that production and store it in a ModuleData attribute. That's interesting. Since the production CompilationUnit can derive the empty string, the entire source file could be something "outside that production" and could include text in some other language virtually identical to Java but with particular interesting extensions, such as AOP or delegates. If I understand your statement, such a source file is still a valid Java(TM) Programming Language compilation unit. Do I have that right? From Alan.Bateman at oracle.com Thu Mar 29 05:05:32 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 29 Mar 2012 13:05:32 +0100 Subject: XML for module descriptor In-Reply-To: <4F735DEF.5000101@univ-mlv.fr> References: <20111221223027.33AD123A6@eggemoggin.niobe.net> <201203261651.22636.jaroslav.tulach@oracle.com> <776EA71639194190A326D87453D99F03@gmail.com> <201203262325.57089.jaroslav.tulach@oracle.com> <4F735DEF.5000101@univ-mlv.fr> Message-ID: <4F74500C.1030605@oracle.com> On 28/03/2012 19:52, R?mi Forax wrote: > : > As I already said, the best is to develop a small parser > that is able to parse a module-info.class and provide > XML events or properties. FWIW, ModuleSystem already defines a parseModuleInfo method for parsing module-info.class files. On XML then the current prototype base module does not include JAXP or any XML support. -Alan. From jonathan.gibbons at oracle.com Fri Mar 30 03:23:20 2012 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Fri, 30 Mar 2012 11:23:20 +0100 Subject: Current javadoc out of date Message-ID: <4F758998.6050700@oracle.com> On the jigsaw project page, the link "current javadoc" takes me to a page dated 2010-06-08 12:55 -0700, which appears to be somewhat less than currenrt. -- Jon From mark.reinhold at oracle.com Fri Mar 30 13:08:31 2012 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Fri, 30 Mar 2012 13:08:31 -0700 Subject: Current javadoc out of date In-Reply-To: jonathan.gibbons@oracle.com; Fri, 30 Mar 2012 11:23:20 BST; <4F758998.6050700@oracle.com> Message-ID: <20120330200831.E916399D@eggemoggin.niobe.net> 2012/3/30 3:23 -0700, jonathan.gibbons at oracle.com: > On the jigsaw project page, the link "current javadoc" takes me to a page dated > 2010-06-08 12:55 -0700, which appears to be somewhat less than currenrt. Updated. (We'll automate this eventually.) - Mark From debasish.raychawdhuri at gmail.com Sat Mar 31 20:56:17 2012 From: debasish.raychawdhuri at gmail.com (Debasish Ray Chawdhuri) Date: Sun, 1 Apr 2012 09:26:17 +0530 Subject: IllegalArgumentException in jmod for modules without version Message-ID: Hi, I am getting java.lang.IllegalArgumentException: default: Version does not start with a number in jmod if I try to install a module without an explicit version. What happens is that if that if there is no explicit version, jmod creates a directory named 'default' instead of a version number, which then fails the test of numeric start of a version number. Is it a bug? -- Debasish Ray Chawdhuri