From mr at sun.com Fri May 1 09:42:05 2009 From: mr at sun.com (Mark Reinhold) Date: Fri, 01 May 2009 09:42:05 -0700 Subject: Initial thoughts on module packaging Message-ID: <20090501164205.103AB5E2F@eggemoggin.niobe.net> At this point we have a basic library CLI and module-system runtime working, and a plan for how javac will read modularized source trees and write to modularized output directories [1]. One of the biggest missing pieces is the packaging tool, so here are some thoughts on how that might work. The primary purpose of the packaging tool is to gather one or more modules from a javac output directory and produce a native package (e.g., rpm, deb, or ips) which, when installed in the usual manner on a target machine, uses the library CLI together with appropriate native tools to install the module. In concrete terms, the workflow would be something like this (on Debian Linux): $ cat src/modules/com.foo.bar/module-info.java module com.foo.bar @ 1.2.3-9 { ... } $ javac -modulesourcepath src/modules -d build/modules $FILES $ jpkg -m build/modules -d build/packages com.foo.bar $ ls build/packages com.foo.bar-1.2.3-9.deb $ sudo dpkg --install build/packages/com.foo.bar-1.2.3-9.deb The version and dependence information in the package's control file would be derived from that of the module-info class file. The data files in the package would be in the on-disk module-library format, minus configuration data. They would install, by default, into the system's module library at, say, /usr/lib/java/modules. Once the files are installed then the package's post-installation script would, essentially, do this: $ jmod config com.foo.bar at 1.2.3-9 This would compute a configuration for com.foo.bar, if it's a root module (i.e., a module with a main class); it would also recompute the configuration of any existing root module in the library that depends on com.foo.bar. To make this work we'll need a new method on the Library class which can write a partial module library to an arbitrary location, and the new jmod config subcommand. These should be straightforward; I'll hack them up today. Dalibor -- To start, I suggest you work on the simple case of pure Java modules. We'll tackle native libraries, resource files, and configuration files later on. - Mark [1] http://openjdk.java.net/projects/jigsaw/doc/ModulesAndJavac.pdf From mr at sun.com Fri May 1 09:56:20 2009 From: mr at sun.com (mr at sun.com) Date: Fri, 01 May 2009 16:56:20 +0000 Subject: hg: jigsaw/jigsaw/jdk: Untabify Message-ID: <20090501165639.752E9ED61@hg.openjdk.java.net> Changeset: ba561d13a142 Author: mr Date: 2009-05-01 09:54 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/ba561d13a142 Untabify ! src/share/classes/java/lang/Class.java ! src/share/classes/java/lang/System.java ! src/share/classes/java/lang/module/Dependence.java ! src/share/classes/java/lang/module/ModuleClassLoader.java ! src/share/classes/java/lang/module/ModuleId.java ! src/share/classes/java/lang/module/ModuleIdQuery.java ! src/share/classes/java/lang/module/ModuleInfoReader.java ! src/share/classes/java/lang/module/ModuleSystem.java ! src/share/classes/java/lang/reflect/Module.java ! src/share/classes/org/openjdk/jigsaw/ClassInfo.java ! src/share/classes/org/openjdk/jigsaw/Configuration.java ! src/share/classes/org/openjdk/jigsaw/ConfigurationException.java ! src/share/classes/org/openjdk/jigsaw/Context.java ! src/share/classes/org/openjdk/jigsaw/FileConstants.java ! src/share/classes/org/openjdk/jigsaw/Files.java ! src/share/classes/org/openjdk/jigsaw/JigsawModuleSystem.java ! src/share/classes/org/openjdk/jigsaw/JigsawVersion.java ! src/share/classes/org/openjdk/jigsaw/JigsawVersionQuery.java ! src/share/classes/org/openjdk/jigsaw/KernelLoader.java ! src/share/classes/org/openjdk/jigsaw/Launcher.java ! src/share/classes/org/openjdk/jigsaw/Library.java ! src/share/classes/org/openjdk/jigsaw/Loader.java ! src/share/classes/org/openjdk/jigsaw/LoaderPool.java ! src/share/classes/org/openjdk/jigsaw/Resolver.java ! src/share/classes/org/openjdk/jigsaw/SimpleLibrary.java ! src/share/classes/org/openjdk/jigsaw/Trace.java ! src/share/classes/org/openjdk/jigsaw/cli/Command.java ! src/share/classes/org/openjdk/jigsaw/cli/Librarian.java From mr at sun.com Fri May 1 10:10:25 2009 From: mr at sun.com (Mark Reinhold) Date: Fri, 01 May 2009 10:10:25 -0700 Subject: javac support for -modulepath Message-ID: <20090501171025.63A3C5E2F@eggemoggin.niobe.net> Jon -- How close are you to having a javac that supports the new -modulepath option, and the corresponding modularized output directory, proposed in the slides that you and Alex put together? Is this something that can be finished independently of hooking up javac itself to the module system? If support for -modulepath will come later rather than sooner then for testing purposes I can hack up a temporary jmod subcommand to convert the current output-directory format to the new one, but if you can get this done sooner then I won't bother. - Mark From Jonathan.Gibbons at Sun.COM Fri May 1 10:19:32 2009 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Fri, 01 May 2009 10:19:32 -0700 Subject: javac support for -modulepath In-Reply-To: <20090501171025.63A3C5E2F@eggemoggin.niobe.net> References: <20090501171025.63A3C5E2F@eggemoggin.niobe.net> Message-ID: <9EFFAB7C-C34D-430A-8D58-62E7BD3AE4E0@Sun.COM> I'm working on it, and it is beginning to come together. The changes to support the new module layout are somewhat bigger than expected: having a module "share the package namespace" enabled me to leverage a lot of code. Last night I uncovered a problem with the combination of code in the unnamed package with module path, but this morning I have a proposed solution to try out. While not so important in itself, it impacts the API the compiler uses to write files in the correct location. All the work so far is independent of hooking it to the Jigsaw module system. I'm currently using a dummy/toy module resolver (even simpler than Alex's so-called ZeroMod.) -- Jon On May 1, 2009, at 10:10 AM, Mark Reinhold wrote: > Jon -- How close are you to having a javac that supports the new > -modulepath option, and the corresponding modularized output > directory, proposed in the slides that you and Alex put together? > > Is this something that can be finished independently of hooking up > javac itself to the module system? > > If support for -modulepath will come later rather than sooner then > for testing purposes I can hack up a temporary jmod subcommand to > convert the current output-directory format to the new one, but if > you can get this done sooner then I won't bother. > > - Mark From mr at sun.com Fri May 1 13:24:45 2009 From: mr at sun.com (mr at sun.com) Date: Fri, 01 May 2009 20:24:45 +0000 Subject: hg: jigsaw/jigsaw/jdk: Pre-installation and explicit configuration Message-ID: <20090501202504.9DA89ED83@hg.openjdk.java.net> Changeset: f26ee715552b Author: mr Date: 2009-05-01 13:22 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/f26ee715552b Pre-installation and explicit configuration ! src/share/classes/org/openjdk/jigsaw/Library.java ! src/share/classes/org/openjdk/jigsaw/SimpleLibrary.java ! src/share/classes/org/openjdk/jigsaw/cli/Librarian.java + test/org/openjdk/jigsaw/preinstall-setup.sh + test/org/openjdk/jigsaw/preinstall.sh From mr at sun.com Fri May 1 13:28:32 2009 From: mr at sun.com (mr at sun.com) Date: Fri, 01 May 2009 20:28:32 +0000 Subject: hg: jigsaw/jigsaw/jdk: Fix unit test Message-ID: <20090501202844.D18C7ED8A@hg.openjdk.java.net> Changeset: 1ac9d70a1339 Author: mr Date: 2009-05-01 13:26 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/1ac9d70a1339 Fix unit test ! test/org/openjdk/jigsaw/preinstall.sh From mr at sun.com Tue May 5 09:00:15 2009 From: mr at sun.com (Mark Reinhold) Date: Tue, 05 May 2009 09:00:15 -0700 Subject: J1 module demo plans? In-Reply-To: karen.kinnear@sun.com; Tue, 05 May 2009 09:41:20 EDT; <4A004200.1040305@sun.com> Message-ID: <20090505160015.3855D5591@eggemoggin.niobe.net> > Date: Tue, 05 May 2009 09:41:20 -0400 > From: karen.kinnear at sun.com > What are our current plans for a J1 demo? What have we gotten done > for that and what is left? The current plan for the J1 demo is to show the download-size and native-packaging benefits of Jigsaw on Ubuntu Linux. The main tasks are: (1) A quick-and-dirty "modularization" of the JDK, done by shuffling around the files from a regular JDK build rather than by rewriting the JDK makefiles (Mark) (2) Add whatever remaining run-time features are needed for (1), e.g., resources, native libraries, and possibly commands (Mark) (3) Write the packaging tool, jpkg, so that we can create .deb files from modules (Dalibor) (4) Upgrade javac to understand -modulepath and the new modularized output-directory format (Jon) (5) (Stretch goal) Upgrade javac to resolve dependences against a module library (Jon) We can have a credible demo without (5), we just won't be able to show how compile-time visibility checking works. At this point I'm pretty sure we won't need any further VM changes for this demo, but I could be wrong. The demo itself will go something like this: - Start with an Ubuntu box that has no Java installed on it. - Using Synaptic, install the "bootstrap" JRE from a nearby package repository. While selecting the package, compare its size to a currently-available full-JRE package. - Show that this minimal JRE can run the usual command-line "Hello, world!" program, packaged as an ordinary stand-alone class file rather than a module. - Show that it can't run even a trivial Swing application. - Install the Swing package, which in turn will cause one or more intermediate packages (e.g., AWT/2D) to be installed. - Show that a simple Swing application can be run. - Repeat this pattern as desired to show the installation of additional components (e.g., XML, JAX-WS, ...). - Show how to write and run the command-line "Hello, world!" program as a module. - Show how to write a simple Swing application as a module. We're not going to have any IDE support for the JSR 294 language constructs in time for J1, so this will be an Emacs-and-shell sort of demo. - Mark From Jonathan.Gibbons at Sun.COM Tue May 5 19:17:08 2009 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Tue, 05 May 2009 19:17:08 -0700 Subject: J1 module demo plans? In-Reply-To: <20090505160015.3855D5591@eggemoggin.niobe.net> References: <20090505160015.3855D5591@eggemoggin.niobe.net> Message-ID: <4A00F324.7040707@sun.com> Mark, How much of the Library interface does the Resolver need? As I understand it, you hope javac will work by providing an implementation of a Library. There are some methods, like Library.listClasses, that javac cannot reasonably implement, because of chicken and egg issues. If the Resolver requires that to be implemented, the elastic on the stretch goal just snapped. Also, the Library API is file-centric, in that the install method works on a classes/ directory. This is not going to work well with javac/JSR 199, since for the most part, javac works in terms of JavaFileObjects returned from a JavaFileManager, and does not assume a File based implementation. How do you envisage javac using the Jigsaw API? -- Jon Mark Reinhold wrote: >> Date: Tue, 05 May 2009 09:41:20 -0400 >> From: karen.kinnear at sun.com >> > > >> What are our current plans for a J1 demo? What have we gotten done >> for that and what is left? >> > > The current plan for the J1 demo is to show the download-size and > native-packaging benefits of Jigsaw on Ubuntu Linux. The main tasks > are: > > (1) A quick-and-dirty "modularization" of the JDK, done by shuffling > around the files from a regular JDK build rather than by rewriting > the JDK makefiles (Mark) > > (2) Add whatever remaining run-time features are needed for (1), e.g., > resources, native libraries, and possibly commands (Mark) > > (3) Write the packaging tool, jpkg, so that we can create .deb files > from modules (Dalibor) > > (4) Upgrade javac to understand -modulepath and the new modularized > output-directory format (Jon) > > (5) (Stretch goal) Upgrade javac to resolve dependences against a > module library (Jon) > > We can have a credible demo without (5), we just won't be able to show > how compile-time visibility checking works. > > At this point I'm pretty sure we won't need any further VM changes for > this demo, but I could be wrong. > > The demo itself will go something like this: > > - Start with an Ubuntu box that has no Java installed on it. > > - Using Synaptic, install the "bootstrap" JRE from a nearby package > repository. While selecting the package, compare its size to a > currently-available full-JRE package. > > - Show that this minimal JRE can run the usual command-line "Hello, > world!" program, packaged as an ordinary stand-alone class file > rather than a module. > > - Show that it can't run even a trivial Swing application. > > - Install the Swing package, which in turn will cause one or more > intermediate packages (e.g., AWT/2D) to be installed. > > - Show that a simple Swing application can be run. > > - Repeat this pattern as desired to show the installation of > additional components (e.g., XML, JAX-WS, ...). > > - Show how to write and run the command-line "Hello, world!" > program as a module. > > - Show how to write a simple Swing application as a module. > > We're not going to have any IDE support for the JSR 294 language > constructs in time for J1, so this will be an Emacs-and-shell sort > of demo. > > - Mark > From mr at sun.com Tue May 5 22:25:36 2009 From: mr at sun.com (mr at sun.com) Date: Wed, 06 May 2009 05:25:36 +0000 Subject: hg: jigsaw/jigsaw/jdk: Resources Message-ID: <20090506052556.25E99E0D2@hg.openjdk.java.net> Changeset: 0449e4f8376e Author: mr Date: 2009-05-05 22:23 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/0449e4f8376e Resources ! make/java/java/FILES_java.gmk ! src/share/classes/org/openjdk/jigsaw/Library.java ! src/share/classes/org/openjdk/jigsaw/Loader.java + src/share/classes/org/openjdk/jigsaw/Manifest.java ! src/share/classes/org/openjdk/jigsaw/SimpleLibrary.java ! src/share/classes/org/openjdk/jigsaw/cli/Librarian.java ! test/org/openjdk/jigsaw/MockLibrary.java ! test/org/openjdk/jigsaw/_Library.java ! test/org/openjdk/jigsaw/library.sh - test/org/openjdk/jigsaw/preinstall-setup.sh ! test/org/openjdk/jigsaw/preinstall.sh + test/org/openjdk/jigsaw/resource.sh ! test/org/openjdk/jigsaw/tester.sh From mr at sun.com Tue May 5 22:33:27 2009 From: mr at sun.com (Mark Reinhold) Date: Tue, 05 May 2009 22:33:27 -0700 Subject: Resources Message-ID: <20090506053327.1C698913F@callebaut.niobe.net> I've pushed code to support "resources", i.e., the getResource(String) and getResources(String) methods declared in java.lang.ClassLoader. >From the explanatory comment in the Loader class: The approach taken here is simply to discover resources at run time. Given a resource name we first search this loader's modules for that resource; we then search the loaders for every other context in this configuration. If we think of a Jigsaw configuration as a "better class path" then this isn't completely unreasonable, but it is meant to be temporary. An eventual fuller treatment of resources will treat them more like classes, resolving them statically, allowing them to be declared private to a module or public, and re-exporting them from one context to another when "requires public" is used. (Aside: I'm reminded once again how utterly horrible it is that the getResource{,s} methods return URLs. We really must define a better alternative, and one that doesn't expose paths into module libraries!) A related addition is the Manifest class, an instance of which describes the files that comprise a module for installation or packaging purposes. The Library install and preInstall methods now take manifest collections rather than bare classes directories. - Mark [1] http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/0449e4f8376e From mr at sun.com Wed May 6 09:19:12 2009 From: mr at sun.com (Mark Reinhold) Date: Wed, 06 May 2009 09:19:12 -0700 Subject: Refactoring the Library API Message-ID: <20090506161912.82CE75591@eggemoggin.niobe.net> The current Library class is definitely file-centric, and I fully expect that we'll need to refactor it, possibly dramatically, in order to hook up javac. The resolver currently needs just three of the Library methods: - List findModuleIds(String moduleName) - ModuleInfo readModuleInfo(ModuleId mid) - List listClasses(ModuleId mid, boolean all) My understanding from our previous discussions is that the first two are easy to implement in javac. The last, as you point out, is impossible. The Library.listClasses method is invoked only in phases 3 and 4 of the resolution algorithm. It won't be used by the alternate compile-time version of phase 3 (still to be written). It's used in phase 4 only to compute the set of packages defined in, and exported from, a context. The compiler can't provide a list of packages up-front either, so it seems that we'll also need an alternate version of phase 4 which, like the alternate phase 3, computes a compile-time approximation of the eventual run-time result. That approximation will most likely be, for each context, its set of remote supplier contexts; finding a class will involve walking the context graph. The eventual minimal "Library" API can declare just the findModuleIds and readModuleInfo methods. It probably makes sense to have that be an interface so that you can implement whatever JavaFileManager plumbing is needed in the compiler. We should think of a better name for it, too; maybe "ModuleInfoSet" or some such? These are nontrivial changes, and the elastic on the stretch goal might still have snapped, but at least we're making progress. - Mark From Jonathan.Gibbons at Sun.COM Wed May 6 09:50:25 2009 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Wed, 06 May 2009 09:50:25 -0700 Subject: Refactoring the Library API In-Reply-To: <20090506161912.82CE75591@eggemoggin.niobe.net> References: <20090506161912.82CE75591@eggemoggin.niobe.net> Message-ID: Mark, Thanks for the update. On the javac side, javac is at the point where it now computes a) the set of "root modules" (plural) for this compilation. This is effectively defined as the set of modules for all compilation units on the command line. b) the set of all modules on the various compilation paths, meaning whatever is the active combination of sourcepath, classpath and modulepath. This information is all available in javac's representation of module- info. I fully expect to have to convert that info into whatever module-info form is required by Jigsaw. --Note, I'll just convert it while accessing Jigsaw; I do not expect to change javac's internal representation for a number of design and bootstrapping reasons. --Note: annotations cannot be evaluated and made available at this stage in the compile pipeline. javac's expectation is that after interacting with Jigsaw it will be able to determine one or perhaps both of the following: i) a map identifying the visible modules for each module. ii) an ordering of the values in that map, such that the modules define a new sort of path to be used internally in javac. With a call back system and sufficient abstraction, it is possible that javac could determine the set of classes or packages in each module. However, that would be such a startup overhead for the compiler that while technically possible, I don't think it would be a good design choice for Jigsaw. It would be much better, IMO, if all the information necessary to evaluate a module dependency graph was directly available in the module-info files themselves. -- Jon On May 6, 2009, at 9:19 AM, Mark Reinhold wrote: > The current Library class is definitely file-centric, and I fully > expect > that we'll need to refactor it, possibly dramatically, in order to > hook > up javac. > > The resolver currently needs just three of the Library methods: > > - List findModuleIds(String moduleName) > > - ModuleInfo readModuleInfo(ModuleId mid) > > - List listClasses(ModuleId mid, boolean all) > > My understanding from our previous discussions is that the first two > are > easy to implement in javac. The last, as you point out, is > impossible. > > The Library.listClasses method is invoked only in phases 3 and 4 of > the > resolution algorithm. It won't be used by the alternate compile-time > version of phase 3 (still to be written). It's used in phase 4 only > to > compute the set of packages defined in, and exported from, a context. > > The compiler can't provide a list of packages up-front either, so it > seems that we'll also need an alternate version of phase 4 which, like > the alternate phase 3, computes a compile-time approximation of the > eventual run-time result. That approximation will most likely be, for > each context, its set of remote supplier contexts; finding a class > will > involve walking the context graph. > > The eventual minimal "Library" API can declare just the > findModuleIds and > readModuleInfo methods. It probably makes sense to have that be an > interface so that you can implement whatever JavaFileManager > plumbing is > needed in the compiler. We should think of a better name for it, too; > maybe "ModuleInfoSet" or some such? > > These are nontrivial changes, and the elastic on the stretch goal > might > still have snapped, but at least we're making progress. > > - Mark From mr at sun.com Thu May 7 12:50:36 2009 From: mr at sun.com (Mark Reinhold) Date: Thu, 07 May 2009 12:50:36 -0700 Subject: Refactoring the Library API In-Reply-To: jonathan.gibbons@sun.com; Wed, 06 May 2009 09:50:25 PDT; Message-ID: <20090507195036.7868F5E96@eggemoggin.niobe.net> > Date: Wed, 06 May 2009 09:50:25 -0700 > From: jonathan.gibbons at sun.com > On the javac side, javac is at the point where it now computes > > a) the set of "root modules" (plural) for this compilation. This is > effectively defined as the set of modules for all compilation units on > the command line. > > b) the set of all modules on the various compilation paths, meaning > whatever is the active combination of sourcepath, classpath and > modulepath. > > This information is all available in javac's representation of module- > info. I fully expect to have to convert that info into whatever > module-info form is required by Jigsaw. > > --Note, I'll just convert it while accessing Jigsaw; I do not expect to > change javac's internal representation for a number of design and > bootstrapping reasons. If you can implement the ModuleInfo interface then no conversion should be necessary, but I can believe there might be reasons why you can't or don't want to do that. > --Note: annotations cannot be evaluated and made available at this > stage in the compile pipeline. Understood. > javac's expectation is that after interacting with Jigsaw it will be > able to determine one or perhaps both of the following: > > i) a map identifying the visible modules for each module. > > ii) an ordering of the values in that map, such that the modules define > a new sort of path to be used internally in javac. Yes, this aligns with what I've been thinking. In fact it might be possible for me to hack up a simple but good-enough implementation fairly quickly. I'll look at that as soon as I finish rewhacking the launcher to understand modules. > With a call back system and sufficient abstraction, it is possible that > javac could determine the set of classes or packages in each > module. However, that would be such a startup overhead for the compiler > that while technically possible, I don't think it would be a good > design choice for Jigsaw. Agreed. > It would be much better, IMO, if all the > information necessary to evaluate a module dependency graph was > directly available in the module-info files themselves. What we have now appears to be enough to compute a decent compile-time approximation to the run-time context graph. We do, however, need some practical experience with it. If we need a better approximation, then I see only two ways to add actual package information to module-info class files: - Have the compiler record member packages in module-info.class. - Extend the module compilation-unit syntax and require developers to declare member packages. I see lots of problems with both alternatives. Let's hope that the simple approximation works well enough. - Mark From Jonathan.Gibbons at Sun.COM Thu May 7 13:28:54 2009 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Thu, 07 May 2009 13:28:54 -0700 Subject: Refactoring the Library API In-Reply-To: <20090507195036.7868F5E96@eggemoggin.niobe.net> References: <20090507195036.7868F5E96@eggemoggin.niobe.net> Message-ID: <4A034486.6060407@sun.com> Mark Reinhold wrote: > [snip] > >> It would be much better, IMO, if all the >> information necessary to evaluate a module dependency graph was >> directly available in the module-info files themselves. >> > > What we have now appears to be enough to compute a decent compile-time > approximation to the run-time context graph. We do, however, need some > practical experience with it. > > If we need a better approximation, then I see only two ways to add actual > package information to module-info class files: > > - Have the compiler record member packages in module-info.class. > > - Extend the module compilation-unit syntax and require developers > to declare member packages. > > I see lots of problems with both alternatives. Let's hope that the > simple approximation works well enough. > > - Mark > I don't like the idea of having to record package information in the module-info file. I would like to think we can identify Jigsaw "modifiers" to put in the requires declaration such that the resolver can create the correct module graph without requiring any knowledge of the packages in a module. Separately, I understand why you want to put package information for a module in the library. That can be done separately, at module installation time, and need not be part of the underlying module resolution algorithm. -- Jon From criscuolo at gmail.com Fri May 8 06:41:20 2009 From: criscuolo at gmail.com (=?ISO-8859-1?Q?Marcelo_Criscuolo_=28Ja=FA=29?=) Date: Fri, 8 May 2009 10:41:20 -0300 Subject: I'd like to contribute Message-ID: <68663b780905080641r7e070659i6505d59779dcebf5@mail.gmail.com> Hi everybody! I'm Marcelo, from Brazil. I've been working with Java (EE, SE and a little bit of ME) for the last 6 years or so. Since I started using Java I missed module features that could enable things like better component encapsulation and ease of configuration management (aka, get rid of Jar hell). Some time ago I came across JSRs 277 and 294 and then ended up knowing about project Jigsaw. Due to my personal interest on modularization (for the JDK and applications), I'd would like to contribute to this project. I've read the four blog entries pointed at the project homepage, so I think I know a little about what's going on. I'm an open source enthusiast, using Linux for 9 years and having Java as my main language for the last 6, as mentioned above. I'm a command-line heavy user, interested on operating systems and user of other programming languages like C. I hold a bachelor's degree on Computer Science and a MSc on Software Engineering. If you guys think it's appropriate, I can send more information about my background later. Now that I've introduced myself, do you think I could help somehow? Is there anything a newcomer could help on the project? Any other resources I could take a look? Any other readings? Thank you! Marcelo Criscuolo From brian at pontarelli.com Fri May 8 07:42:31 2009 From: brian at pontarelli.com (Brian Pontarelli) Date: Fri, 8 May 2009 08:42:31 -0600 Subject: Refactoring the Library API In-Reply-To: <4A034486.6060407@sun.com> References: <20090507195036.7868F5E96@eggemoggin.niobe.net> <4A034486.6060407@sun.com> Message-ID: On May 7, 2009, at 2:28 PM, Jonathan Gibbons wrote: > Mark Reinhold wrote: >> [snip] >> >>> It would be much better, IMO, if all the >>> information necessary to evaluate a module dependency graph was >>> directly available in the module-info files themselves. >>> >> >> What we have now appears to be enough to compute a decent compile- >> time >> approximation to the run-time context graph. We do, however, need >> some >> practical experience with it. >> >> If we need a better approximation, then I see only two ways to add >> actual >> package information to module-info class files: >> >> - Have the compiler record member packages in module-info.class. >> >> - Extend the module compilation-unit syntax and require developers >> to declare member packages. >> >> I see lots of problems with both alternatives. Let's hope that the >> simple approximation works well enough. >> >> - Mark >> > > I don't like the idea of having to record package information in the > module-info file. > I would like to think we can identify Jigsaw "modifiers" to put in > the requires > declaration such that the resolver can create the correct module > graph without > requiring any knowledge of the packages in a module. > > Separately, I understand why you want to put package information for > a module > in the library. That can be done separately, at module installation > time, and > need not be part of the underlying module resolution algorithm. > > -- Jon I'm pretty new to the list and the details of Jigsaw, but based on my knowledge of things and how I do this in Savant, I would generate the module-info file at release time (what you are calling installation time). The only artifact released would be the module JAR file, which could include all the necessary information to generate the module- info.class file, and the installation tool would generate it and release it as a sibling to the JAR (I would highly advise against pulling dependency information from inside the JAR files). That would allow full graph construction without incurring downloading overhead and not force developers to manage two separate pieces. I don't have full information about how Jigsaw defines and manages this information, but Savant uses the primary build file (project.xml) to generate an AMD file (artifact-meta-data) at release time. This is also part of releasing integration builds. -bp From mr at sun.com Fri May 8 08:25:03 2009 From: mr at sun.com (Mark Reinhold) Date: Fri, 08 May 2009 08:25:03 -0700 Subject: Refactoring the Library API In-Reply-To: jonathan.gibbons@sun.com; Thu, 07 May 2009 13:28:54 PDT; <4A034486.6060407@sun.com> Message-ID: <20090508152503.4F9505E2F@eggemoggin.niobe.net> > Date: Thu, 07 May 2009 13:28:54 -0700 > From: jonathan.gibbons at sun.com > I don't like the idea of having to record package information in the > module-info file. I would like to think we can identify Jigsaw > "modifiers" to put in the requires declaration such that the resolver > can create the correct module graph without requiring any knowledge of > the packages in a module. Agreed, and for the compiler's purposes I think we already have what we need in module-info files. > Separately, I understand why you want to put package information for a > module in the library. That can be done separately, at module > installation time, and need not be part of the underlying module > resolution algorithm. I think we're getting confused over what a "resolver" is, and I suspect it's because I've overloaded the term. If a resolver's job is just to compute a module-dependence graph (well, really a context graph) and choose a single version of each module, then that's phases 1 and 2 of what's currently in the Resolver class. Phases 3 and 4 might better be called "configuration", something that's done at install time but not at compile time. This suggests that I should refactor the Resolver class by splitting the last two phases out into a separate Configurator class. I'll try to do that when I revisit that code, likely next week. - Mark From mr at sun.com Fri May 8 08:52:02 2009 From: mr at sun.com (Mark Reinhold) Date: Fri, 08 May 2009 08:52:02 -0700 Subject: Refactoring the Library API In-Reply-To: brian@pontarelli.com; Fri, 08 May 2009 08:42:31 MDT; Message-ID: <20090508155202.DD1A85E2F@eggemoggin.niobe.net> > Date: Fri, 08 May 2009 08:42:31 -0600 > From: Brian Pontarelli > I'm pretty new to the list and the details of Jigsaw, but based on my > knowledge of things and how I do this in Savant, I would generate the > module-info file at release time (what you are calling installation > time). The only artifact released would be the module JAR file, which > could include all the necessary information to generate the module- > info.class file, and the installation tool would generate it and > release it as a sibling to the JAR (I would highly advise against > pulling dependency information from inside the JAR files). That would > allow full graph construction without incurring downloading overhead > and not force developers to manage two separate pieces. I don't have > full information about how Jigsaw defines and manages this > information, but Savant uses the primary build file (project.xml) to > generate an AMD file (artifact-meta-data) at release time. This is > also part of releasing integration builds. One of the primary goals of Jigsaw is to unify, as much as possible, the treatment of modules at compile time vs. run time. The Java language allows you to constrain the accessibility and visibility of interfaces, classes, and members, and the compile-time and run-time views of these constraints are pretty similar -- though not identical. Modules should be no different. So for Jigsaw defining modules is not a release-time activity; it's something done up front, along with all the other code. This way developers have a simple, uniform model of accessibility and visibility rather than one that's split across the compile, release, and run phases of the development cycle. - Mark From Jonathan.Gibbons at Sun.COM Fri May 8 11:04:18 2009 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Fri, 08 May 2009 11:04:18 -0700 Subject: Jigsaw API nit: VersionQuery and Comparator Message-ID: <4A047422.4040305@sun.com> Minor API comment: It's weird having VersionQuery implement Comparator. In principle, this means that different version query instances could provide different implementions of the comparator. It would make more sense for Library or some other "more-singleton" item to provide the one true comparator for the system, without having to cons up VersionQuerys just to get the Comparator functionality. -- Jon From mr at sun.com Fri May 8 12:06:50 2009 From: mr at sun.com (Mark Reinhold) Date: Fri, 08 May 2009 12:06:50 -0700 Subject: Jigsaw API nit: VersionQuery and Comparator In-Reply-To: jonathan.gibbons@sun.com; Fri, 08 May 2009 11:04:18 PDT; <4A047422.4040305@sun.com> Message-ID: <20090508190650.17C5C5E2F@eggemoggin.niobe.net> > Date: Fri, 08 May 2009 11:04:18 -0700 > From: jonathan.gibbons at sun.com > Minor API comment: > > It's weird having VersionQuery implement Comparator. In > principle, this means that different version query instances could > provide different implementions of the comparator. My thinking at the time was that this would allow a query also to express the desired version ordering ("ORDER BY" in SQL terms) in case you wanted something other than the natural order. That was at best a half-formed thought, however, and since I haven't seen an actual need for this yet I'm happy to remove it. > It would make more > sense for Library or some other "more-singleton" item to provide the one > true comparator for the system, without having to cons up VersionQuerys > just to get the Comparator functionality. The Version interface implements Comparable so that specific module systems can define whatever natural ordering is preferred. - Mark From Jonathan.Gibbons at Sun.COM Sun May 10 17:33:07 2009 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Sun, 10 May 2009 17:33:07 -0700 Subject: Jigsaw resolver error handling Message-ID: Mark, I'm sure this is not high on your list of priorities right now, but at some point we should think about how best to handle error handling, at least as far as the ModuleResolver is concerned. The current API leads me to think that you might be formatting English language messages. javac expects to be able to set the Locale used for messages. If nothing else, it would be convenient if in time we identified Jigsaw- specific subtypes of ConfigurationException such that javac could handle/respond to them directly. As an example, consider FileNotFoundException as a special subtype of IOException that is often handled differently, with customized error messages. I would like to take it one step further, and add custom arguments/fields for custom exception subtypes -- something JDK is woefully inadequate at doing. For example, FileNotFoundException should really make available the specific file in question, leaving clients to "guess" that the detail message may be the file name, although that behavior is not specified. In the modules world, I'm sure there will be an similar "ModuleNotFoundException". In this case, it would be useful to know the id of the module in question, and the context (module) within which the reference was found. javac will need this information in a reliable type safe way (no parsing of detail messages, please) so that it can tie individual errors back into the user's source code when necessary. Otherwise, we are going to be left with javac reporting a non-informative error, like "Error: cannot find module M at 2.0" with no context available at all. :-( -- Jon From mr at sun.com Mon May 11 13:36:55 2009 From: mr at sun.com (Mark Reinhold) Date: Mon, 11 May 2009 13:36:55 -0700 Subject: Jigsaw resolver error handling In-Reply-To: jonathan.gibbons@sun.com; Sun, 10 May 2009 17:33:07 PDT; Message-ID: <20090511203655.179625CE0@eggemoggin.niobe.net> > Date: Sun, 10 May 2009 17:33:07 -0700 > From: jonathan.gibbons at sun.com > I'm sure this is not high on your list of priorities right now, but at > some point we should think about how best to handle error handling, at > least as far as the ModuleResolver is concerned. Agreed. Right now "error handling" is limited to throwing pretty much just one exception type (ConfigurationException), and when the resolver fails the message is not at all helpful -- it gives you no clues as to why it failed. > The current API leads me to think that you might be formatting English > language messages. > > javac expects to be able to set the Locale used for messages. If > nothing else, it would be convenient if in time we identified Jigsaw- > specific subtypes of ConfigurationException such that javac could > handle/respond to them directly. Yes. I tend to prefer fairly fine-grained exceptions, in part exactly for this purpose. > As an example, consider FileNotFoundException as a special subtype of > IOException that is often handled differently, with customized error > messages. I would like to take it one step further, and add custom > arguments/fields for custom exception subtypes -- something JDK is > woefully inadequate at doing. For example, FileNotFoundException > should really make available the specific file in question, leaving > clients to "guess" that the detail message may be the file name, > although that behavior is not specified. > > In the modules world, I'm sure there will be an similar > "ModuleNotFoundException". In this case, it would be useful to know > the id of the module in question, and the context (module) within > which the reference was found. javac will need this information in a > reliable type safe way (no parsing of detail messages, please) so that > it can tie individual errors back into the user's source code when > necessary. Otherwise, we are going to be left with javac reporting a > non-informative error, like "Error: cannot find module M at 2.0" with no > context available at all. :-( Couldn't agree more. - Mark From Jonathan.Gibbons at Sun.COM Tue May 12 16:55:15 2009 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Tue, 12 May 2009 16:55:15 -0700 Subject: Jigsaw resolver error handling In-Reply-To: <20090511203655.179625CE0@eggemoggin.niobe.net> References: <20090511203655.179625CE0@eggemoggin.niobe.net> Message-ID: <4A0A0C63.70700@sun.com> We should also consider whether exceptions are the best vehicle to disseminate information about any issues that may arise. Specifically, one fault per diagnostic can potentially put the user into a cycle of "find the next error, fix it and try again" in the case of multiple errors. It would be better to be able to report as many errors as is reasonable, perhaps through a callback error reporting interface. -- Jon Mark Reinhold wrote: >> Date: Sun, 10 May 2009 17:33:07 -0700 >> From: jonathan.gibbons at sun.com >> > > >> I'm sure this is not high on your list of priorities right now, but at >> some point we should think about how best to handle error handling, at >> least as far as the ModuleResolver is concerned. >> > > Agreed. Right now "error handling" is limited to throwing pretty much > just one exception type (ConfigurationException), and when the resolver > fails the message is not at all helpful -- it gives you no clues as to > why it failed. > > >> The current API leads me to think that you might be formatting English >> language messages. >> >> javac expects to be able to set the Locale used for messages. If >> nothing else, it would be convenient if in time we identified Jigsaw- >> specific subtypes of ConfigurationException such that javac could >> handle/respond to them directly. >> > > Yes. I tend to prefer fairly fine-grained exceptions, in part exactly > for this purpose. > > >> As an example, consider FileNotFoundException as a special subtype of >> IOException that is often handled differently, with customized error >> messages. I would like to take it one step further, and add custom >> arguments/fields for custom exception subtypes -- something JDK is >> woefully inadequate at doing. For example, FileNotFoundException >> should really make available the specific file in question, leaving >> clients to "guess" that the detail message may be the file name, >> although that behavior is not specified. >> >> In the modules world, I'm sure there will be an similar >> "ModuleNotFoundException". In this case, it would be useful to know >> the id of the module in question, and the context (module) within >> which the reference was found. javac will need this information in a >> reliable type safe way (no parsing of detail messages, please) so that >> it can tie individual errors back into the user's source code when >> necessary. Otherwise, we are going to be left with javac reporting a >> non-informative error, like "Error: cannot find module M at 2.0" with no >> context available at all. :-( >> > > Couldn't agree more. > > - Mark > From mr at sun.com Wed May 13 08:20:57 2009 From: mr at sun.com (Mark Reinhold) Date: Wed, 13 May 2009 08:20:57 -0700 Subject: I'd like to contribute In-Reply-To: criscuolo@gmail.com; Fri, 08 May 2009 10:41:20 -0300; <68663b780905080641r7e070659i6505d59779dcebf5@mail.gmail.com> Message-ID: <20090513152057.C0C6E5CE0@eggemoggin.niobe.net> > Date: Fri, 08 May 2009 10:41:20 -0300 > From: Marcelo Criscuolo > Hi everybody! Hi! > ... > > Now that I've introduced myself, do you think I could help somehow? > Is there anything a newcomer could help on the project? Any other > resources I could take a look? Any other readings? We'll shortly be posting binary packages for people to play with, so a great way to get started would be to install those, use them, and send feedback. In the meantime you could read up on the work going on in the JSR 292 EG [1] and have a look at the current (somewhat sparse) javadoc [2]. There's also a fair bit of commentary in the code itself; see, e.g., the module-resolution algorithm [3]. - Mark [1] http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-observer [2] http://cr.openjdk.java.net/~mr/jigsaw/api [3] http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/file/tip/src/share/classes/org/openjdk/jigsaw/Resolver.java From Alex.Buckley at Sun.COM Wed May 13 09:56:20 2009 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Wed, 13 May 2009 09:56:20 -0700 Subject: I'd like to contribute In-Reply-To: <20090513152057.C0C6E5CE0@eggemoggin.niobe.net> References: <20090513152057.C0C6E5CE0@eggemoggin.niobe.net> Message-ID: <4A0AFBB4.9060706@sun.com> Mark Reinhold wrote: > the JSR 292 EG [1] and have a look at the current (somewhat sparse) or even the JSR 294 EG [1] :-) > [1] http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-observer Alex From jonathan.gibbons at sun.com Wed May 13 17:47:38 2009 From: jonathan.gibbons at sun.com (jonathan.gibbons at sun.com) Date: Thu, 14 May 2009 00:47:38 +0000 Subject: hg: jigsaw/jigsaw/langtools: initial javac support for -modulepath Message-ID: <20090514004741.D06FDED45@hg.openjdk.java.net> Changeset: 04dbeb7be155 Author: jjg Date: 2009-05-13 17:44 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/04dbeb7be155 initial javac support for -modulepath ! src/share/classes/com/sun/source/util/SimpleTreeVisitor.java ! src/share/classes/com/sun/tools/apt/main/Main.java ! src/share/classes/com/sun/tools/javac/code/Flags.java ! src/share/classes/com/sun/tools/javac/code/Scope.java ! src/share/classes/com/sun/tools/javac/code/Symbol.java ! src/share/classes/com/sun/tools/javac/code/Symtab.java ! src/share/classes/com/sun/tools/javac/comp/Attr.java ! src/share/classes/com/sun/tools/javac/comp/Enter.java ! src/share/classes/com/sun/tools/javac/comp/Flow.java ! src/share/classes/com/sun/tools/javac/comp/Lower.java + src/share/classes/com/sun/tools/javac/comp/Modules.java ! src/share/classes/com/sun/tools/javac/file/BaseFileObject.java ! src/share/classes/com/sun/tools/javac/file/JavacFileManager.java ! src/share/classes/com/sun/tools/javac/file/Paths.java ! src/share/classes/com/sun/tools/javac/file/RegularFileObject.java ! src/share/classes/com/sun/tools/javac/file/ZipArchive.java ! src/share/classes/com/sun/tools/javac/file/ZipFileIndexArchive.java ! src/share/classes/com/sun/tools/javac/jvm/ClassFile.java ! src/share/classes/com/sun/tools/javac/jvm/ClassReader.java ! src/share/classes/com/sun/tools/javac/jvm/ClassWriter.java ! src/share/classes/com/sun/tools/javac/main/JavaCompiler.java ! src/share/classes/com/sun/tools/javac/main/OptionName.java ! src/share/classes/com/sun/tools/javac/main/RecognizedOptions.java ! src/share/classes/com/sun/tools/javac/parser/JavacParser.java ! src/share/classes/com/sun/tools/javac/processing/JavacProcessingEnvironment.java ! src/share/classes/com/sun/tools/javac/resources/compiler.properties ! src/share/classes/com/sun/tools/javac/tree/JCTree.java ! src/share/classes/com/sun/tools/javac/tree/TreeInfo.java ! src/share/classes/com/sun/tools/javac/tree/TreeScanner.java ! src/share/classes/javax/lang/model/element/Element.java + src/share/classes/javax/lang/model/element/ModuleElement.java + src/share/classes/javax/lang/model/util/ModuleResolver.java + src/share/classes/javax/tools/ModuleFileManager.java ! src/share/classes/javax/tools/StandardLocation.java ! test/tools/javac/modules/ModuleAccessTest01.java ! test/tools/javac/modules/ModuleAttributeTest01.java ! test/tools/javac/modules/ModuleClassAttributeTest01.java ! test/tools/javac/modules/ModuleModifierTest01.java + test/tools/javac/modules/ModulePathTest01.java ! test/tools/javac/modules/ModulePermitsAttributeTest01.java ! test/tools/javac/modules/ModuleProvidesAttributeTest01.java ! test/tools/javac/modules/ModuleRequiresAttributeTest01.java + test/tools/javac/modules/ModuleResolverTest01.java + test/tools/javac/modules/ModuleResolverTest02.java + test/tools/javac/modules/ModuleResolverTest03.java + test/tools/javac/modules/ModuleResolverTest04.java ! test/tools/javac/modules/ModuleVersionTest01.java From Jonathan.Gibbons at Sun.COM Thu May 14 09:14:37 2009 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Thu, 14 May 2009 09:14:37 -0700 Subject: Module Resolver roots Message-ID: <4A0C436D.3010900@sun.com> Mark, The current module Resolver only takes a single root at a time. Unless I'm missing something, this makes it hard to resolve a group of mutually dependent modules. While the runtime may be able to have a limited view of a root (i.e. the module containing the main class), javac needs a somewhat more liberal view. javac currently determines the set of roots to be those module-info files given on the command line, together with the module-info files from the source path that are required by the other compilation units on the command line. It would help if the next round of Resolver API could take a set of roots, instead of just one. -- Jon From mr at sun.com Thu May 14 09:38:52 2009 From: mr at sun.com (Mark Reinhold) Date: Thu, 14 May 2009 09:38:52 -0700 Subject: Small VM change for booting from a module Message-ID: <20090514163852.62C2C5E96@eggemoggin.niobe.net> I've made a small VM change to support booting from a module. At the moment it's a bit like the old settable sun.boot.class.path, but I've put "module" in its name because it will, most likely, eventually have semantics beyond just setting the boot path. Webrev: http://cr.openjdk.java.net/~mr/vm-module-boot Karen -- Okay to push? - Mark From Karen.Kinnear at Sun.COM Thu May 14 10:37:11 2009 From: Karen.Kinnear at Sun.COM (Karen Kinnear) Date: Thu, 14 May 2009 13:37:11 -0400 Subject: Small VM change for booting from a module In-Reply-To: <20090514163852.62C2C5E96@eggemoggin.niobe.net> References: <20090514163852.62C2C5E96@eggemoggin.niobe.net> Message-ID: Mark, Thanks for doing this. A couple of minor questions/comments. It appears that meta_index_path and meta_index_dir are by default NULL. I'm not sure what happens on different OSs when classLoader.cpp: setup_meta_index calls fopen(meta_index_path, "r") and passes in a null. You might be safer initializing those values in os::set_boot_path. I'm assuming you want to deal with Class Data Sharing compatibility after J1. If it helps to model usage of the new property, today CDS requires that the bootclasspath contain individual jars, not lists of directories, to reduce having to check timestamp and file size for all .class files, and reduce startup costs. If you want to know what the vm is using for the sysclasspath you can run with -XX:+TraceClassLoading -XX: +Verbose, if you are running fastdebug. Also, there is a comment in ClassLoader.cpp you might want to check into: ... doesn't reorder the bootclasspath which would break java.lang.Package (see PackageInfo). thanks, Karen On May 14, 2009, at 12:38 PM, Mark Reinhold wrote: > I've made a small VM change to support booting from a module. At the > moment it's a bit like the old settable sun.boot.class.path, but I've > put "module" in its name because it will, most likely, eventually have > semantics beyond just setting the boot path. > > Webrev: http://cr.openjdk.java.net/~mr/vm-module-boot > > Karen -- Okay to push? > > - Mark From criscuolo at gmail.com Thu May 14 11:02:31 2009 From: criscuolo at gmail.com (=?ISO-8859-1?Q?Marcelo_Criscuolo_=28Ja=FA=29?=) Date: Thu, 14 May 2009 15:02:31 -0300 Subject: I'd like to contribute In-Reply-To: <20090513152057.C0C6E5CE0@eggemoggin.niobe.net> References: <68663b780905080641r7e070659i6505d59779dcebf5@mail.gmail.com> <20090513152057.C0C6E5CE0@eggemoggin.niobe.net> Message-ID: <68663b780905141102l183c46e3p56a2a4cbbf883c58@mail.gmail.com> Thanks guys! I'm going to check these resources! Marcelo On Wed, May 13, 2009 at 12:20 PM, Mark Reinhold wrote: >> Date: Fri, 08 May 2009 10:41:20 -0300 >> From: Marcelo Criscuolo > >> Hi everybody! > > Hi! > >> ... >> >> ? ? ? Now that I've introduced myself, do you think I could help somehow? >> Is there anything a newcomer could help on the project? Any other >> resources I could take a look? Any other readings? > > We'll shortly be posting binary packages for people to play with, so a > great way to get started would be to install those, use them, and send > feedback. ?In the meantime you could read up on the work going on in > the JSR 292 EG [1] and have a look at the current (somewhat sparse) > javadoc [2]. ?There's also a fair bit of commentary in the code itself; > see, e.g., the module-resolution algorithm [3]. > > - Mark > > > [1] http://cs.oswego.edu/mailman/listinfo/jsr294-modularity-observer > [2] http://cr.openjdk.java.net/~mr/jigsaw/api > [3] http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/file/tip/src/share/classes/org/openjdk/jigsaw/Resolver.java > From mr at sun.com Thu May 14 12:29:42 2009 From: mr at sun.com (Mark Reinhold) Date: Thu, 14 May 2009 12:29:42 -0700 Subject: Small VM change for booting from a module In-Reply-To: karen.kinnear@sun.com; Thu, 14 May 2009 13:37:11 EDT; Message-ID: <20090514192942.18A035E96@eggemoggin.niobe.net> > Date: Thu, 14 May 2009 13:37:11 -0400 > From: karen.kinnear at sun.com > It appears that meta_index_path and meta_index_dir are by default > NULL. I'm not sure what happens on different OSs when classLoader.cpp: > setup_meta_index calls fopen(meta_index_path, "r") and passes in a > null. You might be safer initializing those values in > os::set_boot_path. Good point, but I have no values to which to initialize them, and am unlikely to have such values even in the long run. I've instead added a check to ClassLoader::setup_meta_index so it doesn't open the file if meta_index_path is null. > I'm assuming you want to deal with Class Data Sharing compatibility > after J1. Yes! > If it helps to model usage of the new property, today CDS > requires that the bootclasspath contain individual jars, not lists of > directories, to reduce having to check timestamp and file size for all > .class files, and reduce startup costs. Thanks for pointing that out -- I'll keep it in mind. > If you want to know what the vm is using for the sysclasspath you can > run with -XX:+TraceClassLoading -XX:+Verbose, if you are running > fastdebug. Right, already found those. > Also, there is a comment in ClassLoader.cpp you might want to check > into: > ... doesn't reorder the bootclasspath which would break > java.lang.Package (see PackageInfo). Yes, I already ran across that one too. My present tiny change shouldn't affect this. It turns out that in order to support multi-module packages on the boot class path I'll eventually have to expose this method through a new JVM_ interface, but that's for another day. > Date: Thu, 14 May 2009 14:03:09 -0400 > From: karen.kinnear at sun.com > There does exist a -Xbootclasspath: flag already if you just want to use > that, or -Xbootclasspath/p: if you want to prepend or -Xbootclasspath/a: > to append, a different location. I thought about doing that as a short-term hack, but since the VM will ultimately have to do something module-specific I'd prefer to go with what I have now even if it is, at the moment, technically equivalent to -Xbootclasspath. Updated webrev: http://cr.openjdk.java.net/~mr/vm-module-boot/ (only change is to classLoader.cpp) Thanks, - Mark From Karen.Kinnear at Sun.COM Thu May 14 12:31:57 2009 From: Karen.Kinnear at Sun.COM (Karen Kinnear) Date: Thu, 14 May 2009 15:31:57 -0400 Subject: Small VM change for booting from a module In-Reply-To: <20090514192942.18A035E96@eggemoggin.niobe.net> References: <20090514192942.18A035E96@eggemoggin.niobe.net> Message-ID: <300127D7-0B3E-4B23-BF6B-D457EEB1599C@Sun.COM> Thanks Mark - looks good. thanks, Karen On May 14, 2009, at 3:29 PM, Mark Reinhold wrote: >> Date: Thu, 14 May 2009 13:37:11 -0400 >> From: karen.kinnear at sun.com > >> It appears that meta_index_path and meta_index_dir are by default >> NULL. I'm not sure what happens on different OSs when >> classLoader.cpp: >> setup_meta_index calls fopen(meta_index_path, "r") and passes in a >> null. You might be safer initializing those values in >> os::set_boot_path. > > Good point, but I have no values to which to initialize them, and am > unlikely to have such values even in the long run. I've instead added > a check to ClassLoader::setup_meta_index so it doesn't open the file > if > meta_index_path is null. > >> I'm assuming you want to deal with Class Data Sharing compatibility >> after J1. > > Yes! > >> If it helps to model usage of the new property, today CDS >> requires that the bootclasspath contain individual jars, not lists of >> directories, to reduce having to check timestamp and file size for >> all >> .class files, and reduce startup costs. > > Thanks for pointing that out -- I'll keep it in mind. > >> If you want to know what the vm is using for the sysclasspath you can >> run with -XX:+TraceClassLoading -XX:+Verbose, if you are running >> fastdebug. > > Right, already found those. > >> Also, there is a comment in ClassLoader.cpp you might want to check >> into: >> ... doesn't reorder the bootclasspath which would break >> java.lang.Package (see PackageInfo). > > Yes, I already ran across that one too. My present tiny change > shouldn't > affect this. It turns out that in order to support multi-module > packages > on the boot class path I'll eventually have to expose this method > through > a new JVM_ interface, but that's for another day. > >> Date: Thu, 14 May 2009 14:03:09 -0400 >> From: karen.kinnear at sun.com > >> There does exist a -Xbootclasspath: flag already if you just want >> to use >> that, or -Xbootclasspath/p: if you want to prepend or - >> Xbootclasspath/a: >> to append, a different location. > > I thought about doing that as a short-term hack, but since the VM will > ultimately have to do something module-specific I'd prefer to go with > what I have now even if it is, at the moment, technically equivalent > to > -Xbootclasspath. > > Updated webrev: http://cr.openjdk.java.net/~mr/vm-module-boot/ > (only change is to classLoader.cpp) > > Thanks, > - Mark From Jonathan.Gibbons at Sun.COM Thu May 14 13:50:44 2009 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Thu, 14 May 2009 13:50:44 -0700 Subject: module-info platform dependencies Message-ID: <4A0C8424.90106@sun.com> Mark, Could you (briefly) outline your current ideas regarding platform specification in the module-info file. You alluded to some ideas when I saw you in your office earlier this week. If you have a tangible proposal, I'll see what I can do to accomodate it in javac. -- Jon From mr at sun.com Thu May 14 14:50:59 2009 From: mr at sun.com (mr at sun.com) Date: Thu, 14 May 2009 21:50:59 +0000 Subject: hg: jigsaw/jigsaw/hotspot: 2 new changesets Message-ID: <20090514215106.194E9EE33@hg.openjdk.java.net> Changeset: b195667f639a Author: mr Date: 2009-05-14 14:43 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/b195667f639a Boot from a launcher-specified module ! src/share/vm/classfile/classLoader.cpp ! src/share/vm/runtime/arguments.cpp ! src/share/vm/runtime/arguments.hpp ! src/share/vm/runtime/os.cpp Changeset: 403a1b93874f Author: mr Date: 2009-05-14 14:48 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/403a1b93874f Merge From mr at sun.com Thu May 14 14:52:10 2009 From: mr at sun.com (mr at sun.com) Date: Thu, 14 May 2009 21:52:10 +0000 Subject: hg: jigsaw/jigsaw/jdk: 7 new changesets Message-ID: <20090514215342.8A5C7EE34@hg.openjdk.java.net> Changeset: 5142727e7f1f Author: mr Date: 2009-05-06 14:11 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/5142727e7f1f Stub out Loader methods that still need to be re-implemented ! src/share/classes/org/openjdk/jigsaw/Loader.java Changeset: 4a8540db9810 Author: mr Date: 2009-05-14 14:33 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/4a8540db9810 Fix ClassInfo corner case ! src/share/classes/org/openjdk/jigsaw/ClassInfo.java Changeset: 5724c7f54fd9 Author: mr Date: 2009-05-14 14:36 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/5724c7f54fd9 Install all class files found, ignoring Module attributes ! src/share/classes/org/openjdk/jigsaw/SimpleLibrary.java Changeset: 837855326a7f Author: mr Date: 2009-05-14 14:36 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/837855326a7f Library delegation ! src/share/classes/org/openjdk/jigsaw/Library.java ! src/share/classes/org/openjdk/jigsaw/SimpleLibrary.java ! src/share/classes/org/openjdk/jigsaw/cli/Librarian.java ! test/org/openjdk/jigsaw/MockLibrary.java ! test/org/openjdk/jigsaw/_Library.java Changeset: 4e192f1b952c Author: mr Date: 2009-05-14 14:36 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/4e192f1b952c One big JDK module, with default configuration-time dependence ! make/java/java/FILES_java.gmk + make/modules/Makefile ! src/share/classes/java/lang/ClassLoader.java ! src/share/classes/java/lang/module/ModuleId.java ! src/share/classes/java/lang/module/ModuleInfoReader.java + src/share/classes/org/openjdk/jigsaw/BootLoader.java ! src/share/classes/org/openjdk/jigsaw/Configuration.java ! src/share/classes/org/openjdk/jigsaw/Context.java - src/share/classes/org/openjdk/jigsaw/KernelLoader.java ! src/share/classes/org/openjdk/jigsaw/Loader.java ! src/share/classes/org/openjdk/jigsaw/LoaderPool.java + src/share/classes/org/openjdk/jigsaw/Platform.java ! src/share/classes/org/openjdk/jigsaw/Resolver.java ! src/share/classes/org/openjdk/jigsaw/SimpleLibrary.java ! src/share/classes/org/openjdk/jigsaw/cli/Librarian.java + src/share/modules/jdk/module-info.java ! test/org/openjdk/jigsaw/_Library.java ! test/org/openjdk/jigsaw/_Resolver.java ! test/org/openjdk/jigsaw/cli/jmod-basic.sh ! test/org/openjdk/jigsaw/preinstall.sh ! test/org/openjdk/jigsaw/tester.sh Changeset: c9831eff51ab Author: mr Date: 2009-05-14 14:36 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/c9831eff51ab Upgrade to the module-path-enabled javac ! make/modules/Makefile ! src/share/classes/org/openjdk/jigsaw/Manifest.java ! src/share/classes/org/openjdk/jigsaw/SimpleLibrary.java - test/java/lang/module/Main.java ! test/java/lang/module/_ModuleInfoReader.java + test/java/lang/module/module-info-reader.sh - test/java/lang/module/module-info.java ! test/org/openjdk/jigsaw/_Library.java ! test/org/openjdk/jigsaw/cli/jmod-basic.sh ! test/org/openjdk/jigsaw/library.sh ! test/org/openjdk/jigsaw/preinstall.sh ! test/org/openjdk/jigsaw/resolver.sh ! test/org/openjdk/jigsaw/resource.sh ! test/org/openjdk/jigsaw/tester.sh Changeset: 62a178a8f468 Author: mr Date: 2009-05-14 14:36 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/62a178a8f468 Modularize the JRE launch sequence ! make/Makefile ! make/java/java/FILES_java.gmk ! make/modules/Makefile ! src/share/bin/emessages.h ! src/share/bin/java.c ! src/share/bin/java.h ! src/share/classes/java/lang/ClassLoader.java ! src/share/classes/java/lang/module/ModuleClassLoader.java ! src/share/classes/java/lang/module/ModuleSystem.java + src/share/classes/org/openjdk/jigsaw/Hi.java ! src/share/classes/org/openjdk/jigsaw/Launcher.java ! src/share/classes/org/openjdk/jigsaw/Loader.java ! src/share/classes/org/openjdk/jigsaw/LoaderPool.java ! src/share/classes/sun/launcher/LauncherHelper.java ! src/share/modules/jdk/module-info.java ! test/org/openjdk/jigsaw/_Resolver.java ! test/org/openjdk/jigsaw/preinstall.sh ! test/org/openjdk/jigsaw/resource.sh ! test/org/openjdk/jigsaw/tester.sh From mr at sun.com Thu May 14 14:55:29 2009 From: mr at sun.com (Mark Reinhold) Date: Thu, 14 May 2009 14:55:29 -0700 Subject: Library delegation, the big JDK module, and modular launching Message-ID: <20090514215529.26E505E96@eggemoggin.niobe.net> I just flushed my Mercurial patch queues into the Jigsaw forest. Herewith a summary of what's new. Library delegation A module library can now have a parent library. Each module in the parent that is not also present in the child appears to be in the child, recursively. The Library find and read methods will consider the parent if something is not found in the child, but configuration operations only affect the child. In a related change I removed the Library.visitModules method, which wasn't much used and didn't make much sense in the presence of library delegation. There's instead now a gatherModuleIds method which will return all module ids from a library as well as its parent, if any. When creating a library with the jmod command you can specify its parent with the -P option. If you don't use -P then the parent will default to the JDK module. You can use -N to specify a null parent. (Side note: I figured all along that we'd need this; I was trying to defer it to later but it greatly simplifies testing, so I did it now.) The big JDK module At last! The final phase of the build now creates the jdk at 7-ea module in build/lib/modules, by copying files from build/classes. Classes in this module are loaded by a specialized Jigsaw loader which invokes the VM's bootstrap loader via the new j.l.ClassLoader.findBootClass method. During module configuration, if a module doesn't declare a dependence upon a "jdk" or "jdk.*" module then a dependence upon "jdk at 7-ea" is automagically added. (Hmm, perhaps we could call this the "Miranda" module ...) This is temporary: Ultimately the compiler should add such default dependences, and they should be upon the more abstract "java at 7" module, which the jdk module would provide, but this works for now. This changeset also includes some basic performance work, to cache ModuleId hash codes and to use identity rather than value equality in the Context and Configuration classes. Modular launch sequence This was the fun one. I've hacked the launcher (src/share/bin/java.c) and related logic in j.l.ClassLoader and elsewhere to be able to launch a root module by name, invoking the main method of the class declared in its module-info file. To launch a module you just say: $ java [-L ] -m When the JRE is started up this way the "system" class loader is just the Jigsaw loader for the named module, which -- due to the default library parent and dependence on the jdk at 7-ea module -- delegates to the Jigsaw boot loader to find classes formerly found in rt.jar. (I do have to admit that routing around the old sun.misc.Launcher class and the innate complexity of URLClassLoader and friends was a distinct pleasure.) Of course I've also upgraded all the runtime code and tests to use Jon's new module-path-aware javac (thanks!). (Side note: I heavily used Mercurial patch queues for the first time while working on this set of changes. Highly recommended!) My next task will be to simplify the Resolver interface for Jon, and then it'll be time to start slicing up the big JDK module. Onward! - Mark From mr at sun.com Thu May 14 21:20:45 2009 From: mr at sun.com (Mark Reinhold) Date: Thu, 14 May 2009 21:20:45 -0700 Subject: module-info platform dependencies In-Reply-To: jonathan.gibbons@sun.com; Thu, 14 May 2009 13:50:44 PDT; <4A0C8424.90106@sun.com> Message-ID: <20090515042045.E534A913F@callebaut.niobe.net> > Date: Thu, 14 May 2009 13:50:44 -0700 > From: jonathan.gibbons at sun.com > Could you (briefly) outline your current ideas regarding platform > specification in the module-info file. You alluded to some ideas when I > saw you in your office earlier this week. If you have a tangible > proposal, I'll see what I can do to accomodate it in javac. Sure. The basic idea is that the compiler can ask Jigsaw whether a particular dependence is upon a known, available platform or profile module, and it can also ask what the default is. If a module-info file declares at least one dependence upon a platform module then the compiler does nothing. If it does not declare such a dependence then the compiler inserts a dependence upon the default. The org.openjdk.jigsaw.Platform class already contains a method to tell whether a module name is that of a platform module; it would be trivial to add another to return a module-id query for the default. (The present addPlatformDependenceDefault method isn't really suitable; it assumes that ModuleInfo objects are mutable, which they shouldn't be but are temporarily so that the current configuration-time defaulting mechanism can work.) So from the compiler's point of view the API would be something like: public class org.openjdk.jigsaw.Platform { boolean isPlatformModuleName(String mn); ModuleIdQuery defaultPlatformModule(); } There is a latent security issue here -- we need to ensure that platform modules really are, well, platform modules. That's a problem for the runtime code, however, not the compiler. - Mark From Jonathan.Gibbons at Sun.COM Thu May 14 21:37:24 2009 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Thu, 14 May 2009 21:37:24 -0700 Subject: module-info platform dependencies In-Reply-To: <20090515042045.E534A913F@callebaut.niobe.net> References: <20090515042045.E534A913F@callebaut.niobe.net> Message-ID: <3F8CCC02-8403-4BFA-8404-229C6DA33719@sun.com> On May 14, 2009, at 9:20 PM, Mark Reinhold wrote: >> Date: Thu, 14 May 2009 13:50:44 -0700 >> From: jonathan.gibbons at sun.com > >> Could you (briefly) outline your current ideas regarding platform >> specification in the module-info file. You alluded to some ideas >> when I >> saw you in your office earlier this week. If you have a tangible >> proposal, I'll see what I can do to accomodate it in javac. > > Sure. > > The basic idea is that the compiler can ask Jigsaw whether a > particular > dependence is upon a known, available platform or profile module, > and it > can also ask what the default is. If a module-info file declares at > least one dependence upon a platform module then the compiler does > nothing. If it does not declare such a dependence then the compiler > inserts a dependence upon the default. > > The org.openjdk.jigsaw.Platform class already contains a method to > tell > whether a module name is that of a platform module; it would be > trivial > to add another to return a module-id query for the default. (The > present > addPlatformDependenceDefault method isn't really suitable; it assumes > that ModuleInfo objects are mutable, which they shouldn't be but are > temporarily so that the current configuration-time defaulting > mechanism > can work.) > > So from the compiler's point of view the API would be something like: > > public class org.openjdk.jigsaw.Platform { > boolean isPlatformModuleName(String mn); > ModuleIdQuery defaultPlatformModule(); > } > > There is a latent security issue here -- we need to ensure that > platform > modules really are, well, platform modules. That's a problem for the > runtime code, however, not the compiler. > > - Mark Thanks. There is something of a problem for the compiler as well as the runtime, in that the compiler has expectations of the platform that must be satisfied. It must have a java.lang package (otherwise an immediate fatal error occurs) and within java.lang, there must be Striing, Object, and other obvious basis classes, all with the right signatures. -- Jon From Jonathan.Gibbons at Sun.COM Thu May 14 22:13:49 2009 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Thu, 14 May 2009 22:13:49 -0700 Subject: module-info platform dependencies In-Reply-To: <20090515042045.E534A913F@callebaut.niobe.net> References: <20090515042045.E534A913F@callebaut.niobe.net> Message-ID: <137B6CB4-3237-421C-93AE-2AF5507AA216@sun.com> On May 14, 2009, at 9:20 PM, Mark Reinhold wrote: >> Date: Thu, 14 May 2009 13:50:44 -0700 >> From: jonathan.gibbons at sun.com > >> Could you (briefly) outline your current ideas regarding platform >> specification in the module-info file. You alluded to some ideas >> when I >> saw you in your office earlier this week. If you have a tangible >> proposal, I'll see what I can do to accomodate it in javac. > > Sure. > > The basic idea is that the compiler can ask Jigsaw whether a > particular > dependence is upon a known, available platform or profile module, > and it > can also ask what the default is. If a module-info file declares at > least one dependence upon a platform module then the compiler does > nothing. If it does not declare such a dependence then the compiler > inserts a dependence upon the default. > (snip) > - Mark It's not clear to me why the compiler should be doing this and/or its not clear to me what the right policy is here. It's not clear the compiler should be doing this because so much else is being deferred to the module system, why is this piece of analysis coming back into the compiler? I'm not saying it shouldn't; I'm just saying I don't yet understand why. One reason it may be more of a compiler function is future interaction with the compiler's -source and -target flags. Although rarely used correctly, -target should normally be used in conjunction with -Xbootclasspath. In the JDK 8 timeframe, when it may make sense to compile for JDK 7 or JDK8, the -target switch might interact with and/or influence the default of the platform dependence. If the compiler were to add a default, would you expect it to get compiled into the class file? It also seems to me that a case can be made for the module system to analyze the need for a default for a group of modules, in that if the set of leaf modules do not unambiguously specify a default, then the module system should specify one for the group. But there should not be a need for anyone (such as the compiler) to insert additional dependences into all modules that don't specify a platform when those modules will have an indirect dependence by virtue of the transitive closure of their own (explicit) dependencies. That sort of group wide analysis belongs in the module resolver/system, not in the compiler. -- Jon From mr at sun.com Fri May 15 08:44:06 2009 From: mr at sun.com (Mark Reinhold) Date: Fri, 15 May 2009 08:44:06 -0700 Subject: module-info platform dependencies In-Reply-To: jonathan.gibbons@sun.com; Thu, 14 May 2009 22:13:49 PDT; <137B6CB4-3237-421C-93AE-2AF5507AA216@sun.com> Message-ID: <20090515154406.850295E97@eggemoggin.niobe.net> > Date: Thu, 14 May 2009 22:13:49 -0700 > From: jonathan.gibbons at sun.com > On May 14, 2009, at 9:20 PM, Mark Reinhold wrote: >> The basic idea is that the compiler can ask Jigsaw whether a >> particular dependence is upon a known, available platform or profile >> module, and it can also ask what the default is. If a module-info >> file declares at least one dependence upon a platform module then the >> compiler does nothing. If it does not declare such a dependence then >> the compiler inserts a dependence upon the default. >> > It's not clear to me why the compiler should be doing this and/or its > not clear to me what the right policy is here. > > It's not clear the compiler should be doing this because so much else > is being deferred to the module system, why is this piece of analysis > coming back into the compiler? I'm not saying it shouldn't; I'm just > saying I don't yet understand why. It's a fairly trivial analysis on the compiler's part, and it is in this case deferring to the module system, to identify platform modules and to specify the default. > One reason it may be more of a compiler function is future interaction > with the compiler's -source and -target flags. Although rarely used > correctly, -target should normally be used in conjunction with > -Xbootclasspath. In the JDK 8 timeframe, when it may make sense to > compile for JDK 7 or JDK8, the -target switch might interact with > and/or influence the default of the platform dependence. Exactly. When you compile a module-info file with JDK 8 using -target 7 then the platform dependence would default to, say, jdk@>=7. This reminds me of a related issue which we've discussed but not yet settled: For dependences upon platform modules, at least, and possibly in all cases, javac should probably be asking the module system for the oldest available module satisfying the dependence's version constraint, rather than the newest. > If the compiler were to add a default, would you expect it to get > compiled into the class file? Into the module-info class file? Yes. Into other class files? No. > It also seems to me that a case can be made for the module system to > analyze the need for a default for a group of modules, in that if the > set of leaf modules do not unambiguously specify a default, then the > module system should specify one for the group. But there should not be > a need for anyone (such as the compiler) to insert additional > dependences into all modules that don't specify a platform when those > modules will have an indirect dependence by virtue of the transitive > closure of their own (explicit) dependencies. A module that does not declare a direct dependence upon a platform module will only have an indirect dependence upon a platform module if there is a transitive path of public dependences from that module to a platform module. I don't expect that to be the case, i.e., I expect people will generally write module foo @ 1 { requires jdk @ >=7; } rather than module foo @ 1 { requires public jdk @ >=7; } when declaring a specific platform dependence. > That sort of group wide > analysis belongs in the module resolver/system, not in the compiler. I'm not asking that the compiler do any sort of group-wide analysis, and I completely agree that such analyses are best done by the module system. If the compiler inserts default platform dependences as I've suggested, by doing a simple local analysis of each module, then the group-wide analysis you describe above will be performed by the current resolution algorithm, without change. - Mark From Jonathan.Gibbons at Sun.COM Fri May 15 08:52:49 2009 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Fri, 15 May 2009 08:52:49 -0700 Subject: module-info platform dependencies In-Reply-To: <20090515154406.850295E97@eggemoggin.niobe.net> References: <20090515154406.850295E97@eggemoggin.niobe.net> Message-ID: On May 15, 2009, at 8:44 AM, Mark Reinhold wrote: >> Date: Thu, 14 May 2009 22:13:49 -0700 >> From: jonathan.gibbons at sun.com > >> On May 14, 2009, at 9:20 PM, Mark Reinhold wrote: >>> The basic idea is that the compiler can ask Jigsaw whether a >>> particular dependence is upon a known, available platform or profile >>> module, and it can also ask what the default is. If a module-info >>> file declares at least one dependence upon a platform module then >>> the >>> compiler does nothing. If it does not declare such a dependence >>> then >>> the compiler inserts a dependence upon the default. >>> >> It's not clear to me why the compiler should be doing this and/or its >> not clear to me what the right policy is here. >> >> It's not clear the compiler should be doing this because so much else >> is being deferred to the module system, why is this piece of analysis >> coming back into the compiler? I'm not saying it shouldn't; I'm just >> saying I don't yet understand why. > > It's a fairly trivial analysis on the compiler's part, and it is in > this > case deferring to the module system, to identify platform modules > and to > specify the default. > >> One reason it may be more of a compiler function is future >> interaction >> with the compiler's -source and -target flags. Although rarely used >> correctly, -target should normally be used in conjunction with >> -Xbootclasspath. In the JDK 8 timeframe, when it may make sense to >> compile for JDK 7 or JDK8, the -target switch might interact with >> and/or influence the default of the platform dependence. > > Exactly. When you compile a module-info file with JDK 8 using - > target 7 > then the platform dependence would default to, say, jdk@>=7. > > This reminds me of a related issue which we've discussed but not yet > settled: For dependences upon platform modules, at least, and possibly > in all cases, javac should probably be asking the module system for > the > oldest available module satisfying the dependence's version > constraint, > rather than the newest. > >> If the compiler were to add a default, would you expect it to get >> compiled into the class file? > > Into the module-info class file? Yes. Into other class files? No. > >> It also seems to me that a case can be made for the module system to >> analyze the need for a default for a group of modules, in that if the >> set of leaf modules do not unambiguously specify a default, then the >> module system should specify one for the group. But there should >> not be >> a need for anyone (such as the compiler) to insert additional >> dependences into all modules that don't specify a platform when those >> modules will have an indirect dependence by virtue of the transitive >> closure of their own (explicit) dependencies. > > A module that does not declare a direct dependence upon a platform > module > will only have an indirect dependence upon a platform module if > there is > a transitive path of public dependences from that module to a platform > module. I don't expect that to be the case, i.e., I expect people > will > generally write > > module foo @ 1 { > requires jdk @ >=7; > } > > rather than > > module foo @ 1 { > requires public jdk @ >=7; > } > > when declaring a specific platform dependence. > >> That sort of group wide >> analysis belongs in the module resolver/system, not in the compiler. > > I'm not asking that the compiler do any sort of group-wide analysis, > and > I completely agree that such analyses are best done by the module > system. > > If the compiler inserts default platform dependences as I've > suggested, > by doing a simple local analysis of each module, then the group-wide > analysis you describe above will be performed by the current > resolution > algorithm, without change. > > - Mark So can you give me a method that maps a target number into an equivalent version query, or do I hard-code ("jdk@>=" + targetValue) into the compiler? "jdk" seems like the wrong name to use eventually. If you're writing it into the module-class file, we don't want to constrain apps to run on a large platform. But, it'll do for now. -- Jon From mr at sun.com Fri May 15 09:14:56 2009 From: mr at sun.com (Mark Reinhold) Date: Fri, 15 May 2009 09:14:56 -0700 Subject: module-info platform dependencies In-Reply-To: jonathan.gibbons@sun.com; Fri, 15 May 2009 08:52:49 PDT; Message-ID: <20090515161456.A66155E97@eggemoggin.niobe.net> > Date: Fri, 15 May 2009 08:52:49 -0700 > From: jonathan.gibbons at sun.com > So can you give me a method that maps a target number into an > equivalent version query, or do I hard-code ("jdk@>=" + targetValue) > into the compiler? The former, of course. How about this: public class org.openjdk.jigsaw.Platform { public static boolean isPlatformModuleName(String mn); public static ModuleIdQuery defaultPlatformModule(int target); } > "jdk" seems like the wrong name to use eventually. If you're writing > it into the module-class file, we don't want to constrain apps to run > on a large platform. But, it'll do for now. Actually, "jdk" is the wrong name because it refers to an implementation of the Java platform rather than the platform itself. The default should really be "java@>=7", but I haven't implemented the "provides" feature yet, so we can't do that yet. I do think that the default should be upon the whole platform. That's effectively what it is today. If you want something smaller then you can specify it, or maybe use a tool which can analyze your class dependences, identify the smallest suitable platform module, and insert that into your module-info source file. - Mark From Jonathan.Gibbons at Sun.COM Fri May 15 09:21:53 2009 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Fri, 15 May 2009 09:21:53 -0700 Subject: module-info platform dependencies In-Reply-To: <20090515161456.A66155E97@eggemoggin.niobe.net> References: <20090515161456.A66155E97@eggemoggin.niobe.net> Message-ID: On May 15, 2009, at 9:14 AM, Mark Reinhold wrote: >> Date: Fri, 15 May 2009 08:52:49 -0700 >> From: jonathan.gibbons at sun.com > >> So can you give me a method that maps a target number into an >> equivalent version query, or do I hard-code ("jdk@>=" + targetValue) >> into the compiler? > > The former, of course. How about this: > > public class org.openjdk.jigsaw.Platform { > public static boolean isPlatformModuleName(String mn); > public static ModuleIdQuery defaultPlatformModule(int target); > } Either that (int target) or a nice friendly type-safe enum. > > >> "jdk" seems like the wrong name to use eventually. If you're writing >> it into the module-class file, we don't want to constrain apps to run >> on a large platform. But, it'll do for now. > > Actually, "jdk" is the wrong name because it refers to an > implementation > of the Java platform rather than the platform itself. The default > should > really be "java@>=7", but I haven't implemented the "provides" feature > yet, so we can't do that yet. > > I do think that the default should be upon the whole platform. That's > effectively what it is today. If you want something smaller then > you can > specify it, or maybe use a tool which can analyze your class > dependences, > identify the smallest suitable platform module, and insert that into > your > module-info source file. The default today is that if I compile something like Hello World, it will run on any version of the platform, even the smallest. (Well, perhaps not JavaCard.) > > > - Mark From mr at sun.com Fri May 15 09:34:53 2009 From: mr at sun.com (Mark Reinhold) Date: Fri, 15 May 2009 09:34:53 -0700 Subject: module-info platform dependencies In-Reply-To: jonathan.gibbons@sun.com; Fri, 15 May 2009 09:21:53 PDT; Message-ID: <20090515163453.4D15B5E97@eggemoggin.niobe.net> > Date: Fri, 15 May 2009 09:21:53 -0700 > From: jonathan.gibbons at sun.com > On May 15, 2009, at 9:14 AM, Mark Reinhold wrote: >> The former, of course. How about this: >> >> public class org.openjdk.jigsaw.Platform { >> public static boolean isPlatformModuleName(String mn); >> public static ModuleIdQuery defaultPlatformModule(int target); >> } > > Either that (int target) or a nice friendly type-safe enum. Okay. public class org.openjdk.jigsaw.Platform { public static boolean isPlatformModuleName(String mn); public static enum Target { SEVEN; public static boolean isKnown(int target); public static Target valueOf(int target); } public static ModuleIdQuery defaultPlatformModule(Target t); } >> I do think that the default should be upon the whole platform. That's >> effectively what it is today. If you want something smaller then you >> can specify it, or maybe use a tool which can analyze your class >> dependences, identify the smallest suitable platform module, and >> insert that into your module-info source file. > > The default today is that if I compile something like Hello World, it > will run on any version of the platform, even the smallest. (Well, > perhaps not JavaCard.) Ah, I see what you mean. Need to think about this more, but offhand it still seems like more of a tooling issue, kind of like minimizing import declarations. Minimization of platform dependences could also be done much later, as part of module packaging. - Mark From Jonathan.Gibbons at Sun.COM Fri May 15 09:54:43 2009 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Fri, 15 May 2009 09:54:43 -0700 Subject: module-info platform dependencies In-Reply-To: <20090515163453.4D15B5E97@eggemoggin.niobe.net> References: <20090515163453.4D15B5E97@eggemoggin.niobe.net> Message-ID: <30BCF629-50EB-49E5-838F-4C5C016E38BE@sun.com> > >>> I do think that the default should be upon the whole platform. >>> That's >>> effectively what it is today. If you want something smaller then >>> you >>> can specify it, or maybe use a tool which can analyze your class >>> dependences, identify the smallest suitable platform module, and >>> insert that into your module-info source file. >> >> The default today is that if I compile something like Hello World, it >> will run on any version of the platform, even the smallest. (Well, >> perhaps not JavaCard.) > > Ah, I see what you mean. Need to think about this more, but offhand > it > still seems like more of a tooling issue, kind of like minimizing > import > declarations. Minimization of platform dependences could also be done > much later, as part of module packaging. > > - Mark Right now, within source code, you only get "java.lang" for free. If you need to use anything else in the platform, you have to ask for it. (i.e. import it). The analogy for modules would be that you get some minimal module by default (e.g. java.lang, java.io, java.net, java.util) and that if you want more (e.g, AWT, Swing, javac-API, etc) you have to ask for it. (i.e. require it). Having people declare everything non-basic they need is a good thing, especially for documentation purposes, and for making it clear to programmers what the footprint of their app will likely be. And, we can likely rely on tooling to help out, just like we do for import management Or, if people want the whole platform (the module equivalent of import p.*;) we can give them a virtual module that includes everything. -- Jon From jonathan.gibbons at sun.com Fri May 15 14:49:50 2009 From: jonathan.gibbons at sun.com (jonathan.gibbons at sun.com) Date: Fri, 15 May 2009 21:49:50 +0000 Subject: hg: jigsaw/jigsaw/langtools: remove debug prints; minor cosmetic cleanup Message-ID: <20090515214953.897C1E063@hg.openjdk.java.net> Changeset: 3a9b857151d6 Author: jjg Date: 2009-05-15 14:47 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/3a9b857151d6 remove debug prints; minor cosmetic cleanup ! src/share/classes/com/sun/tools/javac/code/Types.java ! src/share/classes/com/sun/tools/javac/comp/Attr.java ! src/share/classes/com/sun/tools/javac/comp/Resolve.java From jonathan.gibbons at sun.com Fri May 15 15:55:59 2009 From: jonathan.gibbons at sun.com (jonathan.gibbons at sun.com) Date: Fri, 15 May 2009 22:55:59 +0000 Subject: hg: jigsaw/jigsaw/langtools: TEMP: auto-default -source 7 when module-specific items found on the command line Message-ID: <20090515225601.651ABE068@hg.openjdk.java.net> Changeset: d9cf7132c862 Author: jjg Date: 2009-05-15 15:52 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/d9cf7132c862 TEMP: auto-default -source 7 when module-specific items found on the command line ! src/share/classes/com/sun/tools/javac/main/Main.java ! src/share/classes/com/sun/tools/javac/main/RecognizedOptions.java From forax at univ-mlv.fr Fri May 15 16:10:21 2009 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Sat, 16 May 2009 01:10:21 +0200 Subject: hg: jigsaw/jigsaw/langtools: TEMP: auto-default -source 7 when module-specific items found on the command line In-Reply-To: <20090515225601.651ABE068@hg.openjdk.java.net> References: <20090515225601.651ABE068@hg.openjdk.java.net> Message-ID: <4A0DF65D.3070700@univ-mlv.fr> jonathan.gibbons at sun.com a ?crit : > Changeset: d9cf7132c862 > Author: jjg > Date: 2009-05-15 15:52 -0700 > URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/d9cf7132c862 > > TEMP: auto-default -source 7 when module-specific items found on the command line > > ! src/share/classes/com/sun/tools/javac/main/Main.java > ! src/share/classes/com/sun/tools/javac/main/RecognizedOptions.java > > Jon, auto-default is not a good idea. The compiler should emit an error saying to use -source 7. If I compile with -source 1.4, it should compile with -source 1.4. Auto-stuffs are only obvious for the developer that write that code, not for its users. regards, R?mi From Jonathan.Gibbons at Sun.COM Fri May 15 16:24:04 2009 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Fri, 15 May 2009 16:24:04 -0700 Subject: hg: jigsaw/jigsaw/langtools: TEMP: auto-default -source 7 when module-specific items found on the command line In-Reply-To: <4A0DF65D.3070700@univ-mlv.fr> References: <20090515225601.651ABE068@hg.openjdk.java.net> <4A0DF65D.3070700@univ-mlv.fr> Message-ID: <4A0DF994.5090000@sun.com> R?mi Forax wrote: > jonathan.gibbons at sun.com a ?crit : >> Changeset: d9cf7132c862 >> Author: jjg >> Date: 2009-05-15 15:52 -0700 >> URL: >> http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/d9cf7132c862 >> >> TEMP: auto-default -source 7 when module-specific items found on the >> command line >> >> ! src/share/classes/com/sun/tools/javac/main/Main.java >> ! src/share/classes/com/sun/tools/javac/main/RecognizedOptions.java >> >> > Jon, auto-default is not a good idea. > The compiler should emit an error saying to use -source 7. > > If I compile with -source 1.4, it should compile with -source 1.4. I agree. It will. The auto-default only kicks in if no -source is given. This is a temporary hack to ease use of this compiler until JDK 7 changes to set -source 7 as the default. Work to make that happen is underway. This hack provided here will be removed at that time. -- Jon > > Auto-stuffs are only obvious for the developer that write that code, > not for its users. > > regards, > R?mi From mr at sun.com Fri May 15 21:15:47 2009 From: mr at sun.com (mr at sun.com) Date: Sat, 16 May 2009 04:15:47 +0000 Subject: hg: jigsaw/jigsaw/jdk: 3 new changesets Message-ID: <20090516041624.67BE5E0C1@hg.openjdk.java.net> Changeset: 1a3363f36405 Author: mr Date: 2009-05-15 21:11 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/1a3363f36405 Use $JAVA_HOME/lib/modules as the default library ! src/share/classes/org/openjdk/jigsaw/cli/Librarian.java Changeset: 90cb3c1c4a4d Author: mr Date: 2009-05-15 21:11 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/90cb3c1c4a4d jmod's show command should take a module-id query, not a module id ! src/share/classes/org/openjdk/jigsaw/cli/Librarian.java Changeset: 9a2ffe9d1854 Author: mr Date: 2009-05-15 21:11 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/9a2ffe9d1854 Install from simple single-module classes directories too ! src/share/classes/org/openjdk/jigsaw/Manifest.java ! src/share/classes/org/openjdk/jigsaw/SimpleLibrary.java From mr at sun.com Sun May 17 21:41:24 2009 From: mr at sun.com (Mark Reinhold) Date: Sun, 17 May 2009 21:41:24 -0700 Subject: Another temporary javac hack please Message-ID: <20090518044124.04929931A@callebaut.niobe.net> I need to be able to write module-info files like this: module demo @ 0.1 { requires jdk.base @ 7-ea; } Right now the above declaration causes javac to look for a jdk.base module in the module path, and naturally it doesn't find one. That will change over time, but for now javac is getting all JDK classes from the usual boot class path. For demo purposes I need to write such declarations in source files and interpret them at install time. Would it make sense to add another short-term hack which would cause javac to avoid interpreting dependences upon any "platform" module, but still pass them through to the resulting module-info.class file? Thanks, - Mark From Jonathan.Gibbons at Sun.COM Mon May 18 08:05:30 2009 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Mon, 18 May 2009 08:05:30 -0700 Subject: Another temporary javac hack please In-Reply-To: <20090518044124.04929931A@callebaut.niobe.net> References: <20090518044124.04929931A@callebaut.niobe.net> Message-ID: As a short term hack, I can do this, but longer term I still think we need a better way of recognizing names which identify platform modules., either using modifiers or a more general naming scheme that allows for different types of the platform (meaning versions in the sense of Java ME, Java SE, etc. As a right now hack, you could put an empty module-info.java with the right name on your compilation module path. -- Jon On May 17, 2009, at 9:41 PM, Mark Reinhold wrote: > I need to be able to write module-info files like this: > > module demo @ 0.1 { > requires jdk.base @ 7-ea; > } > > Right now the above declaration causes javac to look for a jdk.base > module in the module path, and naturally it doesn't find one. That > will change over time, but for now javac is getting all JDK classes > from the usual boot class path. > > For demo purposes I need to write such declarations in source files > and interpret them at install time. Would it make sense to add > another short-term hack which would cause javac to avoid interpreting > dependences upon any "platform" module, but still pass them through > to the resulting module-info.class file? > > Thanks, > - Mark From mr at sun.com Mon May 18 09:20:20 2009 From: mr at sun.com (Mark Reinhold) Date: Mon, 18 May 2009 09:20:20 -0700 Subject: Another temporary javac hack please In-Reply-To: jonathan.gibbons@sun.com; Mon, 18 May 2009 08:05:30 PDT; Message-ID: <20090518162020.848EF931A@callebaut.niobe.net> > Date: Mon, 18 May 2009 08:05:30 -0700 > From: jonathan.gibbons at sun.com > As a short term hack, I can do this, but longer term I still think we > need a better way of recognizing names which identify platform > modules., either using modifiers or a more general naming scheme > that allows for different types of the platform (meaning versions in > the sense of Java ME, Java SE, etc. Couldn't agree more. To do this right, however, requires implementing "provides" clauses first, and that's unlikely to happen in the next two weeks. Modules named "jdk(\.\w+)?" are ultimately intended merely to be artifacts of our implementation; applications, if they declare an explicit platform dependence, will refer to abstract modules named "java(\.\w+)?". We will actively, and perhaps mechanically, discourage people from referring to "jdk" modules. In the meantime, org.openjdk.jigsaw.Platform.isPlatformModuleName(String) will tell you what is, and is not, a "platform" module. > As a right now hack, you could put an empty module-info.java > with the right name on your compilation module path. Yes, but for demo purposes I'm mainly using single-module source and classes directories, in the traditional layout. - Mark From mr at sun.com Mon May 18 12:55:22 2009 From: mr at sun.com (Mark Reinhold) Date: Mon, 18 May 2009 12:55:22 -0700 Subject: Small VM change #2: JVM_ entry point to extend the boot class path Message-ID: <20090518195523.8B885931A@callebaut.niobe.net> As I hinted last week, the Jigsaw runtime needs the ability to extend the bootstrap class path after the VM has been initialized. Why does it need this? The java launcher knows only about the JDK "boot" module, but almost every application will need other JDK modules as well. To ensure compatibility, and to allow JDK packages to be split statically across multiple modules, the additional JDK modules must be loaded into the same module class loader as the boot module. In theory it's possible for the java launcher to figure out what those modules are, but it's vastly easier to defer this to the Jigsaw runtime code. So what I have working now is a modified BootLoader class which, upon initialization, adds the classes from all the other JDK modules required by the application to the boot class path. (This isn't the only way to solve this problem, and indeed later on it might be better to create a more intimate connection between the VM's boot class loader and the Jigsaw runtime, but this will do for now.) Patch attached, since this is a pretty simple change, though I can produce a webrev if preferred. All I've really done is to expose the existing ClassLoader::update_class_path_entry_list procedure as the new JVM_ entry point JVM_ExtendBootClassPath. I've copied the locking protocol from SystemDictionary::download_and_retry_class_load, which does a very similar thing, but I'm far from an expert on internal locking in HotSpot so please do point out anything I've gotten wrong here. I've updated the Linux mapfiles but not those for Solaris. I'm not doing Solaris (or Windows) builds at the moment, and if I forget to fix those mapfiles later then the problem will be glaringly obvious, so for the sake of expedience I'd like to make this change now and clean up the other builds later on. Karen -- Okay to push? Thanks, - Mark -------------- next part -------------- diff -r 403a1b93874f make/linux/makefiles/mapfile-vers-debug --- a/make/linux/makefiles/mapfile-vers-debug Thu May 14 14:48:35 2009 -0700 +++ b/make/linux/makefiles/mapfile-vers-debug Mon May 18 12:33:07 2009 -0700 @@ -87,6 +87,7 @@ JVM_DumpThreads; JVM_EnableCompiler; JVM_Exit; + JVM_ExtendBootClassPath; JVM_FillInStackTrace; JVM_FindClassFromClass; JVM_FindClassFromClassLoader; diff -r 403a1b93874f make/linux/makefiles/mapfile-vers-product --- a/make/linux/makefiles/mapfile-vers-product Thu May 14 14:48:35 2009 -0700 +++ b/make/linux/makefiles/mapfile-vers-product Mon May 18 12:33:07 2009 -0700 @@ -87,6 +87,7 @@ JVM_DumpThreads; JVM_EnableCompiler; JVM_Exit; + JVM_ExtendBootClassPath; JVM_FillInStackTrace; JVM_FindClassFromClass; JVM_FindClassFromClassLoader; diff -r 403a1b93874f src/share/vm/classfile/classLoader.hpp --- a/src/share/vm/classfile/classLoader.hpp Thu May 14 14:48:35 2009 -0700 +++ b/src/share/vm/classfile/classLoader.hpp Mon May 18 12:33:07 2009 -0700 @@ -190,7 +190,7 @@ // to avoid confusing the zip library static bool get_canonical_path(char* orig, char* out, int len); public: - // Used by the kernel jvm. + // Used by the kernel jvm, and by Jigsaw via JVM_ExtendBootClassPath static void update_class_path_entry_list(const char *path, bool check_for_duplicates); static void print_bootclasspath(); diff -r 403a1b93874f src/share/vm/prims/jvm.cpp --- a/src/share/vm/prims/jvm.cpp Thu May 14 14:48:35 2009 -0700 +++ b/src/share/vm/prims/jvm.cpp Mon May 18 12:33:07 2009 -0700 @@ -851,6 +851,18 @@ (jclass) JNIHandles::make_local(env, Klass::cast(k)->java_mirror()); JVM_END +JVM_ENTRY(void, JVM_ExtendBootClassPath(JNIEnv *env, jclass cls, const char *path)) + JVMWrapper2("JVM_ExtendBootClassPath(%s)", path) + { + // cf. SystemDictionary::download_and_retry_class_load + HandleMark hm(THREAD); + ResourceMark rm(THREAD); + Handle loader_lock(THREAD, SystemDictionary::system_loader_lock()); + ObjectLocker ol(loader_lock, THREAD); + ClassLoader::update_class_path_entry_list(path, true); + } + ClassLoader::print_bootclasspath(); +JVM_END // Reflection support ////////////////////////////////////////////////////////////////////////////// diff -r 403a1b93874f src/share/vm/prims/jvm.h --- a/src/share/vm/prims/jvm.h Thu May 14 14:48:35 2009 -0700 +++ b/src/share/vm/prims/jvm.h Mon May 18 12:33:07 2009 -0700 @@ -438,6 +438,12 @@ jobjectArray constants); /* + * Append a path to the boot class path + */ +JNIEXPORT void JNICALL +JVM_ExtendBootClassPath(JNIEnv *env, jclass cls, const char *path); + +/* * Reflection support functions */ From Karen.Kinnear at Sun.COM Mon May 18 13:47:53 2009 From: Karen.Kinnear at Sun.COM (Karen Kinnear) Date: Mon, 18 May 2009 16:47:53 -0400 Subject: Small VM change #2: JVM_ entry point to extend the boot class path In-Reply-To: <20090518195523.8B885931A@callebaut.niobe.net> References: <20090518195523.8B885931A@callebaut.niobe.net> Message-ID: <4A11C979.6060307@sun.com> Mark, Let's consider this a temporary J1 approach please. After J1 let's have some discussions about alternative approaches please. Question: in jvm.cpp and .hpp in the new JVM_ExtendBootClassPath(JNIEnv *env, jclass cls, const char *path) what is jclass cls used for? also in that method, I suspect you too would like the ClassLoader::print_bootclasspath() to be conditional on TraceClassLoading - unless for the demo you want this to print always. Other than that - just for the demo - go for it. thanks, Karen Mark Reinhold wrote: > As I hinted last week, the Jigsaw runtime needs the ability to extend the > bootstrap class path after the VM has been initialized. > > Why does it need this? The java launcher knows only about the JDK "boot" > module, but almost every application will need other JDK modules as well. > To ensure compatibility, and to allow JDK packages to be split statically > across multiple modules, the additional JDK modules must be loaded into > the same module class loader as the boot module. > > In theory it's possible for the java launcher to figure out what those > modules are, but it's vastly easier to defer this to the Jigsaw runtime > code. So what I have working now is a modified BootLoader class which, > upon initialization, adds the classes from all the other JDK modules > required by the application to the boot class path. > > (This isn't the only way to solve this problem, and indeed later on it > might be better to create a more intimate connection between the VM's > boot class loader and the Jigsaw runtime, but this will do for now.) > > Patch attached, since this is a pretty simple change, though I can > produce a webrev if preferred. All I've really done is to expose the > existing ClassLoader::update_class_path_entry_list procedure as the new > JVM_ entry point JVM_ExtendBootClassPath. I've copied the locking > protocol from SystemDictionary::download_and_retry_class_load, which does > a very similar thing, but I'm far from an expert on internal locking in > HotSpot so please do point out anything I've gotten wrong here. > > I've updated the Linux mapfiles but not those for Solaris. I'm not doing > Solaris (or Windows) builds at the moment, and if I forget to fix those > mapfiles later then the problem will be glaringly obvious, so for the > sake of expedience I'd like to make this change now and clean up the > other builds later on. > > Karen -- Okay to push? > > Thanks, > - Mark > > > > ------------------------------------------------------------------------ > > diff -r 403a1b93874f make/linux/makefiles/mapfile-vers-debug > --- a/make/linux/makefiles/mapfile-vers-debug Thu May 14 14:48:35 2009 -0700 > +++ b/make/linux/makefiles/mapfile-vers-debug Mon May 18 12:33:07 2009 -0700 > @@ -87,6 +87,7 @@ > JVM_DumpThreads; > JVM_EnableCompiler; > JVM_Exit; > + JVM_ExtendBootClassPath; > JVM_FillInStackTrace; > JVM_FindClassFromClass; > JVM_FindClassFromClassLoader; > diff -r 403a1b93874f make/linux/makefiles/mapfile-vers-product > --- a/make/linux/makefiles/mapfile-vers-product Thu May 14 14:48:35 2009 -0700 > +++ b/make/linux/makefiles/mapfile-vers-product Mon May 18 12:33:07 2009 -0700 > @@ -87,6 +87,7 @@ > JVM_DumpThreads; > JVM_EnableCompiler; > JVM_Exit; > + JVM_ExtendBootClassPath; > JVM_FillInStackTrace; > JVM_FindClassFromClass; > JVM_FindClassFromClassLoader; > diff -r 403a1b93874f src/share/vm/classfile/classLoader.hpp > --- a/src/share/vm/classfile/classLoader.hpp Thu May 14 14:48:35 2009 -0700 > +++ b/src/share/vm/classfile/classLoader.hpp Mon May 18 12:33:07 2009 -0700 > @@ -190,7 +190,7 @@ > // to avoid confusing the zip library > static bool get_canonical_path(char* orig, char* out, int len); > public: > - // Used by the kernel jvm. > + // Used by the kernel jvm, and by Jigsaw via JVM_ExtendBootClassPath > static void update_class_path_entry_list(const char *path, > bool check_for_duplicates); > static void print_bootclasspath(); > diff -r 403a1b93874f src/share/vm/prims/jvm.cpp > --- a/src/share/vm/prims/jvm.cpp Thu May 14 14:48:35 2009 -0700 > +++ b/src/share/vm/prims/jvm.cpp Mon May 18 12:33:07 2009 -0700 > @@ -851,6 +851,18 @@ > (jclass) JNIHandles::make_local(env, Klass::cast(k)->java_mirror()); > JVM_END > > +JVM_ENTRY(void, JVM_ExtendBootClassPath(JNIEnv *env, jclass cls, const char *path)) > + JVMWrapper2("JVM_ExtendBootClassPath(%s)", path) > + { > + // cf. SystemDictionary::download_and_retry_class_load > + HandleMark hm(THREAD); > + ResourceMark rm(THREAD); > + Handle loader_lock(THREAD, SystemDictionary::system_loader_lock()); > + ObjectLocker ol(loader_lock, THREAD); > + ClassLoader::update_class_path_entry_list(path, true); > + } > + ClassLoader::print_bootclasspath(); > +JVM_END > > // Reflection support ////////////////////////////////////////////////////////////////////////////// > > diff -r 403a1b93874f src/share/vm/prims/jvm.h > --- a/src/share/vm/prims/jvm.h Thu May 14 14:48:35 2009 -0700 > +++ b/src/share/vm/prims/jvm.h Mon May 18 12:33:07 2009 -0700 > @@ -438,6 +438,12 @@ > jobjectArray constants); > > /* > + * Append a path to the boot class path > + */ > +JNIEXPORT void JNICALL > +JVM_ExtendBootClassPath(JNIEnv *env, jclass cls, const char *path); > + > +/* > * Reflection support functions > */ > From mr at sun.com Mon May 18 15:34:27 2009 From: mr at sun.com (Mark Reinhold) Date: Mon, 18 May 2009 15:34:27 -0700 Subject: Small VM change #2: JVM_ entry point to extend the boot class path In-Reply-To: karen.kinnear@sun.com; Mon, 18 May 2009 16:47:53 EDT; <4A11C979.6060307@sun.com> Message-ID: <20090518223427.C2C2694A9@callebaut.niobe.net> > Date: Mon, 18 May 2009 16:47:53 -0400 > From: karen.kinnear at sun.com > Let's consider this a temporary J1 approach please. After J1 let's > have some discussions about alternative approaches please. Sure, absolutely. > Question: in jvm.cpp and .hpp in the new > JVM_ExtendBootClassPath(JNIEnv *env, jclass cls, const char *path) > what is jclass cls used for? Nothing; that's leftover from an initial version where I thought I was going to use RegisterNatives for this entry point. Removed. > also in that method, I suspect you too would like the > ClassLoader::print_bootclasspath() to be conditional on > TraceClassLoading - unless for the demo you want this to print always. I meant to remove that entirely, but gating it on TraceClassLoading makes sense, so I'll do that. > Other than that - just for the demo - go for it. Thanks. New patch attached FYI. - Mark -------------- next part -------------- diff -r 403a1b93874f make/linux/makefiles/mapfile-vers-debug --- a/make/linux/makefiles/mapfile-vers-debug Thu May 14 14:48:35 2009 -0700 +++ b/make/linux/makefiles/mapfile-vers-debug Mon May 18 15:32:49 2009 -0700 @@ -87,6 +87,7 @@ JVM_DumpThreads; JVM_EnableCompiler; JVM_Exit; + JVM_ExtendBootClassPath; JVM_FillInStackTrace; JVM_FindClassFromClass; JVM_FindClassFromClassLoader; diff -r 403a1b93874f make/linux/makefiles/mapfile-vers-product --- a/make/linux/makefiles/mapfile-vers-product Thu May 14 14:48:35 2009 -0700 +++ b/make/linux/makefiles/mapfile-vers-product Mon May 18 15:32:49 2009 -0700 @@ -87,6 +87,7 @@ JVM_DumpThreads; JVM_EnableCompiler; JVM_Exit; + JVM_ExtendBootClassPath; JVM_FillInStackTrace; JVM_FindClassFromClass; JVM_FindClassFromClassLoader; diff -r 403a1b93874f src/share/vm/classfile/classLoader.cpp --- a/src/share/vm/classfile/classLoader.cpp Thu May 14 14:48:35 2009 -0700 +++ b/src/share/vm/classfile/classLoader.cpp Mon May 18 15:32:49 2009 -0700 @@ -512,12 +512,15 @@ // File or directory found ClassPathEntry* new_entry = NULL; create_class_path_entry((char *)path, st, &new_entry, LazyBootClassLoader); - // The kernel VM adds dynamically to the end of the classloader path and - // doesn't reorder the bootclasspath which would break java.lang.Package - // (see PackageInfo). - // Add new entry to linked list + // The kernel VM, and Jigsaw, add dynamically to the end of the classloader + // path and don't reorder the bootclasspath, which would break + // java.lang.Package (see PackageInfo). if (!check_for_duplicates || !contains_entry(new_entry)) { + // Add new entry to linked list add_to_list(new_entry); + if (TraceClassLoading) { + print_bootclasspath(); + } } } } diff -r 403a1b93874f src/share/vm/classfile/classLoader.hpp --- a/src/share/vm/classfile/classLoader.hpp Thu May 14 14:48:35 2009 -0700 +++ b/src/share/vm/classfile/classLoader.hpp Mon May 18 15:32:49 2009 -0700 @@ -190,7 +190,7 @@ // to avoid confusing the zip library static bool get_canonical_path(char* orig, char* out, int len); public: - // Used by the kernel jvm. + // Used by the kernel jvm, and by Jigsaw via JVM_ExtendBootClassPath static void update_class_path_entry_list(const char *path, bool check_for_duplicates); static void print_bootclasspath(); diff -r 403a1b93874f src/share/vm/prims/jvm.cpp --- a/src/share/vm/prims/jvm.cpp Thu May 14 14:48:35 2009 -0700 +++ b/src/share/vm/prims/jvm.cpp Mon May 18 15:32:49 2009 -0700 @@ -851,6 +851,17 @@ (jclass) JNIHandles::make_local(env, Klass::cast(k)->java_mirror()); JVM_END +JVM_ENTRY(void, JVM_ExtendBootClassPath(JNIEnv *env, const char *path)) + JVMWrapper2("JVM_ExtendBootClassPath(%s)", path) + { + // cf. SystemDictionary::download_and_retry_class_load + HandleMark hm(THREAD); + ResourceMark rm(THREAD); + Handle loader_lock(THREAD, SystemDictionary::system_loader_lock()); + ObjectLocker ol(loader_lock, THREAD); + ClassLoader::update_class_path_entry_list(path, true); + } +JVM_END // Reflection support ////////////////////////////////////////////////////////////////////////////// diff -r 403a1b93874f src/share/vm/prims/jvm.h --- a/src/share/vm/prims/jvm.h Thu May 14 14:48:35 2009 -0700 +++ b/src/share/vm/prims/jvm.h Mon May 18 15:32:49 2009 -0700 @@ -438,6 +438,12 @@ jobjectArray constants); /* + * Append a path to the boot class path + */ +JNIEXPORT void JNICALL +JVM_ExtendBootClassPath(JNIEnv *env, const char *path); + +/* * Reflection support functions */ From Jonathan.Gibbons at Sun.COM Mon May 18 17:10:00 2009 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Mon, 18 May 2009 17:10:00 -0700 Subject: module-info platform dependencies In-Reply-To: <20090515154406.850295E97@eggemoggin.niobe.net> References: <20090515154406.850295E97@eggemoggin.niobe.net> Message-ID: <4A11F8D8.7040604@sun.com> Mark, I'd like to propose the compiler include "synthetic" in the flags for any implicit requires statement generated by the compiler. I think it is important we should be able to differentiate user-written requires from system generated ones. -- Jon Mark Reinhold wrote: >> Date: Thu, 14 May 2009 22:13:49 -0700 >> From: jonathan.gibbons at sun.com >> > > >> On May 14, 2009, at 9:20 PM, Mark Reinhold wrote: >> >>> The basic idea is that the compiler can ask Jigsaw whether a >>> particular dependence is upon a known, available platform or profile >>> module, and it can also ask what the default is. If a module-info >>> file declares at least one dependence upon a platform module then the >>> compiler does nothing. If it does not declare such a dependence then >>> the compiler inserts a dependence upon the default. >>> >>> >> It's not clear to me why the compiler should be doing this and/or its >> not clear to me what the right policy is here. >> >> It's not clear the compiler should be doing this because so much else >> is being deferred to the module system, why is this piece of analysis >> coming back into the compiler? I'm not saying it shouldn't; I'm just >> saying I don't yet understand why. >> > > It's a fairly trivial analysis on the compiler's part, and it is in this > case deferring to the module system, to identify platform modules and to > specify the default. > > >> One reason it may be more of a compiler function is future interaction >> with the compiler's -source and -target flags. Although rarely used >> correctly, -target should normally be used in conjunction with >> -Xbootclasspath. In the JDK 8 timeframe, when it may make sense to >> compile for JDK 7 or JDK8, the -target switch might interact with >> and/or influence the default of the platform dependence. >> > > Exactly. When you compile a module-info file with JDK 8 using -target 7 > then the platform dependence would default to, say, jdk@>=7. > > This reminds me of a related issue which we've discussed but not yet > settled: For dependences upon platform modules, at least, and possibly > in all cases, javac should probably be asking the module system for the > oldest available module satisfying the dependence's version constraint, > rather than the newest. > > >> If the compiler were to add a default, would you expect it to get >> compiled into the class file? >> > > Into the module-info class file? Yes. Into other class files? No. > > >> It also seems to me that a case can be made for the module system to >> analyze the need for a default for a group of modules, in that if the >> set of leaf modules do not unambiguously specify a default, then the >> module system should specify one for the group. But there should not be >> a need for anyone (such as the compiler) to insert additional >> dependences into all modules that don't specify a platform when those >> modules will have an indirect dependence by virtue of the transitive >> closure of their own (explicit) dependencies. >> > > A module that does not declare a direct dependence upon a platform module > will only have an indirect dependence upon a platform module if there is > a transitive path of public dependences from that module to a platform > module. I don't expect that to be the case, i.e., I expect people will > generally write > > module foo @ 1 { > requires jdk @ >=7; > } > > rather than > > module foo @ 1 { > requires public jdk @ >=7; > } > > when declaring a specific platform dependence. > > >> That sort of group wide >> analysis belongs in the module resolver/system, not in the compiler. >> > > I'm not asking that the compiler do any sort of group-wide analysis, and > I completely agree that such analyses are best done by the module system. > > If the compiler inserts default platform dependences as I've suggested, > by doing a simple local analysis of each module, then the group-wide > analysis you describe above will be performed by the current resolution > algorithm, without change. > > - Mark > From mr at sun.com Mon May 18 21:56:28 2009 From: mr at sun.com (Mark Reinhold) Date: Mon, 18 May 2009 21:56:28 -0700 Subject: module-info platform dependencies In-Reply-To: jonathan.gibbons@sun.com; Mon, 18 May 2009 17:10:00 PDT; <4A11F8D8.7040604@sun.com> Message-ID: <20090519045628.9A2C5960A@callebaut.niobe.net> > Date: Mon, 18 May 2009 17:10:00 -0700 > From: jonathan.gibbons at sun.com > I'd like to propose the compiler include "synthetic" in the flags for > any implicit requires statement generated by the compiler. I think it > is important we should be able to differentiate user-written requires > from system generated ones. Makes sense to me. - Mark From Karen.Kinnear at Sun.COM Tue May 19 05:48:52 2009 From: Karen.Kinnear at Sun.COM (Karen Kinnear) Date: Tue, 19 May 2009 08:48:52 -0400 Subject: Small VM change #2: JVM_ entry point to extend the boot class path In-Reply-To: <20090518223427.C2C2694A9@callebaut.niobe.net> References: <20090518223427.C2C2694A9@callebaut.niobe.net> Message-ID: <42C47310-B50B-4957-A6A7-92AADC59B53E@Sun.COM> Thanks Mark. If we were planning to leave this in, I'd ask you to remove the print_bootclasspath from SystemDictionary:download_and_retry_class_load since right now that would have duplicate prints, or better yet have the caller of update_class_path_entry_list do the printing, including the "cause" of the change. But not a big deal since this is temporary. thanks, Karen On May 18, 2009, at 6:34 PM, Mark Reinhold wrote: >> Date: Mon, 18 May 2009 16:47:53 -0400 >> From: karen.kinnear at sun.com > >> Let's consider this a temporary J1 approach please. After J1 let's >> have some discussions about alternative approaches please. > > Sure, absolutely. > >> Question: in jvm.cpp and .hpp in the new >> JVM_ExtendBootClassPath(JNIEnv *env, jclass cls, const char *path) >> what is jclass cls used for? > > Nothing; that's leftover from an initial version where I thought I was > going to use RegisterNatives for this entry point. Removed. > >> also in that method, I suspect you too would like the >> ClassLoader::print_bootclasspath() to be conditional on >> TraceClassLoading - unless for the demo you want this to print >> always. > > I meant to remove that entirely, but gating it on TraceClassLoading > makes > sense, so I'll do that. > >> Other than that - just for the demo - go for it. > > Thanks. New patch attached FYI. > > - Mark > > diff -r 403a1b93874f make/linux/makefiles/mapfile-vers-debug > --- a/make/linux/makefiles/mapfile-vers-debug Thu May 14 14:48:35 > 2009 -0700 > +++ b/make/linux/makefiles/mapfile-vers-debug Mon May 18 15:32:49 > 2009 -0700 > @@ -87,6 +87,7 @@ > JVM_DumpThreads; > JVM_EnableCompiler; > JVM_Exit; > + JVM_ExtendBootClassPath; > JVM_FillInStackTrace; > JVM_FindClassFromClass; > JVM_FindClassFromClassLoader; > diff -r 403a1b93874f make/linux/makefiles/mapfile-vers-product > --- a/make/linux/makefiles/mapfile-vers-product Thu May 14 14:48:35 > 2009 -0700 > +++ b/make/linux/makefiles/mapfile-vers-product Mon May 18 15:32:49 > 2009 -0700 > @@ -87,6 +87,7 @@ > JVM_DumpThreads; > JVM_EnableCompiler; > JVM_Exit; > + JVM_ExtendBootClassPath; > JVM_FillInStackTrace; > JVM_FindClassFromClass; > JVM_FindClassFromClassLoader; > diff -r 403a1b93874f src/share/vm/classfile/classLoader.cpp > --- a/src/share/vm/classfile/classLoader.cpp Thu May 14 14:48:35 > 2009 -0700 > +++ b/src/share/vm/classfile/classLoader.cpp Mon May 18 15:32:49 > 2009 -0700 > @@ -512,12 +512,15 @@ > // File or directory found > ClassPathEntry* new_entry = NULL; > create_class_path_entry((char *)path, st, &new_entry, > LazyBootClassLoader); > - // The kernel VM adds dynamically to the end of the classloader > path and > - // doesn't reorder the bootclasspath which would break > java.lang.Package > - // (see PackageInfo). > - // Add new entry to linked list > + // The kernel VM, and Jigsaw, add dynamically to the end of the > classloader > + // path and don't reorder the bootclasspath, which would break > + // java.lang.Package (see PackageInfo). > if (!check_for_duplicates || !contains_entry(new_entry)) { > + // Add new entry to linked list > add_to_list(new_entry); > + if (TraceClassLoading) { > + print_bootclasspath(); > + } > } > } > } > diff -r 403a1b93874f src/share/vm/classfile/classLoader.hpp > --- a/src/share/vm/classfile/classLoader.hpp Thu May 14 14:48:35 > 2009 -0700 > +++ b/src/share/vm/classfile/classLoader.hpp Mon May 18 15:32:49 > 2009 -0700 > @@ -190,7 +190,7 @@ > // to avoid confusing the zip library > static bool get_canonical_path(char* orig, char* out, int len); > public: > - // Used by the kernel jvm. > + // Used by the kernel jvm, and by Jigsaw via > JVM_ExtendBootClassPath > static void update_class_path_entry_list(const char *path, > bool check_for_duplicates); > static void print_bootclasspath(); > diff -r 403a1b93874f src/share/vm/prims/jvm.cpp > --- a/src/share/vm/prims/jvm.cpp Thu May 14 14:48:35 2009 -0700 > +++ b/src/share/vm/prims/jvm.cpp Mon May 18 15:32:49 2009 -0700 > @@ -851,6 +851,17 @@ > (jclass) JNIHandles::make_local(env, Klass::cast(k)- > >java_mirror()); > JVM_END > > +JVM_ENTRY(void, JVM_ExtendBootClassPath(JNIEnv *env, const char > *path)) > + JVMWrapper2("JVM_ExtendBootClassPath(%s)", path) > + { > + // cf. SystemDictionary::download_and_retry_class_load > + HandleMark hm(THREAD); > + ResourceMark rm(THREAD); > + Handle loader_lock(THREAD, > SystemDictionary::system_loader_lock()); > + ObjectLocker ol(loader_lock, THREAD); > + ClassLoader::update_class_path_entry_list(path, true); > + } > +JVM_END > > // Reflection > support > ////////////////////////////////////////////////////////////////////////////// > > diff -r 403a1b93874f src/share/vm/prims/jvm.h > --- a/src/share/vm/prims/jvm.h Thu May 14 14:48:35 2009 -0700 > +++ b/src/share/vm/prims/jvm.h Mon May 18 15:32:49 2009 -0700 > @@ -438,6 +438,12 @@ > jobjectArray constants); > > /* > + * Append a path to the boot class path > + */ > +JNIEXPORT void JNICALL > +JVM_ExtendBootClassPath(JNIEnv *env, const char *path); > + > +/* > * Reflection support functions > */ From jonathan.gibbons at sun.com Wed May 20 16:19:41 2009 From: jonathan.gibbons at sun.com (jonathan.gibbons at sun.com) Date: Wed, 20 May 2009 23:19:41 +0000 Subject: hg: jigsaw/jigsaw/langtools: javac: Modules -- provide default platform requires if none specified Message-ID: <20090520231944.70151E5E6@hg.openjdk.java.net> Changeset: 9359f5d70528 Author: jjg Date: 2009-05-20 16:14 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/9359f5d70528 javac: Modules -- provide default platform requires if none specified javac: Modules -- use "synthetic" flag on default requires javac: ZeroMod -- ignore platform requires for now ! src/share/classes/com/sun/tools/javac/comp/Modules.java ! src/share/classes/com/sun/tools/javac/util/Names.java ! src/share/classes/javax/lang/model/util/ModuleResolver.java ! test/tools/javac/modules/ModuleRequiresAttributeTest01.java From Dalibor.Topic at Sun.COM Thu May 21 17:55:04 2009 From: Dalibor.Topic at Sun.COM (Dalibor Topic) Date: Fri, 22 May 2009 02:55:04 +0200 Subject: Initial thoughts on module packaging In-Reply-To: <20090501164205.103AB5E2F@eggemoggin.niobe.net> References: <20090501164205.103AB5E2F@eggemoggin.niobe.net> Message-ID: <4A15F7E8.8090102@sun.com> Mark Reinhold wrote: > Dalibor -- To start, I suggest you work on the simple case of pure > Java modules. We'll tackle native libraries, resource files, and > configuration files later on. First cut committed. It creates debian binary packages from jigsaw modules. Tested with the classic greeting routine: $ jpkg -L /tmp/pkg -v -m javac_compiled_modules -d . deb org.astro com.greetings Creating binary Debian package for org.astro Creating binary Debian package for com.greetings $ sudo dpkg -i org.astro_1.2_all.deb com.greetings_0.1_all.deb Selecting previously deselected package org.astro. (Reading database ... 271672 files and directories currently installed.) Unpacking org.astro (from org.astro_1.2_all.deb) ... Selecting previously deselected package com.greetings. Unpacking com.greetings (from com.greetings_0.1_all.deb) ... Setting up org.astro (1.2) ... Setting up com.greetings (0.1) ... $ java -ea -L /tmp/pkg/ -m com.greetings Hello, world! The call interface is jpkg [-v] [-L ] [-m ] [-d ] deb * -v : verbose output -L : library the modules are installed to -m : directory with modules to package -d : destination directory to put the package in You can ask jpkg to create modules for multiple packages, as in the example above. In order to create packages one needs to have jmod, jpkg, which, fakeroot and dpkg-deb on $PATH. The generated packages contain a generated postinst script, that calls jmod to reconfigure the library after a package is installed, and generated preinst script, that creates the library, in case that it doesn't exist yet, as well as as postrm script, that takes care of removing the generated config file. Not handled yet are native files, resources and configuration files. Hints, ideas & patches are welcome. I've tried to push the patch into the repo, but I keep getting the issue at http://mail.openjdk.java.net/pipermail/jdk7-gk/2008-April/000000.html with the jigsaw/jigsaw/jdk repository. So instead the webrev can be found here: http://cr.openjdk.java.net/~robilad/jpkg/webrev.00/ cheers, dalibor topic -- ******************************************************************* Dalibor Topic Tel: (+49 40) 23 646 738 Java F/OSS Ambassador AIM: robiladonaim Sun Microsystems GmbH Mobile: (+49 177) 2664 192 Nagelsweg 55 http://openjdk.java.net D-20097 Hamburg mailto:Dalibor.Topic at sun.com Sitz der Gesellschaft: Sonnenallee 1, D-85551 Kirchheim-Heimstetten Amtsgericht M?nchen: HRB 161028 Gesch?ftsf?hrer: Thomas Schr?der, Wolfgang Engels, Wolf Frenkel Vorsitzender des Aufsichtsrates: Martin H?ring From dalibor.topic at sun.com Fri May 22 15:41:33 2009 From: dalibor.topic at sun.com (dalibor.topic at sun.com) Date: Fri, 22 May 2009 22:41:33 +0000 Subject: hg: jigsaw/jigsaw/jdk: Initial jpkg support for debian packages Message-ID: <20090522224159.5570CE880@hg.openjdk.java.net> Changeset: 41752f479359 Author: robilad Date: 2009-05-23 01:36 +0200 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/41752f479359 Initial jpkg support for debian packages ! make/java/java/FILES_java.gmk ! make/launchers/Makefile ! src/share/classes/org/openjdk/jigsaw/Files.java + src/share/classes/org/openjdk/jigsaw/cli/Packager.java From Dalibor.Topic at Sun.COM Fri May 22 15:52:58 2009 From: Dalibor.Topic at Sun.COM (Dalibor Topic) Date: Sat, 23 May 2009 00:52:58 +0200 Subject: Initial thoughts on module packaging In-Reply-To: <4A15F7E8.8090102@sun.com> References: <20090501164205.103AB5E2F@eggemoggin.niobe.net> <4A15F7E8.8090102@sun.com> Message-ID: <4A172CCA.6040703@sun.com> Dalibor Topic wrote: > Mark Reinhold wrote: >> Dalibor -- To start, I suggest you work on the simple case of pure >> Java modules. We'll tackle native libraries, resource files, and >> configuration files later on. > > First cut committed. It creates debian binary packages from jigsaw > modules. Tested with the classic greeting routine: > > $ jpkg -L /tmp/pkg -v -m javac_compiled_modules -d . deb org.astro com.greetings > Creating binary Debian package for org.astro > Creating binary Debian package for com.greetings > > $ sudo dpkg -i org.astro_1.2_all.deb com.greetings_0.1_all.deb > Selecting previously deselected package org.astro. > (Reading database ... 271672 files and directories currently installed.) > Unpacking org.astro (from org.astro_1.2_all.deb) ... > Selecting previously deselected package com.greetings. > Unpacking com.greetings (from com.greetings_0.1_all.deb) ... > Setting up org.astro (1.2) ... > > Setting up com.greetings (0.1) ... > > $ java -ea -L /tmp/pkg/ -m com.greetings > Hello, world! > > The call interface is > > jpkg [-v] [-L ] [-m ] [-d ] deb * > > -v : verbose output > -L : library the modules are installed to > -m : directory with modules to package > -d : destination directory to put the package in > > You can ask jpkg to create modules for multiple packages, as in > the example above. In order to create packages one needs to have > jmod, jpkg, which, fakeroot and dpkg-deb on $PATH. > > The generated packages contain a generated postinst script, > that calls jmod to reconfigure the library after a package is > installed, and generated preinst script, that creates > the library, in case that it doesn't exist yet, as well as > as postrm script, that takes care of removing the generated > config file. > > Not handled yet are native files, resources and configuration > files. I've added a few more things since, so the interface expanded to: jpkg [-v] [-L ] [-r ] [-i include-dir] [-m ] [-d ] [-c ] deb * new are: -r : directory with resources to bundle as part of the module -i : directory with files to include as part of the package -c : command name for launcher invoking main class so that you could now do something like jpkg -L /tmp/pkg -v -m javac_compiled_modules -d . deb -c greeter -r translations -i debian-extras com.greetings and get a small launcher in /usr/bin/greeter , whose module includes the resources in the subdirectory translations and whose package includes all the additional data in debian-extras, for example the /usr/share/doc/greeter/copyright file, etc. Basically, this allows decoupling the files that are resources used by the module (but not touched by the compiler), like icons, translations, etc to be included in the package along with useful, packaging specific files, like all the good stuff lintian expects to see in a binary package (copyright files, man pages, etc.) that can't be trivially generated from the module information. cheers, dalibor topic -- ******************************************************************* Dalibor Topic Tel: (+49 40) 23 646 738 Java F/OSS Ambassador AIM: robiladonaim Sun Microsystems GmbH Mobile: (+49 177) 2664 192 Nagelsweg 55 http://openjdk.java.net D-20097 Hamburg mailto:Dalibor.Topic at sun.com Sitz der Gesellschaft: Sonnenallee 1, D-85551 Kirchheim-Heimstetten Amtsgericht M?nchen: HRB 161028 Gesch?ftsf?hrer: Thomas Schr?der, Wolfgang Engels, Wolf Frenkel Vorsitzender des Aufsichtsrates: Martin H?ring From Dalibor.Topic at Sun.COM Sun May 24 03:06:47 2009 From: Dalibor.Topic at Sun.COM (Dalibor Topic) Date: Sun, 24 May 2009 12:06:47 +0200 Subject: hg: jigsaw/jigsaw/langtools: javac: Modules -- provide default platform requires if none specified In-Reply-To: <20090520231944.70151E5E6@hg.openjdk.java.net> References: <20090520231944.70151E5E6@hg.openjdk.java.net> Message-ID: <4A191C37.9000903@sun.com> Jonathan.Gibbons at Sun.COM wrote: > Changeset: 9359f5d70528 > Author: jjg > Date: 2009-05-20 16:14 -0700 > URL: http://hg.openjdk.java.net/jigsaw/jigsaw/langtools/rev/9359f5d70528 > > javac: Modules -- provide default platform requires if none specified > javac: Modules -- use "synthetic" flag on default requires > javac: ZeroMod -- ignore platform requires for now > > ! src/share/classes/com/sun/tools/javac/comp/Modules.java > ! src/share/classes/com/sun/tools/javac/util/Names.java > ! src/share/classes/javax/lang/model/util/ModuleResolver.java > ! test/tools/javac/modules/ModuleRequiresAttributeTest01.java > Good morning, this changeset breaks the build as ModuleInfoReader tries to resolve Dependence.Modifier.SYNTHETIC on loading the generated synthetic dependency information, but fails with a illegal argument exception due to Enum.valueOf(Dependency.Modifier.class, "SYNTHETIC") not being an actually defined enum in the class. If you have some time, please have a go at a fix - I'll be on a plane crossing the Atlantic for the next couple of hours without battery power. ;) Otherwise, I'd be happy to push in a fix myself, if Mark & Jon can outline what it should be like. cheers, dalibor topic -- ******************************************************************* Dalibor Topic Tel: (+49 40) 23 646 738 Java F/OSS Ambassador AIM: robiladonaim Sun Microsystems GmbH Mobile: (+49 177) 2664 192 Nagelsweg 55 http://openjdk.java.net D-20097 Hamburg mailto:Dalibor.Topic at sun.com Sitz der Gesellschaft: Sonnenallee 1, D-85551 Kirchheim-Heimstetten Amtsgericht M?nchen: HRB 161028 Gesch?ftsf?hrer: Thomas Schr?der, Wolfgang Engels, Wolf Frenkel Vorsitzender des Aufsichtsrates: Martin H?ring From mr at sun.com Wed May 27 22:40:14 2009 From: mr at sun.com (Mark Reinhold) Date: Wed, 27 May 2009 22:40:14 -0700 Subject: Demo status, and a couple of jpkg enhancement requests Message-ID: <20090528054014.5A22B94A8@callebaut.niobe.net> Yes, it's the week before JavaOne. Utter insanity. I've somehow managed to hack up a reasonably-credible demo. Thanks to Jonathan for the recent compiler changes, and to Dalibor for the packaging tool. I need to clean up my patch queue a bit, but I'll try to push my current changes into the forest tomorrow. I'll also post the demo script, for the curious. Dalibor -- Any chance of getting these two jpkg enhancements done in the next day or so? - Compress class files using pack200 rather than simple gzip. This would make for a huge improvement in our package sizes. It probably makes sense to leave packaged classes in the JAR format. If so, I can easily change the runtime code to look for JAR files in addition to classes/ directories. You can assume that $JAVA_HOME/bin contains the unpack200 binary. (When packing I wouldn't bother to compress the output file; dpkg-deb will compress it with gzip anyway.) - Define additional command-line options for specifying some of the more common package attributes such as maintainer, description, section, installed-size, and maybe home page. Right now the JDK packages show up in Synaptic in a fairly boring way, without the information people expect. Fixes for the launcher-script issues would be welcome too: Use the jmod in the installed $JAVA_HOME/bin, and exec rather than fork the java binary. (I have a tiny patch to fix jpkg's translation of version strings, which I'll include in my next push.) - Mark From Jonathan.Gibbons at Sun.COM Wed May 27 22:48:39 2009 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Wed, 27 May 2009 22:48:39 -0700 Subject: Demo status, and a couple of jpkg enhancement requests In-Reply-To: <20090528054014.5A22B94A8@callebaut.niobe.net> References: <20090528054014.5A22B94A8@callebaut.niobe.net> Message-ID: <7E141206-20FE-4B40-80CB-A2C8DE55A6AA@sun.com> Sounds exciting! -- Jon On May 27, 2009, at 10:40 PM, Mark Reinhold wrote: > Yes, it's the week before JavaOne. Utter insanity. > > I've somehow managed to hack up a reasonably-credible demo. Thanks > to Jonathan for the recent compiler changes, and to Dalibor for the > packaging tool. > > I need to clean up my patch queue a bit, but I'll try to push my > current changes into the forest tomorrow. I'll also post the demo > script, for the curious. > > Dalibor -- Any chance of getting these two jpkg enhancements done > in the next day or so? > > - Compress class files using pack200 rather than simple gzip. > This would make for a huge improvement in our package sizes. > It probably makes sense to leave packaged classes in the JAR > format. If so, I can easily change the runtime code to look > for JAR files in addition to classes/ directories. You can > assume that $JAVA_HOME/bin contains the unpack200 binary. > > (When packing I wouldn't bother to compress the output file; > dpkg-deb will compress it with gzip anyway.) > > - Define additional command-line options for specifying some > of the more common package attributes such as maintainer, > description, section, installed-size, and maybe home page. > Right now the JDK packages show up in Synaptic in a fairly > boring way, without the information people expect. > > Fixes for the launcher-script issues would be welcome too: Use the > jmod in the installed $JAVA_HOME/bin, and exec rather than fork the > java binary. (I have a tiny patch to fix jpkg's translation of > version strings, which I'll include in my next push.) > > - Mark From mr at sun.com Fri May 29 12:55:31 2009 From: mr at sun.com (mr at sun.com) Date: Fri, 29 May 2009 19:55:31 +0000 Subject: hg: jigsaw/jigsaw/jdk: 8 new changesets Message-ID: <20090529195725.289EFEDA4@hg.openjdk.java.net> Changeset: a6773d2669fd Author: mr Date: 2009-05-18 20:39 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/a6773d2669fd Boot loader: Add all boot-context modules to the boot class path ! make/java/java/Exportedfiles.gmk ! make/java/java/FILES_c.gmk ! make/java/java/Makefile ! make/java/java/mapfile-vers ! src/share/classes/org/openjdk/jigsaw/BootLoader.java ! src/share/classes/org/openjdk/jigsaw/Library.java ! src/share/classes/org/openjdk/jigsaw/Platform.java ! src/share/classes/org/openjdk/jigsaw/SimpleLibrary.java ! src/share/javavm/export/jvm.h + src/share/native/org/openjdk/jigsaw/BootLoader.c ! test/org/openjdk/jigsaw/MockLibrary.java Changeset: 780ba3319f63 Author: mr Date: 2009-05-18 20:39 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/780ba3319f63 Installed-module reindexing ! src/share/classes/org/openjdk/jigsaw/Files.java ! src/share/classes/org/openjdk/jigsaw/SimpleLibrary.java ! src/share/classes/org/openjdk/jigsaw/cli/Librarian.java Changeset: a33756ca8a80 Author: mr Date: 2009-05-25 16:03 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/a33756ca8a80 Separate boot/base module ! make/modules/Makefile + make/modules/jdk.boot.ls + make/modules/modularize ! src/share/bin/java.c ! src/share/classes/org/openjdk/jigsaw/Loader.java ! src/share/classes/org/openjdk/jigsaw/Platform.java + src/share/modules/jdk.base/module-info.java + src/share/modules/jdk.boot/module-info.java ! src/share/modules/jdk/module-info.java ! test/org/openjdk/jigsaw/_Library.java Changeset: 5f455d1eaa9a Author: mr Date: 2009-05-25 16:04 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/5f455d1eaa9a Merge ! src/share/classes/org/openjdk/jigsaw/Files.java Changeset: 83170074c90c Author: mr Date: 2009-05-29 12:45 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/83170074c90c jpkg: Map dashes to underscores when computing package versions ! src/share/classes/org/openjdk/jigsaw/cli/Packager.java Changeset: 69acde42810e Author: mr Date: 2009-05-29 12:45 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/69acde42810e Provide better diagnostics at key points in the launch sequence ! src/share/classes/java/text/BreakIterator.java ! src/share/classes/sun/reflect/misc/MethodUtil.java ! src/share/native/java/lang/System.c Changeset: e961084a67f0 Author: mr Date: 2009-05-29 12:45 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/e961084a67f0 Temporary hack to make the legacy launcher work ! src/share/classes/java/lang/ClassLoader.java ! src/share/classes/org/openjdk/jigsaw/BootLoader.java Changeset: c52a7387e8b3 Author: mr Date: 2009-05-29 12:45 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/c52a7387e8b3 Boot, base, awt, and swing modules, and their Debian packages ! make/modules/Makefile + make/modules/image.awt.ls + make/modules/image.boot.ls + make/modules/image.rest.ls + make/modules/imagine.sh + make/modules/jdk.awt.ls ! make/modules/jdk.boot.ls + make/modules/jdk.swing.ls - make/modules/modularize + make/modules/modularize.sh + src/share/modules/jdk.awt/module-info.java ! src/share/modules/jdk.boot/module-info.java + src/share/modules/jdk.swing/module-info.java From mr at sun.com Fri May 29 13:04:19 2009 From: mr at sun.com (mr at sun.com) Date: Fri, 29 May 2009 20:04:19 +0000 Subject: hg: jigsaw/jigsaw/hotspot: 2 new changesets Message-ID: <20090529200424.A8A32EDAF@hg.openjdk.java.net> Changeset: 8879ef58b397 Author: mr Date: 2009-05-29 09:38 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/8879ef58b397 JVM_ExtendBootClassPath ! make/linux/makefiles/mapfile-vers-debug ! make/linux/makefiles/mapfile-vers-product ! src/share/vm/classfile/classLoader.cpp ! src/share/vm/classfile/classLoader.hpp ! src/share/vm/prims/jvm.cpp ! src/share/vm/prims/jvm.h Changeset: dcfe9cb00de5 Author: mr Date: 2009-05-29 12:47 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/hotspot/rev/dcfe9cb00de5 Temporary hack to keep the legacy launcher working in modular mode ! src/share/vm/runtime/os.cpp From mr at sun.com Fri May 29 13:06:37 2009 From: mr at sun.com (Mark Reinhold) Date: Fri, 29 May 2009 13:06:37 -0700 Subject: Modules and packages and demos Message-ID: <20090529200637.D44A35E98@eggemoggin.niobe.net> With my latest changesets we can now build an initial set of platform modules, and corresponding Debian packages. Here are the modules (output of jmod ls -v, with comments added): jdk.boot at 7-ea permits jdk.base, jdk.awt, jdk.swing, jdk # The core of the platform, command-line only jdk.base at 7-ea requires local public jdk.boot@=7-ea # An empty module for applications to require; they can't require # jdk.boot directly since it only permits jdk{,.*} modules jdk.awt at 7-ea requires local public jdk.boot@=7-ea permits jdk.swing # The AWT (Bug: This module can't be required directly by apps; # need to fix that, likely by introducing another jdk.base-like # wrapper module.) jdk.swing at 7-ea requires local public jdk.boot@=7-ea requires local public jdk.awt@=7-ea jdk at 7-ea requires local public jdk.boot@=7-ea # Everything that didn't wind up in the above modules, though # right now this will get you just the JRE, not the full JDK # (i.e., no tools) This initial modularization is (very!) crude; there are, e.g., some XML and java.util.concurrent classes in the Swing module. It's good enough, however, to demonstrate that the basic Jigsaw packaging, install, and runtime components actually work. Here are the corresponding Debian packages: $ ls -l build/jigsaw-pkgs/ total 33107 -rw-r--r-- 1 mr green 6177270 May 29 10:04 jdk.awt_7_ea_all.deb -rw-r--r-- 1 mr green 1206 May 29 10:04 jdk.base_7_ea_all.deb -rw-r--r-- 1 mr green 4127386 May 29 10:10 jdk.boot_7_ea_i386.deb -rw-r--r-- 1 mr green 2777180 May 29 10:04 jdk.swing_7_ea_all.deb -rw-r--r-- 1 mr green 20774498 May 29 10:04 jdk_7_ea_all.deb $ These aren't very well compressed at the moment; in particular, they don't use pack200 for class files or lzma for binaries. The package metadata is singularly uninformative, and also incorrect, e.g., the packages should be labeled "i386" rather than "all". Dalibor is, as I write this, furiously hacking jpkg to address these problems. As to the demo, attached is the output of the current demo script, with commentary. The demo will be smoother once I get the packages properly published on a web server so that apt-get and synaptic just work. My next task will be to merge Jon's latest javac changes, after which I'll extend the demo script to show how to compile and package the "Hello, world" program, and then install it and show that it creates a /usr/bin/hello command to launch the program. - Mark -------------- next part -------------- $ $ # Do we have java here? $ $ java -version bash: java: command not found $ $ # Oops, no, let's see what we can find. $ $ ls -l pkgs/openjdk-6-*.deb -rw-r--r-- 1 mr green 25692258 Apr 14 08:05 pkgs/openjdk-6-jre-headless_6b14_i386.deb $ $ # Hmm, that's mighty big! What about this one? $ $ ls -l pkgs/jdk.boot_7_ea_i386.deb -rw-r--r-- 1 mr green 4127386 May 29 10:10 pkgs/jdk.boot_7_ea_i386.deb $ $ # Ooh, nice and compact! Let's install it. $ $ su dpkg -i pkgs/jdk.boot_7_ea_i386.deb Selecting previously deselected package jdk.boot. (Reading database ... 215229 files and directories currently installed.) Unpacking jdk.boot (from pkgs/jdk.boot_7_ea_i386.deb) ... Setting up jdk.boot (7_ea) ... $ java -version java version "1.7.0-mr_2009.05.29.1022" Java(TM) SE Runtime Environment (build 1.7.0-mr_2009.05.29.1022-b00) OpenJDK Server VM (build 15.0-b02, mixed mode) $ $ # Okay, that worked. Let's run our standard test program. $ $ cat src/org/hello/Main.java package org.hello; public class Main { public static void main(String[] args) { System.out.println("Hello, modular world!"); } } $ javac -d classes src/org/hello/Main.java $ java -cp classes org.hello.Main Hello, modular world! $ $ # Okay, what about a simple Swing application? $ $ cat src/org/hello/swing/Main.java package org.hello.swing; import java.awt.*; import java.awt.event.*; import javax.swing.*; public class Main extends JFrame implements ActionListener { public Main() { super("Hello"); setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE); setLayout(new FlowLayout(FlowLayout.CENTER, 8, 4)); add(new JLabel("Hello, world!")); JButton jb = new JButton("Good-bye"); jb.addActionListener(this); jb.setToolTipText("Click this button to exit"); add(jb); pack(); } public void actionPerformed(ActionEvent e) { System.exit(0); } public static void main(String[] args) { System.setProperty("swing.defaultlaf", "com.sun.java.swing.plaf.gtk.GTKLookAndFeel"); new Main().setVisible(true); } } $ javac -d classes src/org/hello/swing/Main.java $ java -cp classes org.hello.swing.Main Exception in thread "main" java.lang.NoClassDefFoundError: java/awt/event/ActionListener at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:642) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:336) at java.net.URLClassLoader.access$000(URLClassLoader.java:75) at java.net.URLClassLoader$1.run(URLClassLoader.java:271) at java.net.URLClassLoader$1.run(URLClassLoader.java:265) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:264) at java.lang.ClassLoader.loadClass(ClassLoader.java:325) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) at java.lang.ClassLoader.loadClass(ClassLoader.java:270) at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:209) Caused by: java.lang.ClassNotFoundException: java.awt.event.ActionListener at java.net.URLClassLoader$1.run(URLClassLoader.java:276) at java.net.URLClassLoader$1.run(URLClassLoader.java:265) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:264) at java.lang.ClassLoader.loadClass(ClassLoader.java:325) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) at java.lang.ClassLoader.loadClass(ClassLoader.java:270) ... 13 more $ $ # Oops, looks like we haven't installed Swing yet. $ $ jmod ls jdk.boot at 7-ea $ $ # Yep, that's just the boot module. Let's install Swing. $ $ su dpkg -i pkgs/jdk.awt_7_ea_all.deb pkgs/jdk.swing_7_ea_all.deb Selecting previously deselected package jdk.awt. (Reading database ... 219271 files and directories currently installed.) Unpacking jdk.awt (from pkgs/jdk.awt_7_ea_all.deb) ... Selecting previously deselected package jdk.swing. Unpacking jdk.swing (from pkgs/jdk.swing_7_ea_all.deb) ... Setting up jdk.awt (7_ea) ... Setting up jdk.swing (7_ea) ... $ $ # Which modules do we have installed now? $ $ jmod ls jdk.awt at 7-ea jdk.boot at 7-ea jdk.swing at 7-ea $ java -cp classes org.hello.swing.Main $ $ # Good! I saw a "Hello, world!" window on my screen, $ # with a "Good-bye" button, as expected. $ From mr at sun.com Fri May 29 16:21:45 2009 From: mr at sun.com (mr at sun.com) Date: Fri, 29 May 2009 23:21:45 +0000 Subject: hg: jigsaw/jigsaw/jdk: Handle synthetic platform dependences inserted by javac Message-ID: <20090529232207.67FB3EDC8@hg.openjdk.java.net> Changeset: c70945244192 Author: mr Date: 2009-05-29 16:18 -0700 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/c70945244192 Handle synthetic platform dependences inserted by javac ! src/share/classes/java/lang/module/Dependence.java ! src/share/classes/org/openjdk/jigsaw/Platform.java ! src/share/classes/org/openjdk/jigsaw/Resolver.java ! test/java/lang/module/_ModuleInfoReader.java From dalibor.topic at sun.com Fri May 29 19:52:50 2009 From: dalibor.topic at sun.com (dalibor.topic at sun.com) Date: Sat, 30 May 2009 02:52:50 +0000 Subject: hg: jigsaw/jigsaw/jdk: Use lzma compression for the debian packages by default Message-ID: <20090530025310.264ECEDE3@hg.openjdk.java.net> Changeset: 826e0fb90851 Author: robilad Date: 2009-05-30 04:30 +0200 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/826e0fb90851 Use lzma compression for the debian packages by default ! src/share/classes/org/openjdk/jigsaw/cli/Packager.java From Dalibor.Topic at Sun.COM Fri May 29 19:55:33 2009 From: Dalibor.Topic at Sun.COM (Dalibor Topic) Date: Sat, 30 May 2009 04:55:33 +0200 Subject: hg: jigsaw/jigsaw/jdk: Use lzma compression for the debian packages by default In-Reply-To: <20090530025310.264ECEDE3@hg.openjdk.java.net> References: <20090530025310.264ECEDE3@hg.openjdk.java.net> Message-ID: <4A20A025.2030800@Sun.COM> On 05/30/09 04:52, Dalibor.Topic at Sun.COM wrote: > Changeset: 826e0fb90851 > Author: robilad > Date: 2009-05-30 04:30 +0200 > URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/826e0fb90851 > > Use lzma compression for the debian packages by default > > ! src/share/classes/org/openjdk/jigsaw/cli/Packager.java > > This oneliner shaves 6 M off the 20 M in the jdk7_ea package. Transparently. Yay for dpkg-deb & lzma! Next: pack200. cheers, dalibor topic -- ******************************************************************* Dalibor Topic Tel: (+49 40) 23 646 738 Java F/OSS Ambassador AIM: robiladonaim Sun Microsystems GmbH Mobile: (+49 177) 2664 192 Nagelsweg 55 http://openjdk.java.net D-20097 Hamburg mailto:Dalibor.Topic at sun.com Sitz der Gesellschaft: Sonnenallee 1, D-85551 Kirchheim-Heimstetten Amtsgericht M?nchen: HRB 161028 Gesch?ftsf?hrer: Thomas Schr?der, Wolfgang Engels, Dr. Roland B?mer Vorsitzender des Aufsichtsrates: Martin H?ring From dalibor.topic at sun.com Fri May 29 20:08:08 2009 From: dalibor.topic at sun.com (dalibor.topic at sun.com) Date: Sat, 30 May 2009 03:08:08 +0000 Subject: hg: jigsaw/jigsaw/jdk: Removed Architecture field from generated Debian control file Message-ID: <20090530030820.90EB3EDE8@hg.openjdk.java.net> Changeset: a694f5389b98 Author: robilad Date: 2009-05-30 05:04 +0200 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/a694f5389b98 Removed Architecture field from generated Debian control file ! src/share/classes/org/openjdk/jigsaw/cli/Packager.java From dalibor.topic at sun.com Fri May 29 21:30:26 2009 From: dalibor.topic at sun.com (dalibor.topic at sun.com) Date: Sat, 30 May 2009 04:30:26 +0000 Subject: hg: jigsaw/jigsaw/jdk: Use exec for launcher and pass on given parameters Message-ID: <20090530043045.1410AEDFA@hg.openjdk.java.net> Changeset: e4d34adcd44c Author: robilad Date: 2009-05-30 06:25 +0200 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/e4d34adcd44c Use exec for launcher and pass on given parameters ! src/share/classes/org/openjdk/jigsaw/cli/Packager.java From dalibor.topic at sun.com Fri May 29 22:28:08 2009 From: dalibor.topic at sun.com (dalibor.topic at sun.com) Date: Sat, 30 May 2009 05:28:08 +0000 Subject: hg: jigsaw/jigsaw/jdk: Use java.home instead of JAVA_HOME Message-ID: <20090530052824.4A972EDFF@hg.openjdk.java.net> Changeset: bc6b01bdfea5 Author: robilad Date: 2009-05-30 07:24 +0200 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/bc6b01bdfea5 Use java.home instead of JAVA_HOME ! src/share/classes/org/openjdk/jigsaw/cli/Packager.java From dalibor.topic at sun.com Fri May 29 23:48:51 2009 From: dalibor.topic at sun.com (dalibor.topic at sun.com) Date: Sat, 30 May 2009 06:48:51 +0000 Subject: hg: jigsaw/jigsaw/jdk: Use java.home to find jmod Message-ID: <20090530064909.0F8EEEE04@hg.openjdk.java.net> Changeset: 1da16f9838f6 Author: robilad Date: 2009-05-30 08:45 +0200 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/1da16f9838f6 Use java.home to find jmod ! src/share/classes/org/openjdk/jigsaw/cli/Packager.java From dalibor.topic at sun.com Sat May 30 00:10:38 2009 From: dalibor.topic at sun.com (dalibor.topic at sun.com) Date: Sat, 30 May 2009 07:10:38 +0000 Subject: hg: jigsaw/jigsaw/jdk: Use maximum lzma compression Message-ID: <20090530071052.F36F1EE0E@hg.openjdk.java.net> Changeset: 05bf2ba392f7 Author: robilad Date: 2009-05-30 09:06 +0200 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/05bf2ba392f7 Use maximum lzma compression ! src/share/classes/org/openjdk/jigsaw/cli/Packager.java From dalibor.topic at sun.com Sat May 30 05:33:27 2009 From: dalibor.topic at sun.com (dalibor.topic at sun.com) Date: Sat, 30 May 2009 12:33:27 +0000 Subject: hg: jigsaw/jigsaw/jdk: Compress modules using pack200 Message-ID: <20090530123353.509C6EE13@hg.openjdk.java.net> Changeset: b2639f1fed75 Author: robilad Date: 2009-05-30 14:28 +0200 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/b2639f1fed75 Compress modules using pack200 ! src/share/classes/org/openjdk/jigsaw/cli/Packager.java From Dalibor.Topic at Sun.COM Sat May 30 05:39:56 2009 From: Dalibor.Topic at Sun.COM (Dalibor Topic) Date: Sat, 30 May 2009 14:39:56 +0200 Subject: Modules and packages and demos In-Reply-To: <20090529200637.D44A35E98@eggemoggin.niobe.net> References: <20090529200637.D44A35E98@eggemoggin.niobe.net> Message-ID: <4A21291C.6070304@Sun.COM> On 05/29/09 22:06, Mark Reinhold wrote: > Here are the corresponding Debian packages: > > $ ls -l build/jigsaw-pkgs/ > total 33107 > -rw-r--r-- 1 mr green 6177270 May 29 10:04 jdk.awt_7_ea_all.deb > -rw-r--r-- 1 mr green 1206 May 29 10:04 jdk.base_7_ea_all.deb > -rw-r--r-- 1 mr green 4127386 May 29 10:10 jdk.boot_7_ea_i386.deb > -rw-r--r-- 1 mr green 2777180 May 29 10:04 jdk.swing_7_ea_all.deb > -rw-r--r-- 1 mr green 20774498 May 29 10:04 jdk_7_ea_all.deb > $ > > These aren't very well compressed at the moment; in particular, they > don't use pack200 for class files or lzma for binaries. LZMA compression and pack200 compression have just been implemented. Total size of packages for different modules taken together now is about half of the numbers above, i.e. around 18M. That compares (in a somewhat orange apple flavored way) quite favorably with the around 78M for the 6u14 binary JDK RPM installer for Linux, for example. > The package > metadata is singularly uninformative, and also incorrect, e.g., the > packages should be labeled "i386" rather than "all". Dalibor is, as > I write this, furiously hacking jpkg to address these problems. > Yeah, I've temporarily removed the architecture field from the control file. The bugfix for it is forthcoming in a couple of hours. Along with a few more simple to implement arguments to specify maintainer name, e-mail, short and long descriptions of the software, the architecture, and additional metadata in general. cheers, dalibor topic -- ******************************************************************* Dalibor Topic Tel: (+49 40) 23 646 738 Java F/OSS Ambassador AIM: robiladonaim Sun Microsystems GmbH Mobile: (+49 177) 2664 192 Nagelsweg 55 http://openjdk.java.net D-20097 Hamburg mailto:Dalibor.Topic at sun.com Sitz der Gesellschaft: Sonnenallee 1, D-85551 Kirchheim-Heimstetten Amtsgericht M?nchen: HRB 161028 Gesch?ftsf?hrer: Thomas Schr?der, Wolfgang Engels, Dr. Roland B?mer Vorsitzender des Aufsichtsrates: Martin H?ring From Dalibor.Topic at Sun.COM Sat May 30 05:47:02 2009 From: Dalibor.Topic at Sun.COM (Dalibor Topic) Date: Sat, 30 May 2009 14:47:02 +0200 Subject: Demo status, and a couple of jpkg enhancement requests In-Reply-To: <20090528054014.5A22B94A8@callebaut.niobe.net> References: <20090528054014.5A22B94A8@callebaut.niobe.net> Message-ID: <4A212AC6.6010702@Sun.COM> On 05/28/09 07:40, Mark Reinhold wrote: > Dalibor -- Any chance of getting these two jpkg enhancements done > in the next day or so? > > - Compress class files using pack200 rather than simple gzip. > This would make for a huge improvement in our package sizes. > It probably makes sense to leave packaged classes in the JAR > format. If so, I can easily change the runtime code to look > for JAR files in addition to classes/ directories. You can > assume that $JAVA_HOME/bin contains the unpack200 binary. > > (When packing I wouldn't bother to compress the output file; > dpkg-deb will compress it with gzip anyway.) > I've gone for the simplest solution - wrap module in jar file, pack that (without gzipping it), and let dpkg-deb do the LZMA magic on it. Upon installation, the packed file is temporarily turned into a jar file, which is extracted, and upon completion, thrown away, so we end up with classes on disk. > - Define additional command-line options for specifying some > of the more common package attributes such as maintainer, > description, section, installed-size, and maybe home page. > Right now the JDK packages show up in Synaptic in a fairly > boring way, without the information people expect. > Working on it. Should be fairly easy to get right. > Fixes for the launcher-script issues would be welcome too: Use the > jmod in the installed $JAVA_HOME/bin, and exec rather than fork the > java binary. (I have a tiny patch to fix jpkg's translation of > version strings, which I'll include in my next push.) > Thank you for the patch, fixes for the issues have been committed. cheers, dalibor topic -- ******************************************************************* Dalibor Topic Tel: (+49 40) 23 646 738 Java F/OSS Ambassador AIM: robiladonaim Sun Microsystems GmbH Mobile: (+49 177) 2664 192 Nagelsweg 55 http://openjdk.java.net D-20097 Hamburg mailto:Dalibor.Topic at sun.com Sitz der Gesellschaft: Sonnenallee 1, D-85551 Kirchheim-Heimstetten Amtsgericht M?nchen: HRB 161028 Gesch?ftsf?hrer: Thomas Schr?der, Wolfgang Engels, Dr. Roland B?mer Vorsitzender des Aufsichtsrates: Martin H?ring From dalibor.topic at sun.com Sat May 30 05:47:10 2009 From: dalibor.topic at sun.com (dalibor.topic at sun.com) Date: Sat, 30 May 2009 12:47:10 +0000 Subject: hg: jigsaw/jigsaw/jdk: Fixed small typo Message-ID: <20090530124728.9F0FFEE18@hg.openjdk.java.net> Changeset: 679ced810517 Author: robilad Date: 2009-05-30 14:41 +0200 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/679ced810517 Fixed small typo ! src/share/classes/org/openjdk/jigsaw/cli/Packager.java From dalibor.topic at sun.com Sat May 30 14:55:23 2009 From: dalibor.topic at sun.com (dalibor.topic at sun.com) Date: Sat, 30 May 2009 21:55:23 +0000 Subject: hg: jigsaw/jigsaw/jdk: Added -n option for maintainer name Message-ID: <20090530215547.098AEEE1D@hg.openjdk.java.net> Changeset: 9616525cb762 Author: robilad Date: 2009-05-30 23:50 +0200 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/9616525cb762 Added -n option for maintainer name ! src/share/classes/org/openjdk/jigsaw/cli/Packager.java From dalibor.topic at sun.com Sat May 30 15:36:14 2009 From: dalibor.topic at sun.com (dalibor.topic at sun.com) Date: Sat, 30 May 2009 22:36:14 +0000 Subject: hg: jigsaw/jigsaw/jdk: Added -e option for maintainer e-mail Message-ID: <20090530223635.167CEEE25@hg.openjdk.java.net> Changeset: d6ae6f4ee14a Author: robilad Date: 2009-05-31 00:32 +0200 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/d6ae6f4ee14a Added -e option for maintainer e-mail ! src/share/classes/org/openjdk/jigsaw/cli/Packager.java From dalibor.topic at sun.com Sat May 30 17:39:24 2009 From: dalibor.topic at sun.com (dalibor.topic at sun.com) Date: Sun, 31 May 2009 00:39:24 +0000 Subject: hg: jigsaw/jigsaw/jdk: Added -s option for short descriptions Message-ID: <20090531003944.C5E23EE2A@hg.openjdk.java.net> Changeset: dfa40eccd846 Author: robilad Date: 2009-05-31 02:30 +0200 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/dfa40eccd846 Added -s option for short descriptions ! src/share/classes/org/openjdk/jigsaw/cli/Packager.java From dalibor.topic at sun.com Sat May 30 18:28:43 2009 From: dalibor.topic at sun.com (dalibor.topic at sun.com) Date: Sun, 31 May 2009 01:28:43 +0000 Subject: hg: jigsaw/jigsaw/jdk: Added -l option for long descriptions Message-ID: <20090531012857.D83FAEE2F@hg.openjdk.java.net> Changeset: 7c419683a65a Author: robilad Date: 2009-05-31 03:25 +0200 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/7c419683a65a Added -l option for long descriptions ! src/share/classes/org/openjdk/jigsaw/cli/Packager.java From dalibor.topic at sun.com Sat May 30 18:42:45 2009 From: dalibor.topic at sun.com (dalibor.topic at sun.com) Date: Sun, 31 May 2009 01:42:45 +0000 Subject: hg: jigsaw/jigsaw/jdk: Use os.arch for Architecture Message-ID: <20090531014301.2A771EE34@hg.openjdk.java.net> Changeset: 50e7dda87f3c Author: robilad Date: 2009-05-31 03:39 +0200 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/50e7dda87f3c Use os.arch for Architecture ! src/share/classes/org/openjdk/jigsaw/cli/Packager.java From dalibor.topic at sun.com Sat May 30 21:42:28 2009 From: dalibor.topic at sun.com (dalibor.topic at sun.com) Date: Sun, 31 May 2009 04:42:28 +0000 Subject: hg: jigsaw/jigsaw/jdk: Added -x option for additional metadata Message-ID: <20090531044242.638C1EE3F@hg.openjdk.java.net> Changeset: ce7663146d67 Author: robilad Date: 2009-05-31 06:38 +0200 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/ce7663146d67 Added -x option for additional metadata ! src/share/classes/org/openjdk/jigsaw/cli/Packager.java From Dalibor.Topic at Sun.COM Sat May 30 21:56:14 2009 From: Dalibor.Topic at Sun.COM (Dalibor Topic) Date: Sun, 31 May 2009 06:56:14 +0200 Subject: Demo status, and a couple of jpkg enhancement requests In-Reply-To: <4A212AC6.6010702@Sun.COM> References: <20090528054014.5A22B94A8@callebaut.niobe.net> <4A212AC6.6010702@Sun.COM> Message-ID: <4A220DEE.3000300@Sun.COM> On 05/30/09 14:47, Dalibor Topic wrote: > On 05/28/09 07:40, Mark Reinhold wrote: >> - Define additional command-line options for specifying some >> of the more common package attributes such as maintainer, >> description, section, installed-size, and maybe home page. >> Right now the JDK packages show up in Synaptic in a fairly >> boring way, without the information people expect. >> > Working on it. Should be fairly easy to get right. > I've added a bunch of new options to express various bits of non-derivable metadata, like maintainer name, e-mail address, etc. I also added a generic -x filename switch, that is interpreted depending on the packaging system - for debian packages it means please add this file's contents to the DEBIAN/control file. That effectively allows the use of conrol file templates for packaging purposes, where just the derivable information is filled in by jpkg from the contents of the modules, and everything else can be (but doesn't have to be) supplied by the user. cheers, dalibor topic -- ******************************************************************* Dalibor Topic Tel: (+49 40) 23 646 738 Java F/OSS Ambassador AIM: robiladonaim Sun Microsystems GmbH Mobile: (+49 177) 2664 192 Nagelsweg 55 http://openjdk.java.net D-20097 Hamburg mailto:Dalibor.Topic at sun.com Sitz der Gesellschaft: Sonnenallee 1, D-85551 Kirchheim-Heimstetten Amtsgericht M?nchen: HRB 161028 Gesch?ftsf?hrer: Thomas Schr?der, Wolfgang Engels, Dr. Roland B?mer Vorsitzender des Aufsichtsrates: Martin H?ring