From erik.joelsson at oracle.com Thu Mar 21 03:58:29 2013 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 21 Mar 2013 11:58:29 +0100 Subject: RFR: Post-modules and module images build in new buildsystem Message-ID: <514AE7D5.20703@oracle.com> I have converted the post modularization step from the old build system to the new build system in jigsaw. http://cr.openjdk.java.net/~erikj/jigsaw/webrev.post-modules.01/ The images target will now produce the same module and base images as the old build. The legacy images can still be built using the new target "legacy-images". All the post modularization steps happen in the target "post-modules". I have verified the build manually on linux, windows, solaris and mac and also run it successfully through jprt (with a limited set of tests). I've compared the contents of the built modules with the old build (see below for differences). Known issues: * Fastdebug fails, both in old and new build, when running the newly built rmic at the end of the jdk target. * No fast incremental support in the post-modules or images targets. Trying to fix that instead of working on a real modular build seems like a waste of time. * The following differences with the old build have been observed: * Debug info files (*.diz) from hotspot, but not from the jdk, are included in the modules in the old build. I chose to treat all *.diz files the same and chose to exclude them for now. * The swingbeans (dt.jar) are generated during images creation in the old build, but in the new build those classes are compiled with the rest, making them available to the modularization. This makes them present in the jdk.desktop module in the new build. * Similarly META-INF/services/com.sun.tools.xjc.Plugin is generated earlier in the new build and is now present in jdk.tools.jaxws. * sun/text/resources/th/BreakIteratorRules_th.class is showing up in sun.localedata. /Erik From Alan.Bateman at oracle.com Thu Mar 21 06:02:59 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 21 Mar 2013 13:02:59 +0000 Subject: RFR: Post-modules and module images build in new buildsystem In-Reply-To: <514AE7D5.20703@oracle.com> References: <514AE7D5.20703@oracle.com> Message-ID: <514B0503.5090505@oracle.com> On 21/03/2013 10:58, Erik Joelsson wrote: > I have converted the post modularization step from the old build > system to the new build system in jigsaw. > > http://cr.openjdk.java.net/~erikj/jigsaw/webrev.post-modules.01/ > > The images target will now produce the same module and base images as > the old build. The legacy images can still be built using the new > target "legacy-images". All the post modularization steps happen in > the target "post-modules". I have verified the build manually on > linux, windows, solaris and mac and also run it successfully through > jprt (with a limited set of tests). I've compared the contents of the > built modules with the old build (see below for differences). This is great as we need to get moved off the old build system - thank you! I haven't looked through all the changes yet but it may be that we just bring these in now so that we can switch to the new build. Mandy - do you agree? Mandy might wonder about the class analyzer skipping _the files but I think that is fine for now. Do you forsee any issues or complications with future merges? It's possible we might be using post-modules for some time. > > Known issues: > > * Fastdebug fails, both in old and new build, when running the newly > built rmic at the end of the jdk target. I think that's okay as in general, we need to get the rmic usage fixed in the build anyway. > * No fast incremental support in the post-modules or images targets. > Trying to fix that instead of working on a real modular build seems > like a waste of time. I agree. > * The following differences with the old build have been observed: > * Debug info files (*.diz) from hotspot, but not from the jdk, are > included in the modules in the old build. I chose to treat all *.diz > files the same and chose to exclude them for now. We haven't fully explored debug modules so this will need to be re-examined anyway. A possible approach is that there is a -debug module for each JDK module with native code. > * The swingbeans (dt.jar) are generated during images creation in > the old build, but in the new build those classes are compiled with > the rest, making them available to the modularization. This makes them > present in the jdk.desktop module in the new build. This is okay too as this needs to be re-examined and dt.jar eliminated. > * Similarly META-INF/services/com.sun.tools.xjc.Plugin is generated > earlier in the new build and is now present in jdk.tools.jaxws. It's good to have this fixed. We have the "provides service" right and not clear, without digging, why the legacy configuration file was missed in the old build. > * sun/text/resources/th/BreakIteratorRules_th.class is showing up in > sun.localedata. That seems right. -Alan. From erik.joelsson at oracle.com Thu Mar 21 06:43:38 2013 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 21 Mar 2013 14:43:38 +0100 Subject: RFR: Post-modules and module images build in new buildsystem In-Reply-To: <514B0503.5090505@oracle.com> References: <514AE7D5.20703@oracle.com> <514B0503.5090505@oracle.com> Message-ID: <514B0E8A.5060907@oracle.com> On 2013-03-21 14:02, Alan Bateman wrote: > On 21/03/2013 10:58, Erik Joelsson wrote: >> I have converted the post modularization step from the old build >> system to the new build system in jigsaw. >> >> http://cr.openjdk.java.net/~erikj/jigsaw/webrev.post-modules.01/ >> >> The images target will now produce the same module and base images as >> the old build. The legacy images can still be built using the new >> target "legacy-images". All the post modularization steps happen in >> the target "post-modules". I have verified the build manually on >> linux, windows, solaris and mac and also run it successfully through >> jprt (with a limited set of tests). I've compared the contents of the >> built modules with the old build (see below for differences). > This is great as we need to get moved off the old build system - thank > you! > > I haven't looked through all the changes yet but it may be that we > just bring these in now so that we can switch to the new build. Mandy > - do you agree? Mandy might wonder about the class analyzer skipping > _the files but I think that is fine for now. I'm open to suggestions here, but in the new build, files starting with _the are touch files and similar, just handling book keeping for make, that shouldn't be put in the product. > > Do you forsee any issues or complications with future merges? It's > possible we might be using post-modules for some time. > I'm trying my best to keep changes small to not complicate merges unnecessarily, but I suspect some of this could get hairy down the road. /Erik >> >> Known issues: >> >> * Fastdebug fails, both in old and new build, when running the newly >> built rmic at the end of the jdk target. > I think that's okay as in general, we need to get the rmic usage fixed > in the build anyway. > >> * No fast incremental support in the post-modules or images targets. >> Trying to fix that instead of working on a real modular build seems >> like a waste of time. > I agree. > >> * The following differences with the old build have been observed: >> * Debug info files (*.diz) from hotspot, but not from the jdk, are >> included in the modules in the old build. I chose to treat all *.diz >> files the same and chose to exclude them for now. > We haven't fully explored debug modules so this will need to be > re-examined anyway. A possible approach is that there is a -debug > module for each JDK module with native code. > >> * The swingbeans (dt.jar) are generated during images creation in >> the old build, but in the new build those classes are compiled with >> the rest, making them available to the modularization. This makes >> them present in the jdk.desktop module in the new build. > This is okay too as this needs to be re-examined and dt.jar eliminated. > >> * Similarly META-INF/services/com.sun.tools.xjc.Plugin is generated >> earlier in the new build and is now present in jdk.tools.jaxws. > It's good to have this fixed. We have the "provides service" right and > not clear, without digging, why the legacy configuration file was > missed in the old build. > >> * sun/text/resources/th/BreakIteratorRules_th.class is showing up >> in sun.localedata. > That seems right. > > -Alan. From mandy.chung at oracle.com Thu Mar 21 15:30:40 2013 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 21 Mar 2013 15:30:40 -0700 Subject: RFR: Post-modules and module images build in new buildsystem In-Reply-To: <514B0503.5090505@oracle.com> References: <514AE7D5.20703@oracle.com> <514B0503.5090505@oracle.com> Message-ID: <514B8A10.5040803@oracle.com> On 3/21/13 6:02 AM, Alan Bateman wrote: > On 21/03/2013 10:58, Erik Joelsson wrote: >> I have converted the post modularization step from the old build >> system to the new build system in jigsaw. >> >> http://cr.openjdk.java.net/~erikj/jigsaw/webrev.post-modules.01/ >> >> The images target will now produce the same module and base images as >> the old build. The legacy images can still be built using the new >> target "legacy-images". All the post modularization steps happen in >> the target "post-modules". I have verified the build manually on >> linux, windows, solaris and mac and also run it successfully through >> jprt (with a limited set of tests). I've compared the contents of the >> built modules with the old build (see below for differences). > This is great as we need to get moved off the old build system - thank > you! Indeed a big thank you! This is great and we can build jigsaw with the new build! One less dependency on the old build. > > I haven't looked through all the changes yet but it may be that we > just bring these in now so that we can switch to the new build. Mandy > - do you agree? Mandy might wonder about the class analyzer skipping > _the files but I think that is fine for now. I agree. We should switch to the new build. jigsaw/Makefile currently still uses the old build as the default. Can you also try building with the closed repos? > > Do you forsee any issues or complications with future merges? It's > possible we might be using post-modules for some time. > jigsaw is currently sync'ed with jdk8 b76. There are other build changes in jdk8 and we will find out from our next merge if there is any complication. This is good work and we can improve it over time. I support pushing this change. Mandy >> >> Known issues: >> >> * Fastdebug fails, both in old and new build, when running the newly >> built rmic at the end of the jdk target. > I think that's okay as in general, we need to get the rmic usage fixed > in the build anyway. > >> * No fast incremental support in the post-modules or images targets. >> Trying to fix that instead of working on a real modular build seems >> like a waste of time. > I agree. > >> * The following differences with the old build have been observed: >> * Debug info files (*.diz) from hotspot, but not from the jdk, are >> included in the modules in the old build. I chose to treat all *.diz >> files the same and chose to exclude them for now. > We haven't fully explored debug modules so this will need to be > re-examined anyway. A possible approach is that there is a -debug > module for each JDK module with native code. > >> * The swingbeans (dt.jar) are generated during images creation in >> the old build, but in the new build those classes are compiled with >> the rest, making them available to the modularization. This makes >> them present in the jdk.desktop module in the new build. > This is okay too as this needs to be re-examined and dt.jar eliminated. > >> * Similarly META-INF/services/com.sun.tools.xjc.Plugin is generated >> earlier in the new build and is now present in jdk.tools.jaxws. > It's good to have this fixed. We have the "provides service" right and > not clear, without digging, why the legacy configuration file was > missed in the old build. > >> * sun/text/resources/th/BreakIteratorRules_th.class is showing up >> in sun.localedata. > That seems right. > > -Alan. From erik.joelsson at oracle.com Fri Mar 22 01:31:08 2013 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Fri, 22 Mar 2013 09:31:08 +0100 Subject: RFR: Post-modules and module images build in new buildsystem In-Reply-To: <514B8A10.5040803@oracle.com> References: <514AE7D5.20703@oracle.com> <514B0503.5090505@oracle.com> <514B8A10.5040803@oracle.com> Message-ID: <514C16CC.4090901@oracle.com> On 2013-03-21 23:30, Mandy Chung wrote: > On 3/21/13 6:02 AM, Alan Bateman wrote: >> On 21/03/2013 10:58, Erik Joelsson wrote: >>> I have converted the post modularization step from the old build >>> system to the new build system in jigsaw. >>> >>> http://cr.openjdk.java.net/~erikj/jigsaw/webrev.post-modules.01/ >>> >>> The images target will now produce the same module and base images >>> as the old build. The legacy images can still be built using the new >>> target "legacy-images". All the post modularization steps happen in >>> the target "post-modules". I have verified the build manually on >>> linux, windows, solaris and mac and also run it successfully through >>> jprt (with a limited set of tests). I've compared the contents of >>> the built modules with the old build (see below for differences). >> This is great as we need to get moved off the old build system - >> thank you! > > Indeed a big thank you! This is great and we can build jigsaw with > the new build! One less dependency on the old build. >> >> I haven't looked through all the changes yet but it may be that we >> just bring these in now so that we can switch to the new build. Mandy >> - do you agree? Mandy might wonder about the class analyzer skipping >> _the files but I think that is fine for now. > > I agree. We should switch to the new build. jigsaw/Makefile > currently still uses the old build as the default. Can you also try > building with the closed repos? > My jprt run was with closed, but not deploy and install. Are those buildable with jigsaw in the old build? I'm running another job in jprt now with just open to make sure that too works on all platforms. >> Do you forsee any issues or complications with future merges? It's >> possible we might be using post-modules for some time. >> > > jigsaw is currently sync'ed with jdk8 b76. There are other build > changes in jdk8 and we will find out from our next merge if there is > any complication. This is good work and we can improve it over > time. I support pushing this change. > Let me know if you need any help with the merge. /Erik From alan.bateman at oracle.com Sat Mar 23 14:11:35 2013 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Sat, 23 Mar 2013 21:11:35 +0000 Subject: hg: jigsaw/jigsaw: Post-modules and module images build in new buildsystem Message-ID: <20130323211135.BD9D34837D@hg.openjdk.java.net> Changeset: e6cffce41bd8 Author: alanb Date: 2013-03-23 21:09 +0000 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/rev/e6cffce41bd8 Post-modules and module images build in new buildsystem Contributed-by: erik.joelsson at oracle.com ! Makefile ! common/autoconf/boot-jdk.m4 ! common/autoconf/configure.ac ! common/autoconf/generated-configure.sh ! common/autoconf/source-dirs.m4 ! common/autoconf/spec.gmk.in ! common/bin/compare.sh ! common/makefiles/Jprt.gmk ! common/makefiles/Main.gmk From alan.bateman at oracle.com Sat Mar 23 14:12:34 2013 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Sat, 23 Mar 2013 21:12:34 +0000 Subject: hg: jigsaw/jigsaw/jdk: Post-modules and module images build in new buildsystem Message-ID: <20130323211314.56AC34837E@hg.openjdk.java.net> Changeset: 4df6759a08ac Author: alanb Date: 2013-03-23 21:09 +0000 URL: http://hg.openjdk.java.net/jigsaw/jigsaw/jdk/rev/4df6759a08ac Post-modules and module images build in new buildsystem Contributed-by: erik.joelsson at oracle.com ! make/tools/classanalyzer/src/com/sun/classanalyzer/ClassAnalyzer.java ! makefiles/BuildJdk.gmk + makefiles/BuildNativePackages.gmk ! makefiles/CompileJavaClasses.gmk ! makefiles/CompileLaunchers.gmk ! makefiles/CompileNativeLibraries.gmk ! makefiles/CopyFiles.gmk ! makefiles/CreateJars.gmk ! makefiles/GendataFontConfig.gmk ! makefiles/GendataTimeZone.gmk ! makefiles/GenerateData.gmk + makefiles/GenerateModules.gmk + makefiles/GensrcModuleInfo.gmk ! makefiles/Import.gmk + makefiles/ModuleImages.gmk + makefiles/ModulesCommon.gmk + makefiles/ModulesTools.gmk ! makefiles/Setup.gmk ! makefiles/SignJars.gmk From erik.joelsson at oracle.com Thu Mar 28 01:51:38 2013 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 28 Mar 2013 09:51:38 +0100 Subject: The role of submodules Message-ID: <5154049A.9000901@oracle.com> Hello, While working with the jigsaw build, I can't help but wonder what the reason for the submodules concept is. In the old build, each area is assigned a "MODULE", which isn't a full module name, but something shorter. All artifacts produced in an area, like libraries and lib files, automatically get copied to a sub directory of the submodules outputdir. Then in the modularization step, the contents of these submodules is copied to the final (real) modules according to a configured pattern. (ex: base -> jdk.base, javac -> jdk.tools) Is there a reason to keep this model going forward, or can we assign things their final module name directly in the new build? I would like to minimize copying files around for no apparent reason if possible. /Erik From chris.hegarty at oracle.com Thu Mar 28 02:37:08 2013 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 28 Mar 2013 09:37:08 +0000 Subject: The role of submodules In-Reply-To: <5154049A.9000901@oracle.com> References: <5154049A.9000901@oracle.com> Message-ID: <51540F44.5080309@oracle.com> I believe the "submodules" are the fine-grained modules [1]. Not sure if they are still desirable. -Chris. [1] http://openjdk.java.net/projects/jigsaw/doc/draft-java-module-system-requirements-12#fine-grained-java-se-modularization On 03/28/2013 08:51 AM, Erik Joelsson wrote: > Hello, > > While working with the jigsaw build, I can't help but wonder what the > reason for the submodules concept is. In the old build, each area is > assigned a "MODULE", which isn't a full module name, but something > shorter. All artifacts produced in an area, like libraries and lib > files, automatically get copied to a sub directory of the submodules > outputdir. Then in the modularization step, the contents of these > submodules is copied to the final (real) modules according to a > configured pattern. (ex: base -> jdk.base, javac -> jdk.tools) > > Is there a reason to keep this model going forward, or can we assign > things their final module name directly in the new build? I would like > to minimize copying files around for no apparent reason if possible. > > /Erik From mandy.chung at oracle.com Thu Mar 28 12:02:58 2013 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 28 Mar 2013 12:02:58 -0700 Subject: The role of submodules In-Reply-To: <5154049A.9000901@oracle.com> References: <5154049A.9000901@oracle.com> Message-ID: <515493E2.9030500@oracle.com> On 3/28/13 1:51 AM, Erik Joelsson wrote: > Hello, > > While working with the jigsaw build, I can't help but wonder what the > reason for the submodules concept is. In the old build, each area is > assigned a "MODULE", which isn't a full module name, but something > shorter. All artifacts produced in an area, like libraries and lib > files, automatically get copied to a sub directory of the submodules > outputdir. Then in the modularization step, the contents of these > submodules is copied to the final (real) modules according to a > configured pattern. (ex: base -> jdk.base, javac -> jdk.tools) > When we started the modules build, the module graph was not clean to begin with. We grouped classes in a fine-grained module to allow us to do experimentation and fine-tuning the module graph by aggregating more than one small modules (submodules) into one. It also makes it easier for us to determine any undesirable dependency across submodules that may become a non-issue when those submodules are relevant and grouped as a single module. > Is there a reason to keep this model going forward, or can we assign > things their final module name directly in the new build? I would like > to minimize copying files around for no apparent reason if possible. I don't see the need to keep the submodules and we can use the modules currently defined in the new build as you suggest as the module graph is looking good and clean. One thing to keep in mind is that there will likely be some change to the module graph as it is just a prototype and we may need to easily update the module assignment before we do the source restructuring. Thanks Mandy