From erik.joelsson at oracle.com Tue Apr 2 02:04:35 2013 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Tue, 02 Apr 2013 11:04:35 +0200 Subject: The role of submodules In-Reply-To: <515493E2.9030500@oracle.com> References: <5154049A.9000901@oracle.com> <515493E2.9030500@oracle.com> Message-ID: <515A9F23.20205@oracle.com> On 2013-03-28 20:02, Mandy Chung wrote: > On 3/28/13 1:51 AM, Erik Joelsson wrote: >> Hello, >> >> While working with the jigsaw build, I can't help but wonder what the >> reason for the submodules concept is. In the old build, each area is >> assigned a "MODULE", which isn't a full module name, but something >> shorter. All artifacts produced in an area, like libraries and lib >> files, automatically get copied to a sub directory of the submodules >> outputdir. Then in the modularization step, the contents of these >> submodules is copied to the final (real) modules according to a >> configured pattern. (ex: base -> jdk.base, javac -> jdk.tools) >> > > When we started the modules build, the module graph was not clean to > begin with. We grouped classes in a fine-grained module to allow us > to do experimentation and fine-tuning the module graph by aggregating > more than one small modules (submodules) into one. It also makes it > easier for us to determine any undesirable dependency across > submodules that may become a non-issue when those submodules are > relevant and grouped as a single module. > >> Is there a reason to keep this model going forward, or can we assign >> things their final module name directly in the new build? I would >> like to minimize copying files around for no apparent reason if >> possible. > > I don't see the need to keep the submodules and we can use the modules > currently defined in the new build as you suggest as the module graph > is looking good and clean. One thing to keep in mind is that there > will likely be some change to the module graph as it is just a > prototype and we may need to easily update the module assignment > before we do the source restructuring. > That sounds good to me. The design I have assumed for now will look like this. There are two ways to build: 1. The current new build as was recently submitted. Building in legacy mode and then creating modules in a post step. 2. Building from the shuffled source (in */newsrc directories) into modules directly. The first build is needed as input for the source shuffling scripts. The second can only run after shuffling the source. There is a configure switch to choose between the build variants. The default is 2 if restructured source is available and 1 if it isn't. I'm striving to make these builds coexist with minimal changes to ease integrations from jdk8. I'm hoping to post a webrev soon (maybe this week) so you can give it a spin yourselves. So far, this is what build 2 is doing: 1. Build langtools, almost same as in 1. In this step, langtools is compiled twice. A. Bootstrap1. Boot jdk with boot javac is used to compile. Module support is execluded. Target class files runnable on boot jdk. B. Bootstrap2. Boot jdk with Bootstrap1 javac is used to compile. Module support is enabled. Genstubs is used to pull in the needed classes from jdk repo. Classfiles generated are for the target jdk. The class files from bootstrap2 could be put in the final product, but for technical reasons explained below, they aren't. Bootstrap2 will be used to compile the rest of the jdk later. 2. Build hotspot, just as in 1. 3. Generate java source and idls in corba. 4. Enter jdk. Import from hotspot into $(JDK_OUTPUTDIR)/lib. Import classes.jar from langtools (these are the bootstrap2 classes) into $(JDK_OUTPUTDIR)/classes dir. (This is just like build 1 is doing it). Compile the build tools, generate all java source and data. 5. Compile the classes in jdk.base using bootstrap1 (no module awareness) into $(JDK_OUTPUTDIR)/classes. Compile the libraries in jdk.base into $(JDK_OUTPUTDIR)/lib. Copy the files needed into $(JDK_OUTPUTDIR)/lib (currency.data etc). Compile launchers needed into $(JDK_OUTPUTDIR)/bin (java, javac, jmod). 6. The contents of $(JDK_OUTPUTDIR) is now enough to run jmod and javac. Run $(JDK_OUTPUTDIR)/bin/jmod to create a module library in $(JDK_OUTPUTDIR)/lib/modules and then run $(JDK_OUTPUTDIR)/bin/javac to compile all classes, (langtools, corba, jaxws, jaxp and jdk) into $(JDK_OUTPUTDIR)/mclasses. I cannot get this to work without supplying all modules and all classes on the same command line. It will complain that some module or classes belonging to a module is missing. That's how far I am at the moment. Left to do is compiling the rest of the native stuff into the correct module directories. Somehow move the classes into "classes" subdirectories and then create images. Some notes: * I had to add a new option to javac to ignore module-info.java files. Otherwise I can't compile bootstrap2 and I can't compile the classes in jdk.base alone. Javac sees the module-info.java file and refuses to compile in legacy mode. The alternative would be to copy all the java files except module-info.java to a temp dir before compiling, or temporarily remove the module-info.java file. * It would be very nice to have an option to javac to add the /classes subdirectory automatically inside the module directories. People, including me, will go nuts having to write their own logic to copy the class files in each module output dir themselves. (/mclasses/jdk.base/**/*.class -> /modules/jdk.base/classes/**/*.class) /Erik From paul.sandoz at oracle.com Fri Apr 12 01:59:22 2013 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 12 Apr 2013 10:59:22 +0200 Subject: Cannot build jigsaw on Mac OSX 10.8.2 Message-ID: Hi Erik, I cannot build Jigsaw on my Mac running OSX 10.8.2: -- ... Installing module jdk.classpath >>>Finished making modularize @ Thu Apr 11 22:15:15 CEST 2013 ... ## Finished post-modules (build time 00:02:19) ## Starting images sed: RE error: illegal byte sequence make[2]: *** [/Users/sandoz/Projects/jdk8/jigsaw/build/macosx-x86_64-normal-server-release/images/jre-base-image/THIRD_PARTY_README] Error 1 make[2]: *** Waiting for unfinished jobs.... make[1]: *** [images] Error 2 make: *** [images-only] Error 2 -- That is the same error i got previously when making images before the new build system was introduced. Note that i can build tl and lambda without any issue. Paul. From Alan.Bateman at oracle.com Fri Apr 12 03:40:43 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 12 Apr 2013 11:40:43 +0100 Subject: Cannot build jigsaw on Mac OSX 10.8.2 In-Reply-To: References: Message-ID: <5167E4AB.1010001@oracle.com> I think this is just that the jigsaw forest doesn't have 8007387 yet. Can you set LANG=C and see if that resolves it? On 12/04/2013 09:59, Paul Sandoz wrote: > Hi Erik, > > I cannot build Jigsaw on my Mac running OSX 10.8.2: > > -- > > ... > Installing module jdk.classpath >>>> Finished making modularize @ Thu Apr 11 22:15:15 CEST 2013 ... > ## Finished post-modules (build time 00:02:19) > > ## Starting images > sed: RE error: illegal byte sequence > make[2]: *** [/Users/sandoz/Projects/jdk8/jigsaw/build/macosx-x86_64-normal-server-release/images/jre-base-image/THIRD_PARTY_README] Error 1 > make[2]: *** Waiting for unfinished jobs.... > make[1]: *** [images] Error 2 > make: *** [images-only] Error 2 > > -- > > That is the same error i got previously when making images before the new build system was introduced. > > Note that i can build tl and lambda without any issue. > > Paul. From paul.sandoz at oracle.com Fri Apr 12 04:35:53 2013 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 12 Apr 2013 13:35:53 +0200 Subject: Cannot build jigsaw on Mac OSX 10.8.2 In-Reply-To: <5167E4AB.1010001@oracle.com> References: <5167E4AB.1010001@oracle.com> Message-ID: <5A82ED02-C59B-4561-B370-A236D2713D9F@oracle.com> On Apr 12, 2013, at 12:40 PM, Alan Bateman wrote: > > I think this is just that the jigsaw forest doesn't have 8007387 yet. Can you set LANG=C and see if that resolves it? That did not work. But the patch below fixed it. Thanks, Paul. diff -r 4df6759a08ac makefiles/Images.gmk --- a/makefiles/Images.gmk Sat Mar 23 21:09:13 2013 +0000 +++ b/makefiles/Images.gmk Fri Apr 12 13:35:19 2013 +0200 @@ -57,7 +57,7 @@ $(ECHO) $(LOG_INFO) Processing $(patsubst $(OUTPUT_ROOT)/%,%,$@) $(MKDIR) -p $(@D) $(RM) $@ - $(SED) 's/$$//g' $< > $@ + LC_ALL=C $(SED) 's/$$//g' $< > $@ $(CHMOD) 444 $@ endef diff -r 4df6759a08ac makefiles/ModuleImages.gmk --- a/makefiles/ModuleImages.gmk Sat Mar 23 21:09:13 2013 +0000 +++ b/makefiles/ModuleImages.gmk Fri Apr 12 13:35:19 2013 +0200 @@ -129,7 +129,7 @@ $(ECHO) $(LOG_INFO) Processing $(patsubst $(OUTPUT_ROOT)/%,%,$@) $(MKDIR) -p $(@D) $(RM) $@ - $(SED) 's/$$//g' $< > $@ + LC_ALL=C $(SED) 's/$$//g' $< > $@ $(CHMOD) 444 $@ endef > > > On 12/04/2013 09:59, Paul Sandoz wrote: >> Hi Erik, >> >> I cannot build Jigsaw on my Mac running OSX 10.8.2: >> >> -- >> >> ... >> Installing module jdk.classpath >>>>> Finished making modularize @ Thu Apr 11 22:15:15 CEST 2013 ... >> ## Finished post-modules (build time 00:02:19) >> >> ## Starting images >> sed: RE error: illegal byte sequence >> make[2]: *** [/Users/sandoz/Projects/jdk8/jigsaw/build/macosx-x86_64-normal-server-release/images/jre-base-image/THIRD_PARTY_README] Error 1 >> make[2]: *** Waiting for unfinished jobs.... >> make[1]: *** [images] Error 2 >> make: *** [images-only] Error 2 >> >> -- >> >> That is the same error i got previously when making images before the new build system was introduced. >> >> Note that i can build tl and lambda without any issue. >> >> Paul. > From Alan.Bateman at oracle.com Fri Apr 12 05:00:11 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 12 Apr 2013 13:00:11 +0100 Subject: Cannot build jigsaw on Mac OSX 10.8.2 In-Reply-To: <5A82ED02-C59B-4561-B370-A236D2713D9F@oracle.com> References: <5167E4AB.1010001@oracle.com> <5A82ED02-C59B-4561-B370-A236D2713D9F@oracle.com> Message-ID: <5167F74B.8020009@oracle.com> On 12/04/2013 12:35, Paul Sandoz wrote: > : > That did not work. But the patch below fixed it. I'm surprised that setting LANG=C didn't work. Hopefully we can sync up jigaw/jigsaw again soon as that has the fix to JDK-8007387 (which includes the patch you used). We could we push it earlier if needed. -Alan. From naoto.sato at oracle.com Fri Apr 12 09:29:30 2013 From: naoto.sato at oracle.com (Naoto Sato) Date: Fri, 12 Apr 2013 09:29:30 -0700 Subject: Cannot build jigsaw on Mac OSX 10.8.2 In-Reply-To: <5167F74B.8020009@oracle.com> References: <5167E4AB.1010001@oracle.com> <5A82ED02-C59B-4561-B370-A236D2713D9F@oracle.com> <5167F74B.8020009@oracle.com> Message-ID: <5168366A.60205@oracle.com> On 4/12/13 5:00 AM, Alan Bateman wrote: > On 12/04/2013 12:35, Paul Sandoz wrote: >> : >> That did not work. But the patch below fixed it. > I'm surprised that setting LANG=C didn't work. Probably LC_CTYPE was set to some value which was not "C". It supersedes LANG. Naoto