The role of submodules

[Erik Joelsson]

On 2013-03-28 20:02, Mandy Chung wrote:
> On 3/28/13 1:51 AM, Erik Joelsson wrote:
>> Hello,
>>
>> While working with the jigsaw build, I can't help but wonder what the 
>> reason for the submodules concept is. In the old build, each area is 
>> assigned a "MODULE", which isn't a full module name, but something 
>> shorter. All artifacts produced in an area, like libraries and lib 
>> files, automatically get copied to a sub directory of the submodules 
>> outputdir. Then in the modularization step, the contents of these 
>> submodules is copied to the final (real) modules according to a 
>> configured pattern. (ex: base -> jdk.base, javac -> jdk.tools)
>>
>
> When we started the modules build, the module graph was not clean to 
> begin with.  We grouped classes in a fine-grained module to allow us 
> to do experimentation and fine-tuning the module graph by aggregating 
> more than one small modules (submodules) into one.  It also makes it 
> easier for us to determine any undesirable dependency across 
> submodules that may become a non-issue when those submodules are 
> relevant and grouped as a single module.
>
>> Is there a reason to keep this model going forward, or can we assign 
>> things their final module name directly in the new build? I would 
>> like to minimize copying files around for no apparent reason if 
>> possible.
>
> I don't see the need to keep the submodules and we can use the modules 
> currently defined in the new build as you suggest as the module graph 
> is looking good and clean.  One thing to keep in mind is that there 
> will likely be some change to the module graph as it is just a 
> prototype and we may need to easily update the module assignment 
> before we do the source restructuring.
>
That sounds good to me. The design I have assumed for now will look like 
this. There are two ways to build:

1. The current new build as was recently submitted. Building in legacy 
mode and then creating modules in a post step.
2. Building from the shuffled source (in */newsrc directories) into 
modules directly.

The first build is needed as input for the source shuffling scripts. The 
second can only run after shuffling the source. There is a configure 
switch to choose between the build variants. The default is 2 if 
restructured source is available and 1 if it isn't.

I'm striving to make these builds coexist with minimal changes to ease 
integrations from jdk8. I'm hoping to post a webrev soon (maybe this 
week) so you can give it a spin yourselves.

So far, this is what build 2 is doing:

1. Build langtools, almost same as in 1. In this step, langtools is 
compiled twice.
   A. Bootstrap1. Boot jdk with boot javac is used to compile. Module 
support is execluded. Target class files runnable on boot jdk.
   B. Bootstrap2. Boot jdk with Bootstrap1 javac is used to compile. 
Module support is enabled. Genstubs is used to pull in the needed 
classes from jdk repo. Classfiles generated are for the target jdk.
The class files from bootstrap2 could be put in the final product, but 
for technical reasons explained below, they aren't. Bootstrap2 will be 
used to compile the rest of the jdk later.

2. Build hotspot, just as in 1.

3. Generate java source and idls in corba.

4. Enter jdk. Import from hotspot into $(JDK_OUTPUTDIR)/lib. Import 
classes.jar from langtools (these are the bootstrap2 classes) into 
$(JDK_OUTPUTDIR)/classes dir. (This is just like build 1 is doing it). 
Compile the build tools, generate all java source and data.

5. Compile the classes in jdk.base using bootstrap1 (no module 
awareness) into $(JDK_OUTPUTDIR)/classes. Compile the libraries in 
jdk.base into $(JDK_OUTPUTDIR)/lib. Copy the files needed into 
$(JDK_OUTPUTDIR)/lib (currency.data etc). Compile launchers needed into 
$(JDK_OUTPUTDIR)/bin (java, javac, jmod).

6. The contents of $(JDK_OUTPUTDIR) is now enough to run jmod and javac. 
Run $(JDK_OUTPUTDIR)/bin/jmod to create a module library in 
$(JDK_OUTPUTDIR)/lib/modules and then run $(JDK_OUTPUTDIR)/bin/javac to 
compile all classes, (langtools, corba, jaxws, jaxp and jdk) into 
$(JDK_OUTPUTDIR)/mclasses. I cannot get this to work without supplying 
all modules and all classes on the same command line. It will complain 
that some module or classes belonging to a module is missing.

That's how far I am at the moment. Left to do is compiling the rest of 
the native stuff into the correct module directories. Somehow move the 
classes into "classes" subdirectories and then create images.

Some notes:

* I had to add a new option to javac to ignore module-info.java files. 
Otherwise I can't compile bootstrap2 and I can't compile the classes in 
jdk.base alone. Javac sees the module-info.java file and refuses to 
compile in legacy mode. The alternative would be to copy all the java 
files except module-info.java to a temp dir before compiling, or 
temporarily remove the module-info.java file.

* It would be very nice to have an option to javac to add the /classes 
subdirectory automatically inside the module directories. People, 
including me, will go nuts having to write their own logic to copy the 
class files in each module output dir themselves. 
(/mclasses/jdk.base/**/*.class -> /modules/jdk.base/classes/**/*.class)

/Erik