converting javac to be a modular app

[Mandy Chung]

Jon,

This note is helpful and it gives an idea how complicate the world we 
are in!  My comment inlined below.

On 05/24/10 19:25, Jonathan Gibbons wrote:
> On the face of it, it ought to be easy to change javac to be executed 
> with module mode instead of legacy mode. You might think it is 
> sufficient to simply remove the reference to -J-Xmode:legacy in the 
> appropriate line of jdk/make/launchers/Makefile, as in the following 
> diff:
>
>> $ hg diff -U 0
>> diff -r 7e1d7e037819 make/launchers/Makefile
>> --- a/make/launchers/Makefile   Mon May 24 14:27:31 2010 -0700
>> +++ b/make/launchers/Makefile   Mon May 24 16:43:30 2010 -0700
>> @@ -64,1 +64,1 @@
>> -$(call make-launcher, javac, com.sun.tools.javac.Main, 
>> -J-Xmode:legacy, )
>> +$(call make-launcher, javac, com.sun.tools.javac.Main, , )
>
> If only it were that simple ...
>
> To understand the real problem, you need to understand javac's role in 
> building JDK. For a full description of the build process, see Kelly's 
> blog, "Anatomy of the JDK Build":
>     http://blogs.sun.com/kto/entry/anatomy_of_the_jdk_build
>
> Here's a quick summary:
>
> 1. Build the latest javac with the BOOTJAVA javac to create a hybrid 
> javac (call it hybrid-1) that can run on BOOTJAVA but which can 
> compile JDK7 code.
> 2. Use hybrid-1 javac to compile all the langtools repository for 
> delivery into the final bundles.
> 3. Use hybrid-1 javac to compile the corba, jaxws, jaxp, repositories
> 4. Build the jdk repository using hybrid-1 javac, and including all 
> the classes compiled in steps 1, 2 and 3, to produce candidate JDK 
> bundles.
>
> Many developers stop at this point.  But up to this point, the JDK 
> created in step 4 has not been tested, so a common step is to perform 
> the build with SKIP_BOOT_CYCLE=false. If that variable is set, the 
> build will repeat steps 1-4 using the JDK from step 4 (call it 
> "interim JDK") as the setting for BOOTJAVA.
>
> 5. Build the latest javac with the interim JDK javac to create a 
> hybrid javac (call it hybrid-2) that can run on the interim JDK and 
> which can compile JDK7 code.
> 6. Use hybrid-2 javac to compile all the langtools repository for 
> delivery into the final bundles.
> 7. Use hybrid-2 javac to compile the corba, jaxws, jaxp, repositories
> 8. Build the jdk repository using hybrid-2 javac, and including all 
> the classes compiled in steps 1, 2 and 3, to produce final JDK bundles.
>
> So where's the problem?  Not surprisingly, it's in those hybrid-1 and 
> hybrid-2 versions of javac.  Currently, we synthesize those hybrid 
> versions of javac by using -J-Xbootclasspath/p: to override the 
> location of the javac classes being used by the underlying JDK. As 
> long as we build javac to execute in legacy mode, that works just 
> fine. But once we switch javac to use module mode, we lose the ability 
> to use -J-Xbootclasspath/p:.  Instead, we have to use the new "-L 
> library" option, which replaces "-Xbootclasspath/p:classes". What does 
> that entail?
>
> Currently, there is no explicit artifact that is identifiable as 
> "hybrid-1" or "hybrid-2" javac. Instead, the langtools part of the 
> build delivers a set of classes, to be used as part of a hybrid javac, 
> and each downstream repository in the forest uses those classes with 
> -J-Xbootclasspath to synthesize the hybrid javac. So, there are two 
> sets of changes to be made.
> - The langtools repository must determine if it has been given a 
> legacy JDK or a modular JDK for its bootstrap. If it has been given a 
> modular JDK, it needs to create a module library to be used for 
> downstream compilations.
> - All downstream builds need to determine if they have been given a 
> legacy JDK or a modular JDK for their bootstrap, and so should 
> synthesize their hybrid javac using "-Xbootclasspath/p:classes" or "-L 
> library" (or rather, "-J-Xbootclasspath/p:classes" or "-J-L 
> -Jlibrary". Mild uugh!)
>
> Note that we cannot hard-code the change to use "-L library" through 
> the life of JDK7. In other words, for as long as we need to start the 
> build using a legacy JDK, such as JDK 6, the build must be "bimodal", 
> able to work with both a legacy JDK, or a modular JDK such as may just 
> have been built.
>
> The changes to have the langtools repository build a module library 
> are essentially as described here:
>     
> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2010-May/000960.html
>
> However, the changes described there concerned a developer build of 
> javac; the changes need to be replicated to apply to the bootstrap 
> builds as well. This corresponds to langtools having settings for two 
> different versions of JDK -- one to build with, and one to execute 
> what it builds: the langtools build needs to be prepared for each 
> setting to be given either a legacy JDK or a modular JDK.
>
> The changes to have each downstream repository determine whether to 
> use  -Xbootclasspath or L are not particularly difficult. One minor 
> complication to take into account is that some repositories use Ant 
> instead of "make".  Another complication is whether or not this needs 
> to be a synchronized build change across all repositories. One way to 
> avoid that would be a two phase approach, such that we first change 
> all repositories to explicitly force the use of legacy mode for javac, 
> then we can change the default mode for javac, then finally update the 
> repositories to work with the default.
>

I think the javac would have to stay in legacy mode for jdk build for 
some time until the jdk compiles classes in its module directory (i.e. 
setting -modulepath for javac to use).   In step 4 & 8 (building jdk), 
the hybrid javac overrides the bootclasspath with the classes directory 
as it compiles itself.  The  module library for the jdk is created at 
the end of the jdk build.  It can use -L option to set to use the 
langtools module library but it still needs a way to compile jdk sources 
with the newly built classes during the jdk build.

What steps would you imagine it takes to update the langtools repo to 
work with the default?  I guess langtools can remain being compiled in 
single module mode using the -sourcepath option?   It doesn't require 
langtools to be compiled as modules for phase 2.  I try to put things 
together in my head - do I get this right?

> That's almost it. You'd think.
>
> If only it were that simple ...
>
> Who spotted that sometimes developers perform "partial builds" and 
> don't necessarily use and build a copy of the langtools repository?  
> Instead, you can set ALT_JDK_IMPORT_PATH to a recent build of JDK.  So 
> yes, we need to make the build work in that case as well.  And, for 
> now, that is left as an exercise for the reader.
>

That's a good point.  If ALT_JDK_IMPORT_PATH would be a modular JDK, the 
current workaround is to copy classes directly from the imported module 
library.   The jdk makefiles have be modified to import corba, jaxp and 
jaxws from a modular JDK but yet to import langtools correctly.

In the long-term, I think the jdk build would import from the langtools 
module files (and other components) from a promoted build rather than a 
module image.

Mandy