converting javac to be a modular app

[Jonathan Gibbons]

Comments inline.

-- Jon


On 05/26/2010 09:20 AM, Mandy Chung wrote:
> Jon,
>
> This note is helpful and it gives an idea how complicate the world we 
> are in!  My comment inlined below.
>
> On 05/24/10 19:25, Jonathan Gibbons wrote:
>> On the face of it, it ought to be easy to change javac to be executed 
>> with module mode instead of legacy mode. You might think it is 
>> sufficient to simply remove the reference to -J-Xmode:legacy in the 
>> appropriate line of jdk/make/launchers/Makefile, as in the following 
>> diff:
>>
>>> $ hg diff -U 0
>>> diff -r 7e1d7e037819 make/launchers/Makefile
>>> --- a/make/launchers/Makefile   Mon May 24 14:27:31 2010 -0700
>>> +++ b/make/launchers/Makefile   Mon May 24 16:43:30 2010 -0700
>>> @@ -64,1 +64,1 @@
>>> -$(call make-launcher, javac, com.sun.tools.javac.Main, 
>>> -J-Xmode:legacy, )
>>> +$(call make-launcher, javac, com.sun.tools.javac.Main, , )
>>
>> If only it were that simple ...
>>
>> To understand the real problem, you need to understand javac's role 
>> in building JDK. For a full description of the build process, see 
>> Kelly's blog, "Anatomy of the JDK Build":
>>     http://blogs.sun.com/kto/entry/anatomy_of_the_jdk_build
>>
>> Here's a quick summary:
>>
>> 1. Build the latest javac with the BOOTJAVA javac to create a hybrid 
>> javac (call it hybrid-1) that can run on BOOTJAVA but which can 
>> compile JDK7 code.
>> 2. Use hybrid-1 javac to compile all the langtools repository for 
>> delivery into the final bundles.
>> 3. Use hybrid-1 javac to compile the corba, jaxws, jaxp, repositories
>> 4. Build the jdk repository using hybrid-1 javac, and including all 
>> the classes compiled in steps 1, 2 and 3, to produce candidate JDK 
>> bundles.
>>
>> Many developers stop at this point.  But up to this point, the JDK 
>> created in step 4 has not been tested, so a common step is to perform 
>> the build with SKIP_BOOT_CYCLE=false. If that variable is set, the 
>> build will repeat steps 1-4 using the JDK from step 4 (call it 
>> "interim JDK") as the setting for BOOTJAVA.
>>
>> 5. Build the latest javac with the interim JDK javac to create a 
>> hybrid javac (call it hybrid-2) that can run on the interim JDK and 
>> which can compile JDK7 code.
>> 6. Use hybrid-2 javac to compile all the langtools repository for 
>> delivery into the final bundles.
>> 7. Use hybrid-2 javac to compile the corba, jaxws, jaxp, repositories
>> 8. Build the jdk repository using hybrid-2 javac, and including all 
>> the classes compiled in steps 1, 2 and 3, to produce final JDK bundles.
>>
>> So where's the problem?  Not surprisingly, it's in those hybrid-1 and 
>> hybrid-2 versions of javac.  Currently, we synthesize those hybrid 
>> versions of javac by using -J-Xbootclasspath/p: to override the 
>> location of the javac classes being used by the underlying JDK. As 
>> long as we build javac to execute in legacy mode, that works just 
>> fine. But once we switch javac to use module mode, we lose the 
>> ability to use -J-Xbootclasspath/p:.  Instead, we have to use the new 
>> "-L library" option, which replaces "-Xbootclasspath/p:classes". What 
>> does that entail?
>>
>> Currently, there is no explicit artifact that is identifiable as 
>> "hybrid-1" or "hybrid-2" javac. Instead, the langtools part of the 
>> build delivers a set of classes, to be used as part of a hybrid 
>> javac, and each downstream repository in the forest uses those 
>> classes with -J-Xbootclasspath to synthesize the hybrid javac. So, 
>> there are two sets of changes to be made.
>> - The langtools repository must determine if it has been given a 
>> legacy JDK or a modular JDK for its bootstrap. If it has been given a 
>> modular JDK, it needs to create a module library to be used for 
>> downstream compilations.
>> - All downstream builds need to determine if they have been given a 
>> legacy JDK or a modular JDK for their bootstrap, and so should 
>> synthesize their hybrid javac using "-Xbootclasspath/p:classes" or 
>> "-L library" (or rather, "-J-Xbootclasspath/p:classes" or "-J-L 
>> -Jlibrary". Mild uugh!)
>>
>> Note that we cannot hard-code the change to use "-L library" through 
>> the life of JDK7. In other words, for as long as we need to start the 
>> build using a legacy JDK, such as JDK 6, the build must be "bimodal", 
>> able to work with both a legacy JDK, or a modular JDK such as may 
>> just have been built.
>>
>> The changes to have the langtools repository build a module library 
>> are essentially as described here:
>>     
>> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2010-May/000960.html
>>
>> However, the changes described there concerned a developer build of 
>> javac; the changes need to be replicated to apply to the bootstrap 
>> builds as well. This corresponds to langtools having settings for two 
>> different versions of JDK -- one to build with, and one to execute 
>> what it builds: the langtools build needs to be prepared for each 
>> setting to be given either a legacy JDK or a modular JDK.
>>
>> The changes to have each downstream repository determine whether to 
>> use  -Xbootclasspath or L are not particularly difficult. One minor 
>> complication to take into account is that some repositories use Ant 
>> instead of "make".  Another complication is whether or not this needs 
>> to be a synchronized build change across all repositories. One way to 
>> avoid that would be a two phase approach, such that we first change 
>> all repositories to explicitly force the use of legacy mode for 
>> javac, then we can change the default mode for javac, then finally 
>> update the repositories to work with the default.
>>
>
> I think the javac would have to stay in legacy mode for jdk build for 
> some time until the jdk compiles classes in its module directory (i.e. 
> setting -modulepath for javac to use).   In step 4 & 8 (building jdk), 
> the hybrid javac overrides the bootclasspath with the classes 
> directory as it compiles itself.  The  module library for the jdk is 
> created at the end of the jdk build.  It can use -L option to set to 
> use the langtools module library but it still needs a way to compile 
> jdk sources with the newly built classes during the jdk build.

Ah, but that's the "other" bootclasspath, isn't it?   The hybrid javac's 
are all about the use of the JVM bootclasspath, and changing to set the 
JVM module library.   Here, you're talking about the javac bootclasspath 
and -L option.


>
> What steps would you imagine it takes to update the langtools repo to 
> work with the default?  I guess langtools can remain being compiled in 
> single module mode using the -sourcepath option?   It doesn't require 
> langtools to be compiled as modules for phase 2.  I try to put things 
> together in my head - do I get this right?
>
>> That's almost it. You'd think.
>>
>> If only it were that simple ...
>>
>> Who spotted that sometimes developers perform "partial builds" and 
>> don't necessarily use and build a copy of the langtools repository?  
>> Instead, you can set ALT_JDK_IMPORT_PATH to a recent build of JDK.  
>> So yes, we need to make the build work in that case as well.  And, 
>> for now, that is left as an exercise for the reader.
>>
>
> That's a good point.  If ALT_JDK_IMPORT_PATH would be a modular JDK, 
> the current workaround is to copy classes directly from the imported 
> module library.   The jdk makefiles have be modified to import corba, 
> jaxp and jaxws from a modular JDK but yet to import langtools correctly.

Hmmm, yes, I was just thinking of the javac bootstrap  issues; I missed 
the need to import classes from the ALT_JDK_IMPORT_PATH.

>
> In the long-term, I think the jdk build would import from the 
> langtools module files (and other components) from a promoted build 
> rather than a module image.

What is the difference that you're thinking of here?

>
> Mandy
>
>



In the short term, my plan is to push changes that will have the 
langtools build create modules from the langtools bootstrap classes for 
optional use downstream if desired.  We can then work on determining how 
to use those modules in the downstream repos.

-- Jon