converting javac to be a modular app

[Mandy Chung]

On 05/26/10 09:33, Jonathan Gibbons wrote:
> Comments inline.
>
> -- Jon
>
>
> On 05/26/2010 09:20 AM, Mandy Chung wrote:
>> Jon,
>>
>> This note is helpful and it gives an idea how complicate the world we 
>> are in!  My comment inlined below.
>>
>> On 05/24/10 19:25, Jonathan Gibbons wrote:
>>> On the face of it, it ought to be easy to change javac to be 
>>> executed with module mode instead of legacy mode. You might think it 
>>> is sufficient to simply remove the reference to -J-Xmode:legacy in 
>>> the appropriate line of jdk/make/launchers/Makefile, as in the 
>>> following diff:
>>>
>>>> $ hg diff -U 0
>>>> diff -r 7e1d7e037819 make/launchers/Makefile
>>>> --- a/make/launchers/Makefile   Mon May 24 14:27:31 2010 -0700
>>>> +++ b/make/launchers/Makefile   Mon May 24 16:43:30 2010 -0700
>>>> @@ -64,1 +64,1 @@
>>>> -$(call make-launcher, javac, com.sun.tools.javac.Main, 
>>>> -J-Xmode:legacy, )
>>>> +$(call make-launcher, javac, com.sun.tools.javac.Main, , )
>>>
>>> If only it were that simple ...
>>>
>>> To understand the real problem, you need to understand javac's role 
>>> in building JDK. For a full description of the build process, see 
>>> Kelly's blog, "Anatomy of the JDK Build":
>>>     http://blogs.sun.com/kto/entry/anatomy_of_the_jdk_build
>>>
>>> Here's a quick summary:
>>>
>>> 1. Build the latest javac with the BOOTJAVA javac to create a hybrid 
>>> javac (call it hybrid-1) that can run on BOOTJAVA but which can 
>>> compile JDK7 code.
>>> 2. Use hybrid-1 javac to compile all the langtools repository for 
>>> delivery into the final bundles.
>>> 3. Use hybrid-1 javac to compile the corba, jaxws, jaxp, repositories
>>> 4. Build the jdk repository using hybrid-1 javac, and including all 
>>> the classes compiled in steps 1, 2 and 3, to produce candidate JDK 
>>> bundles.
>>>
>>> Many developers stop at this point.  But up to this point, the JDK 
>>> created in step 4 has not been tested, so a common step is to 
>>> perform the build with SKIP_BOOT_CYCLE=false. If that variable is 
>>> set, the build will repeat steps 1-4 using the JDK from step 4 (call 
>>> it "interim JDK") as the setting for BOOTJAVA.
>>>
>>> 5. Build the latest javac with the interim JDK javac to create a 
>>> hybrid javac (call it hybrid-2) that can run on the interim JDK and 
>>> which can compile JDK7 code.
>>> 6. Use hybrid-2 javac to compile all the langtools repository for 
>>> delivery into the final bundles.
>>> 7. Use hybrid-2 javac to compile the corba, jaxws, jaxp, repositories
>>> 8. Build the jdk repository using hybrid-2 javac, and including all 
>>> the classes compiled in steps 1, 2 and 3, to produce final JDK bundles.
>>>
>>> So where's the problem?  Not surprisingly, it's in those hybrid-1 
>>> and hybrid-2 versions of javac.  Currently, we synthesize those 
>>> hybrid versions of javac by using -J-Xbootclasspath/p: to override 
>>> the location of the javac classes being used by the underlying JDK. 
>>> As long as we build javac to execute in legacy mode, that works just 
>>> fine. But once we switch javac to use module mode, we lose the 
>>> ability to use -J-Xbootclasspath/p:.  Instead, we have to use the 
>>> new "-L library" option, which replaces "-Xbootclasspath/p:classes". 
>>> What does that entail?
>>>
>>> Currently, there is no explicit artifact that is identifiable as 
>>> "hybrid-1" or "hybrid-2" javac. Instead, the langtools part of the 
>>> build delivers a set of classes, to be used as part of a hybrid 
>>> javac, and each downstream repository in the forest uses those 
>>> classes with -J-Xbootclasspath to synthesize the hybrid javac. So, 
>>> there are two sets of changes to be made.
>>> - The langtools repository must determine if it has been given a 
>>> legacy JDK or a modular JDK for its bootstrap. If it has been given 
>>> a modular JDK, it needs to create a module library to be used for 
>>> downstream compilations.
>>> - All downstream builds need to determine if they have been given a 
>>> legacy JDK or a modular JDK for their bootstrap, and so should 
>>> synthesize their hybrid javac using "-Xbootclasspath/p:classes" or 
>>> "-L library" (or rather, "-J-Xbootclasspath/p:classes" or "-J-L 
>>> -Jlibrary". Mild uugh!)
>>>
>>> Note that we cannot hard-code the change to use "-L library" through 
>>> the life of JDK7. In other words, for as long as we need to start 
>>> the build using a legacy JDK, such as JDK 6, the build must be 
>>> "bimodal", able to work with both a legacy JDK, or a modular JDK 
>>> such as may just have been built.
>>>
>>> The changes to have the langtools repository build a module library 
>>> are essentially as described here:
>>>     
>>> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2010-May/000960.html
>>>
>>> However, the changes described there concerned a developer build of 
>>> javac; the changes need to be replicated to apply to the bootstrap 
>>> builds as well. This corresponds to langtools having settings for 
>>> two different versions of JDK -- one to build with, and one to 
>>> execute what it builds: the langtools build needs to be prepared for 
>>> each setting to be given either a legacy JDK or a modular JDK.
>>>
>>> The changes to have each downstream repository determine whether to 
>>> use  -Xbootclasspath or L are not particularly difficult. One minor 
>>> complication to take into account is that some repositories use Ant 
>>> instead of "make".  Another complication is whether or not this 
>>> needs to be a synchronized build change across all repositories. One 
>>> way to avoid that would be a two phase approach, such that we first 
>>> change all repositories to explicitly force the use of legacy mode 
>>> for javac, then we can change the default mode for javac, then 
>>> finally update the repositories to work with the default.
>>>
>>
>> I think the javac would have to stay in legacy mode for jdk build for 
>> some time until the jdk compiles classes in its module directory 
>> (i.e. setting -modulepath for javac to use).   In step 4 & 8 
>> (building jdk), the hybrid javac overrides the bootclasspath with the 
>> classes directory as it compiles itself.  The  module library for the 
>> jdk is created at the end of the jdk build.  It can use -L option to 
>> set to use the langtools module library but it still needs a way to 
>> compile jdk sources with the newly built classes during the jdk build.
>
> Ah, but that's the "other" bootclasspath, isn't it?   The hybrid 
> javac's are all about the use of the JVM bootclasspath, and changing 
> to set the JVM module library.   Here, you're talking about the javac 
> bootclasspath and -L option.
>

That's right - that's the other bootclasspath!   JDK will continue to 
compile in single module mode (i.e. legacy compilation) with javac's 
-Xbootclasspath option.  javac will run with the latest langtools 
classes via the JVM bootclasspath or JVM module library (the -J 
version).  I clearly mixed them up when I first read this but now it's 
clear.  Thanks.

>
>>
>> What steps would you imagine it takes to update the langtools repo to 
>> work with the default?  I guess langtools can remain being compiled 
>> in single module mode using the -sourcepath option?   It doesn't 
>> require langtools to be compiled as modules for phase 2.  I try to 
>> put things together in my head - do I get this right?
>>
>>> That's almost it. You'd think.
>>>
>>> If only it were that simple ...
>>>
>>> Who spotted that sometimes developers perform "partial builds" and 
>>> don't necessarily use and build a copy of the langtools repository?  
>>> Instead, you can set ALT_JDK_IMPORT_PATH to a recent build of JDK.  
>>> So yes, we need to make the build work in that case as well.  And, 
>>> for now, that is left as an exercise for the reader.
>>>
>>
>> That's a good point.  If ALT_JDK_IMPORT_PATH would be a modular JDK, 
>> the current workaround is to copy classes directly from the imported 
>> module library.   The jdk makefiles have be modified to import corba, 
>> jaxp and jaxws from a modular JDK but yet to import langtools correctly.
>
> Hmmm, yes, I was just thinking of the javac bootstrap  issues; I 
> missed the need to import classes from the ALT_JDK_IMPORT_PATH.
>
>>
>> In the long-term, I think the jdk build would import from the 
>> langtools module files (and other components) from a promoted build 
>> rather than a module image.
>
> What is the difference that you're thinking of here?
>

It's related to the jdk partial build issue and how it should import 
components in the modular world.  I wanted to point it out as langtools 
will be building jmod files that would work well in the future.

ALT_JDK_IMPORT_PATH refers to a JDK image. In the legacy world, we build 
only the legacy image and thus we can only import from a JDK image 
(that's ALT_JDK_IMPORT_PATH).  In the modular world, we will bundle the 
jdk module files in addition to the jre-base-image.  For doing a partial 
jdk build (no langtools build), some time later the jdk build should 
support to import from the jdk module files (e.g. javac.jmod, 
corba.jmod, etc).

>
>
> In the short term, my plan is to push changes that will have the 
> langtools build create modules from the langtools bootstrap classes 
> for optional use downstream if desired.  We can then work on 
> determining how to use those modules in the downstream repos.
>  
Sounds good.

Mandy