The future of partial builds

[Anthony Petrov]

Magnus,

You've only explained how incremental builds could work for Java classes 
in the new build-infra. What about incremental builds of native code? 
E.g. in AWT we often do the following:

$ cd make/sun/awt (or make/java/awt, or make/sun/lwawt)
$ make

And this re-builds both AWT classes and AWT native code (and some 2D 
stuff too, btw). What has build-infra got to offer for incremental 
builds of native code?

Also, how fast are those incremental solutions (both the temporary, and 
the expected final one) in build-infra? Performing the above commands 
with the current build system takes literally just a few seconds on any 
platform (given you've built the JDK repo once before, and it might took 
like 10-30 minutes for the first build).

--
best regards,
Anthony

On 9/10/2012 6:00 PM, Magnus Ihse Bursie wrote:
> On 2012-09-10 14:13, Alan Bateman wrote:
>> When you say "sub-directory builds" then I think you mean incremental 
>> builds, or "poor-man increment builds" as I call it. I think the 
>> majority of people working in the jdk repository, at least in Oracle, 
>> do this because they know the area and know which make files that 
>> re-build the files that they have changed. Incremental builds are 
>> generally not very reliable but seem to be "good enough" for most 
>> people needs, at least those that aren't doing significant 
>> refactoring. The main motive of course is to be able to re-build in a 
>> few seconds and the #1 goal of any new build system has to be fast 
>> incremental builds.
> 
> In the new build system, fast incremental builds of Java code is 
> dependent on the new "smart javac", which unfortunately has still not 
> proven stable enough to be enabled by default, even in the experimental 
> build-infra forest. It is still our hope that it will turn out to be 
> good enough to be used as default, but I don't want to trust hope alone.
> 
> As an interim solution, I have just created the following two shortcuts. 
> Both of these bypass the proper dependency detection. This means that 
> make will cut to the chase and recompile faster, but it also means that 
> you need to take over the responsibility of making sure dependencies are 
> correct. This is very similar to the "*-only" (e.g. hotspot-only) 
> targets that are current available in the new build system.
> 
> First shortcut is to skip parts of the JDK makefile. By typing e.g. 
> "make jdk JDK_TARGET=classes", the JDK build will stop after building 
> the "classes" target (which compiles the majority of the JDK java 
> files). An even faster, but less safe, version is "make jdk-only 
> JDK_TARGET=classes-only", which will jump directly to the JDK Makefile, 
> and only execute the "classes" target. In this way, absolutely no time 
> is spent before starting to compile Java classes.
> 
> Second shortcut is to limit the classes that are recompiled by the JDK 
> "classes" target. Currently, *all* (more or less) Java classes in the 
> JDK is recompiled at once. While this speeds up things considerably at a 
> first time compilation, it makes all subsequent builds take the same 
> time. By specifying e.g. JDK_FILTER=java/nio, only files in the java/nio 
> package will get compiled. This is, I think, similar (but not strictly 
> not identical) to the effect of "sub-directory builds". It is possible 
> to use an even more fine-grained filter, separated by comma, e.g. 
> JDK_FILTER=java/nio/channels,java/nio/charset -- the smaller number of 
> Java files your filter includes, the faster the build will go (more or 
> less).
> 
> These two can of course be combined.
> 
> I would like to point out that this is hopefully a temporary solution -- 
> if proper dependency checking and incremental builds get fast enough, 
> this kind of trickery should not be needed. If this happens, we will 
> remove the support for these shortcuts since they do carry a risk of 
> improper builds.
> 
> /Magnus