The future of partial builds

[Magnus Ihse Bursie]

On 2012-09-10 14:13, Alan Bateman wrote:
> When you say "sub-directory builds" then I think you mean incremental 
> builds, or "poor-man increment builds" as I call it. I think the 
> majority of people working in the jdk repository, at least in Oracle, 
> do this because they know the area and know which make files that 
> re-build the files that they have changed. Incremental builds are 
> generally not very reliable but seem to be "good enough" for most 
> people needs, at least those that aren't doing significant 
> refactoring. The main motive of course is to be able to re-build in a 
> few seconds and the #1 goal of any new build system has to be fast 
> incremental builds.

In the new build system, fast incremental builds of Java code is 
dependent on the new "smart javac", which unfortunately has still not 
proven stable enough to be enabled by default, even in the experimental 
build-infra forest. It is still our hope that it will turn out to be 
good enough to be used as default, but I don't want to trust hope alone.

As an interim solution, I have just created the following two shortcuts. 
Both of these bypass the proper dependency detection. This means that 
make will cut to the chase and recompile faster, but it also means that 
you need to take over the responsibility of making sure dependencies are 
correct. This is very similar to the "*-only" (e.g. hotspot-only) 
targets that are current available in the new build system.

First shortcut is to skip parts of the JDK makefile. By typing e.g. 
"make jdk JDK_TARGET=classes", the JDK build will stop after building 
the "classes" target (which compiles the majority of the JDK java 
files). An even faster, but less safe, version is "make jdk-only 
JDK_TARGET=classes-only", which will jump directly to the JDK Makefile, 
and only execute the "classes" target. In this way, absolutely no time 
is spent before starting to compile Java classes.

Second shortcut is to limit the classes that are recompiled by the JDK 
"classes" target. Currently, *all* (more or less) Java classes in the 
JDK is recompiled at once. While this speeds up things considerably at a 
first time compilation, it makes all subsequent builds take the same 
time. By specifying e.g. JDK_FILTER=java/nio, only files in the java/nio 
package will get compiled. This is, I think, similar (but not strictly 
not identical) to the effect of "sub-directory builds". It is possible 
to use an even more fine-grained filter, separated by comma, e.g. 
JDK_FILTER=java/nio/channels,java/nio/charset -- the smaller number of 
Java files your filter includes, the faster the build will go (more or 
less).

These two can of course be combined.

I would like to point out that this is hopefully a temporary solution -- 
if proper dependency checking and incremental builds get fast enough, 
this kind of trickery should not be needed. If this happens, we will 
remove the support for these shortcuts since they do carry a risk of 
improper builds.

/Magnus