The future of partial builds

[Fredrik Öhrström]

11 sep 2012 kl. 21:59 skrev Alan Bateman:
> With a sjavac config I touched one source file and the incremental build took 36s so much better. In this case the one class caused 77 classes to be re-compiled, 10 native files, and 2 shared libraries to be re-linked. On the other hand, the old make does it in 2s, mostly because it just re-compiled the 1 class without a care in the world as to who is using it.

Excellent. I hope you realize how valuable it is that the build system recompiled the proper source files, then proceed to generate the the jni headers output
because of native methods in those classes that were recompiled, then proceed to recompile only the shared libraries that implement these jni interfaces.

Later we will make sure that if the recompile of the java source file does not change the jni header, it will not be updated/touched.

Have you tried making an actual change in a Java source file that changes the public api? To see how the build system recompiles all packages
that import from that changed package?

Most of 36 seconds are spent in javac. Javac is extremely fast, but has a huuuuge startup time. In particular 
because the option -Xprefer:source is enabled in Setup.gmk inside jdk/makefiles

This means that even though you recompile only a package, it will recompile all imported classes from source.
For the jdk repository, thats a lot of classes!

When I touch for example SignerOutputStream.java and remake, it takes 16 seconds to rebuild.
When I remove -Xprefer:source in jdk/makefiles/Setup.gmk, touch, make (takes longer because the command line changes
thus it had to drop the javac_state file) but then touch, then make again,
it takes only 7 seconds to rebuild. Please try, and tell us what speed improvement you gained.

We should now (perhaps) remove -Xprefer:source, and let javac pickup already compiled classes from the (correct?) class path.

However, this begets the question why -Xprefer:source was added in the first place. If I remember correctly,
this was over a year ago. It turns out that the default behavior for implicit compilation (which is necessary for sjavac:s multi core support) 
is to not recompile java source files when the class has a more recent timestamp than the source file.
Sounds reasonable, until you realize that javac also takes into account the timestamps in rt.jar inside the jdk.
Soooo, if you check out the source code on Monday, then download a newly built jdk on tuesday,
and use this as the boot jdk, then the rt. jar timestamps will be newer than the source files you are trying to 
compile, and this somehow messes up the build!

Perhaps this is only relevant when compiling langtools where there are other bootstrap intricasies.
Perhaps not.  Any masters of the javac universe out there who can help resolve this conundrum?

//Fredrik