Faster incremental OpenJDK compilation

[Jan Lahoda]

Hi,

Triggered by Magnus' recent e-mail on adjusting the location of the IDE 
files, I looked at possibilities to improve speed of incremental 
compilation using make. About 3 years ago, we have sped up incremental 
build by only rebuilding modules when API of modules they depend on 
changed. But the module which contains modified sources is always 
compiled in full. So, for changes in java.base, this change improved the 
incremental build time significantly (by not recompiling e.g. 
java.desktop), but it can still take many seconds to build java.base 
after a trivial change. So, this time, I am thinking of speeding up 
module builds by not rebuilding all the source if possible.

What I am thinking of is a relatively simple approach: detect changed 
files in a module and check if their "interface" changed. If it did, 
recompile the whole module. If it didn't, only compile the modified 
files. As a consequence, a small change inside a method body should lead 
to a fast build. Changes outside of method bodies may trigger longer 
build, but my hope would be that these would be less common.

So far, I unfortunately don't know how to efficiently do this as nicely 
as the API digests used for inter-module dependencies. The approach that 
seems might work is this: the Depend plugin hooks itself into javac 
internals, and filters the incoming files - it first parses the modified 
ones, and if it cannot find a significant change, it will throw away the 
unmodified files, and only compile the modified ones. (In principle, it 
could also do dependency analysis, if we at some point decide it is 
important.)

For a simple "touch Object.java && make", the wall-clock time is less 
then 5s, which sounds interesting:
---
$ touch src/java.base/share/classes/java/lang/Object.java && time make
Building target 'default (exploded-image)' in configuration 
'linux-x86_64-server-release'
Compiling 3028 files for java.base
Stopping sjavac server
Finished building target 'default (exploded-image)' in configuration 
'linux-x86_64-server-release'

real    0m3,104s
user    0m5,731s
sys     0m1,086s
---

My current prototype is in the jdk/sandbox repository, branch 
"jlahoda-depend-in-module":
http://hg.openjdk.java.net/jdk/sandbox/shortlog/jlahoda-depend-in-module

I wonder if this would sound interesting to developers working on base 
modules, like java.base.

What do you think? Any ideas/feedback?

Thanks,
      Jan