Faster incremental OpenJDK compilation

[Jonathan Gibbons]

Jan,

This seems like an interesting approach.

How are you determining "significant change"?  I could imagine trying to 
do this by looking at the changed lines, to see if they only within 
method bodies and comments (for example), or by doing some sort of 
lexical hash on the signatures, assuming that folk aren't messing with 
imports to change the resolution of simple type names in the signature.

-- Jon

On 5/6/20 8:30 AM, Jan Lahoda wrote:
> Hi,
>
> Triggered by Magnus' recent e-mail on adjusting the location of the 
> IDE files, I looked at possibilities to improve speed of incremental 
> compilation using make. About 3 years ago, we have sped up incremental 
> build by only rebuilding modules when API of modules they depend on 
> changed. But the module which contains modified sources is always 
> compiled in full. So, for changes in java.base, this change improved 
> the incremental build time significantly (by not recompiling e.g. 
> java.desktop), but it can still take many seconds to build java.base 
> after a trivial change. So, this time, I am thinking of speeding up 
> module builds by not rebuilding all the source if possible.
>
> What I am thinking of is a relatively simple approach: detect changed 
> files in a module and check if their "interface" changed. If it did, 
> recompile the whole module. If it didn't, only compile the modified 
> files. As a consequence, a small change inside a method body should 
> lead to a fast build. Changes outside of method bodies may trigger 
> longer build, but my hope would be that these would be less common.
>
> So far, I unfortunately don't know how to efficiently do this as 
> nicely as the API digests used for inter-module dependencies. The 
> approach that seems might work is this: the Depend plugin hooks itself 
> into javac internals, and filters the incoming files - it first parses 
> the modified ones, and if it cannot find a significant change, it will 
> throw away the unmodified files, and only compile the modified ones. 
> (In principle, it could also do dependency analysis, if we at some 
> point decide it is important.)
>
> For a simple "touch Object.java && make", the wall-clock time is less 
> then 5s, which sounds interesting:
> ---
> $ touch src/java.base/share/classes/java/lang/Object.java && time make
> Building target 'default (exploded-image)' in configuration 
> 'linux-x86_64-server-release'
> Compiling 3028 files for java.base
> Stopping sjavac server
> Finished building target 'default (exploded-image)' in configuration 
> 'linux-x86_64-server-release'
>
> real    0m3,104s
> user    0m5,731s
> sys     0m1,086s
> ---
>
> My current prototype is in the jdk/sandbox repository, branch 
> "jlahoda-depend-in-module":
> http://hg.openjdk.java.net/jdk/sandbox/shortlog/jlahoda-depend-in-module
>
> I wonder if this would sound interesting to developers working on base 
> modules, like java.base.
>
> What do you think? Any ideas/feedback?
>
> Thanks,
>      Jan
>