Faster incremental OpenJDK compilation
jonathan.gibbons at oracle.com
Wed May 6 17:23:46 UTC 2020
You could refine the imports by filtering them according to whether they
are 'star-imports' or the simple name matches a simple name that has
been seen by the TreeScanner. In other words, try and filter out
imports that are definitely not relevant.
On 5/6/20 9:49 AM, Jan Lahoda wrote:
> Hi Jon,
> Good question. I was first experimenting with hashing Elements, as we
> do currently for the module API hashes, but it was too slow to enter
> the sources (although it is not completely impossible). So I did a new
> hashing based on the AST - basically, it is a TreeScanner sending Tree
> kinds, names and sub-trees into a MessageDigest. It ignores things
> like method bodies, class initializers, and private class members.
> Does not currently skip imports, although that may be seen as too
> conservative. But tweaks to the AST hashing are surely still needed,
> so we can tweak the exact meaning of a "significant change".
> On 06. 05. 20 18:07, Jonathan Gibbons wrote:
>> This seems like an interesting approach.
>> How are you determining "significant change"? I could imagine trying
>> to do this by looking at the changed lines, to see if they only
>> within method bodies and comments (for example), or by doing some
>> sort of lexical hash on the signatures, assuming that folk aren't
>> messing with imports to change the resolution of simple type names in
>> the signature.
>> -- Jon
>> On 5/6/20 8:30 AM, Jan Lahoda wrote:
>>> Triggered by Magnus' recent e-mail on adjusting the location of the
>>> IDE files, I looked at possibilities to improve speed of incremental
>>> compilation using make. About 3 years ago, we have sped up
>>> incremental build by only rebuilding modules when API of modules
>>> they depend on changed. But the module which contains modified
>>> sources is always compiled in full. So, for changes in java.base,
>>> this change improved the incremental build time significantly (by
>>> not recompiling e.g. java.desktop), but it can still take many
>>> seconds to build java.base after a trivial change. So, this time, I
>>> am thinking of speeding up module builds by not rebuilding all the
>>> source if possible.
>>> What I am thinking of is a relatively simple approach: detect
>>> changed files in a module and check if their "interface" changed. If
>>> it did, recompile the whole module. If it didn't, only compile the
>>> modified files. As a consequence, a small change inside a method
>>> body should lead to a fast build. Changes outside of method bodies
>>> may trigger longer build, but my hope would be that these would be
>>> less common.
>>> So far, I unfortunately don't know how to efficiently do this as
>>> nicely as the API digests used for inter-module dependencies. The
>>> approach that seems might work is this: the Depend plugin hooks
>>> itself into javac internals, and filters the incoming files - it
>>> first parses the modified ones, and if it cannot find a significant
>>> change, it will throw away the unmodified files, and only compile
>>> the modified ones. (In principle, it could also do dependency
>>> analysis, if we at some point decide it is important.)
>>> For a simple "touch Object.java && make", the wall-clock time is
>>> less then 5s, which sounds interesting:
>>> $ touch src/java.base/share/classes/java/lang/Object.java && time make
>>> Building target 'default (exploded-image)' in configuration
>>> Compiling 3028 files for java.base
>>> Stopping sjavac server
>>> Finished building target 'default (exploded-image)' in configuration
>>> real 0m3,104s
>>> user 0m5,731s
>>> sys 0m1,086s
>>> My current prototype is in the jdk/sandbox repository, branch
>>> I wonder if this would sound interesting to developers working on
>>> base modules, like java.base.
>>> What do you think? Any ideas/feedback?
More information about the ide-support-dev