Faster incremental OpenJDK compilation
jan.lahoda at oracle.com
Wed May 6 16:52:54 UTC 2020
FWIW, I should explain this only relates to the OpenJDK build. This is
not a proposal to do any kind of tweaks like this in javac for other
compilations. I realized I only wrote that into the subject, but not the
body. Sorry for that.
On 06. 05. 20 18:37, Jan Lahoda wrote:
> On 06. 05. 20 17:56, Remi Forax wrote:
>> ----- Mail original -----
>>> De: "jan lahoda" <jan.lahoda at oracle.com>
>>> À: ide-support-dev at openjdk.java.net, "compiler-dev"
>>> <compiler-dev at openjdk.java.net>
>>> Envoyé: Mercredi 6 Mai 2020 17:30:52
>>> Objet: Faster incremental OpenJDK compilation
>>> Triggered by Magnus' recent e-mail on adjusting the location of the IDE
>>> files, I looked at possibilities to improve speed of incremental
>>> compilation using make. About 3 years ago, we have sped up incremental
>>> build by only rebuilding modules when API of modules they depend on
>>> changed. But the module which contains modified sources is always
>>> compiled in full. So, for changes in java.base, this change improved the
>>> incremental build time significantly (by not recompiling e.g.
>>> java.desktop), but it can still take many seconds to build java.base
>>> after a trivial change. So, this time, I am thinking of speeding up
>>> module builds by not rebuilding all the source if possible.
>>> What I am thinking of is a relatively simple approach: detect changed
>>> files in a module and check if their "interface" changed. If it did,
>>> recompile the whole module. If it didn't, only compile the modified
>>> files. As a consequence, a small change inside a method body should lead
>>> to a fast build. Changes outside of method bodies may trigger longer
>>> build, but my hope would be that these would be less common.
>>> So far, I unfortunately don't know how to efficiently do this as nicely
>>> as the API digests used for inter-module dependencies. The approach that
>>> seems might work is this: the Depend plugin hooks itself into javac
>>> internals, and filters the incoming files - it first parses the modified
>>> ones, and if it cannot find a significant change, it will throw away the
>>> unmodified files, and only compile the modified ones. (In principle, it
>>> could also do dependency analysis, if we at some point decide it is
>> I think you are missing the dependencies that are changing the
>> generated code,
> Do you mean for the current hashing, or for the new hashing?
> The new hashing is more sketchy now (I just realized modifiers are not
> reflected in the hash properly). But in both cases, I believe the intent
> is to make the hashing conservative (and easy), potentially causing more
> rebuilds than necessary, although I could see some arguments for being
> more optimistic. (This is basically only for people developing OpenJDK,
> after all.)
>> - in a switch, javac generates a different code depending on the order
>> of the enum constants,
> I believe that any change to order of significant members will change
> the API hashes. (Which, admittedly, might be overly conservative.)
>> - static final are inlined in the code,
> For the existing API hash, changes to compile-time constants should
> change the hash. For the new intra-module hash, currently, (significant)
> fields initializers are included in full (this is probably overly
>> - for valhalla, changing a class to an inline class or vice versa
>> change all the method/field descriptors
> As long as that is reflected in the modifiers, we should be able to
> handle fine. If not, then the hashing may need some more tweaks.
>> and i'm sure there are more ..
> Possibly - and it is entirely possible the current hashes are way too
> conservative. But what I wonder about is if people developing e.g.
> java.base would find such faster incremental builds useful.
>>> For a simple "touch Object.java && make", the wall-clock time is less
>>> then 5s, which sounds interesting:
>>> $ touch src/java.base/share/classes/java/lang/Object.java && time make
>>> Building target 'default (exploded-image)' in configuration
>>> Compiling 3028 files for java.base
>>> Stopping sjavac server
>>> Finished building target 'default (exploded-image)' in configuration
>>> real 0m3,104s
>>> user 0m5,731s
>>> sys 0m1,086s
>>> My current prototype is in the jdk/sandbox repository, branch
>>> I wonder if this would sound interesting to developers working on base
>>> modules, like java.base.
>>> What do you think? Any ideas/feedback?
More information about the ide-support-dev