RFR: JDK-8146403: Windows build can be faster

[Erik Joelsson]

Hello,

New webrev: http://cr.openjdk.java.net/~erikj/8146403/webrev.02/

Fixed the below comments. Also adjusted some more gensrc log output to 
make it more uniform, even if it doesn't exactly address Windows 
performance.

I also fixed a bug in the bootstrap make logic. The module-deps.gmk 
wasn't correctly generated in all cases. Typically when building more 
than one conf at a time or when doing a compare build. I added explicit 
calls to GenerateModuleDeps.gmk to fix this.

/Erik

On 2016-01-13 12:48, Magnus Ihse Bursie wrote:
> Hi,
>
> In InitSupport.gmk, please restore the comment:
>  # Only do this if make has not been restarted, and if we do not force 
> it.
>
> In jdk/make/gensrc/GensrcExceptions.gmk:
> The old construct resulted in the output "Generating exceptions 
> classes" (but a bit irregularly), could you please re-add it as a 
> LogInfo?
>
> In make/CompileTools.gmk, the copyright header says 2014, which is at 
> least one year too early. :-) (In fact, all modified files should 
> really get bumped to 2016).
>
> Apart from this, it looks good to me.
>
> /Magnus
>
> On 2016-01-05 11:58, Erik Joelsson wrote:
>> Hello,
>>
>> During the hotspot makefile conversion, we have been reminded of 
>> inefficiencies when running make in Cygwin. We still have a pretty 
>> severe performance regression in the new hotspot build compared to 
>> the old on Windows in certain situations, my laptop being one such 
>> situation. A recent comparison there between just the old and new 
>> hotspot build:
>>
>> release new 00:04:30
>> release old 00:03:10
>> fastdebug new 00:04:59
>> fastdebug old 00:03:37
>>
>> Much of the extra time is spent spawning various shell processes for 
>> book keeping, like saving failure logs, creating header dependency 
>> files etc.
>>
>> It also takes a very long time to do a "do nothing" rebuild when 
>> nothing has changed. On my laptop, repeating "make jimages" often 
>> takes as long as 40 seconds to figure out that nothing needs to be 
>> rebuilt. In this case there are several culprits.
>>
>> I have been working on improvements in these areas to reduce the 
>> overhead. The "do nothing" rebuild of jimages is down to around 25 
>> seconds. A full images build is around 1-2 minutes faster from 24 to 
>> 22 minutes, but fluctuates quite a bit. The new hotspot build is also 
>> improved:
>>
>> release: 3:44
>> fastdebug: 4:02
>>
>> Here is a list of the kinds of changes I've made:
>>
>> * Rewrote logger.sh to use a different construct. It drastically 
>> reduces the number of bash processes being spawned. Basically you can 
>> pipe like this: "> >(tee logfile) 2> >(tee logfile >&2)". I also 
>> managed to reduce the extra sed/grep commands needed for the header 
>> dependencies even more. A side effect of this is that the log files 
>> for each native compilation unit are always saved instead of being 
>> deleted on successful compilation. I see no issue with this.
>>
>> * Changed all recipes that contain echo for logging to instead use 
>> new LogInfo macro, which in turn calls $(info ) if appropriate.
>>
>> * Changed all recipes that contain mkdir to instead use MakeDir 
>> macro, which only executes mkdir if the directory doesn't exist.
>>
>> * In HotspotWrapper.gmk there is an optimization to avoid calling the 
>> hotspot makefiles at all if nothing has changed. This runs a find 
>> over the whole hotspot repository. I reduced this to only run over 
>> src and make sub dirs to avoid the pretty large test dir (since 
>> test/closed has grown quite a bit).
>>
>> * Split Tools.gmk into CompileTools.gmk and Tools.gmk, to avoid 
>> having the buildtools compilation being reevaluated by all makefiles 
>> needing the tools definitions.
>>
>> * Split Modules.gmk to avoid having the module deps generation being 
>> reevaluated multiple times. Made the new GenerateModuleDeps.gmk an 
>> explicit call from Init.gmk.
>>
>> Since these improvements affect much more than just the new hotspot 
>> build, I intend to push this to JDK 9 directly.
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8146403
>> Webrev: http://cr.openjdk.java.net/~erikj/8146403/webrev.01/
>>
>> /Erik
>