Stop using precompiled headers for Linux?

Thu Nov 1 23:58:20 UTC 2018

On 2/11/2018 9:53 AM, Ioi Lam wrote:
> Maybe precompiled.hpp can be periodically (weekly?) updated by a robot, 
> which parses the dependencies files generated by gcc, and pick the most 
> popular N files?

Do we need to use precompiled.hpp at all? Can we not use the list of 
files contained in precompiled.hpp and precompile them individually?

Might be an interesting experiment.

David

> - Ioi
> 
> 
> On 11/1/18 4:38 PM, David Holmes wrote:
>> It's not at all obvious to me that the way we use PCH is the 
>> right/best way to use it. We dump every header we think it would be 
>> good to precompile into precompiled.hpp and then only ask gcc to 
>> precompile it. That results in a ~250MB file that has to be read into 
>> and processed for every source file! That doesn't seem very efficient 
>> to me.
>>
>> Cheers,
>> David
>>
>> On 2/11/2018 3:18 AM, Erik Joelsson wrote:
>>> Hello,
>>>
>>> My point here, which wasn't very clear, is that Mac and Linux seem to 
>>> lose just as much real compile time. The big difference in these 
>>> tests was rather the number of cpus in the machine (32 threads in the 
>>> linux box vs 8 on the mac). The total amount of work done was 
>>> increased when PCH was disabled, that's the user time. Here is my 
>>> theory on why the real (wall clock) time was not consistent with user 
>>> time between these experiments can be explained:
>>>
>>> With pch the time line (simplified) looks like this:
>>>
>>> 1. Single thread creating PCH
>>> 2. All cores compiling C++ files
>>>
>>> When disabling pch it's just:
>>>
>>> 1. All cores compiling C++ files
>>>
>>> To gain speed with PCH, the time spent in 1 much be less than the 
>>> time saved in 2. The potential time saved in 2 goes down as the 
>>> number of cpus go up. I'm pretty sure that if I repeated the 
>>> experiment on Linux on a smaller box (typically one we use in CI), 
>>> the results would look similar to Macosx, and similarly, if I had 
>>> access to a much bigger mac, it would behave like the big Linux box. 
>>> This is why I'm saying this should be done for both or none of these 
>>> platforms.
>>>
>>> In addition to this, the experiment only built hotspot. If you we 
>>> would instead build the whole JDK, then the time wasted in 1 in the 
>>> PCH case would be negated to a large extent by other build targets 
>>> running concurrently, so for a full build, PCH is still providing value.
>>>
>>> The question here is that if the value of PCH isn't very big, perhaps 
>>> it's not worth it if it's also creating as much grief as described 
>>> here. There is no doubt that there is value however. And given the 
>>> examination done by Magnus, it seems this value could be increased.
>>>
>>> The main reason why we haven't disabled PCH in CI before this. We 
>>> really really want to get CI builds fast. We don't have a ton of over 
>>> capacity to just throw at it. PCH made builds faster, so we used 
>>> them. My other reason is consistency between builds. Supporting 
>>> multiple different modes of building creates the potential for 
>>> inconsistencies. For that reason I would definitely not support 
>>> having PCH on by default, but turned off in our CI/dev-submit. We 
>>> pick one or the other as the official build configuration, and we 
>>> stick with the official build configuration for all builds of any 
>>> official capacity (which includes CI).
>>>
>>> In the current CI setup, we have a bunch of tiers that execute one 
>>> after the other. The jdk-submit currently only runs tier1. In tier2 
>>> I've put slowdebug builds with PCH disabled, just to help verify a 
>>> common developer configuration. These builds are not meant to be used 
>>> for testing or anything like that, they are just run for 
>>> verification, which is why this is ok. We could argue that it would 
>>> make sense to move the linux-x64-slowdebug without pch build to tier1 
>>> so that it's included in dev-submit.
>>>
>>> /Erik
>>>
>>> On 2018-11-01 03:38, Magnus Ihse Bursie wrote:
>>>>
>>>>
>>>> On 2018-10-31 00:54, Erik Joelsson wrote:
>>>>> Below are the corresponding numbers from a Mac, (Mac Pro (Late 
>>>>> 2013), 3.7 GHz, Quad-Core Intel Xeon E5, 16 GB). To be clear, the 
>>>>> -npch is without precompiled headers. Here we see a slight 
>>>>> degradation when disabling on both user time and wall clock time. 
>>>>> My guess is that the user time increase is about the same, but 
>>>>> because of a lower cpu count, the extra load is not as easily covered.
>>>>>
>>>>> These tests were run with just building hotspot. This means that 
>>>>> the precompiled header is generated alone on one core while nothing 
>>>>> else is happening, which would explain this degradation in build 
>>>>> speed. If we were instead building the whole product, we would see 
>>>>> a better correlation between user and real time.
>>>>>
>>>>> Given the very small benefit here, it could make sense to disable 
>>>>> precompiled headers by default for Linux and Mac, just as we did 
>>>>> with ccache.
>>>>>
>>>>> I do know that the benefit is huge on Windows though, so we cannot 
>>>>> remove the feature completely. Any other comments?
>>>>
>>>> Well, if you show that it is a loss in time on macosx to disable 
>>>> precompiled headers, and no-one (as far as I've seen) has complained 
>>>> about PCH on mac, then why not keep them on as default there? That 
>>>> the gain is small is no argument to lose it. (I remember a time when 
>>>> you were hunting seconds in the build time ;-))
>>>>
>>>> On linux, the story seems different, though. People experience PCH 
>>>> as a problem, and there is a net loss of time, at least on selected 
>>>> testing machines. It makes sense to turn it off as default, then.
>>>>
>>>> /Magnus
>>>>
>>>>>
>>>>> /Erik
>>>>>
>>>>> macosx-x64
>>>>> real     4m13.658s
>>>>> user     27m17.595s
>>>>> sys     2m11.306s
>>>>>
>>>>> macosx-x64-npch
>>>>> real     4m27.823s
>>>>> user     30m0.434s
>>>>> sys     2m18.669s
>>>>>
>>>>> macosx-x64-debug
>>>>> real     5m21.032s
>>>>> user     35m57.347s
>>>>> sys     2m20.588s
>>>>>
>>>>> macosx-x64-debug-npch
>>>>> real     5m33.728s
>>>>> user     38m10.311s
>>>>> sys     2m27.587s
>>>>>
>>>>> macosx-x64-slowdebug
>>>>> real     3m54.439s
>>>>> user     25m32.197s
>>>>> sys     2m8.750s
>>>>>
>>>>> macosx-x64-slowdebug-npch
>>>>> real     4m11.987s
>>>>> user     27m59.857s
>>>>> sys     2m18.093s
>>>>>
>>>>>
>>>>> On 2018-10-30 14:00, Erik Joelsson wrote:
>>>>>> Hello,
>>>>>>
>>>>>> On 2018-10-30 13:17, Aleksey Shipilev wrote:
>>>>>>> On 10/30/2018 06:26 PM, Ioi Lam wrote:
>>>>>>>> Is there any advantage of using precompiled headers on Linux?
>>>>>>> I have measured it recently on shenandoah repositories, and 
>>>>>>> fastdebug/release build times have not
>>>>>>> improved with or without PCH. Actually, it gets worse when you 
>>>>>>> touch a single header that is in PCH
>>>>>>> list, and you end up recompiling the entire Hotspot. I would be 
>>>>>>> in favor of disabling it by default.
>>>>>> I just did a measurement on my local workstation (2x8 cores x2 ht 
>>>>>> Ubuntu 18.04 using Oracle devkit GCC 7.3.0). I ran "time make 
>>>>>> hotspot" with clean build directories.
>>>>>>
>>>>>> linux-x64:
>>>>>> real    4m6.657s
>>>>>> user    61m23.090s
>>>>>> sys    6m24.477s
>>>>>>
>>>>>> linux-x64-npch
>>>>>> real    3m41.130s
>>>>>> user    66m11.824s
>>>>>> sys    4m19.224s
>>>>>>
>>>>>> linux-x64-debug
>>>>>> real    4m47.117s
>>>>>> user    75m53.740s
>>>>>> sys    8m21.408s
>>>>>>
>>>>>> linux-x64-debug-npch
>>>>>> real    4m42.877s
>>>>>> user    84m30.764s
>>>>>> sys    4m54.666s
>>>>>>
>>>>>> linux-x64-slowdebug
>>>>>> real    3m54.564s
>>>>>> user    44m2.828s
>>>>>> sys    6m22.785s
>>>>>>
>>>>>> linux-x64-slowdebug-npch
>>>>>> real    3m23.092s
>>>>>> user    55m3.142s
>>>>>> sys    4m10.172s
>>>>>>
>>>>>> These numbers support your claim. Wall clock time is actually 
>>>>>> increased with PCH enabled, but total user time is decreased. Does 
>>>>>> not seem worth it to me.
>>>>>>>> It's on by default and we keep having
>>>>>>>> breakage where someone would forget to add #include. The latest 
>>>>>>>> instance is JDK-8213148.
>>>>>>> Yes, we catch most of these breakages in CIs. Which tells me 
>>>>>>> adding it to jdk-submit would cover
>>>>>>> most of the breakage during pre-integration testing.
>>>>>> jdk-submit is currently running what we call "tier1". We do have 
>>>>>> builds of Linux slowdebug with precompiled headers disabled in 
>>>>>> tier2. We also build solaris-sparcv9 in tier1 which does not 
>>>>>> support precompiled headers at all, so to not be caught in 
>>>>>> jdk-submit you would have to be in Linux specific code. The 
>>>>>> example bug does not seem to be that. Mach5/jdk-submit was down 
>>>>>> over the weekend and yesterday so my suspicion is the offending 
>>>>>> code in this case was never tested.
>>>>>>
>>>>>> That said, given that we get practically no benefit from PCH on 
>>>>>> Linux/GCC, we should probably just turn it off by default for 
>>>>>> Linux and/or GCC. I think we need to investigate Macos as well here.
>>>>>>
>>>>>> /Erik
>>>>>>> -Aleksey
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
> 