RFR: JDK-8213339 Update precompiled.hpp with headers based on current frequency

Mon Nov 5 09:41:39 UTC 2018

On 2018-11-05 08:28, David Holmes wrote:
> Hi Thomas,
>
> On 5/11/2018 5:17 PM, Thomas Stüfe wrote:
>> Hi David,
>>
>> On Sun, Nov 4, 2018 at 11:09 PM David Holmes 
>> <david.holmes at oracle.com> wrote:
>>>
>>> On 4/11/2018 5:27 PM, Magnus Ihse Bursie wrote:
>>>>
>>>>> 3 nov. 2018 kl. 23:24 skrev David Holmes <david.holmes at oracle.com>:
>>>>>
>>>>>> On 3/11/2018 10:09 PM, David Holmes wrote:
>>>>>> Looks okay - thanks for doing all the experiments! (Though I'm 
>>>>>> still curious what happens if you recompile individual header 
>>>>>> files :) ).
>>>>>
>>>>> s/recompile/precompile/  :)
>>>>
>>>> What do you mean? To have more than one set of PCH? As far as I 
>>>> know, all compilers we use only support a single PCH per source 
>>>> file to compile. So while you could have a PCH tailored per source 
>>>> file, that would only make sense in a scenario were you recompile 
>>>> all source files (but no header files) often.
>>>
>>> The GCC description for PCH describes it as a per-file header file
>>> optimization. You can precompile each .hpp file and if it is located in
>>> the same directory as the .hpp (or is on an include directory ahead of
>>> the .hpp) then the PCH file will be used instead. So rather than
>>> precompiling precompiled.hpp and having source files include it, you
>>> would precompile each individual (chosen) header file and have that in
>>> the special PCH include directory, and they would then be included 
>>> where
>>> a source file asked for the corresponding .hpp file.
>>>
>>
>> I assume precompiling a header means precompiling it itself and any
>> header it includes, no? In that case, would precompiling each header
>> individually not cause us to implicitly precompile included headers
>> many times (e.g. globalDefinitions.hpp)?
>
> I presume you'd have to figure out the order in which to do the 
> precompilation to avoid that. The makefile aspects of this seems 
> non-trivial compared to precompiled.hpp.
I see. I think this is something we might have in mind for perhaps some 
kind of future improvement. The gcc behavior you describe is not shared 
by any other compiler, as far as I know, which also makes this a bit of 
a special solution. Since additional speed gains on gcc seems not like 
our most pressing issue, I can't say I'm interested right now in making 
a special solution.

If I were to spend more time on PCH, I'd rather spend it on either 
Windows (which is the slowest build platform in our CI), or Solaris, 
which does not have precompiled headers at all. (I actually looked into 
solstudio PCH as part of this fix to see if the new set of headers would 
allow us to enable it. Unfortunately, it seems like they need a single 
PCH per directory of source code files (e.g. all files in share/logging 
can share PCH with each other, but they cannot share PCH with the files 
in share/utilities. As I understood the problem, at least; the error 
messages were unclear and undocumented.)

/Magnus

>
> Cheers,
> David
>
>> Thanks, Thomas
>>
>>> That would also remove the problem of things not compiling with PCH
>>> disabled as a single set of includes would be used for both PCH and
>>> non-PCH building.
>>>
>>> I don't know if the other compilers support PCH in a similar manner 
>>> to gcc.
>>>
>>> Cheers,
>>> David
>>>
>>>> /Magnus
>>>>
>>>>>
>>>>> David
>>>>>
>>>>>>    25 // Precompiled headers are turned off for Sun Studio,
>>>>>> May as well change to Solaris Studio if you're going to fix the 
>>>>>> typo :)
>>>>>> Thanks.
>>>>>> David
>>>>>>> On 3/11/2018 7:06 PM, Magnus Ihse Bursie wrote:
>>>>>>> The reasons for the current set of files included in 
>>>>>>> precompiled.hpp is somewhat lost in the mists of history. 
>>>>>>> However, it is clear that it is not optimal.
>>>>>>>
>>>>>>> This patch replaces the current set with a new set, based on how 
>>>>>>> often a header file is included in a C++ file. This selection 
>>>>>>> contains all header files that are included by at least 130 C++ 
>>>>>>> files. Testing has shown that this is around the optimal value 
>>>>>>> -- include many more, and too many "innocent" files get hurt by 
>>>>>>> unneeded work, leave out many more, and we miss out on 
>>>>>>> optimization possibilities.
>>>>>>>
>>>>>>> The same set turned out to work well for both clang and gcc. 
>>>>>>> However, files named "*.inline.hpp" did hurt rather than help 
>>>>>>> performance, so those have been left out. For visual studio, the 
>>>>>>> same set was also optimal, as long as the inline files were 
>>>>>>> included. Presumably, visual studio is better than gcc/clang on 
>>>>>>> handling precompiled headers containing inlined code.
>>>>>>>
>>>>>>> Here are some rough comparisons from our internal CI system, for 
>>>>>>> building the target "hotspot" from scratch.
>>>>>>>
>>>>>>> macosx-x64:
>>>>>>> old: 00:05:00
>>>>>>> new: 00:03:47
>>>>>>>
>>>>>>> linux-x64:
>>>>>>> old: 00:05:43
>>>>>>> new: 00:04:51
>>>>>>>
>>>>>>> windows-x64:
>>>>>>> old: 00:05:18
>>>>>>> new: 00:04:33
>>>>>>>
>>>>>>> linux-aarch64:
>>>>>>> old: 00:07:57
>>>>>>> new: 00:03:48
>>>>>>>
>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213339
>>>>>>> WebRev: 
>>>>>>> http://cr.openjdk.java.net/~ihse/JDK-8213339-update-precompiled-headers/webrev.01
>>>>>>>
>>>>>>> /Magnus
>>>>