RFR(XS): 7175914: Usage of gcc with precompiled headers produces wrong build dependencies

[Bengt Rutisson]

Volker and Dmitry,

Thanks for the extra explanations!

I feel much more comfortable with this change now - which is good since 
it has already been pushed ;-)

Thanks,
Bengt

On 2012-06-12 15:20, Volker Simonis wrote:
> On Tue, Jun 12, 2012 at 2:25 PM, Dmitry Samersoff
> <Dmitry.Samersoff at oracle.com>  wrote:
>> Bengt,
>>
>> By default gcc *will not* list dependency to particular file if this
>> file was included into precompiled header.
>>
> Yes, that's right and that was the exact problem.
>
> You can also build on Linux without PCH by passing
> USE_PRECOMPILED_HEADER=0 to the build.
> In that case, no headers will be included into precompiled.hpp and for
> each compilation unit headers
> will be only included as explicitly requested by each .cpp file. As
> you mentioned, the explicit includes
> should be right "on their own" because there are platforms which do
> not support PCH.
> A common mistake is that users working with PCH don't notice that they
> break the non-PCH
> build because they forget to explicitly include required include files.
>
>> -fpch-deps instruct gcc to list dependency to particular file ever if it
>> included into PCH and gcc doesn't use it during compilation.
>>
>> i.e. dependency would be almost the same with or without PCH.
>>
> Well, almost...
> I've only looked at some concrete files. So for example frame.cpp:
>
> - without PCH, frame.o is dependant on 276 header files (273 of which
> are unique) in frame.o.d
> - with PCH, frame.o is dependant on 408 header files (380 of which are
> unique) in frame.o.d
>
> With PCH, the object files depend on more header files because each
> object depends on all the
> headers which are included into precompiled.hpp (plus the ones
> explicitly included into the
> compilation unit and not into precompiled.h). This may lead to
> unnecessary recompiles if
> PCHs are used but this is out-weighted by the fact that PCH compiles
> are usually a magnitude
> faster than usual builds.
>
> Before this change we could have omitted necessary recompiles which is
> much worse because
> it could lead to incorrect build results.
>
> I consider it a gcc bug, that the generated dependencies contain
> duplicate entries (28 in the PCH
> example, 3 in the example without PCH) but I hope that make will
> handle them gracefully.
>
>> So I think it's good changes.
>>
> Thank you. So do I:)
>
>> -Dmitry
>>
>>
>> On 2012-06-12 14:58, Bengt Rutisson wrote:
>>> Hi Volker,
>>>
>>> I am in no way an expert on precompiled headers, but for my own
>>> understanding I have a question regarding this change.
>>>
>>> In hotspot there should be no files that rely on dependencies only being
>>> included through precompiled headers. On Solaris we don't support
>>> precompiled headers, so unless there are issues with Linux specific
>>> files, there should not be any files that only rely on includes through
>>> precompiled headers.
>>>
>>> You mention that the documentation for -fpch-deps is a bit fuzzy and I
>>> have to agree. My question is: if gcc resolves a dependency through
>>> precompiled headers will it then skip listing that dependency even if it
>>> is explicitly present in the source file as an #include? Is this what
>>> -fpch-deps fixes?
>>>
>>> Thanks,
>>> Bengt
>>>
>>> On 2012-06-12 11:34, Frederic Parain wrote:
>>>> Looks good.
>>>>
>>>> Could you just fix copyright year in make/bsd/makefiles/gcc.make
>>>> and make/solaris/makefiles/gcc.make.
>>>>
>>>> Thanks,
>>>>
>>>> Fred
>>>>
>>>> On 06/11/12 06:32 PM, Volker Simonis wrote:
>>>>> Hi,
>>>>>
>>>>> please review and push this little change which fixes the
>>>>> dependencies for
>>>>> gcc builds which use precompiled headers:
>>>>>
>>>>> http://cr.openjdk.java.net/~simonis/7175914/
>>>>>
>>>>> Below is some background:
>>>>>
>>>>> Dependencies which are resolved from the precompiled headers file during
>>>>> compilation are not by default listed as dependencies in the
>>>>> dependencies file
>>>>> generated by gcc with the '-MMD' option. This leads to incomplete
>>>>> dependencies
>>>>> and potentially bogus incremental builds. To fix the problem, we have to
>>>>> additionally use the '-fpch-deps' option which instructs gcc to include
>>>>> dependencies from the precompiled headers file as well.
>>>>>
>>>>> Notice that the gcc documentation (at least for gcc 4.1 and 4.4) is
>>>>> imprecise
>>>>> with respect to precompiled headers and dependencies genereation. The
>>>>> description of the flags for the generation of the dependency-files
>>>>> (-M, -MM,
>>>>> -MMD, ..) don't even mention that these flags may be impacted by the
>>>>> use of
>>>>> precompiled headers. The description of '-fpch-deps' is wrong in the
>>>>> sense that
>>>>> it claims that if it is not set, a dependency for the precompiled
>>>>> header file
>>>>> will be generated. This is not true for gcc 4.1 and 4.4. If
>>>>> '-fpch-deps' is not
>>>>> set, the generated dependencies file only contains dependencies which
>>>>> are not
>>>>> resolved trough the precompiled headers file. If '-fpch-deps' is set,
>>>>> the
>>>>> dependencies file contains all the dependencies, some of them even
>>>>> several
>>>>> times! While this is not nice and unnecessarily bloats the
>>>>> dependencies files,
>>>>> it is at least correct.
>>>>>
>>>>> Another undocumented pitfall is the fact that the precompiled header
>>>>> file has
>>>>> to be prepared with '-M' as well in order to make '-fpch-deps' work.
>>>>> This is at
>>>>> least suggested by the gcc bug
>>>>> "28483 - Dependency tracking should be on by default when copiling
>>>>> .gchs" [1].
>>>>> Fortunately, the command line used by the HotSpot build to create the
>>>>> precompiled headers file already contains these parameters, so we
>>>>> don't have to
>>>>> do anything in this case.
>>>>>
>>>>> And there's also another unresolved gcc bug
>>>>> "14933 - missing pre-compiled header depends with -MD" [2]
>>>>> which seems to be obsoleted by the '-fpch-deps' option but which I
>>>>> want to mention
>>>>> here for reference.
>>>>>
>>>>> Regards,
>>>>> Volker
>>>>>
>>>>> [1] http://gcc.gnu.org/bugzilla/show_bug.cgi?id=28483
>>>>> [2] http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14933
>>
>> --
>> Dmitry Samersoff
>> Java Hotspot development team, SPB04
>> * There will come soft rains ...