RFR(XS): 7175914: Usage of gcc with precompiled headers produces wrong build dependencies

[Volker Simonis]

On Tue, Jun 12, 2012 at 2:25 PM, Dmitry Samersoff
<Dmitry.Samersoff at oracle.com> wrote:
> Bengt,
>
> By default gcc *will not* list dependency to particular file if this
> file was included into precompiled header.
>

Yes, that's right and that was the exact problem.

You can also build on Linux without PCH by passing
USE_PRECOMPILED_HEADER=0 to the build.
In that case, no headers will be included into precompiled.hpp and for
each compilation unit headers
will be only included as explicitly requested by each .cpp file. As
you mentioned, the explicit includes
should be right "on their own" because there are platforms which do
not support PCH.
A common mistake is that users working with PCH don't notice that they
break the non-PCH
build because they forget to explicitly include required include files.

> -fpch-deps instruct gcc to list dependency to particular file ever if it
> included into PCH and gcc doesn't use it during compilation.
>
> i.e. dependency would be almost the same with or without PCH.
>

Well, almost...
I've only looked at some concrete files. So for example frame.cpp:

- without PCH, frame.o is dependant on 276 header files (273 of which
are unique) in frame.o.d
- with PCH, frame.o is dependant on 408 header files (380 of which are
unique) in frame.o.d

With PCH, the object files depend on more header files because each
object depends on all the
headers which are included into precompiled.hpp (plus the ones
explicitly included into the
compilation unit and not into precompiled.h). This may lead to
unnecessary recompiles if
PCHs are used but this is out-weighted by the fact that PCH compiles
are usually a magnitude
faster than usual builds.

Before this change we could have omitted necessary recompiles which is
much worse because
it could lead to incorrect build results.

I consider it a gcc bug, that the generated dependencies contain
duplicate entries (28 in the PCH
example, 3 in the example without PCH) but I hope that make will
handle them gracefully.

> So I think it's good changes.
>

Thank you. So do I:)

> -Dmitry
>
>
> On 2012-06-12 14:58, Bengt Rutisson wrote:
>>
>> Hi Volker,
>>
>> I am in no way an expert on precompiled headers, but for my own
>> understanding I have a question regarding this change.
>>
>> In hotspot there should be no files that rely on dependencies only being
>> included through precompiled headers. On Solaris we don't support
>> precompiled headers, so unless there are issues with Linux specific
>> files, there should not be any files that only rely on includes through
>> precompiled headers.
>>
>> You mention that the documentation for -fpch-deps is a bit fuzzy and I
>> have to agree. My question is: if gcc resolves a dependency through
>> precompiled headers will it then skip listing that dependency even if it
>> is explicitly present in the source file as an #include? Is this what
>> -fpch-deps fixes?
>>
>> Thanks,
>> Bengt
>>
>> On 2012-06-12 11:34, Frederic Parain wrote:
>>> Looks good.
>>>
>>> Could you just fix copyright year in make/bsd/makefiles/gcc.make
>>> and make/solaris/makefiles/gcc.make.
>>>
>>> Thanks,
>>>
>>> Fred
>>>
>>> On 06/11/12 06:32 PM, Volker Simonis wrote:
>>>> Hi,
>>>>
>>>> please review and push this little change which fixes the
>>>> dependencies for
>>>> gcc builds which use precompiled headers:
>>>>
>>>> http://cr.openjdk.java.net/~simonis/7175914/
>>>>
>>>> Below is some background:
>>>>
>>>> Dependencies which are resolved from the precompiled headers file during
>>>> compilation are not by default listed as dependencies in the
>>>> dependencies file
>>>> generated by gcc with the '-MMD' option. This leads to incomplete
>>>> dependencies
>>>> and potentially bogus incremental builds. To fix the problem, we have to
>>>> additionally use the '-fpch-deps' option which instructs gcc to include
>>>> dependencies from the precompiled headers file as well.
>>>>
>>>> Notice that the gcc documentation (at least for gcc 4.1 and 4.4) is
>>>> imprecise
>>>> with respect to precompiled headers and dependencies genereation. The
>>>> description of the flags for the generation of the dependency-files
>>>> (-M, -MM,
>>>> -MMD, ..) don't even mention that these flags may be impacted by the
>>>> use of
>>>> precompiled headers. The description of '-fpch-deps' is wrong in the
>>>> sense that
>>>> it claims that if it is not set, a dependency for the precompiled
>>>> header file
>>>> will be generated. This is not true for gcc 4.1 and 4.4. If
>>>> '-fpch-deps' is not
>>>> set, the generated dependencies file only contains dependencies which
>>>> are not
>>>> resolved trough the precompiled headers file. If '-fpch-deps' is set,
>>>> the
>>>> dependencies file contains all the dependencies, some of them even
>>>> several
>>>> times! While this is not nice and unnecessarily bloats the
>>>> dependencies files,
>>>> it is at least correct.
>>>>
>>>> Another undocumented pitfall is the fact that the precompiled header
>>>> file has
>>>> to be prepared with '-M' as well in order to make '-fpch-deps' work.
>>>> This is at
>>>> least suggested by the gcc bug
>>>> "28483 - Dependency tracking should be on by default when copiling
>>>> .gchs" [1].
>>>> Fortunately, the command line used by the HotSpot build to create the
>>>> precompiled headers file already contains these parameters, so we
>>>> don't have to
>>>> do anything in this case.
>>>>
>>>> And there's also another unresolved gcc bug
>>>> "14933 - missing pre-compiled header depends with -MD" [2]
>>>> which seems to be obsoleted by the '-fpch-deps' option but which I
>>>> want to mention
>>>> here for reference.
>>>>
>>>> Regards,
>>>> Volker
>>>>
>>>> [1] http://gcc.gnu.org/bugzilla/show_bug.cgi?id=28483
>>>> [2] http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14933
>>>
>>
>
>
> --
> Dmitry Samersoff
> Java Hotspot development team, SPB04
> * There will come soft rains ...