RFR: JDK-8217739: Cannot reuse java.base UnixConstants.java from target in BuildJDK when cross compiling

[Erik Joelsson]

Here is a new webrev: http://cr.openjdk.java.net/~erikj/8217739/webrev.03

The change is in buildjdk-spec.gmk.in where CPP is now also overridden.

/Erik

On 2019-06-05 10:33, Erik Joelsson wrote:
> Never mind, I found the issue. The reason it worked for me was that we 
> set SYSROOT_CFLAGS for both the target and the build compiler (from 
> using devkits for them). The preprocessor for the target would then 
> act correctly given the --sysroot parameter pointing to a sysroot for 
> the build system.
>
> I'm working on a patch where PPC gets overridden with the build 
> compiler in buildjdk-spec.gmk.
>
> /Erik
>
> On 2019-06-05 08:37, Erik Joelsson wrote:
>> Hello Ao Qi,
>>
>> In my testing, I tried building ppc64le (which I first identified as 
>> generating a UnixConstants.java that differed from an x64 build). 
>> With this patch, the buildjdk and target version of 
>> UnixConstants.java differ the same way. How did you configure your 
>> build? I'm suspecting something in the setup of the buildjdk 
>> compilers is different from my configuration.
>>
>> /Erik
>>
>> On 2019-06-04 21:48, Ao Qi wrote:
>>> Hi Erik,
>>>
>>> Thank you a lot for taking care of this. I mainly like your approach.
>>> I tested the patch by cross building mips64-zero, by which we found
>>> this problem. However, it seems the problem still exists. The reason I
>>> found is that UnixConstants.java of buildjdk is generated by cross
>>> compiler, so the macro definitions are defined by the target. Here is
>>> a preliminary workaround for your reference only (only worked for
>>> gcc):
>>>
>>> $ hg diff make/gensrc/GensrcMisc.gmk
>>> diff -r f4df9d4201cc make/gensrc/GensrcMisc.gmk
>>> --- a/make/gensrc/GensrcMisc.gmk Tue Jun 04 16:50:25 2019 -0700
>>> +++ b/make/gensrc/GensrcMisc.gmk Wed Jun 05 12:39:43 2019 +0800
>>> @@ -61,6 +61,13 @@
>>>     CPP_FLAGS += -nologo
>>>   endif
>>>
>>> +ifeq ($(CREATING_BUILDJDK), true)
>>> +  # Only worked for gcc
>>> +  TEMP_CPP=gcc -E
>>> +else
>>> +  TEMP_CPP=$(CPP)
>>> +endif
>>> +
>>>   # Generate a java source file from a template through the C
>>> preprocessor for the
>>>   # target system. First extract the copyright notice at the start 
>>> of the file.
>>>   # Run the preprocessor. Filter out the default compiler stderr 
>>> output on
>>> @@ -71,7 +78,7 @@
>>>   define generate-preproc-src
>>>    $(call MakeDir, $(@D))
>>>    ( $(NAWK) '/@@END_COPYRIGHT@@/{exit}1' $< && \
>>> -   $(CPP) $(CPP_FLAGS) $(SYSROOT_CFLAGS) $(CFLAGS_JDKLIB) $< \
>>> +   $(TEMP_CPP) $(CPP_FLAGS) $(SYSROOT_CFLAGS) $(CFLAGS_JDKLIB) $< \
>>>          2> >($(GREP) -v '^$(<F)$$' >&2) \
>>>          | $(NAWK) '/@@START_HERE@@/,0' \
>>>          |  $(SED) -e 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED
>>> FILE - DO NOT EDIT/' \
>>>
>>>
>>> Cheers,
>>> Ao Qi
>>>
>>> On Wed, Jun 5, 2019 at 12:00 AM Erik Joelsson 
>>> <erik.joelsson at oracle.com> wrote:
>>>> Thanks!
>>>>
>>>> During testing, I found an intermittent race issue. The command line
>>>> that runs the interim langtools version of javac still always 
>>>> points to
>>>> the buildtools/interim-langtools dir of the normal build. This 
>>>> variable
>>>> also needs to be rewritten in buildjdk-spec.gmk.in. New webrev, only
>>>> changed in buildjdk-spec.gmk.in:
>>>>
>>>> http://cr.openjdk.java.net/~erikj/8217739/webrev.02/
>>>>
>>>> /Erik
>>>>
>>>> On 2019-06-04 07:53, Tim Bell wrote:
>>>>> Erik:
>>>>>
>>>>> Looks good.
>>>>>
>>>>> Tim
>>>>>
>>>>>> This is a fix for a problem brought up on this list [1]. The main 
>>>>>> issue
>>>>>> is that when cross compiling, and creating a buildjdk during the 
>>>>>> build,
>>>>>> the UnixConstants.java cannot necessarily be reused between the 
>>>>>> target
>>>>>> JDK and the buildjdk. While investigating this, I came to the 
>>>>>> conclusion
>>>>>> that we really should stop trying to reuse anything between the 
>>>>>> buildjdk
>>>>>> and target JDK. It's just becoming too complex to maintain (due 
>>>>>> to the
>>>>>> intricate interactions with the interim-image, 
>>>>>> generate-link-opt-data
>>>>>> and exploded-image-optimize targets, causing circular 
>>>>>> dependencies and
>>>>>> endless incremental rebuilds).
>>>>>>
>>>>>> So this fix simply removes all the reuse of buildtools, gensrc 
>>>>>> and java
>>>>>> compilation when creating a buildjdk. Hopefully this is making 
>>>>>> Main.gmk
>>>>>> a bit less convoluted.
>>>>>>
>>>>>> While working on this, I came up with the idea of adding a log 
>>>>>> prefix
>>>>>> functionality, to more easily discern which log lines were 
>>>>>> printed by
>>>>>> the buildjdk build and which were part of the normal build. I 
>>>>>> liked this
>>>>>> a lot so including that in the same fix, and applied it to the 
>>>>>> bootcycle
>>>>>> build as well.
>>>>>>
>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217739
>>>>>>
>>>>>> Webrev: 
>>>>>> http://cr.openjdk.java.net/~erikj/8217739/webrev.01/index.html
>>>>>>
>>>>>> /Erik
>>>>>>
>>>>>> [1]
>>>>>> http://mail.openjdk.java.net/pipermail/build-dev/2019-January/024729.html 
>>>>>>
>>>>>>
>>>>>
>>>>>