RFR: JDK-8217739: Cannot reuse java.base UnixConstants.java from target in BuildJDK when cross compiling

[Erik Joelsson]

Never mind, I found the issue. The reason it worked for me was that we 
set SYSROOT_CFLAGS for both the target and the build compiler (from 
using devkits for them). The preprocessor for the target would then act 
correctly given the --sysroot parameter pointing to a sysroot for the 
build system.

I'm working on a patch where PPC gets overridden with the build compiler 
in buildjdk-spec.gmk.

/Erik

On 2019-06-05 08:37, Erik Joelsson wrote:
> Hello Ao Qi,
>
> In my testing, I tried building ppc64le (which I first identified as 
> generating a UnixConstants.java that differed from an x64 build). With 
> this patch, the buildjdk and target version of UnixConstants.java 
> differ the same way. How did you configure your build? I'm suspecting 
> something in the setup of the buildjdk compilers is different from my 
> configuration.
>
> /Erik
>
> On 2019-06-04 21:48, Ao Qi wrote:
>> Hi Erik,
>>
>> Thank you a lot for taking care of this. I mainly like your approach.
>> I tested the patch by cross building mips64-zero, by which we found
>> this problem. However, it seems the problem still exists. The reason I
>> found is that UnixConstants.java of buildjdk is generated by cross
>> compiler, so the macro definitions are defined by the target. Here is
>> a preliminary workaround for your reference only (only worked for
>> gcc):
>>
>> $ hg diff make/gensrc/GensrcMisc.gmk
>> diff -r f4df9d4201cc make/gensrc/GensrcMisc.gmk
>> --- a/make/gensrc/GensrcMisc.gmk Tue Jun 04 16:50:25 2019 -0700
>> +++ b/make/gensrc/GensrcMisc.gmk Wed Jun 05 12:39:43 2019 +0800
>> @@ -61,6 +61,13 @@
>>     CPP_FLAGS += -nologo
>>   endif
>>
>> +ifeq ($(CREATING_BUILDJDK), true)
>> +  # Only worked for gcc
>> +  TEMP_CPP=gcc -E
>> +else
>> +  TEMP_CPP=$(CPP)
>> +endif
>> +
>>   # Generate a java source file from a template through the C
>> preprocessor for the
>>   # target system. First extract the copyright notice at the start of 
>> the file.
>>   # Run the preprocessor. Filter out the default compiler stderr 
>> output on
>> @@ -71,7 +78,7 @@
>>   define generate-preproc-src
>>    $(call MakeDir, $(@D))
>>    ( $(NAWK) '/@@END_COPYRIGHT@@/{exit}1' $< && \
>> -   $(CPP) $(CPP_FLAGS) $(SYSROOT_CFLAGS) $(CFLAGS_JDKLIB) $< \
>> +   $(TEMP_CPP) $(CPP_FLAGS) $(SYSROOT_CFLAGS) $(CFLAGS_JDKLIB) $< \
>>          2> >($(GREP) -v '^$(<F)$$' >&2) \
>>          | $(NAWK) '/@@START_HERE@@/,0' \
>>          |  $(SED) -e 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED
>> FILE - DO NOT EDIT/' \
>>
>>
>> Cheers,
>> Ao Qi
>>
>> On Wed, Jun 5, 2019 at 12:00 AM Erik Joelsson 
>> <erik.joelsson at oracle.com> wrote:
>>> Thanks!
>>>
>>> During testing, I found an intermittent race issue. The command line
>>> that runs the interim langtools version of javac still always points to
>>> the buildtools/interim-langtools dir of the normal build. This variable
>>> also needs to be rewritten in buildjdk-spec.gmk.in. New webrev, only
>>> changed in buildjdk-spec.gmk.in:
>>>
>>> http://cr.openjdk.java.net/~erikj/8217739/webrev.02/
>>>
>>> /Erik
>>>
>>> On 2019-06-04 07:53, Tim Bell wrote:
>>>> Erik:
>>>>
>>>> Looks good.
>>>>
>>>> Tim
>>>>
>>>>> This is a fix for a problem brought up on this list [1]. The main 
>>>>> issue
>>>>> is that when cross compiling, and creating a buildjdk during the 
>>>>> build,
>>>>> the UnixConstants.java cannot necessarily be reused between the 
>>>>> target
>>>>> JDK and the buildjdk. While investigating this, I came to the 
>>>>> conclusion
>>>>> that we really should stop trying to reuse anything between the 
>>>>> buildjdk
>>>>> and target JDK. It's just becoming too complex to maintain (due to 
>>>>> the
>>>>> intricate interactions with the interim-image, generate-link-opt-data
>>>>> and exploded-image-optimize targets, causing circular dependencies 
>>>>> and
>>>>> endless incremental rebuilds).
>>>>>
>>>>> So this fix simply removes all the reuse of buildtools, gensrc and 
>>>>> java
>>>>> compilation when creating a buildjdk. Hopefully this is making 
>>>>> Main.gmk
>>>>> a bit less convoluted.
>>>>>
>>>>> While working on this, I came up with the idea of adding a log prefix
>>>>> functionality, to more easily discern which log lines were printed by
>>>>> the buildjdk build and which were part of the normal build. I 
>>>>> liked this
>>>>> a lot so including that in the same fix, and applied it to the 
>>>>> bootcycle
>>>>> build as well.
>>>>>
>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217739
>>>>>
>>>>> Webrev: 
>>>>> http://cr.openjdk.java.net/~erikj/8217739/webrev.01/index.html
>>>>>
>>>>> /Erik
>>>>>
>>>>> [1]
>>>>> http://mail.openjdk.java.net/pipermail/build-dev/2019-January/024729.html 
>>>>>
>>>>>
>>>>
>>>>