RFR: JDK-8217739: Cannot reuse java.base UnixConstants.java from target in BuildJDK when cross compiling

[Ao Qi]

On Thu, Jun 6, 2019 at 4:46 AM Erik Joelsson <erik.joelsson at oracle.com> wrote:
>
> Here is a new webrev: http://cr.openjdk.java.net/~erikj/8217739/webrev.03

I (as an Author) am ok with this fix. mips64el zero cross build passed
and javac (to test open a file) worked.

UnixConstants.java of build:
.../usr/bin/cc -E -x c -I...

UnixConstants.java of target:
.../usr/bin/mips64el-linux-gnuabi64-gcc -E -x c
--sysroot=/home/aoqi//chroots/mips64el_stretch/ -I...

Thanks,
Ao Qi
>
> The change is in buildjdk-spec.gmk.in where CPP is now also overridden.
>
> /Erik
>
> On 2019-06-05 10:33, Erik Joelsson wrote:
> > Never mind, I found the issue. The reason it worked for me was that we
> > set SYSROOT_CFLAGS for both the target and the build compiler (from
> > using devkits for them). The preprocessor for the target would then
> > act correctly given the --sysroot parameter pointing to a sysroot for
> > the build system.
> >
> > I'm working on a patch where PPC gets overridden with the build
> > compiler in buildjdk-spec.gmk.
> >
> > /Erik
> >
> > On 2019-06-05 08:37, Erik Joelsson wrote:
> >> Hello Ao Qi,
> >>
> >> In my testing, I tried building ppc64le (which I first identified as
> >> generating a UnixConstants.java that differed from an x64 build).
> >> With this patch, the buildjdk and target version of
> >> UnixConstants.java differ the same way. How did you configure your
> >> build? I'm suspecting something in the setup of the buildjdk
> >> compilers is different from my configuration.
> >>
> >> /Erik
> >>
> >> On 2019-06-04 21:48, Ao Qi wrote:
> >>> Hi Erik,
> >>>
> >>> Thank you a lot for taking care of this. I mainly like your approach.
> >>> I tested the patch by cross building mips64-zero, by which we found
> >>> this problem. However, it seems the problem still exists. The reason I
> >>> found is that UnixConstants.java of buildjdk is generated by cross
> >>> compiler, so the macro definitions are defined by the target. Here is
> >>> a preliminary workaround for your reference only (only worked for
> >>> gcc):
> >>>
> >>> $ hg diff make/gensrc/GensrcMisc.gmk
> >>> diff -r f4df9d4201cc make/gensrc/GensrcMisc.gmk
> >>> --- a/make/gensrc/GensrcMisc.gmk Tue Jun 04 16:50:25 2019 -0700
> >>> +++ b/make/gensrc/GensrcMisc.gmk Wed Jun 05 12:39:43 2019 +0800
> >>> @@ -61,6 +61,13 @@
> >>>     CPP_FLAGS += -nologo
> >>>   endif
> >>>
> >>> +ifeq ($(CREATING_BUILDJDK), true)
> >>> +  # Only worked for gcc
> >>> +  TEMP_CPP=gcc -E
> >>> +else
> >>> +  TEMP_CPP=$(CPP)
> >>> +endif
> >>> +
> >>>   # Generate a java source file from a template through the C
> >>> preprocessor for the
> >>>   # target system. First extract the copyright notice at the start
> >>> of the file.
> >>>   # Run the preprocessor. Filter out the default compiler stderr
> >>> output on
> >>> @@ -71,7 +78,7 @@
> >>>   define generate-preproc-src
> >>>    $(call MakeDir, $(@D))
> >>>    ( $(NAWK) '/@@END_COPYRIGHT@@/{exit}1' $< && \
> >>> -   $(CPP) $(CPP_FLAGS) $(SYSROOT_CFLAGS) $(CFLAGS_JDKLIB) $< \
> >>> +   $(TEMP_CPP) $(CPP_FLAGS) $(SYSROOT_CFLAGS) $(CFLAGS_JDKLIB) $< \
> >>>          2> >($(GREP) -v '^$(<F)$$' >&2) \
> >>>          | $(NAWK) '/@@START_HERE@@/,0' \
> >>>          |  $(SED) -e 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED
> >>> FILE - DO NOT EDIT/' \
> >>>
> >>>
> >>> Cheers,
> >>> Ao Qi
> >>>
> >>> On Wed, Jun 5, 2019 at 12:00 AM Erik Joelsson
> >>> <erik.joelsson at oracle.com> wrote:
> >>>> Thanks!
> >>>>
> >>>> During testing, I found an intermittent race issue. The command line
> >>>> that runs the interim langtools version of javac still always
> >>>> points to
> >>>> the buildtools/interim-langtools dir of the normal build. This
> >>>> variable
> >>>> also needs to be rewritten in buildjdk-spec.gmk.in. New webrev, only
> >>>> changed in buildjdk-spec.gmk.in:
> >>>>
> >>>> http://cr.openjdk.java.net/~erikj/8217739/webrev.02/
> >>>>
> >>>> /Erik
> >>>>
> >>>> On 2019-06-04 07:53, Tim Bell wrote:
> >>>>> Erik:
> >>>>>
> >>>>> Looks good.
> >>>>>
> >>>>> Tim
> >>>>>
> >>>>>> This is a fix for a problem brought up on this list [1]. The main
> >>>>>> issue
> >>>>>> is that when cross compiling, and creating a buildjdk during the
> >>>>>> build,
> >>>>>> the UnixConstants.java cannot necessarily be reused between the
> >>>>>> target
> >>>>>> JDK and the buildjdk. While investigating this, I came to the
> >>>>>> conclusion
> >>>>>> that we really should stop trying to reuse anything between the
> >>>>>> buildjdk
> >>>>>> and target JDK. It's just becoming too complex to maintain (due
> >>>>>> to the
> >>>>>> intricate interactions with the interim-image,
> >>>>>> generate-link-opt-data
> >>>>>> and exploded-image-optimize targets, causing circular
> >>>>>> dependencies and
> >>>>>> endless incremental rebuilds).
> >>>>>>
> >>>>>> So this fix simply removes all the reuse of buildtools, gensrc
> >>>>>> and java
> >>>>>> compilation when creating a buildjdk. Hopefully this is making
> >>>>>> Main.gmk
> >>>>>> a bit less convoluted.
> >>>>>>
> >>>>>> While working on this, I came up with the idea of adding a log
> >>>>>> prefix
> >>>>>> functionality, to more easily discern which log lines were
> >>>>>> printed by
> >>>>>> the buildjdk build and which were part of the normal build. I
> >>>>>> liked this
> >>>>>> a lot so including that in the same fix, and applied it to the
> >>>>>> bootcycle
> >>>>>> build as well.
> >>>>>>
> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217739
> >>>>>>
> >>>>>> Webrev:
> >>>>>> http://cr.openjdk.java.net/~erikj/8217739/webrev.01/index.html
> >>>>>>
> >>>>>> /Erik
> >>>>>>
> >>>>>> [1]
> >>>>>> http://mail.openjdk.java.net/pipermail/build-dev/2019-January/024729.html
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>