RFR (XS): Enable new build on Linux/PPC64 (jdk part)
Vladimir Kozlov
vladimir.kozlov at oracle.com
Tue Jul 2 07:56:34 UTC 2013
JPRT uses own target to build forest:
jprt_build_product: sanity all_product_build
http://hg.openjdk.java.net/ppc-aix-port/stage/file/c156084add48/make/jprt.gmk
That is why 'make sanity' is called I think.
Note the next warning was present in JPRT logs on all platforms:
Warning: The generated configure file contains changes not present in the custom generated file.
I verified that submitted source common/autoconf/generated-configure.sh is matching file in stage repo plus changes from
patch.
Hmm, may be the problem is next change in generated-configure.sh:
-DATE_WHEN_GENERATED=1371547824
+DATE_WHEN_GENERATED=1372238067
It is date when Volker generated generated-configure.sh but the date of changed .m4 files is when I applied patch which
is later. It looks like I need to regenerate generated-configure.sh file myself as Erik said. I will give it a try tomorrow.
Vladimir
On 7/2/13 12:31 AM, Volker Simonis wrote:
> Hi Erik,
>
> thank you for looking into this.
>
> On Tue, Jul 2, 2013 at 9:16 AM, Erik Joelsson <erik.joelsson at oracle.com> wrote:
>> This looks like you have applied a change to configure input files (*.m4)
>> and haven't regenerated configure locally afterwards. The JPRT machines
>> don't have autoconf setup so they can't do it. In this case it looks like
>> one of them, probably the mac, has a faulty version of autoconf that gets
>> picked up. I have seen this happen before on mac.
>>
>
> Actually, that was my guess as well. On the other hand that's still
> strange, because my patch also contains an updated
> 'common/autoconf/generated-
> configure.sh' and it works on the other platforms (and also locally
> for me on MacOS X) without regenerating it. I just saw that the mail I
> wrote yesterday wasn't send to the lists so I quote my assumptions
> here:
>
> ...
> From your logs I only see that the make process does not find any
> configuration, but the configuration should have been created by
> running configure. Then it tries to call configure but doesn't succeed
> because it says "The generated configure file contains changes not
> present in the custom generated file." That's also strange because my
> changesets also patch 'common/autoconf/generated-
> configure.sh' so it should not be re-generated. And finally the build
> fails because there's not autoconf on Windows and probably because the
> autoconf on Mac is too old (the checked-in file is generated by
> autoconf 2.8).
>
> I've just synced jdk8/jdk8 and applied the two patches. I could build
> without any problems on MacOS X 10.8 with the following command line.
>
> sh /usr/work/d046063/OpenJDK/ppc-aix-port/stage/configure
> --with-boot-jdk=<path_to>/darwinintel64/output/sapjvm_7
> --with-target-bits=64 --with-debug-level=release
>
> make images LOG=debug
>
> Attached you can find the output I get from running configure.
>
> Could you please retry one more time? If you still have problems, can
> you please run configure by hand and post the output.
>
> PS: I also saw from your logs that '/usr/bin/make sanity' is called.
> This also seems strange to me because I think there aren't any sanity
> targets any more in the new build system. Maybe you have a mismatch
> between old and new build system?
> ...
>
> Erik, could you please try this locally on Mac or Windows and post you
> findings. Alternatively, could you please explain post the complete
> JPRT output and explain what JPRT is trying to do. From Vladimir's log
> snippets it's hard for me t understand the whole picture.
>
> Thank you and best regards,
> Volker
>
>> The fix is to execute "bash common/autoconf/autogen.sh" locally and then
>> resubmit to jprt. You need to have autoconf version 2.67 or newer installed
>> for it to work.
>>
>> /Erik
>>
>>
>> On 2013-07-02 05:48, Vladimir Kozlov wrote:
>>>
>>> Need help from JDK build experts.
>>>
>>> I applied next 2 changesets:
>>>
>>> http://cr.openjdk.java.net/~simonis/webrevs/8017568_toplevel/
>>> http://cr.openjdk.java.net/~simonis/webrevs/8017568_jdk/
>>>
>>> and got JPRT build problems (-control build) only on MacOS and Win64:
>>>
>>>
>>> http://bus2001067.us.oracle.com/archives/2013/06/2013-06-28-213927.vkozlov.ppc64_jdk_build_test/
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> macosx_x64_10.7-product (details from log file)
>>>
>>> ...
>>>
>>> Warning: The generated configure file contains changes not present in the
>>> custom generated file.
>>> Running autogen.sh to correct the situation
>>> Autoconf found: /usr/bin/autoconf
>>> Autoconf-2.67 found:
>>> Generating generated-configure.sh with /usr/bin/autoconf
>>> /usr/bin/gm4:stdin:187: bad expression in eval: 32 > <dynamic>
>>> autom4te: /usr/bin/gm4 failed with exit status: 1
>>> Generating custom generated-configure.sh
>>> /usr/bin/gm4:stdin:187: bad expression in eval: 32 > <dynamic>
>>> autom4te: /usr/bin/gm4 failed with exit status: 1
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> windows_x64_5.2-fastdebug (details from log file)
>>>
>>> ...
>>>
>>> Warning: The generated configure file contains changes not present in the
>>> custom generated file.
>>> Error: Cannot continue
>>> Cannot locate autoconf, unable to correct situation.
>>> Please install autoconf and run 'bash autogen.sh' to update the generated
>>> files.
>>> make: *** [bridge2configure] Error 1
>>>
>>>
>>> ------------------------------------------------------------------------------
>>>
>>> Without these changes the output is:
>>>
>>> Running custom generated-configure.sh
>>> configure: Configuration created at Thu Jun 27 16:32:25 EDT 2013.
>>> configure: configure script generated at timestamp 1371547939.
>>>
>>> Thanks,
>>> Vladimir
>>>
>>> On 6/28/13 12:04 AM, Volker Simonis wrote:
>>>>
>>>> Ok, that's fine.
>>>>
>>>> Could you please let me know when you've verified these changes. I
>>>> will then push them to the staging repository.
>>>>
>>>>
>>>> Regards,
>>>> Volker
>>>>
>>>>
>>>> On Thu, Jun 27, 2013 at 7:35 PM, Vladimir Kozlov
>>>> <vladimir.kozlov at oracle.com> wrote:
>>>>>
>>>>> On 6/27/13 10:16 AM, Iris Clark wrote:
>>>>>>
>>>>>>
>>>>>> Hi, Volker.
>>>>>>
>>>>>> I think that the right thing for this change [1] is for you to push
>>>>>> into
>>>>>> ppc-aix-port/stage once you get the necessary reviews (presumably Erik
>>>>>> and
>>>>>> possibly Alan). While your changeset contains some general purpose
>>>>>> updates,
>>>>>> it also contains PPC/AIX-specific files which can't be added to a JDK
>>>>>> release repository until stage is pushed into the a JDK release.
>>>>>>
>>>>>> The recommendation to push to stage of course assumes that Vladimir
>>>>>> doesn't think that this will adversely affect the Hotspot work already
>>>>>> being
>>>>>> pushed to stage.
>>>>>
>>>>>
>>>>>
>>>>> This should not affect Hotspot in stage repo. Me or Albert will do JPRT
>>>>> bootstrap control build of jdk with this changes to make sure it works.
>>>>> After that Volker can push it into stage.
>>>>>
>>>>> When I talked about pushing *general* changes into main sources I meant
>>>>> changes with no ppc64 specific code. The example of such changes was
>>>>> recent
>>>>> Goetz's fix for '8017531: 8010460 changes broke
>>>>> bytecodeInterpreter.cpp'.
>>>>>
>>>>> Thanks,
>>>>> Vladimir
>>>>>
>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> iris
>>>>>>
>>>>>> [1]: http://cr.openjdk.java.net/~simonis/webrevs/8017568_jdk/
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Volker Simonis [mailto:volker.simonis at gmail.com]
>>>>>> Sent: Thursday, June 27, 2013 9:23 AM
>>>>>> To: Erik Joelsson
>>>>>> Cc: Kumar Srinivasan; build-dev; ppc-aix-port-dev at openjdk.java.net;
>>>>>> Alan
>>>>>> Bateman
>>>>>> Subject: Re: RFR (XS): Enable new build on Linux/PPC64 (jdk part)
>>>>>>
>>>>>> Hi Erik,
>>>>>>
>>>>>> we have no polices which are carved in stone:) It's all done informally
>>>>>> and by common sense.
>>>>>>
>>>>>> The main reason for the ppc-aix-port/stage repository is to have a
>>>>>> sandbox
>>>>>> for in-depth review and testing of changes we had to make in shared
>>>>>> code
>>>>>> before pushing them to the main repository (and this especially applies
>>>>>> to
>>>>>> hotspot changes). If you feel comfortable with the current changes and
>>>>>> don't
>>>>>> think that they will break anything (e.g. by running tests build on
>>>>>> your
>>>>>> supported platforms including the closed source ones) I'd really
>>>>>> appreciate
>>>>>> if you could push them to the build repository.
>>>>>>
>>>>>> Otherwise I'll push them to the staging repository and you'll get them
>>>>>> once we're finished with the integration of the port.
>>>>>>
>>>>>> Thank you and best regards,
>>>>>> Volker
>>>>>>
>>>>>> On Thu, Jun 27, 2013 at 1:51 PM, Erik Joelsson
>>>>>> <erik.joelsson at oracle.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 2013-06-27 13:00, Volker Simonis wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jun 27, 2013 at 12:16 PM, Erik Joelsson
>>>>>>> <erik.joelsson at oracle.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Hello Volker,
>>>>>>>>
>>>>>>>> I wasn't aware of this project until now. From what I (now)
>>>>>>>> understand, generic patches can go into jdk8 repos, but port specific
>>>>>>>> things need to go to staging and go in with the rest later. These
>>>>>>>> changes contain a couple of port specific things so as it looks now
>>>>>>>> they would need to go through staging.
>>>>>>>>
>>>>>>>
>>>>>>> Yes, that's the general approach. But I'd argue that for the most part
>>>>>>> this changes are generic (enable CPP-interpreter, enable CORE build,
>>>>>>> fix for broken ld on SuSE, replacing OPENWIN_LIB by X_LIB, fix typos)
>>>>>>> with only very few PPC64 specific parts (map-files and a few defines).
>>>>>>> The problem we want to avoid is that some of our fixes go into the
>>>>>>> main repositories in parallel which would result in merging pain when
>>>>>>> integrating the staging into the main repository.
>>>>>>>
>>>>>>> So if you think you don't need any of the general fixes any time soon,
>>>>>>> I'll push the changes into the staging repository. Otherwise I think
>>>>>>> it would be better to push them right into the main repositories.
>>>>>>>
>>>>>>> Several of the general fixes in there are good and I don't want to
>>>>>>> hinder those getting in. I also don't want to break policies I'm not
>>>>>>> familiar with.
>>>>>>>
>>>>>>> /Erik
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Volker
>>>>>>>
>>>>>>>>
>>>>>>>> /Erik
>>>>>>>>
>>>>>>>>
>>>>>>>> On 2013-06-27 12:03, Volker Simonis wrote:
>>>>>>>>
>>>>>>>> Hi Erik,
>>>>>>>>
>>>>>>>> as Vladimir explained, we have a special staging repository for our
>>>>>>>> PPC64/AIX port:
>>>>>>>>
>>>>>>>> I would be happy if you could push the changes right into the
>>>>>>>> build-infra repositories and we will then get them into our staging
>>>>>>>> repository via jdk8/jdk8.
>>>>>>>>
>>>>>>>> Otherwise I'll have to push them to our staging repository and later
>>>>>>>> when the whole port is completed in the staging repo they will be
>>>>>>>> bulk-integrated into jdk8.
>>>>>>>>
>>>>>>>> What do you think?
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Volker
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Jun 26, 2013 at 6:53 PM, Vladimir Kozlov
>>>>>>>> <vladimir.kozlov at oracle.com> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Erik,
>>>>>>>>>
>>>>>>>>> We have special staging forest for PPC64/AIX port:
>>>>>>>>>
>>>>>>>>> http://hg.openjdk.java.net/ppc-aix-port/stage
>>>>>>>>>
>>>>>>>>> We collect Hotspot and JDK changes there for testing before pushing
>>>>>>>>> into main sources in a future. But some general fixes we push
>>>>>>>>> directly into our main sources.
>>>>>>>>> If you think these build changes are acceptable for main, please,
>>>>>>>>> ask someone sponsor these changes.
>>>>>>>>>
>>>>>>>>> Alan is our official contact for PPC64/AIX project. I CC him.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Vladimir
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 6/26/13 8:56 AM, Erik Joelsson wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Hello,
>>>>>>>>>>
>>>>>>>>>> If you by staging area mean the build-infra forest, that's more or
>>>>>>>>>> less dead now.
>>>>>>>>>>
>>>>>>>>>> I think these changes look good now (both top level and jdk). It
>>>>>>>>>> should be fine to push this to jdk8/build.
>>>>>>>>>>
>>>>>>>>>> /Erik
>>>>>>>>>>
>>>>>>>>>> On 2013-06-26 17:28, Volker Simonis wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Hi Erik,
>>>>>>>>>>>
>>>>>>>>>>> thank you for looking at this. I've prepared a new webrev at:
>>>>>>>>>>>
>>>>>>>>>>> http://cr.openjdk.java.net/~simonis/webrevs/8017568_jdk/
>>>>>>>>>>> <http://cr.openjdk.java.net/%7Esimonis/webrevs/8017568_jdk/>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> What do you think, do you want to push this directly into the
>>>>>>>>>>> build repositories or should I push it into the staging repository
>>>>>>>>>>> first?
>>>>>>>>>>>
>>>>>>>>>>> Please see my further comments inline.
>>>>>>>>>>>
>>>>>>>>>>> Thank you and best regards,
>>>>>>>>>>> Volker
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Jun 25, 2013 at 12:41 PM, Erik Joelsson
>>>>>>>>>>> <erik.joelsson at oracle.com <mailto:erik.joelsson at oracle.com>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 2013-06-25 12:27, Erik Joelsson wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hello Volker,
>>>>>>>>>>>
>>>>>>>>>>> On 2013-06-24 19:23, Volker Simonis wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> could somebody please review the following change
>>>>>>>>>>> and
>>>>>>>>>>> create an appropriate
>>>>>>>>>>> Bug ID for it:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> http://cr.openjdk.java.net/~simonis/webrevs/linux_ppc_build_jdk/
>>>>>>>>>>>
>>>>>>>>>>> <http://cr.openjdk.java.net/%7Esimonis/webrevs/linux_ppc_build_jdk
>>>>>>>>>>> />
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> The patch contains two little changes which are
>>>>>>>>>>> required
>>>>>>>>>>> to build the class
>>>>>>>>>>> library part of the OpenJDK on Linux/PPC64. Most of
>>>>>>>>>>> the
>>>>>>>>>>> build magic is
>>>>>>>>>>> contained in the top-level part of this change which
>>>>>>>>>>> is
>>>>>>>>>>> separately reviewed
>>>>>>>>>>> at
>>>>>>>>>>>
>>>>>>>>>>> http://cr.openjdk.java.net/~simonis/webrevs/linux_ppc_build_top
>>>>>>>>>>>
>>>>>>>>>>> <http://cr.openjdk.java.net/%7Esimonis/webrevs/linux_ppc_build_top
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> CompileLaunchers.gmk
>>>>>>>>>>>
>>>>>>>>>>> Remove mapfile from build instructions of
>>>>>>>>>>> BUILD_UNPACKEXE:
>>>>>>>>>>>
>>>>>>>>>>> CFLAGS_macosx:=-fPIC, \
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> MAPFILE:=$(JDK_TOPDIR)/makefiles/mapfiles/libunpack/mapfile-vers-unpack200,\
>>>>>>>>>>> LDFLAGS:=$(UNPACKEXE_ZIPOBJS),\
>>>>>>>>>>>
>>>>>>>>>>> I think it makes no sense to use a version script
>>>>>>>>>>> file
>>>>>>>>>>> for an executable
>>>>>>>>>>> and older linkers (e.g. on SLES 10) complain with:
>>>>>>>>>>> "*Invalid version tag
>>>>>>>>>>> `SUNWprivate_1.1'. Only anonymous version tag is
>>>>>>>>>>> allowed
>>>>>>>>>>> in executable.*"
>>>>>>>>>>> The GNU ld
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> manual<http://ftp.gnu.org/old-gnu/Manuals/ld-2.9.1/html_node/ld_25.html>states:
>>>>>>>>>>> "*Version scripts are only meaningful when creating
>>>>>>>>>>> shared
>>>>>>>>>>> libraries.*"
>>>>>>>>>>> Morover unpack200 was the only executable which used
>>>>>>>>>>> a
>>>>>>>>>>> version script file.
>>>>>>>>>>>
>>>>>>>>>>> Unpackexe has some weirdness and this isn't surprising
>>>>>>>>>>> me.
>>>>>>>>>>> Would be good if someone with more historic knowledge
>>>>>>>>>>> could
>>>>>>>>>>> fill in on the reason for this. Someone apparently went
>>>>>>>>>>> through the trouble of creating a special mapfile for
>>>>>>>>>>> this
>>>>>>>>>>> executable. Also, if not using it, should it be removed?
>>>>>>>>>>>
>>>>>>>>>>> I looked closer at this. These mapfiles were explicitly
>>>>>>>>>>> added in
>>>>>>>>>>> http://bugs.sun.com/view_bug.do?bug_id=7033954, but it was
>>>>>>>>>>> noted
>>>>>>>>>>> that it broke builds for architectures that didn't have
>>>>>>>>>>> mapfiles
>>>>>>>>>>> defined. If you look at the launchers, the mapfile is only
>>>>>>>>>>> set
>>>>>>>>>>> if
>>>>>>>>>>> the arch specific one exists. I think a safer change here
>>>>>>>>>>> would
>>>>>>>>>>> be
>>>>>>>>>>> to make the mapfile conditional on platform or arch for
>>>>>>>>>>> unpackexe.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I still do not fully understand why we need map-files for
>>>>>>>>>>> executables, but I also understand that you don't want to change
>>>>>>>>>>> the
>>>>>>>>>>> current setup.
>>>>>>>>>>> So I went the hard (and hopefully right:) way and implemented a
>>>>>>>>>>> detection of the buggy linkers on older SuSE distros (e.g. on SLES
>>>>>>>>>>> 10) which complain with: "Invalid version tag `SUNWprivate_1.1'
>>>>>>>>>>> during the configure step (see top-level change). Unfortunately we
>>>>>>>>>>> still have quite a lot of these systems so we really need the
>>>>>>>>>>> build with that buggy ld.
>>>>>>>>>>>
>>>>>>>>>>> I've therefore added map files with anonymous version tags for
>>>>>>>>>>> these buggy linkers which are only used if the buggy linker was
>>>>>>>>>>> detected during the configure step (i.e. if
>>>>>>>>>>> USING_BROKEN_SUSE_LD=yes). Notice that this is no PPC64 specific
>>>>>>>>>>> problem but a occurs on all SuSE 10 platforms.
>>>>>>>>>>>
>>>>>>>>>>> And you've been right. I also had to add the arch specific map
>>>>>>>>>>> files for ppc64 in order to use them for the other launchers.
>>>>>>>>>>>
>>>>>>>>>>> Kumar, you made the change referred to here, do you have
>>>>>>>>>>> anything
>>>>>>>>>>> to add?
>>>>>>>>>>>
>>>>>>>>>>> /Erik
>>>>>>>>>>>
>>>>>>>>>>> Fix typo (replace 'defalt: all' by 'default')
>>>>>>>>>>>
>>>>>>>>>>> default: all
>>>>>>>>>>>
>>>>>>>>>>> CompileNativeLibraries.gmk
>>>>>>>>>>>
>>>>>>>>>>> Only use $(OPENWIN_LIB) for linking LIBSPLASHSCREEN
>>>>>>>>>>> on
>>>>>>>>>>> Solaris! The old
>>>>>>>>>>> code worked only accidentally when the X-libraries
>>>>>>>>>>> are
>>>>>>>>>>> in
>>>>>>>>>>> the default
>>>>>>>>>>> linker path anyway. The right solution is to use
>>>>>>>>>>> $(X_LIBS)
>>>>>>>>>>> if not on
>>>>>>>>>>> Windows or Solaris.
>>>>>>>>>>>
>>>>>>>>>>> Append -DX_ARCH=X_PPC64 to LIBJSOUND_CFLAGS on
>>>>>>>>>>> PPC64.
>>>>>>>>>>> The value of
>>>>>>>>>>> X_ARCHisn't actually used on the PPC architectures,
>>>>>>>>>>> but
>>>>>>>>>>> there's a
>>>>>>>>>>> check to verify
>>>>>>>>>>> that it is set.
>>>>>>>>>>>
>>>>>>>>>>> Fix typo (replace 'defalt: all' by 'default')
>>>>>>>>>>>
>>>>>>>>>>> default: all
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Otherwise looks good.
>>>>>>>>>>>
>>>>>>>>>>> /Erik
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>
>
More information about the build-dev
mailing list