New builds from the build-infra team

Vincent Ryan vincent.x.ryan at oracle.com
Tue Nov 6 09:34:19 PST 2012 

I picked up the 2 changesets that were pushed over the weekend. The build makes more
progress but it still fails due to missing C lib. I guess there are a few more corner cases.

Is there a way to activate more debugging during the build? I'd like to examine the exact
compiler flags that are in use when the failure occurs.



On 6 Nov 2012, at 16:33, Kelly O'Hair wrote:

> If you get a chance, could you try this experiment again with the latest in jdk8/build forest?
> I fixed a few more libraries that I think had a missing -lc.
> 
> I don't have access to a Solaris SPARC 11.1 system, and so far the issues you have run into seem to
> be unique to SPARCV9 builds on 11.1, the Solaris X64 11.1 systems I have access to don't seem to reproduce this issue.
> I can only assume that the compiler or the system behaves differently between SPARCV9 and X64 on 11.1. :^(
> 
> -kto
> 
> On Nov 2, 2012, at 1:08 PM, Vincent Ryan wrote:
> 
>> Thanks Kelly.
>> 
>> I re-ran the build with your patch and got better results but there
>> seems to be another linker problem later in the jdk build.
>> 
>> Here's the tail of the build output:
>> 
>> :
>> :
>> Undefined                       first referenced
>> symbol                             in file
>> free /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libjava_crw_demo/java_crw_demo.o
>> __iob /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libjava_crw_demo/java_crw_demo.o
>> abort /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libjava_crw_demo/java_crw_demo.o
>> snprintf /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libjava_crw_demo/java_crw_demo.o
>> calloc /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libjava_crw_demo/java_crw_demo.o
>> malloc /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libjava_crw_demo/java_crw_demo.o
>> memcpy /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libjava_crw_demo/java_crw_demo.o
>> memset /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libjava_crw_demo/java_crw_demo.o
>> strcmp /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libjava_crw_demo/java_crw_demo.o
>> strlen /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libjava_crw_demo/java_crw_demo.o
>> realloc /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libjava_crw_demo/java_crw_demo.o
>> strncmp /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libjava_crw_demo/java_crw_demo.o
>> fprintf /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libjava_crw_demo/java_crw_demo.o
>> ld: fatal: symbol referencing errors. No output written to /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/lib/sparcv9/libjava_crw_demo.so
>> gmake[3]: *** [/export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/lib/sparcv9/libjava_crw_demo.so] Error 1
>> gmake[3]: *** Waiting for unfinished jobs....
>> Undefined                       first referenced
>> symbol                             in file
>> exit /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libnpt/npt.o
>> free /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libnpt/npt.o
>> __iob /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libnpt/npt.o
>> abort /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libnpt/utf.o
>> iconv /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libnpt/utf_md.o
>> __ctype /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libnpt/utf.o
>> iconv_open /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libnpt/utf_md.o
>> calloc /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libnpt/npt.o
>> memcpy /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libnpt/utf_md.o
>> strcmp /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libnpt/npt.o
>> strdup /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libnpt/npt.o
>> strlen /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libnpt/utf.o
>> nl_langinfo /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libnpt/utf_md.o
>> sprintf /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libnpt/utf.o
>> setlocale /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libnpt/utf_md.o
>> iconv_close /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libnpt/utf_md.o
>> fprintf /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libnpt/npt.o
>> ld: fatal: symbol referencing errors. No output written to /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/lib/sparcv9/libnpt.so
>> gmake[3]: *** [/export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/lib/sparcv9/libnpt.so] Error 1
>> gmake[3]: Leaving directory `/export/home/vinryan/jdk8-master/jdk/makefiles'
>> gmake[2]: *** [libs-only] Error 2
>> gmake[2]: Leaving directory `/export/home/vinryan/jdk8-master/jdk/makefiles'
>> make[1]: *** [jdk-only] Error 2
>> make[1]: Leaving directory `/export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release'
>> make: *** [all] Error 2
>> t4%
>> 
>> 
>> 
>> 
>> 
>> On 02/11/2012 18:38, Kelly O'Hair wrote:
>>> 
>>> On Nov 2, 2012, at 11:17 AM, Vincent Ryan wrote:
>>> 
>>>> That attachment got dropped when I re-sent my message (previously I wasn't a member of the build-infra-dev email alias). I've attached it again in case it proves useful.
>>>> 
>>>> <config.status>
>>>> 
>>>> 
>>>> I realize Solaris 11 is not an official build platform for JDK8 but I thought I'd give it a spin on a reasonably
>>>> beefy server.
>>> 
>>> It's great that you tried this, thanks.
>>> 
>>> As far as I can tell, this seems to be a sparcv9 Solaris 11.1 issue, because I tried Solaris 11.1 X64 and it worked fine.
>>> However, the link of libhprof.so IS missing the -lc option.
>>> So why sparcv9 would complain and amd64 would not is a bit puzzling.
>>> 
>>> The change I think needs to be made is:
>>> 
>>> diff --git a/makefiles/CompileNativeLibraries.gmk b/makefiles/CompileNativeLibraries.gmk
>>> --- a/makefiles/CompileNativeLibraries.gmk
>>> +++ b/makefiles/CompileNativeLibraries.gmk
>>> @@ -1807,7 +1807,7 @@
>>> BUILD_LIBHPROF_LDFLAGS:=
>>> 
>>> ifeq ($(OPENJDK_TARGET_OS),solaris)
>>> -     BUILD_LIBHPROF_LDFLAGS += -lsocket -lnsl
>>> +     BUILD_LIBHPROF_LDFLAGS += -lsocket -lnsl -lc
>>> endif
>>> 
>>> LIBHPROF_OPTIMIZATION:=HIGHEST
>>> 
>>> 
>>> 
>>> But I have not tested it yet.
>>> Did you want to give this patch a spin?
>>> 
>>> -kto
>>> 
>>>> Thanks.
>>>> 
>>>> 
>>>> On 2 Nov 2012, at 17:45, Kelly O'Hair wrote:
>>>> 
>>>>> The attachment is missing.
>>>>> 
>>>>> This looks like the  -lc   is missing from the creation of hprof, we should never let libc be an implicit dependency.
>>>>> 
>>>>> I wonder if Solaris 11.1 has tightened up the rules on implicit dependencies. We have not seen this on Solaris 10.
>>>>> We just got svc6 setup as a 11.1 system, I'll see if it reproduces there.
>>>>> 
>>>>> And thank you for reporting this. Although Solaris 11.1 is not one of our critical systems, it's very important that this works.
>>>>> 
>>>>> -kto
>>>>> 
>>>>> On Nov 2, 2012, at 9:38 AM, Vincent Ryan wrote:
>>>>> 
>>>>>>> From: Vincent Ryan <vincent.x.ryan at oracle.com>
>>>>>>> Subject: Re: New builds from the build-infra team
>>>>>>> Date: 2 November 2012 15:48:03 GMT
>>>>>>> To: build-infra-dev at openjdk.java.net
>>>>>>> 
>>>>>>> I tried this out on the latest Solaris 11 Update 1 (sparc) but the build encountered problems locating libc when
>>>>>>> building jdk. (BTW the old build works correctly, just slower)
>>>>>>> 
>>>>>>> :
>>>>>>> :
>>>>>>> strerror                            /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libhprof_jvmti/hprof_md.o  (symbol belongs to implicit dependency /lib/sparcv9/libc.so.1)
>>>>>>> vfprintf                            /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libhprof_jvmti/hprof_error.o  (symbol belongs to implicit dependency /lib/sparcv9/libc.so.1)
>>>>>>> fprintf                             /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libhprof_jvmti/hprof_init.o  (symbol belongs to implicit dependency /lib/sparcv9/libc.so.1)
>>>>>>> gethrvtime                          /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/objs/libhprof_jvmti/hprof_md.o  (symbol belongs to implicit dependency /lib/sparcv9/libc.so.1)
>>>>>>> ld: fatal: symbol referencing errors. No output written to /export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/lib/sparcv9/libhprof.so
>>>>>>> gmake[3]: *** [/export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release/jdk/lib/sparcv9/libhprof.so] Error 1
>>>>>>> gmake[3]: Leaving directory `/export/home/vinryan/jdk8-master/jdk/makefiles'
>>>>>>> gmake[2]: *** [libs-only] Error 2
>>>>>>> gmake[2]: Leaving directory `/export/home/vinryan/jdk8-master/jdk/makefiles'
>>>>>>> make[1]: *** [jdk-only] Error 2
>>>>>>> make[1]: Leaving directory `/export/home/vinryan/jdk8-master/build/solaris-sparcv9-normal-server-release'
>>>>>>> make: *** [all] Error 2
>>>>>>> t4%
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> FYI I've attached the config script that was generated by configure.sh.
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> On 1 Nov 2012, at 18:38, Kelly O'Hair wrote:
>>>>>>> 
>>>>>>>> 
>>>>>>>> Pardon the wide email, but this impacts everyone building the OpenJDK jdk8/jdk8 derived forests.
>>>>>>>> 
>>>>>>>> Please only reply to the build-infra-dev mailing list, or just me.
>>>>>>>> 
>>>>>>>> With some recent integrations from the build-infra project into jdk8/jdk8 repositories, the build-infra team
>>>>>>>> would like to get more exposure of the new builds. These jdk8/jdk8 changes will start showing up in various
>>>>>>>> jdk8 and team forests over the next few weeks. The default is still the old builds, but both builds work in most
>>>>>>>> cases for OpenJDK as far as we know.
>>>>>>>> 
>>>>>>>> At a very high level, the intent is that once you get a forest:
>>>>>>>> hg clone http://hg.openjdk.java.net/jdk8/jdk8    j8
>>>>>>>> cd j8
>>>>>>>> sh ./get_source.sh
>>>>>>>> 
>>>>>>>> You should be able to simply configure&&make (the ultimate goal is this simple anyway), e.g.
>>>>>>>> sh ./configure
>>>>>>>> make NEWBUILD=true     # The NEWBUILD=true will become the default when we formally switch.
>>>>>>>> 
>>>>>>>> Where "make" is GNU make 3.81, and your system has all the requires packages and PATH contains the
>>>>>>>> needed tools. Note that on Windows, MKS unix utilities cannot be used with the new builds, just CYGWIN
>>>>>>>> is recommended at this time.
>>>>>>>> 
>>>>>>>> Of course, we know, it's never as easy as a simple configure&&make, and often you will need to pass in
>>>>>>>> configure options.
>>>>>>>> 
>>>>>>>> What we would like to know is where a simple configure&&make does not work, and anything people had
>>>>>>>> to do to make it work.
>>>>>>>> 
>>>>>>>> I know many of you are quite used to the old builds, so I have a temporary  "bridgeBuild" target
>>>>>>>> people can try that will attempt to map the ALT_* environment variables to an appropriate configure command
>>>>>>>> and then run that configure command and do the build, e.g.
>>>>>>>> 
>>>>>>>> make NEWBUILD=true bridgeBuild
>>>>>>>> 
>>>>>>>> People willing to do comparisons between the old and new builds could:
>>>>>>>> rm -f -r build
>>>>>>>> time make NEWBUILD=true bridgeBuild
>>>>>>>> rm -f -r build
>>>>>>>> time make NO_DOCS=true     # Old builds do not generate javadocs by default
>>>>>>>> 
>>>>>>>> Any observations about speed of the builds would be appreciated, as will any impressions on what you see.
>>>>>>>> 
>>>>>>>> At this time, we think this is working pretty well with a few caveats:
>>>>>>>> * GNU make with the new builds is doing much more parallel processing and this can stress out a system
>>>>>>>> - Use "make JOBS=1" if you suspect a problem, then try adjusting it up slowly.
>>>>>>>> * Partial builds are limited, right now full builds of the entire OpenJDK is the target
>>>>>>>> - Hotspot can still be built on it's own, but everyone else needs to build hotspot at least once
>>>>>>>> * Paths with multiple names can cause problems, e.g. being on system svc6, and access an exported share
>>>>>>>> area as /net/svc6/export/foobar  instead of /export/foobar  will cause problems. Use local paths.
>>>>>>>> 
>>>>>>>> We know there are still issues and we will be focusing heavily on the critical ones in the next few weeks, but
>>>>>>>> we do need the community to tell us what the critical issues really are.
>>>>>>>> 
>>>>>>>> Our number one priority at this time is that everyone that was able to build the old way, should be able to build
>>>>>>>> with the new build-infra makefiles. Please help us verify that.
>>>>>>>> 
>>>>>>>> -kto
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>> 
>> 
>

More information about the build-infra-dev mailing list