New builds from the build-infra team

Fri Nov 9 06:15:58 PST 2012

On 11/8/2012 12:30 AM, Kelly O'Hair wrote:
> On Nov 7, 2012, at 7:25 AM, Anthony Petrov wrote:
> 
>> A follow up message. Generally, the new builds work fine. A couple suggestions:
>>
>> 1. Can we make hotspot builds less noisy? Unless there are warnings or errors, there's no need to print out the name of each file being compiled.
> 
> It's hard to know before hand if a file has errors or warnings but the hotspot makefiles are actually
> unchanged for the most part, so you are seeing the old and traditional hotspot build output.
> 
> There is a great deal of debate on this noisy vs. quiet logging issue, and Magnus at least tried to deal with
> it by adding a LOG option, so you can try:
>    make LOG=warn
> if I remember right. But it may be a while before we can get the hotspot makefiles to obey this setting.

I see. I just seen how much less noise the JDK build produces now (and 
this is cool, IMO), and been under impression that HotSpot makefiles 
differ a little in this respect. It appears they differ a lot... yet. 
All right, will wait then till HotSpot folks fix them.

>> 2. There's a lot of warnings generated when compiling JDK native code. I think some of them can be related to the new build. After they are analyzed, can somebody from build-infra file bugs against appropriate teams to fix them up?
> 
> I don't think the new build is adding warnings, but I can't be sure. These should be the same warnings as the old
> build, but may be more visible when the compile lines are not echo'd.

Exactly!

> The last time I filed a whole bunch of bugs against warnings in the build, it was a royal pain, like herding cats
> to get all the various teams to do anything about it. But at some point we will do another warning hunt fix,
> just not sure the build team will be driving it or not.

Well, I don't think we should actively drive this effort. But at least 
just filing the bugs would be a very good thing to do. As we slowly 
switching to the new build, I think we should just tell people to file 
bugs against warnings whenever they see them. I guess that after closing 
dozens of duplicates, the responsible teams will finally fix them.

And, btw, -Werror would make the issues P1. Just saying. :))

--
best regards,
Anthony

> 
> -kto
> 
>> --
>> best regards,
>> Anthony
>>
>> On 11/7/2012 7:14 PM, Anthony Petrov wrote:
>>> Update on issue #3: after cloning the closed repos my build succeeded. Must be an issue related to OPENJDK=true builds only. Still, I'd like to get some comments regarding #1 and #2 below. Thanks in advance.
>>> -- 
>>> best regards,
>>> Anthony
>>> On 11/7/2012 6:16 PM, Anthony Petrov wrote:
>>>> Hi All,
>>>>
>>>> (please keep me in CC since I'm not subscribed to this list)
>>>>
>>>> (I'm building on Linux i586)
>>>>
>>>> 1. Could we add the /java/re/jdk/<version>/promoted/latest/binaries/<platform>/ directories to check for a Boot JDK to ./configure? E.g. /java/re/jdk/7u10/promoted/latest/binaries/linux-i586. I realize that this is almost useless outside of Oracle, but it would be really helpful for internal builds when one has already set up the /java directory structure properly.
>>>>
>>>> 2. What is the ./build/linux-x86-normal-server-release directory, and why is it not ./build/linux-i586 ? What does 'normal' stand for? The same question about the 'server' part.
>>>>
>>>> 3. I get the following error and the build stops:
>>>>
>>>>> /export/anthony/8-50-full-new-build/build/linux-x86-normal-server-release/jdk/objs/liblcms/cmspack.o: In function `UnrollHalfToFloat':
>>>>> cmspack.c:(.text+0x1f25): undefined reference to `_cmsHalf2Float'
>>>>> cmspack.c:(.text+0x1f73): undefined reference to `_cmsHalf2Float'
>>>>> /export/anthony/8-50-full-new-build/build/linux-x86-normal-server-release/jdk/objs/liblcms/cmspack.o: In function `UnrollHalfTo16':
>>>>> cmspack.c:(.text+0x261d): undefined reference to `_cmsHalf2Float'
>>>>> cmspack.c:(.text+0x26bd): undefined reference to `_cmsHalf2Float'
>>>>> /export/anthony/8-50-full-new-build/build/linux-x86-normal-server-release/jdk/objs/liblcms/cmspack.o: In function `PackHalfFromFloat':
>>>>> cmspack.c:(.text+0x36bb): undefined reference to `_cmsFloat2Half'
>>>>> cmspack.c:(.text+0x3710): undefined reference to `_cmsFloat2Half'
>>>>> cmspack.c:(.text+0x37af): undefined reference to `_cmsFloat2Half'
>>>>> /export/anthony/8-50-full-new-build/build/linux-x86-normal-server-release/jdk/objs/liblcms/cmspack.o: In function `PackHalfFrom16':
>>>>> cmspack.c:(.text+0x38a8): undefined reference to `_cmsFloat2Half'
>>>>> cmspack.c:(.text+0x3906): undefined reference to `_cmsFloat2Half'
>>>>> /export/anthony/8-50-full-new-build/build/linux-x86-normal-server-release/jdk/objs/liblcms/cmspack.o:cmspack.c:(.text+0x39a7): more undefined references to `_cmsFloat2Half' follow
>>>>> collect2: ld returned 1 exit status
>>>>> gmake[3]: *** [/export/anthony/8-50-full-new-build/build/linux-x86-normal-server-release/jdk/lib/i386/liblcms.so] Error 1
>>>>> gmake[3]: *** Waiting for unfinished jobs....
>>>>> gmake[3]: Leaving directory `/export/anthony/8-50-full-new-build/jdk/makefiles'
>>>>> gmake[2]: *** [libs-only] Error 2
>>>>> gmake[2]: Leaving directory `/export/anthony/8-50-full-new-build/jdk/makefiles'
>>>>> make[1]: *** [jdk-only] Error 2
>>>>> make[1]: Leaving directory `/export/anthony/8-50-full-new-build/build/linux-x86-normal-server-release' 
>>>>> make: *** [all] Error 2
>>>> Is this a known issue with lcms? Note that I've almost exclusively always built JDK w/ closed repos, and this time I'm building OpenJDK repos only. I'll try to clone the closed parts and see if this eliminates the issue #3.
>>>>
>>>> -- 
>>>> best regards,
>>>> Anthony
>>>>
>>>> On 11/1/2012 10:38 PM, Kelly O'Hair wrote:
>>>>> Pardon the wide email, but this impacts everyone building the OpenJDK jdk8/jdk8 derived forests.
>>>>>
>>>>> Please only reply to the build-infra-dev mailing list, or just me.
>>>>>
>>>>> With some recent integrations from the build-infra project into jdk8/jdk8 repositories, the build-infra team
>>>>> would like to get more exposure of the new builds. These jdk8/jdk8 changes will start showing up in various
>>>>> jdk8 and team forests over the next few weeks. The default is still the old builds, but both builds work in most
>>>>> cases for OpenJDK as far as we know.
>>>>>
>>>>> At a very high level, the intent is that once you get a forest:
>>>>>  hg clone http://hg.openjdk.java.net/jdk8/jdk8    j8
>>>>>  cd j8
>>>>>  sh ./get_source.sh
>>>>>
>>>>> You should be able to simply configure&&make (the ultimate goal is this simple anyway), e.g.
>>>>>   sh ./configure
>>>>>   make NEWBUILD=true     # The NEWBUILD=true will become the default when we formally switch.
>>>>>
>>>>> Where "make" is GNU make 3.81, and your system has all the requires packages and PATH contains the
>>>>> needed tools. Note that on Windows, MKS unix utilities cannot be used with the new builds, just CYGWIN
>>>>> is recommended at this time.
>>>>>
>>>>> Of course, we know, it's never as easy as a simple configure&&make, and often you will need to pass in
>>>>> configure options.
>>>>>
>>>>> What we would like to know is where a simple configure&&make does not work, and anything people had
>>>>> to do to make it work.
>>>>>
>>>>> I know many of you are quite used to the old builds, so I have a temporary  "bridgeBuild" target
>>>>> people can try that will attempt to map the ALT_* environment variables to an appropriate configure command
>>>>> and then run that configure command and do the build, e.g.
>>>>>
>>>>>  make NEWBUILD=true bridgeBuild
>>>>>
>>>>> People willing to do comparisons between the old and new builds could:
>>>>>  rm -f -r build
>>>>>  time make NEWBUILD=true bridgeBuild
>>>>>  rm -f -r build
>>>>>  time make NO_DOCS=true     # Old builds do not generate javadocs by default
>>>>>
>>>>> Any observations about speed of the builds would be appreciated, as will any impressions on what you see.
>>>>>
>>>>> At this time, we think this is working pretty well with a few caveats:
>>>>>  * GNU make with the new builds is doing much more parallel processing and this can stress out a system
>>>>>    - Use "make JOBS=1" if you suspect a problem, then try adjusting it up slowly.
>>>>>  * Partial builds are limited, right now full builds of the entire OpenJDK is the target
>>>>>    - Hotspot can still be built on it's own, but everyone else needs to build hotspot at least once
>>>>>  * Paths with multiple names can cause problems, e.g. being on system svc6, and access an exported share
>>>>>    area as /net/svc6/export/foobar  instead of /export/foobar  will cause problems. Use local paths.
>>>>>
>>>>> We know there are still issues and we will be focusing heavily on the critical ones in the next few weeks, but
>>>>> we do need the community to tell us what the critical issues really are.
>>>>>
>>>>> Our number one priority at this time is that everyone that was able to build the old way, should be able to build
>>>>> with the new build-infra makefiles. Please help us verify that.
>>>>>
>>>>> -kto
>>>>>
> 