New builds from the build-infra team

[Kelly O'Hair]

On Nov 7, 2012, at 7:25 AM, Anthony Petrov wrote:

> A follow up message. Generally, the new builds work fine. A couple suggestions:
> 
> 1. Can we make hotspot builds less noisy? Unless there are warnings or errors, there's no need to print out the name of each file being compiled.

It's hard to know before hand if a file has errors or warnings but the hotspot makefiles are actually
unchanged for the most part, so you are seeing the old and traditional hotspot build output.

There is a great deal of debate on this noisy vs. quiet logging issue, and Magnus at least tried to deal with
it by adding a LOG option, so you can try:
   make LOG=warn
if I remember right. But it may be a while before we can get the hotspot makefiles to obey this setting.

> 
> 2. There's a lot of warnings generated when compiling JDK native code. I think some of them can be related to the new build. After they are analyzed, can somebody from build-infra file bugs against appropriate teams to fix them up?

I don't think the new build is adding warnings, but I can't be sure. These should be the same warnings as the old
build, but may be more visible when the compile lines are not echo'd.

The last time I filed a whole bunch of bugs against warnings in the build, it was a royal pain, like herding cats
to get all the various teams to do anything about it. But at some point we will do another warning hunt fix,
just not sure the build team will be driving it or not.

-kto

> 
> --
> best regards,
> Anthony
> 
> On 11/7/2012 7:14 PM, Anthony Petrov wrote:
>> Update on issue #3: after cloning the closed repos my build succeeded. Must be an issue related to OPENJDK=true builds only. Still, I'd like to get some comments regarding #1 and #2 below. Thanks in advance.
>> -- 
>> best regards,
>> Anthony
>> On 11/7/2012 6:16 PM, Anthony Petrov wrote:
>>> Hi All,
>>> 
>>> (please keep me in CC since I'm not subscribed to this list)
>>> 
>>> (I'm building on Linux i586)
>>> 
>>> 1. Could we add the /java/re/jdk/<version>/promoted/latest/binaries/<platform>/ directories to check for a Boot JDK to ./configure? E.g. /java/re/jdk/7u10/promoted/latest/binaries/linux-i586. I realize that this is almost useless outside of Oracle, but it would be really helpful for internal builds when one has already set up the /java directory structure properly.
>>> 
>>> 2. What is the ./build/linux-x86-normal-server-release directory, and why is it not ./build/linux-i586 ? What does 'normal' stand for? The same question about the 'server' part.
>>> 
>>> 3. I get the following error and the build stops:
>>> 
>>>> /export/anthony/8-50-full-new-build/build/linux-x86-normal-server-release/jdk/objs/liblcms/cmspack.o: In function `UnrollHalfToFloat':
>>>> cmspack.c:(.text+0x1f25): undefined reference to `_cmsHalf2Float'
>>>> cmspack.c:(.text+0x1f73): undefined reference to `_cmsHalf2Float'
>>>> /export/anthony/8-50-full-new-build/build/linux-x86-normal-server-release/jdk/objs/liblcms/cmspack.o: In function `UnrollHalfTo16':
>>>> cmspack.c:(.text+0x261d): undefined reference to `_cmsHalf2Float'
>>>> cmspack.c:(.text+0x26bd): undefined reference to `_cmsHalf2Float'
>>>> /export/anthony/8-50-full-new-build/build/linux-x86-normal-server-release/jdk/objs/liblcms/cmspack.o: In function `PackHalfFromFloat':
>>>> cmspack.c:(.text+0x36bb): undefined reference to `_cmsFloat2Half'
>>>> cmspack.c:(.text+0x3710): undefined reference to `_cmsFloat2Half'
>>>> cmspack.c:(.text+0x37af): undefined reference to `_cmsFloat2Half'
>>>> /export/anthony/8-50-full-new-build/build/linux-x86-normal-server-release/jdk/objs/liblcms/cmspack.o: In function `PackHalfFrom16':
>>>> cmspack.c:(.text+0x38a8): undefined reference to `_cmsFloat2Half'
>>>> cmspack.c:(.text+0x3906): undefined reference to `_cmsFloat2Half'
>>>> /export/anthony/8-50-full-new-build/build/linux-x86-normal-server-release/jdk/objs/liblcms/cmspack.o:cmspack.c:(.text+0x39a7): more undefined references to `_cmsFloat2Half' follow
>>>> collect2: ld returned 1 exit status
>>>> gmake[3]: *** [/export/anthony/8-50-full-new-build/build/linux-x86-normal-server-release/jdk/lib/i386/liblcms.so] Error 1
>>>> gmake[3]: *** Waiting for unfinished jobs....
>>>> gmake[3]: Leaving directory `/export/anthony/8-50-full-new-build/jdk/makefiles'
>>>> gmake[2]: *** [libs-only] Error 2
>>>> gmake[2]: Leaving directory `/export/anthony/8-50-full-new-build/jdk/makefiles'
>>>> make[1]: *** [jdk-only] Error 2
>>>> make[1]: Leaving directory `/export/anthony/8-50-full-new-build/build/linux-x86-normal-server-release' 
>>>> make: *** [all] Error 2
>>> 
>>> Is this a known issue with lcms? Note that I've almost exclusively always built JDK w/ closed repos, and this time I'm building OpenJDK repos only. I'll try to clone the closed parts and see if this eliminates the issue #3.
>>> 
>>> -- 
>>> best regards,
>>> Anthony
>>> 
>>> On 11/1/2012 10:38 PM, Kelly O'Hair wrote:
>>>> Pardon the wide email, but this impacts everyone building the OpenJDK jdk8/jdk8 derived forests.
>>>> 
>>>> Please only reply to the build-infra-dev mailing list, or just me.
>>>> 
>>>> With some recent integrations from the build-infra project into jdk8/jdk8 repositories, the build-infra team
>>>> would like to get more exposure of the new builds. These jdk8/jdk8 changes will start showing up in various
>>>> jdk8 and team forests over the next few weeks. The default is still the old builds, but both builds work in most
>>>> cases for OpenJDK as far as we know.
>>>> 
>>>> At a very high level, the intent is that once you get a forest:
>>>>  hg clone http://hg.openjdk.java.net/jdk8/jdk8    j8
>>>>  cd j8
>>>>  sh ./get_source.sh
>>>> 
>>>> You should be able to simply configure&&make (the ultimate goal is this simple anyway), e.g.
>>>>   sh ./configure
>>>>   make NEWBUILD=true     # The NEWBUILD=true will become the default when we formally switch.
>>>> 
>>>> Where "make" is GNU make 3.81, and your system has all the requires packages and PATH contains the
>>>> needed tools. Note that on Windows, MKS unix utilities cannot be used with the new builds, just CYGWIN
>>>> is recommended at this time.
>>>> 
>>>> Of course, we know, it's never as easy as a simple configure&&make, and often you will need to pass in
>>>> configure options.
>>>> 
>>>> What we would like to know is where a simple configure&&make does not work, and anything people had
>>>> to do to make it work.
>>>> 
>>>> I know many of you are quite used to the old builds, so I have a temporary  "bridgeBuild" target
>>>> people can try that will attempt to map the ALT_* environment variables to an appropriate configure command
>>>> and then run that configure command and do the build, e.g.
>>>> 
>>>>  make NEWBUILD=true bridgeBuild
>>>> 
>>>> People willing to do comparisons between the old and new builds could:
>>>>  rm -f -r build
>>>>  time make NEWBUILD=true bridgeBuild
>>>>  rm -f -r build
>>>>  time make NO_DOCS=true     # Old builds do not generate javadocs by default
>>>> 
>>>> Any observations about speed of the builds would be appreciated, as will any impressions on what you see.
>>>> 
>>>> At this time, we think this is working pretty well with a few caveats:
>>>>  * GNU make with the new builds is doing much more parallel processing and this can stress out a system
>>>>    - Use "make JOBS=1" if you suspect a problem, then try adjusting it up slowly.
>>>>  * Partial builds are limited, right now full builds of the entire OpenJDK is the target
>>>>    - Hotspot can still be built on it's own, but everyone else needs to build hotspot at least once
>>>>  * Paths with multiple names can cause problems, e.g. being on system svc6, and access an exported share
>>>>    area as /net/svc6/export/foobar  instead of /export/foobar  will cause problems. Use local paths.
>>>> 
>>>> We know there are still issues and we will be focusing heavily on the critical ones in the next few weeks, but
>>>> we do need the community to tell us what the critical issues really are.
>>>> 
>>>> Our number one priority at this time is that everyone that was able to build the old way, should be able to build
>>>> with the new build-infra makefiles. Please help us verify that.
>>>> 
>>>> -kto
>>>> 
>>>