MILESTONE and BUILD_NUMBER with new build system

[Mike Duigou]

On Sep 24 2012, at 17:24 , David Holmes wrote:

> On 25/09/2012 9:43 AM, Mike Duigou wrote:
>> On Sep 24 2012, at 16:25 , David Holmes wrote:
>>> On 25/09/2012 6:38 AM, Henri Gomez wrote:
>>>> OpenJDK reports like this :
>>>> 
>>>> openjdk version "1.8.0-internal"
>>>> OpenJDK Runtime Environment (build 1.8.0-internal-myuser_2012_09_24_15_47-b00)
>>>> OpenJDK Server VM (build 24.0-b21, mixed mode)
>>>> 
>>>> With old build system I used to set MILESTONE and BUILD_NUMBER env
>>>> vars to have JDK reports like this :
>>>> 
>>>> openjdk version "1.8.0-b56"
>>>> OpenJDK Runtime Environment (build 1.8.0-b56-20120918)
>>>> OpenJDK 64-Bit Server VM (build 24.0-b21, mixed mode)
>>>> 
>>>> How did we set them now with new build system ?
>>> 
>>> I recently raised this problem with build-infra. Presently the MILESTONE
>>> and JDK_BUILD_NUMBER (note name change) are used for these but they get
>>> overridden by the values in the version.numbers file. So you can either
>>> put the right numbers in that file, or delete those variables from the
>>> file and set them via the environment.
>>> 
>> 
>> (Not addressed to David but in response to his informative comment)
>> 
>> Ugh. Magic files and magic environment variables. That's very disappointing. Why are we using configure again?
>> 
>> I really hoped that the new build-infra system would relieve us of this kind of hocus pocus. The build system should only be looking to the environment for ephemerals that have no visible effect on the resulting bits. This would include such things as build verbosity control, temp dir location, target dir, etc. (perhaps max-mem and max-cores too)
>> 
>> Build parameters that do have an effect on the output bits really *must* be configure parameters. The sense of version.numbers also appears to be backwards. It should be defaults with configure params overriding. User provided params should not be overridden, silently or otherwise.
> 
> The current situation is not build-infra's fault, they are copying (to some extent) current practice. Current practice allows, eg, RE to set things like build number and milestone when they build and that is done via environment variables. "we" use this capability all the time to control version info etc.

The version.numbers file seems be a reasonable place to get those defaults. The use of configure for build number and milestone would only be to override the defaults.

> I do not agree that these things should be configure variables at all! Why should I have to re-run configure because I've done a "hg pull -u" and am now building b57 instead of b56 ?

In this case I would hope that you would have not have had to specify version parameters to configure and the default would have been pulled from version.numbers.

Regarding re-running configure. It's been unclear to me when I can safely omit deleting my builds and re-running configure after a pull. With an abundance of paranoia I've been re-running configure after every pull and doing a clean build. Almost certainly this is overkill but absent a reliable recipe to determine when it can be skipped, I'll keep doing reconfigure and clean build. Full automake actually handles this situation fairly well. I wonder if perhaps a check in the common/makefiles/Makefile similar to "find . -cnewer ../../build/linux-x64-normal-server-fastdebug/config.status -type f" might be warranted.

> Having the build/milestone in version.numbers is okay provided that they get updated on each promotion.

Seems like a problem if they don't...

> This currently happens with hotspot version info for example. But until/if that happens version.numbers breaks your ability to set things via the environment.

If version.numbers is defaults only then setting via configure or environment is OK. 

> Regardless I should always be able to override these things via the environment.

I worry about what else is going to be specifiable, overridable via the environment. Are the environment settings you are using documented anywhere? What other settings are there?

The old build system has dozens or hundreds of environment variables that can be messed with. After several years of working on the JDK I still don't have any idea about even half the knobs available to me let alone under what conditions I should use them. Instead, like most OpenJDK developers I have a "recipe"[1] and am oblivious to what other options might even exist. I am sure there are loads of forgotten and broken options unknown and unused by anyone.

The ideal is that most developers and especially newbies don't ever have to do anything beyond:

hg clone http://hg.openjdk.java.net/jdk8/jdk8
cd jdk8
sh ./get_source.sh
./configure
make images javadoc

(I know that the current build isn't quite this simple yet).

Setting configure flags, shell environment variables or make variables (make foo=bar) should not be part of the "default user experience".

One other point: If the build does pick up anything from the environment it should definitely produce a log statement that it did so. I previously had to debug a build which turned out to be conditionalized based upon "TERM" environment variable of the compiling host. Compiling from rlogin terminal produced a working build, compiling from an xterm did not. Incredibly frustrating to track down. It should always be possible to determine where every bit of build config came from.

Mike

[1] LANG=C ALT_BOOTDIR=~/jdk7/ ALLOW_DOWNLOADS=true HOTSPOT_BUILD_JOBS=16 PARALLEL_COMPILE_JOBS=16 SKIP_FASTDEBUG_BUILD=true NO_DOCS=true NO_DEMOS=true NO_SAMPLES=true make


> David
> -----
> 
>> Mike
>> 
>>> 
>>>> Cheers
>>> 
>>