RFR: JDK-8136695 Automatic build comparison with COMPARE_BUILD

[Magnus Ihse Bursie]

On 2015-09-17 17:14, Erik Joelsson wrote:
> Nice work!
>
> I wonder what happens if you invoke this target from the build output 
> directory? I can imagine it not working too well, but would like a 
> graceful exit in that case.

I keep forgetting about that use case, since I don't use it regularly 
myself. :-(

It was not too bad, though. I've updated the patch slightly, to cd to 
topdir before moving away the old directory. It works fine on my Linux 
computer, but I imagine there might be problems on Windows to rename a 
directory if it's in use.

I also noticed a problem with the MAKE=<target> part, due to a 
last-minute rename of the argument, which i fixed, and I added the new 
make target(s) to the output so it's correct about what is to be run.

Updated webrev, only changed in Init.gmk:
http://cr.openjdk.java.net/~ihse/JDK-8136695-compare_build/webrev.02

/Magnus

>
> /Erik
>
> On 2015-09-17 06:41, Magnus Ihse Bursie wrote:
>> A common scenario in when working in the build system is to want to 
>> check if a particular change results in a change in the build output 
>> or not, or a more detailed analysis of such changes.
>>
>> The compare.sh tool will provide such an analysis, but only if you 
>> have a pre-built baseline to compare with.
>>
>> To facilitate testing changes, especially on distributed job servers, 
>> we should add a COMPARE_BUILD make control variable. If this variable 
>> is present, the build will run twice, the second time with the code 
>> that should be tested (particular make targets, configure arguments, 
>> or with a patch file applied), and then automatically run the compare 
>> script afterwards against the baseline.
>>
>> The suggested format allows for a very fine-grained control, by 
>> further parsing the value of COMPARE_BUILD into separate parts. The 
>> syntax looks like this:
>> COMPARE_BUILD=CONF=<configure options>:PATCH=<patch file>:MAKE=<make 
>> targets>:COMP_OPTS=<compare script options>|<default>
>>
>> If neither CONF or PATCH is given, assume <default> means CONF if it 
>> begins with "--", otherwise assume it means PATCH. Either CONF or 
>> PATCH must be given, directly specified or by using the default 
>> syntax. The rest of the key/value pairs are optional, and they may 
>> come in any order.
>>
>> MAKE and COMP_OPTS can only be used with CONF and/or PATCH specified. 
>> If any value contains "+", it will be replaced by space.
>> If just a single value is specified, it can contain spaces, e.g. 
>> "COMPARE_BUILD=CONF=--enable-debug --with-special-sauce". (For 
>> technical reasons, this is not possible if using multiple, 
>> colon-separated key=value pairs).
>>
>> Values given in CONF and MAKE are appended to the normal configure 
>> command line and make target list, respectively, when building the 
>> second comparison round.
>>
>> The patch file, if specified, could be either absolute or relative to 
>> TOPDIR.
>>
>> COMP_OPTS is additional options to pass to compare.sh.
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8136695
>> WebRev: 
>> http://cr.openjdk.java.net/~ihse/JDK-8136695-compare_build/webrev.01
>>
>> /Magnus
>>
>