RFR: 8041151: More concurrent hgforest.

[Staffan Larsen]

I ran the patch on OS X and it worked there too. Thanks!

You mention “status code from subprocesses” which got me thinking of a problem I frequently run into when I have some uncommitted changes in one of the repos:

                    ...
                    .:   searching for changes
                    .:   adding changesets
                    .:   adding manifests
                    .:   adding file changes
                    .:   added 1 changesets with 2 changes to 2 files (+1 heads)
                    .:   not updating: not a linear update
                    .:   (merge or update --check to force update)
                    ...

This output is frequently lost to me in all the other output, would it be possible to catch these errors and somehow highlight the failure at the end of the run? I guess I could go spelunking in the code to find out, but I’m too lazy and prefer to bother someone else. :)

Thanks,
/Staffan

On 19 apr 2014, at 01:21, Mike Duigou <mike.duigou at oracle.com> wrote:

> Hello all;
> 
> This is an improvement to hgforest to increase it's concurrency behaviour. Currently hgforest.sh limits the rate at which it starts new sub-processes because it wants to limit the number of concurrent tasks. The naive approach it takes can cause unnecessary delays. For sequences of operations that are entirely local the overhead of waiting is significant. The revised implementation uses fifos for completion notification on capable platforms or compares started task count to completed task count in a shorter sleep loop.
> 
> The intention is to use the enhanced concurrency to allow for a fancier get_source.sh that can handle rebasing for mq patches. This involves running a half dozen commands through hgforest. With my current in-development get_source.sh script changes these hgforest changes provide a 10X speedup. (4s vs 40s)
> 
> The changeset also incorporates a build-dev suggested improvement to extra base repo url handling and other minor fixes (status code from subprocesses).
> 
> JBSBUG: https://bugs.openjdk.java.net/browse/JDK-8041151
> WEBREV: http://cr.openjdk.java.net/~mduigou/JDK-8041151/0/webrev/
> 
> I've so far tested it on successfully on Linux (Ubuntu 13.10) Solaris (10u9) and Cygwin x64
> 
> Cheers,
> 
> Mike
>