RFR: race with nested repos in hgforest.sh

[Chris Hegarty]

On 4 Feb 2013, at 19:36, Ioi Lam <ioi.lam at oracle.com> wrote:

> How about adding a message to indicate the sleep, just in case the directory is never created for whatever reason. If you don't like too many such messages, you can print the message after the initial wait.

I understand the reason for your suggestion, but in my environment it can take more than a minute before these directories are created, which will be very noisy with six of these nested repos. If these repos are never created then you have bigger problems, disk space, I/O, etc, which such be indicated by the process that encounters the problem.

-Chris.

> 
> 177           while [ ! -d "$path" ]   ## nested repo, ensure containing dir exists
> 178           do
> 179             sleep 5
>                 if[ ! -d "$path" ] ; then
>                     echo "Sleeping to wait for '$path' to be created";
>                 fi
> 180           done
> 181         fi
> 
> - Ioi
> 
> On 02/01/2013 02:37 PM, David Holmes wrote:
>> Hi Chris,
>> 
>> On 2/02/2013 12:40 AM, Chris Hegarty wrote:
>>> [ to build-dev and core-libs-dev, expect reviewer from either, but will
>>> integrate through jdk8/tl ]
>>> 
>>> This issue is mainly of interest to Oracle engineers, but it effects the
>>> public hgforest script.
>>> 
>>> When hgforest.sh is run with an addition argument to specify a closed
>>> server, there is a problem/race between the creation of the directories
>>> to hold nested repositories and the clone itself. These directories need
>>> to be created before the clone command is executed, otherwise it will
>>> fail, as below.
>> 
>> I think I reported this myself just last week - probably internally to build or build-infra.
>> 
>> The weird thing is that based on the list of repos it shouldn't be grabbing the closed one yet.
>> 
>>> The trivial fix is to back off these nested repos until their containing
>>> directory exists.
>> 
>> That seems to assume/require that the nested repo will always be later in the list. They happen to be but ....
>> 
>> Workaround is to run get_source.sh twice first without the "extra base" then with it.
>> 
>> David
>> -----
>> 
>>> 
>>> Webrev:
>>> http://cr.openjdk.java.net/~chegar/hgforest_nestedRepos/webrev/
>>> 
>>> sh ./get_source.sh http://xxx.yyy.oracle.com | & tee clone.log
>>> # Repositories: corba jaxp jaxws langtools jdk hotspot jdk/src/closed
>>> jdk/make/closed jdk/test/closed hotspot/make/closed hotspot/src/closed
>>> hotspot/test/closed deploy install sponsors pubs
>>> 
>>> Waiting 5 secs before spawning next background command.
>>> Waiting 5 secs before spawning next background command.
>>> Waiting 5 secs before spawning next background command.
>>> jdk/src/closed: /java/devtools/sparc/mercurial/0.9.5/bin/python -u
>>> /java/devtools/sparc/mercurial/latest/bin/hg clone
>>> http://xxx.yyy.oracle.com/jdk8/tl/jdk/src/closed jdk/src/closed
>>> jdk/src/closed: abort: No such file or directory: jdk/src/closed
>>> jdk/make/closed: /java/devtools/sparc/mercurial/0.9.5/bin/python -u
>>> /java/devtools/sparc/mercurial/latest/bin/hg clone
>>> http://xxx.yyy.oracle.com/jdk8/tl/jdk/make/closed jdk/make/closed
>>> jdk/make/closed: abort: No such file or directory: jdk/make/closed
>>> Waiting 5 secs before spawning next background command.
>>> ....
>>> 
>>> 
>>> -Chris.
>