hg clone is unbelievably slow

[Mario Torre]

Nice, I didn't know about that extension, but seems exactly what we
are searching for.

What would it take to add this support to the repos? From the linked
documentation I understand this is transparent for clients, so we
should see immediate improvements just by enabling it on the servers.

Cheers,
Mario

2018-02-06 16:04 GMT+01:00 Erik Helin <erik.helin at oracle.com>:
> There is no need to re-invent the wheel :)
>
> Mercurial (hg) has had support for clonebundles for a some time by now, see
> [0] for details. If the servers where configured to use clonebundles then
> users can continue to just use `hg clone` but get the performance of
> downloading compressed bundles. clonebundles can be stored on various CDN
> servers, then `hg clone` will download the closest bundle, unpack it and
> eventually do `hg pull --update` to get the latest commits (that might not
> be present in the bundle).
>
> For more information about clonebundles, just run
> `hg help -e clonebundles`.
>
> Thanks,
> Erik
>
> [0]: https://www.mercurial-scm.org/wiki/ClonebundlesExtension
>
>
> On 02/06/2018 03:42 PM, Mario Torre wrote:
>>
>> I think it should be a fully functional mercurial repository once is
>> unpacked.
>>
>> What I have in mind is something that support this workflow (or similar):
>>
>> $ hg clone  -r 12345 http://hg.openjdk.java.net/jdk10/client-gz
>> $ tar xvfz client-gz.tgz && cd client-gz
>> (do whatever, then some time later)
>> $ hg log || hg pull -u || commit etc...
>>
>> This should be just a few lines of scripting away.
>>
>> Cheers,
>> Mario
>>
>>
>>
>> 2018-02-06 15:35 GMT+01:00 Thomas Stüfe <thomas.stuefe at gmail.com>:
>>>
>>> Side question, we would only need to tar the .hg folder, no? And revive
>>> the
>>> workspace with hg update?
>>>
>>> On Tue, Feb 6, 2018 at 3:10 PM, Mario Torre
>>> <neugens.limasoftware at gmail.com>
>>> wrote:
>>>>
>>>>
>>>> I think it should probably be done on demand, it may be more expensive
>>>> to keep a commit based tgz otherwise.
>>>>
>>>> In fact the most awesome thing would be to have a special repository
>>>> that we clone but gives us back only the tgz to unpack locally, with
>>>> all the history and correct hg paths.
>>>>
>>>> Cheers,
>>>> Mario
>>>>
>>>> 2018-02-06 15:04 GMT+01:00 Remi Forax <forax at univ-mlv.fr>:
>>>>>
>>>>> +1
>>>>>
>>>>> creating a tgz at each commit doesn't seems that difficult ...
>>>>>
>>>>> Rémi
>>>>>
>>>>> ----- Mail original -----
>>>>>>
>>>>>> De: "Andrew Haley" <aph at redhat.com>
>>>>>> À: "jdk-dev" <jdk-dev at openjdk.java.net>
>>>>>> Envoyé: Mardi 6 Février 2018 11:50:07
>>>>>> Objet: hg clone is unbelievably slow
>>>>>
>>>>>
>>>>>> Half an hour or more here.  AFAIK the problem is due to the
>>>>>> inefficiency of Mercurial itself and the hg protocol.
>>>>>>
>>>>>> Aleksey Shipilev has done an experiment whereby trees are regularly
>>>>>> cloned and compressed tarballs created; these can be downloaded in a
>>>>>> couple of minutes.  But really we don't want to depend on the largesse
>>>>>> of one developer: if we could download the OpenJDK trees directly by
>>>>>> means of wget (or something similar) we would reduce the load on the
>>>>>> servers and reduce the time taken to download as well.
>>>>>>
>>>>>> --
>>>>>> Andrew Haley
>>>>>> Java Platform Lead Engineer
>>>>>> Red Hat UK Ltd. <https://www.redhat.com>
>>>>>> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> pgp key: http://subkeys.pgp.net/ PGP Key ID: 80F240CF
>>>> Fingerprint: BA39 9666 94EC 8B73 27FA  FC7C 4086 63E3 80F2 40CF
>>>>
>>>> Java Champion - Blog: http://neugens.wordpress.com - Twitter: @neugens
>>>> Proud GNU Classpath developer: http://www.classpath.org/
>>>> OpenJDK: http://openjdk.java.net/projects/caciocavallo/
>>>>
>>>> Please, support open standards:
>>>> http://endsoftpatents.org/
>>>
>>>
>>>
>>
>>
>>
>



-- 
pgp key: http://subkeys.pgp.net/ PGP Key ID: 80F240CF
Fingerprint: BA39 9666 94EC 8B73 27FA  FC7C 4086 63E3 80F2 40CF

Java Champion - Blog: http://neugens.wordpress.com - Twitter: @neugens
Proud GNU Classpath developer: http://www.classpath.org/
OpenJDK: http://openjdk.java.net/projects/caciocavallo/

Please, support open standards:
http://endsoftpatents.org/