hg clone is unbelievably slow
Erik Helin
erik.helin at oracle.com
Tue Feb 6 16:08:49 UTC 2018
On 02/06/2018 05:08 PM, Thomas Stüfe wrote
> On Tue, Feb 6, 2018 at 4:58 PM, Erik Helin <erik.helin at oracle.com> wrote:
> On 02/06/2018 04:18 PM, Mario Torre wrote:
> From the linked
> documentation I understand this is transparent for clients, so we
> should see immediate improvements just by enabling it on the
> servers.
>
>
> Yes, all hg clients that understand clonebundles will use it (and
> thereby get the performance improvement) and older hg clients will
> just continue to use the traditional way of cloning.
>
>
> This sounds neat. Assuming we would not update the cached bundle for
> every push, so the cached bundle gets outdated, would the client pull
> the remaining changes from the real repository?
Yep, that is exactly how it works.
Thanks,
Erik
> ..Thomas
>
>
> Thanks,
> Erik
>
>
> Cheers,
> Mario
>
> 2018-02-06 16:04 GMT+01:00 Erik Helin <erik.helin at oracle.com
> <mailto:erik.helin at oracle.com>>:
>
> There is no need to re-invent the wheel :)
>
> Mercurial (hg) has had support for clonebundles for a some
> time by now, see
> [0] for details. If the servers where configured to use
> clonebundles then
> users can continue to just use `hg clone` but get the
> performance of
> downloading compressed bundles. clonebundles can be stored
> on various CDN
> servers, then `hg clone` will download the closest bundle,
> unpack it and
> eventually do `hg pull --update` to get the latest commits
> (that might not
> be present in the bundle).
>
> For more information about clonebundles, just run
> `hg help -e clonebundles`.
>
> Thanks,
> Erik
>
> [0]:
> https://www.mercurial-scm.org/wiki/ClonebundlesExtension
> <https://www.mercurial-scm.org/wiki/ClonebundlesExtension>
>
>
> On 02/06/2018 03:42 PM, Mario Torre wrote:
>
>
> I think it should be a fully functional mercurial
> repository once is
> unpacked.
>
> What I have in mind is something that support this
> workflow (or similar):
>
> $ hg clone -r 12345
> http://hg.openjdk.java.net/jdk10/client-gz
> <http://hg.openjdk.java.net/jdk10/client-gz>
> $ tar xvfz client-gz.tgz && cd client-gz
> (do whatever, then some time later)
> $ hg log || hg pull -u || commit etc...
>
> This should be just a few lines of scripting away.
>
> Cheers,
> Mario
>
>
>
> 2018-02-06 15:35 GMT+01:00 Thomas Stüfe
> <thomas.stuefe at gmail.com <mailto:thomas.stuefe at gmail.com>>:
>
>
> Side question, we would only need to tar the .hg
> folder, no? And revive
> the
> workspace with hg update?
>
> On Tue, Feb 6, 2018 at 3:10 PM, Mario Torre
> <neugens.limasoftware at gmail.com
> <mailto:neugens.limasoftware at gmail.com>>
> wrote:
>
>
>
> I think it should probably be done on demand, it
> may be more expensive
> to keep a commit based tgz otherwise.
>
> In fact the most awesome thing would be to have
> a special repository
> that we clone but gives us back only the tgz to
> unpack locally, with
> all the history and correct hg paths.
>
> Cheers,
> Mario
>
> 2018-02-06 15:04 GMT+01:00 Remi Forax
> <forax at univ-mlv.fr <mailto:forax at univ-mlv.fr>>:
>
>
> +1
>
> creating a tgz at each commit doesn't seems
> that difficult ...
>
> Rémi
>
> ----- Mail original -----
>
>
> De: "Andrew Haley" <aph at redhat.com
> <mailto:aph at redhat.com>>
> À: "jdk-dev" <jdk-dev at openjdk.java.net
> <mailto:jdk-dev at openjdk.java.net>>
> Envoyé: Mardi 6 Février 2018 11:50:07
> Objet: hg clone is unbelievably slow
>
>
>
> Half an hour or more here. AFAIK the
> problem is due to the
> inefficiency of Mercurial itself and the
> hg protocol.
>
> Aleksey Shipilev has done an experiment
> whereby trees are regularly
> cloned and compressed tarballs created;
> these can be downloaded in a
> couple of minutes. But really we don't
> want to depend on the largesse
> of one developer: if we could download
> the OpenJDK
> <https://maps.google.com/?q=OpenJDK&entry=gmail&source=g>trees
> directly by
> means of wget (or something similar) we
> would reduce the load on the
> servers and reduce the time taken to
> download as well.
>
> --
> Andrew Haley
> Java Platform Lead Engineer
> Red Hat UK Ltd. <https://www.redhat.com>
> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035
> 332F A671
>
>
>
>
>
> --
> pgp key: http://subkeys.pgp.net/ PGP Key ID:
> 80F240CF
> Fingerprint: BA39 9666 94EC 8B73 27FA FC7C 4086
> 63E3 80F2 40CF
>
> Java Champion - Blog:
> http://neugens.wordpress.com - Twitter: @neugens
> Proud GNU Classpath developer:
> http://www.classpath.org/
> OpenJDK:
> http://openjdk.java.net/projects/caciocavallo/
> <http://openjdk.java.net/projects/caciocavallo/>
>
> Please, support open standards:
> http://endsoftpatents.org/
>
>
>
>
>
>
>
>
>
>
>
>
More information about the jdk-dev
mailing list