hg repos/forests backups (Was: serving forests over http)

[Mark Wielaard]

Hi,

On Sun, 2009-01-04 at 20:31 +0100, Mark Wielaard wrote:
> I was setting up a backup of the forests to provide some redundancy in
> case the server goes down again. But I am unable to convince
> hgwebdir.cgi (or actually hg serve) to provide forest support through
> http. fclone locally or through ssh works fine though.
> How did you configure hgwebdir to make fclone over http work?

Thanks to Kelly I figured some stuff out.

There is now a mirror of the IcedTea repos (icedtea, icedtea6, brandweg,
fedora, icepick, pulseaudio and the testrepo), the main OpenJDK repos
(jdk6 and jdk7 master forests) and a copy of the old openjdk svn and
openjdk6 code drop hg mirrors at: http://classpath.wildebeest.org/hg/
This is a read-only mirror that is synced once an hour.

The main issues setting this up were:

- The forest repo itself has moved. There are two:
  http://hg.akoha.org/hgforest/ (the main repo)
  http://www.bitbucket.org/pmezard/hgforest-crew/
  (for making forest work with the latest and greatest - but see below)

- hgwebdir.config isn't a real hgrc file, only parts of it are parsed.
  (In particular [extensions] aren't parsed)

- Some versions of hgwebdir don't pick up the default hgrc files.
  You might have to play with something like:
  import os
  os.environ["HGRCPATH"] = "/var/lib/hg/.hgrc:/etc/mercurial/hgrc"
  (where one of those hgrc files has the forest extension defined)

- It seems that whatever version of forest you use, the hgweb/http
  support only reliably works with mercurial 0.95. I couldn't get it to
  work with anything newer or older (local file system fclones or
  fclones through ssh do work fine with newer versions though, just the
  http versions - also those created with hg serve - don't.)

- Unfortunately 0.95 hgwebdir doesn't work nicely with nested (forest)
  repos. You will end up having to define every forest in full in your
  hgwebdir.config like:

  [paths]
  jdk6/corba = /hg/jdk6/corba
  jdk6/hotspot = /hg/jdk6/hotspot
  jdk6/jaxp = /hg/jdk6/jaxp
  jdk6/jaxws = /hg/jdk6/jaxws
  jdk6/jdk = /hg/jdk6/jdk
  jdk6/langtools = /hg/jdk6/langtools

  Which is a pity since with 1.1.2 you could just do:
  [paths]
  / = /hg/**
  (all assuming you have a top level /hg dir with all your repos)

- Mercurial 1.1 and higher support the fncache repo format, Mercurial
  0.95 won't be able to read these. So if you experimented with 1.1+
  then you will have to convert your repo first:
  http://www.selenic.com/mercurial/wiki/index.cgi/fncacheRepoFormat

- The hg --debug fclone ... option is a good way to see if the forest
  capabilities have been picked up, it will should show you something
  like:

  sending capabilities command
  capabilities: unbundle=HG10GZ,HG10BZ,HG10UN lookup forests changegroupsubset
  sending forests command

  If it doesn't work then the client will print:
  abort: Remote forests cannot be cloned because the other repository 
  doesn't support the forest extension.

Cheers,

Mark