JDK 7 Repository updates

[Volker Simonis]

Hi Neil,

thanks a lot for your detailed mail.
I'm really always happy to learn some new and cool Mercurial tricks:)

Please find my comments inline:

On Wed, Feb 6, 2013 at 4:05 PM, Neil Richards <neil.richards at ngmr.net> wrote:
> Hi Volker,
> I'm a little intrigued that you want to sync up to jdk7u-dev (i.e. the
> component forest), rather than jdk7u (the master).
>
> (Of course, I may just be being pedantic here, as the contents of these
> two forests tend to be similar - there's just a greater theoretical risk
> of abnormal perturbation by pulling fresh changes from the component
> forest, rather than pulling changes once they've been promoted to the
> master forest.)
>

Sorry, that was just my looseness that I confused jdk7u-dev and jdk7u.
I really meant jdk7u (which in the context of my initial mail also meant
that we would had to rename either the old or the newly created repository,
which is not nice as well)

>
> Of course, the syncing operation that you're considering is not a
> one-time operation - the ppc forest is only "current" after a sync until
> the next change is made in jdk7u, at which point another sync is needed
> to get back to being "current".
>
> So creating a new or replacement forest each time there's a sync doesn't
> seem like a good strategy to me.
>
> However, I think there is a variation on your second strategy that
> doesn't require the creation of a new forest that might be viable ...
>
> ----
>
> Consider this conceptual starting position, with two ppc changes (p.1 &
> p.2) made against a version of code from jdk7u (o.1):
>
>         @       p.2
>         |
>         o       p.1
>         |
>         o       o.1
>
>
> (I've simplified things for the sake of discussion).
>
> In a local clone, new change(s) from jdk7u are pulled in, so there's a
> new version of code (o.2) to be combined (in some fashion) with the ppc
> changes:
>
>
>         o       o.2
>         |
>         | @     p.2
>         | |
>         | o     p.1
>         |/
>         o       o.1
>
>
> Work is done (via rebase and/or mq import / export) to get the ppc
> changes on top of o.2:
>
>           @     p.2'
>           |
>           o     p.1'
>          /
>         o       o.2
>         |
>         o       o.1
>
>
> The original p.1 & p.2 changes are still in the master ppc forest, and
> can be pulled back down to the local clone:
>
>           @     p.2'
>           |
>           o     p.1'
>          /
>         o       o.2
>         |
>         | o     p.2
>         | |
>         | o     p.1
>         |/
>         o       o.1
>
>
> The old "spur" containing the original p.1 & p.2 can now be marked as
> "closed", by updating to p.2 using:
>         hg ci --close-branch -m "Code rebased to $(hg log -r tip
>         --template '{node|short}')"
>
> That results in a tree (with c.1 being the close branch changeset) like
> this:
>
>             x   c.1
>             |
>           o |   p.2'
>           | |
>           o |   p.1'
>          /  |
>         o   |   o.2
>         |  /
>         | o     p.2
>         | |
>         | o     p.1
>         |/
>         o       o.1
>
> Then, use mq to resequence the changes, such that the close branch
> change occurs before p.1' & p.2':
>

If I understand this right, it would be also possible to:
1. pull
2. hg ci --close-branch
3. hg update -r 'o.2'
4. hg export p.1 && hg qimport p.1
5. hg export p.2 && hg qimport p.2

This would avoid the step to 'resequence' the changes p.1' and p.2'

>           @     p.2'
>           |
>           o     p.1'
>           |
>           | x   c.1
>          /  |
>         o   |   o.2
>         |  /
>         | o     p.2
>         | |
>         | o     p.1
>         |/
>         o       o.1
>
> Then push the new changes (c.1, p.1' & p.2') up to the master in one
> fell swoop.
>
> NB: This push will need the (usually dangerous) '-f' option, as the
> receiving ends sees the number of head being increased (by one), because
> it doesn't recognise that c.1 reduces that count (back down to 1).
>

Are you really sure that this is possible with the OpenJDK repositories.
I always thought that they don't allow multi-head configurations.

I really like your approach but I'm afraid that's not possible with OpenJDK
repositories. It is true that 'hg heads' will only show one head with your
approach, but actually that's just sham. You really have two heads afterwards
(you can see them with 'hg heads -c') and that's also the reason why you
have to use the '-f' option for 'hg push'.

> However, the net effect after the push is that there is still one active
> head, and the "tip" change in the master repository is that for the
> active head.
>

Unfortunately I'm pretty sure that the OpenJDK repos don't allow this (and
actually there is no single example of a closed branch in the hotspot or
jdk repositories). I therefore think that your approach is only possible for
local repositories to manage own changes with respect to a constantly
moving upstream repository.

> ----
>
> This is how we have been handling our historic changes internally, as we
> work them into shape for contribution to OpenJDK.
>
> As you say, there is some overhead in working with the changes in this
> manner (instead of using 'hg merge'), but the benefits in doing so may
> outweigh the extra cost.
>
> By marking the old spurs "closed" (rather than trying to eliminate their
> existence from history, as moving to a new forest would do), I think
> that it would more accurately reflect the nature of the actions that are
> being performed.
>
> Any thoughts / comments on this would be gratefully received.
>

I think we may do a "no-op merge" as described in:
http://mercurial.selenic.com/wiki/PruningDeadBranches#No-Op_Merges
(i.e. we pull from 7u, do a "no-op merge" to totally eliminate our
changes and then selectively re-apply them).
That's not beautiful nor elegant, but it may be the only possible solution
if we want to stay with one repository (which I agree is worth while).
And because we will not directly push from our porting repositories into
the main jdk repos the intermediate "no-op merges" won't bother anybody.

Do you see other ways of doing this?

Thank you and best regards,
Volker

> Regards,
> Neil
>
>
> On Tue, 2013-02-05 at 11:13 +0100, Volker Simonis wrote:
>> Hi,
>>
>> we have internally discussed the best way for syncing up with the
>> upstream jdk7u-dev repository forest. Actually there are two
>> possibilities:
>>
>> 1. "pull -u" and merge
>> pros: simple and fast
>> cons: one big merge changeset which may hide changes which have
>> previously been intentionally and carefully split into different
>> changes.
>>
>> 2. create a new forest (as clone of jdk7u-dev) and transplant our
>> changes from 7u into the new 7u-dev
>> pros: clean new repo with the chance to manually merge related
>> changesets from our current repository
>> cons: a little more time consuming than 1
>>
>> We are very much in favour of alternative 2 because we would like to
>> combine some of our changes (especially from the HotSpot repository).
>> We initially already tried to bring in self contained, meanigful
>> changes but in reality they had often been followed by small bug fix
>> changes. With the new jdk7u-dev forest we would get the opportunity to
>> clean this up. This would also be an intermediate step towards the
>> ultimate goal to provide a set of clean, reviewable changes for jdk8.
>>
>> We would like to start with this work within a week or two. Once we
>> had the new jdk7u-dev forest in our porting repository we would
>> sequentially bring over all changes from the current 7u forest. We
>> expect this process will not take more than about two weeks.
>> Afterwards, the 7u forest will be closed and all further work will be
>> done in the 7u-dev forest (i.e. nightly build and tests will use this
>> new forest). The plan is to synchronize this new forest more regularly
>> with the upstream 7u-dev repository.
>>
>> This would give us a current jdk 7 version (i.e. something which
>> should be very close to the upcoming 7u14 release) and a good starting
>> point for the next step which would be the integration into jdk8.
>> Especially we don't expect that the HotSpot repository in 7u-dev will
>> need much attention in the future any more because it seems the the
>> currently integrated HotSpot Express 24 (hsx24) is the last new
>> HotSpot version which has been backported into the Java 7 releases.
>>
>> Please let us know if you are comfortable with this approach.
>>
>> Regards,
>> Volker
>
>
> --
> Unless stated above:
> IBM email: neil_richards at uk.ibm.com
> IBM United Kingdom Limited - Registered in England and Wales with number 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>