JDK 7 Repository updates
Volker Simonis
volker.simonis at gmail.com
Fri Feb 22 07:57:24 PST 2013
Thanks Neil,
sounds like a perfect plan;)
Regards,
Volker
On Fri, Feb 22, 2013 at 3:51 PM, Neil Richards <neil.richards at ngmr.net> wrote:
> Hi Volker, Goetz,
>
> Sorry for the delay in getting back to you on this.
>
> For the jdk code, I'd like to go with the sync & merge approach
> (simplicity wins!).
>
> (I'm currently looking through the merge conflicts when merging the jdk
> code up with the latest from jdk7u, jdk7u14-b13.)
>
> If you find that the hotspot code updated to hs24 works with the current
> jdk code, I'm happy for you to submit the hs changes, and I'll look to
> follow suit with the jdk changes as soon as I've completed the jdk
> merge.
>
> Regards,
> Neil
>
> On Fri, 2013-02-22 at 11:11 +0100, Lindenmaier, Goetz wrote:
>> Hi,
>>
>> actually, our changes apply nicely to hs24, so I'm
>> ready to submit a first part already, and I'd like to do so.
>>
>> So we need a decision how to proceed.
>>
>> As I found out, hs24 works with the jdk we currently build.
>> So maybe we could just go ahead and submit the hs changes,
>> and then pull the jdk to the current state of jdk7u.
>>
>> Best regards,
>> Goetz.
>>
>>
>>
>> -----Original Message-----
>> From: ppc-aix-port-dev-bounces at openjdk.java.net [mailto:ppc-aix-port-dev-bounces at openjdk.java.net] On Behalf Of Volker Simonis
>> Sent: Mittwoch, 20. Februar 2013 18:36
>> To: Neil Richards
>> Cc: ppc-aix-port-dev at openjdk.java.net
>> Subject: Re: JDK 7 Repository updates
>>
>> Hi all,
>>
>> we've started working on hs24 (i.e. the newest HotSpot version from
>> 7u). After the discussions in this thread we've decided to use the
>> Mercurial Queue extension ('mq') to simplify this and future
>> synchronisations of the 'hotspot' repository with new upstream
>> versions. We will convert and consolidate our changes into a Mercurial
>> patch queue. We will then synchronise from upstream (i.e. pull from
>> 7u), do a "no-op merge" to eliminate our changes and finally reapply
>> the changes from our patch queue. Future synchronisations from
>> upstream will be handled similarly.
>>
>> We hope we will be ready to checkin the new HotSpot version by the end
>> of next week. We promise to be careful but it may happen that the
>> ppc-aix-port/jdk7u will become unstable for a few days after the
>> update.
>>
>> We wanted to ask you (i.e. *@ibm) if you want to take a similar
>> approach for the class library changes? Actually, the jdk repository
>> has been synced one time (in mid January) with the latest changes from
>> jdk7u6 (without the need for manual resolving). This was necessary to
>> get the certification finished.
>>
>> If you don't plan to prepare a patch queue we will just sync and merge
>> the jdk repository after we've finished the hotspot one (notice that
>> his is necessary this time because of the new JSR292 implementation in
>> hs24 which has dependencies on the class library).
>>
>> I you want to create a patch queue yourself (or consolidate and
>> reapply the porting changes in some other way) please let us know so
>> we can synchronise (because of the mentioned dependencies).
>>
>> Regards,
>> Volker
>>
>>
>> On Thu, Feb 7, 2013 at 3:46 PM, Neil Richards <neil.richards at ngmr.net> wrote:
>> > Hi Volker,
>> > Please find some further comments inline below:
>> >
>> > On Wed, 2013-02-06 at 19:31 +0100, Volker Simonis wrote:
>> >> Hi Neil,
>> >>
>> >> thanks a lot for your detailed mail.
>> >> I'm really always happy to learn some new and cool Mercurial tricks:)
>> >>
>> >> Please find my comments inline:
>> >>
>> >> On Wed, Feb 6, 2013 at 4:05 PM, Neil Richards <neil.richards at ngmr.net> wrote:
>> >> > Hi Volker,
>> >> > I'm a little intrigued that you want to sync up to jdk7u-dev (i.e. the
>> >> > component forest), rather than jdk7u (the master).
>> >> >
>> >> > (Of course, I may just be being pedantic here, as the contents of these
>> >> > two forests tend to be similar - there's just a greater theoretical risk
>> >> > of abnormal perturbation by pulling fresh changes from the component
>> >> > forest, rather than pulling changes once they've been promoted to the
>> >> > master forest.)
>> >> >
>> >>
>> >> Sorry, that was just my looseness that I confused jdk7u-dev and jdk7u.
>> >> I really meant jdk7u (which in the context of my initial mail also meant
>> >> that we would had to rename either the old or the newly created repository,
>> >> which is not nice as well)
>> >>
>> >> >
>> >> > Of course, the syncing operation that you're considering is not a
>> >> > one-time operation - the ppc forest is only "current" after a sync until
>> >> > the next change is made in jdk7u, at which point another sync is needed
>> >> > to get back to being "current".
>> >> >
>> >> > So creating a new or replacement forest each time there's a sync doesn't
>> >> > seem like a good strategy to me.
>> >> >
>> >> > However, I think there is a variation on your second strategy that
>> >> > doesn't require the creation of a new forest that might be viable ...
>> >> >
>> >> > ----
>> >> >
>> >> > Consider this conceptual starting position, with two ppc changes (p.1 &
>> >> > p.2) made against a version of code from jdk7u (o.1):
>> >> >
>> >> > @ p.2
>> >> > |
>> >> > o p.1
>> >> > |
>> >> > o o.1
>> >> >
>> >> >
>> >> > (I've simplified things for the sake of discussion).
>> >> >
>> >> > In a local clone, new change(s) from jdk7u are pulled in, so there's a
>> >> > new version of code (o.2) to be combined (in some fashion) with the ppc
>> >> > changes:
>> >> >
>> >> >
>> >> > o o.2
>> >> > |
>> >> > | @ p.2
>> >> > | |
>> >> > | o p.1
>> >> > |/
>> >> > o o.1
>> >> >
>> >> >
>> >> > Work is done (via rebase and/or mq import / export) to get the ppc
>> >> > changes on top of o.2:
>> >> >
>> >> > @ p.2'
>> >> > |
>> >> > o p.1'
>> >> > /
>> >> > o o.2
>> >> > |
>> >> > o o.1
>> >> >
>> >> >
>> >> > The original p.1 & p.2 changes are still in the master ppc forest, and
>> >> > can be pulled back down to the local clone:
>> >> >
>> >> > @ p.2'
>> >> > |
>> >> > o p.1'
>> >> > /
>> >> > o o.2
>> >> > |
>> >> > | o p.2
>> >> > | |
>> >> > | o p.1
>> >> > |/
>> >> > o o.1
>> >> >
>> >> >
>> >> > The old "spur" containing the original p.1 & p.2 can now be marked as
>> >> > "closed", by updating to p.2 using:
>> >> > hg ci --close-branch -m "Code rebased to $(hg log -r tip
>> >> > --template '{node|short}')"
>> >> >
>> >> > That results in a tree (with c.1 being the close branch changeset) like
>> >> > this:
>> >> >
>> >> > x c.1
>> >> > |
>> >> > o | p.2'
>> >> > | |
>> >> > o | p.1'
>> >> > / |
>> >> > o | o.2
>> >> > | /
>> >> > | o p.2
>> >> > | |
>> >> > | o p.1
>> >> > |/
>> >> > o o.1
>> >> >
>> >> > Then, use mq to resequence the changes, such that the close branch
>> >> > change occurs before p.1' & p.2':
>> >> >
>> >>
>> >> If I understand this right, it would be also possible to:
>> >> 1. pull
>> >> 2. hg ci --close-branch
>> >> 3. hg update -r 'o.2'
>> >> 4. hg export p.1 && hg qimport p.1
>> >> 5. hg export p.2 && hg qimport p.2
>> >>
>> >> This would avoid the step to 'resequence' the changes p.1' and p.2'
>> >
>> > It would, though at the cost of not being able to reference the p.2'
>> > changeset in the commit comment for the close branch (c.1) changeset
>> > (because c.1 is committed before the changeset id for p.2' is known).
>> >
>> > I reckon that this extra reference makes the history a little more
>> > legible when reading it back later.
>> > (This is particularly so if original changes have been reformed and/or
>> > folded together.)
>> >
>> >> > @ p.2'
>> >> > |
>> >> > o p.1'
>> >> > |
>> >> > | x c.1
>> >> > / |
>> >> > o | o.2
>> >> > | /
>> >> > | o p.2
>> >> > | |
>> >> > | o p.1
>> >> > |/
>> >> > o o.1
>> >> >
>> >> > Then push the new changes (c.1, p.1' & p.2') up to the master in one
>> >> > fell swoop.
>> >> >
>> >> > NB: This push will need the (usually dangerous) '-f' option, as the
>> >> > receiving ends sees the number of head being increased (by one), because
>> >> > it doesn't recognise that c.1 reduces that count (back down to 1).
>> >> >
>> >>
>> >> Are you really sure that this is possible with the OpenJDK repositories.
>> >> I always thought that they don't allow multi-head configurations.
>> >
>> > You are correct that using close-branch mechanisms is verboten, at least
>> > within the "main" reference-implementation OpenJDK projects (e.g. jdk7,
>> > jdk7u, jdk8, hsx).
>> >
>> > These projects also have other restrictions on the changes that they
>> > accept, such as each (non-merge, non-tag change) being associated with a
>> > unique, valid Java bug id.
>> > This does not apply to *all* projects in OpenJDK - my understanding is
>> > that the Project Lead determines the rules which apply to each project.
>> >
>> > Any policing of these restrictions at the OpenJDK mercurial location
>> > will be done by check-in hooks. For the "main" OpenJDK projects, I think
>> > this includes the use of 'jcheck'. I'm not sure what hooks there are for
>> > ppc-aix.
>> >
>> > (I suppose one could try my suggested approach & see if the changes can
>> > be pushed up - if they can't, then there's some commit checking that
>> > prevents it, and you know a different choice of approach is needed).
>> >
>> >> I really like your approach but I'm afraid that's not possible with OpenJDK
>> >> repositories. It is true that 'hg heads' will only show one head with your
>> >> approach, but actually that's just sham. You really have two heads afterwards
>> >> (you can see them with 'hg heads -c') and that's also the reason why you
>> >> have to use the '-f' option for 'hg push'.
>> >
>> > True, the number of total heads has increased by 1 (as has the number of
>> > closed heads).
>> > The number of active heads remains at 1.
>> >
>> >> > However, the net effect after the push is that there is still one active
>> >> > head, and the "tip" change in the master repository is that for the
>> >> > active head.
>> >> >
>> >>
>> >> Unfortunately I'm pretty sure that the OpenJDK repos don't allow this (and
>> >> actually there is no single example of a closed branch in the hotspot or
>> >> jdk repositories). I therefore think that your approach is only possible for
>> >> local repositories to manage own changes with respect to a constantly
>> >> moving upstream repository.
>> >
>> > As I say above, I think you may find that it is under the jurisdiction
>> > of the ppc-aix Project Lead to choose what mercurial conventions to
>> > follow (subject to any universally enforced commit hooks).
>> >
>> > As Project Lead, you might decide that using the same conventions as the
>> > "main" OpenJDK projects makes everything simpler and clearer.
>> > The convention followed by the "main" projects (in their component
>> > forests) for syncing up with later changes (from their master forest) is
>> > to use 'hg merge', of course, so following this course might sway you
>> > back towards going with your option 1).
>> >
>> >
>> >> > ----
>> >> >
>> >> > This is how we have been handling our historic changes internally, as we
>> >> > work them into shape for contribution to OpenJDK.
>> >> >
>> >> > As you say, there is some overhead in working with the changes in this
>> >> > manner (instead of using 'hg merge'), but the benefits in doing so may
>> >> > outweigh the extra cost.
>> >> >
>> >> > By marking the old spurs "closed" (rather than trying to eliminate their
>> >> > existence from history, as moving to a new forest would do), I think
>> >> > that it would more accurately reflect the nature of the actions that are
>> >> > being performed.
>> >> >
>> >> > Any thoughts / comments on this would be gratefully received.
>> >> >
>> >>
>> >> I think we may do a "no-op merge" as described in:
>> >> http://mercurial.selenic.com/wiki/PruningDeadBranches#No-Op_Merges
>> >> (i.e. we pull from 7u, do a "no-op merge" to totally eliminate our
>> >> changes and then selectively re-apply them).
>> >> That's not beautiful nor elegant, but it may be the only possible solution
>> >> if we want to stay with one repository (which I agree is worth while).
>> >> And because we will not directly push from our porting repositories into
>> >> the main jdk repos the intermediate "no-op merges" won't bother anybody.
>> >
>> > This would be anther viable approach, and would effectively eliminate
>> > the need to refer to the p.2' changeset id in the commit comment of c.1
>> > (as the parent changests of the no-op merge changeset would effectively
>> > record that association).
>> >
>> > These no-op branches would cause the dead branches to be listed as part
>> > of the history of the latest changes.
>> >
>> > Perhaps if the dead branches were marked as closed and then no-op
>> > merged, dead branch history could be identified and pruned out of any
>> > listing, using 'hg log -P', though the revset incantation to achieve
>> > this would be fairly complex, I suspect.
>> >
>> > (Whichever approach is taken, I agree that it doesn't affect what
>> > happens when changes are adopted into the main jdk forest(s)).
>> >
>> >> Do you see other ways of doing this?
>> >
>> > No, I think we've those three techniques are all that I can think of.
>> >
>> > One other point, regarding the combining of changesets into "meaningful"
>> > changes ...
>> >
>> > The natural temptation when doing this is to tend to combine more
>> > changes together, to end up with fewer, larger changes.
>> >
>> > In general, it is good to resist giving in to this temptation, as
>> > smaller, more fine-grained changes tend to be easier (for everyone) to
>> > consider during activities like promotion to the "mainline".
>> >
>> > It's also a lot easier in mercurial to combine small changes together
>> > "on-demand" with 'hg qfold' than it is to split them apart again.
>> > So any such "folding" can be fairly safely deferred until close to
>> > mainline "promotion", whilst there is benefit for any suitable
>> > "splitting" to be done early.
>> >
>> > Hope this helps.
>> > Regards,
>> > Neil
>> >
>> >> Thank you and best regards,
>> >> Volker
>> >>
>> >> > Regards,
>> >> > Neil
>> >> >
>> >> >
>> >> > On Tue, 2013-02-05 at 11:13 +0100, Volker Simonis wrote:
>> >> >> Hi,
>> >> >>
>> >> >> we have internally discussed the best way for syncing up with the
>> >> >> upstream jdk7u-dev repository forest. Actually there are two
>> >> >> possibilities:
>> >> >>
>> >> >> 1. "pull -u" and merge
>> >> >> pros: simple and fast
>> >> >> cons: one big merge changeset which may hide changes which have
>> >> >> previously been intentionally and carefully split into different
>> >> >> changes.
>> >> >>
>> >> >> 2. create a new forest (as clone of jdk7u-dev) and transplant our
>> >> >> changes from 7u into the new 7u-dev
>> >> >> pros: clean new repo with the chance to manually merge related
>> >> >> changesets from our current repository
>> >> >> cons: a little more time consuming than 1
>> >> >>
>> >> >> We are very much in favour of alternative 2 because we would like to
>> >> >> combine some of our changes (especially from the HotSpot repository).
>> >> >> We initially already tried to bring in self contained, meanigful
>> >> >> changes but in reality they had often been followed by small bug fix
>> >> >> changes. With the new jdk7u-dev forest we would get the opportunity to
>> >> >> clean this up. This would also be an intermediate step towards the
>> >> >> ultimate goal to provide a set of clean, reviewable changes for jdk8.
>> >> >>
>> >> >> We would like to start with this work within a week or two. Once we
>> >> >> had the new jdk7u-dev forest in our porting repository we would
>> >> >> sequentially bring over all changes from the current 7u forest. We
>> >> >> expect this process will not take more than about two weeks.
>> >> >> Afterwards, the 7u forest will be closed and all further work will be
>> >> >> done in the 7u-dev forest (i.e. nightly build and tests will use this
>> >> >> new forest). The plan is to synchronize this new forest more regularly
>> >> >> with the upstream 7u-dev repository.
>> >> >>
>> >> >> This would give us a current jdk 7 version (i.e. something which
>> >> >> should be very close to the upcoming 7u14 release) and a good starting
>> >> >> point for the next step which would be the integration into jdk8.
>> >> >> Especially we don't expect that the HotSpot repository in 7u-dev will
>> >> >> need much attention in the future any more because it seems the the
>> >> >> currently integrated HotSpot Express 24 (hsx24) is the last new
>> >> >> HotSpot version which has been backported into the Java 7 releases.
>> >> >>
>> >> >> Please let us know if you are comfortable with this approach.
>> >> >>
>> >> >> Regards,
>> >> >> Volker
>> >> >
>> >> >
>> >> > --
>> >> > Unless stated above:
>> >> > IBM email: neil_richards at uk.ibm.com
>> >> > IBM United Kingdom Limited - Registered in England and Wales with number 741598.
>> >> > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>> >> >
>> >
>> >
>> > --
>> > Unless stated above:
>> > IBM email: neil_richards at uk.ibm.com
>> > IBM United Kingdom Limited - Registered in England and Wales with number 741598.
>> > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>> >
>
>
> --
> Unless stated above:
> IBM email: neil_richards at uk.ibm.com
> IBM United Kingdom Limited - Registered in England and Wales with number 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>
More information about the ppc-aix-port-dev
mailing list