JDK 7 Repository updates
Neil Richards
neil.richards at ngmr.net
Fri Feb 22 06:51:40 PST 2013
Hi Volker, Goetz,
Sorry for the delay in getting back to you on this.
For the jdk code, I'd like to go with the sync & merge approach
(simplicity wins!).
(I'm currently looking through the merge conflicts when merging the jdk
code up with the latest from jdk7u, jdk7u14-b13.)
If you find that the hotspot code updated to hs24 works with the current
jdk code, I'm happy for you to submit the hs changes, and I'll look to
follow suit with the jdk changes as soon as I've completed the jdk
merge.
Regards,
Neil
On Fri, 2013-02-22 at 11:11 +0100, Lindenmaier, Goetz wrote:
> Hi,
>
> actually, our changes apply nicely to hs24, so I'm
> ready to submit a first part already, and I'd like to do so.
>
> So we need a decision how to proceed.
>
> As I found out, hs24 works with the jdk we currently build.
> So maybe we could just go ahead and submit the hs changes,
> and then pull the jdk to the current state of jdk7u.
>
> Best regards,
> Goetz.
>
>
>
> -----Original Message-----
> From: ppc-aix-port-dev-bounces at openjdk.java.net [mailto:ppc-aix-port-dev-bounces at openjdk.java.net] On Behalf Of Volker Simonis
> Sent: Mittwoch, 20. Februar 2013 18:36
> To: Neil Richards
> Cc: ppc-aix-port-dev at openjdk.java.net
> Subject: Re: JDK 7 Repository updates
>
> Hi all,
>
> we've started working on hs24 (i.e. the newest HotSpot version from
> 7u). After the discussions in this thread we've decided to use the
> Mercurial Queue extension ('mq') to simplify this and future
> synchronisations of the 'hotspot' repository with new upstream
> versions. We will convert and consolidate our changes into a Mercurial
> patch queue. We will then synchronise from upstream (i.e. pull from
> 7u), do a "no-op merge" to eliminate our changes and finally reapply
> the changes from our patch queue. Future synchronisations from
> upstream will be handled similarly.
>
> We hope we will be ready to checkin the new HotSpot version by the end
> of next week. We promise to be careful but it may happen that the
> ppc-aix-port/jdk7u will become unstable for a few days after the
> update.
>
> We wanted to ask you (i.e. *@ibm) if you want to take a similar
> approach for the class library changes? Actually, the jdk repository
> has been synced one time (in mid January) with the latest changes from
> jdk7u6 (without the need for manual resolving). This was necessary to
> get the certification finished.
>
> If you don't plan to prepare a patch queue we will just sync and merge
> the jdk repository after we've finished the hotspot one (notice that
> his is necessary this time because of the new JSR292 implementation in
> hs24 which has dependencies on the class library).
>
> I you want to create a patch queue yourself (or consolidate and
> reapply the porting changes in some other way) please let us know so
> we can synchronise (because of the mentioned dependencies).
>
> Regards,
> Volker
>
>
> On Thu, Feb 7, 2013 at 3:46 PM, Neil Richards <neil.richards at ngmr.net> wrote:
> > Hi Volker,
> > Please find some further comments inline below:
> >
> > On Wed, 2013-02-06 at 19:31 +0100, Volker Simonis wrote:
> >> Hi Neil,
> >>
> >> thanks a lot for your detailed mail.
> >> I'm really always happy to learn some new and cool Mercurial tricks:)
> >>
> >> Please find my comments inline:
> >>
> >> On Wed, Feb 6, 2013 at 4:05 PM, Neil Richards <neil.richards at ngmr.net> wrote:
> >> > Hi Volker,
> >> > I'm a little intrigued that you want to sync up to jdk7u-dev (i.e. the
> >> > component forest), rather than jdk7u (the master).
> >> >
> >> > (Of course, I may just be being pedantic here, as the contents of these
> >> > two forests tend to be similar - there's just a greater theoretical risk
> >> > of abnormal perturbation by pulling fresh changes from the component
> >> > forest, rather than pulling changes once they've been promoted to the
> >> > master forest.)
> >> >
> >>
> >> Sorry, that was just my looseness that I confused jdk7u-dev and jdk7u.
> >> I really meant jdk7u (which in the context of my initial mail also meant
> >> that we would had to rename either the old or the newly created repository,
> >> which is not nice as well)
> >>
> >> >
> >> > Of course, the syncing operation that you're considering is not a
> >> > one-time operation - the ppc forest is only "current" after a sync until
> >> > the next change is made in jdk7u, at which point another sync is needed
> >> > to get back to being "current".
> >> >
> >> > So creating a new or replacement forest each time there's a sync doesn't
> >> > seem like a good strategy to me.
> >> >
> >> > However, I think there is a variation on your second strategy that
> >> > doesn't require the creation of a new forest that might be viable ...
> >> >
> >> > ----
> >> >
> >> > Consider this conceptual starting position, with two ppc changes (p.1 &
> >> > p.2) made against a version of code from jdk7u (o.1):
> >> >
> >> > @ p.2
> >> > |
> >> > o p.1
> >> > |
> >> > o o.1
> >> >
> >> >
> >> > (I've simplified things for the sake of discussion).
> >> >
> >> > In a local clone, new change(s) from jdk7u are pulled in, so there's a
> >> > new version of code (o.2) to be combined (in some fashion) with the ppc
> >> > changes:
> >> >
> >> >
> >> > o o.2
> >> > |
> >> > | @ p.2
> >> > | |
> >> > | o p.1
> >> > |/
> >> > o o.1
> >> >
> >> >
> >> > Work is done (via rebase and/or mq import / export) to get the ppc
> >> > changes on top of o.2:
> >> >
> >> > @ p.2'
> >> > |
> >> > o p.1'
> >> > /
> >> > o o.2
> >> > |
> >> > o o.1
> >> >
> >> >
> >> > The original p.1 & p.2 changes are still in the master ppc forest, and
> >> > can be pulled back down to the local clone:
> >> >
> >> > @ p.2'
> >> > |
> >> > o p.1'
> >> > /
> >> > o o.2
> >> > |
> >> > | o p.2
> >> > | |
> >> > | o p.1
> >> > |/
> >> > o o.1
> >> >
> >> >
> >> > The old "spur" containing the original p.1 & p.2 can now be marked as
> >> > "closed", by updating to p.2 using:
> >> > hg ci --close-branch -m "Code rebased to $(hg log -r tip
> >> > --template '{node|short}')"
> >> >
> >> > That results in a tree (with c.1 being the close branch changeset) like
> >> > this:
> >> >
> >> > x c.1
> >> > |
> >> > o | p.2'
> >> > | |
> >> > o | p.1'
> >> > / |
> >> > o | o.2
> >> > | /
> >> > | o p.2
> >> > | |
> >> > | o p.1
> >> > |/
> >> > o o.1
> >> >
> >> > Then, use mq to resequence the changes, such that the close branch
> >> > change occurs before p.1' & p.2':
> >> >
> >>
> >> If I understand this right, it would be also possible to:
> >> 1. pull
> >> 2. hg ci --close-branch
> >> 3. hg update -r 'o.2'
> >> 4. hg export p.1 && hg qimport p.1
> >> 5. hg export p.2 && hg qimport p.2
> >>
> >> This would avoid the step to 'resequence' the changes p.1' and p.2'
> >
> > It would, though at the cost of not being able to reference the p.2'
> > changeset in the commit comment for the close branch (c.1) changeset
> > (because c.1 is committed before the changeset id for p.2' is known).
> >
> > I reckon that this extra reference makes the history a little more
> > legible when reading it back later.
> > (This is particularly so if original changes have been reformed and/or
> > folded together.)
> >
> >> > @ p.2'
> >> > |
> >> > o p.1'
> >> > |
> >> > | x c.1
> >> > / |
> >> > o | o.2
> >> > | /
> >> > | o p.2
> >> > | |
> >> > | o p.1
> >> > |/
> >> > o o.1
> >> >
> >> > Then push the new changes (c.1, p.1' & p.2') up to the master in one
> >> > fell swoop.
> >> >
> >> > NB: This push will need the (usually dangerous) '-f' option, as the
> >> > receiving ends sees the number of head being increased (by one), because
> >> > it doesn't recognise that c.1 reduces that count (back down to 1).
> >> >
> >>
> >> Are you really sure that this is possible with the OpenJDK repositories.
> >> I always thought that they don't allow multi-head configurations.
> >
> > You are correct that using close-branch mechanisms is verboten, at least
> > within the "main" reference-implementation OpenJDK projects (e.g. jdk7,
> > jdk7u, jdk8, hsx).
> >
> > These projects also have other restrictions on the changes that they
> > accept, such as each (non-merge, non-tag change) being associated with a
> > unique, valid Java bug id.
> > This does not apply to *all* projects in OpenJDK - my understanding is
> > that the Project Lead determines the rules which apply to each project.
> >
> > Any policing of these restrictions at the OpenJDK mercurial location
> > will be done by check-in hooks. For the "main" OpenJDK projects, I think
> > this includes the use of 'jcheck'. I'm not sure what hooks there are for
> > ppc-aix.
> >
> > (I suppose one could try my suggested approach & see if the changes can
> > be pushed up - if they can't, then there's some commit checking that
> > prevents it, and you know a different choice of approach is needed).
> >
> >> I really like your approach but I'm afraid that's not possible with OpenJDK
> >> repositories. It is true that 'hg heads' will only show one head with your
> >> approach, but actually that's just sham. You really have two heads afterwards
> >> (you can see them with 'hg heads -c') and that's also the reason why you
> >> have to use the '-f' option for 'hg push'.
> >
> > True, the number of total heads has increased by 1 (as has the number of
> > closed heads).
> > The number of active heads remains at 1.
> >
> >> > However, the net effect after the push is that there is still one active
> >> > head, and the "tip" change in the master repository is that for the
> >> > active head.
> >> >
> >>
> >> Unfortunately I'm pretty sure that the OpenJDK repos don't allow this (and
> >> actually there is no single example of a closed branch in the hotspot or
> >> jdk repositories). I therefore think that your approach is only possible for
> >> local repositories to manage own changes with respect to a constantly
> >> moving upstream repository.
> >
> > As I say above, I think you may find that it is under the jurisdiction
> > of the ppc-aix Project Lead to choose what mercurial conventions to
> > follow (subject to any universally enforced commit hooks).
> >
> > As Project Lead, you might decide that using the same conventions as the
> > "main" OpenJDK projects makes everything simpler and clearer.
> > The convention followed by the "main" projects (in their component
> > forests) for syncing up with later changes (from their master forest) is
> > to use 'hg merge', of course, so following this course might sway you
> > back towards going with your option 1).
> >
> >
> >> > ----
> >> >
> >> > This is how we have been handling our historic changes internally, as we
> >> > work them into shape for contribution to OpenJDK.
> >> >
> >> > As you say, there is some overhead in working with the changes in this
> >> > manner (instead of using 'hg merge'), but the benefits in doing so may
> >> > outweigh the extra cost.
> >> >
> >> > By marking the old spurs "closed" (rather than trying to eliminate their
> >> > existence from history, as moving to a new forest would do), I think
> >> > that it would more accurately reflect the nature of the actions that are
> >> > being performed.
> >> >
> >> > Any thoughts / comments on this would be gratefully received.
> >> >
> >>
> >> I think we may do a "no-op merge" as described in:
> >> http://mercurial.selenic.com/wiki/PruningDeadBranches#No-Op_Merges
> >> (i.e. we pull from 7u, do a "no-op merge" to totally eliminate our
> >> changes and then selectively re-apply them).
> >> That's not beautiful nor elegant, but it may be the only possible solution
> >> if we want to stay with one repository (which I agree is worth while).
> >> And because we will not directly push from our porting repositories into
> >> the main jdk repos the intermediate "no-op merges" won't bother anybody.
> >
> > This would be anther viable approach, and would effectively eliminate
> > the need to refer to the p.2' changeset id in the commit comment of c.1
> > (as the parent changests of the no-op merge changeset would effectively
> > record that association).
> >
> > These no-op branches would cause the dead branches to be listed as part
> > of the history of the latest changes.
> >
> > Perhaps if the dead branches were marked as closed and then no-op
> > merged, dead branch history could be identified and pruned out of any
> > listing, using 'hg log -P', though the revset incantation to achieve
> > this would be fairly complex, I suspect.
> >
> > (Whichever approach is taken, I agree that it doesn't affect what
> > happens when changes are adopted into the main jdk forest(s)).
> >
> >> Do you see other ways of doing this?
> >
> > No, I think we've those three techniques are all that I can think of.
> >
> > One other point, regarding the combining of changesets into "meaningful"
> > changes ...
> >
> > The natural temptation when doing this is to tend to combine more
> > changes together, to end up with fewer, larger changes.
> >
> > In general, it is good to resist giving in to this temptation, as
> > smaller, more fine-grained changes tend to be easier (for everyone) to
> > consider during activities like promotion to the "mainline".
> >
> > It's also a lot easier in mercurial to combine small changes together
> > "on-demand" with 'hg qfold' than it is to split them apart again.
> > So any such "folding" can be fairly safely deferred until close to
> > mainline "promotion", whilst there is benefit for any suitable
> > "splitting" to be done early.
> >
> > Hope this helps.
> > Regards,
> > Neil
> >
> >> Thank you and best regards,
> >> Volker
> >>
> >> > Regards,
> >> > Neil
> >> >
> >> >
> >> > On Tue, 2013-02-05 at 11:13 +0100, Volker Simonis wrote:
> >> >> Hi,
> >> >>
> >> >> we have internally discussed the best way for syncing up with the
> >> >> upstream jdk7u-dev repository forest. Actually there are two
> >> >> possibilities:
> >> >>
> >> >> 1. "pull -u" and merge
> >> >> pros: simple and fast
> >> >> cons: one big merge changeset which may hide changes which have
> >> >> previously been intentionally and carefully split into different
> >> >> changes.
> >> >>
> >> >> 2. create a new forest (as clone of jdk7u-dev) and transplant our
> >> >> changes from 7u into the new 7u-dev
> >> >> pros: clean new repo with the chance to manually merge related
> >> >> changesets from our current repository
> >> >> cons: a little more time consuming than 1
> >> >>
> >> >> We are very much in favour of alternative 2 because we would like to
> >> >> combine some of our changes (especially from the HotSpot repository).
> >> >> We initially already tried to bring in self contained, meanigful
> >> >> changes but in reality they had often been followed by small bug fix
> >> >> changes. With the new jdk7u-dev forest we would get the opportunity to
> >> >> clean this up. This would also be an intermediate step towards the
> >> >> ultimate goal to provide a set of clean, reviewable changes for jdk8.
> >> >>
> >> >> We would like to start with this work within a week or two. Once we
> >> >> had the new jdk7u-dev forest in our porting repository we would
> >> >> sequentially bring over all changes from the current 7u forest. We
> >> >> expect this process will not take more than about two weeks.
> >> >> Afterwards, the 7u forest will be closed and all further work will be
> >> >> done in the 7u-dev forest (i.e. nightly build and tests will use this
> >> >> new forest). The plan is to synchronize this new forest more regularly
> >> >> with the upstream 7u-dev repository.
> >> >>
> >> >> This would give us a current jdk 7 version (i.e. something which
> >> >> should be very close to the upcoming 7u14 release) and a good starting
> >> >> point for the next step which would be the integration into jdk8.
> >> >> Especially we don't expect that the HotSpot repository in 7u-dev will
> >> >> need much attention in the future any more because it seems the the
> >> >> currently integrated HotSpot Express 24 (hsx24) is the last new
> >> >> HotSpot version which has been backported into the Java 7 releases.
> >> >>
> >> >> Please let us know if you are comfortable with this approach.
> >> >>
> >> >> Regards,
> >> >> Volker
> >> >
> >> >
> >> > --
> >> > Unless stated above:
> >> > IBM email: neil_richards at uk.ibm.com
> >> > IBM United Kingdom Limited - Registered in England and Wales with number 741598.
> >> > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
> >> >
> >
> >
> > --
> > Unless stated above:
> > IBM email: neil_richards at uk.ibm.com
> > IBM United Kingdom Limited - Registered in England and Wales with number 741598.
> > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
> >
--
Unless stated above:
IBM email: neil_richards at uk.ibm.com
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
More information about the ppc-aix-port-dev
mailing list