[External] : Re: Switch jdk8u development to Git/Skara

[Andrew Hughes]

On 06:22 Fri 01 Oct     , erik.joelsson at oracle.com wrote:
> Hello Andrew,
> 
> Thank you for your detailed reply!
> 
> On 2021-09-30 19:25, Andrew Hughes wrote:
> > Hi. Sorry for the delayed reply, but we needed some time to think
> > this over in detail.
> No worries.

You're welcome and again, sorry for taking a while to reply; the October
security update is consuming most of my time at the minute.

snip...

> >
> > 1. End of November 2021 (Rampdown of 8u322 / Start of 8u332 Development)
> >
> > * Convert 8u & 8u-dev to Mercurial mono repositories.
> > Given they should be identical at this point, it should be possible to
> > do one conversion of 8u, provide copies at both 8u and 8u-dev, and
> > then let each copy diverge.
> >
> > * Create live read-only mirrors of 8u & 8u-dev in git
> >
> > 2. End of February 2022 (Rampdown of 8u332 / Start of 8u342 Development)
> >
> > * 8u-dev development switches to Git & Skara, with the previous read-only
> > git mirror being used directly.
> > * 8u remains on Mercurial for the release of 8u332.  8u can be synced
> > to 8u-dev from the 8u git mirror.
> >
> > 3. April 2022 (Release of 8u332)
> >
> > * 8u git is used directly for promotions of 8u342.
> >
> > I think this provides the best balance of risk, while also allowing 8u
> > to be consumed from git within the next few months.

> I think this plan looks reasonable, but I will need to check internally 
> before I can give a final go ahead. Could you narrow down the potential 
> cut over windows to date ranges?

Certainly.

1. The monorepo transition and creation of live read-only git mirrors
would be after the rampdown of 8u322, which is scheduled for Friday,
November the 26th, 2021.  So I would expect the transition to begin on
the week beginning Monday, November the 29th, 2021.  One of the
reasons we chose this particular period for what is expected to be the
most involved work is that the December period is usually quiet, as
many people ramp down themselves for the holiday period. So a delay
should be less problematic than at other times.

2. The 8u-dev git repo should go live the week beginning Monday,
February 28th, 2022, following rampdown on Friday the 25th.
Technically, this should hopefully just mean making the repository
writable. The main issue is in adjusting our processes to using
git and Skara.

3. The 8u git repo should go live after the 8u332 release is pushed,
on Tuesday, April the 19th 2022.  As 8u is only used at this point
for the build promotions I'm making, a delay here doesn't
directly affect developers; it would just mean a later build promotion.

I've updated the 8u wiki with the 8u322 & 8u332 schedules as well:

https://wiki.openjdk.java.net/display/jdk8u/Main

> > The most demand I'm seeing for 8u to be in git is from downstream
> > users, who expect it to be on github with the rest, while the most
> > change is required from developers, who have to adapt to the markedly
> > different process used by Skara.  The above allows an additional cycle
> > for developers to adapt to this change, see the live git mirrors in
> > operation and perhaps experiment with development in later JDK trees.
> > Meanwhile, downstream users should be able to start working with
> > 8u in git from December.
> >
> > Please let us know if the above seems feasible.  My understanding
> > is that the majority of work will be at the end of November,
> > and the second and third changes should just be a matter of making
> > it possible to commit to the git repositories.
> 
> Yes, the consolidation is the trickiest part with the most amount of 
> unknowns that could potentially delay things. I do believe I have it 
> under control however. You can look at the current prototype here:
> 
> https://hg.openjdk.java.net/jdk8u/consol-proto/
>

Thanks. Are the commits from duke usual? The one for 8u312-b05
particularly catches my attention, as I'd expect several tags
from myself, as with b04, instead.

> > A couple of additional questions:
> >
> > 1. Would the consolidation to a monorepo take place on top of
> > the existing root repository or be an entirely new Mercurial
> > repository? I've been trying to recall from the jdk10 transition,
> > but can't remember. Either way, the change for developers should
> > just be a one-time switch to the repository and the process
> > of submitting patches remains exactly the same as it is now,
> > except everything is from the one repository.

> No, it's a new repository. In the case of the OpenJDK 8u consolidation, 
> it will be based on the existing JDK 10 consolidated repository and 
> share history up until jdk8-b120, where the history started to diverge 
> in the original repositories. The change for developers should be 
> minimal as you say. The guarantee we give is that each promotion tag is 
> equivalent between the monorepo and the forest.

That makes sense. In the worst case, I assume the original forest will
still be available for reference.

One of the reasons I suggest the rampdown for the consolidation is we
should be able to just do the process once for 8u, and then 8u-dev can
just be a duplicate that diverges afterwards.

I presume once the hg monorepos are there, we can have read-only git
mirrors which will eventually become the live repos?

> > 2. Is the consolidated repository expected to be identical
> > to a fully-checked out copy of the forest (i.e. the root
> > repository with each subrepo as a top-level subdirectory).
> > I know some files moved with jdk10, but I think we'd want
> > everything to stay the same with 8u.

> Yes, the file layout is identical. We have no plans to rearrange things 
> for 8u. You may want to do some cleanup afterwards, like removing the 
> hgforest.sh/get_source.sh scripts that are no longer useful and remove 
> the no longer used .hgtags files from old nested repo directories.

That's good to hear. Yes, having just been looking through the 11.0.13
changes, I see quite a few alterations to adapt to using git in
build instructions, testing, etc. I expect we'll want to do similar
over the 8u322 & 8u332 lifecycles.

> > 3. As mentioned in another reply, could Skara's backport
> > command be adapted to "shuffle" the paths to fit the
> > 8u layout? This may be quite tricky as currently it
> > requires both the 10u->9u conversion and the 9u->8u
> > conversion, but without this, the backport command
> > is effectively unusable and they all would have to
> > be performed manually.

> I can see this being a very useful feature, but also a non trivial 
> effort to implement. I can't say at this time if I would personally be 
> able to allocate time for this.

Ok, I was thinking it might be quite difficult, especially as it needs
to adapt over time to changes being made in the different repos.
Developers should be used to doing it themselves by now anyway ;-)

> > 4. Could Skara be adapted to enforce the jdk8u-fix-yes /
> > jdk8u-critical-yes JIRA labels so the change can't be pushed until it
> > has been approved?
> 
> This sounds like a rather simple feature in comparison. Please file an 
> enhancement request for SKARA.
>

Oh, excellent, will do.

> /Erik
> 

Thanks,
-- 
Andrew :)
Pronouns: he / him or they / them
Senior Free Java Software Engineer
OpenJDK Package Owner
Red Hat, Inc. (http://www.redhat.com)

PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net)
Fingerprint = 5132 579D D154 0ED2 3E04  C5A0 CFDA 0F9B 3596 4222