JDK 11.0.3 Update process

Wed Feb 13 18:30:22 UTC 2019

On Wed, Feb 13, 2019 at 6:19 PM Andrew Haley <aph at redhat.com> wrote:
>
> On 2/12/19 7:11 PM, Volker Simonis wrote:
> > Hi all,
> >
> > I think this are very valid questions for which we have to find an
> > answer pretty soon. Please find my further comments inline:
> >
> > On Mon, Feb 11, 2019 at 8:11 PM Langer, Christoph
> > <christoph.langer at sap.com> wrote:
> >>
> >> while we know that the jdk11 updates project is in the phase of
> >> transitioning to its new maintainers and this will take a little
> >> while to settle down, the next quarterly update JDK 11.0.3,
> >> scheduled for April, is in the process of ramping up.
> >>
> >> When discussing the JDK update process today in our team at SAP, we
> >> found a few questions open which we think need to be answered soon.
> >>
> >> One question is, whether changes brought into the jdk11u repo as of
> >> today will still target for 11.0.3? I assume that's the case as no
> >> communication has been made telling otherwise. Can you confirm
> >> this? Furthermore, if that's true, when will be the cutoff point?
> >> Or will we maybe pick a certain change in the jdk11u repo that is
> >> known as of today to be the base for 11.0.3?
>
> The cutoff date must be before when we branch for the CPU, and that
> will be when the patches drop into the VG. I take your point that it
> would be nice to know in advance exactly when the cutoff will be, but
> on the other hand I don't think it's necessary or desirable to cut off
> early. On the other hand, we will start working on the CPU as soon as
> we can.
>
> > I suppose you wanted to write "known to be the base for
> > 11.0.3-oracle". I don't think we should look for Oracle here. First
> > of all, we don't know which was the cutoff point at which Oracle has
> > branched its 11.0.3-oracle consolidation branch. And even if they
> > would tell us, this would only make sense for our 11.0.3-openjdk
> > release. For all future releases, Oracle will branch their
> > consolidation branches from their internal, 11u-oracle repository
> > which will be different from 11u-openjdk and to which we don't have
> > access anyway.
>
> Yes, indeed so.
>
> >> Furthermore, can we assume a new open consolidation repository will
> >> be created on hg.openjdk.java.net where update release ramp up for
> >> OpenJDK 11 updates will take place?
> >
> > We've discussed this at SAP and we would definitely vote for two
> > repos: a development AND a consolidation repository (i.e.
> > "jdk-updates/jdk11u-dev" as "always-open" for approved downports and
> > "jdk-updates/jdk11u" for consolidating the next update release). I've
> > used this notation in the following picture which tries to explain our
> > current proposal (please bear with me, I'm not an artist :)
>
> Yes, that sounds right. The development repo continues after the
> snapshot.
>
> > http://cr.openjdk.java.net/~simonis/webrevs/2019/jdk11u-project/11u-update-project.png
> >
> > The following explanations try to make the picture (and proposed
> > process) more understandable:
> >
> > - repositories are being drawn as a linear sequence of changes to
> > simplify the picture
>
> > - jdk/jdk is the "always-open" jdk development repo with a stream of
> > new changes ('a' to 'h')
>
> > - jdk-updates/jdk11u-dev is the "always-open" development repository
> > for 11u updates. After approval, changes from jdk/jdk can be
> > downported to jdk-updates/jdk11u-dev any time (e.g. 'a', 'c' and 'e'
> > in the picture).
>
> jdk/jdk is the head jdk development repo. Some changes, mostly bug
> fixes, will be backported to jdk-updates/jdk11u-dev.
>
> > - at a given time (usually within a week or two after the previous 11u
> > update (jdk-11.0.2 in the picture) has been released) the current head
> > of  jdk-updates/jdk11u-dev will be tagged with the '-rdp2' tag of the
> > next update release (jdk11.0.3 in the picture).
>
> > - the change with this tag will be pushed to (or pulled from)
> > jdk-updates/jdk11u. This effectively merges all the changes which have
> > been accumulated in jdk-updates/jdk11u-dev into jdk-updates/jdk11u
> > (e.g. 'a' and 'c' in the picture).
>
> Right.
>
> > - jdk-updates/jdk11u will now be used to prepare the next update
> > release (11.0.3 in this example). Only P2 (and later P1 bugs,
> > depending on the rules defined by the Maintainer) will be pushed to
> > jdk-updates/jdk11u. These changes can either come from jdk/jdk or they
> > can be specific to jdk-updates/jdk11u like 'y' and 'z' in the picture.
>
> It depends if the bug exists in later releases. If it does, it must be
> fixed there too in order to be accepted by 11u.
>
> > - in any case jdk-updates/jdk11u will be regularly (e.g. weekly,
> > depending on the Maintainers decision) merged back into
> > jdk-updates/jdk11u-dev. This saves committers from having to push the
> > same change into both, jdk-updates/jdk11u and jdk-updates/jdk11u-dev,
> > and works similar to the automated push from a new feature release
> > repo back into jdk/jdk. In the picture these are the changes 'y' and
> > 'z' which get pushed to (or pulled from) jdk-updates/jdk11u-dev and
> > merged into jdk-updates/jdk11u-dev with merge change 'm3'.
>
> Again, that seems reasonable.
>
> > - the jdk11u Maintainer needs to keep an additional, "closed"
> > repository, where he can integrate and test the closed security fixes
> > for the upcoming update release which he receives through the OpenJDK
> > Vulnerability Groups mailing list. In the picture this is called
> > jdk-updates/jdk11u-sec.
>
> Yes.
>
> > - jdk-updates/jdk11u-sec starts by merging in the corresponding
> > 'jdk-...-rdp2' change from jdk-updates/jdk11u (merge changeset 'm2' in
> > the picture).
>
> > - after that, jdk-updates/jdk11u-sec is ready for taking security
> > fixes (e.g. 'p' in the picture)
>
> > - at the same time jdk-updates/jdk11u-sec will be regularly synced
> > with jdk-updates/jdk11u
>
> Maybe. I'm not promising anything about that.

I meant (and probably better had written) "jdk-updates/jdk11u-sec
regularly pulls from from jdk-updates/jdk11u". How else would you be
able to test and prepare the upcoming release, if you don't merge the
non-security AND the security fixes into one repository?

>
> > by pulling from there (ideally this can happen
> > at the same time at which jdk-updates/jdk11u and
> > jdk-updates/jdk11u-dev are synced to keep things simple). In the
> > picture this happens with merge change 'm5' which pulls 'y' and 'z'
> > from jdk-updates/jdk11u.
>
> > - new security fixes can be pushed to jdk-updates/jdk11u-sec at any
> > time by the Maintainer (e.g. 'q' in the picture).
>
> > - right before the patch day, the head revision of
> > jdk-updates/jdk11u-sec will be tagged with the corresponding '-ga' tag
> > ('jdk-11.0.3-ga' in the picture).
>
> > - on the patch day, the Maintainer can release its binary releases
> > based on the '-ga' tag and at the same time push that tag back to
> > jdk-updates/jdk11u (merge 'm6' in the picture) and
> > jdk-updates/jdk11u-dev (merge 'm7' in the picture). This will give all
> > the other down-stream distributors and packager the possibility to
> > build their binary packages based on the same '-ga' tagged changeset.
>
> This sounds reasonable.
>
> > - at this point in time, jdk-updates/jdk11u and jdk-updates/jdk11u-sec
> > will be the same, while jdk-updates/jdk11u-dev will have all the
> > changes from jdk-updates/jdk11u-sec (or jdk-updates/jdk11u) PLUS the
> > additional downports which have been collected in
> > jdk-updates/jdk11u-dev since the last RDP2 change was tagged (in the
> > picture these additional changes are 'e' and 'g'.
>
> > - shortly after the update has been released, we're ready to tag the
> > head of jdk-updates/jdk11u-dev with the next '-rdp2' tag and the same
> > procedure begins again for the next update release.
> >
> > Why do we think it is important to have two update repositories (i.e.
> > jdk-updates/jdk11u-dev AND jdk-updates/jdk11u) ?
>
> > - having both repos gives other down-stream distributors the
> > possibility to test (and potentially fix) all of the non-security
> > changes which will be included in the next update release. This is
> > important because the 11u Maintainer will probably not have the
> > possibility to test on all the supported platforms and all the
> > different configurations on his own
>
> > - it will also make it transparent to the community which non-security
> > changes will be integrated into the next update release.
>
> There are potentially some issues here. We may have to integrate
> non-security fixes into the hidden CPU tree, and we may have to do
> that in a way that people outside the VG cannot see. So yes, that's
> highly desirable, but no promises.
>

Yes, but this should be only for the fairly rare cases, where a
security fix has some dependency on a non-security change and pulling
that non-security change in via jdk-updates/jdk11u would disclose
something about the upcoming security change. Otherwise, I still think
it would be advisable to do such changes in jdk-updates/jdk11u and
regularly merge them into jdk-updates/jdk11u-sec.

Sorry if my insistence in doing as much as possible of the
non-security work in in jdk-updates/jdk11u (and pulling it from there
into jdk-updates/jdk11u-sec) seems stubborn. I understand that it may
produce a little extra-overhead on the side of the
jdk-updates/jdk11u-sec maintainers. On the other side, it makes the
live for all the other down-stream consumers easier, leads to better
test coverage for the update release and keeps the VG mailing list
focused on vulnerability issues. Otherwise, it would be necessary to
also communicate and discuss the non-security fixes done in the
"closed" jdk-updates/jdk11u-sec over the VG mailing list.

> > - having both these repos gives other down-stream distributors, who
> > are also members of the OpenJDK Vulnerability Group, a chance to test
> > the non-security PLUS the security fixes together (which should be
> > pretty close to the version Maintainer is preparing and testing) on
> > all their platforms and for all their configurations.
> >
> > Please let me know what you think about this proposal. I'm obviously
> > mostly interested in the opinion of the new, designated jdk11u
> > Maintener, but I think this is an important topic for all potential
> > jdk11u downstream distributors.
> >
> > I would also be happy to add this write-up and picture (or a
> > consolidated version thereof) to the JDK11u Wiki at
> > https://wiki.openjdk.java.net/display/JDKUpdates/JDK11u if somebody
> > thinks that it my be useful.
>
> This doesn't sound significantly different from what we already do for
> jdk7. We must have two repositories, one for development and one
> stabilization branch. We must also have an internal repository for
> security updates.
>
> We need flexibility to do what we need to do, to begin with, and then
> we'll see how things settle out.
>
> I take it that you, or some of your people, will be happy to do some
> of the work you've described above, at least the open part?
>

Yes, sure - why not :)

> --
> Andrew Haley
> Java Platform Lead Engineer
> Red Hat UK Ltd. <https://www.redhat.com>
> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671