Preparation of update releases

Wed Oct 24 07:15:01 UTC 2018

On Wed, Oct 24, 2018 at 8:55 AM Rob McKenna <rob.mckenna at oracle.com> wrote:
>
>
>
> > On 23 Oct 2018, at 19:15, Rob McKenna <rob.mckenna at oracle.com> wrote:
> >
> >> On 23/10/18 19:41, Volker Simonis wrote:
> >>> On Tue, Oct 23, 2018 at 6:23 PM Rob McKenna <rob.mckenna at oracle.com> wrote:
> >>>
> >>>> On 23/10/18 17:45, Volker Simonis wrote:
> >>>>> On Fri, Oct 19, 2018 at 6:19 PM Rob McKenna <rob.mckenna at oracle.com> wrote:
> >>>>>
> >>>>> 7 too many!
> >>>>>
> >>>>> So, with the new bar for approvals in the jdk-updates project, this
> >>>>> should become less of an issue. Which is to say, a bug that can be
> >>>>> pushed via the open repo should be.
> >>>>>
> >>>>
> >>>> Yes, but this doesn't solve the problem of duplicate changes and the
> >>>> inability of the community to follow which non-security changes will
> >>>> be in the next security update if such a changes get manually
> >>>> "cherry-picked" from the open updates repo into the closed security
> >>>> updates repository.
> >>>
> >>> Prior to RDP2 all open contributions will end up in the corresponding
> >>> update release.
> >>>
> >>
> >> What do you mean here? Prior to RDP2 of 11.0.1 all changes to
> >> jdk/jdk11 and jdk-updates/jdk11u will end up in jdk-11.0.1 and later?
> >> When exactly was RDP2 of 11.0.1?
> >
> > jdk/jdk11 is the JDK11 GA feature release. All changes in this repo
> > should be in JDK 11.
> >
> > jdk-updates/jdk11u is the updates-release repo. Any changes pushed here
> > prior to the RDP2 date of an update release will make it into that
> > update release.
> >
> > The RDP2 date for 11.0.1 was in late July. (you'll note this was prior
> > to the creation of the openjdk jdk11u repo, which was delayed while we
> > figured a few things out with the release model - hopefully that delay
> > will be avoided in future)
> >
> >
> >>
> >>> Kevin gave a jql query below which should help though, is that
> >>> insufficient?
> >>>
> >>
> >> I get more and more comfortable with it :)
> >>
> >> The problem is that I still see a lot of changes in 11.0.1 (e.g. for
> >> "8204667: Resources not freed on exception") which have not been
> >> communicated on the vuln-dev mailing list. At the same time, I can't
> >> look at their JBS issues. So I wonder if they are considered "security
> >> fixes" but not communicated on vuln-dev (which would be bad) or if
> >> their JBS issues are just not visible. But in the latter case (i.e. if
> >> they are not security fixes) shouldn't they appear on Kevin's query?
> >> "8204667" for example isn't visible there.
> >
> > This does appear on Kevin's list, but I don't believe it needs to be
> > confidential.
>
> Sorry, it doesn’t appear because it’s confidential presumably. I’m not sure we can avoid this situation completely. (there may be non-vuln fixes that depend on a vuln fix for example)

But that's exactly the problem! I'll bring this up on vuln-dev as well
(which is a closed list unfortunately, so I can't CC it here). If such
changes can't be opened but at the same time they aren't
communicated/distributed on the vuln-dev list, there's no chance for
down-stream builders/packagers to prepare and test a security release
which is equivalent to the up-stream one. And this is true even if the
down-stream builders/packagers are members of the vulnerability group.

Any idea how this problem could be solved?

Thanks,
Volker

>
>     -Rob
>
> >
> >    -Rob
> >
> >
> >>
> >>>    -Rob
> >>>
> >>>>
> >>>> That's why I was suggesting the creation of an "open" clone for every
> >>>> security update repository in order to transparently collect the
> >>>> non-security changes there. This still doesn't solve the issue with
> >>>> "duplicate" changes after merging the security-repo back into the main
> >>>> updates repo, but it makes it clear for everybody which non-security
> >>>> changes will be in an security update.
> >>>>
> >>>>>    -Rob
> >>>>>
> >>>>>> On 19/10/18 07:43, Kevin Rushforth wrote:
> >>>>>> Correction, there are 7 new public fixes in 11.0.1.
> >>>>>>
> >>>>>> -- Kevin
> >>>>>>
> >>>>>>
> >>>>>>> On 10/19/2018 7:35 AM, Kevin Rushforth wrote:
> >>>>>>> I'm sure Rob will respond to the rest of this, but here is a correct
> >>>>>>> filter to find all bugs that are newly fixed in 11.0.1:
> >>>>>>>
> >>>>>>> https://bugs.openjdk.java.net/issues/?jql=project%20%3D%20JDK%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%2011.0.1%20AND%20(labels%20is%20EMPTY%20OR%20labels%20not%20in%20(hgupdate-sync))
> >>>>>>>
> >>>>>>>
> >>>>>>> There are a only 8 new public fixes that went into 11.0.1. The
> >>>>>>> "hgupdate-sync" label indicates a fix synced in from an earlier release
> >>>>>>> in the same release family, so excluding those will give you an accurate
> >>>>>>> picture.
> >>>>>>>
> >>>>>>> -- Kevin
> >>>>>>>
> >>>>>>>
> >>>>>>>> On 10/19/2018 7:07 AM, Volker Simonis wrote:
> >>>>>>>> Hi,
> >>>>>>>>
> >>>>>>>> after 11.0.1 has been successfully released I'd like to describe some
> >>>>>>>> of my observations on how this release has been prepared and suggest
> >>>>>>>> some improvements to the process:
> >>>>>>>>
> >>>>>>>> - I first, naively expected that 11.0.1 will only contain security
> >>>>>>>> fixes (i.e. the fixes circulated and discussed on the vuln-dev mailing
> >>>>>>>> list)
> >>>>>>>> - in the end I was a little surprised that in addition to the ~20
> >>>>>>>> security fixes 11.0.1 also contained ~130 other changes
> >>>>>>>> - so in the end 11.0.1 is not strictly speaking a "security release"
> >>>>>>>> but more a kind of combined "security" and "maintenance" release.
> >>>>>>>> - because 11.0.1 was prepared in a hidden clone inside Oracle, it is
> >>>>>>>> hard for others to understand which of the changes in jdk11u will also
> >>>>>>>> be integrated into 11.0.1. (I know I can list all the issues fixed in
> >>>>>>>> 11.0.1 in JBS, but this gives me more than 1000 changes which is not
> >>>>>>>> near the additional ~130 changes which are in 11.0.1 compared to 11).
> >>>>>>>> - in the end, this makes it hard for anybody outside Oracle to prepare
> >>>>>>>> and test a security update like 11.0.1, which will be close to what
> >>>>>>>> will be finally released in the OpenJDK updates project, even if he
> >>>>>>>> has access to the required security changes (because he simply can not
> >>>>>>>> know which other changes Oracle integrates into the corresponding
> >>>>>>>> security update).
> >>>>>>>>
> >>>>>>>> I would therefore like to propose the following procedure for the
> >>>>>>>> creation of future security updates:
> >>>>>>>>
> >>>>>>>> - for every security update, create a corresponding "xxx-open" clone
> >>>>>>>> in the OpenJDK updates project. E.g. for 11.0.1 we could have created
> >>>>>>>> "jdk-updates/jdk11u-11.0.1-open"
> >>>>>>>> - the hidden repository required for the collection of security
> >>>>>>>> changes should be cloned from the corresponding "-open" repo
> >>>>>>>> - the hidden repository should only be used for pushing security fixes
> >>>>>>>> - all other fixes intended for that respective security release should
> >>>>>>>> be downported from the corresponding general updates repository (e.g.
> >>>>>>>> jdk11u) where they have to be downported first, into the corresponding
> >>>>>>>> "-open" repository (e.g. jdk11u-11-0.1-open).
> >>>>>>>> - the hidden repository should regularly merge in the changes from the
> >>>>>>>> corresponding "-open" repository.
> >>>>>>>>
> >>>>>>>> The benefits of this approach would be:
> >>>>>>>>  - it is transparent for everybody what non-security changes will be
> >>>>>>>> in the next release
> >>>>>>>>  - for parties having access to the security fixes it would be trivial
> >>>>>>>> to prepare and test an update release behind closed walls which will
> >>>>>>>> exactly correspond to what the final OpenJDK update will be.
> >>>>>>>>
> >>>>>>>> The only downside I can see so far is:
> >>>>>>>>  - the extra effort of creating the new "-open" clone in the updates
> >>>>>>>> project (but notice that such a clone will in general only live for
> >>>>>>>> about three month after which he can be switched to read-only mode or
> >>>>>>>> even deleted after the corresponding hidden repository was merged in
> >>>>>>>> the general updates repo.)
> >>>>>>>>
> >>>>>>>> What do you think? I'm of course especially interested in the opinions
> >>>>>>>> of Oracle and potential future Update Project maintainers.
> >>>>>>>>
> >>>>>>>> Thank you and best regards,
> >>>>>>>> Volker
> >>>>>>>
> >>>>>>
>