[gnu.andrew at redhat.com: Re: OpenJDK 8u and Backport Bugs]

[Severin Gehwolf]

On Tue, 2020-12-15 at 10:20 +0000, Langer, Christoph wrote:
> Hi Severin,
> 
> > > sorry for being late on this discussion, I just now found the time to
> > > go through the mail thread on 8u-dev in detail. But, hold on, why are
> > > you proposing this change to the process? What issues are you aiming
> > > to solve?
> > 
> > Note that we've largely discussed this in the context of 8u. But in
> > general we'd like to have a process that works for most (all?) updates
> > releases.
> 
> I wasn't aware of this discussion - I just saw the mails from last Friday.
> 
> > The main goal is for us to have a way to distribute backporting work.
> > The intention is to solve a couple of problems with it
> > 
> > - Allow for proper reporting. Time a backport bug gets opened until
> >   it is resolved kind of metrics. Using labels for this isn't working.
> > - Assign specific backport issues to a certain user. This is a more
> >   formal approach than labels. It has the advantage of getting email
> >   notifications when certain events happen. Like a backport bug is
> >   created and assigned to someone. Somebody other than the backporter
> >   can assign bugs. The assignee would notice this by getting email
> >   notifications.
> > - For CSR-needing bugs we need to do the explicit backport bug
> >   anyway. It would no longer be an exception to the rule.
> > - Commenting on the master bug that somebody is doing the backport
> >   isn't really machine readable. Getting a report quickly as to who
> >   is doing the backport is difficult, especially figuring this out
> >   for different releases.
> > 
> > I understand this also has the drawback of an additional step (creating
> > the explicit bug), but we haven't found an alternative solution to
> > cover all the cases. If you've got suggestions, I'd be all ears.
> 
> I can see your points.
> 
> > > So far, and it has worked very well at least in 11u, the process was
> > > to add fix request labels in the parent/master bug that is to be
> > > backported. Also, when somebody wants/needs to claim that they are
> > > working on a backport, they would claim it in a comment on the master
> > > bug. Eventually, this comment could also be modified to be the actual
> > > "Fix Request" comment with more details for the approver.
> > 
> > The theory is that it works well when only few people do the work. It
> > has a scalability problem. This process is rather implicit and relies
> > on the process knowledge of the backporter. We feel the explicit bug is
> > easier to understand for new contributors.
> 
> I think there's always some process to be understood when one wants
> to contribute to OpenJDK updates maintenance. I at least can't see
> any major improvement when I have to check linked backports vs.
> comments in an issue.

Agreed. There shouldn't be a need to check both: The master bug *and*
the backport bug. Keeping relevant things for approval and decision
making in the master bug makes sense.

Right now the only difference would be to create an explicit backport
bug for tracking/assigning people.

> > 
> > > For most cases, there is no need to create a backport item in advance
> > > as hgupdater will create it at the moment the backport is pushed.
> > > Exceptions are when e.g. CSRs have to be done. So, after all, we
> > > should not encourage to create backport items in advance. This can as
> > > well lead to orphaned items, e.g. when the person doing the backport
> > > then steps away and forgets about it or when the target release is
> > > labeled wrong and hgupdater creates another bug, leaving the intended
> > > backport alone.
> > 
> > Makes sense. I wonder though, how it's different to the current
> > process. One could go comment on the bug "I'm doing a backport of this
> > for OpenJDK X" and then step away. Note that the explicit backport bug
> > would remain unresolved in that case. Again, it would show up in the
> > reporting and would get flagged.
> > 
> > > I know about the issues with 8-pool and 11-pool. For 8-pool,
> > > resolving OpenJDK backport releases doesn't work at all. And for
> > > 11-
> > > pool there's a danger that hgupdater will seize a backport that
> > > is
> > > intended for OpenJDK 11u backports when Oracle does a backport at
> > > the
> > > same time. Both can be circumvented by explicitly specifying the
> > > intended target update release, such as "openjdk8u291" or
> > > "11.0.11".
> > > In that case, however, there's a certain danger if the backporter
> > > misses a release and pushes the backport to a different release
> > > than
> > > set at the backport. Then a fresh backport issue would be created
> > > and
> > > the already existing one gets orphaned. So, both approaches have
> > > potential issues and need specific attention by the committers.
> > > Hence
> > > the safest way is to not open a backport bug at all and let
> > > hgupdater
> > > do the work whenever possible.
> > 
> > We've been testing this in 8u with setting the explicit bug to "Fix
> > Version" -> openjdk8u292 (current jdk8u-dev). Then the bug would get
> > properly resolved as you say. What we thought we'd do to avoid mistakes
> > is for the maintainer to set the version correctly when the bug gets
> > approved for push. So it wouldn't be the backporters responsibility.
> > For 11u it's a non-issue as 11-pool works fine. It would sometimes need
> > coordination with Oracle, but it would be an OK compromise IMO.
> 
> I think actually 11-pool is a major issue here since there will be
> conflicts with Oracle. Especially for backport items that are open
> for a long time as once can never know if and when Oracle does
> backports and whether their engineer will also check JBS at the time
> of pushing. So, if we manually create the backport items, we should
> then set the intended target version explicitly. In fact, this seems
> to be what Oracle does as well. When they are working on a backport,
> they also sometimes create backport items with a certain target
> version. This would make life easier for the maintainers as well as
> they will only have to check whether the version is set correctly. 

OK. So is the concern that by creating explicit backport bugs, setting
Fix Version to "11-pool" makes it indistinguishable from a bug created
by Oracle for their work? Setting the fix version to something like
11.0.11 (current jdk11u-dev) seems fine to me if that alleviates the
concern.

My understanding was that 11-pool bugs would be OpenJDK 11 bugs.
Perhaps that's not the case.

> 
> > > As for the issue of copying all the labels to backport bugs,
> > > especially those fix request/approval labels that will flood the
> > > open/unpushed backport lists, that is a known issue. The Skara
> > > tooling has addressed this already ([0], [1]), so for 13u this isn't
> > > an issue any longer. And for 11u this will be solved once we go to
> > > Skara (Another reason for not waiting too long until migrating to
> > > Skara with 11u ��).
> > 
> > Right. I'm not convinced Skara will be a solution for 8u, though. So
> > there is that problem.

Perhaps we shouldn't rule out Skara :) It might well be part of the
solution. I should keep an open mind about that. Mea culpa.

> But 8u is only at the end of the chain... If no release higher than 8
> generates this bug pollution, I think 8 won't suffer.

I'm not sure how skara would address the bug-distribution/assignment
issue, though. Do you know of something that would handle it?

> > 
Thanks,
Severin