Proposal: use backports for working with release stabilization repositories

Mon May 15 15:22:54 UTC 2023

On 5/15/23 9:14 AM, Erik Duveblad wrote:
>> On 21 Apr 2023, at 22:44, Daniel Daugherty <daniel.daugherty at oracle.com> wrote:
>>
>> Greetings,
> Hey Dan,
>
> first of all, sorry for the very late reply, I somehow missed your e-mail :(

No worries.

>
>> On 21 Apr 2023, at 22:44, Daniel Daugherty <daniel.daugherty at oracle.com> wrote:
>> Erik this is a very well thought out proposal!
>>
> Thanks!
>
>> On 21 Apr 2023, at 22:44, Daniel Daugherty <daniel.daugherty at oracle.com> wrote:
>> As one of the other gatekeepers that sometimes has to do the forward port
>> work, I'd like to chime in that I'm in favor of this approach. I agree that
>> for time critical fixes, we can work both PRs in parallel since that what
>> we would do now even with the current process. Think bad failure in code that
>> starts being tested in Tier3 or Tier4 and we don't discover it until after
>> the stabilization repo has already been forward ported to the main repo...
>>
>> The only piece that I have reservations about is this part:
>>
>>> - Q: What happens with tags created in the release stabilization repository?
>>> - A: They will be pushed to the main-line repository by the Skara bots. Note that
>>>       only the tags, and the commits they refer to, will be pushed to the main-line
>>>       repository. The commits referred to by the tags will *not* be merged into the
>>>       "master" branch in the main-line repository.
>> I don't understand why we would put tags from the stabilization repo that are
>> non-functional outside of that context in the main repo. In the current process,
>> those tags are copied to the main repo because we forward ported the changesets
>> from the stabilization repo to the main repo.
>>
>> I think copying the tags from the stabilization repo will just lead to questions
>> and confusion about whether changesets are missing (or not)...
> I think you might be confusing how Mercurial tags works here with how Git tags work. Git tags are just references to commits, much like branches in Git (except tags aren't mutable, you cannot change the commit a tag refers to in Git). With Mercurial tags are regular commits, they just update a special file (.hgtags), so a very different concept. With Mercurial that meant we had to merge in the commits into the "default" branch just to get the tags. With Git we don't have to do that, instead you will see the tags in the main-line repo but the commits referred to by the tags will *not* be on the "master" branch.
>
> Pushing the tags to the main-line repo means that a user of the main-line repo can run:
>
>      $ git switch jdk-20-ga
>
> If we didn't push the tags to the main-line repo the above would have been two commands:
>
>      $ git fetch --tags https://git.openjdk.org/jdk20
>      $ git switch jdk-20-ga
>
> For code archeology it is much more convenient to have the tags in the main-line repository, and since the commits are *not* merged into the "master" branch, I don't think people will be confused. The tags will be there when you need them and otherwise they (and the commits they refer to) will stay out of sight :)
>
> Hope this answers your question!

It does answer my question quite nicely. I presumed that just the tag would
be in the repo and not the changeset to which the tag referred. So the tags
will indeed be functional and that's a good thing from a code archeology 
POV.

It doesn't matter to me that the changesets are not merged into the "master"
branch. From my POV, the changeset is available for code archeology and 
that's
what is important (to me anyway).

Erik, thanks for closing the loop on this topic.

Dan

>
> Thanks,
> Erik
>
>> Dan
>>
>>
>> On 4/11/23 8:55 AM, Erik Duveblad wrote:
>>> Hi all,
>>>
>>> this is a proposal to change the process for how we work with commits that
>>> should be applied to _both_ the main-line repository (jdk [0]) and a release
>>> stabilization repository (e.g. jdk19 [1], jdk20 [2]).
>>>
>>> The current process we are using today consist of contributors first creating
>>> a pull request targeting the release stabilization repository (e.g. jdk20).
>>> Jesper will then periodically merge changes from the release stabilization
>>> repository to the main-line (jdk) repository. If the merge contains
>>> commits that should *only* be applied to the release stabilization repository,
>>> then those commits have to be backed out [3] from the main-line repository.
>>> The process of Jesper periodically merging changes from the release
>>> stabilization repository to the main-line repository is sometimes referred to as
>>> "forward porting". For more information on the current process, see the
>>> section on "Forward Ports" in the OpenJDK Developer's Guide [4].
>>>
>>> I propose that we change to use backports for working with the release
>>> stabilization repositories. With this new process contributors would first
>>> create a pull request targeting the main-line (jdk) repository. Once the
>>> pull request has been reviewed and integrated into the main-line repository
>>> contributors will backport the commit to the release stabilization repository
>>> (e.g. jdk20). The backport can be created in numerous ways but most
>>> contributors will likely use Skara's [5] "/backport" command [6] to create the
>>> backport. For changes that should *only* go into the release stabilization
>>> repository, contributors will create a pull request targeting the release
>>> stabilization repository. Many of you will recognize this process as being
>>> essentially the same one that is being used for backporting changes to update
>>> releases (modulo JBS labels, approvals, etc.).
>>>
>>> All the relevant Skara tooling supporting this proposal has been in place for a
>>> long time and is being used by other OpenJDK projects. With this new process we no
>>> longer have to care about backing out release stabilization specific commits
>>> from the main-line repository, there is a much lower risk of issues being introduced
>>> due to incorrect merges and the backport process has proven itself when used by
>>> other OpenJDK projects.
>>>
>>> I suggest that we adopt this proposal for JDK 21.
>>>
>>> Please see below for some common questions and answers:
>>> - Q: How can we ensure that no contributor forgets to backport a commit to the
>>>       release stabilization repository?
>>> - A: An e-mail reminder will be sent to jdk-dev [7] prior to entering the
>>>       rampdown phase two (RDP 2) [8] and release candidate (RC) [8] phases. The
>>>       e-mail will contain a link to a JBS [6] query that contributors can click to
>>>       view issues that they have fixed in the main-line repository that are
>>>       applicable to being backported.
>>>
>>>       For example, prior to entering RDP 2 for a release an e-mail will be sent
>>>       to jdk-dev with a JBS query that shows a user the P3 issues he or she
>>>       has fixed in the main-line repository since the release stabilization
>>>       repository was created that has *not* been backported to the release
>>>       stabilization repository. Every contributor can read through their own
>>>       list to verify that they haven't forgotten to backport any P3 issues
>>>       before the release enters RDP 2.
>>>
>>> - Q: What happens with tags created in the release stabilization repository?
>>> - A: They will be pushed to the main-line repository by the Skara bots. Note that
>>>       only the tags, and the commits they refer to, will be pushed to the main-line
>>>       repository. The commits referred to by the tags will *not* be merged into the
>>>       "master" branch in the main-line repository.
>>>
>>> - Q: Will "clean" backports [10] have to be re-reviewed?
>>> - A: Yes. Skara [5] has a feature where "clean" backports can be configured
>>>       (per repository) to be allowed to be integrated without a review. This is
>>>       feature is utilized by some projects in OpenJDK. For the JDK project and for
>>>       release stabilization repositories this feature will be turned off, meaning
>>>       that all backports will require reviews (regardless whether they are
>>>       "clean" or not).
>>>
>>> - Q: Can someone other than the original contributor do the backport?
>>> - A: Yes, this is fully supported by Skara. This often happens when an developer
>>>       sponsors a bugfix from a newer contributor who is not yet a committer, and
>>>       that bugfix should be backported to an older release. In that case the
>>>       sponsor is often the one doing the backport.
>>>
>>> If this proposal is accepted then information about this process will be added
>>> to the OpenJDK Developer's Guide [11] and to JEP 3 [8]. Please let me know if you have
>>> any feedback, comments or questions!
>>>
>>> Thanks,
>>> Erik
>>>
>>> [0]:  https://git.openjdk.org/jdk
>>> [1]:  https://git.openjdk.org/jdk19
>>> [2]:  https://git.openjdk.org/jdk20
>>> [3]:  https://openjdk.org/guide/#backing-out-a-change
>>> [4]:  https://openjdk.org/guide/#forward-ports
>>> [5]:  https://openjdk.org/projects/skara/
>>> [6]:  https://wiki.openjdk.org/display/SKARA/Pull+Request+Commands#PullRequestCommands-/backport
>>> [7]:  jdk-dev at openjdk.org
>>> [8]:  https://openjdk.org/jeps/3
>>> [9]:  https://bugs.openjdk.org
>>> [10]: https://wiki.openjdk.org/display/SKARA/Backports#Backports-Terminology
>>> [11]: https://openjdk.org/guide
>>>