Moving to GitHub (project Skara)

Mon Sep 9 12:28:15 UTC 2019

On 9/9/19 12:38 PM, Marcus Hirt wrote:
> (Replying so that Erik, who just joined the mailing list, can easily
> join the discussion thread.)

Thanks Marcus, and thank you for inviting me to join this discussion. I 
have been following via the archives as I usually try to *not* get 
involved when projects' are discussing using Skara. Whether JMC should 
use Skara or not is the JMC project's decision and I don't want to sway 
that in any direction. That said, I'm happy to answer questions and/or 
clarify any concerns.

One thing I immediately realized following the discussion on jmc-dev is 
that I need to describe various workflows in much more detail on the 
Skara wiki [0].

> On Fri, Sep 6, 2019 at 3:29 PM Mario Torre <neugens at redhat.com> wrote:
>>
>> On 06/09/2019 14:56, Marcus Hirt wrote:
>>> Hi Mario,
>>>
>>> The normal way to do this is to have one fork that you work in, for
>>> all your PRs. Branching is a very quick operation, and anyone involved
>>> in open source projects on GitHub will already know this workflow
>>> intimately.
>>
>> Branching may be quick, but maintaing a fork isn't simple. Also 80% of
>> the work we do is based on fixing short lived issues, the remaining 20%
>> involves significant work, that may already happen in a forked
>> repository. The difference here is between being forced the overhead of
>> maintaining a fork and the external steps of using the web interface to
>> submit and merge the work done on each forked repository, with the
>> occasional need to have one. It's fine to keep those as option for
>> people who want them, but we shouldn't be all forced to adapt.
>>
>> I'm not very happy to change to the dictation of another tooling imposed
>> workflow to be honest.

Neither are we in project Skara, so it is our hope that we have managed 
to create a solution where you can pick a workflow that suits you. That 
said, and as I describe further below in this email, we could not come 
up with a workflow that was step-by-step identical to what we use today.

>>> Other advantages include using an environment known to most developers
>>> out there - an environment not requiring any OpenJDK special scripts
>>> or workflows (easy on-boarding of new developers), contributors
>>> getting recognition where it matters (many companies/recruiters go
>>> look at GitHub commit history), starting using git (hg tool support is
>>> starting to wane) etc.
>>
>> Well, except that we are already OpenJDK developers, so we already know
>> all the tooling and workflow. It's exactly why I resist the GitHub way,
>> it may be nice for some but not for all.

Yes, if an OpenJDK project makes any changes to its workflow, then that 
means change for those accustomed to the current workflow. What we have 
tried to do with Skara is to spend significant time investigating how we 
can:

1. Minimize the disruption of changing to another workflow,
2. Enable contributors to use multiple different workflows at the same
    time so that each contributor can use the workflow that suits him or
    her the best.

That said, changing to _a_ source code hosting provider (Skara supports 
multiple) will mean some slight changes for contributors that enjoy the 
current OpenJDK workflow, but I think (hope) we have managed to come up 
with something very similar. I have worked with OpenJDK for 7+ years now 
so hopefully I know something about our current practices :)

One obvious change will be the switch from Mercurial (hg) to Git. For 
those contributors wanting to stay with hg I'm happy report that hg-git 
[1] works just fine with both github.com and gitlab.com. I created the 
following small change 
https://github.com/edvbld/jmc/commit/73bd425a0f9782eaf683855c0a2169a9ba130233 
using:

$ hg clone git+ssh://github.com/edvbld/jmc
$ cd jmc
$ hg bookmark readme
$ vim README.md
$ hg commit -m 'Add information about hg-git'
$ hg push --bookmark readme

I will create a page on the Skara OpenJDK wiki describing how to set up 
Mercurial to work with a Git source code hosting provider. But yes, as 
you see, this still forces you to use bookmarks in hg (since hg-git maps 
hg bookmarks to git branches). You would also have to add a path (see 
`hg help paths`) "upstream" pointing to https://github.com/openjdk/jmc 
so that you from time to time can run `hg pull upstream` (this is 
basically all there is to keep your personal fork up to date). I also 
want to add here that we are eagerly awaiting feedback on how 
complicated/tricky contributors think the above is so that we can create 
some custom tooling to help ease the experience if necessary.

Please also note that all CLI tools developed as part of Skara supports 
hg just as well as git, we did this precisely for the contributors 
wanting to use hg-git (I'm about to send out the final parts of the 
patch enabling hg support today).

In a previous email you described not wanting to use the browser and 
this is something we care about as well. Skara has implemented extensive 
CLI tooling to allow pull requests to be created, approved and 
integrated from the command line. Continuing the above example, still 
using hg, you would execute:

$ hg pr create # will open your $EDITOR for writing the message

You can also list pull requests from the command-line:

$ hg pr list
66   Upgrade JUnit 5 to 5.5.1                        sormuras build,rfr
120  Escape quotes in shell code stored in gitconfig rose00 rfr

You can approve pull requests from the command-line:

$ hg pr approve 66

And finally you can integrate pull requests from the command-line:

$ hg pr integrate 120

All of the above of course uses the public API's of the source code 
hosting provider but none of the above forces you to use the source code 
hosting provider's web UI for interacting with the hosting provider. 
Think of the web UI as being "just another interface" to the source code 
hosting provider, many source code hosting providers have very extensive 
API's work with.

Finally, for commenting on pull requests we have bi-directional syncing 
of comments between the OpenJDK mailing lists and source code hosting 
providers. That means if someone creates a pull request then a 
corresponding RFR will be created. The RFR email will contain an 
automatically generated webrev, links to any JBS issues, direct links to 
the patch and instructions for fetching the changes for local review. We 
also wait to send out the RFR email until the pull request have passed 
jcheck so that reviewers don't waste time looking at patches that don't 
even pass jcheck. As a reviewer, if you have any comments and/or 
feedback, then you can just reply to the RFR email. I know that you, me 
and Marcus did run into some early bugs in the mailing list bridge, it 
has improved significantly since then and Robin is currently doing the 
final touches on version two of the mailing list bridge.

Please note however that you cannot directly send an RFR email like we 
do today, you need to create a pull request which will trigger the 
automatic generation of an RFR email. You must also formally approve a 
PR by using `hg pr approve` (or another tool interacting with the 
platform), just replying "Looks good" to an RFR email will not work. 
These are the steps I mentioned in the beginning of the email that 
differ from our current workflow.

This email is already long enough as it is, I will try to summarize it 
on the Skara wiki and I'm happy to provide more detailed information if 
you have any questions. I would also like to say that we would be happy 
to work with you and the JMC project to figure out what can be improved 
and how we can offer a workflow that is as similar to our current one as 
possible (for those that want to use exactly that workflow).

>> Maintaining versions of webrevs is indeed a problem, but the process of
>> creating them is trivial and can be done via command line from the same
>> location you do the coding work, i.e. I don't need to
>> clone/fork/merge/remember to merge with a different repository too/then
>> open the browser click around until I find the right place and create a
>> pull request. If you use rsync you don't even need to create versions,
>> just keep overriding the same location.

As a Reviewer in the JDK project I just want to chime in here that I do 
expect to get an incremental webrev if someone has updated their patch ;)

Thanks,
Erik

[0]: https://wiki.openjdk.java.net/display/skara#Skara-Workflows
[1]: https://bitbucket.org/durin42/hg-git/

>>> Also, awesomely enough, the project can start
>>> using GitHub integrations like CI infrastructure to have tests run
>>> before allowing to merge etc.
>>
>> We already have most of this infrastructure in place, so if anything we
>> would need to spend time and resources to adapt it.
>>
>> Cheers,
>> Mario
>>
>>
>> --
>> Mario Torre
>> Associate Manager, Software Engineering
>> Red Hat GmbH <https://www.redhat.com>
>> 9704 A60C B4BE A8B8 0F30  9205 5D7E 4952 3F65 7898