Clarification regarding the GitHub Actions pre-submit testing

[Stuart Marks]

On 3/18/21 1:36 AM, Aleksey Shipilev wrote:
> On 3/18/21 3:55 AM, Stuart Marks wrote:
>> On 3/17/21 6:43 AM, Aleksey Shipilev wrote:
>>> TL;DR: I view GHA as the OpenJDK-wide litmus test for the "green" master.
>>
>> I think this is premature. Who is responsible for maintaining the systems on which
>> GHA workflows run? Are they configured correctly and consistently? Who keeps the
>> configurations up to date? If they are broken, who fixes them? Do they produce
>> reliable results? etc.
> 
> I believe the answers to some of these questions are in GHA workflow history:
>    
> https://github.com/openjdk/jdk/commits/master/.github/workflows/submit.yml

Based on Magnus' message, it seems like this is a group of volunteers. While having 
people volunteer is helpful, it's not the same as being responsible for something.

> Yes, systems-wise with GHA, we are in somewhat awkward position to trust that GH 
> Cloud (Azure?) is configured correctly, runs on proper hardware, and is being 
> maintained. There is a fair amount of trust in this equation. I haven't seen that 
> trust broken in the last half a year the GHA were running for OpenJDK (both for 
> mainline JDK and Codetools projects), so I am keeping hopeful.

Well this "awkwardness" is exactly what I'm concerned about. But "trust" is I think 
the wrong concept. There's no one there to trust.

The word you're looking for is "hope". The systems are there, we're using them, they 
seem to work, a few volunteers tinker with how we use them, and we hope that continues.

> The fact that GHA is not ideal does not make it less useful. As I note in the reply 
> to Mark, the number of follow-up trivial build/test fixes had dropped palpably after 
> GHA were introduced in JDK mainline repo.

I didn't say they're not useful. In fact, they're useful in that for most developers 
they get run automatically with zero effort, and they'll probably pick up gross 
errors (like forgetting to check in a file) fairly quickly.

I object, and continue to object to GHA as a litmus test.

Well, maybe we should be clear on what you mean by a litmus test. When I read it, I 
thought, "GHA must be green before integration." If that's what you mean, then I 
would disagree with this policy. (And I note Magnus did as well.)

But if you mean something different by "litmus test", perhaps you could clarify 
that; I'm making a bunch of assumptions about what you meant that might be incorrect.

>> My main takeaway from Magnus' message is that his team doesn't have these
>> responsibilities. Ok, then who does? Or... is nobody responsible? (Seems plausible.)
> 
> Again, at present this seems to be the self-appointed duty of some OpenJDK 
> contributors, look at GHA workflow history. If the only thing missing here is formal 
> assignment to maintain this, we can solve that. I am not sure that is a problem 
> worth solving, though, based on our current experience: there are interested parties 
> that do that outside the formal assignment.

In other words, we're hoping there are sufficient volunteers to keep this thing going.

Having a volunteer crew is probably sufficient to keep it running as a convenience, 
but if it turns into a gatekeeper, then it's not sufficient.

s'marks