Clarification regarding the GitHub Actions pre-submit testing

Aleksey Shipilev shade at
Wed Mar 17 13:43:26 UTC 2021


A significant part of it feels like Oracle-specific matter.
But no worries, we can discuss OpenJDK here.

TL;DR: I view GHA as the OpenJDK-wide litmus test for the "green" master.

On 3/17/21 12:42 PM, Magnus Ihse Bursie wrote:
> * It is not a requirement to have green results from GHA tests to
> integrate a fix. Nothing in the OpenJDK bylaws, or the JDK project rules
> requires this.

IIRC, GHA workflows were introduced as the replacement for Oracle's jdk-submit forest, which was not 
implemented for Git after the move to Skara. Anyhow, jdk-submit was also a recommendation, as far as 
I can remember. Still, Oracle folks asked contributors to pass it before push, and contributors 
generally obliged. It seems fair to ask for reciprocity here.

> * That being said, you should of course only integrate correct and
> working code. If you believe GHA is a tool for you to discover bugs, by
> all means, use it. But if it shows up red, that does not mean *in
> itself* that your code is broken.

That is true. And like with jdk-submit before, you can reasonably ignore some failures, by 
rationally arguing they are not related to the current PR.

> * The Build Team will not engage in solving problems with building or
> running tests on GHA machines. We have no access to these machines, and
> no ability to fix problems with them. Any requests to fix issues with
> GHA will be ignored.

Now that I am thinking about it, is the crux of the issue in the implicit _mapping_ of every 
.github/workflows/ problem to infrastructure/build and build-dev@? I can see how that can be 
frustrating and arguably outside the build-infra team charter. If that is the issue, we should be 
addressing it: e.g. by adding more members to build-infra (I volunteer here), or remapping GHA 
worklows out of build-infra to something more generic e.g. "jdk"?

That is to say, even though Magnus, being the build-infra lead, can clarify that build team is not 
responsible for GHA issues, the OpenJDK Project as whole would still benefit from functional GHA. I 
am fixing GHA problems when I discover them, and I invite others to join in analyzing/fixing 
problems with it. Over last few months, many Oracle engineers did offer a helping hand in this, and 
I hope that would continue, even if at lower priority.

This is why I am concerned to see this:

> * Oracle employees should not engage with GHA issues. Oracle has its own
> internal, and much superior, building and testing system. Oracle
> employees should use and rely on this system for building and testing
> code, instead of GHA.

"Should not engage" sounds odd here. Was that meant as "${vendor} employees are required to pass 
${vendor}-internal testing, and GHA is not enough alone"? That looks fine, and what I expect 
${vendor} to have as policy. Otherwise it can be sadly read -- while within the ${vendor}'s freedom 
to conduct their business as they want -- as the order to ignore the problems in shared community 

Generally speaking:

It is beyond doubt that many OpenJDK vendors have testing systems that are more comprehensive than 
what is currently done in GHA -- which is absolutely expected, given that GHA is essentially a 
litmus test. This is not specific for Oracle's testing. To shamelessly plug myself, I have my own CI 
farm for, and in many areas it is also more comprehensive than GHA. Still, it 
does not replace GHA, because only myself has access to it. Having my own testing capability does 
not limit me from maintaining and using GHA.

Technically speaking, in the last few months we saw issues that slipped through Oracle's build-test 
system. Arch-specific build breakages were caught by GHA. Not sure if Oracle systems run any 32-bit 
tests, so 32-bit build/test failures were caught. There were issues when a minor 
post-internal-testing fix actually broke the code, and it was caught by GHA, which runs on every sync.

Also, GHA improves the OpenJDK community story: external contributors, especially first time 
contributors, and/or those not employed by big OpenJDK vendors, benefit from GHA greatly by getting 
their patches built/tested on the systems they cannot otherwise build/test at all.

This is why I view GHA as the important community service that is complementary for vendors' 
testing. For the sake of the shared project health, let's keep GHA stable, reliable, green.


More information about the jdk-dev mailing list