RFR: 8256127: Add cross-compiled foreign architectures builds to submit workflow [v5]

Thu Nov 12 10:10:58 UTC 2020

On Thu, 12 Nov 2020 09:34:51 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Maybe it would be possible to build hotspot only as this is the part where things often break? This would save a lot of CPU time.
>> E.g. I think it'd be valueable to have at least one Big Endian platform included.
>> I noticed that some Oracle people use cross builds to check their changes on these platforms. It would save a bit of their time as well.
>
>> I'm not at all sure I want to accept these changes. I understand that you care about these exotic platforms, and so do I, but adding them to the submit workflow for _all_ changes seem like the wrong way to proceed.
> 
> I understand the first reaction, but please read the updated description. It includes the discussion what bugs it tries to catch, and updated cost estimates. 
> 
> Submit workflow is a convenience thing for better testing coverage. Having the early warnings for fixable issues in foreign architectures helps to avoid follow-up churn without increasing the costs all too much. Do note that it does not require contributors to fix the runtime issues in foreign architectures, it only asks for passing a rather low bar of _not breaking the builds_ for them, which is, in my experience, an easily achievable goal. It is easily achievable because --  I am speaking from experience of fixing a lot of them over the years! -- the overwhelming majority of arch-specific build breakages are about silly things that are obvious from the build logs.
> 
> So I am leaving this PR open, letting others to chime in. If there is really a need for a wider discussion here, I could start a thread at jdk-dev@, but so far it does not look as a good use of our collective time. Please read through the description, and see if that soothes your concerns.
> 
> Aside: the issues about costs were [raised before](https://mail.openjdk.java.net/pipermail/skara-dev/2020-October/003581.html), but I think all of them resolve as "enjoy your free tier". If we target to optimize the GH action costs, IMO we should instead consider to avoid triggering them on every push, as [I hinted before](https://mail.openjdk.java.net/pipermail/jdk-dev/2020-September/004739.html), and which I think requires better Skara integration (i.e. for `/test` command). Meanwhile, this PR gives us extended build coverage at fraction of the additional cost.

I think this is a really useful feature and I'm a little surprised that it gets rejected. Please find my comments inline.

> Aleksey,
> 
> I'm not at all sure I want to accept these changes. I understand that you care about these exotic platforms, and so do I, but adding them to the submit workflow for _all_ changes seem like the wrong way to proceed.
> 
> My concern vary all the way from increased build time, to increased usage of the Github CPU quota (which, as I understand it, is fairly large but not unlimited), to the elephant in the room: the question on who the burden lays to fix bugs in these exotic platforms. It is not at all obvious that these builds should be tested before commit.
> 

I think "on who's the burden to fix issues" on *exotic* platforms and "testing changes on all platforms" at  commit time are two totally distinct questions. While I agree that problems on a *exotic* platform shouldn't block reviewed changes for an unreasonable amount of time, I also think it is very valuable for both, authors of changes and maintainers of *exotic* platforms to at least get early notifications of problems on other platforms.

My experience as a maintainer of the AIX7PowerPC/s390 ports has always been very positive with regards to the collaboration with authors of changes which impact these ports. Authors are anxious to not break other platforms but usually don't have the possibility to test on these platforms. Aleksey's change comes in very handy here. It will help authors to fix trivial build problems right in their initial submission which I'm sure they'll appreciate. For more complex problems it gives Authors the possibility to notify port maintainers early, before a change gets pushed. I'm sure this alerting could even be automatized in the future if builds/tests fail on certain platforms.

Of course such problems shouldn't block changes which are otherwise reviewed and ready to push for an unreasonable amount of time, so I have nothing against making the test on such platform optional submit requirements.

But overall I think Aleksey's enhancement are a great means to improve the development experience and to keep the OpenJDK repos clean of trivial follow-up fixes.

PS: I specially emphasized *exotic platforms* because it seems to me that your comment implies that all not Oracle-supported platforms are considered *exotic* for you. I think in an open source project it should be decided by the community which platforms are considered *main* (or *primary*) and which are considered *secondary* platforms. 

> I think this change warrants a much wider and deeper discussion about the purpose of the Github commit hook actions, how the free Github CPU quota should be spent, and on whom the responsibility lays to either make sure no. commit is pushed that breaks an exotic platform, or to rectify the problem with a follow-up issue.
> 

I agree that we should we frugal with resources, especially if these resources are being offered to us at no charge. But from my understanding, the resources for GitHub actions are accounted on a personal account base for every developer and not to the upstream project. So every developer who clones OpenJDK and submits changes to his fork will use his personal "GitHub Actions" budget. According to [the documentation](https://docs.github.com/en/free-pro-team@latest/actions/reference/usage-limits-billing-and-administration#usage-limits) and [@rwestberg's explanations on skara-dev](https://mail.openjdk.java.net/pipermail/skara-dev/2020-October/003581.html) this budget contains 20 parallel jobs (with up to 6 hours runtime) 24/7. Even with Aleksey's changes I think we're well in this budget.

Are you afraid tha GitHub/Microsoft will cut down this free-tier in general or for for OpenJDK users? If that's the case we should probably try to get an offical statement from GitHub/Microsoft on this topic before unnecessarily restricting ourselves.

As Aleksey explained, the additional builds should still finish before the corresponding Windows/MacOS builds and tests terminate, so there shouldn't be any usability issue either.

> I recommend you withdraw this PR and instead initiate a wider discussion on a suitable mailing list, e.g. jdk-dev.

I really like this change and I'd appreciate if we could get it merged. I'm of course always open to discussion on details (e.g. if the new biuilds/tests should be mandatory or. optional, how to notify port maintainers, etc...).

-------------

PR: https://git.openjdk.java.net/jdk/pull/1147