RFR: 8258477: Pre-submit testing using GitHub Actions should merge changes from target branch

[Magnus Ihse Bursie]

On Thu, 21 Jan 2021 09:03:57 GMT, Robin Westberg <rwestberg at openjdk.org> wrote:

>> Normally when running GitHub Actions on a pull request, what is checked out is the merge of the pull request with the latest changes on the target branch. This ensure that what is tested is as close as possible to what will actually be the result of integrating said pull request. 
>> 
>> In our use of GitHub Actions, we don't run directly on pull requests but instead allow contributors to run them in their own personal forks. In that context, there is no notion of a target branch. However, we can infer the target project and branch by encoding this in the workflow file. This allows us to perform the same merge manually, and achieve the same behaviour. 
>> 
>> The change also adds an option to disable this automated merge when launching the workflow manually, which could be useful when investigating unexpected test failures.
>> 
>> Note that downstream projects picking up this change will have to adapt the workflow file to keep using these actions for pre-submit testing. This has been a common request from downstream projects, but could also be done in a separate change (or not at all).
>
> So, is there anyone who would like to review this? I still think it will improve the effectiveness of the GitHub Actions-based testing.

I might be missing something here, but it sounds like it muddies the water on exactly what is tested: if an automatic merge is possible, this will be done and tested. If the automatic merge fails, the original, un-merged, branch is tested. For instance, it will have the ironic effect of a small non-compatible change that passes the automatic merge will fail the test, but an even larger non-compatible change that do not pass the automatic merge will succeed!

I understand the intention, but I'm just worried it decreases the transparency of what GHA tests even more. 

How can you know what was tested?

-------------

PR: https://git.openjdk.java.net/jdk/pull/1801