RFR: 8253865: Pre-submit testing using GitHub Actions does not detect failures reliably

[Erik Joelsson]

On Wed, 30 Sep 2020 16:58:25 GMT, Robin Westberg <rwestberg at openjdk.org> wrote:

> The pre-submit test definitions utilizing GitHub Actions can sometimes report success even when there are failing
> tests. The failure detection depended on make returning a non-zero exit code on failures, which doesn't seem to work.
> To properly determine test outcome we should check the "test-summary.txt" result file for the string "TEST SUCCESS". If
> it isn't found, the tests must have failed. This change also includes a reliability improvement.
> Here's an example of a run with the updated test definitions: https://github.com/rwestberg/jdk/actions/runs/280529475
> 
> Note! There is a failure now properly detected in langtools/tier1 on Windows
> (tools/javac/launcher/SourceLauncherTest.java) which is tracked by https://bugs.openjdk.java.net/browse/JDK-8249095 -
> could be of interest to either fix or ProblemList that one before integrating this change.

Marked as reviewed by erikj (Reviewer).

-------------

PR: https://git.openjdk.java.net/jdk/pull/437