Questions about pre-submit testing using GitHub Actions

Volker Simonis volker.simonis at gmail.com
Fri Oct 9 15:47:50 UTC 2020


Hi,

first of all thanks a lot for adding support for pre-submit testing
using GitHub Actions. I think this is a really useful feature!

Unfortunately, I couldn't find much documentation about it except some
emails on this list and the initial PR [1], so I still have a few
questions :)

1. All the pre-submit test that have been run for me until now (e.g.
[2]) have failed for "Windows x64 (hs/tier1 gc)" presumably because of
a generic "out of disc space" error:

Error: There is not enough space on the disk. :
'D:\a\_temp\_runner_file_commands\add_path_1a4cdd0c-7bc7-4870-9105-dd1bb009be71'

This starts at "TEST: gc/shenandoah/compiler/TestClone.java#id5" after
which all subsequent tests fail because of insufficient disc space:
2020-10-09T12:03:09.6467073Z
D:\a\jdk\jdk\test\hotspot\jtreg\gc\shenandoah\compiler\TestClone.java:209:
error: error while writing TestClone: There is not enough space on the
disk
...
2020-10-09T12:03:09.6475592Z TEST RESULT: Error. Write to temp. JTR
file failed (old JTR intact):
D:\a\jdk\jdk\build\run-test-prebuilt\test-support\jtreg_test_hotspot_jtreg_tier1_gc\gc\shenandoah\compiler\TestClone_id5.jtr.0.tmp

Is this a known problem? Is somebody working on fixing it?

2. When looking at "Linux x64 (hs/tier1 gc)" Ican see that there are
44 test failures:

2020-10-09T11:40:16.8365547Z Test results: passed: 255; failed: 44
2020-10-09T11:40:22.0971166Z ==============================
2020-10-09T11:40:22.0973305Z Test summary
2020-10-09T11:40:22.0973969Z ==============================
2020-10-09T11:40:22.0974393Z    TEST
           TOTAL  PASS  FAIL ERROR
2020-10-09T11:40:22.0974844Z >> jtreg:test/hotspot/jtreg:tier1_gc
             299   255    44     0 <<
2020-10-09T11:40:22.0975267Z ==============================

But the test group has a green check mark and is flagged as passed in
the report: "Linux x64 (hs/tier1 gc) succeeded (3 annotations)". How
can this happen? Is this a known problem?

By the way, the 44 failing test all fail on Windows and MacOS as well
because of a crash in:
2020-10-09T12:03:06.0162304Z #  Internal Error
(d:\a\jdk\jdk\jdk\src\hotspot\share\runtime/vmThread.hpp:103),
pid=6344, tid=3700
2020-10-09T12:03:06.0163057Z #
assert(Thread::current()->is_VM_thread()) failed: Must be

It looks like the framework only recognizes jtReg "Errors" but not
"Failures" as errors.

3. Would it be possible to save the hs_err files along with the .jtr
files in the case of crashes? I think this would be extremely useful
for quickly diagnosing problems.

4. I don't exactly understand the "free tier" you mention in your PR
[1]. From the GitHub actions side as described in their "Usage limits"
documentation [3] it seems we're on the safe side. But that
information somehow contradicts with the information about "GitHub
Free for user accounts" [4] which mentions that free user accounts
only have 2000 free GitHub Actions minutes per month. In addition
"About billing for GitHub Actions" [5] mentions that these free
minutes are subject to a multiplier which is one for Linux, two for
Windows and ten for macOS which effectively means that after running
200 minutes on macOS our personal free minutes for a month will
already be consumed. So to me it seems that GitHub Actions "Usage
limits" documentation [3] only sets an upper bound for job and
workflow execution time of Actions but this has to be interpreted in
the context of the totally available, free minutes for Actions. But
maybe I'm wrong?

If you know better :) could you please explain which of these limits
apply and how many automatic tests jobs they are approximately
covering?

Thank you and best regards,
Volker

[1] https://github.com/openjdk/jdk/pull/284
[2] https://github.com/simonis/jdk/runs/1230911240
[3] https://docs.github.com/en/free-pro-team@latest/actions/reference/usage-limits-billing-and-administration#usage-limits
[4] https://docs.github.com/en/free-pro-team@latest/github/getting-started-with-github/githubs-products#github-free-for-user-accounts
[5] https://docs.github.com/en/free-pro-team@latest/github/setting-up-and-managing-billing-and-payments-on-github/about-billing-for-github-actions#about-billing-for-github-actions


More information about the skara-dev mailing list