RFR: 8295970: Add jdk_vector tests in GHA

[Erik Joelsson]

On Thu, 27 Oct 2022 03:53:13 GMT, Jie Fu <jiefu at openjdk.org> wrote:

> Hi all,
> 
> As discussed here https://github.com/openjdk/jdk/pull/10807#pullrequestreview-1150314487 , it would be better to add the vector api tests in GHA.
> 
> Thanks.
> Best regards,
> Jie

The default set of tests included in GHA is currently defined to be exactly tier1. That is the set of tests that we have collectively agreed on represents a reasonably sized set that gives the best coverage for the time spent (at least that's the intention of tier1). The time spent here is probably the most important variable, as whatever we put here, everyone runs all the time in their PRs and PFs. Adding 5 minutes may not seem like much, but multiplied with platforms and all the users and PRs it adds up to a lot. Also imagine if more components would want to add to the set, pointing to this change as a precedent ,adding another 5 minutes, and then another 5 minutes, eventually the reasonable size of the test set becomes completely unreasonable.

My point is that we can't just add more and more to GHA on a whim. The current agreement is for it to be exactly tier1. If you can't motivate that your tests should be part of tier1, then I find it pretty hard to motivate that it should be part of GHA. I don't know enough about these tests to know if they should fit in tier1/GHA or not, I'm just pointing out that we need to keep a pretty high bar to avoid bloating the set.

For this particular issue, would it be possible to define a minimal set of vector api tests that would have discovered the problem in https://github.com/openjdk/jdk/pull/10807 without taking 5 minutes? Perhaps the better solution here would be to allow a single or very few specific tests into tier1?

-------------

PR: https://git.openjdk.org/jdk/pull/10879