RFR: 7903935: --verify-exclude existence check misses tests with failing requires

[Ludvig Janiuk]

On Tue, 21 Jan 2025 14:20:25 GMT, Ludvig Janiuk <lujaniuk at openjdk.org> wrote:

> The current implementation relies on
>   getResultsIterator
> which in turn calls into
>   TestResultTable.getIterator(filters)
> passing into jtharness. That seems to currently filter out tests that don't meet the local "requires" criteria. Thus we fail to collect all "existing" tests in to a set to compare with.
> 
> The fix seems to be to explicitly pass an empty list of filters into
>   TestResultTable.getIterator(filters)
> as this will make
>   TRT_Iterator.wouldAccept()
> accept every test.

As a way to test these changes on real-world data, I ran against the OpenJDK problemlist files. The following lists how many tests in each were listed as problematic:

test/lib-test/ProblemList.txt: 0
test/jaxp/ProblemList.txt: 0
test/docs/ProblemList.txt: 0
test/hotspot/jtreg/ProblemList-AotJdk.txt: 1
test/hotspot/jtreg/ProblemList.txt: 3
test/hotspot/jtreg/ProblemList-Virtual.txt: 0
test/hotspot/jtreg/ProblemList-Xcomp.txt: 0
test/hotspot/jtreg/ProblemList-zgc.txt: 14
test/jdk/ProblemList-AotJdk.txt: 0
test/jdk/ProblemList-zgc.txt: 0
test/jdk/ProblemList-shenandoah.txt: 0
test/jdk/ProblemList-Virtual.txt: 4
test/jdk/ProblemList.txt: 8
test/jdk/ProblemList-Xcomp.txt: 0
test/langtools/ProblemList.txt: 0

After a manual look at `test/hotspot/jtreg/ProblemList-zgc.txt`, it does look like it references a lot of tests that don't exist. These were:

serviceability/sa/ClhsdbFindPC.java#apa                       8307393   generic-all
serviceability/sa/ClhsdbLauncher.java                         8307393   generic-all
serviceability/sa/ClhsdbPmap.java                             8307393   generic-all
serviceability/sa/ClhsdbScanOops.java                         8307393   generic-all
serviceability/sa/LingeredAppSysProps.java                    8307393   generic-all
serviceability/sa/LingeredAppWithDefaultMethods.java          8307393   generic-all
serviceability/sa/LingeredAppWithEnum.java                    8307393   generic-all
serviceability/sa/LingeredAppWithInterface.java               8307393   generic-all
serviceability/sa/LingeredAppWithInvokeDynamic.java           8307393   generic-all
serviceability/sa/LingeredAppWithLock.java                    8307393   generic-all
serviceability/sa/LingeredAppWithNativeMethod.java            8307393   generic-all
serviceability/sa/LingeredAppWithRecComputation.java          8307393   generic-all
serviceability/sa/jmap-hprof/JMapHProfLargeHeapProc.java      8307393   generic-all
serviceability/sa/sadebugd/DebugdUtils.java                   8307393   generic-all

Indeed, manual sampling reveals these lines are invalid, e.g. because `ClhsdbFindPC.java` does not have a test with id `apa`, or because `serviceability/sa/LingeredAppWithInterface.java` is not a jtreg test at all (no `@test` comment).

-------------

PR Comment: https://git.openjdk.org/jtreg/pull/243#issuecomment-2615342921