[aarch64-port-dev ] RFR: exclude list (was Re: server JTREG results for OpenJDK 8 on AArch64)

[Andrew McDermott]

Edward Nevill <edward.nevill at linaro.org> writes:

> On Wed, 2014-02-12 at 12:42 +0000, Andrew Haley wrote:
>> On 02/10/2014 03:15 PM, OpenJDK Automated Test wrote:
>> > This is a summary of the JTREG test results for OpenJDK 8 on AArch64.
>> 
>> I've been investigating one of these tests.
>> 
>> CdsSameObjectAlignment fails with ObjectAlignmentInBytes=32 because
>> the initial heap is too small.  Set the -Xms correctly and it's fine.
>> There is nothing wrong with the functionality being tested, and the
>> test fails in the same way on x86.
>> 
>> I wonder what we should do.  Bogus test failures are very distracting
>> (not to mention dispiriting.)  Maybe we should get a list of x86 test
>> failures and use those to filter Aarch64 test falures.  Sure, they're
>> still failures, but we should be concentrating on Aarch64.
>> 
>> Andrew.
>
> Hi Andrew,
>
> I think the first thing to do is add an exclude list.
>
> Here is my first stab at such. It basically excludes everything which is not implemented (such as +G1GC).
>
> It is a reduced version of the one used for the nightly tests. I have
> reduced it because I was unhappy about some of the reasons for
> excluding tests in the nightly test (for example, some tests were
> excluded because they had long timeouts).
>
> I have put it in jdk8/test/exclude_aarch64.txt

>From a scripting P.O.V it would be convenient if there was one per
top-level JTREG test category. There's already jdk/test/ProblemList.txt,
so could this become hotspot/test/ProblemList.txt? And ditto for
langtools, nashorn, et al should there be a need to exclude tests.

>
> Andrew McDermott: Once this is in place can you modify the nightly
> scripts to use this one, rather than the one that is checked into
> git.linaro.org. I think the correct thing is for the exclude list to
> be checked into the sources they belong to rather than checked into
> the scripts on g.l.o.
>
> All the best,
> Ed.
>
> --- CUT HERE ---
> exporting patch:
> # HG changeset patch
> # User Edward Nevill edward.nevill at linaro.org
> # Date 1392213105 0
> #      Wed Feb 12 13:51:45 2014 +0000
> # Node ID 8f072f620113f8e39f3222ac57058977425a73ab
> # Parent  0c189a35ec9a939c862670794f08105f8d02d82a
> Initial rev. of a JTReg exclude list for aarch64
>
> diff -r 0c189a35ec9a -r 8f072f620113 test/exclude_aarch64.txt
> --- /dev/null	Thu Jan 01 00:00:00 1970 +0000
> +++ b/test/exclude_aarch64.txt	Wed Feb 12 13:51:45 2014 +0000
> @@ -0,0 +1,40 @@
> +# Added: Fri Feb  7 2014
> +# ----------------------
> +gc/g1/TestHumongousCodeCacheRoots.java	generic-all
> +
> +# Added: Wed Feb  5 2014
> +# ----------------------
> +compiler/regalloc/C1ObjectSpillInLogicOp.java	generic-all
> +gc/TestSystemGC.java	generic-all
> +gc/arguments/TestMaxNewSize.java	generic-all
> +gc/g1/TestHumongousAllocInitialMark.java	generic-all
> +runtime/8001071/Test8001071.java	generic-all
> +java/lang/management/MemoryMXBean/ResetPeakMemoryUsage.java	generic-all
> +
> +# These tests all require the G1 Garbage Collector, which is not
> +# implemented in the aarch64-port.  They are guaranteed to fail.
> +# Unfortunately, some of them run multiple tests with multiple garbage
> +# collectors, but as soon as they get to the G1 collector, they abort
> +# with an Unimplemented assertion.
> +gc/8000311/Test8000311.java	generic-all
> +gc/arguments/TestAlignmentToUseLargePages.java	generic-all
> +gc/arguments/TestG1HeapRegionSize.java	generic-all
> +gc/arguments/TestG1HeapSizeFlags.java	generic-all
> +gc/arguments/TestMaxHeapSizeTools.java	generic-all
> +gc/arguments/TestUseCompressedOopsErgo.java	generic-all
> +gc/g1/TestPrintGCDetails.java	generic-all
> +gc/g1/TestPrintRegionRememberedSetInfo.java	generic-all
> +gc/g1/TestRegionAlignment.java	generic-all
> +gc/g1/TestShrinkToOneRegion.java	generic-all
> +gc/metaspace/G1AddMetaspaceDependency.java	generic-all
> +gc/metaspace/TestMetaspacePerfCounters.java	generic-all
> +gc/startup_warnings/TestG1.java	generic-all
> +gc/TestG1ZeroPGCTJcmdThreadPrint.java	generic-all
> +
> +# ClassSpaceSize (aka PermGen) is not supported by Java 8
> +gc/metaspace/CompressedClassSpaceSizeInJmapHeap.java	generic-all
> +
> +# This collector is not implemented for server; for simplicity also ignoring on client tests.
> +gc/concurrentMarkSweep/CheckAllocateAndSystemGC.java    generic-all
> +gc/concurrentMarkSweep/GuardShrinkWarning.java  generic-all
> +gc/concurrentMarkSweep/SystemGCOnForegroundCollector.java       generic-all
> --- CUT HERE ---
>
>

-- 
Andy McDermott | Kernel Baselines Engineer
Linaro <www.linaro.org> │ Open source software for ARM SoCs