testing new Graal backends for Java conformance

[Doug Simon]

Currently we have the gate tests plus a bunch of benchmarks we run for every push. The latter are run post-gate.

Since we only have a fully functioning AMD64 backend, we currently use only AMD64 machines in our gate/benchmarking infrastructure.

We don’t yet do any rigorous TCK/JCK testing. This kind of testing will occur at the point Graal is adopted into the mainline HotSpot code base.

On Nov 4, 2013, at 5:36 PM, Venkatachalam, Vasanth <Vasanth.Venkatachalam at amd.com> wrote:

> Doug or others-
> 
> Do you have a way of testing new backends that are developed for Graal for Java compliance (against the TCK)?
> 
> If so, can you explain how we can plug our HSAIL backend into this process so that it also gets tested for Java compliance?
> 
> A related question has to do with unit testing. So far we've been writing separate unit tests (in compiler.hsail.test) for each new Java operator that we add support for (e.g., +, -, *)  as well as program constructs (e.g., conditionals, loops) to ensure that we are generating the correct code. Is there already a common infrastructure in Graal that could be leveraged for this purpose?

The JTT tests (com.oracle.graal.jtt) provide fairly good coverage of basic Java functionality. Additions are always welcome ;-)

> On a related note, can you explain at a high level how you are ensuring that you have complete testing coverage (for the x86 code gen) in compiler.test/ ?

We don’t have much else apart from what is listed above. However, in combination with our automated benchmarking system, we think the coverage is fine for now (even though it can always be better of course). We also use JaCoCo for analyzing the code paths covered by the gate (see the '--jacocout’ option of the ‘mx gate’ command).

If you want to see the list of tests (apart from doing a text search for “@Test”), use the MX_TESTFILE variable to capture the set of tests run by ‘mx unittest’ to a file:

% mx help unittest
...
% env MX_TESTFILE=test.list mx unittest
% cat test.list

-Doug