RFR: 8257147: [TESTBUG] Set a larger default loop count for the VectorAPI jtreg tests

[Xiaohong Gong]

On Tue, 8 Dec 2020 20:25:18 GMT, Paul Sandoz <psandoz at openjdk.org> wrote:

>> The current default loop count of VectorAPI tests is too small (10 for load/store tests, 100 for arithmetic tests) to make the tests compiled with C2. This makes some potential issues not be reported as expected.
>> 
>> This patch fixes it by setting a larger default iteration.
>> 
>> The whole tests running time increases with this patch. Here is the results on two different types of machines:
>> 
>> Running with "`-conc:10`":
>>              Before     After
>> System A     5m30s      6m43s
>> System B     5m1s       6m31s
>> 
>> Running with "`-conc:1`":
>>              Before     After
>> System A     26m52s     45m6s
>> System B     30m3s      43m30s
>
> It's good to revisit this, but I am concerned about the increase in test execution time. I wonder if we can approach this a little differently.
> 
> First, I think we should switch off tiered compilation, `-XX:-TieredCompilation` (maybe reconsider later some limited form of execution with tiered compilation enabled).
> For such a configuration I observe, when using the current invocation count and looking at inline traces, far more lines with intrinsic `VectorSupport` methods.
> 
> Second, we could lower the compile threshold from 10000 e.g. `-XX:CompileThreshold=1000`. We can grep the inline trace and count the lines containing intrinsic `VectorSupport` methods.
> 
> Using both these approaches i think we can increase intrinsification without such a large increase in test execution time.
> 
> WDYT?

Hi @PaulSandoz , I tested the jtreg tests with these two options these days, and the results do not show better as expected. 

With `-XX:-TieredCompilation`, the tests can work well without increasing the loop count. However, I think it's not an optimal solution. First, with this flag, some issues that might only happen with tiered compilation might not be reported as expected (Currently we do have met a similar issue).  Second, a better way to add the option for the test is adding it to the test file (e.g. ` @run testng/othervm -XX:-TieredCompilation`).  However, this might override the same options people specified when running jtreg.

For `-XX:CompileThreshold=1000`, unfortunately it seems this flag doesn't have any effect that I still need to increase the loop count to make the tests effective.  Good news is that I found the flags `-XX:CompileThresholdScaling=0.1 -Xbatch` can work well even if I do not increase the loop count. However, this still increase the whole execution time a lot when running with `-conc:1`. Here is part of the comparison for the time:

Running with "-conc:10":
             Before     After
System A     5m21s      5m29s
System B     5m16s      6m7s

Running with "-conc:1":
             Before     After
System A     33m20s     62m14s
System B     36m17s     71m59s

So do you have tried these before? Or any better idea about this? Thanks!

Xiaohong Gong

-------------

PR: https://git.openjdk.java.net/jdk/pull/1621