RFR: 8257147: [TESTBUG] Set a larger default loop count for the VectorAPI jtreg tests

[Xiaohong Gong]

On Thu, 17 Dec 2020 17:46:40 GMT, Paul Sandoz <psandoz at openjdk.org> wrote:

>> Hi @PaulSandoz , I tested the jtreg tests with these two options these days, and the results do not show better as expected. 
>> 
>> With `-XX:-TieredCompilation`, the tests can work well without increasing the loop count. However, I think it's not an optimal solution. First, with this flag, some issues that might only happen with tiered compilation might not be reported as expected (Currently we do have met a similar issue).  Second, a better way to add the option for the test is adding it to the test file (e.g. ` @run testng/othervm -XX:-TieredCompilation`).  However, this might override the same options people specified when running jtreg.
>> 
>> For `-XX:CompileThreshold=1000`, unfortunately it seems this flag doesn't have any effect that I still need to increase the loop count to make the tests effective.  Good news is that I found the flags `-XX:CompileThresholdScaling=0.1 -Xbatch` can work well even if I do not increase the loop count. However, this still increase the whole execution time a lot when running with `-conc:1`. Here is part of the comparison for the time:
>> 
>> Running with "-conc:10":
>>              Before     After
>> System A     5m21s      5m29s
>> System B     5m16s      6m7s
>> 
>> Running with "-conc:1":
>>              Before     After
>> System A     33m20s     62m14s
>> System B     36m17s     71m59s
>> 
>> So do you have tried these before? Or any better idea about this? Thanks!
>> 
>> Xiaohong Gong
>
> Thanks for looking at this in more detail. Before suggesting i did try some rough experiments which did indicate some upside to running without tiered with reduced compiler thresholds, but i only selected a few test classes to run against.
> 
> It's true that some issues have been found in tiered compilation. However, i think there is a tradeoff, as these tests are really designed exercise C2. 
> 
> My suggestion for now is to just turn off tiered compilation, which gets us reasonably far, and perhaps keep pushing on compiler threshold investigation as a follow up.
> 
>  I will follow up with our Hotspot test engineers to see if there is a way to run these tests less frequently under tiered compilation (as we do for other flags to stress the HotSpot compilers, such as `-Xcomp`).

Hi @PaulSandoz, I'v updated the patch. Could you please take a look at it again? Thanks so much!

-------------

PR: https://git.openjdk.java.net/jdk/pull/1621