[9] RFR(S): 8050407: Add jtreg compiler tests to Hotspot JPRT jobs

Vladimir Kozlov vladimir.kozlov at oracle.com
Thu Sep 4 19:33:03 UTC 2014


http://cr.openjdk.java.net/~zmajo/8050407/webrev.01/

This looks better.

On 9/4/14 1:42 AM, Albert wrote:
> Hi,
>
> doesn't it make more sense to select tests based on their 'usefulness'?

I agree. But we need to take into account its running time too.
I assume Zoltan did a quick review of tests on usefulness.

>
> For example, it seems to me that adding a regression test that was
> introduced
> based on a fix of a segmentation fault in compiled code makes more sense
> than adding a test that checks the consistency of rarely used command line
> arguments.

Agree. Which tests in the list do flags testing?

We should also exclude unstable tests (have to look on history of the test).

>
> How about adding tests based on their priorities? Maybe it makes more sense
> to add a test of a corresponding P1 bug than adding a test of a P5 bug?

We can't add P1 test if it takes more then 1 min to run. I would prefer 
to run 5 x 10 sec P3 tests instead. We would test more code this way.
Also Bug's priority was not accurate before as it is now.

Thanks,
Vladimir

>
> Thanks,
> Albert
>
> On 08/28/2014 08:02 PM, Zoltán Majó wrote:
>> Hi,
>>
>>
>> please review the following patch.
>>
>>
>> Bug: Add jtreg compiler tests to Hotspot JPRT jobs
>>
>>
>> Problem: The test/TEST.groups file lists JTREG compiler tests executed
>> in JPRT (target name: 'hotspot_compiler'). We need to determine the
>> list of tests from test/compiler that should be executed in JPRT. The
>> total time of execution should be less than 10 minutes on slowest
>> platform.
>>
>>
>> Solution: The slowest platform in JPRT is currently solaris_sparcv9. I
>> executed all open JTREG tests from test/compiler on solaris_sparcv9
>> and measured the "work time" of each test. Then, tests were sorted in
>> ascending order of their work time. To construct the subset, I first
>> added the test with the lowest work time to the subset and then
>> continued adding tests until a time limit L is reached.
>>
>> Limit L is set to 80% of the original time budget (10 minutes). L is
>> set conservatively to account also for JPRT "cleanup", "init", and
>> "finishing" time, as well as for the variation of tests' "work time".
>> (The profiling measurements contain only "work time".)
>>
>> The subset contains 77 tests. The longest executing test in the subset
>> takes 6.9 seconds.
>>
>>
>> Webrev: http://cr.openjdk.java.net/~zmajo/8050407/webrev.00/
>>
>>
>> Testing: Ran subset on all platforms in both west and stockholm JPRT.
>> Execution times of tests are as follows:
>>
>> West:
>>   linux_i586-fastdebug-c1-hotspot_compiler         success(03m 01s)
>>   linux_i586-fastdebug-c2-hotspot_compiler         success(03m 39s)
>>   linux_x64-fastdebug-c2-hotspot_compiler          success(03m 32s)
>>   macosx_x64-fastdebug-c2-hotspot_compiler         success(03m 48s)
>>   solaris_sparcv9-fastdebug-c2-hotspot_compiler     success(08m 41s)
>>   solaris_x64-fastdebug-c2-hotspot_compiler        success(03m 00s)
>>   windows_i586-fastdebug-c1-hotspot_compiler       success(03m 39s)
>>   windows_i586-fastdebug-c2-hotspot_compiler       success(03m 54s)
>>   windows_x64-fastdebug-c2-hotspot_compiler        success(04m 51s)
>>
>> Stockholm:
>>   linux_i586-fastdebug-c1-hotspot_compiler         success(02m 31s)
>>   linux_i586-fastdebug-c2-hotspot_compiler         success(02m 39s)
>>   linux_x64-fastdebug-c2-hotspot_compiler          success(02m 53s)
>>   macosx_x64-fastdebug-c2-hotspot_compiler         success(04m 10s)
>>   solaris_sparcv9-fastdebug-c2-hotspot_compiler     success(05m 55s)
>>   solaris_x64-fastdebug-c2-hotspot_compiler        success(03m 06s)
>>   windows_i586-fastdebug-c1-hotspot_compiler       success(02m 34s)
>>   windows_i586-fastdebug-c2-hotspot_compiler       success(03m 07s)
>>   windows_x64-fastdebug-c2-hotspot_compiler        success(04m 33s)
>>
>> We get close to the 10 minute budget (87% usage on West for
>> solaris_sparcv9).
>>
>>
>> Thank you and best regards,
>>
>>
>> Zoltan
>>
>>
>>
>


More information about the hotspot-compiler-dev mailing list