[9] RFR(S): 8050407: Add jtreg compiler tests to Hotspot JPRT jobs

[Albert]

Hi,

doesn't it make more sense to select tests based on their 'usefulness'?

For example, it seems to me that adding a regression test that was 
introduced
based on a fix of a segmentation fault in compiled code makes more sense
than adding a test that checks the consistency of rarely used command line
arguments.

How about adding tests based on their priorities? Maybe it makes more sense
to add a test of a corresponding P1 bug than adding a test of a P5 bug?

Thanks,
Albert

On 08/28/2014 08:02 PM, Zoltán Majó wrote:
> Hi,
>
>
> please review the following patch.
>
>
> Bug: Add jtreg compiler tests to Hotspot JPRT jobs
>
>
> Problem: The test/TEST.groups file lists JTREG compiler tests executed 
> in JPRT (target name: 'hotspot_compiler'). We need to determine the 
> list of tests from test/compiler that should be executed in JPRT. The 
> total time of execution should be less than 10 minutes on slowest 
> platform.
>
>
> Solution: The slowest platform in JPRT is currently solaris_sparcv9. I 
> executed all open JTREG tests from test/compiler on solaris_sparcv9 
> and measured the "work time" of each test. Then, tests were sorted in 
> ascending order of their work time. To construct the subset, I first 
> added the test with the lowest work time to the subset and then 
> continued adding tests until a time limit L is reached.
>
> Limit L is set to 80% of the original time budget (10 minutes). L is 
> set conservatively to account also for JPRT "cleanup", "init", and 
> "finishing" time, as well as for the variation of tests' "work time". 
> (The profiling measurements contain only "work time".)
>
> The subset contains 77 tests. The longest executing test in the subset 
> takes 6.9 seconds.
>
>
> Webrev: http://cr.openjdk.java.net/~zmajo/8050407/webrev.00/
>
>
> Testing: Ran subset on all platforms in both west and stockholm JPRT. 
> Execution times of tests are as follows:
>
> West:
>   linux_i586-fastdebug-c1-hotspot_compiler         success(03m 01s)
>   linux_i586-fastdebug-c2-hotspot_compiler         success(03m 39s)
>   linux_x64-fastdebug-c2-hotspot_compiler          success(03m 32s)
>   macosx_x64-fastdebug-c2-hotspot_compiler         success(03m 48s)
>   solaris_sparcv9-fastdebug-c2-hotspot_compiler     success(08m 41s)
>   solaris_x64-fastdebug-c2-hotspot_compiler        success(03m 00s)
>   windows_i586-fastdebug-c1-hotspot_compiler       success(03m 39s)
>   windows_i586-fastdebug-c2-hotspot_compiler       success(03m 54s)
>   windows_x64-fastdebug-c2-hotspot_compiler        success(04m 51s)
>
> Stockholm:
>   linux_i586-fastdebug-c1-hotspot_compiler         success(02m 31s)
>   linux_i586-fastdebug-c2-hotspot_compiler         success(02m 39s)
>   linux_x64-fastdebug-c2-hotspot_compiler          success(02m 53s)
>   macosx_x64-fastdebug-c2-hotspot_compiler         success(04m 10s)
>   solaris_sparcv9-fastdebug-c2-hotspot_compiler     success(05m 55s)
>   solaris_x64-fastdebug-c2-hotspot_compiler        success(03m 06s)
>   windows_i586-fastdebug-c1-hotspot_compiler       success(02m 34s)
>   windows_i586-fastdebug-c2-hotspot_compiler       success(03m 07s)
>   windows_x64-fastdebug-c2-hotspot_compiler        success(04m 33s)
>
> We get close to the 10 minute budget (87% usage on West for 
> solaris_sparcv9).
>
>
> Thank you and best regards,
>
>
> Zoltan
>
>
>