[9] RFR(S): 8050407: Add jtreg compiler tests to Hotspot JPRT jobs

[Nils Eliasson]

Have you considered excluding the expensive tests instead of including 
the cheap? Then we could avoid mistakes when people write new jtreg 
tests, run them through JPRT and forgetting the tests will be disabled 
by default.

//Nils

On 2014-08-28 20:02, Zoltán Majó wrote:
> Hi,
>
>
> please review the following patch.
>
>
> Bug: Add jtreg compiler tests to Hotspot JPRT jobs
>
>
> Problem: The test/TEST.groups file lists JTREG compiler tests executed 
> in JPRT (target name: 'hotspot_compiler'). We need to determine the 
> list of tests from test/compiler that should be executed in JPRT. The 
> total time of execution should be less than 10 minutes on slowest 
> platform.
>
>
> Solution: The slowest platform in JPRT is currently solaris_sparcv9. I 
> executed all open JTREG tests from test/compiler on solaris_sparcv9 
> and measured the "work time" of each test. Then, tests were sorted in 
> ascending order of their work time. To construct the subset, I first 
> added the test with the lowest work time to the subset and then 
> continued adding tests until a time limit L is reached.
>
> Limit L is set to 80% of the original time budget (10 minutes). L is 
> set conservatively to account also for JPRT "cleanup", "init", and 
> "finishing" time, as well as for the variation of tests' "work time". 
> (The profiling measurements contain only "work time".)
>
> The subset contains 77 tests. The longest executing test in the subset 
> takes 6.9 seconds.
>
>
> Webrev: http://cr.openjdk.java.net/~zmajo/8050407/webrev.00/
>
>
> Testing: Ran subset on all platforms in both west and stockholm JPRT. 
> Execution times of tests are as follows:
>
> West:
>   linux_i586-fastdebug-c1-hotspot_compiler         success(03m 01s)
>   linux_i586-fastdebug-c2-hotspot_compiler         success(03m 39s)
>   linux_x64-fastdebug-c2-hotspot_compiler          success(03m 32s)
>   macosx_x64-fastdebug-c2-hotspot_compiler         success(03m 48s)
>   solaris_sparcv9-fastdebug-c2-hotspot_compiler     success(08m 41s)
>   solaris_x64-fastdebug-c2-hotspot_compiler        success(03m 00s)
>   windows_i586-fastdebug-c1-hotspot_compiler       success(03m 39s)
>   windows_i586-fastdebug-c2-hotspot_compiler       success(03m 54s)
>   windows_x64-fastdebug-c2-hotspot_compiler        success(04m 51s)
>
> Stockholm:
>   linux_i586-fastdebug-c1-hotspot_compiler         success(02m 31s)
>   linux_i586-fastdebug-c2-hotspot_compiler         success(02m 39s)
>   linux_x64-fastdebug-c2-hotspot_compiler          success(02m 53s)
>   macosx_x64-fastdebug-c2-hotspot_compiler         success(04m 10s)
>   solaris_sparcv9-fastdebug-c2-hotspot_compiler     success(05m 55s)
>   solaris_x64-fastdebug-c2-hotspot_compiler        success(03m 06s)
>   windows_i586-fastdebug-c1-hotspot_compiler       success(02m 34s)
>   windows_i586-fastdebug-c2-hotspot_compiler       success(03m 07s)
>   windows_x64-fastdebug-c2-hotspot_compiler        success(04m 33s)
>
> We get close to the 10 minute budget (87% usage on West for 
> solaris_sparcv9).
>
>
> Thank you and best regards,
>
>
> Zoltan
>
>
>