[9] RFR(S): 8050407: Add jtreg compiler tests to Hotspot JPRT jobs
Vladimir Kozlov
vladimir.kozlov at oracle.com
Thu Sep 11 16:37:32 UTC 2014
So we have 93 tests left in open and about 150 in closed/compiler.
Add 4th group compiler_closed and collect fast closed tests there.
It would be nice to have tests from the same directory in one group, for example for 5091921.
Also if you want to run all tests from one directory it could be possible to specify only directory, for example
compiler/5091921 and compiler/intrinsics. Please, try it.
I don't think we need tests which verify generated assemble, like intrinsics/bmi/verifycode/.
Also don't need WB tests which verifies compilations and inlinining, like compiler/whitebox/SetDontInlineMethodTest.java
Exclude compiler/rtm/ because they may not run at all due to old CPUs we use in JPRT.
Thanks,
Vladimir
On 9/11/14 8:24 AM, Zoltán Majó wrote:
> Hi,
>
>
> On 09/10/2014 11:46 AM, Zoltán Majó wrote:
>> On 09/05/2014 08:34 PM, Vladimir Kozlov wrote:
>>> Zoltan,
>>>
>>> Christian Tornqvist gave a great suggestion in chat room. We can split tests in several sets each taking no more than
>>> 10 min. Runtime has hotspot_runtime and hotspot_runtime_closed. So we can have several hotspot_compiler_tests1, etc too.
>>>
>>> Could you, please, investigate this?
>>
>> Sure, I'll look into it.
>
> I created 3 subsets, hotspot_compiler_[1-3]. The subsets include all quick-running JTREG open tests (test with < 10
> seconds execution time) that are useful (at least in my view). There are 83/69/58 tests in subset 1/2/3.
>
> Here is the new webrev:
>
> http://cr.openjdk.java.net/~zmajo/8050407/webrev.03/
>
> Below are the execution times ("work time") for all 3 subsets. There are 3 runs for each subset on both Stockholm and
> West queues.
>
>
> STOCKHOLM:
>
>
> hotspot_compiler_1
>
> solaris_sparcv9-fastdebug-c2-hotspot_compiler_1 success(08m 22s)
> USED: hostname=sthjprtsol33 platform=solaris_sparc_5.11 osname=solaris osarch=sparc cpus=12 parallelcount=12
> ram=24576MB cpufreqmhz=2848 instance=P1
> TIMING: clean=1s init=27s work=7m52s fini=1s
>
> solaris_sparcv9-fastdebug-c2-hotspot_compiler_1 success(09m 11s)
> USED: hostname=sthjprtsol35 platform=solaris_sparc_5.11 osname=solaris osarch=sparc cpus=12 parallelcount=12
> ram=24576MB cpufreqmhz=2848 instance=P1
> TIMING: clean=1s init=26s work=8m41s fini=1s
>
> solaris_sparcv9-fastdebug-c2-hotspot_compiler_1 success(08m 36s)
> USED: hostname=sthjprtsol40 platform=solaris_sparc_5.11 osname=solaris osarch=sparc cpus=12 parallelcount=12
> ram=24576MB cpufreqmhz=2848 instance=P1
> TIMING: clean=2s init=21s work=8m10s fini=1s
>
> hotspot_compiler_2
>
> solaris_sparcv9-fastdebug-c2-hotspot_compiler_2 success(09m 33s)
> USED: hostname=sthjprtsol41 platform=solaris_sparc_5.11 osname=solaris osarch=sparc cpus=12 parallelcount=12
> ram=24576MB cpufreqmhz=2848 instance=P1
> TIMING: clean=2s init=27s work=9m2s fini=1s
>
> solaris_sparcv9-fastdebug-c2-hotspot_compiler_2 success(09m 27s)
> USED: hostname=sthjprtsol42 platform=solaris_sparc_5.11 osname=solaris osarch=sparc cpus=12 parallelcount=12
> ram=24576MB cpufreqmhz=2848 instance=P1
> TIMING: clean=6s init=14s work=9m2s fini=1s
>
> solaris_sparcv9-fastdebug-c2-hotspot_compiler_2 success(08m 38s)
> USED: hostname=sthjprtsol45 platform=solaris_sparc_5.11 osname=solaris osarch=sparc cpus=12 parallelcount=12
> ram=24576MB cpufreqmhz=2848 instance=P1
> TIMING: clean=1s init=22s work=8m13s fini=1s
>
> hotspot_compiler_3
>
> solaris_sparcv9-fastdebug-c2-hotspot_compiler_3 success(09m 20s)
> USED: hostname=sthjprtsol43 platform=solaris_sparc_5.11 osname=solaris osarch=sparc cpus=12 parallelcount=12
> ram=24576MB cpufreqmhz=2848 instance=P1
> TIMING: clean=1s init=26s work=8m49s fini=1s
>
> solaris_sparcv9-fastdebug-c2-hotspot_compiler_3 success(09m 09s)
> USED: hostname=sthjprtsol48 platform=solaris_sparc_5.11 osname=solaris osarch=sparc cpus=12 parallelcount=12
> ram=24576MB cpufreqmhz=2848 instance=P1
> TIMING: clean=1s init=18s work=8m46s fini=1s
>
> solaris_sparcv9-fastdebug-c2-hotspot_compiler_3 success(09m 16s)
> USED: hostname=sthjprtsol34 platform=solaris_sparc_5.11 osname=solaris osarch=sparc cpus=12 parallelcount=12
> ram=24576MB cpufreqmhz=2848 instance=P1
> TIMING: clean=2s init=21s work=8m51s fini=1s
>
>
> WEST:
>
>
> hotspot_compiler_1
>
> solaris_sparcv9-fastdebug-c2-hotspot_compiler_1 success(09m 55s)
> USED: hostname=sc11152412 platform=solaris_sparc_5.11 osname=solaris osarch=sparc cpus=2 parallelcount=2
> ram=7900MB cpufreqmhz=2848 instance=T1
> TIMING: clean=11s init=50s work=8m45s fini=2s
>
> solaris_sparcv9-fastdebug-c2-hotspot_compiler_1 success(10m 59s)
> USED: hostname=sca00bvx platform=solaris_sparc_5.11 osname=solaris osarch=sparc cpus=6 parallelcount=6
> ram=31744MB cpufreqmhz=2848 instance=P1
> TIMING: clean=6s init=42s work=10m3s fini=2s
>
> solaris_sparcv9-fastdebug-c2-hotspot_compiler_1 success(07m 35s)
> USED: hostname=sca00bnr platform=solaris_sparc_5.11 osname=solaris osarch=sparc cpus=16 parallelcount=16
> ram=32768MB cpufreqmhz=3600 instance=P1
> TIMING: clean=3s init=46s work=6m36s fini=3s
>
> hotspot_compiler_2
>
> solaris_sparcv9-fastdebug-c2-hotspot_compiler_2 success(10m 43s)
> USED: hostname=sca00bwb platform=solaris_sparc_5.11 osname=solaris osarch=sparc cpus=6 parallelcount=6
> ram=31744MB cpufreqmhz=2848 instance=P1
> TIMING: clean=6s init=26s work=9m58s fini=7s
>
> solaris_sparcv9-fastdebug-c2-hotspot_compiler_2 success(10m 47s)
> USED: hostname=sca00bwa platform=solaris_sparc_5.11 osname=solaris osarch=sparc cpus=6 parallelcount=6
> ram=31744MB cpufreqmhz=2848 instance=P1
> TIMING: clean=6s init=28s work=10m2s fini=2s
>
> solaris_sparcv9-fastdebug-c2-hotspot_compiler_2 success(10m 56s)
> USED: hostname=sca00bvw platform=solaris_sparc_5.11 osname=solaris osarch=sparc cpus=6 parallelcount=6
> ram=31744MB cpufreqmhz=2848 instance=P1
> TIMING: clean=1s init=26s work=10m25s fini=2s
>
> hotspot_compiler_3
>
> solaris_sparcv9-fastdebug-c2-hotspot_compiler_3 success(11m 13s)
> USED: hostname=sc11152346 platform=solaris_sparc_5.11 osname=solaris osarch=sparc cpus=2 parallelcount=2
> ram=7900MB cpufreqmhz=2848 instance=T1
> TIMING: clean=2s init=34s work=10m31s fini=5s
>
> solaris_sparcv9-fastdebug-c2-hotspot_compiler_3 success(11m 27s)
> USED: hostname=sc11152481 platform=solaris_sparc_5.11 osname=solaris osarch=sparc cpus=2 parallelcount=2
> ram=7900MB cpufreqmhz=2848 instance=T1
> TIMING: clean=6s init=29s work=10m25s fini=17s
>
> solaris_sparcv9-fastdebug-c2-hotspot_compiler_3 success(07m 44s)
> USED: hostname=sca00bnq platform=solaris_sparc_5.11 osname=solaris osarch=sparc cpus=16 parallelcount=16
> ram=32768MB cpufreqmhz=3600 instance=P1
> TIMING: clean=2s init=46s work=6m42s fini=2s
>
>
> On the West queue work time is sometimes a bit above the 10 minute limit (at most 31 seconds). I don't think that is a
> problem, but please correct me if I'm wrong.
>
> Thank you and best regards,
>
>
> Zoltan
>
>
>>
>> Best,
>>
>>
>> Zoltan
>>
>>> Thanks,
>>> Vladimir
>>>
>>> On 9/5/14 7:17 AM, Zoltán Majó wrote:
>>>> Hi,
>>>>
>>>>
>>>> Thank you for the feedback.
>>>>
>>>> On 09/04/2014 09:33 PM, Vladimir Kozlov wrote:
>>>>> http://cr.openjdk.java.net/~zmajo/8050407/webrev.01/
>>>>>
>>>>> This looks better.
>>>>>
>>>>> On 9/4/14 1:42 AM, Albert wrote:
>>>>>> Hi,
>>>>>>
>>>>>> doesn't it make more sense to select tests based on their 'usefulness'?
>>>>>
>>>>> I agree. But we need to take into account its running time too.
>>>>> I assume Zoltan did a quick review of tests on usefulness.
>>>>
>>>> I went through the list of tests once more. I found 15 tests not that
>>>> useful (at least in my view), so I replaced them with 14 tests that I
>>>> found more useful.
>>>>
>>>> We have now 83 tests in total, here is the new webrev:
>>>>
>>>> http://cr.openjdk.java.net/~zmajo/8050407/webrev.02/
>>>>
>>>> I ran the tests three times on both Stockholm and West. Here are the
>>>> timings for the runs:
>>>>
>>>> WEST:
>>>>
>>>> solaris_sparcv9-fastdebug-c2-hotspot_compiler success(09m 49s)
>>>> USED: hostname=sca00bvw platform=solaris_sparc_5.11
>>>> osname=solaris osarch=sparc cpus=6 parallelcount=6 ram=31744MB
>>>> cpufreqmhz=2848 instance=P1
>>>>
>>>> solaris_sparcv9-fastdebug-c2-hotspot_compiler success(06m 36s)
>>>> USED: hostname=sca00bnp platform=solaris_sparc_5.11
>>>> osname=solaris osarch=sparc cpus=16 parallelcount=16 ram=32768MB
>>>> cpufreqmhz=3600 instance=P1
>>>>
>>>> solaris_sparcv9-fastdebug-c2-hotspot_compiler success(09m 32s)
>>>> USED: hostname=sc11152346 platform=solaris_sparc_5.11
>>>> osname=solaris osarch=sparc cpus=2 parallelcount=2 ram=7900MB
>>>> cpufreqmhz=2848 instance=T1
>>>>
>>>>
>>>> STOCKHOLM:
>>>>
>>>> solaris_sparcv9-fastdebug-c2-hotspot_compiler success(08m 05s)
>>>> USED: hostname=sthjprtsol40 platform=solaris_sparc_5.11
>>>> osname=solaris osarch=sparc cpus=12 parallelcount=12 ram=24576MB
>>>> cpufreqmhz=2848 instance=P1
>>>>
>>>> solaris_sparcv9-fastdebug-c2-hotspot_compiler success(08m 27s)
>>>> USED: hostname=sthjprtsol32 platform=solaris_sparc_5.11
>>>> osname=solaris osarch=sparc cpus=12 parallelcount=12 ram=24576MB
>>>> cpufreqmhz=2848 instance=P1
>>>>
>>>> solaris_sparcv9-fastdebug-c2-hotspot_compiler success(08m 14s)
>>>> USED: hostname=sthjprtsol38 platform=solaris_sparc_5.11
>>>> osname=solaris osarch=sparc cpus=12 parallelcount=12 ram=24576MB
>>>> cpufreqmhz=2848 instance=P1
>>>>
>>>>
>>>> We get close to the 10-minute limit on both queues (9:49 on West and
>>>> 8:27 on Stockholm).
>>>>
>>>> Thank you and best regards,
>>>>
>>>>
>>>> Zoltan
>>>>
>>>>>
>>>>>>
>>>>>> For example, it seems to me that adding a regression test that was
>>>>>> introduced
>>>>>> based on a fix of a segmentation fault in compiled code makes more sense
>>>>>> than adding a test that checks the consistency of rarely used command
>>>>>> line
>>>>>> arguments.
>>>>>
>>>>> Agree. Which tests in the list do flags testing?
>>>>>
>>>>> We should also exclude unstable tests (have to look on history of the
>>>>> test).
>>>>>
>>>>>>
>>>>>> How about adding tests based on their priorities? Maybe it makes more
>>>>>> sense
>>>>>> to add a test of a corresponding P1 bug than adding a test of a P5 bug?
>>>>>
>>>>> We can't add P1 test if it takes more then 1 min to run. I would
>>>>> prefer to run 5 x 10 sec P3 tests instead. We would test more code
>>>>> this way.
>>>>> Also Bug's priority was not accurate before as it is now.
>>>>>
>>>>> Thanks,
>>>>> Vladimir
>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> Albert
>>>>>>
>>>>>> On 08/28/2014 08:02 PM, Zoltán Majó wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>>
>>>>>>> please review the following patch.
>>>>>>>
>>>>>>>
>>>>>>> Bug: Add jtreg compiler tests to Hotspot JPRT jobs
>>>>>>>
>>>>>>>
>>>>>>> Problem: The test/TEST.groups file lists JTREG compiler tests executed
>>>>>>> in JPRT (target name: 'hotspot_compiler'). We need to determine the
>>>>>>> list of tests from test/compiler that should be executed in JPRT. The
>>>>>>> total time of execution should be less than 10 minutes on slowest
>>>>>>> platform.
>>>>>>>
>>>>>>>
>>>>>>> Solution: The slowest platform in JPRT is currently solaris_sparcv9. I
>>>>>>> executed all open JTREG tests from test/compiler on solaris_sparcv9
>>>>>>> and measured the "work time" of each test. Then, tests were sorted in
>>>>>>> ascending order of their work time. To construct the subset, I first
>>>>>>> added the test with the lowest work time to the subset and then
>>>>>>> continued adding tests until a time limit L is reached.
>>>>>>>
>>>>>>> Limit L is set to 80% of the original time budget (10 minutes). L is
>>>>>>> set conservatively to account also for JPRT "cleanup", "init", and
>>>>>>> "finishing" time, as well as for the variation of tests' "work time".
>>>>>>> (The profiling measurements contain only "work time".)
>>>>>>>
>>>>>>> The subset contains 77 tests. The longest executing test in the subset
>>>>>>> takes 6.9 seconds.
>>>>>>>
>>>>>>>
>>>>>>> Webrev: http://cr.openjdk.java.net/~zmajo/8050407/webrev.00/
>>>>>>>
>>>>>>>
>>>>>>> Testing: Ran subset on all platforms in both west and stockholm JPRT.
>>>>>>> Execution times of tests are as follows:
>>>>>>>
>>>>>>> West:
>>>>>>> linux_i586-fastdebug-c1-hotspot_compiler success(03m 01s)
>>>>>>> linux_i586-fastdebug-c2-hotspot_compiler success(03m 39s)
>>>>>>> linux_x64-fastdebug-c2-hotspot_compiler success(03m 32s)
>>>>>>> macosx_x64-fastdebug-c2-hotspot_compiler success(03m 48s)
>>>>>>> solaris_sparcv9-fastdebug-c2-hotspot_compiler success(08m 41s)
>>>>>>> solaris_x64-fastdebug-c2-hotspot_compiler success(03m 00s)
>>>>>>> windows_i586-fastdebug-c1-hotspot_compiler success(03m 39s)
>>>>>>> windows_i586-fastdebug-c2-hotspot_compiler success(03m 54s)
>>>>>>> windows_x64-fastdebug-c2-hotspot_compiler success(04m 51s)
>>>>>>>
>>>>>>> Stockholm:
>>>>>>> linux_i586-fastdebug-c1-hotspot_compiler success(02m 31s)
>>>>>>> linux_i586-fastdebug-c2-hotspot_compiler success(02m 39s)
>>>>>>> linux_x64-fastdebug-c2-hotspot_compiler success(02m 53s)
>>>>>>> macosx_x64-fastdebug-c2-hotspot_compiler success(04m 10s)
>>>>>>> solaris_sparcv9-fastdebug-c2-hotspot_compiler success(05m 55s)
>>>>>>> solaris_x64-fastdebug-c2-hotspot_compiler success(03m 06s)
>>>>>>> windows_i586-fastdebug-c1-hotspot_compiler success(02m 34s)
>>>>>>> windows_i586-fastdebug-c2-hotspot_compiler success(03m 07s)
>>>>>>> windows_x64-fastdebug-c2-hotspot_compiler success(04m 33s)
>>>>>>>
>>>>>>> We get close to the 10 minute budget (87% usage on West for
>>>>>>> solaris_sparcv9).
>>>>>>>
>>>>>>>
>>>>>>> Thank you and best regards,
>>>>>>>
>>>>>>>
>>>>>>> Zoltan
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>
>
More information about the hotspot-compiler-dev
mailing list