[9] RFR(S): 8050407: Add jtreg compiler tests to Hotspot JPRT jobs
Zoltán Majó
zoltan.majo at oracle.com
Thu Sep 11 15:24:20 UTC 2014
Hi,
On 09/10/2014 11:46 AM, Zoltán Majó wrote:
> On 09/05/2014 08:34 PM, Vladimir Kozlov wrote:
>> Zoltan,
>>
>> Christian Tornqvist gave a great suggestion in chat room. We can
>> split tests in several sets each taking no more than 10 min. Runtime
>> has hotspot_runtime and hotspot_runtime_closed. So we can have
>> several hotspot_compiler_tests1, etc too.
>>
>> Could you, please, investigate this?
>
> Sure, I'll look into it.
I created 3 subsets, hotspot_compiler_[1-3]. The subsets include all
quick-running JTREG open tests (test with < 10 seconds execution time)
that are useful (at least in my view). There are 83/69/58 tests in
subset 1/2/3.
Here is the new webrev:
http://cr.openjdk.java.net/~zmajo/8050407/webrev.03/
Below are the execution times ("work time") for all 3 subsets. There are
3 runs for each subset on both Stockholm and West queues.
STOCKHOLM:
hotspot_compiler_1
solaris_sparcv9-fastdebug-c2-hotspot_compiler_1 success(08m 22s)
USED: hostname=sthjprtsol33 platform=solaris_sparc_5.11
osname=solaris osarch=sparc cpus=12 parallelcount=12 ram=24576MB
cpufreqmhz=2848 instance=P1
TIMING: clean=1s init=27s work=7m52s fini=1s
solaris_sparcv9-fastdebug-c2-hotspot_compiler_1 success(09m 11s)
USED: hostname=sthjprtsol35 platform=solaris_sparc_5.11
osname=solaris osarch=sparc cpus=12 parallelcount=12 ram=24576MB
cpufreqmhz=2848 instance=P1
TIMING: clean=1s init=26s work=8m41s fini=1s
solaris_sparcv9-fastdebug-c2-hotspot_compiler_1 success(08m 36s)
USED: hostname=sthjprtsol40 platform=solaris_sparc_5.11
osname=solaris osarch=sparc cpus=12 parallelcount=12 ram=24576MB
cpufreqmhz=2848 instance=P1
TIMING: clean=2s init=21s work=8m10s fini=1s
hotspot_compiler_2
solaris_sparcv9-fastdebug-c2-hotspot_compiler_2 success(09m 33s)
USED: hostname=sthjprtsol41 platform=solaris_sparc_5.11
osname=solaris osarch=sparc cpus=12 parallelcount=12 ram=24576MB
cpufreqmhz=2848 instance=P1
TIMING: clean=2s init=27s work=9m2s fini=1s
solaris_sparcv9-fastdebug-c2-hotspot_compiler_2 success(09m 27s)
USED: hostname=sthjprtsol42 platform=solaris_sparc_5.11
osname=solaris osarch=sparc cpus=12 parallelcount=12 ram=24576MB
cpufreqmhz=2848 instance=P1
TIMING: clean=6s init=14s work=9m2s fini=1s
solaris_sparcv9-fastdebug-c2-hotspot_compiler_2 success(08m 38s)
USED: hostname=sthjprtsol45 platform=solaris_sparc_5.11
osname=solaris osarch=sparc cpus=12 parallelcount=12 ram=24576MB
cpufreqmhz=2848 instance=P1
TIMING: clean=1s init=22s work=8m13s fini=1s
hotspot_compiler_3
solaris_sparcv9-fastdebug-c2-hotspot_compiler_3 success(09m 20s)
USED: hostname=sthjprtsol43 platform=solaris_sparc_5.11
osname=solaris osarch=sparc cpus=12 parallelcount=12 ram=24576MB
cpufreqmhz=2848 instance=P1
TIMING: clean=1s init=26s work=8m49s fini=1s
solaris_sparcv9-fastdebug-c2-hotspot_compiler_3 success(09m 09s)
USED: hostname=sthjprtsol48 platform=solaris_sparc_5.11
osname=solaris osarch=sparc cpus=12 parallelcount=12 ram=24576MB
cpufreqmhz=2848 instance=P1
TIMING: clean=1s init=18s work=8m46s fini=1s
solaris_sparcv9-fastdebug-c2-hotspot_compiler_3 success(09m 16s)
USED: hostname=sthjprtsol34 platform=solaris_sparc_5.11
osname=solaris osarch=sparc cpus=12 parallelcount=12 ram=24576MB
cpufreqmhz=2848 instance=P1
TIMING: clean=2s init=21s work=8m51s fini=1s
WEST:
hotspot_compiler_1
solaris_sparcv9-fastdebug-c2-hotspot_compiler_1 success(09m 55s)
USED: hostname=sc11152412 platform=solaris_sparc_5.11
osname=solaris osarch=sparc cpus=2 parallelcount=2 ram=7900MB
cpufreqmhz=2848 instance=T1
TIMING: clean=11s init=50s work=8m45s fini=2s
solaris_sparcv9-fastdebug-c2-hotspot_compiler_1 success(10m 59s)
USED: hostname=sca00bvx platform=solaris_sparc_5.11
osname=solaris osarch=sparc cpus=6 parallelcount=6 ram=31744MB
cpufreqmhz=2848 instance=P1
TIMING: clean=6s init=42s work=10m3s fini=2s
solaris_sparcv9-fastdebug-c2-hotspot_compiler_1 success(07m 35s)
USED: hostname=sca00bnr platform=solaris_sparc_5.11
osname=solaris osarch=sparc cpus=16 parallelcount=16 ram=32768MB
cpufreqmhz=3600 instance=P1
TIMING: clean=3s init=46s work=6m36s fini=3s
hotspot_compiler_2
solaris_sparcv9-fastdebug-c2-hotspot_compiler_2 success(10m 43s)
USED: hostname=sca00bwb platform=solaris_sparc_5.11
osname=solaris osarch=sparc cpus=6 parallelcount=6 ram=31744MB
cpufreqmhz=2848 instance=P1
TIMING: clean=6s init=26s work=9m58s fini=7s
solaris_sparcv9-fastdebug-c2-hotspot_compiler_2 success(10m 47s)
USED: hostname=sca00bwa platform=solaris_sparc_5.11
osname=solaris osarch=sparc cpus=6 parallelcount=6 ram=31744MB
cpufreqmhz=2848 instance=P1
TIMING: clean=6s init=28s work=10m2s fini=2s
solaris_sparcv9-fastdebug-c2-hotspot_compiler_2 success(10m 56s)
USED: hostname=sca00bvw platform=solaris_sparc_5.11
osname=solaris osarch=sparc cpus=6 parallelcount=6 ram=31744MB
cpufreqmhz=2848 instance=P1
TIMING: clean=1s init=26s work=10m25s fini=2s
hotspot_compiler_3
solaris_sparcv9-fastdebug-c2-hotspot_compiler_3 success(11m 13s)
USED: hostname=sc11152346 platform=solaris_sparc_5.11
osname=solaris osarch=sparc cpus=2 parallelcount=2 ram=7900MB
cpufreqmhz=2848 instance=T1
TIMING: clean=2s init=34s work=10m31s fini=5s
solaris_sparcv9-fastdebug-c2-hotspot_compiler_3 success(11m 27s)
USED: hostname=sc11152481 platform=solaris_sparc_5.11
osname=solaris osarch=sparc cpus=2 parallelcount=2 ram=7900MB
cpufreqmhz=2848 instance=T1
TIMING: clean=6s init=29s work=10m25s fini=17s
solaris_sparcv9-fastdebug-c2-hotspot_compiler_3 success(07m 44s)
USED: hostname=sca00bnq platform=solaris_sparc_5.11
osname=solaris osarch=sparc cpus=16 parallelcount=16 ram=32768MB
cpufreqmhz=3600 instance=P1
TIMING: clean=2s init=46s work=6m42s fini=2s
On the West queue work time is sometimes a bit above the 10 minute limit
(at most 31 seconds). I don't think that is a problem, but please
correct me if I'm wrong.
Thank you and best regards,
Zoltan
>
> Best,
>
>
> Zoltan
>
>> Thanks,
>> Vladimir
>>
>> On 9/5/14 7:17 AM, Zoltán Majó wrote:
>>> Hi,
>>>
>>>
>>> Thank you for the feedback.
>>>
>>> On 09/04/2014 09:33 PM, Vladimir Kozlov wrote:
>>>> http://cr.openjdk.java.net/~zmajo/8050407/webrev.01/
>>>>
>>>> This looks better.
>>>>
>>>> On 9/4/14 1:42 AM, Albert wrote:
>>>>> Hi,
>>>>>
>>>>> doesn't it make more sense to select tests based on their
>>>>> 'usefulness'?
>>>>
>>>> I agree. But we need to take into account its running time too.
>>>> I assume Zoltan did a quick review of tests on usefulness.
>>>
>>> I went through the list of tests once more. I found 15 tests not that
>>> useful (at least in my view), so I replaced them with 14 tests that I
>>> found more useful.
>>>
>>> We have now 83 tests in total, here is the new webrev:
>>>
>>> http://cr.openjdk.java.net/~zmajo/8050407/webrev.02/
>>>
>>> I ran the tests three times on both Stockholm and West. Here are the
>>> timings for the runs:
>>>
>>> WEST:
>>>
>>> solaris_sparcv9-fastdebug-c2-hotspot_compiler success(09m 49s)
>>> USED: hostname=sca00bvw platform=solaris_sparc_5.11
>>> osname=solaris osarch=sparc cpus=6 parallelcount=6 ram=31744MB
>>> cpufreqmhz=2848 instance=P1
>>>
>>> solaris_sparcv9-fastdebug-c2-hotspot_compiler success(06m 36s)
>>> USED: hostname=sca00bnp platform=solaris_sparc_5.11
>>> osname=solaris osarch=sparc cpus=16 parallelcount=16 ram=32768MB
>>> cpufreqmhz=3600 instance=P1
>>>
>>> solaris_sparcv9-fastdebug-c2-hotspot_compiler success(09m 32s)
>>> USED: hostname=sc11152346 platform=solaris_sparc_5.11
>>> osname=solaris osarch=sparc cpus=2 parallelcount=2 ram=7900MB
>>> cpufreqmhz=2848 instance=T1
>>>
>>>
>>> STOCKHOLM:
>>>
>>> solaris_sparcv9-fastdebug-c2-hotspot_compiler success(08m 05s)
>>> USED: hostname=sthjprtsol40 platform=solaris_sparc_5.11
>>> osname=solaris osarch=sparc cpus=12 parallelcount=12 ram=24576MB
>>> cpufreqmhz=2848 instance=P1
>>>
>>> solaris_sparcv9-fastdebug-c2-hotspot_compiler success(08m 27s)
>>> USED: hostname=sthjprtsol32 platform=solaris_sparc_5.11
>>> osname=solaris osarch=sparc cpus=12 parallelcount=12 ram=24576MB
>>> cpufreqmhz=2848 instance=P1
>>>
>>> solaris_sparcv9-fastdebug-c2-hotspot_compiler success(08m 14s)
>>> USED: hostname=sthjprtsol38 platform=solaris_sparc_5.11
>>> osname=solaris osarch=sparc cpus=12 parallelcount=12 ram=24576MB
>>> cpufreqmhz=2848 instance=P1
>>>
>>>
>>> We get close to the 10-minute limit on both queues (9:49 on West and
>>> 8:27 on Stockholm).
>>>
>>> Thank you and best regards,
>>>
>>>
>>> Zoltan
>>>
>>>>
>>>>>
>>>>> For example, it seems to me that adding a regression test that was
>>>>> introduced
>>>>> based on a fix of a segmentation fault in compiled code makes more
>>>>> sense
>>>>> than adding a test that checks the consistency of rarely used command
>>>>> line
>>>>> arguments.
>>>>
>>>> Agree. Which tests in the list do flags testing?
>>>>
>>>> We should also exclude unstable tests (have to look on history of the
>>>> test).
>>>>
>>>>>
>>>>> How about adding tests based on their priorities? Maybe it makes more
>>>>> sense
>>>>> to add a test of a corresponding P1 bug than adding a test of a P5
>>>>> bug?
>>>>
>>>> We can't add P1 test if it takes more then 1 min to run. I would
>>>> prefer to run 5 x 10 sec P3 tests instead. We would test more code
>>>> this way.
>>>> Also Bug's priority was not accurate before as it is now.
>>>>
>>>> Thanks,
>>>> Vladimir
>>>>
>>>>>
>>>>> Thanks,
>>>>> Albert
>>>>>
>>>>> On 08/28/2014 08:02 PM, Zoltán Majó wrote:
>>>>>> Hi,
>>>>>>
>>>>>>
>>>>>> please review the following patch.
>>>>>>
>>>>>>
>>>>>> Bug: Add jtreg compiler tests to Hotspot JPRT jobs
>>>>>>
>>>>>>
>>>>>> Problem: The test/TEST.groups file lists JTREG compiler tests
>>>>>> executed
>>>>>> in JPRT (target name: 'hotspot_compiler'). We need to determine the
>>>>>> list of tests from test/compiler that should be executed in JPRT.
>>>>>> The
>>>>>> total time of execution should be less than 10 minutes on slowest
>>>>>> platform.
>>>>>>
>>>>>>
>>>>>> Solution: The slowest platform in JPRT is currently
>>>>>> solaris_sparcv9. I
>>>>>> executed all open JTREG tests from test/compiler on solaris_sparcv9
>>>>>> and measured the "work time" of each test. Then, tests were
>>>>>> sorted in
>>>>>> ascending order of their work time. To construct the subset, I first
>>>>>> added the test with the lowest work time to the subset and then
>>>>>> continued adding tests until a time limit L is reached.
>>>>>>
>>>>>> Limit L is set to 80% of the original time budget (10 minutes). L is
>>>>>> set conservatively to account also for JPRT "cleanup", "init", and
>>>>>> "finishing" time, as well as for the variation of tests' "work
>>>>>> time".
>>>>>> (The profiling measurements contain only "work time".)
>>>>>>
>>>>>> The subset contains 77 tests. The longest executing test in the
>>>>>> subset
>>>>>> takes 6.9 seconds.
>>>>>>
>>>>>>
>>>>>> Webrev: http://cr.openjdk.java.net/~zmajo/8050407/webrev.00/
>>>>>>
>>>>>>
>>>>>> Testing: Ran subset on all platforms in both west and stockholm
>>>>>> JPRT.
>>>>>> Execution times of tests are as follows:
>>>>>>
>>>>>> West:
>>>>>> linux_i586-fastdebug-c1-hotspot_compiler success(03m 01s)
>>>>>> linux_i586-fastdebug-c2-hotspot_compiler success(03m 39s)
>>>>>> linux_x64-fastdebug-c2-hotspot_compiler success(03m 32s)
>>>>>> macosx_x64-fastdebug-c2-hotspot_compiler success(03m 48s)
>>>>>> solaris_sparcv9-fastdebug-c2-hotspot_compiler success(08m 41s)
>>>>>> solaris_x64-fastdebug-c2-hotspot_compiler success(03m 00s)
>>>>>> windows_i586-fastdebug-c1-hotspot_compiler success(03m 39s)
>>>>>> windows_i586-fastdebug-c2-hotspot_compiler success(03m 54s)
>>>>>> windows_x64-fastdebug-c2-hotspot_compiler success(04m 51s)
>>>>>>
>>>>>> Stockholm:
>>>>>> linux_i586-fastdebug-c1-hotspot_compiler success(02m 31s)
>>>>>> linux_i586-fastdebug-c2-hotspot_compiler success(02m 39s)
>>>>>> linux_x64-fastdebug-c2-hotspot_compiler success(02m 53s)
>>>>>> macosx_x64-fastdebug-c2-hotspot_compiler success(04m 10s)
>>>>>> solaris_sparcv9-fastdebug-c2-hotspot_compiler success(05m 55s)
>>>>>> solaris_x64-fastdebug-c2-hotspot_compiler success(03m 06s)
>>>>>> windows_i586-fastdebug-c1-hotspot_compiler success(02m 34s)
>>>>>> windows_i586-fastdebug-c2-hotspot_compiler success(03m 07s)
>>>>>> windows_x64-fastdebug-c2-hotspot_compiler success(04m 33s)
>>>>>>
>>>>>> We get close to the 10 minute budget (87% usage on West for
>>>>>> solaris_sparcv9).
>>>>>>
>>>>>>
>>>>>> Thank you and best regards,
>>>>>>
>>>>>>
>>>>>> Zoltan
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>
>
More information about the hotspot-compiler-dev
mailing list