RFR JDK-8059510 Compact symbol table layout inside shared archive

[David Holmes]

On 14/10/2014 2:17 AM, Jiangli Zhou wrote:
> Hi David and Gerard,
>
> On 10/13/2014 07:59 AM, Gerard Ziemski wrote:
>> hi David,
>>
>> On 10/12/2014 8:32 PM, David Holmes wrote:
>>>> ==============================================================================
>>>>
>>>>
>>>> logs.specjvm.before:
>>>>    Benchmark           Samples        Mean     Stdev Geomean Weight
>>>>    specjvm98                 8       81.33      1.47
>>>> ==============================================================================
>>>>
>>>>
>>>> logs.specjvm.after:
>>>>    Benchmark           Samples        Mean     Stdev   %Diff P
>>>> Significant
>>>>    specjvm98                 8       81.72      0.70    0.48
>>>> 0.509            *
>>>> ==============================================================================
>>>>
>>>
>>> Sample size is too small to give meaningful results.
>>
>> 8 is a too small sample size?
>>
>> I'm running refworkload benchmarks myself, so I'm curious - what is a
>> meaningful sample size for refworkload benchmarks?
>
> 8 is definitely not small for specjvm. In some cases, 3 is enough.

I was always told to run at least 10 samples. YMMV.

With regard to the machine, we can easily see large variances in 
executions time through other system activity. When trying to compare 
your results against someone else's it help to establish your testing 
environment is up to the task.

David

> Thanks,
> Jiangli
>
>>
>>
>> cheers
>