[aarch64-port-dev ] aarch64 and arm64 jdk benchmarks result sharing

[Zhongwei Yao]

Hi, Stuart,
Thanks for your suggestions.

The tests were running on dedicated clean machines. I don't think there
were any non-negligible background processes. The physical memory is large
enough (>32G), but we run this benchmark with default JVM heap
configurations.

We updated kernel/firmware in one of the test machines, and the result of
these Dacapo benchmarks looks more stable, but still with relative high
standard deviation.


On 20 April 2017 at 18:39, Stuart Monteith <stuart.monteith at linaro.org>
wrote:

> Zhongwei - I'd like you to check the OS as it is running. Are there
> any background processes running?



> What is the memory pressure like?
>


> Are there any errors in the kernel logs?



> Does a pristine machine
> exhibit the same behaviour?



> I'm concerned there might not be the
> controls necessary to get clean results.
>
> BR,
>    Stuart
>
>
> On 20 April 2017 at 10:07, Andrew Haley <aph at redhat.com> wrote:
> > On 20/04/17 03:03, Zhongwei Yao wrote:
> >> On 19 April 2017 at 19:30, Andrew Haley <aph at redhat.com> wrote:
> >>
> >>> Looking at this some more, I can't tell which benchmarks are
> >>> throughput (big is good) and which are time (small is good).  This
> >>> makes it very hard for me to see where community AArch64 is lagging
> >>> performance.
> >>>
> >>> I think Dacapo is throughput?  And TeraSort is time?
> >>>
> >>> The results for Partner B's hardware on Dacapo are so weird that I
> >>> think there must be something wrong.  lusearch didn't work at all, so
> >>> there must be a code generation problem somewhere.
> >>
> >> On our Partner B's hardware, lusearch could run but it failed to
> converge
> >> at warmup stage at every time. So the result is set to 0 in our result.
> >>
> >> And for sunflow case, it is in similar case like lusearch. Sunflow could
> >> run but it failed to converge at most time. I've checked it succeeds to
> >> converge only once in our result. So the standard deviation is 0
> because of
> >> there are only one. I should have added note for this case. Sorry for
> >> confusion.
> >
> > OK, so I think we have to reject all of the results on Partner B's
> hardware
> > because something is severely broken.
> >
> > Andrew.
> >
>



-- 
Best regards,
Zhongwei