Graal benchmarking

[Aleksandar Prokopec]

Hi Konstantin,

There is a standard way to run the benchmarks used in Graal with our 
build tool mx.
You need to run the following for DaCapo benchmarks:

$ mx build

$ mx benchmark 'dacapo:*'

To run a specific benchmark, e.g. fop, you need to do:

$ mx benchmark dacapo:fop

To get a detailed list of all the options and benchmark suites, you can run:

$ mx benchmark --help

These benchmarks will typically show you the running times. To see other 
things, such as the generated codesize, you will need to activate one of 
Graal's debug counters or output VM statistics.

$ mx benchmark dacapo:fop -- -Dgraal.Count=

...

     |-> BytecodesParsed=18058
     |-> CE_ImprovedPhis=39
     |-> CE_KilledIfs=16
     |-> CanonicalizationConsideredNodes=56611
     |-> CanonicalizedNodes=3398
     |-> CompiledAndInstalledBytecodes=17619
     |-> CompiledBytecodes=17619

...

Since you want to track code size after register allocation, an even 
better way is to use VM flags:

$ mx benchmark dacapo:fop -- -XX:+PrintNMethodStatistics

You should look for JVMCI-compiled methods here (not C1, that's a 
different compiler):

...
Statistics for 315 bytecoded nmethods for JVMCI:
  total in heap  = 487096
  header         = 98280
  relocation     = 15816
  main code      = 249815
  stub code      = 4753
  oops           = 1552
  metadata       = 14352
  scopes data    = 48256
  scopes pcs     = 47328
  dependencies   = 3232
  handler table  = 192
...

Alternatively, this option will give you some additional information 
about the total compiled bytecode size and the total compiled code size:

$ mx benchmark dacapo:fop -- -XX:+CITime

Hope this helps,
Alex


On 11.06.2017 21:55, Новиков, Константин Михайлович wrote:
> Hello, everyone
>
> I'm working on graal register allocation optimization and have troubles
> with testing. I've seen many presentations that tell how to build and run
> distinct tests, but they don't describe how to run whole suits, only
> results of such testings.
> Is there the way to run for example a DaCapo suit using mx or other tool?
> How could one test not only performance, but change in generated codesize?
>
> P.S. graaltest also fails to give correct output because clean repository
> fails in four tests and it uses seconds as the unit of measurement
>
> Thankfully,
> Bachelor of Moscow Institute of Physics and Technology
> Konstantin Novikov