Poor performance with invokedynamic on Graal

[Thomas Wuerthinger]

Hi Charlie!

Thanks for the regression report. Our policy with Graal is to not accept any significant peak slow-down on realistic workloads compared to C2. We will create a ticket to track progress. Can you also specify the version of the JDK used with Graal (i.e., the full “-version” output of the JVM)?

We did not (wittingly) change the behavior of Graal related to invokedynamic. Maybe some VM internals have changed that we did not yet adopt for.

Obviously, we recommend on the long run to use Truffle+JRuby. We see fairly convincing numbers with Truffle enabled for the mentioned red-black tree benchmark on our performance tracking infrastructure.

- thomas


> On 06 Aug 2016, at 03:35, Charles Oliver Nutter <headius at headius.com> wrote:
> 
> Hey all!
> 
> I've started to poke at running JRuby on Graal and I've been pretty
> disappointed with the results so far. It appears that Graal is not doing a
> good job of optimizing through invokedynamic call sites, which largely
> defeats our best opportunity for optimization.
> 
> I've tried two benchmarks today against graal-core HEAD.
> 
> The first is trivial: a recursive fib benchmark. JRuby+indy on Hotspot
> (8u92) runs fib(35) in around 0.32s. JRuby+indy on Graal takes 0.52s or so
> per iteration.
> 
> https://gist.github.com/headius/492fb995e15b50d3db42145339f21968
> 
> The second benchmark is more robust: a pure-Ruby red/black tree
> implementation. The benchmark builds up a large tree, searches it, deletes
> from it, etc. JRuby+indy on Hotspot can run an iteration in about 0.41s. On
> Graal, it takes 1s or more.
> 
> This is all especially unfortunate because some time ago I was able to see
> a 5x improvement on a mandelbrot benchmark. I don't see that on recent
> Graal builds.
> 
> Here's how to reproduce...
> 
> * Grab a master build of JRuby from http://ci.jruby.org. Look for
> 9.1.3.0-snapshot "bin" tarball.
> * Unpack, add bin/ to PATH so JRuby gets picked up.
> * Run the benchmarks I've gisted below.
> 
> https://gist.github.com/headius/5eb47848fbe486266942695891b0d9f6
> 
> Let me know if you have question! I want to see this run as well as I
> believe it should!
> 
> - Charlie