Poor performance with invokedynamic on Graal

[Doug Simon]

Hi Charlie,

I’ve created an issue to track our investigation into the reported performance issue:

https://github.com/graalvm/graal-core/issues/202

Feel free to add an extra discoveries you make there.

Thanks,

-Doug

> On 08 Aug 2016, at 19:49, Thomas Wuerthinger <thomas.wuerthinger at oracle.com> wrote:
> 
> Hi Charlie!
> 
> Thanks for the regression report. Our policy with Graal is to not accept any significant peak slow-down on realistic workloads compared to C2. We will create a ticket to track progress. Can you also specify the version of the JDK used with Graal (i.e., the full “-version” output of the JVM)?
> 
> We did not (wittingly) change the behavior of Graal related to invokedynamic. Maybe some VM internals have changed that we did not yet adopt for.
> 
> Obviously, we recommend on the long run to use Truffle+JRuby. We see fairly convincing numbers with Truffle enabled for the mentioned red-black tree benchmark on our performance tracking infrastructure.
> 
> - thomas
> 
> 
>> On 06 Aug 2016, at 03:35, Charles Oliver Nutter <headius at headius.com> wrote:
>> 
>> Hey all!
>> 
>> I've started to poke at running JRuby on Graal and I've been pretty
>> disappointed with the results so far. It appears that Graal is not doing a
>> good job of optimizing through invokedynamic call sites, which largely
>> defeats our best opportunity for optimization.
>> 
>> I've tried two benchmarks today against graal-core HEAD.
>> 
>> The first is trivial: a recursive fib benchmark. JRuby+indy on Hotspot
>> (8u92) runs fib(35) in around 0.32s. JRuby+indy on Graal takes 0.52s or so
>> per iteration.
>> 
>> https://gist.github.com/headius/492fb995e15b50d3db42145339f21968
>> 
>> The second benchmark is more robust: a pure-Ruby red/black tree
>> implementation. The benchmark builds up a large tree, searches it, deletes
>> from it, etc. JRuby+indy on Hotspot can run an iteration in about 0.41s. On
>> Graal, it takes 1s or more.
>> 
>> This is all especially unfortunate because some time ago I was able to see
>> a 5x improvement on a mandelbrot benchmark. I don't see that on recent
>> Graal builds.
>> 
>> Here's how to reproduce...
>> 
>> * Grab a master build of JRuby from http://ci.jruby.org. Look for
>> 9.1.3.0-snapshot "bin" tarball.
>> * Unpack, add bin/ to PATH so JRuby gets picked up.
>> * Run the benchmarks I've gisted below.
>> 
>> https://gist.github.com/headius/5eb47848fbe486266942695891b0d9f6
>> 
>> Let me know if you have question! I want to see this run as well as I
>> believe it should!
>> 
>> - Charlie
>