Truffle language benchmarks

[Arthur Peters]

Hello there,

I'm trying to decide how to focus programming efforts in my Truffle
based research language (Orc/PorcE [0]
<https://github.com/orc-lang/orc/tree/master/PorcE>). So I'm looking for
some performance numbers. Any guidance would be appreciated.

What I want is an order-of-magnitude estimate of how much
dispatch-into-Java performance will improve if I rewrite the dispatch to
be entirely visible to the Truffle partial evaluator. By dispatch I mean
collecting and converting arguments and then invoking the target method.
Currently, I'm using old dispatch code from our interpreted
implementation. [1]

Does anyone have a sense for how much I could expect to gain? (in terms
of dispatch time reduction, not in terms of overall performance, obviously)

If not, I plan to estimate based on benchmarks comparing Truffle
language implementations with traditional Java implementations of those
language. Any benchmark will do, since I just need to get a general
sense. I found a comparison for Ruby [2]
<https://pragtob.wordpress.com/2017/01/24/benchmarking-a-go-ai-in-ruby-cruby-vs-rubinius-vs-jruby-vs-truffle-a-year-later/>.
I was not able to find comparisons for ZipPy vs Jython, or Rhino vs
Nashorn vs Graal.JS.

Does anyone know of such numbers (Python, JS)? I don't need them for
formal comparison in a paper or anything so preliminary numbers are
totally acceptable. I won't distribute them farther.

Regardless of my needs, it might be pretty useful to have a repository
of performance numbers from Truffle languages and other implementations.
It would be quite interesting from a historical prospective.

Thanks!!!
-Arthur


[0] https://github.com/orc-lang/orc/tree/master/PorcE

[1] I want to maintain some degree of backwards compatibility, so just
switching to Truffle Interop is not a good option. Also I want to
maintain the same value conversion semantics as our old implementation
if possible.

[2]
https://pragtob.wordpress.com/2017/01/24/benchmarking-a-go-ai-in-ruby-cruby-vs-rubinius-vs-jruby-vs-truffle-a-year-later/