Are We Fast Yet? Comparing Language Implementations with Objects, Closures, and Arrays

[Stefan Marr]

Hi:

One question Truffle language implementers like I might have is: is my language implementation actually fast?

To help us to answer this question at least to some degree, I started to build a set of benchmarks for Java, JavaScript, Ruby, and my SOMns.

The main idea is to use a common ‘core’ language that is portable across languages and relies mainly on objects, closures, and arrays. For me the goal was to have the benchmarks as identical as possible between the languages to identify optimization opportunities on the level of the Truffle AST.

So far, that was pretty successful, I was able to speed up SOMns quite a bit, and a performance bug in Graal’s array handling was fixed as well to solve issues in JRuby+Truffle. 

More details of the project and the benchmarks are available here:
 https://github.com/smarr/are-we-fast-yet

The benchmarks should be portable to a wide range of dynamic languages for instance also Python or Lua.
Whether they are applicable to other types of languages is still an open question. But it would be interesting to see whether there are somewhat idiomatic implementations possible that still are comparable.

So, if you got a Truffle-based language, I’d be highly interested to hear whether this could be useful for you.
Feedback on the approach and the set of benchmarks is highly welcome, too.

Best regards
Stefan

-- 
Stefan Marr
Johannes Kepler Universität Linz
http://stefan-marr.de/research/