Dynamic languages on the JVM / jmh

[Thomas Wuerthinger]

Moving this to jmh-dev then. The discussion topic is whether support for dynamic language on the JVM to include jmh APIs can be useful and how this could be added.

With JavaScript as the example: One could imagine providing a clojure with the benchmark code together with a JSON object with configuration data for executing a benchmark. Internally, the clojure could be represented via a method handle and the configuration data via a hash map. This would be the scenario (a) that you outline below. I believe that it would be easier to use and also less likely to measure additional overheads such as the javax.script implementation of the dynamic language. What I read from your message is that you feel that this is out of scope for JMH.

- thomas


On 23 Sep 2014, at 17:36, Marcus Lagergren <marcus.lagergren at oracle.com> wrote:

> Guys, guys! This is my review thread for JDK-8025435. Start another one for this, please…
> 
> /M
> 
> On 23 Sep 2014, at 17:29, Aleksey Shipilev <aleksey.shipilev at oracle.com> wrote:
> 
>> On 09/23/2014 05:58 PM, Thomas Wuerthinger wrote:
>>> I understood your remark. I just believe that this restriction is not
>>> fundamental to JMH and there *could* be support for languages that do
>>> not statically compile to Java bytecodes as well.
>> 
>> I think this thread side-tracks.
>> 
>> Bottom-line: There are only two ways to hook up dynamically-compiled
>> language runtime into JMH (say, JS engine), depending who is in control:
>> 
>> a) Ask JS engine to start executing the JS benchmark, *and* call back
>> into JMH with an AST/bytecode/whatever in hands, asking to measure it.
>> We would need to generate the synthetic Java code, compile it, load it,
>> and only then run it, calling back to JS runtime for execution of our
>> payload -- all in flight, and requiring the tight cohesion.
>> 
>> b) Ask JMH to compile and execute a Java benchmark, and call into JS
>> engine with a JS script in hands, asking to compile and run it. There,
>> users statically compile the benchmark JAR, and it runs with minimal
>> dynamic dances, also going through the standard APIs (javax.script).
>> 
>> While there is nothing fundamental preventing us from exploring the
>> route (a), it is not as practical as going the well-established route
>> (b). In other words, "could be done" != "should be done".
>> 
>> -Aleksey.
>> 
>