graal + node.js: how does it work?

[Christian Wirth]

Hi Adam,

thanks for your work and for providing the benchmarks.

We ran them on our machines and think you allowed too little warmup (you 
write "long jvm warmup" - Can you quantify how long you warmed up before 
measuring?) Due to its architecture, Graal.js requires more warmup 
compared to natively implemented engines [1].

On the fileTest.js benchmark, given more warmup time, we measure node9 
being roughly 3x faster, and node6 roughly 1.7x faster than Graal.js. As 
Graal.js is currently based on node6 source code, that's what we can 
compare against. We are currently upgrading to a newer Node version, and 
expect Graal.js performance to then go up, accordingly.

Still, there is some room for improvement, we will look into closing 
this gap. Thanks again for reporting, feedback like that is tremendously 
valuable for us.

Thanks,
Christian Wirth


[1] external blog-post about that (the article is about the Ruby 
implementation, but the problem is basically identical for our 
JavaScript engine): 
https://pragtob.wordpress.com/2017/08/29/careful-what-you-measure-2-1-times-slower-to-4-2-times-faster-mjit-versus-truffle-ruby/




Am 23.12.2017 um 22:21 schrieb Adam McMahon:
> Jaroslav and group,
> 
> I had a chance to run some tests running node.js on the JVM using the
> Vert.x library. I call this project node.v (node running on vert.x).
> Included are some benchmarks of my approach vs graal-node and native node.
> 
> *[1] node.v*
> 
> Project is found at https://github.com/AdamMcM/node.v (the readme has setup
> instructions if anyone is interested).
> 
> Vert.x is a polyglot inspired node library for the JVM that has many
> features similar to node already written in Javascript using Nashorn. For
> most of the functionality in Node, One can implement node.js by writing a
> small wrapper that calls down to the JS Vert.x API. For example, Vert.x
> already has a method called readFile which is very similar to node’s
> fs.readFile: https://github.com/AdamMcM/node.v/blob/master/fs.js
> 
> 
> *[2] Performance*
> 
> Take this all with a grain of salt :)
> 
> In the two examples I ran, Node.v is almost as performant as native
> node.js, and it is significantly faster than graal’s node implementation.
> In fact, graal-node runs so much slower than native node that I am
> wondering if I have a setup issue. Based on Devoxx presentations, I was
> expecting it to be near native node performance.
> 
> Here are some benchmarks.
> 
> *[2a] HelloWorld. *
> https://github.com/AdamMcM/node.v/blob/master/helloTest.js
> 
> 10 concurrent clients, Node.v is the fastest, with graal-native the
> slowest. *Node.v x3* indictes that 3 instnces of vert.x were used. This is
> similar to node’s cluster module, but does not require any change to the
> script, just adding “-–instances 3” to the command line, it will
> automatically scale to 3 event loops. No other code changes are necessary.
> 
> numbers are in requests per second
> 
> node.v x3:  14182          (long jvm warmup: 100k requests)
> node.v : 10103               (long jvm warmup)
> node9:  8172
> node-graal-jvm: 5065       (long jvm warmup)
> node-graal-native: 4101
> 
> [2a] FileTest https://github.com/AdamMcM/node.v/blob/master/fileTest.js
> 
> Read a small file, 36 bytes, and send it back to the client 10 times using
> chunked encoding.
> 
> Node9 3593
> node.v x3: 3354 (after long warm up)
> node.v : 1900 (after long warm up)
> node-graal-native: 228
> node-graal-jvm: 228
> 
> 
> *[3] Discussion*
> 
> Node.v is a small proof of concept that node.js can be relatively easily
> implemented onto the JVM using the Vert.x library. To do so, one can create
> wrapper functions down to the Javascript Vert.x API. Unless there was a
> setup issue, node.v is significantly faster than node-graal in these
> examples. Node.v uses Nashorn, perhaps it could be improved even more by
> using graal.js instead.
> 
> This was just a proof of concept project for me to learn more about node.js
> on the JVM, I will likely add a few more examples and flush out a few more
> features, but I do not intend (at this time) to implement the entire
> node.js API (as my day job keeps me pretty busy).
> 
> Best and Happy Holidays,
> 
> -Adam
> 
> 
> 
> 
> 
> On Wed, Dec 20, 2017 at 3:40 AM, Jaroslav Tulach <jaroslav.tulach at oracle.com
>> wrote:
> 
>> Hello Adam,
>> a small note on performance:
>>
>> On úterý 19. prosince 2017 22:14:35 CET Adam McMahon wrote:
>>> But in terms of
>>> performance, my approach runs about the same as vert.x (if you are
>> familiar
>>> with that performance), as mine is just a thin call down to vert.x from
>> JS.
>>
>> This would be the kind of reply I'd use before joining the Graal team.
>> However
>> now I have to ask:
>>
>> - what does a "thin call from JS" mean?
>> - how fast the JS implementation is?
>>
>> First of all, from all the Java based JavaScript implementations, only
>> Graal.js is comparable in terms of speed with V8. All other ones are
>> significantly slower. Secondly, the "thin call from JS" may require
>> argument
>> mangling and possibly also inlining boundary. Again, this is something
>> Graal +
>> Graal.js can eliminate and make the execution faster.
>>
>>> ... share my code on github once I get an example together a half-decent
>>> example that works well on all 3 systems (node.js, graal+node, node
>> +vertxWrapper).
>>
>> Of course, the performance depends on the actual usage. Once your example
>> works, it would be nice to turn it into a benchmark and measure the # of
>> operations per second each version can handle.
>>
>> Good luck.
>> -jt
>>
>>