Nashorn performance regression from JDK8u5 to JDK8u25?

[Marcus Lagergren]

Are you running with optimistic types enabled? (—optimistic-types=true) and the compiler team is still working on various performance regressions that appeared when Lambda Form caching came in to HotSpot (b30->b31 for 9), backported to 8.



Regards
Marcus

> On 18 Nov 2014, at 03:21, Bernard Liang <bliang at linkedin.com> wrote:
> 
> Michel et al,
> 
> I’ve run our local test battery using the link you provided, and while in some cases there is improvement, overall the performance still seems to be closer to u25 levels than u5 levels. For what it’s worth, I did notice that the performance improvements from u25 to u40 were generally better in pooled environments than ones where a single instance of the execution environment was running per thread. This leads itself to a few questions, some of which are reiterated from the original inquiry:
> 
>  *   Is anyone familiar with (significant) specific changes in the Nashorn libraries from u5 => u25 => u40 that might be related to this regression and could explain the u25 and/or u40 changes in more detail (that might have led to the recommendation to use u40)?
>  *   Do you have any performance suites (internal or other) that test various Nashorn benchmarks across different releases (of JDK8, for instance)? Do the results of those correlate with our findings?
> 
> Regards,
> Bernard Liang
> 
> PS. The output of `java -version` most recently tested was as follows:
> 
> java version "1.8.0_40-ea"
> 
> Java(TM) SE Runtime Environment (build 1.8.0_40-ea-b12)
> 
> Java HotSpot(TM) 64-Bit Server VM (build 25.40-b16, mixed mode)
> 
> Previous versions tested:
> 
> java version "1.8.0_25"
> 
> Java(TM) SE Runtime Environment (build 1.8.0_25-b17)
> 
> Java HotSpot(TM) 64-Bit Server VM (build 25.25-b02, mixed mode)
> 
> java version "1.8.0_05"
> 
> Java(TM) SE Runtime Environment (build 1.8.0_05-b13)
> 
> Java HotSpot(TM) 64-Bit Server VM (build 25.5-b02, mixed mode)
> 
> From: Michel Trudeau <michel.trudeau at oracle.com<mailto:michel.trudeau at oracle.com>>
> Date: Monday, November 17, 2014 at 1:12 PM
> To: Bernard Liang <bliang at linkedin.com<mailto:bliang at linkedin.com>>
> Cc: "nashorn-dev at openjdk.java.net<mailto:nashorn-dev at openjdk.java.net>" <nashorn-dev at openjdk.java.net<mailto:nashorn-dev at openjdk.java.net>>
> Subject: Re: Nashorn performance regression from JDK8u5 to JDK8u25?
> 
> Bernard,
> 
> It'd be great if you could try the latest 8u40 stable build.   We are planning to release 8u40 early in the new year.
> 
> https://jdk8.java.net/download.html
> 
> We also have an optional optimizer in 8u40, enable it with the command line argument '-ot'.
> 
> Thanks,
> Michel
> 
> 
> 
> 
> 
> Bernard Liang wrote:
> 
> Hello,
> 
> After running some performance tests on the Cartesian product of ([JDK8u5, JDK8u25] x [simple template, complex templates] x [all-or-nothing, streaming chunks] x [single dust instance per thread, pooled dust instances] x [blank Dust instances, Dust instances with templates preloaded]), we find that JDK8u25 performance is very consistently considerably worse than JDK8u5 (by roughly 10-100%, with the average falling somewhere between there). The relevant code has been executed enough times (on the order of 10,000 times) to reach reasonably warmed-up states. If some of the items on the axes of the Cartesian product don’t make much sense, you can ignore the fuzzy parts of the detailed breakdown for now, with the general understanding that various different environments have been tested and shown to yield the same results.
> 
> Some additional high-level context:
> 
> Dust is basically a templating language used to render JSON data into HTML with compilable “templates": https://github.com/linkedin/dustjs (we are at the v2.4.2 tag)
> 
> “simple template” = ~150 bytes each of one (precompiled) template + context JSON (attached)
> “complex templates” = ~350 compiled templates spanning ~245KB in compiled JS + ~75KB of JSON context (proprietary data)
> 
>> From https://blogs.oracle.com/nashorn/entry/latest_news_from_the_nashorn, it sounded like there were some recent updates made to Nashorn performance around the u20 mark, but that seems to have caused a regression rather than an improvement. Is this something that nashorn-dev is aware of? Is there any way we can help diagnose the issue further using publicly safe data? (If you’re looking for a way to reproduce this, the attached basic Dust template + JSON context should be adequate under almost any environment.)
> 
> Regards,
> Bernard Liang
>