Nashorn Octane JMH code publicly available?

[Marcus Lagergren]

No results are very unstable - using JMH will bring significant stability to them, but you have to realize that in some cases HotSpot needs to do hundreds of runs before plateauing out because of LambdaForms etc. Aleksey knows more.

/M

> On 24 Jun 2015, at 17:50, Volker Simonis <volker.simonis at gmail.com> wrote:
> 
> Hi Chris, Marcus,
> 
> thanks for your responses.
> 
> Some colleagues of mine who set up Octane in our environment
> complained about the test results being extremely unstable. And even
> more so if comparing two different JDK versions (like our internal
> build with an "official" Oracle-JDK build" which should
> "theoretically" behave the same).
> 
> So I wanted to look at the differences and what I found was the quote
> from the Nashorn Wiki which says: "the octane base.js harness .. has a
> very primitive and deviation prone way of measuring benchmark time."
> That's why I thought to better use JMH harness as described on that
> page before starting to look at "real" problems.
> 
> 
> @Marcus: do you really see stable benchmark results when using your
> internal JMH Octane harness or is this a pointless undertaking
> anywhere?
> 
> @Chris: if it isn't too much of an effort for you, I'd appreciate to
> have a look at your harness.
> 
> Regards,
> Volker
> 
> On Wed, Jun 24, 2015 at 5:38 PM, Chris Newland
> <cnewland at chrisnewland.com> wrote:
>> Hi Volker,
>> 
>> In the end I wrote my own JMH harness around octane but I can't get the
>> results stable even with 3 day runs so my JMH code could be buggy.
>> 
>> Will share later off list if you're interested.
>> 
>> Regards,
>> 
>> Chris
>> @chriswhocodes
>> 
>> On Wed, June 24, 2015 16:00, Marcus Lagergren wrote:
>>> Not sure if it’s available publically as part of JMH, but you can get
>>> the octane benchmark code itself from
>>> 
>>> svn co http://octane-benchmark.googlecode.com/svn/trunk
>>> <http://octane-benchmark.googlecode.com/svn/trunk>
>>> 
>>> 
>>> /M
>>> 
>>> 
>>>> On 24 Jun 2015, at 16:41, Volker Simonis <volker.simonis at gmail.com>
>>>> wrote:
>>>> 
>>>> 
>>>> I just had the same question and couldn't find any answer with Google
>>>> except this mail thread.
>>>> 
>>>> So I'll just as again: is the Octane benchmark as part of JMH as
>>>> described in the Nashorn Wiki [1] publicly available somewhere?
>>>> 
>>>> Thanks,
>>>> Volker
>>>> 
>>>> 
>>>> [1]
>>>> https://wiki.openjdk.java.net/display/Nashorn/Monitoring+Nashorn+Perfor
>>>> mance
>>>> 
>>>> On Thu, Apr 30, 2015 at 9:18 AM, Marcus Lagergren
>>>> <marcus.lagergren at oracle.com> wrote:
>>>> 
>>>>> 
>>>>>> On 29 Apr 2015, at 09:08, Marcus Lagergren
>>>>>> <marcus.lagergren at oracle.com> wrote:
>>>>>> 
>>>>>> 
>>>>>> The benchmark code itself is public, but I don’t know if the
>>>>>> benchmarks as part of JMH are. CC Aleksey /M
>>>>>> 
>>>>>> 
>>>>>>> On 28 Apr 2015, at 19:24, Chris Newland
>>>>>>> <cnewland at chrisnewland.com> wrote:
>>>>>>> 
>>>>>>> 
>>>>>>> Hi,
>>>>>>> 
>>>>>>> 
>>>>>>> https://wiki.openjdk.java.net/display/Nashorn/Monitoring+Nashorn+
>>>>>>> Performance
>>>>>>> indicates the JMH microbenchmarks for octane are internal to
>>>>>>> Oracle (VPN
>>>>>>> needed).
>>>>>>> 
>>>>>>> I can't see them elsewhere in the OpenJDK repos, are they public?
>>>>>>> 
>>>>>>> 
>>>>>>> Asking because the results I'm getting from
>>>>>>> 
>>>>>>> 
>>>>>>> nashorn/test/script/external/octane$ jjs run.js
>>>>>>> 
>>>>>>> are wildly variable (as per above wiki page).
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> 
>>>>>>> 
>>>>>>> Chris
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>> 
>>> 
>> 
>>