Nashorn Octane JMH code publicly available?

[Volker Simonis]

Hi Chris, Marcus,

thanks for your responses.

Some colleagues of mine who set up Octane in our environment
complained about the test results being extremely unstable. And even
more so if comparing two different JDK versions (like our internal
build with an "official" Oracle-JDK build" which should
"theoretically" behave the same).

So I wanted to look at the differences and what I found was the quote
from the Nashorn Wiki which says: "the octane base.js harness .. has a
very primitive and deviation prone way of measuring benchmark time."
That's why I thought to better use JMH harness as described on that
page before starting to look at "real" problems.


@Marcus: do you really see stable benchmark results when using your
internal JMH Octane harness or is this a pointless undertaking
anywhere?

@Chris: if it isn't too much of an effort for you, I'd appreciate to
have a look at your harness.

Regards,
Volker

On Wed, Jun 24, 2015 at 5:38 PM, Chris Newland
<cnewland at chrisnewland.com> wrote:
> Hi Volker,
>
> In the end I wrote my own JMH harness around octane but I can't get the
> results stable even with 3 day runs so my JMH code could be buggy.
>
> Will share later off list if you're interested.
>
> Regards,
>
> Chris
> @chriswhocodes
>
> On Wed, June 24, 2015 16:00, Marcus Lagergren wrote:
>> Not sure if it’s available publically as part of JMH, but you can get
>> the octane benchmark code itself from
>>
>> svn co http://octane-benchmark.googlecode.com/svn/trunk
>> <http://octane-benchmark.googlecode.com/svn/trunk>
>>
>>
>> /M
>>
>>
>>> On 24 Jun 2015, at 16:41, Volker Simonis <volker.simonis at gmail.com>
>>> wrote:
>>>
>>>
>>> I just had the same question and couldn't find any answer with Google
>>> except this mail thread.
>>>
>>> So I'll just as again: is the Octane benchmark as part of JMH as
>>> described in the Nashorn Wiki [1] publicly available somewhere?
>>>
>>> Thanks,
>>> Volker
>>>
>>>
>>> [1]
>>> https://wiki.openjdk.java.net/display/Nashorn/Monitoring+Nashorn+Perfor
>>> mance
>>>
>>> On Thu, Apr 30, 2015 at 9:18 AM, Marcus Lagergren
>>> <marcus.lagergren at oracle.com> wrote:
>>>
>>>>
>>>>> On 29 Apr 2015, at 09:08, Marcus Lagergren
>>>>> <marcus.lagergren at oracle.com> wrote:
>>>>>
>>>>>
>>>>> The benchmark code itself is public, but I don’t know if the
>>>>> benchmarks as part of JMH are. CC Aleksey /M
>>>>>
>>>>>
>>>>>> On 28 Apr 2015, at 19:24, Chris Newland
>>>>>> <cnewland at chrisnewland.com> wrote:
>>>>>>
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>>
>>>>>> https://wiki.openjdk.java.net/display/Nashorn/Monitoring+Nashorn+
>>>>>> Performance
>>>>>> indicates the JMH microbenchmarks for octane are internal to
>>>>>> Oracle (VPN
>>>>>> needed).
>>>>>>
>>>>>> I can't see them elsewhere in the OpenJDK repos, are they public?
>>>>>>
>>>>>>
>>>>>> Asking because the results I'm getting from
>>>>>>
>>>>>>
>>>>>> nashorn/test/script/external/octane$ jjs run.js
>>>>>>
>>>>>> are wildly variable (as per above wiki page).
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>>
>>>>>> Chris
>>>>>>
>>>>>>
>>>>>
>>>>
>>
>>
>
>