From davidmichaelkarr at gmail.com  Sat Dec  2 05:22:10 2017
From: davidmichaelkarr at gmail.com (David Karr)
Date: Fri, 1 Dec 2017 21:22:10 -0800
Subject: Getting "No benchmarks to run"
Message-ID: <CAA5t8VrwqOk4ZviQa0VQYTNxrUVqOcTb3GDXUUs3QFEF_99pNg@mail.gmail.com>

I'm trying to compare the performance of two methods for operating on
a map with populated random data.

The following is my current benchmark class:
--------------------------
@State(Scope.Benchmark)
public class Streams {
    private static final int    NUM_ELEMENTS    = 100;

    private Map<String, Long>    scores;

    public static void main(String[] args) {

        Streams streams = new Streams();
        streams.doit();
    }

    private void doit() {
        setup();
        System.out.println("scores[" + scores + "]");
    }

    public String randomString() { return RandomStringUtils.random(16); }
    public long randomLong() { return RandomUtils.nextLong(0L, 100L); }

    @Setup
    public void setup() {
        scores  = IntStream.range(0,
NUM_ELEMENTS).asLongStream().mapToObj(v ->
randomString()).collect(Collectors.toMap(Function.identity(), val ->
randomLong()));
    }

    @Benchmark
    public void measureGroupByScores1() {
        Streams.groupByScores1(scores);
    }

    public static Map<Long, List<String>> groupByScores1(Map<String,
Long> scores) {
        return scores.keySet().stream().collect(Collectors.groupingBy(scores::get));
    }

    @Benchmark
    public void measureGroupByScores2() {
        Streams.groupByScores2(scores);
    }

    public static Map<Long, List<String>> groupByScores2(Map<String,
Long> scores) {
        return scores.entrySet().stream().collect(Collectors.groupingBy(Map.Entry::getValue,
Collectors.mapping(Map.Entry::getKey, Collectors.toList())));
    }
}

----------------

I'm using the Gradle JMH plugin (https://github.com/melix/jmh-gradle-plugin).

In this project, I also have another class with benchmarks, although
I'd prefer to only run the benchmarks in this "Streams" class for now.

Currently, when I run the "jmh" task, I get this (many lines elided):
-----------------
:compileJmhJava UP-TO-DATE
:processJmhResources NO-SOURCE
:jmhClasses UP-TO-DATE
:jmhRunBytecodeGenerator
Processing 16 classes from
C:\Users\<userid>\workspace\jsonpathexperiments\build\classes\java\jmh
with "reflection" generator
Writing out Java source to
C:\Users\<userid>\workspace\jsonpathexperiments\build\jmh-generated-sources
and resources to
C:\Users\<userid>\workspace\jsonpathexperiments\build\jmh-generated-classes
:jmhCompileGeneratedClasses
:jmhJar
Encountered duplicate path "enums/Enums$Thingie1.class" during copy
operation configured with DuplicatesStrategy.WARN
...
Encountered duplicate path "streams/generated/Streams_jmhType.class"
during copy operation configured with DuplicatesStrategy.WARN
...
Encountered duplicate path
"enums/generated/Enums_enumValuesLoop_jmhTest.class" during copy
operation configured with DuplicatesStrategy.WARN
...
Encountered duplicate path "streams/generated/Streams_jmhType.class"
during copy operation configured with DuplicatesStrategy.WARN
...
:jmh FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':jmh'.
> A failure occurred while executing me.champeau.gradle.IsolatedRunner
   > Error during execution of benchmarks

* Try:
Run with --info or --debug option to get more log output.

* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':jmh'.
...
Caused by: No benchmarks to run; check the include/exclude regexps.
    at org.openjdk.jmh.runner.Runner.internalRun(Runner.java:261)
    at org.openjdk.jmh.runner.Runner.run(Runner.java:206)
    at me.champeau.gradle.IsolatedRunner.run(IsolatedRunner.java:38)
    ... 12 more
--------------------

The "jmh" block in my build script is just this:
-------------
jmh {
    //verbosity = 'EXTRA' // Verbosity mode. Available modes are:
[SILENT, NORMAL, EXTRA]
    jvmArgs = ['-Djmh.ignoreLock=true']
    jmhVersion = '1.19'
    fork = 2
    duplicateClassesStrategy = 'warn'
    iterations = 40
    //include = ['Streams']
}

-------------

Not sure what's going on here.

From vitaliy.se at gmail.com  Mon Dec 11 16:57:42 2017
From: vitaliy.se at gmail.com (Vitaliy Semochkin)
Date: Mon, 11 Dec 2017 19:57:42 +0300
Subject: assertions for jmh performance test results
Message-ID: <CAHyKpfPgB-Cgn4J4pnHNMs5Z5iV5_JzV9YBoATYCqsuBTFPpog@mail.gmail.com>

Hi,

Is there any way to create assertions for JMH performance test rests?

E.g I expect avg time of execution to be no longer than T milliseconds
and I want the test to fail if the expectation is not met.

Regards,
Vitaliy

From nitsanw at yahoo.com  Tue Dec 12 07:13:30 2017
From: nitsanw at yahoo.com (Nitsan Wakart)
Date: Tue, 12 Dec 2017 07:13:30 +0000 (UTC)
Subject: assertions for jmh performance test results
In-Reply-To: <CAHyKpfPgB-Cgn4J4pnHNMs5Z5iV5_JzV9YBoATYCqsuBTFPpog@mail.gmail.com>
References: <CAHyKpfPgB-Cgn4J4pnHNMs5Z5iV5_JzV9YBoATYCqsuBTFPpog@mail.gmail.com>
Message-ID: <75176233.3450406.1513062810956@mail.yahoo.com>

JMH does not offer this capability at the moment.
 

    On Monday, December 11, 2017 6:58 PM, Vitaliy Semochkin <vitaliy.se at gmail.com> wrote:
 

 Hi,

Is there any way to create assertions for JMH performance test rests?

E.g I expect avg time of execution to be no longer than T milliseconds
and I want the test to fail if the expectation is not met.

Regards,
Vitaliy


   

From akarnokd at gmail.com  Tue Dec 12 10:36:05 2017
From: akarnokd at gmail.com (=?UTF-8?Q?D=C3=A1vid_Karnok?=)
Date: Tue, 12 Dec 2017 11:36:05 +0100
Subject: [PATCH]: Forking benchmarks with long classpath on Windows (7)
Message-ID: <CAAWwtm-ynUmOtb+cMTmixE+C42eCe8egAjexvVN1Dyaw5mbonA@mail.gmail.com>

Hello!

I'd like to submit a patch (attached) to JMH's
org.openjdk.jmh.runner.Runner.java that creates a temporary jar file with
all classpath entries on Windows so that the command line doesn't overflow.



2017-10-16 19:09 GMT+02:00 D?vid Karnok <akarnokd at gmail.com>:

> Hi.
>
> I have found those posts but they don't work for me as the code to be
> benchmarked is part of a library, not a Java app or Spring boot.
>
> As I understand, the main JMH program is started with the very same large
> set of classpath entries successfully. The problem after that is with the
> forking of new JMH instances by a parent JMH itself.
>
> 2017-10-16 18:56 GMT+02:00 Dmitry Chuyko <dmitry.chuyko at bell-sw.com>:
>
>> Hi D?vid,
>>
>> Looks more like common Gradle problem. It could be probably solved by
>> using manifest instead, like
>>
>> http://tuhrig.de/gradles-bootrun-and-windows-command-length-limit/
>>
>> There's probably also an option to process dependencies and remove
>> transitive ones.
>>
>> -Dmitry
>>
>>
>> On 10/16/2017 05:18 PM, D?vid Karnok wrote:
>>
>>> Hi,
>>>
>>> I've been running JMH via Gradle from some time now on Windows (and Java
>>> 8u144), but my recent upgrade to Gradle 4+ lead to some problems. Gradle
>>> 4+
>>> seems to add so many of its own jar files (irrelevant from the target
>>> benchmark's perspective) to the classpath that when JMH tries to fork an
>>> instance, the generated command line (in Runner::getForkedMainCommand,
>>> System.getProperty("java.class.path")) creates a long String that, along
>>> with the other short parameters of JMH, now exceed Windows' 32kb limit on
>>> command lines, resulting in an IOException pointing to  CreateProcess.
>>> Linux doesn't have this problem, however, the project needs at least
>>> someone who tests and benchmarks on Windows.
>>>
>>> Would it be possible that the Runner passed this long classpath string to
>>> the to-be-forked java.exe in some other way (a temporary pathing jar)?
>>> Alternatively, how feasible would be to support some kind of exclude
>>> filter
>>> at this point in the Runner?
>>>
>>> Unfortunately, I couldn't find a way to prevent Gradle (or the JMH
>>> plugin)
>>> itself from passing that many classpath entries when it runs JMH.
>>>
>>>
>>
>
>
> --
> Best regards,
> David Karnok
>



-- 
Best regards,
David Karnok

From vitaliy.se at gmail.com  Tue Dec 12 20:16:04 2017
From: vitaliy.se at gmail.com (Vitaliy Semochkin)
Date: Tue, 12 Dec 2017 23:16:04 +0300
Subject: assertions for jmh performance test results
In-Reply-To: <75176233.3450406.1513062810956@mail.yahoo.com>
References: <CAHyKpfPgB-Cgn4J4pnHNMs5Z5iV5_JzV9YBoATYCqsuBTFPpog@mail.gmail.com>
 <75176233.3450406.1513062810956@mail.yahoo.com>
Message-ID: <CAHyKpfPrOAcfHPgpOdjnJD4ttvnKYFN-hCh-QXhgCWycRuW2cA@mail.gmail.com>

Thank you very much.
Does it  provide an API so that a developer could get performance
metrics not from logging?
(If yes, it won't be difficult to write a JUnit test or a Rule).

On Tue, Dec 12, 2017 at 10:13 AM, Nitsan Wakart <nitsanw at yahoo.com> wrote:
> JMH does not offer this capability at the moment.
>
>
>
> On Monday, December 11, 2017 6:58 PM, Vitaliy Semochkin
> <vitaliy.se at gmail.com> wrote:
>
>
> Hi,
>
> Is there any way to create assertions for JMH performance test rests?
>
> E.g I expect avg time of execution to be no longer than T milliseconds
> and I want the test to fail if the expectation is not met.
>
> Regards,
> Vitaliy
>
>

From dmitry.timofeiev at gmail.com  Tue Dec 12 20:50:22 2017
From: dmitry.timofeiev at gmail.com (Dmitry Timofeiev)
Date: Tue, 12 Dec 2017 22:50:22 +0200
Subject: assertions for jmh performance test results
In-Reply-To: <CAHyKpfPrOAcfHPgpOdjnJD4ttvnKYFN-hCh-QXhgCWycRuW2cA@mail.gmail.com>
References: <CAHyKpfPgB-Cgn4J4pnHNMs5Z5iV5_JzV9YBoATYCqsuBTFPpog@mail.gmail.com>
 <75176233.3450406.1513062810956@mail.yahoo.com>
 <CAHyKpfPrOAcfHPgpOdjnJD4ttvnKYFN-hCh-QXhgCWycRuW2cA@mail.gmail.com>
Message-ID: <de74cac5-0bd2-a140-1989-95b30f5496ac@gmail.com>

Hi Vitaliy,

I may suggest two approaches to get the metrics:

(1) construct your performance tests parameters using an OptionsBuilder, 
run them and then analyze a Collection<RunResult> returned by 
org.openjdk.jmh.runner.Runner#run 
(http://hg.openjdk.java.net/code-tools/jmh/file/1ddf31f810a3/jmh-samples/src/main/java/org/openjdk/jmh/samples/JMHSample_03_States.java#l125).

(2) run your tests using the CLI of the benchmarks.jar, but tell it to 
store the results in machine-readable format (AFAIR, CSV & JSON are 
supported):

 >? $ java -jar target/benchmark.jar -h

--------------------------------8<--------------------------------

-rf <type>????????????????? Format type for machine-readable results. These
 ????????????????????????????? results are written to a separate file 
(see -rff).
 ????????????????????????????? See the list of available result formats 
with -lrf.
 ????????????????????????????? (default: CSV)
-------------------------------->8--------------------------------

There might be some other, possibly better, approaches.

Cheers,
Dmitry


On 12.12.17 22:16, Vitaliy Semochkin wrote:
> Thank you very much.
> Does it  provide an API so that a developer could get performance
> metrics not from logging?
> (If yes, it won't be difficult to write a JUnit test or a Rule).
>
> On Tue, Dec 12, 2017 at 10:13 AM, Nitsan Wakart <nitsanw at yahoo.com> wrote:
>> JMH does not offer this capability at the moment.
>>
>>
>>
>> On Monday, December 11, 2017 6:58 PM, Vitaliy Semochkin
>> <vitaliy.se at gmail.com> wrote:
>>
>>
>> Hi,
>>
>> Is there any way to create assertions for JMH performance test rests?
>>
>> E.g I expect avg time of execution to be no longer than T milliseconds
>> and I want the test to fail if the expectation is not met.
>>
>> Regards,
>> Vitaliy
>>
>>


From henri.tremblay at gmail.com  Wed Dec 13 03:09:50 2017
From: henri.tremblay at gmail.com (Henri Tremblay)
Date: Tue, 12 Dec 2017 22:09:50 -0500
Subject: assertions for jmh performance test results
In-Reply-To: <de74cac5-0bd2-a140-1989-95b30f5496ac@gmail.com>
References: <CAHyKpfPgB-Cgn4J4pnHNMs5Z5iV5_JzV9YBoATYCqsuBTFPpog@mail.gmail.com>
 <75176233.3450406.1513062810956@mail.yahoo.com>
 <CAHyKpfPrOAcfHPgpOdjnJD4ttvnKYFN-hCh-QXhgCWycRuW2cA@mail.gmail.com>
 <de74cac5-0bd2-a140-1989-95b30f5496ac@gmail.com>
Message-ID: <CADZL2=vCfqJG84QXqWYbx1R2cGDC3WZdaP=b9eoDMpSnO9TVvw@mail.gmail.com>

This would work nicely. The alternative is to do your own OutputFormat that
does the assertion.

I don't exactly know how. You will need your own Runner I think so it's a
bit more involved.

On 12 December 2017 at 15:50, Dmitry Timofeiev <dmitry.timofeiev at gmail.com>
wrote:

> Hi Vitaliy,
>
> I may suggest two approaches to get the metrics:
>
> (1) construct your performance tests parameters using an OptionsBuilder,
> run them and then analyze a Collection<RunResult> returned by
> org.openjdk.jmh.runner.Runner#run (http://hg.openjdk.java.net/co
> de-tools/jmh/file/1ddf31f810a3/jmh-samples/src/main/java/
> org/openjdk/jmh/samples/JMHSample_03_States.java#l125).
>
> (2) run your tests using the CLI of the benchmarks.jar, but tell it to
> store the results in machine-readable format (AFAIR, CSV & JSON are
> supported):
>
> >  $ java -jar target/benchmark.jar -h
>
> --------------------------------8<--------------------------------
>
> -rf <type>                  Format type for machine-readable results. These
>                               results are written to a separate file (see
> -rff).
>                               See the list of available result formats
> with -lrf.
>                               (default: CSV)
> -------------------------------->8--------------------------------
>
> There might be some other, possibly better, approaches.
>
> Cheers,
> Dmitry
>
>
>
> On 12.12.17 22:16, Vitaliy Semochkin wrote:
>
>> Thank you very much.
>> Does it  provide an API so that a developer could get performance
>> metrics not from logging?
>> (If yes, it won't be difficult to write a JUnit test or a Rule).
>>
>> On Tue, Dec 12, 2017 at 10:13 AM, Nitsan Wakart <nitsanw at yahoo.com>
>> wrote:
>>
>>> JMH does not offer this capability at the moment.
>>>
>>>
>>>
>>> On Monday, December 11, 2017 6:58 PM, Vitaliy Semochkin
>>> <vitaliy.se at gmail.com> wrote:
>>>
>>>
>>> Hi,
>>>
>>> Is there any way to create assertions for JMH performance test rests?
>>>
>>> E.g I expect avg time of execution to be no longer than T milliseconds
>>> and I want the test to fail if the expectation is not met.
>>>
>>> Regards,
>>> Vitaliy
>>>
>>>
>>>
>

From henri.tremblay at gmail.com  Wed Dec 13 16:25:37 2017
From: henri.tremblay at gmail.com (Henri Tremblay)
Date: Wed, 13 Dec 2017 11:25:37 -0500
Subject: DTrace asm profiler for Mac OS X
In-Reply-To: <0b213005-161b-a2f8-c5c4-c2df4d61c3ed@redhat.com>
References: <CAEH0ZEGMKxi2WCrAOvy_UH-H80aFGGXCS6kAswdk-MU0Zbq_iw@mail.gmail.com>
 <CADZL2=uKU=9_fHa5e5VMPBb_fEN+X1iu0xXmQs+UgCrm=vZ6qw@mail.gmail.com>
 <CADZL2=tn5T9CjOhUxM1wUF6VV2qi457kpmPCsamkK30SySi7Xw@mail.gmail.com>
 <CAEH0ZEEiDR8N-vWpg=v-oJ5TMBvxB56C9gAJ49ZoBWFrnR4vQg@mail.gmail.com>
 <CAEH0ZEHzrdtkbM_-N1g_O-4_0P0tASiJKHfn_6-cOPonG0yiYg@mail.gmail.com>
 <0b213005-161b-a2f8-c5c4-c2df4d61c3ed@redhat.com>
Message-ID: <CADZL2=taY19Svv12PU2nSEU=tTkvoX7ivHLgGFFKLysYn8=sHQ@mail.gmail.com>

A bit late but my only problem right now is that I don't get any hot
section. Which is weird.

With perf, I would get some content. With dtrace, nothing.

However, I am not an expert in using both. So maybe some javac or java
arguments are required to get nice results. Is it the case?

Thanks,
Henri

On 23 November 2017 at 13:04, Aleksey Shipilev <shade at redhat.com> wrote:

> On 11/23/2017 09:09 AM, Vsevolod Tolstopyatov wrote:
> > Hello,
> >
> > Any news about this patch? Is it going into jmh?
>
> It will. Just let me figure out some Mac testing.
>
> -Aleksey
>
>

From alex.averbuch at neo4j.com  Fri Dec 15 09:21:15 2017
From: alex.averbuch at neo4j.com (Alex Averbuch)
Date: Fri, 15 Dec 2017 10:21:15 +0100
Subject: beforeTrial/afterTrial not called for warmup forks
Message-ID: <CAMvjqRwSCevqt6TQnZ4ngiFKwhBme0nCEC3WgFN1rckuOL4t=w@mail.gmail.com>

I've noticed that ExternalProfiler#addJVMInvokeOptions &
ExternalProfiler#addJVMOptions are invoked for warmup forks, but that
ExternalProfiler#beforeTrial & ExternalProfiler#afterTrial are not.

On first impression it feels inconsistent.
Is this intended behavior?

Thanks

From alex.averbuch at neo4j.com  Fri Dec 15 09:21:15 2017
From: alex.averbuch at neo4j.com (Alex Averbuch)
Date: Fri, 15 Dec 2017 10:21:15 +0100
Subject: beforeTrial/afterTrial not called for warmup forks
Message-ID: <CAMvjqRwSCevqt6TQnZ4ngiFKwhBme0nCEC3WgFN1rckuOL4t=w@mail.gmail.com>

I've noticed that ExternalProfiler#addJVMInvokeOptions &
ExternalProfiler#addJVMOptions are invoked for warmup forks, but that
ExternalProfiler#beforeTrial & ExternalProfiler#afterTrial are not.

On first impression it feels inconsistent.
Is this intended behavior?

Thanks

From jzaugg at gmail.com  Mon Dec 18 05:26:53 2017
From: jzaugg at gmail.com (Jason Zaugg)
Date: Mon, 18 Dec 2017 05:26:53 +0000
Subject: Regression in "7902041: Make sure JMH builds and runs with JDK 9 GA"
Message-ID: <CAG3_yeV8ABCytxR6ycx0Vrc6Xv6mUhzXWJwnEf5h0J-b-zJgbA@mail.gmail.com>

There appears to be a bug in
http://hg.openjdk.java.net/code-tools/jmh/rev/1ddf31f810a3#l6.34 The method
name should be "getInternal$TYPECounter", but TYPE is hardcoded to
"Runtime".

Regards,

Jason Zaugg

From ashipile at redhat.com  Wed Dec 20 19:53:15 2017
From: ashipile at redhat.com (ashipile at redhat.com)
Date: Wed, 20 Dec 2017 19:53:15 +0000
Subject: hg: code-tools/jmh: 7902085: hs_* profilers regression: typo in
 getInternalRuntimeCounter, should be getInternalTYPECounter
Message-ID: <201712201953.vBKJrFjt007947@aojmv0008.oracle.com>

Changeset: 335dfeac13dc
Author:    shade
Date:      2017-12-20 20:47 +0100
URL:       http://hg.openjdk.java.net/code-tools/jmh/rev/335dfeac13dc

7902085: hs_* profilers regression: typo in getInternalRuntimeCounter, should be getInternalTYPECounter

+ jmh-core-it/src/test/java/org/openjdk/jmh/it/profilers/AbstractHotspotProfilerTest.java
+ jmh-core-it/src/test/java/org/openjdk/jmh/it/profilers/HotspotClassloadingProfilerTest.java
+ jmh-core-it/src/test/java/org/openjdk/jmh/it/profilers/HotspotCompilationProfilerTest.java
+ jmh-core-it/src/test/java/org/openjdk/jmh/it/profilers/HotspotMemoryProfilerTest.java
+ jmh-core-it/src/test/java/org/openjdk/jmh/it/profilers/HotspotRuntimeProfilerTest.java
+ jmh-core-it/src/test/java/org/openjdk/jmh/it/profilers/HotspotThreadProfilerTest.java
! jmh-core/src/main/java/org/openjdk/jmh/profile/AbstractHotspotProfiler.java
! jmh-core/src/main/java/org/openjdk/jmh/profile/HotspotClassloadingProfiler.java
! jmh-core/src/main/java/org/openjdk/jmh/profile/HotspotCompilationProfiler.java
! jmh-core/src/main/java/org/openjdk/jmh/profile/HotspotMemoryProfiler.java
! jmh-core/src/main/java/org/openjdk/jmh/profile/HotspotRuntimeProfiler.java
! jmh-core/src/main/java/org/openjdk/jmh/profile/HotspotThreadProfiler.java


From shade at redhat.com  Wed Dec 20 19:50:16 2017
From: shade at redhat.com (Aleksey Shipilev)
Date: Wed, 20 Dec 2017 20:50:16 +0100
Subject: Regression in "7902041: Make sure JMH builds and runs with JDK 9
 GA"
In-Reply-To: <CAG3_yeV8ABCytxR6ycx0Vrc6Xv6mUhzXWJwnEf5h0J-b-zJgbA@mail.gmail.com>
References: <CAG3_yeV8ABCytxR6ycx0Vrc6Xv6mUhzXWJwnEf5h0J-b-zJgbA@mail.gmail.com>
Message-ID: <b1d898e9-2a77-c736-12bf-161cf04dcb81@redhat.com>

On 12/18/2017 06:26 AM, Jason Zaugg wrote:
> There appears to be a bug in
> http://hg.openjdk.java.net/code-tools/jmh/rev/1ddf31f810a3#l6.34 The method
> name should be "getInternal$TYPECounter", but TYPE is hardcoded to
> "Runtime".

Thanks! That is indeed an unfortunate regression, fixed with:
  https://bugs.openjdk.java.net/browse/CODETOOLS-7902085

-Aleksey


From ashipile at redhat.com  Thu Dec 21 07:25:12 2017
From: ashipile at redhat.com (ashipile at redhat.com)
Date: Thu, 21 Dec 2017 07:25:12 +0000
Subject: hg: code-tools/jmh: 7902088: External profilers should be called with
 before/afterTrial during warmup forks
Message-ID: <201712210725.vBL7PCDQ028675@aojmv0008.oracle.com>

Changeset: fbe1b55eadf8
Author:    shade
Date:      2017-12-21 08:21 +0100
URL:       http://hg.openjdk.java.net/code-tools/jmh/rev/fbe1b55eadf8

7902088: External profilers should be called with before/afterTrial during warmup forks

+ jmh-core-it/src/test/java/org/openjdk/jmh/it/profilers/CountingExternalProfiler.java
+ jmh-core-it/src/test/java/org/openjdk/jmh/it/profilers/CountingExternalProfilerTest.java
! jmh-core/src/main/java/org/openjdk/jmh/runner/Runner.java


From shade at redhat.com  Thu Dec 21 07:23:06 2017
From: shade at redhat.com (Aleksey Shipilev)
Date: Thu, 21 Dec 2017 08:23:06 +0100
Subject: beforeTrial/afterTrial not called for warmup forks
In-Reply-To: <CAMvjqRwSCevqt6TQnZ4ngiFKwhBme0nCEC3WgFN1rckuOL4t=w@mail.gmail.com>
References: <CAMvjqRwSCevqt6TQnZ4ngiFKwhBme0nCEC3WgFN1rckuOL4t=w@mail.gmail.com>
Message-ID: <0a8dac2b-5958-2d7c-90b6-925ff2aec97c@redhat.com>

Hi,

On 12/15/2017 10:21 AM, Alex Averbuch wrote:
> I've noticed that ExternalProfiler#addJVMInvokeOptions &
> ExternalProfiler#addJVMOptions are invoked for warmup forks, but that
> ExternalProfiler#beforeTrial & ExternalProfiler#afterTrial are not.
> 
> On first impression it feels inconsistent.
> Is this intended behavior?

It is inconsistent. The intent was to ignore profiler results for warmup forks, and it was crudely
implemented by doing the warmup forks separately, this is why {after,before}Trial is called only
with measurement forks. But, JVM still forks with the added of ExternalProfiler JVM options during
warmup forks anyway.

It seems more consistent to run ExternalProfilers during warmup forks too, and just ignore their
results. This is less surprising, and aligns better with what InternalProfilers are doing --
internal profilers also run during warmups and their results ignored.

Fixed with:
  https://bugs.openjdk.java.net/browse/CODETOOLS-7902088

Thanks,
-Aleksey



From shade at redhat.com  Thu Dec 21 07:24:24 2017
From: shade at redhat.com (Aleksey Shipilev)
Date: Thu, 21 Dec 2017 08:24:24 +0100
Subject: [PATCH]: Forking benchmarks with long classpath on Windows (7)
In-Reply-To: <CAAWwtm-ynUmOtb+cMTmixE+C42eCe8egAjexvVN1Dyaw5mbonA@mail.gmail.com>
References: <CAAWwtm-ynUmOtb+cMTmixE+C42eCe8egAjexvVN1Dyaw5mbonA@mail.gmail.com>
Message-ID: <372409f3-0a34-ded5-7290-bfa49f782a63@redhat.com>

Hi David,

On 12/12/2017 11:36 AM, D?vid Karnok wrote:
> I'd like to submit a patch (attached) to JMH's
> org.openjdk.jmh.runner.Runner.java that creates a temporary jar file with
> all classpath entries on Windows so that the command line doesn't overflow.

The patch got stripped by the mailing list, apparently. (Wrong MIME type?)

Thanks,
-Aleksey


From akarnokd at gmail.com  Thu Dec 21 10:01:51 2017
From: akarnokd at gmail.com (=?UTF-8?Q?D=C3=A1vid_Karnok?=)
Date: Thu, 21 Dec 2017 11:01:51 +0100
Subject: [PATCH]: Forking benchmarks with long classpath on Windows (7)
In-Reply-To: <372409f3-0a34-ded5-7290-bfa49f782a63@redhat.com>
References: <CAAWwtm-ynUmOtb+cMTmixE+C42eCe8egAjexvVN1Dyaw5mbonA@mail.gmail.com>
 <372409f3-0a34-ded5-7290-bfa49f782a63@redhat.com>
Message-ID: <CAAWwtm9-vPzuCYJ1AWeJGZueBFfxQ0vN71ASbR_HQNQpZDSW3g@mail.gmail.com>

Index: jmh-core/src/main/java/org/openjdk/jmh/runner/Runner.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- jmh-core/src/main/java/org/openjdk/jmh/runner/Runner.java (revision
1434:1ddf31f810a3100b9433c3fedf24615e85b1d1a7)
+++ jmh-core/src/main/java/org/openjdk/jmh/runner/Runner.java (revision
1434+:1ddf31f810a3+)
@@ -47,6 +47,8 @@
 import java.nio.channels.OverlappingFileLockException;
 import java.util.*;
 import java.util.concurrent.TimeUnit;
+import java.util.jar.*;
+import java.util.zip.*;

 /**
  * Runner executes JMH benchmarks.
@@ -826,7 +828,34 @@
         // assemble final process command
         command.add("-cp");
         if (Utils.isWindows()) {
-            command.add('"' + System.getProperty("java.class.path") + '"');
+            try {
+                File tmpFile = File.createTempFile("jmh-classpath-file",
".jar");
+                tmpFile.deleteOnExit();
+
+                StringBuilder pw = new StringBuilder();
+                for (String cp :
System.getProperty("java.class.path").split(";")) {
+                    pw.append("/");
+                    pw.append(cp.replace('\\', '/').replace(" ", "%20"));
+                    if (!cp.endsWith(".jar")) {
+                        pw.append('/');
+                    }
+                    pw.append(" ");
+                }
+
+                Manifest manifest = new Manifest();
+                Attributes attributes = manifest.getMainAttributes();
+                attributes.put(Attributes.Name.MANIFEST_VERSION, "1.0");
+                attributes.putValue("Class-Path", pw.toString().trim());
+
+                try (JarOutputStream zout = new JarOutputStream(new
FileOutputStream(tmpFile), manifest)) {
+                    ZipEntry ze = new ZipEntry("META-INF/");
+                    zout.putNextEntry(ze);
+                }
+
+                command.add("\"" + tmpFile.getAbsolutePath() + "\"");
+            } catch (IOException ex) {
+                command.add('"' + System.getProperty("java.class.path") +
'"');
+            }
         } else {
             command.add(System.getProperty("java.class.path"));
         }






2017-12-21 8:24 GMT+01:00 Aleksey Shipilev <shade at redhat.com>:

> Hi David,
>
> On 12/12/2017 11:36 AM, D?vid Karnok wrote:
> > I'd like to submit a patch (attached) to JMH's
> > org.openjdk.jmh.runner.Runner.java that creates a temporary jar file
> with
> > all classpath entries on Windows so that the command line doesn't
> overflow.
>
> The patch got stripped by the mailing list, apparently. (Wrong MIME type?)
>
> Thanks,
> -Aleksey
>
>


-- 
Best regards,
David Karnok

From alex.averbuch at neo4j.com  Fri Dec 22 09:43:25 2017
From: alex.averbuch at neo4j.com (Alex Averbuch)
Date: Fri, 22 Dec 2017 10:43:25 +0100
Subject: beforeTrial/afterTrial not called for warmup forks
In-Reply-To: <0a8dac2b-5958-2d7c-90b6-925ff2aec97c@redhat.com>
References: <CAMvjqRwSCevqt6TQnZ4ngiFKwhBme0nCEC3WgFN1rckuOL4t=w@mail.gmail.com>
 <0a8dac2b-5958-2d7c-90b6-925ff2aec97c@redhat.com>
Message-ID: <CAMvjqRzYHCgCcBxXVqnoWQRb1bJ+zP79APPFMv+zJ3CjcomD4g@mail.gmail.com>

Thanks, Aleksey.

On Thu, Dec 21, 2017 at 8:23 AM, Aleksey Shipilev <shade at redhat.com> wrote:

> Hi,
>
> On 12/15/2017 10:21 AM, Alex Averbuch wrote:
> > I've noticed that ExternalProfiler#addJVMInvokeOptions &
> > ExternalProfiler#addJVMOptions are invoked for warmup forks, but that
> > ExternalProfiler#beforeTrial & ExternalProfiler#afterTrial are not.
> >
> > On first impression it feels inconsistent.
> > Is this intended behavior?
>
> It is inconsistent. The intent was to ignore profiler results for warmup
> forks, and it was crudely
> implemented by doing the warmup forks separately, this is why
> {after,before}Trial is called only
> with measurement forks. But, JVM still forks with the added of
> ExternalProfiler JVM options during
> warmup forks anyway.
>
> It seems more consistent to run ExternalProfilers during warmup forks too,
> and just ignore their
> results. This is less surprising, and aligns better with what
> InternalProfilers are doing --
> internal profilers also run during warmups and their results ignored.
>
> Fixed with:
>   https://bugs.openjdk.java.net/browse/CODETOOLS-7902088
>
> Thanks,
> -Aleksey
>
>
>

From shade at redhat.com  Fri Dec 22 17:28:23 2017
From: shade at redhat.com (Aleksey Shipilev)
Date: Fri, 22 Dec 2017 18:28:23 +0100
Subject: [PATCH]: Forking benchmarks with long classpath on Windows (7)
In-Reply-To: <CAAWwtm9-vPzuCYJ1AWeJGZueBFfxQ0vN71ASbR_HQNQpZDSW3g@mail.gmail.com>
References: <CAAWwtm-ynUmOtb+cMTmixE+C42eCe8egAjexvVN1Dyaw5mbonA@mail.gmail.com>
 <372409f3-0a34-ded5-7290-bfa49f782a63@redhat.com>
 <CAAWwtm9-vPzuCYJ1AWeJGZueBFfxQ0vN71ASbR_HQNQpZDSW3g@mail.gmail.com>
Message-ID: <5132e129-d2f6-7e25-8531-14c7367bbd1c@redhat.com>

Thanks!

So I have a few problems with this patch: it takes a completely different route for one particular
OS, which seems odd, and complicates testing, plus (I think) it relies on relative paths to
classpath entries being the same as the path to the generated JAR, etc.

Here's my take:
  http://cr.openjdk.java.net/~shade/jmh/long-classpath.patch

Hacking this further, my lingering doubt it is a JMH place to handle this kind of failure escalated
to a firm concern. It would seems that Gradle is responsible for adding cruft to the classpath on
the platforms that are sensitive to this. In that light, the workaround is disabled by default and
can enabled with -Djmh.separateClasspathJAR=true. How about you ask Gradle and/or Gradle JMH plugin
maintainers to fix their stack, and until then use this optional workaround?

-Aleksey


On 12/21/2017 11:01 AM, D?vid Karnok wrote:
> Index: jmh-core/src/main/java/org/openjdk/jmh/runner/Runner.java
> IDEA additional info:
> Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
> <+>UTF-8
> ===================================================================
> --- jmh-core/src/main/java/org/openjdk/jmh/runner/Runner.java(revision
> 1434:1ddf31f810a3100b9433c3fedf24615e85b1d1a7)
> +++ jmh-core/src/main/java/org/openjdk/jmh/runner/Runner.java(revision 1434+:1ddf31f810a3+)
> @@ -47,6 +47,8 @@
> ?import java.nio.channels.OverlappingFileLockException;
> ?import java.util.*;
> ?import java.util.concurrent.TimeUnit;
> +import java.util.jar.*;
> +import java.util.zip.*;
> ?
> ?/**
> ? * Runner executes JMH benchmarks.
> @@ -826,7 +828,34 @@
> ? ? ? ? ?// assemble final process command
> ? ? ? ? ?command.add("-cp");
> ? ? ? ? ?if (Utils.isWindows()) {
> -? ? ? ? ? ? command.add('"' + System.getProperty("java.class.path") + '"');
> +? ? ? ? ? ? try {
> +? ? ? ? ? ? ? ? File tmpFile = File.createTempFile("jmh-classpath-file", ".jar");
> +? ? ? ? ? ? ? ? tmpFile.deleteOnExit();
> +
> +? ? ? ? ? ? ? ? StringBuilder pw = new StringBuilder();
> +? ? ? ? ? ? ? ? for (String cp : System.getProperty("java.class.path").split(";")) {
> +? ? ? ? ? ? ? ? ? ? pw.append("/");
> +? ? ? ? ? ? ? ? ? ? pw.append(cp.replace('\\', '/').replace(" ", "%20"));
> +? ? ? ? ? ? ? ? ? ? if (!cp.endsWith(".jar")) {
> +? ? ? ? ? ? ? ? ? ? ? ? pw.append('/');
> +? ? ? ? ? ? ? ? ? ? }
> +? ? ? ? ? ? ? ? ? ? pw.append(" ");
> +? ? ? ? ? ? ? ? }
> +
> +? ? ? ? ? ? ? ? Manifest manifest = new Manifest();
> +? ? ? ? ? ? ? ? Attributes attributes = manifest.getMainAttributes();
> +? ? ? ? ? ? ? ? attributes.put(Attributes.Name.MANIFEST_VERSION, "1.0");
> +? ? ? ? ? ? ? ? attributes.putValue("Class-Path", pw.toString().trim());
> +
> +? ? ? ? ? ? ? ? try (JarOutputStream zout = new JarOutputStream(new FileOutputStream(tmpFile),
> manifest)) {
> +? ? ? ? ? ? ? ? ? ? ZipEntry ze = new ZipEntry("META-INF/");
> +? ? ? ? ? ? ? ? ? ? zout.putNextEntry(ze);
> +? ? ? ? ? ? ? ? }
> +
> +? ? ? ? ? ? ? ? command.add("\"" + tmpFile.getAbsolutePath() + "\"");
> +? ? ? ? ? ? } catch (IOException ex) {
> +? ? ? ? ? ? ? ? command.add('"' + System.getProperty("java.class.path") + '"');
> +? ? ? ? ? ? }
> ? ? ? ? ?} else {
> ? ? ? ? ? ? ?command.add(System.getProperty("java.class.path"));
> ? ? ? ? ?}
> 
> 
> 
> 
> 
> 
> 2017-12-21 8:24 GMT+01:00 Aleksey Shipilev <shade at redhat.com <mailto:shade at redhat.com>>:
> 
>     Hi David,
> 
>     On 12/12/2017 11:36 AM, D?vid Karnok wrote:
>     > I'd like to submit a patch (attached) to JMH's
>     > org.openjdk.jmh.runner.Runner.java that creates a temporary jar file with
>     > all classpath entries on Windows so that the command line doesn't overflow.
> 
>     The patch got stripped by the mailing list, apparently. (Wrong MIME type?)
> 
>     Thanks,
>     -Aleksey
> 
> 
> 
> 
> -- 
> Best regards,
> David Karnok



From akarnokd at gmail.com  Fri Dec 22 17:56:01 2017
From: akarnokd at gmail.com (=?UTF-8?Q?D=C3=A1vid_Karnok?=)
Date: Fri, 22 Dec 2017 18:56:01 +0100
Subject: [PATCH]: Forking benchmarks with long classpath on Windows (7)
In-Reply-To: <5132e129-d2f6-7e25-8531-14c7367bbd1c@redhat.com>
References: <CAAWwtm-ynUmOtb+cMTmixE+C42eCe8egAjexvVN1Dyaw5mbonA@mail.gmail.com>
 <372409f3-0a34-ded5-7290-bfa49f782a63@redhat.com>
 <CAAWwtm9-vPzuCYJ1AWeJGZueBFfxQ0vN71ASbR_HQNQpZDSW3g@mail.gmail.com>
 <5132e129-d2f6-7e25-8531-14c7367bbd1c@redhat.com>
Message-ID: <CAAWwtm96_jQHdz8=aas0j3ZQ7D8WBu2jBbO7u5QbJwnXENkHfA@mail.gmail.com>

Thanks Aleksey,

The OS dependence is by no coincidence. The problem my patch tries to solve
manifests itself on Windows only due to fundamental limits on how long
command lines can be on that platform. I think this temp file trick is
unnecessary on Linux/Mac.

Indeed, the classpath entries could be relative paths which my patch
unconditionally turns into incorrect paths by prepending that forward slash
that is otherwise required by absolute paths to work (example:
"/C:/somepath/file.jar").

Yes, Gradle is the main source of the lengthy path, but it uses a similar
technique on Java 8 to work around the Windows' limit. Unfortunately, when
the parent JMH is run, Java collects all those classpath entries in its JAR
file into the java.class.path string that JMH is then uses in its entirety.
I'd say Gradle's only fault is to dump all of its JAR files when the
initial JMH Java process is forked. The issue was brought up for Gradle and
the JMH plugin to no avail. Gradle would likely need deep changes to
introduce filtering and the JMH plugin has no control over the classpath
expansion which happens *after* it due to how Gradle works.

(Afaik, Java 9 started supporting the "java @file" notion in command line
where "file" can contain any number and any length of command line options.
Of course, JMH being Java 8+, this option doesn't work.)

Your take on the patch looks okay, I'll try to test it locally through the
JMH Gradle plugin.



2017-12-22 18:28 GMT+01:00 Aleksey Shipilev <shade at redhat.com>:

> Thanks!
>
> So I have a few problems with this patch: it takes a completely different
> route for one particular
> OS, which seems odd, and complicates testing, plus (I think) it relies on
> relative paths to
> classpath entries being the same as the path to the generated JAR, etc.
>
> Here's my take:
>   http://cr.openjdk.java.net/~shade/jmh/long-classpath.patch
>
> Hacking this further, my lingering doubt it is a JMH place to handle this
> kind of failure escalated
> to a firm concern. It would seems that Gradle is responsible for adding
> cruft to the classpath on
> the platforms that are sensitive to this. In that light, the workaround is
> disabled by default and
> can enabled with -Djmh.separateClasspathJAR=true. How about you ask
> Gradle and/or Gradle JMH plugin
> maintainers to fix their stack, and until then use this optional
> workaround?
>
> -Aleksey
>
>
> On 12/21/2017 11:01 AM, D?vid Karnok wrote:
> > Index: jmh-core/src/main/java/org/openjdk/jmh/runner/Runner.java
> > IDEA additional info:
> > Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
> > <+>UTF-8
> > ===================================================================
> > --- jmh-core/src/main/java/org/openjdk/jmh/runner/Runner.java(revision
> > 1434:1ddf31f810a3100b9433c3fedf24615e85b1d1a7)
> > +++ jmh-core/src/main/java/org/openjdk/jmh/runner/Runner.java(revision
> 1434+:1ddf31f810a3+)
> > @@ -47,6 +47,8 @@
> >  import java.nio.channels.OverlappingFileLockException;
> >  import java.util.*;
> >  import java.util.concurrent.TimeUnit;
> > +import java.util.jar.*;
> > +import java.util.zip.*;
> >
> >  /**
> >   * Runner executes JMH benchmarks.
> > @@ -826,7 +828,34 @@
> >          // assemble final process command
> >          command.add("-cp");
> >          if (Utils.isWindows()) {
> > -            command.add('"' + System.getProperty("java.class.path") +
> '"');
> > +            try {
> > +                File tmpFile = File.createTempFile("jmh-classpath-file",
> ".jar");
> > +                tmpFile.deleteOnExit();
> > +
> > +                StringBuilder pw = new StringBuilder();
> > +                for (String cp : System.getProperty("java.class.path").split(";"))
> {
> > +                    pw.append("/");
> > +                    pw.append(cp.replace('\\', '/').replace(" ",
> "%20"));
> > +                    if (!cp.endsWith(".jar")) {
> > +                        pw.append('/');
> > +                    }
> > +                    pw.append(" ");
> > +                }
> > +
> > +                Manifest manifest = new Manifest();
> > +                Attributes attributes = manifest.getMainAttributes();
> > +                attributes.put(Attributes.Name.MANIFEST_VERSION,
> "1.0");
> > +                attributes.putValue("Class-Path",
> pw.toString().trim());
> > +
> > +                try (JarOutputStream zout = new JarOutputStream(new
> FileOutputStream(tmpFile),
> > manifest)) {
> > +                    ZipEntry ze = new ZipEntry("META-INF/");
> > +                    zout.putNextEntry(ze);
> > +                }
> > +
> > +                command.add("\"" + tmpFile.getAbsolutePath() + "\"");
> > +            } catch (IOException ex) {
> > +                command.add('"' + System.getProperty("java.class.path")
> + '"');
> > +            }
> >          } else {
> >              command.add(System.getProperty("java.class.path"));
> >          }
> >
> >
> >
> >
> >
> >
> > 2017-12-21 8:24 GMT+01:00 Aleksey Shipilev <shade at redhat.com <mailto:
> shade at redhat.com>>:
> >
> >     Hi David,
> >
> >     On 12/12/2017 11:36 AM, D?vid Karnok wrote:
> >     > I'd like to submit a patch (attached) to JMH's
> >     > org.openjdk.jmh.runner.Runner.java that creates a temporary jar
> file with
> >     > all classpath entries on Windows so that the command line doesn't
> overflow.
> >
> >     The patch got stripped by the mailing list, apparently. (Wrong MIME
> type?)
> >
> >     Thanks,
> >     -Aleksey
> >
> >
> >
> >
> > --
> > Best regards,
> > David Karnok
>
>
>


-- 
Best regards,
David Karnok

From akarnokd at gmail.com  Fri Dec 22 20:12:29 2017
From: akarnokd at gmail.com (=?UTF-8?Q?D=C3=A1vid_Karnok?=)
Date: Fri, 22 Dec 2017 21:12:29 +0100
Subject: [PATCH]: Forking benchmarks with long classpath on Windows (7)
In-Reply-To: <CAAWwtm96_jQHdz8=aas0j3ZQ7D8WBu2jBbO7u5QbJwnXENkHfA@mail.gmail.com>
References: <CAAWwtm-ynUmOtb+cMTmixE+C42eCe8egAjexvVN1Dyaw5mbonA@mail.gmail.com>
 <372409f3-0a34-ded5-7290-bfa49f782a63@redhat.com>
 <CAAWwtm9-vPzuCYJ1AWeJGZueBFfxQ0vN71ASbR_HQNQpZDSW3g@mail.gmail.com>
 <5132e129-d2f6-7e25-8531-14c7367bbd1c@redhat.com>
 <CAAWwtm96_jQHdz8=aas0j3ZQ7D8WBu2jBbO7u5QbJwnXENkHfA@mail.gmail.com>
Message-ID: <CAAWwtm-uWojmAfKG0MdK37ioLkogzA-TPM-j-J2Rbb7nXgRQ7A@mail.gmail.com>

Unfortunately, the system property approach doesn't work. The JMH plugin
(0.4.5) can only set the `jvmArgs*` and not the -D param on the task that
runs the parent JMH process. (I.e., it would require a change here:
https://github.com/melix/jmh-gradle-plugin/blob/19ff7a4ed013e7980ef162534e0fefc91536d12c/src/main/groovy/me/champeau/gradle/JMHTask.java#L64
)

I don't have much hope the plugin would evolve in reasonable time to
accomodate this property so I suggest adding a workaround
for checking for the flag:

String jvmargs = ""
    + options.getJvmArgs().orElse(Collections.<String>emptyList())
    + options.getJvmArgsPrepend().orElse(Collections.<String>emptyList())
    + options.getJvmArgsAppend().orElse(Collections.<String>emptyList());


if (Boolean.getBoolean("jmh.separateClasspathJAR")
        || jvmargs.contains("jmh.separateClasspathJAR=true")) {


With this change, my benchmarks executed under Gradle 4.3.1, JMH Plugin
0.4.5 and setting jvmArgs.

2017-12-22 18:56 GMT+01:00 D?vid Karnok <akarnokd at gmail.com>:

> Thanks Aleksey,
>
> The OS dependence is by no coincidence. The problem my patch tries to
> solve manifests itself on Windows only due to fundamental limits on how
> long command lines can be on that platform. I think this temp file trick is
> unnecessary on Linux/Mac.
>
> Indeed, the classpath entries could be relative paths which my patch
> unconditionally turns into incorrect paths by prepending that forward slash
> that is otherwise required by absolute paths to work (example:
> "/C:/somepath/file.jar").
>
> Yes, Gradle is the main source of the lengthy path, but it uses a similar
> technique on Java 8 to work around the Windows' limit. Unfortunately, when
> the parent JMH is run, Java collects all those classpath entries in its JAR
> file into the java.class.path string that JMH is then uses in its entirety.
> I'd say Gradle's only fault is to dump all of its JAR files when the
> initial JMH Java process is forked. The issue was brought up for Gradle and
> the JMH plugin to no avail. Gradle would likely need deep changes to
> introduce filtering and the JMH plugin has no control over the classpath
> expansion which happens *after* it due to how Gradle works.
>
> (Afaik, Java 9 started supporting the "java @file" notion in command line
> where "file" can contain any number and any length of command line options.
> Of course, JMH being Java 8+, this option doesn't work.)
>
> Your take on the patch looks okay, I'll try to test it locally through the
> JMH Gradle plugin.
>
>
>
> 2017-12-22 18:28 GMT+01:00 Aleksey Shipilev <shade at redhat.com>:
>
>> Thanks!
>>
>> So I have a few problems with this patch: it takes a completely different
>> route for one particular
>> OS, which seems odd, and complicates testing, plus (I think) it relies on
>> relative paths to
>> classpath entries being the same as the path to the generated JAR, etc.
>>
>> Here's my take:
>>   http://cr.openjdk.java.net/~shade/jmh/long-classpath.patch
>>
>> Hacking this further, my lingering doubt it is a JMH place to handle this
>> kind of failure escalated
>> to a firm concern. It would seems that Gradle is responsible for adding
>> cruft to the classpath on
>> the platforms that are sensitive to this. In that light, the workaround
>> is disabled by default and
>> can enabled with -Djmh.separateClasspathJAR=true. How about you ask
>> Gradle and/or Gradle JMH plugin
>> maintainers to fix their stack, and until then use this optional
>> workaround?
>>
>> -Aleksey
>>
>>
>> On 12/21/2017 11:01 AM, D?vid Karnok wrote:
>> > Index: jmh-core/src/main/java/org/openjdk/jmh/runner/Runner.java
>> > IDEA additional info:
>> > Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
>> > <+>UTF-8
>> > ===================================================================
>> > --- jmh-core/src/main/java/org/openjdk/jmh/runner/Runner.java(revision
>> > 1434:1ddf31f810a3100b9433c3fedf24615e85b1d1a7)
>> > +++ jmh-core/src/main/java/org/openjdk/jmh/runner/Runner.java(revision
>> 1434+:1ddf31f810a3+)
>> > @@ -47,6 +47,8 @@
>> >  import java.nio.channels.OverlappingFileLockException;
>> >  import java.util.*;
>> >  import java.util.concurrent.TimeUnit;
>> > +import java.util.jar.*;
>> > +import java.util.zip.*;
>> >
>> >  /**
>> >   * Runner executes JMH benchmarks.
>> > @@ -826,7 +828,34 @@
>> >          // assemble final process command
>> >          command.add("-cp");
>> >          if (Utils.isWindows()) {
>> > -            command.add('"' + System.getProperty("java.class.path") +
>> '"');
>> > +            try {
>> > +                File tmpFile = File.createTempFile("jmh-classpath-file",
>> ".jar");
>> > +                tmpFile.deleteOnExit();
>> > +
>> > +                StringBuilder pw = new StringBuilder();
>> > +                for (String cp : System.getProperty("java.class.path").split(";"))
>> {
>> > +                    pw.append("/");
>> > +                    pw.append(cp.replace('\\', '/').replace(" ",
>> "%20"));
>> > +                    if (!cp.endsWith(".jar")) {
>> > +                        pw.append('/');
>> > +                    }
>> > +                    pw.append(" ");
>> > +                }
>> > +
>> > +                Manifest manifest = new Manifest();
>> > +                Attributes attributes = manifest.getMainAttributes();
>> > +                attributes.put(Attributes.Name.MANIFEST_VERSION,
>> "1.0");
>> > +                attributes.putValue("Class-Path",
>> pw.toString().trim());
>> > +
>> > +                try (JarOutputStream zout = new JarOutputStream(new
>> FileOutputStream(tmpFile),
>> > manifest)) {
>> > +                    ZipEntry ze = new ZipEntry("META-INF/");
>> > +                    zout.putNextEntry(ze);
>> > +                }
>> > +
>> > +                command.add("\"" + tmpFile.getAbsolutePath() + "\"");
>> > +            } catch (IOException ex) {
>> > +                command.add('"' + System.getProperty("java.class.path")
>> + '"');
>> > +            }
>> >          } else {
>> >              command.add(System.getProperty("java.class.path"));
>> >          }
>> >
>> >
>> >
>> >
>> >
>> >
>> > 2017-12-21 8:24 GMT+01:00 Aleksey Shipilev <shade at redhat.com <mailto:
>> shade at redhat.com>>:
>> >
>> >     Hi David,
>> >
>> >     On 12/12/2017 11:36 AM, D?vid Karnok wrote:
>> >     > I'd like to submit a patch (attached) to JMH's
>> >     > org.openjdk.jmh.runner.Runner.java that creates a temporary jar
>> file with
>> >     > all classpath entries on Windows so that the command line doesn't
>> overflow.
>> >
>> >     The patch got stripped by the mailing list, apparently. (Wrong MIME
>> type?)
>> >
>> >     Thanks,
>> >     -Aleksey
>> >
>> >
>> >
>> >
>> > --
>> > Best regards,
>> > David Karnok
>>
>>
>>
>
>
> --
> Best regards,
> David Karnok
>



-- 
Best regards,
David Karnok

From anuraaga at gmail.com  Sun Dec 24 15:48:43 2017
From: anuraaga at gmail.com (Anuraag Agrawal)
Date: Mon, 25 Dec 2017 00:48:43 +0900
Subject: JMH does not correctly handle Enum params that override toString()
Message-ID: <CAOWvS+0PwpzFtnA3PcD1Z8mSm1Y5hsnJ0e8f-opN6wkG8CEgjw@mail.gmail.com>

Hi all,

Currently, JMH uses Enum.toString() to convert enum params to string for
specification from the @Param annotation or the command line. This string,
is then passed into Enum.valueOf when running the benchmark.

However, Java explicitly allows enums to override toString() to something
that doesn't match the descriptor name

https://docs.oracle.com/javase/7/docs/api/java/lang/Enum.html#toString()

Instead, name() should be used when round-tripping through valueOf is
required

https://docs.oracle.com/javase/7/docs/api/java/lang/Enum.html#name()

This affects the reflection and ASM generators. Annotation processor is
unaffected since it directly reads the declaration token.

The patch is simple and I've attached it, but could not find any tests that
round-trip reflection/ASM benchmarks from compile -> run (the issue only
affects running a benchmark, the compile succeeds). if this patch makes
sense, any advice on writing a regression test would be appreciated.

Thanks,
- Anuraag

From shade at redhat.com  Sun Dec 24 15:54:37 2017
From: shade at redhat.com (Aleksey Shipilev)
Date: Sun, 24 Dec 2017 16:54:37 +0100
Subject: JMH does not correctly handle Enum params that override toString()
In-Reply-To: <CAOWvS+0PwpzFtnA3PcD1Z8mSm1Y5hsnJ0e8f-opN6wkG8CEgjw@mail.gmail.com>
References: <CAOWvS+0PwpzFtnA3PcD1Z8mSm1Y5hsnJ0e8f-opN6wkG8CEgjw@mail.gmail.com>
Message-ID: <8b75a1ca-f071-4cbd-0f48-3e25c2d81c68@redhat.com>

On 12/24/2017 04:48 PM, Anuraag Agrawal wrote:
> Instead, name() should be used when round-tripping through valueOf is
> required
> 
> https://docs.oracle.com/javase/7/docs/api/java/lang/Enum.html#name()
> 
> This affects the reflection and ASM generators. Annotation processor is
> unaffected since it directly reads the declaration token.

That makes sense.

> The patch is simple and I've attached it, 

Patch got stripped :( Please put it inline?

> but could not find any tests that
> round-trip reflection/ASM benchmarks from compile -> run (the issue only
> affects running a benchmark, the compile succeeds). if this patch makes
> sense, any advice on writing a regression test would be appreciated.

Run the build with appropriate profile, and the tests would run with appropriate generator:

 $ mvn clean install -Pasm
 $ mvn clean install -Preflection

Thanks,
-Aleksey


From shade at redhat.com  Sun Dec 24 15:57:46 2017
From: shade at redhat.com (Aleksey Shipilev)
Date: Sun, 24 Dec 2017 16:57:46 +0100
Subject: JMH does not correctly handle Enum params that override toString()
In-Reply-To: <8b75a1ca-f071-4cbd-0f48-3e25c2d81c68@redhat.com>
References: <CAOWvS+0PwpzFtnA3PcD1Z8mSm1Y5hsnJ0e8f-opN6wkG8CEgjw@mail.gmail.com>
 <8b75a1ca-f071-4cbd-0f48-3e25c2d81c68@redhat.com>
Message-ID: <2d362dad-903b-1d6c-9e3b-e5ff11e2edca@redhat.com>

On 12/24/2017 04:54 PM, Aleksey Shipilev wrote:
> On 12/24/2017 04:48 PM, Anuraag Agrawal wrote:
>> Instead, name() should be used when round-tripping through valueOf is
>> required
>>
>> https://docs.oracle.com/javase/7/docs/api/java/lang/Enum.html#name()
>>
>> This affects the reflection and ASM generators. Annotation processor is
>> unaffected since it directly reads the declaration token.
> 
> That makes sense.
> 
>> The patch is simple and I've attached it, 
> 
> Patch got stripped :( Please put it inline?
> 
>> but could not find any tests that
>> round-trip reflection/ASM benchmarks from compile -> run (the issue only
>> affects running a benchmark, the compile succeeds). if this patch makes
>> sense, any advice on writing a regression test would be appreciated.
> 
> Run the build with appropriate profile, and the tests would run with appropriate generator:
> 
>  $ mvn clean install -Pasm
>  $ mvn clean install -Preflection

Ah, also make the regression test that trips on this issue, and run with some profile above. Our CI
runs all tests with all profiles, so that would be a valid regression test. Since this is runtime
failure, put the test somewhere in jmh-core-it.

Thanks,
-Aleksey



From anuraaga at gmail.com  Sun Dec 24 16:32:34 2017
From: anuraaga at gmail.com (Anuraag Agrawal)
Date: Mon, 25 Dec 2017 01:32:34 +0900
Subject: JMH does not correctly handle Enum params that override toString()
In-Reply-To: <2d362dad-903b-1d6c-9e3b-e5ff11e2edca@redhat.com>
References: <CAOWvS+0PwpzFtnA3PcD1Z8mSm1Y5hsnJ0e8f-opN6wkG8CEgjw@mail.gmail.com>
 <8b75a1ca-f071-4cbd-0f48-3e25c2d81c68@redhat.com>
 <2d362dad-903b-1d6c-9e3b-e5ff11e2edca@redhat.com>
Message-ID: <CAOWvS+0JqrsdAPgvOFhf=zH=c=R6g=NEMQj8mY7gd25FVoqs4w@mail.gmail.com>

Thanks Alexey, I will take those pointers and try producing a regression
test tomorrow and submit a more complete patch.
For reference, the original patch I sent that got stripped is this (just
changes toString() to name() since we know the variables are Enum):

# HG changeset patch
# User Anuraag Agrawal <anuraaga at gmail.com>
# Date 1512560148 -32400
#      Wed Dec 06 20:35:48 2017 +0900
# Node ID c8e5e85046656062bb2070318e51fac4cff4b4e4
# Parent  1ddf31f810a3100b9433c3fedf24615e85b1d1a7
Use Enum.name() instead of Enum.toString() for determining the string value
of enum params. Only Enum.name() is guaranteed to round-trip through
Enum.valueOf during execution, the specification explicitly allows
overriding toString() to something that will not round-trip.

diff --git
a/jmh-generator-asm/src/main/java/org/openjdk/jmh/generators/asm/ASMClassInfo.java
b/jmh-generator-asm/src/main/java/org/openjdk/jmh/generators/asm/ASMClassInfo.java
---
a/jmh-generator-asm/src/main/java/org/openjdk/jmh/generators/asm/ASMClassInfo.java
+++
b/jmh-generator-asm/src/main/java/org/openjdk/jmh/generators/asm/ASMClassInfo.java
@@ -221,7 +221,7 @@
             try {
                 Collection<String> res = new ArrayList<>();
                 for (Object cnst : Class.forName(origQualifiedName, false,
Thread.currentThread().getContextClassLoader()).getEnumConstants()) {
-                    res.add(cnst.toString());
+                    res.add(((Enum<?>) cnst).name());
                 }
                 return res;
             } catch (ClassNotFoundException e) {
diff --git
a/jmh-generator-reflection/src/main/java/org/openjdk/jmh/generators/reflection/RFClassInfo.java
b/jmh-generator-reflection/src/main/java/org/openjdk/jmh/generators/reflection/RFClassInfo.java
---
a/jmh-generator-reflection/src/main/java/org/openjdk/jmh/generators/reflection/RFClassInfo.java
+++
b/jmh-generator-reflection/src/main/java/org/openjdk/jmh/generators/reflection/RFClassInfo.java
@@ -162,7 +162,7 @@
     public Collection<String> getEnumConstants() {
         Collection<String> res = new ArrayList<>();
         for (Object cnst : klass.getEnumConstants()) {
-            res.add(cnst.toString());
+            res.add(((Enum<?>) cnst).name());
         }
         return res;
     }


On Mon, Dec 25, 2017 at 12:57 AM, Aleksey Shipilev <shade at redhat.com> wrote:

> On 12/24/2017 04:54 PM, Aleksey Shipilev wrote:
> > On 12/24/2017 04:48 PM, Anuraag Agrawal wrote:
> >> Instead, name() should be used when round-tripping through valueOf is
> >> required
> >>
> >> https://docs.oracle.com/javase/7/docs/api/java/lang/Enum.html#name()
> >>
> >> This affects the reflection and ASM generators. Annotation processor is
> >> unaffected since it directly reads the declaration token.
> >
> > That makes sense.
> >
> >> The patch is simple and I've attached it,
> >
> > Patch got stripped :( Please put it inline?
> >
> >> but could not find any tests that
> >> round-trip reflection/ASM benchmarks from compile -> run (the issue only
> >> affects running a benchmark, the compile succeeds). if this patch makes
> >> sense, any advice on writing a regression test would be appreciated.
> >
> > Run the build with appropriate profile, and the tests would run with
> appropriate generator:
> >
> >  $ mvn clean install -Pasm
> >  $ mvn clean install -Preflection
>
> Ah, also make the regression test that trips on this issue, and run with
> some profile above. Our CI
> runs all tests with all profiles, so that would be a valid regression
> test. Since this is runtime
> failure, put the test somewhere in jmh-core-it.
>
> Thanks,
> -Aleksey
>
>
>

From anuraaga at gmail.com  Tue Dec 26 10:15:56 2017
From: anuraaga at gmail.com (Anuraag Agrawal)
Date: Tue, 26 Dec 2017 19:15:56 +0900
Subject: [PATCH] Support enum params that override Object#toString()
Message-ID: <CAOWvS+1LdGZz-5v69DuDju9YzXTdQymeZRg0ReC2EZ=KtfHXYg@mail.gmail.com>

Hi all,

As discussed in
http://mail.openjdk.java.net/pipermail/jmh-dev/2017-December/002678.html
currently JMH does not handle enum params where the enum overrides toString
when using the reflection or ASM generators. This is because the spec
explicitly allows enums to override toString() to something that doesn't
match the descriptor's name, but restoring the enum using valueOf can only
happen with the descriptor's name.

https://docs.oracle.com/javase/7/docs/api/java/lang/Enum.html#toString()

This patch changes use of toString() to name(), which is always guaranteed
to be the descriptor's name and round-trippable.

https://docs.oracle.com/javase/7/docs/api/java/lang/Enum.html#name()

The patch has been attached and inlined below, including a regression test
which fails without the logic change.

Note, I just signed and emailed the Oracle CLA, so it may take some time
for that to get processed.

Thanks. Patch follows

# HG changeset patch
# User Anuraag Agrawal <anuraaga at gmail.com>
# Date 1512560148 -32400
#      Wed Dec 06 20:35:48 2017 +0900
# Node ID 4eefe5751280399fdbde2df655d2dcf43f2de557
# Parent  1ddf31f810a3100b9433c3fedf24615e85b1d1a7
Use Enum.name() instead of Enum.toString() for determining the string value
of enum params. Only Enum.name() is guaranteed to round-trip through
Enum.valueOf during execution, the specification explicitly allows
overriding toString() to something that will not round-trip.

diff --git
a/jmh-core-it/src/test/java/org/openjdk/jmh/it/params/EnumParamSequenceTest.java
b/jmh-core-it/src/test/java/org/openjdk/jmh/it/params/EnumParamToStringOverridingTest.java
copy from
jmh-core-it/src/test/java/org/openjdk/jmh/it/params/EnumParamSequenceTest.java
copy to
jmh-core-it/src/test/java/org/openjdk/jmh/it/params/EnumParamToStringOverridingTest.java
---
a/jmh-core-it/src/test/java/org/openjdk/jmh/it/params/EnumParamSequenceTest.java
+++
b/jmh-core-it/src/test/java/org/openjdk/jmh/it/params/EnumParamToStringOverridingTest.java
@@ -45,56 +45,32 @@
 @Warmup(iterations = 1, time = 100, timeUnit = TimeUnit.MICROSECONDS)
 @Fork(1)
 @State(Scope.Thread)
-public class EnumParamSequenceTest {
+public class EnumParamToStringOverridingTest {

     @Param({"VALUE_A", "VALUE_B", "VALUE_C"})
     public SampleEnumA a;

-    @Param({"VALUE_A", "VALUE_B", "VALUE_C"})
-    public SampleEnumB b;
-
     @Benchmark
     public void test() {
         Fixtures.work();
     }

     @Test
-    public void full() throws RunnerException {
+    public void normal() throws RunnerException {
         Options opts = new OptionsBuilder()
                 .include(Fixtures.getTestMask(this.getClass()))
                 .shouldFailOnError(true)
                 .build();

-        Assert.assertEquals(3 * 3, new Runner(opts).run().size());
-    }
-
-    @Test
-    public void constrainedA() throws RunnerException {
-        Options opts = new OptionsBuilder()
-                .include(Fixtures.getTestMask(this.getClass()))
-                .shouldFailOnError(true)
-                .param("a", SampleEnumA.VALUE_A.name())
-                .build();
-
-        Assert.assertEquals(1 * 3, new Runner(opts).run().size());
-    }
-
-    @Test
-    public void constrainedB() throws RunnerException {
-        Options opts = new OptionsBuilder()
-                .include(Fixtures.getTestMask(this.getClass()))
-                .shouldFailOnError(true)
-                .param("b", SampleEnumB.VALUE_A.name())
-                .build();
-
-        Assert.assertEquals(1*3, new Runner(opts).run().size());
+        Assert.assertEquals(3, new Runner(opts).run().size());
     }

     public enum SampleEnumA {
-        VALUE_A, VALUE_B, VALUE_C
-    }
+        VALUE_A, VALUE_B, VALUE_C;

-    public enum SampleEnumB {
-        VALUE_A, VALUE_B, VALUE_C
+        @Override
+        public String toString() {
+            return name().toLowerCase();
+        }
     }
 }
diff --git
a/jmh-generator-asm/src/main/java/org/openjdk/jmh/generators/asm/ASMClassInfo.java
b/jmh-generator-asm/src/main/java/org/openjdk/jmh/generators/asm/ASMClassInfo.java
---
a/jmh-generator-asm/src/main/java/org/openjdk/jmh/generators/asm/ASMClassInfo.java
+++
b/jmh-generator-asm/src/main/java/org/openjdk/jmh/generators/asm/ASMClassInfo.java
@@ -221,7 +221,7 @@
             try {
                 Collection<String> res = new ArrayList<>();
                 for (Object cnst : Class.forName(origQualifiedName, false,
Thread.currentThread().getContextClassLoader()).getEnumConstants()) {
-                    res.add(cnst.toString());
+                    res.add(((Enum<?>) cnst).name());
                 }
                 return res;
             } catch (ClassNotFoundException e) {
diff --git
a/jmh-generator-reflection/src/main/java/org/openjdk/jmh/generators/reflection/RFClassInfo.java
b/jmh-generator-reflection/src/main/java/org/openjdk/jmh/generators/reflection/RFClassInfo.java
---
a/jmh-generator-reflection/src/main/java/org/openjdk/jmh/generators/reflection/RFClassInfo.java
+++
b/jmh-generator-reflection/src/main/java/org/openjdk/jmh/generators/reflection/RFClassInfo.java
@@ -162,7 +162,7 @@
     public Collection<String> getEnumConstants() {
         Collection<String> res = new ArrayList<>();
         for (Object cnst : klass.getEnumConstants()) {
-            res.add(cnst.toString());
+            res.add(((Enum<?>) cnst).name());
         }
         return res;
     }

- Anuraag
-------------- next part --------------
A non-text attachment was scrubbed...
Name: enum-name.patch
Type: text/x-patch
Size: 4491 bytes
Desc: not available
URL: <http://mail.openjdk.java.net/pipermail/jmh-dev/attachments/20171226/081311bb/enum-name.patch>

From qwwdfsad at gmail.com  Tue Dec 26 19:19:41 2017
From: qwwdfsad at gmail.com (Vsevolod Tolstopyatov)
Date: Tue, 26 Dec 2017 22:19:41 +0300
Subject: DTrace asm profiler for Mac OS X
In-Reply-To: <CADZL2=taY19Svv12PU2nSEU=tTkvoX7ivHLgGFFKLysYn8=sHQ@mail.gmail.com>
References: <CAEH0ZEGMKxi2WCrAOvy_UH-H80aFGGXCS6kAswdk-MU0Zbq_iw@mail.gmail.com>
 <CADZL2=uKU=9_fHa5e5VMPBb_fEN+X1iu0xXmQs+UgCrm=vZ6qw@mail.gmail.com>
 <CADZL2=tn5T9CjOhUxM1wUF6VV2qi457kpmPCsamkK30SySi7Xw@mail.gmail.com>
 <CAEH0ZEEiDR8N-vWpg=v-oJ5TMBvxB56C9gAJ49ZoBWFrnR4vQg@mail.gmail.com>
 <CAEH0ZEHzrdtkbM_-N1g_O-4_0P0tASiJKHfn_6-cOPonG0yiYg@mail.gmail.com>
 <0b213005-161b-a2f8-c5c4-c2df4d61c3ed@redhat.com>
 <CADZL2=taY19Svv12PU2nSEU=tTkvoX7ivHLgGFFKLysYn8=sHQ@mail.gmail.com>
Message-ID: <CAEH0ZEFybwYhL2E6DnmvDZLQYM4k8X8q-B1mTwJW9kQLkemC=Q@mail.gmail.com>

Hi, could you share your benchmark?
I've just re-applied my patch over clean repo and
run JMHSample_37_CacheAccess with dtrace-profiler, everything works as
expected, so maybe your hottest region lies in kernel code.

>With perf, I would get some content. With dtrace, nothing.
Are you running both on Linux?



--
Best regards,
Tolstopyatov Vsevolod

On Wed, Dec 13, 2017 at 7:25 PM, Henri Tremblay <henri.tremblay at gmail.com>
wrote:

> A bit late but my only problem right now is that I don't get any hot
> section. Which is weird.
>
> With perf, I would get some content. With dtrace, nothing.
>
> However, I am not an expert in using both. So maybe some javac or java
> arguments are required to get nice results. Is it the case?
>
> Thanks,
> Henri
>
> On 23 November 2017 at 13:04, Aleksey Shipilev <shade at redhat.com> wrote:
>
>> On 11/23/2017 09:09 AM, Vsevolod Tolstopyatov wrote:
>> > Hello,
>> >
>> > Any news about this patch? Is it going into jmh?
>>
>> It will. Just let me figure out some Mac testing.
>>
>> -Aleksey
>>
>>
>

From henri.tremblay at gmail.com  Wed Dec 27 14:56:34 2017
From: henri.tremblay at gmail.com (Henri Tremblay)
Date: Wed, 27 Dec 2017 09:56:34 -0500
Subject: DTrace asm profiler for Mac OS X
In-Reply-To: <CAEH0ZEFybwYhL2E6DnmvDZLQYM4k8X8q-B1mTwJW9kQLkemC=Q@mail.gmail.com>
References: <CAEH0ZEGMKxi2WCrAOvy_UH-H80aFGGXCS6kAswdk-MU0Zbq_iw@mail.gmail.com>
 <CADZL2=uKU=9_fHa5e5VMPBb_fEN+X1iu0xXmQs+UgCrm=vZ6qw@mail.gmail.com>
 <CADZL2=tn5T9CjOhUxM1wUF6VV2qi457kpmPCsamkK30SySi7Xw@mail.gmail.com>
 <CAEH0ZEEiDR8N-vWpg=v-oJ5TMBvxB56C9gAJ49ZoBWFrnR4vQg@mail.gmail.com>
 <CAEH0ZEHzrdtkbM_-N1g_O-4_0P0tASiJKHfn_6-cOPonG0yiYg@mail.gmail.com>
 <0b213005-161b-a2f8-c5c4-c2df4d61c3ed@redhat.com>
 <CADZL2=taY19Svv12PU2nSEU=tTkvoX7ivHLgGFFKLysYn8=sHQ@mail.gmail.com>
 <CAEH0ZEFybwYhL2E6DnmvDZLQYM4k8X8q-B1mTwJW9kQLkemC=Q@mail.gmail.com>
Message-ID: <CADZL2=tt3ki-XxF=27OD2hEhVL_+mij7cyTu5u2cFYTsKGrhFw@mail.gmail.com>

No. One was Linux (perf), the other was OSX (dtrace). Let me put the
benchmark out.

On 26 December 2017 at 14:19, Vsevolod Tolstopyatov <qwwdfsad at gmail.com>
wrote:

> Hi, could you share your benchmark?
> I've just re-applied my patch over clean repo and
> run JMHSample_37_CacheAccess with dtrace-profiler, everything works as
> expected, so maybe your hottest region lies in kernel code.
>
> >With perf, I would get some content. With dtrace, nothing.
> Are you running both on Linux?
>
>
>
> --
> Best regards,
> Tolstopyatov Vsevolod
>
> On Wed, Dec 13, 2017 at 7:25 PM, Henri Tremblay <henri.tremblay at gmail.com>
> wrote:
>
>> A bit late but my only problem right now is that I don't get any hot
>> section. Which is weird.
>>
>> With perf, I would get some content. With dtrace, nothing.
>>
>> However, I am not an expert in using both. So maybe some javac or java
>> arguments are required to get nice results. Is it the case?
>>
>> Thanks,
>> Henri
>>
>> On 23 November 2017 at 13:04, Aleksey Shipilev <shade at redhat.com> wrote:
>>
>>> On 11/23/2017 09:09 AM, Vsevolod Tolstopyatov wrote:
>>> > Hello,
>>> >
>>> > Any news about this patch? Is it going into jmh?
>>>
>>> It will. Just let me figure out some Mac testing.
>>>
>>> -Aleksey
>>>
>>>
>>
>

From henri.tremblay at gmail.com  Thu Dec 28 19:35:48 2017
From: henri.tremblay at gmail.com (Henri Tremblay)
Date: Thu, 28 Dec 2017 14:35:48 -0500
Subject: DTrace asm profiler for Mac OS X
In-Reply-To: <CADZL2=tt3ki-XxF=27OD2hEhVL_+mij7cyTu5u2cFYTsKGrhFw@mail.gmail.com>
References: <CAEH0ZEGMKxi2WCrAOvy_UH-H80aFGGXCS6kAswdk-MU0Zbq_iw@mail.gmail.com>
 <CADZL2=uKU=9_fHa5e5VMPBb_fEN+X1iu0xXmQs+UgCrm=vZ6qw@mail.gmail.com>
 <CADZL2=tn5T9CjOhUxM1wUF6VV2qi457kpmPCsamkK30SySi7Xw@mail.gmail.com>
 <CAEH0ZEEiDR8N-vWpg=v-oJ5TMBvxB56C9gAJ49ZoBWFrnR4vQg@mail.gmail.com>
 <CAEH0ZEHzrdtkbM_-N1g_O-4_0P0tASiJKHfn_6-cOPonG0yiYg@mail.gmail.com>
 <0b213005-161b-a2f8-c5c4-c2df4d61c3ed@redhat.com>
 <CADZL2=taY19Svv12PU2nSEU=tTkvoX7ivHLgGFFKLysYn8=sHQ@mail.gmail.com>
 <CAEH0ZEFybwYhL2E6DnmvDZLQYM4k8X8q-B1mTwJW9kQLkemC=Q@mail.gmail.com>
 <CADZL2=tt3ki-XxF=27OD2hEhVL_+mij7cyTu5u2cFYTsKGrhFw@mail.gmail.com>
Message-ID: <CADZL2=snhXypM-aWf-P7BizXdXZ656+_i6pK24eLAPut5Z+ApA@mail.gmail.com>

I am far far far from being an expert here so I'm pretty sure you will
throw some stupid mistake in my face but here it goes.

You can use
https://github.com/JCTools/JCTools/tree/master/jctools-benchmarks.

I did on Linux:
java -jar target/microbenchmarks.jar -f 1 --prof=perfasm
org.jctools.maps.nhbm_test.jmh.ConcurrentMapThroughput

And got (yes, with an error on PrintAssembly):

ERROR: No address lines detected in assembly capture, make sure your JDK is
PrintAssembly-enabled:
    https://wiki.openjdk.java.net/display/HotSpot/PrintAssembly

Perf output processed (skipped 2.844 seconds):
 Column 1: cycles (12218 events)
 Column 2: instructions (12169 events)

Hottest code regions (>10.00% "cycles" events):

....[Hottest Region
1]..............................................................................
perf-52432.map, [unknown] (177 bytes)

 <no assembly is recorded, native region>
....................................................................................................
 19.81%   11.78%  <total for region 1>

....[Hottest Region
2]..............................................................................
perf-52432.map, [unknown] (381 bytes)

 <no assembly is recorded, native region>
....................................................................................................
 15.03%   12.21%  <total for region 2>

....[Hottest Region
3]..............................................................................
perf-52432.map, [unknown] (138 bytes)

 <no assembly is recorded, native region>
....................................................................................................
 10.38%    6.35%  <total for region 3>

....[Hottest
Regions]...............................................................................
 19.81%   11.78%      perf-52432.map  [unknown] (177 bytes)
 15.03%   12.21%      perf-52432.map  [unknown] (381 bytes)
 10.38%    6.35%      perf-52432.map  [unknown] (138 bytes)
  9.82%   37.09%      perf-52432.map  [unknown] (447 bytes)
  8.22%    2.47%      perf-52432.map  [unknown] (72 bytes)
  7.89%    1.69%      perf-52432.map  [unknown] (28 bytes)
  7.65%    1.69%      perf-52432.map  [unknown] (33 bytes)
  5.20%    2.94%      perf-52432.map  [unknown] (173 bytes)
  1.98%    1.59%      perf-52432.map  [unknown] (287 bytes)
  1.85%    4.54%      perf-52432.map  [unknown] (59 bytes)
  1.81%    4.48%      perf-52432.map  [unknown] (55 bytes)
  1.51%    0.96%      perf-52432.map  [unknown] (116 bytes)
  1.47%    1.83%      perf-52432.map  [unknown] (71 bytes)
  1.26%    1.25%              kernel  [unknown] (2 bytes)
  1.15%    0.53%      perf-52432.map  [unknown] (95 bytes)
  0.89%    0.40%      perf-52432.map  [unknown] (75 bytes)
  0.56%    0.05%              kernel  [unknown] (0 bytes)
  0.53%    2.34%      perf-52432.map  [unknown] (92 bytes)
  0.45%    1.16%      perf-52432.map  [unknown] (8 bytes)
  0.44%    2.47%      perf-52432.map  [unknown] (8 bytes)
  2.11%    2.14%  <...other 199 warm regions...>
....................................................................................................
100.00%   99.99%  <totals>

....[Hottest Methods (after
inlining)]..............................................................
 96.95%   97.51%      perf-52432.map  [unknown]
  2.76%    2.10%              kernel  [unknown]
  0.03%    0.07%           libjvm.so  fileStream::write
  0.02%    0.01%        libc-2.12.so  __strlen_sse42
  0.02%                 libc-2.12.so  _IO_file_xsputn@@GLIBC_2.2.5
  0.02%                 libc-2.12.so  __printf_fp
  0.01%                    libjvm.so  CompileBroker::set_last_compile
  0.01%                    libjvm.so  CodeCache::allocate
  0.01%           libpthread-2.12.so  pthread_mutex_unlock
  0.01%                    libjvm.so  os::set_priority
  0.01%                    libjvm.so
DebugInformationRecorder::find_sharable_decode_offset
  0.01%           libpthread-2.12.so  pthread_cond_wait@@GLIBC_2.3.2
  0.01%                    libjvm.so
CompileBroker::invoke_compiler_on_method
  0.01%                    libjvm.so  ciEnv::get_klass_by_index_impl
  0.01%    0.01%           libjvm.so  PhiResolverState::reset
  0.01%                    libjvm.so  CompilerOracle::should_exclude
  0.01%                    libjvm.so  CompilerOracle::has_option_string
  0.01%                    libjvm.so  LinearScan::compute_local_live_sets
  0.01%                    libjvm.so  OptoRuntime::new_instance_C
  0.01%                    libjvm.so  ChunkPool::allocate
  0.10%    0.02%  <...other 12 warm methods...>
....................................................................................................
100.00%   99.71%  <totals>

....[Distribution by
Source]........................................................................
 96.95%   97.51%      perf-52432.map
  2.76%    2.10%              kernel
  0.22%    0.31%           libjvm.so
  0.05%    0.06%        libc-2.12.so
  0.02%           libpthread-2.12.so
....................................................................................................
100.00%   99.99%  <totals>

But on OSX when I do

java -jar target/microbenchmarks.jar -f 1 --prof=dtraceasm
org.jctools.maps.nhbm_test.jmh.ConcurrentMapThroughput

I get:

PrintAssembly processed: 193901 total address lines.
Perf output processed (skipped 6.097 seconds):
 Column 1: sampled_pc (0 events)

WARNING: No hottest code region above the threshold (10.00%) for
disassembly.
Use "hotThreshold" profiler option to lower the filter threshold.

....[Hottest
Regions]...............................................................................
....................................................................................................
         <totals>

....[Hottest Methods (after
inlining)]..............................................................
....................................................................................................
         <totals>

....[Distribution by
Source]........................................................................
....................................................................................................
         <totals>

WARNING: The perf event count is suspiciously low (0). The performance data
might be
inaccurate or misleading. Try to do the profiling again, or tune up the
sampling frequency.

Which seem pretty empty.

Henri

On 27 December 2017 at 09:56, Henri Tremblay <henri.tremblay at gmail.com>
wrote:

> No. One was Linux (perf), the other was OSX (dtrace). Let me put the
> benchmark out.
>
> On 26 December 2017 at 14:19, Vsevolod Tolstopyatov <qwwdfsad at gmail.com>
> wrote:
>
>> Hi, could you share your benchmark?
>> I've just re-applied my patch over clean repo and
>> run JMHSample_37_CacheAccess with dtrace-profiler, everything works as
>> expected, so maybe your hottest region lies in kernel code.
>>
>> >With perf, I would get some content. With dtrace, nothing.
>> Are you running both on Linux?
>>
>>
>>
>> --
>> Best regards,
>> Tolstopyatov Vsevolod
>>
>> On Wed, Dec 13, 2017 at 7:25 PM, Henri Tremblay <henri.tremblay at gmail.com
>> > wrote:
>>
>>> A bit late but my only problem right now is that I don't get any hot
>>> section. Which is weird.
>>>
>>> With perf, I would get some content. With dtrace, nothing.
>>>
>>> However, I am not an expert in using both. So maybe some javac or java
>>> arguments are required to get nice results. Is it the case?
>>>
>>> Thanks,
>>> Henri
>>>
>>> On 23 November 2017 at 13:04, Aleksey Shipilev <shade at redhat.com> wrote:
>>>
>>>> On 11/23/2017 09:09 AM, Vsevolod Tolstopyatov wrote:
>>>> > Hello,
>>>> >
>>>> > Any news about this patch? Is it going into jmh?
>>>>
>>>> It will. Just let me figure out some Mac testing.
>>>>
>>>> -Aleksey
>>>>
>>>>
>>>
>>
>