JMH and ocperf

[Sergey Melnikov]

I got your idea.

There are few my points:

1. It's common approach to replace real executable name with
(environment) variable. For example, JDK makefiles uses
CD/AWK/CAT/BASENANE/CXX/... variables. Commonly used tools like
cmake/gn/... let user to override executable name (or specify
additional flags) by overriding particular environment variables.
2. I'm not sure it's good idea to name wrapper the same way as wrapped
executable and it should be done only as "last resort". It's a bit
confusing, user should remember he invokes either wrapper of wrapped
executable. It makes constantly take care of PATH variable ("what do I
use either wrapper or wrapped") and makes a bit more difficult to
switch between wrapper and wrapped (It makes to use tricks with PATH
variable like "PATH=/where/my/wrapper/is:$PATH java ...").
3. We leave in real world, so, I think Andi will not be happy to
rename his tool only because we decided here it's better.

--Sergey

On Fri, Nov 18, 2016 at 2:16 PM, Aleksey Shipilev <shade at redhat.com> wrote:
> What you are explaining is the *output* story: sure, JMH will receive
> the same output from either perf or ocperf.py.
>
> My question is about the *input* story: why can't ocperf.py be the
> drop-in compatible wrapper for perf? JMH does not do anything magical
> about perf, it just invokes "perf <blah-blah>", and if any external tool
> is providing the same input interface (e.g. accepts the same command
> line the perf does) -- nothing needs changing in JMH.
>
> I do believe this is a better interop story for the tool that claims to
> be the perf wrapper: both input and output stories should match the ones
> from the tool you are wrapping.
>
> Thanks,
> -Aleksey
>
> On 11/18/2016 12:01 PM, Sergey Melnikov wrote:
>> Hi,
>>
>> Correct me, if I didn't understand you right.
>>
>> In fact, pmu-tools/ocperf does it. Let's treat ocperf as one more
>> system-wide event-based profiler. But ocperf is just a wrapper over
>> perf which passes flags to perf. Therefore, ocperf's output format and
>> perf's one are equal. So, there is not need to write one more profiler
>> parser.
>>
>> So, I believe there are two options:
>>
>> 1. Use current Perf infrastructure (classes) in JMH and let user to
>> overwrite executable name via environment variable. Thus, we will use
>> the same parser as perfasm and changes are minimal. Moreover, user is
>> fully responsible for issues of incompatibility between actual
>> executable output and perf's reference output.  I suggested this
>> approach in my first letter.
>> 2. Add one more profiler visible from '-lprof' flag output. In this
>> approach it'd required to introduce new classes responsible for new
>> profiles and register them as 'Profiler' in JMH internals. These
>> classes should share code with perfasm's parser (we don't want to have
>> 2 parsers for the same format, isn't it?). I believe it'll make JMH
>> code slightly more difficult to understand. Also, it makes us to
>> support this parser and be responsible for all issues of this parser.
>>
>> Yes, I agree the second way is a bit 'more right', but I'm not sure it
>> worth to be implemented because there is no so many people who is
>> interested in such a deep analysis. Also, environment variable
>> overriding is user's area of responsibility and don't make us to
>> support additional code.
>>
>> But anyway, it's just my point of view. If you disagree, I can
>> implement the second approach.
>>
>> --Sergey
>>
>> On Fri, Nov 18, 2016 at 12:35 PM, Aleksey Shipilev <shade at redhat.com> wrote:
>>> Hi,
>>>
>>> On 11/17/2016 09:57 PM, Sergey Melnikov wrote:
>>>> So, I'd like to propose to modify JMH a bit: to let user to overwrite
>>>> 'perf' executable name via environment variable. It let to use ocperf
>>>> with JMH and to specify symbolic names via JMH parameters.
>>>>
>>>> Therefore, JMH run will look like:
>>>>
>>>> $ PERF=ocperf.py java -jar target/benchmarks.jar -prof
>>>> perfasm:event=br_inst_retired.far_branch
>>>>
>>>> Alexey, what do you think, if I implement this functionality, will it
>>>> be possible to merge fix to trunk?
>>>
>>> I am not resistant to the idea, but I am missing something here.
>>>
>>> Why can't pmu-tools project provide the system/user-wide alternative for
>>> perf, so that JMH and others just use that instead of hacking the
>>> specifics in? This would seems like a proper way to do interop between
>>> the tools.
>>>
>>> Thanks,
>>> -Aleksey
>>>
>>
>>
>>
>



-- 

--Sergey