RFR: 7903722: JMH: Add xctrace-based perfnorm profiler for macOS

[Galder Zamarreño]

On Mon, 24 Jun 2024 15:49:54 GMT, Filipp Zhinkin <fzhinkin at openjdk.org> wrote:

>> Implementation of a perfnorm-alike profiler for macOS based on `xctrace` command line tool bundled with Xcode.
>> 
>> While the profiler is tested and seems to be working well, I consider it rather a preliminary version and open to a discussion on what and how it should measure.
>> 
>> Currently, the profiler only supports PMU counters sampling using `CPU Counters` instrument provided by the Instruments app / xctrace.
>> Unfortunately, `CPU Counters` instrument has no default settings, unlike `Time Profiler` and `CPU Profiler` instruments used by the recently merged `xctraceasm` profiler.
>> To use `CPU Counters`, a user has to create a template in the Instruments UI, select PMU events, save the template and then supply to `xctracenorm` as an argument.
>> 
>> This workflow not only prevents use of the profiler without preliminary manual configuration, but also tends to be annoying when it comes to measuring multiple events, as xctrace, unlike perf_events, does not support events multiplexing.
>> 
>> Thankfully, command-line-based configuration and default parameters could be emulated by building a custom Instruments package that imports data from `CPU Counters` and also supplies all required parameters.
>> As you can guess, there's no way to get information about supported PMU events directly from xctrace, but it could be fetched from KPEP database files, stored is `/usr/share/kpep`.
>> `xctracenorm` relies on that data to validate events specified by a user, if any, and also to print a help message that gives some insights into what could be sampled.
>> 
>> To sum up, there are a few things that were implemented to make `xctracenorm` profiler works:
>> - CPU model deletion using `sysctl`;
>> - KPEP file parsing to extract information about the PMU and all supported events;
>> - selected performance events validation;
>> - Instruments package building (generate XML, call a builder tool), packages are cached in `~/Library/Caches/org.openjdk.jmh`;
>> - xctrace execution, resulting samples extraction, and aggregation;
>> - samples postprocessing to calculate some additional metrics, like CPI and branch missprediction ratio.
>> 
>> Currently, if a user didn't specify any additional options, `xctracenorm` will sample instructions, cycles, branches and mispredicted branches events. 
>> These were selected as events that should be supported in all hardware macOS runs on; only 4 events were selected for the same reason.
>> 
>> Profiling results look like this on M2-based MacBook:
>> 
>> j...
>
> Gentle ping to preserve the PR from being closed.

Great work @fzhinkin! I've been trying this out and thought I'd share my experience:

* `xctraceasm` seems to be working well, but some characters do not appear to be printed well:


   0.03%  ??? ??? ?   0x00000001168cf5d8:   sbfiz		x10, x6, #2, #0x20  ;*iaload {reexecute=0 rethrow=0 return_oop=0}


The `?` characters should instead be the vertical lines that bind together loops.

* `xctracenorm` not working for me, it stops with error:


Can't create an Instruments package:

Finished running test ...


I do have `instrumentbuilder` locally but any calls to it get killed:


$ ls -l /Applications/Xcode.app/Contents/Developer/usr/bin/instrumentbuilder
-rwxr-xr-x  1 galder  staff   184K  6 Oct  2023 /Applications/Xcode.app/Contents/Developer/usr/bin/instrumentbuilder

$ /Applications/Xcode.app/Contents/Developer/usr/bin/instrumentbuilder -h
[1]    98880 killed     /Applications/Xcode.app/Contents/Developer/usr/bin/instrumentbuilder -h


I'm on Sonoma 14.5.

-------------

PR Comment: https://git.openjdk.org/jmh/pull/131#issuecomment-2249327388