RFR: 8254723: add diagnostic command to write Linux perf map file [v2]

[Thomas Stuefe]

On Wed, 21 Oct 2020 09:13:08 GMT, Nick Gasson <ngasson at openjdk.org> wrote:

>> When using the Linux "perf" tool to do system profiling, symbol names of
>> running Java methods cannot be decoded, resulting in unhelpful output
>> such as:
>> 
>>   10.52% [JIT] tid 236748 [.] 0x00007f6fdb75d223
>> 
>> Perf can read a simple text file format describing the mapping between
>> address ranges and symbol names for a particular process [1].
>> 
>> It's possible to generate this already for Java processes using a JVMTI
>> plugin such as perf-map-agent [2]. However this requires compiling
>> third-party code and then loading the agent into your Java process. It
>> would be more convenient if Hotspot could write this file directly using
>> a diagnostic command. The information required is almost identical to
>> that of the existing Compiler.codelist command.
>> 
>> This patch adds a Compiler.perfmap diagnostic command on Linux only. To
>> use, first run "jcmd <pid> Compiler.perfmap" and then "perf top" or
>> "perf record" and the report should show decoded Java symbol names for
>> that process.
>> 
>> As this just writes a snapshot of the code cache when the command is
>> run, it will become stale if methods are compiled later or unloaded.
>> However this shouldn't be a big problem in practice if the map file is
>> generated after the application has warmed up.
>> 
>> [1] https://github.com/torvalds/linux/blob/master/tools/perf/Documentation/jit-interface.txt
>> [2] https://github.com/jvm-profiling-tools/perf-map-agent
>
> Nick Gasson has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Update for review comments

Hi Nick,

this is a very useful idea! I missed this in the past.

Some remarks. I'll try to keep bikeshedding to a minimum since you already have enough input. Mostly ergonomics.

1) Like Alexey, I would really wish for an print-at-exit switch. The common naming seems to be xxxAtExit (so not, OnExit). "PrintXxx" seems to be printing stuff out to tty, "DumpXxxx" for writing separate files (e.g. CDS map). So I would name it DumpPerfMapAtExit.

2) Dumping to /tmp is unexpected for me, I would prefer if the default were dumping to the current directory. That seems to be the default for other files too (cds map, hs-err file etc).

3) Not necessary but nice would be a an option to specify location of the dump file.

4) I think it would be nice to have these switches always available, so real product switches. Which would require you to write up a small CSR but I still think it would make sense.

Cheers, Thomas

src/hotspot/share/code/codeCache.hpp line 194:

> 192:   static void print_summary(outputStream* st, bool detailed = true); // Prints a summary of the code cache usage
> 193:   static void log_state(outputStream* st);
> 194:   static void write_perf_map(outputStream* st);

Seems weird for this function to have an outputStream parameter only to write the dump to an unrelated file and ignore the stream for everything but the final message.

I would either pass in the file name as an option - preferably configurable - and write the last message out here; or just write the whole perf dump to the outputstream itself, piping it to jcmd and let the caller do what he wants with it (e.g. just redirecting). The latter is what most commands do. Not sure how large the perf dump gets though, may be impractical.

src/hotspot/share/code/codeCache.cpp line 1562:

> 1560: }
> 1561: 
> 1562: void CodeCache::write_perf_map(outputStream* st) {

Could this whole function possibly live inside os/linux in an own file? Or would that expose too many code heap internals?

-------------

PR: https://git.openjdk.java.net/jdk/pull/760