RFR: 8317453: NMT: Performance benchmarks are needed to measure speed and memory

[Gerard Ziemski]

On Tue, 1 Apr 2025 09:07:02 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> Please review this addition of an internal benchmark, mostly of interest to those working with NMT.
>> 
>> This benchmark allows us to record a pattern of memory allocation operations (i.e. `malloc`, `realloc` and `free`) as well as the virtual memory allocations (i.e. `VirtualMemoryTracker::add_reserved_region`, etc.) and record those into files.
>> 
>> Later we can use that recording to _play back_ the pattern with different code or settings to compare the performance.
>> 
>> The goal of this benchmark is for anyone working on NMT to be able to measure and prove whether their improvement helps or regresses the performance.
>> 
>> ### To use it:
>> 
>> To record pattern of allocations of memory calls:
>> 
>> `NMTRecordMemoryAllocations=0x7FFFFFFF ./build/macosx-aarch64-server-release/xcode/build/jdk/bin/java -XX:+UnlockDiagnosticVMOptions -XX:NativeMemoryTracking=summary -jar build/macosx-aarch64-server-release/images/jdk/demo/jfc/J2Ddemo/J2Ddemo.jar`
>> 
>> OR to record pattern of allocations of virtual memory calls:
>> 
>> `NMTRecordVirtualMemoryAllocations=0x7FFFFFFF ./build/macosx-aarch64-server-release/xcode/build/jdk/bin/java -XX:+UnlockDiagnosticVMOptions -XX:NativeMemoryTracking=summary -jar build/macosx-aarch64-server-release/images/jdk/demo/jfc/J2Ddemo/J2Ddemo.jar`
>> 
>> This will result in the file:
>> -  hs_nmt_pid22770_allocs_record.log (is the chronological record of the the desired operations)
>> OR
>> - hs_nmt_pid22770_virtual_allocs_record.log (is the chronological record of the desired operations)
>> 
>> And 2 additional files:
>> - hs_nmt_pid22770_info_record.log (is the record of default NMT memory overhead and the NMT state)
>> - hs_nmt_pid22770_threads_record.log (is the record of thread names that can be retrieved later when processing)
>> 
>> 
>> then to actually run the benchmark:
>> 
>> NMTBenchmarkRecordedPID=22770 ./build/macosx-aarch64-server-release/xcode/build/jdk/bin/java -XX:+UnlockDiagnosticVMOptions -XX:NativeMemoryTracking=summary
>> 
>> ### Usage:
>> 
>> See the issue for more details and the design document.
>
> src/hotspot/share/nmt/memTracker.cpp line 57:
> 
>> 55: 
>> 56: void MemTracker::initialize() {
>> 57:   NMT_LogRecorder::initialize();
> 
> Why we initialize the log-recorder even if NMT is not enabled?

We allow the NMTBenchmark to run without NMT, so that we can compare memory usage with NMT on or off.

NMTBenchmark doesn't require NMT to be ON.

Now that you brought it up, the name might be confusing , but the main usage was to compare apps with NMT on and off.

> src/hotspot/share/nmt/memTracker.cpp line 100:
> 
>> 98:   }
>> 99: 
>> 100:   NMT_LogRecorder::replay();
> 
> If the pid is saved in the corresponding env-var and there is a recording file, `replay()` will run and play at every time JVM starts (all the JTREG tests for example). Perhaps we don't want this.

The user can always unset `NMTBenchmarkRecordedPID` and the code will not activate:


void NMT_LogRecorder::replay() {
  char* NMTBenchmarkRecordedPID = getenv("NMTBenchmarkRecordedPID");
  if (NMTBenchmarkRecordedPID != nullptr) {
    int pid = atoi(NMTBenchmarkRecordedPID);
    NMT_MemoryLogRecorder::instance()->replay(pid);
    NMT_VirtualMemoryLogRecorder::instance()->replay(pid);
    os::exit(0);
  }
}



I think this is fine, but let me know if it really bothers you so we can give it more thought.

> src/hotspot/share/nmt/memTracker.hpp line 1:
> 
>> 1: /*
> 
> In memTracker, there are also functions for `Arena` allocations. Why they are not include in the LogRecording?

We absolutely could include them (in a follow-up). For 1st release I just picked malloc and virtual alloc.

We should absolutely keep eyes open for the kind of things we want to benchmark - great suggestion!

Filed [NMT: add Arenas to NMTBenchmark](https://bugs.openjdk.org/browse/JDK-8353855)

> src/hotspot/share/nmt/memTracker.hpp line 91:
> 
>> 89:       ptr = MallocTracker::record_malloc(mem_base, size, mem_tag, stack, old_base);
>> 90:     }
>> 91:     NMT_MemoryLogRecorder::record_malloc(mem_tag, size, mem_base, &stack, old_base);
> 
> Can we move the `NMT_MemoryLogRecorder::record_malloc` inside the `MallocTracker::record_malloc`?
> Then we are also sure that thee recording only happens when NMT is enabled.
> The same applies to other malloc methods and `VMT::record_xxxxx` methods.

Currently` NMT_MemoryLogRecorder::record_malloc()` is inside `record_malloc()`

Where would you like to move it? I am all for simplifying the code, I'm just unsure where you would like to see them moved...

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23786#discussion_r2052907488
PR Review Comment: https://git.openjdk.org/jdk/pull/23786#discussion_r2052910951
PR Review Comment: https://git.openjdk.org/jdk/pull/23786#discussion_r2031210075
PR Review Comment: https://git.openjdk.org/jdk/pull/23786#discussion_r2052915049