RFR: 8205051: UseNUMA memory interleaving vs cpunodebind & localalloc [v3]

[Thomas Schatzl]

On Fri, 13 Dec 2024 10:27:58 GMT, Swati Sharma <duke at openjdk.org> wrote:

>> Hi All,
>> 
>> The PR handles the performance issues related to flag UseNUMA. We disable the UseNUMA flag when the process gets invoked with incorrect node alignment.
>> We check the cpunodebind and membind(or interleave for interleave policy) bitmask equality and disable UseNUMA when they are not equal.
>> For example on a 4 NUMA node system:
>> 0123 Node Number
>> 1100  cpunodebind bitmask
>> 1111 membind bitmask 
>> Disable UseNUMA as CPU and memory bitmask are not equal.
>> 
>> 0123 Node Number  
>> 1100  cpunodebind bitmask
>> 1100 membind bitmask 
>> Enable UseNUMA as CPU and memory bitmask are equal.
>> 
>> This covers all the cases with all policies and tested this with below command
>> numactl --cpunodebind=0,1 --localalloc java -Xlog:gc*=info -XX:+UseParallelGC -XX:+UseNUMA -version
>> 
>> For localalloc and preferred policies the membind bitmask returns true for all nodes, hence if cpunodebind is not bound to all nodes then the UseNUMA will be disabled.
>> 
>> This PR covers disabling the UseNUMA flag for all GC's hence we observed an improvement of ~25% on G1GC , ~20% on ZGC and ~7-8% on PGC in both throughput  and latency on SPECjbb2015 on a 2 NUMA node SRF-SP system with 6Group configuration.
>> 
>> Please review and provide your valuable comments.
>> 
>> Thanks,
>> Swati Sharma
>> Intel
>
> Swati Sharma has updated the pull request incrementally with one additional commit since the last revision:
> 
>   8205051: Added an extra method disable_numa to handle individual cases, minor comment change

I also gave a suggestion for a better name in the CR as the current one is effectively nondescriptive about the actual problem. Feel free to improve and then match witch this PR title.

src/hotspot/os/linux/os_linux.cpp line 4495:

> 4493:       // Only issue a warning if the user explicitly asked for NUMA support
> 4494:       log_warning(os)("NUMA support is disabled as libnuma failed to initialize");
> 4495:     }

Afaics this block can also use `disable_numa()`.

src/hotspot/os/linux/os_linux.hpp line 372:

> 370:   // Check if bound to only one numa node.
> 371:   // Returns true if bound to a single numa node, otherwise returns false.
> 372:   static bool is_bound_to_single_mem_node() {

Suggestion:

  // Check if memory is bound to only one numa node.
  // Returns true if memory is bound to a single numa node, otherwise returns false.
  static bool is_bound_to_single_mem_node() {

-------------

Changes requested by tschatzl (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/22395#pullrequestreview-2502080248
PR Review Comment: https://git.openjdk.org/jdk/pull/22395#discussion_r1883800167
PR Review Comment: https://git.openjdk.org/jdk/pull/22395#discussion_r1883794487