[11] RFR(M): 8189922: UseNUMA memory interleaving vs membind

[Gustavo Romero]

Hi Swati,

On 06/16/2018 02:52 PM, Swati Sharma wrote:
> Hi All,
> 
> This is my first patch,I would appreciate if anyone can review the fix:
> 
> Bug : https://bugs.openjdk.java.net/browse/JDK-8189922 <https://bugs.openjdk.java.net/browse/JDK-8189922>
> Webrev :http://cr.openjdk.java.net/~gromero/8189922/v1
> 
> The bug is about JVM flag UseNUMA which bypasses the user specified numactl --membind option and divides the whole heap in lgrps according to available numa nodes.
> 
> The proposed solution is to disable UseNUMA if bound to single numa node. In case more than one numa node binding, create the lgrps according to bound nodes.If there is no binding, then JVM will divide the whole heap based on the number of NUMA nodes available on the system.
> 
> I appreciate Gustavo's help for fixing the thread allocation based on numa distance for membind which was a dangling issue associated with main patch.

Thanks. I have no further comments on it. LGTM.


Best regards,
Gustavo

PS: Please, provide numactl -H information when possible. It helps to grasp
promptly the actual NUMA topology in question :)

> Tested the fix by running specjbb2015 composite workload on 8 NUMA node system.
> Case 1 : Single NUMA node bind
> numactl --cpunodebind=0 --membind=0 java -Xmx24g -Xms24g -Xmn22g -XX:+UseNUMA -Xlog:gc*=debug:file=gc.log:time,uptimemillis <composite_application>
> Before Patch: gc.log
> eden space 22511616K(22GB), 12% used
>      lgrp 0 space 2813952K, 100% used
>      lgrp 1 space 2813952K, 0% used
>      lgrp 2 space 2813952K, 0% used
>      lgrp 3 space 2813952K, 0% used
>      lgrp 4 space 2813952K, 0% used
>      lgrp 5 space 2813952K, 0% used
>      lgrp 6 space 2813952K, 0% used
>      lgrp 7 space 2813952K, 0% used
> After Patch : gc.log
> eden space 46718976K(45GB), 99% used(NUMA disabled)
> 
> Case 2 : Multiple NUMA node bind
> numactl --cpunodebind=0,7 –membind=0,7 java -Xms50g -Xmx50g -Xmn45g  -XX:+UseNUMA -Xlog:gc*=debug:file=gc.log:time,uptimemillis <composite_application>
> Before Patch :gc.log
> eden space 46718976K, 6% used
>      lgrp 0 space 5838848K, 14% used
>      lgrp 1 space 5838848K, 0% used
>      lgrp 2 space 5838848K, 0% used
>      lgrp 3 space 5838848K, 0% used
>      lgrp 4 space 5838848K, 0% used
>      lgrp 5 space 5838848K, 0% used
>      lgrp 6 space 5838848K, 0% used
>      lgrp 7 space 5847040K, 35% used
> After Patch : gc.log
> eden space 46718976K(45GB), 99% used
>       lgrp 0 space 23359488K(23.5GB), 100% used
>       lgrp 7 space 23359488K(23.5GB), 99% used
> 
> 
> Note: The proposed solution is only for numactl membind option.The fix is not for --cpunodebind and localalloc which is a separate bug bug https://bugs.openjdk.java.net/browse/JDK-8205051 and fix is in progress on this.
> 
> Thanks,
> Swati Sharma
> Software Engineer -2 at AMD
>