Enabling use of hugepages with java

[Richard Achmatowicz]

Hi Stefan

Thanks very much for your reply. I wish I had seen it before wading 
through the JDK sources (virtualspace.cpp, os_linux.cpp) :-)

Through running some tests, I have found that when using the option 
-XX:+UseSHM, in addition to raising shmmax, the process requesting the 
large pages also needs to be a member of the group vm.hugetlb_shm_group:

// Running as me (who is not in group 0)
[nrla at localhost hugepages]$ sysctl -a | grep shm
kernel.shm_next_id = -1
kernel.shm_rmid_forced = 0
kernel.shmall = 18446744073692774399
kernel.shmmax = 18446744073692774399
kernel.shmmni = 4096
vm.hugetlb_shm_group = 0

[nrla at localhost hugepages]$ java -Xmx7G -XX:+UseLargePages -XX:+UseSHM 
HugePagesTest
Java HotSpot(TM) 64-Bit Server VM warning: Failed to reserve shared 
memory. (error = 1)
Java HotSpot(TM) 64-Bit Server VM warning: Failed to reserve shared 
memory. (error = 1)
Allocating (int) array of 4294967296 bytes (4.0 Gb).
Allocated (int) array successfully.
Press "ENTER" to continue...
Shutting down.

// Running as root (who is in group 0)
[root at localhost hugepages]# java -Xmx7G -XX:+UseLargePages -XX:+UseSHM 
HugePagesTest
Allocating (int) array of 4294967296 bytes (4.0 Gb).
Allocated (int) array successfully.
Press "ENTER" to continue...
Shutting down.

The large memory usage, after successful allocation and then after 
termination:
[nrla at localhost ~]$ cat /proc/meminfo | grep Huge
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
HugePages_Total:    4096
HugePages_Free:     2045
HugePages_Rsvd:     1645
HugePages_Surp:        0
Hugepagesize:       2048 kB

[nrla at localhost ~]$ cat /proc/meminfo | grep Huge
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
HugePages_Total:    4096
HugePages_Free:     4096
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

Richard

On 03/06/2018 03:30 AM, Stefan Karlsson wrote:
> Hi Richard,
>
> On 2018-02-28 23:04, David Holmes wrote:
>> Hi Richard,
>>
>> Moving to hotspot-dev as the appropriate list.
>>
>> David
>>
>> On 1/03/2018 1:20 AM, Richard Achmatowicz wrote:
>>> Hi
>>>
>>> I hope that I am directing this question to the correct mailing list.
>>>
>>> I have a question concerning the OS setup on Linux required for 
>>> correct use of the java option -XX:+UseLargePages in JDK 8.
>>>
>>> Official Oracle documentation 
>>> (http://www.oracle.com/technetwork/java/javase/tech/largememory-jsp-137182.html) 
>>> suggests that in order to make use of large memory pages, in 
>>> addition to setting the flag -XX:+UseLargePages, an OS option shmmax 
>>> needs to be tuned to be larger than the java heap size.
>>>
>>>  From looking at the java documentation, there are various ways of 
>>> enabling the use of huge pages: -XX:+UseHugeTLBFS, 
>>> -XX:+UseTransparentHugePages, -XX:+UseSHM and, if I understand 
>>> correctly, these correspond in part to making use of different 
>>> OS-level APIs for accessing huge pages (via shared memory, 
>>> hugetlbfs, and other means).
>>>
>>> My question is this: is setting the shmmax OS value only relevant if 
>>> we are using -XX:+UseSHM? In other words, if we are using 
>>> -XX:+UseHugeTLBFS to enable use of hugepages by the JVM, is it the 
>>> case that setting the shmmax OS setting has no effect on the use of 
>>> hugepages by the JVM?
>
> Yes, your understanding is correct.
>
> The document you link to seems to be from a time before 
> -XX:+UseHugeTLBFS was added.
>
> This document clarifies this a bit more:
>  https://docs.oracle.com/javase/8/docs/technotes/tools/unix/java.html#large_pages 
>
>
>  "If you are using the option -XX:+UseSHM (instead of 
> -XX:+UseHugeTLBFS), then increase the SHMMAX value."
>
> Cheers,
> StefanK
>
>>>
>>> Thanks in advance
>>>
>>> Richard
>>>
>