Trying to understand ZGC

[charlie hunt]

Hi Stefan,

Response to your large / huge pages question below.

hths,

Charlie

On 11/28/18 1:09 PM, Stefan Reich wrote:
> Hi Per!
>
> On Tue, 13 Nov 2018 at 20:22, Per Liden <per.liden at oracle.com> wrote:
>
>> The RSS accounting on Linux isn't always telling the complete truth and
>> it can even vary depending on if you're using small or large pages. ZGC
>> does heap multi-mapping, which means it will map the same heap memory in
>> three different locations in the virtual address space. When using small
>> pages, Linux isn't clever enough to detect that it's the same memory
>> being mapped multiple times, and so it accounts for each mapping as if
>> it was new/different, inflating the RSS by 3x. This typically doesn't
>> happen when using large pages (-XX:+UseLargePages).
>>
> Thanks. I would call this an actual bug in Linux then. Counting memory
> twice is really not OK.
>
> Hm... are large pages really problematic as suggested here?
> https://www.oracle.com/technetwork/java/javase/tech/largememory-jsp-137182.html

You are probably referring to this paragraph from that article, right?

> However please note sometimes using large page memory can negatively 
> affect system performance. For example, when a large mount of memory 
> is pinned by an application, it may create a shortage of regular 
> memory and cause excessive paging in other applications and slow down 
> the entire system. Also please note for a system that has been up for 
> a long time, excessive fragmentation can make it impossible to reserve 
> enough large page memory. When it happens, either the OS or JVM will 
> revert to using regular pages.
This paragraph applies to a system that has multiple applications 
running on it, and/or applies to a situation where there is not a lot of 
available memory above what you have configured as large pages.

With some hand waving, and generally speaking, if you have a lot of 
memory available on your system, or you do not have a situation where 
there are multiple applications running that could push you close to 
exhausting available physical memory or a need for large segments of 
contiguous memory, then configuring large pages as described should work 
fine.

Another tip that helps with configuring large pages is to reboot your 
system prior to configuring it for large pages. It is usually not 
required. But, it does make it easier to find contiguous pages to lock 
into memory as large pages. You might also consider adding 
-XX:+AlwaysPreTouch in addition to -XX:+UseLargePages as JVM command 
line options.

You can use transparent huge pages also. If you want to go down that 
path I can send you instructions. Just let me know.