RFR: Add NMT support for Java heap

[Per Liden]

Hi,

On 2017-12-12 14:29, Erik Österlund wrote:
> Hi Per,
>
> I suppose this does indeed depend on how you view things. To me, when
> there are different interpretations what should be reported, I try to
> understand why we are reporting it. If we report a number, then there
> should be some kind of question to which the reported number provides an
> answer.
>
> The number you propose now is accurate in terms of what is called by
> mmap, but I do not know what useful question it answers. It will
> seemingly report ~17 TB always regardless of potentially interesting
> things such as, e.g. how close we are to hitting the roof of max memory
> that may be committed. In other words, I am not sure what useful
> question there is to which the answer is how much virtual address space
> has been reserved by the Java heap.

Given how NMT records/displays data, especially in "detail" mode where 
it's not just one number but a ranges of reserved and committed, I think 
we have two options here:

1) Register all heap views as reserved.
2) Register only one heap view as reserved.

Only registering "max heap size" as reserved wouldn't really work, 
again, since it's a range and we don't necessarily later commit memory 
within that reserved range.

None of the alternatives above are optimal, but at this time I think #1 
is slightly better, since at least it can answer two questions 
truthfully (the usefulness of those questions is debatable):

1) What part of the address spaces does the GC actually reserve for the 
Java heap. Could be useful to know if you want to e.g. set VA limits 
(think ulimit -v).
2) And, as I mentioned before, how does this or that line in 
/proc/.../maps correlate to the JVM memory usage?

So, I'll suggest we do #1 for now. We'll change it if we find a good 
reason to do so.

cheers,
Per

>
> Thanks,
> /Erik
>
> On 2017-12-12 12:20, Per Liden wrote:
>> On 2017-12-12 11:50, Aleksey Shipilev wrote:
>>> On 12/12/2017 11:42 AM, Per Liden wrote:
>>>> As Aleksey noticed, we don't register the Java heap with the native
>>>> memory tracker. Here's a patch
>>>> to do that.
>>>>
>>>> http://cr.openjdk.java.net/~pliden/zgc/nmt_java_heap/webrev.0/
>>>
>>> Patch looks good, but the NMT "reserved" data is off the charts:
>>>
>>> Total: reserved=17180280855KB, committed=17143487KB
>>> -                 Java Heap (reserved=17179869184KB,
>>> committed=16777216KB)
>>>                             (mmap: reserved=17179869184KB,
>>> committed=16777216KB)
>>>
>>> I guess this should not pass ZAddressSpaceSize, and instead tell the
>>> reserved space of the first
>>> mapping?
>>>
>>> +  // Register address space with native memory tracker
>>> +  nmt_reserve(ZAddressSpaceStart, ZAddressSpaceSize);
>>
>> I think this is correct actually. But it depends on how one views
>> things I guess. As I see it, I want to be able to look in
>> /proc/../maps and with NMT see what the different mappings correlate
>> to. If we only registered the first heap view, then there would be a
>> big mysterious reservation that would go unaccounted. That doesn't
>> sound right to me, but I'm open for hearing other opinions on this.
>> The big number there covers all addresses for all heap views/mappings
>> (i.e. the actual address space that is reserved). It should be noted
>> that, in ZGC, the heap address space doesn't have a 1:1 relation with
>> max heap size.
>>
>> cheers,
>> Per
>>
>>>
>>> Thanks,
>>> -Aleksey
>>>
>