jdk 8u40 - MaxMetaspaceSize and CompressedClassSpaceSize

Wed Apr 8 01:38:13 UTC 2015

Hi Vitaly --

Let me give an example: In a multi-threaded process, if we want some piece
of anon memory to be placed at a specific spot in our (i.e. process's) VA
address space, we can first "reserve" that address space (tit is in that
sense purely local to the process, there are no physical pages backing it,
no swap reservation, no OS accounting, nothing). We then can decide at
leisure what to do with that address space, knowing that that address space
in our process is reserved for us and "no one can take that away from us."
This is what the JVM calls an address space reservation.

What the JVM calls "commit" is when we reserve the virtual memory with the
OS. Committed memory may or may not be backed by physical pages. (For
example, on Linux wjere you allow overcommit.) It's only when you access
that page will it be faulted in from the set of free pages in RAM (if there
are any left and you aren't overcommitted).

What is confusing you (and many others) is the use of the term "reserved".
Here reserved is used in the sense of reserving the address space in the
process. Nothing else. No OS book-keeping to reserve virtual *memory*,
nothing.

At least that was what it was a couple of years ago, but I haven't looked
at the code in recent times (but given this discussion, I'll go and do so
soon!).
-- ramki

On Tue, Apr 7, 2015 at 5:47 PM, Vitaly Davidovich <vitalyd at gmail.com> wrote:

> Ramki,
>
> I'm not sure I fully understand your distinction between virtual address
> space vs virtual memory.  If you disable swap and overcommit and you
> mmap/brk from the kernel, it has to ensure that the reservation could be
> backed up if it were committed; if you take away swap and don't allow linux
> to overpromise (i.e. overcommit), the malloc will fail.  See here for what
> I'm talking about: http://www.etalabs.net/overcommit.html
>
> The committed memory is memory that is now backed by physical ram (or
> swap), it has either been touched or manually committed - it's not a
> reservation (which has no physical backing, just kernel VM accounting
> behind it).
>
> Did I misunderstand you?
>
> sent from my phone
> Hi Vitaly --
>
> The allocated (but not necessarily touched/faulted) memory reported above
> (by the jvm) is the one corresponding to "committed". The "reservation"
> reported above is only of the virtual *address space* within the process,
> not virtual *memory*  (i.e. no swap reservations or the like). I didn't
> know that ulimits could prevent virtual address space (_not_ virtual
> memory) reservation.
>
> I think your question is valid, in that looking at the VA reservation, one
> feels that the JVM might eventually decide to commit beyond the max space
> you asked for in your command-line spec. At the very least it is confusing,
> and someone who knows why could explain the reason for the larger than
> user-requested VA space (not VA memory yet in yr snippet) reservation when
> it won't be used  -- if it won't be used, and here I am giving the benefit
> of the doubt to the JVM in that it'll never try to commit those pages
> despite having reserved that in its virtual address space.
>
> -- ramki
>
> On Tue, Apr 7, 2015 at 5:01 PM, Vitaly Davidovich <vitalyd at gmail.com>
> wrote:
>
>> By the way, forgot to mention that while doing some searching around
>> this, saw this: https://issues.apache.org/jira/browse/HADOOP-11364.  So
>> anyone with multi-tenant servers that impose mem limits may hit this.  But
>> again, I'm trying to understand the flags involved and not saying this is a
>> big problem.
>>
>> sent from my phone
>> On Apr 7, 2015 7:55 PM, "Vitaly Davidovich" <vitalyd at gmail.com> wrote:
>>
>>> Hi Ramki,
>>>
>>> I agree in principle that a 1GB reservation on 64bit isn't an issue in
>>> and of itself.  The problem comes up, however, if a program is run with
>>> ulimit on VA and someone sized their java app appropriately in, say, java
>>> 7.  With java 8, there can be regressions.
>>>
>>> Second, if you disable swap and overcommit on linux, larger virtual mem
>>> allocations may fail.
>>>
>>> Having said all that, I'm curious whether there's a way to restrict this
>>> with 1 flag in java 8 and whether the linked documentation is correct or
>>> not.
>>>
>>> Thanks
>>>
>>> sent from my phone
>>> On Apr 7, 2015 7:26 PM, "Srinivas Ramakrishna" <ysr1729 at gmail.com>
>>> wrote:
>>>
>>>> Hi Vitaly --
>>>>
>>>> Because reserved but unused virtual address space is, for all practical
>>>> purposes today, irrelevant for 64 bit processes,
>>>> a question to ask is what you see when you do this for 32-bit processes
>>>> (where of course the oop compression
>>>> comments wouldn't of course apply, but the amount of VA space used
>>>> would be much more relevant and parsimony
>>>> -- or lack thereof -- in VA space reservation would visibly affect the
>>>> bottom-line in a 32-bit VA space).
>>>>
>>>> -- ramki
>>>>
>>>> On Tue, Apr 7, 2015 at 12:27 PM, Vitaly Davidovich <vitalyd at gmail.com>
>>>> wrote:
>>>>
>>>>> Hi guys,
>>>>>
>>>>> It seems that putting a cap on MaxMetaspaceSize has no impact on
>>>>> amount of
>>>>> memory reserved *unless* CompressedClassSpaceSize is also set.  For
>>>>> example,
>>>>>
>>>>> java -Xmx64m -XX:MaxMetaspaceSize=16m Test
>>>>>
>>>>> yields the following:
>>>>>
>>>>>  Metaspace       used 2421K, capacity 4486K, committed 4864K, reserved
>>>>> 1056768K
>>>>>   class space    used 261K, capacity 386K, committed 512K, reserved
>>>>> 1048576K
>>>>>
>>>>> Adding -XX:CompressedClassSpaceSize=16m yields:
>>>>>
>>>>> Metaspace       used 2421K, capacity 4486K, committed 4864K, reserved
>>>>> 24576K
>>>>>   class space    used 261K, capacity 386K, committed 512K, reserved
>>>>> 16384K
>>>>>
>>>>> According to
>>>>>
>>>>> http://docs.oracle.com/javase/8/docs/technotes/guides/vm/gctuning/considerations.html
>>>>> :
>>>>>
>>>>> >
>>>>> > If UseCompressedOops is turned on and UseCompressedClassesPointers is
>>>>> > used, then two logically different areas of native memory are used
>>>>> for
>>>>> > class metadata. UseCompressedClassPointers uses a 32-bit offset to
>>>>> > represent the class pointer in a 64-bit process as does
>>>>> UseCompressedOops for
>>>>> > Java object references. A region is allocated for these compressed
>>>>> class
>>>>> > pointers (the 32-bit offsets). The size of the region can be set with
>>>>> > CompressedClassSpaceSize and is 1 gigabyte (GB) by default. The
>>>>> space for
>>>>> > the compressed class pointers is reserved as space allocated by mmap
>>>>> at
>>>>> > initialization and committed as needed. The MaxMetaspaceSize applies
>>>>> to
>>>>> > the sum of the committed compressed class space and the space for
>>>>> the other
>>>>> > class metadata.
>>>>>
>>>>>
>>>>> Unless I'm misunderstanding, the last sentence there seems to imply
>>>>> that
>>>>> MaxMetaspaceSize would put a cap on the compressed class space as
>>>>> well.  Or
>>>>> is the "committed" part of the last sentence the key? There's no way
>>>>> to cap
>>>>> the *reserved* amount with 1 flag? Basically, what's the java 8
>>>>> equivalent
>>>>> to MaxPermSize=X in java 7 and earlier?
>>>>>
>>>>> The reason this came up is because java 8 seems to have a much higher
>>>>> virtual mem reserve charge than java 7, even for an app with a simple
>>>>> main
>>>>> that does nothing, and metaspace/compressed class space reservation
>>>>> appears
>>>>> to be the biggest contributor.
>>>>>
>>>>> Thanks
>>>>>
>>>>
>>>>
>