jdk 8u40 - MaxMetaspaceSize and CompressedClassSpaceSize
Stefan Karlsson
stefan.karlsson at oracle.com
Wed Apr 8 14:29:08 UTC 2015
Hi Vitaly,
On 2015-04-08 15:06, Vitaly Davidovich wrote:
>
> Hi Stefan,
>
> Thanks for clarifying some of this. The one thing I'm still unclear
> about is why more VA is reserved when a max committed amount is set to
> a lower value. I'm assuming that if I set MaxMetaspaceSize to, say,
> 32MB then trying to use/commit more than that will result in an OOME.
>
That's correct.
> Yet the VA reservation stays at 1GB.
>
Yes, that's how the compressed class space was chosen to be implemented
to make sure that the Klasses end up at an address that we can compress
down to 32-bits. This could be changed to reserve less memory when
MaxMetaspaceSize is set to a lower value.
> Or are you saying that 32MB only restricts the non-class metadata and
> thus you need to leave additional room for class metadata? Does
> MaxMetaspaceSize actually cap both class and non-class metadata? The
> doc I linked to earlier suggests it caps both, but if I'm reading your
> email right, you seem to be saying that even if MaxMetaspaceSize is
> set you still need additional room for class metadata.
>
MaxMetspaceSize caps the combined _committed_ memory, so the process
will not commit more space than what you've specified on the command line.
What I'm trying to say is that having the current implementation of
compressed class space forces us to pre-reserve at least 32MB even if we
are not going to use/commit all that memory. Since we don't know the
distribution between Klasses and non-class metadata, we also have to
allow the non-class metadata spaces to reserve up to 32MB as well. So,
in the extreme case we will reserve 64MB but we'll not commit more than
32MB.
However, as you've noted we pre-reserve 1GB, and not 32 MB, for the
compressed class space. That can be fixed by either running with
-XX:CompressedClassSpaceSize=32m, or by a patch to change the
implementation to only reserve MIN(CompressedClassSpaceSize,
MaxMetaspaceSize) for the compressed class space.
Thanks,
StefanK
> Thanks
>
> sent from my phone
>
> On Apr 8, 2015 5:23 AM, "Stefan Karlsson" <stefan.karlsson at oracle.com
> <mailto:stefan.karlsson at oracle.com>> wrote:
>
> Hi Vitaly,
>
> On 2015-04-08 04:26, Vitaly Davidovich wrote:
>
> Thanks Ramki, that does clear it up a bit. So you're saying
> the JVM notion
> of reserved is basically (on linux) mmap()'ing anon memory
> with PROT_NONE
> equivalent - i.e. no commit charge. Is that right?
>
>
> That's sounds right to me.
>
> The code path to the mmap:
> os::reserve_memory
> os::pd_reserve_memory
> anon_mmap
> mmap( ... PROT_NONE ... , ... MAP_NORESERVE ... )
>
> I did an experiment with some time ago that can be found at:
> http://mail.openjdk.java.net/pipermail/hotspot-dev/2013-July/010226.html
>
> Independent of the over-commit settings in the Linux kernel, I
> could still "reserve" 2TB of memory on my 8GB machine, if
> PROT_NONE was used.
>
>
> Ok, so the issue with ulimit is probably that it actually
> restricts address
> space, and not virtual memory.
>
> As for the reservation in Hotspot, why is it reserving that
> address space
> if it has no intention of using it?
>
>
> I agree that we shouldn't have to reserve 1 GB of virtual address
> space for the compressed class space when MaxMetaspaceSize is set
> to a lower value. It should be enough to allocate
> MaxMetaspaceSize of compressed class space. Unfortunately, we
> don't know how much Metadata will be put inside the compressed
> class space vs the other metaspace memory areas (containing the
> non-class metadata). So, we would probably end up allowing ~ 2 x
> MaxMetaspaceSize of memory to be reserved if the program committed
> MaxMetaspaceSize of non-class metadata.
>
> We could reserve a contiguous memory area if user has set a low
> MaxMetaspaceSize, say below 1 GB, and use that area for both
> classes and non-class metadata. With that in place the
> MaxMetaspaceSize flag would limit both the reserved and committed
> memory. Unfortunately, this has a draw-back that mmap will destroy
> the reservation of the memory area if a sub-sequent "commit"
> request would fail. We would have to handle that somehow.
>
> Or if it does use it, I'm assuming
> it'll change protection on it and then possibly oom at that
> point (if
> commit charge is over the limit).
>
> But anyway, back to the question at hand. Is there a single
> flag that
> would cap the amount of VA requested? It seems odd that
> MaxMetaspaceSize by
> itself doesn't change VA reservation, but adding
> CompressedClassSpaceSize
> does. Moreover, if I'm capping max size, why even reserve
> more VA in that
> case?
>
>
> MaxMetaspaceSize is designed to limit the amount of committed
> memory and not to limit the amount of reserved memory. At this
> moment, there are no flag to limit the amount of reserved memory
> by the metaspaces. If that feature is needed, it would have to be
> implemented.
>
> Thanks,
> StefanK
>
>
> Thanks
>
> sent from my phone
> On Apr 7, 2015 9:38 PM, "Srinivas Ramakrishna"
> <ysr1729 at gmail.com <mailto:ysr1729 at gmail.com>> wrote:
>
> Hi Vitaly --
>
> Let me give an example: In a multi-threaded process, if we
> want some piece
> of anon memory to be placed at a specific spot in our
> (i.e. process's) VA
> address space, we can first "reserve" that address space
> (tit is in that
> sense purely local to the process, there are no physical
> pages backing it,
> no swap reservation, no OS accounting, nothing). We then
> can decide at
> leisure what to do with that address space, knowing that
> that address space
> in our process is reserved for us and "no one can take
> that away from us."
> This is what the JVM calls an address space reservation.
>
> What the JVM calls "commit" is when we reserve the virtual
> memory with the
> OS. Committed memory may or may not be backed by physical
> pages. (For
> example, on Linux wjere you allow overcommit.) It's only
> when you access
> that page will it be faulted in from the set of free pages
> in RAM (if there
> are any left and you aren't overcommitted).
>
> What is confusing you (and many others) is the use of the
> term "reserved".
> Here reserved is used in the sense of reserving the
> address space in the
> process. Nothing else. No OS book-keeping to reserve
> virtual *memory*,
> nothing.
>
> At least that was what it was a couple of years ago, but I
> haven't looked
> at the code in recent times (but given this discussion,
> I'll go and do so
> soon!).
> -- ramki
>
>
> On Tue, Apr 7, 2015 at 5:47 PM, Vitaly Davidovich
> <vitalyd at gmail.com <mailto:vitalyd at gmail.com>>
> wrote:
>
> Ramki,
>
> I'm not sure I fully understand your distinction
> between virtual address
> space vs virtual memory. If you disable swap and
> overcommit and you
> mmap/brk from the kernel, it has to ensure that the
> reservation could be
> backed up if it were committed; if you take away swap
> and don't allow linux
> to overpromise (i.e. overcommit), the malloc will
> fail. See here for what
> I'm talking about: http://www.etalabs.net/overcommit.html
>
> The committed memory is memory that is now backed by
> physical ram (or
> swap), it has either been touched or manually
> committed - it's not a
> reservation (which has no physical backing, just
> kernel VM accounting
> behind it).
>
> Did I misunderstand you?
>
> sent from my phone
> Hi Vitaly --
>
> The allocated (but not necessarily touched/faulted)
> memory reported above
> (by the jvm) is the one corresponding to "committed".
> The "reservation"
> reported above is only of the virtual *address space*
> within the process,
> not virtual *memory* (i.e. no swap reservations or
> the like). I didn't
> know that ulimits could prevent virtual address space
> (_not_ virtual
> memory) reservation.
>
> I think your question is valid, in that looking at the
> VA reservation,
> one feels that the JVM might eventually decide to
> commit beyond the max
> space you asked for in your command-line spec. At the
> very least it is
> confusing, and someone who knows why could explain the
> reason for the
> larger than user-requested VA space (not VA memory yet
> in yr snippet)
> reservation when it won't be used -- if it won't be
> used, and here I am
> giving the benefit of the doubt to the JVM in that
> it'll never try to
> commit those pages despite having reserved that in its
> virtual address
> space.
>
> -- ramki
>
> On Tue, Apr 7, 2015 at 5:01 PM, Vitaly Davidovich
> <vitalyd at gmail.com <mailto:vitalyd at gmail.com>>
> wrote:
>
> By the way, forgot to mention that while doing
> some searching around
> this, saw this:
> https://issues.apache.org/jira/browse/HADOOP-11364. So
> anyone with multi-tenant servers that impose mem
> limits may hit this. But
> again, I'm trying to understand the flags involved
> and not saying this is a
> big problem.
>
> sent from my phone
> On Apr 7, 2015 7:55 PM, "Vitaly Davidovich"
> <vitalyd at gmail.com <mailto:vitalyd at gmail.com>> wrote:
>
> Hi Ramki,
>
> I agree in principle that a 1GB reservation on
> 64bit isn't an issue in
> and of itself. The problem comes up, however,
> if a program is run with
> ulimit on VA and someone sized their java app
> appropriately in, say, java
> 7. With java 8, there can be regressions.
>
> Second, if you disable swap and overcommit on
> linux, larger virtual mem
> allocations may fail.
>
> Having said all that, I'm curious whether
> there's a way to restrict
> this with 1 flag in java 8 and whether the
> linked documentation is correct
> or not.
>
> Thanks
>
> sent from my phone
> On Apr 7, 2015 7:26 PM, "Srinivas Ramakrishna"
> <ysr1729 at gmail.com <mailto:ysr1729 at gmail.com>>
> wrote:
>
> Hi Vitaly --
>
> Because reserved but unused virtual
> address space is, for all
> practical purposes today, irrelevant for
> 64 bit processes,
> a question to ask is what you see when you
> do this for 32-bit
> processes (where of course the oop compression
> comments wouldn't of course apply, but the
> amount of VA space used
> would be much more relevant and parsimony
> -- or lack thereof -- in VA space
> reservation would visibly affect the
> bottom-line in a 32-bit VA space).
>
> -- ramki
>
> On Tue, Apr 7, 2015 at 12:27 PM, Vitaly
> Davidovich <vitalyd at gmail.com
> <mailto:vitalyd at gmail.com>>
> wrote:
>
> Hi guys,
>
> It seems that putting a cap on
> MaxMetaspaceSize has no impact on
> amount of
> memory reserved *unless*
> CompressedClassSpaceSize is also set. For
> example,
>
> java -Xmx64m -XX:MaxMetaspaceSize=16m Test
>
> yields the following:
>
> Metaspace used 2421K, capacity
> 4486K, committed 4864K, reserved
> 1056768K
> class space used 261K, capacity
> 386K, committed 512K, reserved
> 1048576K
>
> Adding
> -XX:CompressedClassSpaceSize=16m yields:
>
> Metaspace used 2421K, capacity
> 4486K, committed 4864K, reserved
> 24576K
> class space used 261K, capacity
> 386K, committed 512K, reserved
> 16384K
>
> According to
>
> http://docs.oracle.com/javase/8/docs/technotes/guides/vm/gctuning/considerations.html
> :
>
> If UseCompressedOops is turned on
> and UseCompressedClassesPointers
>
> is
>
> used, then two logically different
> areas of native memory are used
>
> for
>
> class metadata.
> UseCompressedClassPointers uses a
> 32-bit offset to
> represent the class pointer in a
> 64-bit process as does
>
> UseCompressedOops for
>
> Java object references. A region
> is allocated for these compressed
>
> class
>
> pointers (the 32-bit offsets). The
> size of the region can be set
>
> with
>
> CompressedClassSpaceSize and is 1
> gigabyte (GB) by default. The
>
> space for
>
> the compressed class pointers is
> reserved as space allocated by
>
> mmap at
>
> initialization and committed as
> needed. The MaxMetaspaceSize
>
> applies to
>
> the sum of the committed
> compressed class space and the
> space for
>
> the other
>
> class metadata.
>
>
> Unless I'm misunderstanding, the last
> sentence there seems to imply
> that
> MaxMetaspaceSize would put a cap on
> the compressed class space as
> well. Or
> is the "committed" part of the last
> sentence the key? There's no way
> to cap
> the *reserved* amount with 1 flag?
> Basically, what's the java 8
> equivalent
> to MaxPermSize=X in java 7 and earlier?
>
> The reason this came up is because
> java 8 seems to have a much higher
> virtual mem reserve charge than java
> 7, even for an app with a simple
> main
> that does nothing, and
> metaspace/compressed class space
> reservation
> appears
> to be the biggest contributor.
>
> Thanks
>
>
>
More information about the hotspot-runtime-dev
mailing list