Java Heap and -Xms/-Xmx

[Keith Holdaway]

Many thanks gentlemen - much appreciated.

keith
________________________________________
From: Peter.Kessler at Sun.COM [Peter.Kessler at Sun.COM]
Sent: Tuesday, July 01, 2008 2:55 PM
To: Y Srinivas Ramakrishna
Cc: Keith Holdaway; hotspot-gc-dev at openjdk.java.net; Jon.Masamitsu at Sun.COM
Subject: Re: Java Heap and -Xms/-Xmx

The only tweak I'd add to your explanation is that in addition
to the -Xmx that sets the amount of virtual address space we
reserve, and the -Xms that represented the initial (and minimum)
amount of virtual address space that we commit, there's the
permanent generation, which is reserved at start-up and committed
as needed.  But it's part of the contiguous block of the "heap"
that we need, and in some cases is a modest fraction of our total
reservation.

One can follow how much is reserved and committed for each
generation of the heap with the -XX:+PrintHeapAtGC flag, where
lines like

      PSPermGen       total 16384K, used 2107K [0xcfc00000, 0xd0c00000, 0xd3c00000)

mean we have reserved [0xcfc00000, 0xd3c00000) and committed
[0xcfc00000, 0xd0c00000).

Oh, and the caveat that we round reservations and commits to page
sizes, which can also be significant on some operating systems.

                        ... peter

Y Srinivas Ramakrishna wrote:

> Hi Keith --
>
>> If I set -Xms1g -Xmx2g and I have 4gb RAM, then will the VM always
>> look to commit the 2GB Java heap to RAM? Or will it simply commit 2GB
>> to virtual memory? Opening with -Xms of 1GB to RAM, but perhaps at
>> runtime as the heap grows, the remaining 1GB java heap could be paged
>> out. I understand the Java heap should not be paged out by the OS on
>> account of performance overtones, but at start up when the heap is not
>> stuck, by setting -Xms < -Xmx, what happens vis-à-vis the addressable
>> memory that is the contiguous java heap? Spread out ENTIRELY on RAM,
>> if available, or if available, could it still be RAM and non-RAM?
>>
>
> When you set -Xms1G -Xmx2G, the committed heap (i.e. the physical heap
> that we will actually use) starts at 1 G. That means there are pages
> of memory that contain that state of the 1 G heap. Whether those
> pages are physically on RAM or are in swap is something that the JVM
> does not explicitly control. It's something that the OS controls, and
> it depends on the "environment" (memory pressure) and the state of the host.
> The 2G of address space is however reserved in the virtual address
> space of the process. In other words, that space is not available for
> use by other things in the process, such as C-heap, thread stacks or
> mmapped libraries. However, at the start there may not be any physical
> state (in an abstract sense) associated with that address space, because
> there is no "memory" backing it, it's uninitialized unused memory that
> has just been reserved. It may not even be available to be had when
> we want to use it; see below. When the JVM believes it needs to expand the
> Java heap to use that space, it will "commit" some of that reserved VA space,
> as it expands the physical Java heap. When this happens, actual memory (i wanted
> to say "physical", but it may not be RAM pages at that stage; it may just
> be a swap reservation that is all) backs that virtual address space.
> Typically, when that space is actually touched by a thread (say when copying
> data into it for the first time) the OS will then allocate physical RAM pages
> for it. [The swap backing that space is allocated at the time the process
> "commits" that space, indicating an intention to use it -- that's how it
> works in Solaris; some Linuxes will not actually reserve the swap space
> until the physical pages are touched, by the way, so in those cases thewe may fail when actually touching the page for the first time because
> we find that the swap is not to be had because we are maxed out on swap;
> this can make Linuxes fragile wrt the JVM's current implementation.
> I don't know if we catch this and translate it into a suitable OOM
> on the vulnerable Linuxes, i suspect not. (That is my understanding
> from a while ago; things may have changed since to deal with that
> fragility.)
>
> Once again however the question of whether the committed and used heap
> is entirely in physical RAM or on RAM and swap depends on the OS
> and the conditions of the host with the JVM not directly controlling
> it.
>
> There was some talk of adding JVM options for locking the heap in
> core (to use an archaic term), but I don't believe that has actually
> been done in Java SE JDK. So my guess is that in order to lock the
> heap in core, the user would need to do that separately via appropriate
> OS interfaces.
>
> Others please correct any mistakes I might have made in my description
> above based on what might be somewhat obsolete information/understanding
> of the JVM's status wrt these issues. (In particular there has been
> talk on and off both with dealing with Linux's commit semantics
> and with locking the Java heap in core via suitable JVM options.)
>
> thanks.
> -- ramki
>
>> Hope my question is not too confusing.
>>
>> thanks
>>
>> keith
>>
>> Keith R Holdaway
>> Java Development Technologies
>>
>> SAS The Power to Know
>>
>> Carpe Diem
>>
>>
>> -----Original Message-----
>> From: Jon.Masamitsu at Sun.COM [mailto:Jon.Masamitsu at Sun.COM]
>> Sent: Tuesday, July 01, 2008 1:56 PM
>> To: Keith Holdaway
>> Cc: hotspot-gc-dev at openjdk.java.net
>> Subject: Re: Java Heap and -Xms/-Xmx
>>
>> Keith,
>>
>> Is this your question with numbers.
>>
>> If flag -Xmx1g -Xms500m are set on the command line
>> and there is 2g of physical memory, will the VM
>> increase the maximum heap size to 2g?
>>
>> No, the VM would not set the maximum heap size to
>> 2g.  In general, the user gets what is requested
>> on the command line.
>>
>> Was that the question?
>>
>> Jon
>>
>> Keith Holdaway wrote:
>>> If -Xms is set to a smaller value than -Xmx, and -Xmx is set to a
>> value less than the total RAM, will the VM commit the -Xmx setting to
>> physical addressable space ONLY?
>>> thanks
>>>
>>> Keith R Holdaway
>>> Java Development Technologies
>>>
>>> SAS The Power to Know
>>>
>>> Carpe Diem
>>>
>>>