Stupid question

Tue May 22 16:38:31 PDT 2012

If you add -XX:+PrintGCDetails to your command line, when the program exits it will print out where the various pieces of the Java heap are, which might help you connect the mmaps with parts of the heap.  E.g., 

   $ $Deployed/JDK-7/bin/java -d64 -Xms512m -Xmx9g -XX:+PrintGCDetails -cp $Play/Java HelloWorld
    Hello world!
    Heap
     PSYoungGen      total 154112K, used 5284K [0xfffffd7f3a600000, 0xfffffd7f45200000, 0xfffffd7ffa600000)
      eden space 132096K, 4% used [0xfffffd7f3a600000,0xfffffd7f3ab29028,0xfffffd7f42700000)
      from space 22016K, 0% used [0xfffffd7f43c80000,0xfffffd7f43c80000,0xfffffd7f45200000)
      to   space 22016K, 0% used [0xfffffd7f42700000,0xfffffd7f42700000,0xfffffd7f43c80000)
     PSOldGen        total 350208K, used 0K [0xfffffd7dba600000, 0xfffffd7dcfc00000, 0xfffffd7f3a600000)
      object space 350208K, 0% used [0xfffffd7dba600000,0xfffffd7dba600000,0xfffffd7dcfc00000)
     PSPermGen       total 22528K, used 2538K [0xfffffd7db5400000, 0xfffffd7db6a00000, 0xfffffd7dba600000)
      object space 22528K, 11% used [0xfffffd7db5400000,0xfffffd7db567a868,0xfffffd7db6a00000)

The generation triples are [start address, committed to address, end address).  The space triples are [start address, current top address, end address), but I don't think you need those here.  With the -XX:+UseParallelGC collector, the generations are perm, old, young.  Other collectors lay the generations out differently.  You can also use those addresses to find the pieces of the Java heap in a pmap.

			... peter

Greg Bowyer wrote:
>   Done, I get this
> 
> The "Leaky program" bit is stupid simple test program that slowly leaks 
> memory, mostly to test the jvm simply
> 
> Printf's for commint memory and anon_mmap added
> 
> I notice that we get this one -> anon_mmap addr=0x5bae00000 bytes=9749659648 fixed=0
> 
> greg at localhost 
> ~/projects/openjdk/jdk7/build/linux-amd64-fastdebug/j2sdk-image $ ./bin/java -Xms512m -Xmx9g -XX:+LockHeapInMemory test
> VM option '+LockHeapInMemory'
> commit_memory addr=0x7f2d66c49000 bytes=12288
> anon_mmap addr=(nil) bytes=50331648 fixed=0
> commit_memory addr=0x7f2d60bb5000 bytes=2555904
> anon_mmap addr=(nil) bytes=786432 fixed=0
> commit_memory addr=0x7f2d66b89000 bytes=40960
> anon_mmap addr=0x5bae00000 bytes=9749659648 fixed=0
> anon_mmap addr=(nil) bytes=19046400 fixed=0
> commit_memory addr=0x7f2d60bb4000 bytes=4096
> commit_memory addr=0x740000000 bytes=178913280
> commit_memory addr=0x7f2d605b4000 bytes=352256
> commit_memory addr=0x5c0000000 bytes=357957632
> anon_mmap addr=(nil) bytes=12582912 fixed=0
> commit_memory addr=0x7f2d5f9b4000 bytes=700416
> commit_memory addr=0x7f2d5ed8b000 bytes=700416
> commit_memory addr=0x5bae00000 bytes=21757952
> anon_mmap addr=(nil) bytes=167936 fixed=0
> commit_memory addr=0x7f2d5f98b000 bytes=45056
> commit_memory addr=0x7f2d5ed62000 bytes=45056
> commit_memory addr=0x7f2d5d7e5000 bytes=12288
> commit_memory addr=0x7f2d5d6e4000 bytes=12288
> commit_memory addr=0x7f2d5d3f5000 bytes=12288
> commit_memory addr=0x7f2d5d2f4000 bytes=12288
> commit_memory addr=0x7f2d5d1f3000 bytes=12288
> commit_memory addr=0x7f2d5d0f2000 bytes=12288
> Leaking program is slowly leaking 1
> Leaking program is slowly leaking 2
> Leaking program is slowly leaking 3
> ....
> ^Ccommit_memory addr=0x7f2d5cef0000 bytes=12288
> 
> On 22/05/12 13:35, Vitaly Davidovich wrote:
>>
>> can you add a print statement to commit_memory() to see what arguments 
>> are being passed to it? I'm sure someone on this mailing list will 
>> have a better suggestion, but that's what I'd try just to establish 
>> that some code is indeed trying to expand the heap to its max allotted 
>> size.
>>
>> Sent from my phone
>>
>> On May 22, 2012 4:08 PM, "Greg Bowyer" <gbowyer at fastmail.co.uk 
>> <mailto:gbowyer at fastmail.co.uk>> wrote:
>>
>>     Yes thats basically it
>>
>>     java -Xms512m -Xmx9g test
>>
>>     On 22/05/12 12:27, Vitaly Davidovich wrote:
>>>
>>>     Hi Greg,
>>>
>>>     As an aside, in os::commit_memory, I think the tertiary condition
>>>     that sets the flags is inverted - you want MAP_LOCKED if
>>>     LockHeapInMemory is set, I believe.
>>>
>>>     As to your question, what args are being passed to the above
>>>     function in your run? I take it -Xms is less than Xmx in your trial?
>>>
>>>     Sent from my phone
>>>
>>>     On May 22, 2012 3:07 PM, "Greg Bowyer" <gbowyer at fastmail.co.uk
>>>     <mailto:gbowyer at fastmail.co.uk>> wrote:
>>>
>>>         Not sure if this is the right venue for this, or if I am insane
>>>
>>>         I have been playing with the openjdk code with a view to lock
>>>         the heap in memory (or at least suggest to the OS that it
>>>         wants to be locked in memory).
>>>
>>>         My use case is for java processes that are greedy in memory
>>>         and typically have a virtual size larger than physical ram.
>>>         This is not where an end user allocates a java heap beyond
>>>         ram but rather things like Lucene / Cassandra, where
>>>         typically the JVM heap is large but limited to say 1/4 of the
>>>         total physical ram and the rest of the process virtual size
>>>         is taken up with mmap()'d files.
>>>
>>>         There are java projects that currently do this with a call
>>>         out via JNA / JNI to mlockall()
>>>
>>>         Asking the OS to use the MAP_LOCKED flag in the mmap calls in
>>>         os_linux.cpp effectively does an mlock / mlockall which I
>>>         think means that when the OS chooses pages to page out; then
>>>         it should (for some measure of should) avoid paging out the
>>>         JVM heap.
>>>
>>>         This means that horrors between CMS and paging do not cause
>>>         hateful pauses (hopefully)
>>>
>>>         The thing I cant understand is that when I start a new VM
>>>         with this code (attached) it appears to lock the full size of
>>>         the heap (-Xmx) (even though the given memory is not used by
>>>         the VM), this seems to make the entire space resident.
>>>
>>>         any ideas ?
>>>
>>>         _______________________________________________
>>>         hotspot-gc-use mailing list
>>>         hotspot-gc-use at openjdk.java.net
>>>         <mailto:hotspot-gc-use at openjdk.java.net>
>>>         http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
>>>
>>
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> hotspot-gc-use mailing list
> hotspot-gc-use at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use