Stupid question

[Greg Bowyer]

To clarify slightly I see this here

through /proc/meminfo it does not look like the used pages are locked, 
just unevictable

greg at localhost ~ $ cat /proc/meminfo
MemTotal:       12286752 kB
MemFree:          209512 kB
Buffers:           21612 kB
Cached:           487636 kB
SwapCached:          352 kB
Active:          1573468 kB
Inactive:         435696 kB
Active(anon):    1325624 kB
Inactive(anon):   178984 kB
Active(file):     247844 kB
Inactive(file):   256712 kB
Unevictable:     9602276 kB
Mlocked:          246888 kB

the process itself suggests that its fully wired

PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+ COMMAND
31019 greg      20   0 9523m 9.2g  11m S    0 78.4   0:03.96 java

On 22/05/12 14:27, Greg Bowyer wrote:
> Done, I get this
>
> The "Leaky program" bit is stupid simple test program that slowly 
> leaks memory, mostly to test the jvm simply
>
> Printf's for commint memory and anon_mmap added
>
> I notice that we get this one -> anon_mmap addr=0x5bae00000 
> bytes=9749659648 fixed=0
>
> greg at localhost 
> ~/projects/openjdk/jdk7/build/linux-amd64-fastdebug/j2sdk-image $ 
> ./bin/java -Xms512m -Xmx9g -XX:+LockHeapInMemory test
> VM option '+LockHeapInMemory'
> commit_memory addr=0x7f2d66c49000 bytes=12288
> anon_mmap addr=(nil) bytes=50331648 fixed=0
> commit_memory addr=0x7f2d60bb5000 bytes=2555904
> anon_mmap addr=(nil) bytes=786432 fixed=0
> commit_memory addr=0x7f2d66b89000 bytes=40960
> anon_mmap addr=0x5bae00000 bytes=9749659648 fixed=0
> anon_mmap addr=(nil) bytes=19046400 fixed=0
> commit_memory addr=0x7f2d60bb4000 bytes=4096
> commit_memory addr=0x740000000 bytes=178913280
> commit_memory addr=0x7f2d605b4000 bytes=352256
> commit_memory addr=0x5c0000000 bytes=357957632
> anon_mmap addr=(nil) bytes=12582912 fixed=0
> commit_memory addr=0x7f2d5f9b4000 bytes=700416
> commit_memory addr=0x7f2d5ed8b000 bytes=700416
> commit_memory addr=0x5bae00000 bytes=21757952
> anon_mmap addr=(nil) bytes=167936 fixed=0
> commit_memory addr=0x7f2d5f98b000 bytes=45056
> commit_memory addr=0x7f2d5ed62000 bytes=45056
> commit_memory addr=0x7f2d5d7e5000 bytes=12288
> commit_memory addr=0x7f2d5d6e4000 bytes=12288
> commit_memory addr=0x7f2d5d3f5000 bytes=12288
> commit_memory addr=0x7f2d5d2f4000 bytes=12288
> commit_memory addr=0x7f2d5d1f3000 bytes=12288
> commit_memory addr=0x7f2d5d0f2000 bytes=12288
> Leaking program is slowly leaking 1
> Leaking program is slowly leaking 2
> Leaking program is slowly leaking 3
> ....
> ^Ccommit_memory addr=0x7f2d5cef0000 bytes=12288
>
> On 22/05/12 13:35, Vitaly Davidovich wrote:
>>
>> can you add a print statement to commit_memory() to see what 
>> arguments are being passed to it? I'm sure someone on this mailing 
>> list will have a better suggestion, but that's what I'd try just to 
>> establish that some code is indeed trying to expand the heap to its 
>> max allotted size.
>>
>> Sent from my phone
>>
>> On May 22, 2012 4:08 PM, "Greg Bowyer" <gbowyer at fastmail.co.uk 
>> <mailto:gbowyer at fastmail.co.uk>> wrote:
>>
>>     Yes thats basically it
>>
>>     java -Xms512m -Xmx9g test
>>
>>     On 22/05/12 12:27, Vitaly Davidovich wrote:
>>>
>>>     Hi Greg,
>>>
>>>     As an aside, in os::commit_memory, I think the tertiary
>>>     condition that sets the flags is inverted - you want MAP_LOCKED
>>>     if LockHeapInMemory is set, I believe.
>>>
>>>     As to your question, what args are being passed to the above
>>>     function in your run? I take it -Xms is less than Xmx in your trial?
>>>
>>>     Sent from my phone
>>>
>>>     On May 22, 2012 3:07 PM, "Greg Bowyer" <gbowyer at fastmail.co.uk
>>>     <mailto:gbowyer at fastmail.co.uk>> wrote:
>>>
>>>         Not sure if this is the right venue for this, or if I am insane
>>>
>>>         I have been playing with the openjdk code with a view to
>>>         lock the heap in memory (or at least suggest to the OS that
>>>         it wants to be locked in memory).
>>>
>>>         My use case is for java processes that are greedy in memory
>>>         and typically have a virtual size larger than physical ram.
>>>         This is not where an end user allocates a java heap beyond
>>>         ram but rather things like Lucene / Cassandra, where
>>>         typically the JVM heap is large but limited to say 1/4 of
>>>         the total physical ram and the rest of the process virtual
>>>         size is taken up with mmap()'d files.
>>>
>>>         There are java projects that currently do this with a call
>>>         out via JNA / JNI to mlockall()
>>>
>>>         Asking the OS to use the MAP_LOCKED flag in the mmap calls
>>>         in os_linux.cpp effectively does an mlock / mlockall which I
>>>         think means that when the OS chooses pages to page out; then
>>>         it should (for some measure of should) avoid paging out the
>>>         JVM heap.
>>>
>>>         This means that horrors between CMS and paging do not cause
>>>         hateful pauses (hopefully)
>>>
>>>         The thing I cant understand is that when I start a new VM
>>>         with this code (attached) it appears to lock the full size
>>>         of the heap (-Xmx) (even though the given memory is not used
>>>         by the VM), this seems to make the entire space resident.
>>>
>>>         any ideas ?
>>>
>>>         _______________________________________________
>>>         hotspot-gc-use mailing list
>>>         hotspot-gc-use at openjdk.java.net
>>>         <mailto:hotspot-gc-use at openjdk.java.net>
>>>         http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
>>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-use/attachments/20120522/aeac79cb/attachment-0001.html