C-heap memory leak of IceTea

[Zhu Han]

As far as I know, cassandra does not have any native code itself,  except
JNA. I will try to remove the JNA library to see whether there is still
memory leak.

Does sun JDK and Iceatea have any difference on this problem?

Because the node is a VPS over OpenVZ, so the kernel is quite old.
$ uname -a
Linux nutstore 2.6.18-194.17.1.el5.028stab070.7 #1 SMP Fri Oct 1 14:17:14
MSD 2010 x86_64 GNU/Linux

Thank you!

best regards,
hanzhu


On Wed, Dec 22, 2010 at 5:12 PM, Andrew Haley <aph at redhat.com> wrote:

> On 12/22/2010 05:12 AM, Zhu Han wrote:
>
>> By the way, cassandra use JNA to invoke native APIs. Is there any know
>> problem of JNA over openJDK?
>>
>
> No.  But I suspect your native code is leaking memory.
>
>
>  On Mon, Dec 20, 2010 at 6:04 PM, Andrew Haley<aph at redhat.com>  wrote:
>>
>>  On 12/19/2010 02:59 AM, Zhu Han wrote:
>>>
>>>  Hi,
>>>>
>>>> I might observe C-heap leak of IcedTea6 1.9.2 on Ubuntu 10.04. This bug
>>>> was
>>>> reported against cassandra[1]. But I'm afraid it is a general openJDK
>>>> problem. Here is the symptoms. [1] may has more information on it.
>>>>
>>>> For a infrequently accessed openJDK node, the memory usage is climbed up
>>>> every day, and I'm pretty sure it is not caused by leak on Java object
>>>> heap.
>>>> This is the GC settings:
>>>>
>>>> " /usr/bin/java -ea -Xms1G -Xmx1G -XX:+UseParNewGC
>>>> -XX:+UseConcMarkSweepGC
>>>> -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8
>>>> -XX:MaxTenuringThreshold=1
>>>> -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly
>>>> -XX:+HeapDumpOnOutOfMemoryError ..."
>>>>
>>>> So the size of JVM heap is constant. Use pmap against the JVM process
>>>> for
>>>> the last 3 days, I observed a region expanded every day. It can be
>>>> expanded
>>>> to multiple gigabytes finally[1].
>>>>
>>>> on 12/19: 00000000010c3000 178548K rw---    [ anon ]
>>>> on 12/18: 00000000010c3000 110320K rw---    [ anon ]
>>>> on 12/17: 00000000010c3000  39256K rw---    [ anon ]
>>>>
>>>> I notice the region is on lower address of process address space. So I
>>>> suspect it is the c-heap of JVM. The map of object heap is as below,
>>>> which
>>>> is constant in the last few days.
>>>>
>>>> 00002b58433c0000 1069824K rw---    [ anon ]
>>>>
>>>> Both the heap and non-heap memory usage of java code is quite normal:
>>>>
>>>> 12/19/2010 10:56:13 +0800 org.archive.jmx.Client HeapMemoryUsage:
>>>> committed: 1065025536
>>>> init: 1073741824
>>>> max: 1065025536
>>>> used: 160743000
>>>>
>>>> 12/19/2010 10:56:23 +0800 org.archive.jmx.Client NonHeapMemoryUsage:
>>>> committed: 24313856
>>>> init: 24313856
>>>> max: 224395264
>>>> used: 20477424
>>>>
>>>>
>>>> There are hundreds of alive threads in the JVM.
>>>>
>>>> 12/19/2010 10:52:20 +0800 org.archive.jmx.Client PeakThreadCount: 244
>>>>
>>>>
>>>>  Originally, I suspect it is bug 6824570[2]. But After I upgraded JDK
>>>> from
>>>> IceTea 1.8.2 to IceTea 1.9.2, the problem is still not fixed. [3] said
>>>> it
>>>> is
>>>> fixed in IceTea 1.9.0.
>>>>
>>>> And there is another JVM instances running on the same node with
>>>> ParNewGC
>>>> enabled, I did not observe the memory leak.  Although the peak thread
>>>> count
>>>> is quite low of that instance, which is 43.
>>>>
>>>> Do you have any hints to diagnostic this problem? I'd like to help.
>>>>
>>>>
>>> This may, of course, be a leak in JNI code.  What Java program was
>>> running?
>>>
>>> Andrew.
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/distro-pkg-dev/attachments/20101222/a22a047f/attachment.html