C-heap memory leak of IceTea

[Zhu Han]

BTW: the load on the JVM is very light.


best regards,
hanzhu


On Mon, Dec 20, 2010 at 7:47 PM, Zhu Han <schumi.han at gmail.com> wrote:

> The program running is apache cassandra 0.6.8[1].  Based on the report of
> cassandra community, most of them does not meet it even in their production
> environment. Some of the users are big web company.
>
> I have no idea whether sun jdk has same problem. Maybe I should have a try.
>
> Do you have any hints on how to get more diagnostic information?
>
> [1] http://cassandra.apache.org/
>
> best regards,
> hanzhu
>
>
>
> On Mon, Dec 20, 2010 at 6:04 PM, Andrew Haley <aph at redhat.com> wrote:
>
>> On 12/19/2010 02:59 AM, Zhu Han wrote:
>>
>>> Hi,
>>>
>>> I might observe C-heap leak of IcedTea6 1.9.2 on Ubuntu 10.04. This bug
>>> was
>>> reported against cassandra[1]. But I'm afraid it is a general openJDK
>>> problem. Here is the symptoms. [1] may has more information on it.
>>>
>>> For a infrequently accessed openJDK node, the memory usage is climbed up
>>> every day, and I'm pretty sure it is not caused by leak on Java object
>>> heap.
>>> This is the GC settings:
>>>
>>> " /usr/bin/java -ea -Xms1G -Xmx1G -XX:+UseParNewGC
>>> -XX:+UseConcMarkSweepGC
>>> -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8
>>> -XX:MaxTenuringThreshold=1
>>> -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly
>>> -XX:+HeapDumpOnOutOfMemoryError ..."
>>>
>>> So the size of JVM heap is constant. Use pmap against the JVM process for
>>> the last 3 days, I observed a region expanded every day. It can be
>>> expanded
>>> to multiple gigabytes finally[1].
>>>
>>> on 12/19: 00000000010c3000 178548K rw---    [ anon ]
>>> on 12/18: 00000000010c3000 110320K rw---    [ anon ]
>>> on 12/17: 00000000010c3000  39256K rw---    [ anon ]
>>>
>>> I notice the region is on lower address of process address space. So I
>>> suspect it is the c-heap of JVM. The map of object heap is as below,
>>> which
>>> is constant in the last few days.
>>>
>>> 00002b58433c0000 1069824K rw---    [ anon ]
>>>
>>> Both the heap and non-heap memory usage of java code is quite normal:
>>>
>>> 12/19/2010 10:56:13 +0800 org.archive.jmx.Client HeapMemoryUsage:
>>> committed: 1065025536
>>> init: 1073741824
>>> max: 1065025536
>>> used: 160743000
>>>
>>> 12/19/2010 10:56:23 +0800 org.archive.jmx.Client NonHeapMemoryUsage:
>>> committed: 24313856
>>> init: 24313856
>>> max: 224395264
>>> used: 20477424
>>>
>>>
>>> There are hundreds of alive threads in the JVM.
>>>
>>> 12/19/2010 10:52:20 +0800 org.archive.jmx.Client PeakThreadCount: 244
>>>
>>>
>>>  Originally, I suspect it is bug 6824570[2]. But After I upgraded JDK
>>> from
>>> IceTea 1.8.2 to IceTea 1.9.2, the problem is still not fixed. [3] said it
>>> is
>>> fixed in IceTea 1.9.0.
>>>
>>> And there is another JVM instances running on the same node with ParNewGC
>>> enabled, I did not observe the memory leak.  Although the peak thread
>>> count
>>> is quite low of that instance, which is 43.
>>>
>>> Do you have any hints to diagnostic this problem? I'd like to help.
>>>
>>
>> This may, of course, be a leak in JNI code.  What Java program was
>> running?
>>
>> Andrew.
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/distro-pkg-dev/attachments/20101220/6f6dd870/attachment.html