OpenJDK OOM issue -
Zhengyu Gu
zgu at redhat.com
Thu Sep 21 14:07:36 UTC 2017
Hi Tim,
Try to run with -XX:NativeMemoryTracking=summary , this should give you
some hints on native memory side.
In your case, not be able to create native thread, more likely to be on
native side than heap.
Thanks,
-Zhengyu
On 09/20/2017 11:22 PM, Yu, Tim (NSB - CN/Chengdu) wrote:
> Hi Poonam & Vladimir
>
> After add " -XX:-UseCompressedOops" flag, the OMM still happens. The corresponding GC log is as below and no heap is printed out. So, what's the next step to do? Please help on this and many thanks :)
>
> OpenJDK 64-Bit Server VM (25.131-b11) for linux-amd64 JRE (1.8.0_131-b11), built on Apr 13 2017 17:56:19 by "mockbuild" with gcc 4.4.7 20120313 (Red Hat 4.4.7-18)
> Memory: 4k page, physical 11163792k(551024k free), swap 16777212k(16722204k free)
> CommandLine flags: -XX:InitialHeapSize=178620672 -XX:MaxHeapSize=2857930752 -XX:+PrintGC -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:-UseCompressedOops -XX:+UseParallelGC
>
> Br,
> Tim
>
> -----Original Message-----
> From: Poonam Parhar [mailto:poonam.bajaj at oracle.com]
> Sent: Thursday, September 21, 2017 12:37 AM
> To: Vladimir Kozlov <vladimir.kozlov at oracle.com>; David Holmes <david.holmes at oracle.com>; Yu, Tim (NSB - CN/Chengdu) <tim.yu at nokia-sbell.com>; Andrew Haley <aph at redhat.com>; hotspot-dev developers <hotspot-dev at openjdk.java.net>
> Cc: Shen, David (NSB - CN/Chengdu) <david.shen at nokia-sbell.com>
> Subject: RE: OpenJDK OOM issue -
>
> Hi Vladimir,
>
>> -----Original Message-----
>> From: Vladimir Kozlov
>> Sent: Wednesday, September 20, 2017 9:30 AM
>> To: Poonam Parhar; David Holmes; Yu, Tim (NSB - CN/Chengdu); Andrew
>> Haley; hotspot-dev developers
>> Cc: Shen, David (NSB - CN/Chengdu)
>> Subject: Re: OpenJDK OOM issue -
>>
>> On Linux we should not have java heap's low address memory problem.
>>
>> Small swap space? Also memory left is less than 1Gb ("MemFree: 898332
>> kB").
>>
>> Also 5326 processes it a lot. Overloaded system?
>>
>> Poonam, do we have bug for this? Can you attached hs_err file to it.
>>
>
> No, there is no bug for this.
>
> Thanks,
> Poonam
>
>> Thanks,
>> Vladimir
>>
>> On 9/20/17 9:15 AM, Poonam Parhar wrote:
>>> Hello Tim,
>>>
>>> From the hs_err_pid12678.log file, the java heap is based at
>> 0x715a00000 which is 28gb, so there should be plenty of space available
>> for the native heap.
>>>
>>> Memory map:
>>> ...
>>> 00600000-00601000 rw-p 00000000 fc:01 17950
>> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.131-
>> 0.b11.el6_9.x86_64/jre/bin/java
>>> 019bc000-019dd000 rw-p 00000000 00:00 0
>> [heap]
>>> 715a00000-71cd00000 rw-p 00000000 00:00 0
>>> 71cd00000-787380000 ---p 00000000 00:00 0 ...
>>>
>>> Heap
>>> PSYoungGen total 51200K, used 13258K [0x0000000787380000,
>> 0x000000078ac80000, 0x00000007c0000000)
>>> eden space 44032K, 30% used
>> [0x0000000787380000,0x0000000788072ad0,0x0000000789e80000)
>>> from space 7168K, 0% used
>> [0x000000078a580000,0x000000078a580000,0x000000078ac80000)
>>> to space 7168K, 0% used
>> [0x0000000789e80000,0x0000000789e80000,0x000000078a580000)
>>> ParOldGen total 117760K, used 0K [0x0000000715a00000,
>> 0x000000071cd00000, 0x0000000787380000)
>>> object space 117760K, 0% used
>> [0x0000000715a00000,0x0000000715a00000,0x000000071cd00000)
>>> Metaspace used 10485K, capacity 10722K, committed 11008K,
>> reserved 1058816K
>>> class space used 1125K, capacity 1227K, committed 1280K,
>> reserved 1048576K
>>>
>>>
>>> To narrow down the issue, would it be possible for you to test with -
>> XX:-UseCompressedOops?
>>>
>>> Thanks,
>>> Poonam
>>>
>>>> -----Original Message-----
>>>> From: David Holmes
>>>> Sent: Wednesday, September 20, 2017 4:43 AM
>>>> To: Yu, Tim (NSB - CN/Chengdu); Andrew Haley; Poonam Parhar;
>> hotspot-
>>>> dev developers
>>>> Cc: Shen, David (NSB - CN/Chengdu)
>>>> Subject: Re: OpenJDK OOM issue -
>>>>
>>>> Tim,
>>>>
>>>> Please note attachments get stripped from the mailing lists.
>>>>
>>>> All - please drop the jdk8-dev and jdk8u-dev mailing lists from this
>>>> and leave it just on hotspot-dev. I've tried to bcc those lists.
>>>>
>>>> Thank you.
>>>>
>>>> David
>>>>
>>>> On 20/09/2017 6:44 PM, Yu, Tim (NSB - CN/Chengdu) wrote:
>>>>> Hi All
>>>>>
>>>>> Thank you all for the quick response.
>>>>> The environment information is listed as below, could you please
>>>>> help
>>>> to further check?
>>>>>
>>>>> 1. What OS is this?
>>>>> # cat /etc/redhat-release
>>>>> Red Hat Enterprise Linux Server release 6.9 (Santiago) # uname -a
>>>>> Linux cloudyvm16 2.6.32-696.6.3.el6.x86_64 #1 SMP Fri Jun 30
>>>>> 13:24:18 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux
>>>>>
>>>>> 2.GC log is listed as below. The heap information cannot be printed
>>>> out in gc-2017_09_20-09_21_15.log when OOM happens. In gc-
>> 2017_09_20-
>>>> 09_21_17.log, you can see the heap begins with 0x0000000787380000
>> and
>>>> it should be not the first 4G virtual memory address.
>>>>> -rw-r--r-- 1 19477 Sep 20 09:21 hs_err_pid12678.log
>>>>> -rw-r--r-- 1 570 Sep 20 09:21 gc-2017_09_20-09_21_15.log
>>>>> -rw-r--r-- 1 17741 Sep 20 09:21 hs_err_pid12706.log
>>>>> -rw-r--r-- 1 1297 Sep 20 09:21 gc-2017_09_20-09_21_17.log
>>>>> -rw-r--r-- 1 1722 Sep 20 09:21 gc-2017_09_20-09_21_18.log
>>>>> -rw-r--r-- 1 1297 Sep 20 09:21 gc-2017_09_20-09_21_19.log
>>>>> -rw-r--r-- 1 1297 Sep 20 09:21 gc-2017_09_20-09_21_20.log
>>>>>
>>>>> 3. This issue happens occasionally but frequently. We periodically
>>>> launch a JAVA program to use JMX to monitor service status of
>> another
>>>> JAVA service.
>>>>>
>>>>> Br,
>>>>> Tim
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> -----Original Message-----
>>>>> From: Andrew Haley [mailto:aph at redhat.com]
>>>>> Sent: Tuesday, September 19, 2017 9:13 PM
>>>>> To: Yu, Tim (NSB - CN/Chengdu) <tim.yu at nokia-sbell.com>;
>>>>> jdk8-dev at openjdk.java.net; jdk8u-dev at openjdk.java.net
>>>>> Cc: Shen, David (NSB - CN/Chengdu) <david.shen at nokia-sbell.com>
>>>>> Subject: Re: OpenJDK OOM issue -
>>>>>
>>>>> On 19/09/17 09:50, Yu, Tim (NSB - CN/Chengdu) wrote:
>>>>>> Hi OpenJDK dev group
>>>>>>
>>>>>> We meet one issue that the VM failed to initialize. The error log
>>>>>> is
>>>> as below. We checked both memory usage and thread number. They do
>> not
>>>> hit the limit. So could you please help to confirm why
>>>> "java.lang.OutOfMemoryError: unable to create new native thread"
>>>> error occurs? Many thanks.
>>>>>
>>>>> What OS is this?
>>>>>
More information about the hotspot-dev
mailing list