RFR: 8012241 NMT huge memory footprint, it usually leads to OOME

[Daniel D. Daugherty]

On 7/11/13 10:49 AM, Zhengyu Gu wrote:
> Hi Dan,
>
> Thanks for the review.
>
>
> On 7/11/2013 12:11 PM, Daniel D. Daugherty wrote:
>> > http://cr.openjdk.java.net/%7Ezgu/8012241/webrev.00
>>
>> src/share/vm/services/memTracker.cpp
>>     I noticed that the NMT shutdown code path calls:
>>
>>     200   // shutdown shared decoder instance, since it is only
>>     201   // used by native memory tracking so far.
>>     202   Decoder::shutdown();
>>
>>     Isn't the Decoder used by the trouble shooting code at this point?
>>     If so, will this NMT shutdown cause hs_err_pid generation to not
>>     be able to generate crash info with symbols?
>>
> VMError uses a private instance of decoder, since error handling path 
> can not take any lock.
> NMT uses shared instance. Right  now, only NMT uses this instance, and 
> the instance can be re-created.

Thanks for clarifying.  Thumbs up!

Dan


>
> Thanks,
>
> -Zhengyu
>
>
>> Dan
>>
>>
>> On 7/11/13 7:07 AM, Zhengyu Gu wrote:
>>> The patch is a workaround that prevents JVM from OOM when running 
>>> Kitchensink with NMT on some slow machines.
>>>
>>> Kitchensink normally runs with about 100 threads, peaks around 150 
>>> threads. On some slow machines, NMT worker thread does not process 
>>> raw NMT data quick enough, that causes huge backlogs, which consumes 
>>> large amount of memory and can lead JVM OOM. By enforcing the new 
>>> threshold, NMT will shutdown itself and free NMT memory.
>>>
>>> If native memory tracking is important to diagnostic memory issue, 
>>> user can use -XX:-AutoShutdownNMT diagnostic flag to override above 
>>> behavior. Turning off auto shutdown NMT flag, NMT also tries slow 
>>> down malloc/free rate, to give NMT worker thread a chance to catch up.
>>>
>>>
>>> JBS: https://jbs.oracle.com/bugs/browse/JDK-8012241
>>> Exteneral bug: not available
>>> Webrev: http://cr.openjdk.java.net/~zgu/8012241/webrev.00/ 
>>> <http://cr.openjdk.java.net/%7Ezgu/8012241/webrev.00/>
>>>
>>>
>>> Tests:
>>>   Christian Tornqvist has verified on Windows and Solaris, that 
>>> patch prevented JVM from OOM.
>>>
>>> Thanks,
>>>
>>> -Zhengyu
>>
>