RFR(S): 8141356: Explicitly stop CMS threads during VM termination.

[Jungwoo Ha]

I don't think it has anything to do with the user threads.
The crash was during the VM termination. The launcher initiated the
termination (I guess through DestroyJavaVM or similar call), and crashed
inside CMS. Following is the stack trace.

*** SIGSEGV (@(nil)), see go/cppdebug received by PID 782912 (TID 783045);
stack trace: ***
PC: @     0x7fc61e2e1e6c  (unknown)  CMSCollector::collect_in_background()
    @          0x29f09a0       1312  FailureSignalHandler()
    @     0x7fc61e6fd147        208  os::Linux::chained_handler()
    @     0x7fc61e701994        336  JVM_handle_linux_signal
    @     0x7fc61e6f64b8         64  signalHandler()
    @     0x7fc621578340       1888  __restore_rt
    @     0x7fc61e2e92f9         48  ConcurrentMarkSweepThread::run()
    @     0x7fc61e6f78a2         64  java_start()
    @     0x7fc62156e800        176  start_thread
    @     0x7fc6211d22ed  (unknown)  clone

tcmalloc writes 0xcdcdcdcd when it deallocates the memory (in this case,
Java heap), and core dump show that CMS thread was interpreting that word
as an oop and crashed.


On Wed, Nov 4, 2015 at 9:51 AM, Jon Masamitsu <jon.masamitsu at oracle.com>
wrote:

>
>
> On 11/4/2015 9:33 AM, Jungwoo Ha wrote:
>
> Unfortunately, we cannot provide the test publicly.
> But we do have internal tests that triggers this bug, and got fixed with
> this patch.
>
>
> Can you describe the circumstances that provoke the bug?  Lots of
> concurrent threads?
> Lots of concurrent work in progress?
>
> Jon
>
>
> On Wed, Nov 4, 2015 at 9:25 AM, Jon Masamitsu <jon.masamitsu at oracle.com>
> wrote:
>
>>
>>
>> On 11/3/2015 5:32 PM, Jungwoo Ha wrote:
>>
>> Somehow I didn't receive Jon's email.
>> We've first noticed this on JDK8, but I think the bug is more triggered
>> with the tcmalloc + custom launcher.
>>
>>
>> Jungwoo,
>>
>> Do you have a test that exhibits the bug?
>>
>> Jon
>>
>>
>>
>> On Tue, Nov 3, 2015 at 4:14 PM, Carsten Varming < <varming at gmail.com>
>> varming at gmail.com> wrote:
>>
>>> Dear Jon,
>>>
>>> I am pretty sure this has been that way for a long time. I have never
>>> seen a problem, but I would like to see Hotspot shut down its threads as
>>> well.
>>>
>>> Carsten
>>>
>>> On Tue, Nov 3, 2015 at 5:40 PM, Jon Masamitsu <
>>> <jon.masamitsu at oracle.com>jon.masamitsu at oracle.com> wrote:
>>>
>>>> Jungwoo,
>>>>
>>>> Did you first see this bug in jdk8?  I'm wondering if it
>>>> was always broken.
>>>>
>>>> Jon
>>>>
>>>> On 11/03/2015 01:14 PM, Jungwoo Ha wrote:
>>>>
>>>> BUG:  <https://bugs.openjdk.java.net/browse/JDK-8141356>
>>>> https://bugs.openjdk.java.net/browse/JDK-8141356
>>>> Webrev:  <http://cr.openjdk.java.net/%7Ejwha/8141356/webrev.00/>
>>>> http://cr.openjdk.java.net/~jwha/8141356/webrev.00/
>>>>
>>>> Inside before_exit(), it says
>>>>
>>>>   // Stop concurrent GC threads
>>>>   Universe::heap()->stop();
>>>>
>>>> but GenCollectedHeap never implemented stop() method, and thus calling
>>>> empty SharedHeap::stop() method.
>>>> This causes CMS threads to run during the VM termination and ends up
>>>> crashing.
>>>> ConcurrentMarkSweepThread::stop() is currently a dead code and never
>>>> gets called.
>>>> The patch just implemented GenCollectedHeap::stop() to call
>>>> ConcurrentMarkSweepThread::stop().
>>>>
>>>> Can someone sponsor this patch?
>>>>
>>>> --
>>>> Jungwoo Ha | Google | <jwha at google.com>jwha at google.com
>>>>
>>>>
>>>>
>>>
>>
>>
>> --
>> Jungwoo Ha | Java Platform Team | <jwha at google.com>jwha at google.com
>>
>>
>>
>
>
> --
> Jungwoo Ha | Java Platform Team | jwha at google.com
>
>
>


-- 
Jungwoo Ha | Java Platform Team | jwha at google.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20151104/dde874d8/attachment.htm>