Safepoint Deadlock Diagnosis

[David Holmes]

I'd suggest turning on all class-loading/resolution related tracing. 
Seems to me that deopt is forcing a call-site to be re-resolved while a 
safepoint is pending. But the re-resolution seems to be livelocking 
somewhere.

David

Peng Du said the following on 04/14/10 08:42:
> Thanks, Ramki and David
> 
> Sorry for the confusion, I was running a custom HotSpot that I have 
> modified.
> So it's definitely my fault :)
> 
> After the previous post, I did more investigations by trying the various 
> -XX
> options, especially those on safepoints, e.g. TraceSafepoint, 
> ShowSafepointMsgs,
> and etc. I saw lots of consecutive:
> 
> ======
> Safepoint synchronization initiated. (7)
> Entering safepoint region: ForceSafepoint
> do_cleanup_tasks takes      0ms
> Leaving safepoint region
> ======
> 
> I am not sure if that was the normal behavior. Then, I tried: 
> TraceRuntimeCalls,
> which confirmed that HotSpot kept spinning on 
> SharedRuntime::handle_wrong_method
> a large number of times. So, I guess David is right. It seems like a 
> live lock.
> 
> Can someone explain why would HotSpot keep calling handle_wrong_method()?
> 
> Thanks
> 
> 
> Peng
> 
> 
> 
> On 04/13/2010 05:15 PM, Y. Srinivas Ramakrishna wrote:
>> Try the debug jvm if not already doing so and it might tell you
>> why you are going to deadlock. If not, then you might have some
>> other kind of class loading deadlock which you can learn more about
>> from the class loading cognoscenti on this list. (May be use
>> TraceLoaderConstraints and TraceClass* flags for more visibility
>> into the sequence of events leading to this.) Since deadlocks
>> involve more than a single thread, you might have wanted to
>> look at other threads to see where they were. It wasn't clear
>> where this thread was stuck (was it acquiring a lock or waiting
>> for notification on a cond_var)? You might be able to trace the
>> waits-for chain on the JVM locks at issue here to understand
>> the nature of the deadlock if any.
>>
>> -- ramki
>>
>> On 04/13/10 13:40, Peng Du wrote:
>>> Hello, list
>>>
>>> I have been witnessing deadlocks of the HotSpot I am working on. I 
>>> tried everything I know
>>> to diagnose this. Here is what I found: the "main" JavaThread always 
>>> ended up somewhere
>>> in the stub SharedRuntime::handle_wrong_method(). HotSpot would then 
>>> deadlock at this point.
>>> In to the thread dump, I saw all threads are: _thread_blocked. The 
>>> method that was patched
>>> with this stub is "java_lang_math_log" (frame 12 @ 0x00007f832173b40e).
>>>
>>> I have attached the backtrace and thread dump in the email. Can 
>>> someone hint on what might
>>> be the reason for this deadlock?
>>>
>>> Thanks!
>>>
>>>
>>> -Peng
>>>
>>
>