jstack can't attach to JVM (OS X)

[Staffan Larsen]

On 26 aug 2013, at 20:19, Martin Traverso <mtraverso at gmail.com> wrote:

> Great! Thanks for the explanation.
> 
> Just curious, why does running jstack at least once before the code reaches that method call prevent the problem from happening at all?

No idea. :(  Can't figure it out.

/Staffan

> 
> Martin
> 
> 
> On Mon, Aug 26, 2013 at 5:31 AM, Staffan Larsen <staffan.larsen at oracle.com> wrote:
> Here is what's happening:
> 
> - The VM sets the signal mask for all threads (except the internal VMThread) to mask out SIGQUIT using pthread_sigmask(). So the process will still respond to SIGQUIT, but only one thread.
> - The verifier code calls setjmp() to save the calling context. On OS X this also saves the signal mask. 
> - The example code causes a verification error somewhere which causes the verifier to call longjmp().
> - longjmp will restore the signal mask using sigprocmask() which sets the signal mask for the _process_.
> - Now the process has a signal mask that masks out SIGQUIT and attach stops working.
> 
> This only happens on OS X because setjmp/longjmp does not save and restore the signal mask on other platforms. There are functions _setjmp/_longjmp on OS X to skip the signal mask save/restore and I think this is what we need to use in the verifier (also need to check other uses of setjmp/longjmp in the JDK).
> 
> I have filed bug 8023720 for this.
> 
> Thanks again for the report and reproducer.
> 
> /Staffan
> 
> 
> 
> 
> On 23 aug 2013, at 14:44, Staffan Larsen <staffan.larsen at oracle.com> wrote:
> 
>> Thanks for the excellent bugreport!
>> 
>> I've had a quick look today but haven't yet figured out what the problem is. What happens is that somehow the process becomes immune to the SIGQUIT that jstack sends to it when it wants to attach. Instead of running jstack, one can also do "kill -QUIT <pid>" and with this code no thread stacks are printed (as they normally will be).
>> 
>> I'll continue looking,
>> /Staffan 
>> 
>> On 23 aug 2013, at 07:20, Martin Traverso <mtraverso at gmail.com> wrote:
>> 
>>> I have a program that causes jstack and jcmd to fail to attach to the JVM on OS X. Unfortunately, it uses a third-party library, so I'm not sure what's the magic incantation that causes this to happen. 
>>> 
>>> I wrote a small test case using this library that reproduces the problem. You can find the code here: https://github.com/martint/jstackissue.
>>> 
>>> I've seen the problem on both 1.7.0_25-b15 and 1.8.0-ea-b100 on OS X, but not on Java 6. It doesn't seem to happen on Linux, either.
>>> 
>>> One interesting data point is that if I run jstack before the code makes the call to the library, it works, and continues to work even after that call is complete. On the other hand, if run jstack for the first time *after* the call to the library, it won't work at all.
>>> 
>>> Any ideas?
>>> 
>>> Thanks!
>>> Martin
>> 
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20130827/06924e02/attachment.html