Can't get hs_err log on native stack overflow on Linux

[Yasumasa Suenaga]

Hi, David,

Thank you for checking the history.

> What I can say is that the stack-banging that we do with the guard pages
> was considered generally more reliable, and could be applied the same
> way across all platforms. (The Solaris version also dropped all use of
> alternate signal stacks for other reasons.)

I've understood the history. 
I guess that is "-XX:AltStackSize" .
http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html


However, at least, VM stack guard page (RedZone: -XX:StackRedPages) does not
work in now implementation (on Linux x86 / AMD64). So, I think that we should
fix this problem to work this function.


Thanks,

Yasumasa

(2011/08/09 17:16), David Holmes wrote:
> Well I was right about there being history and wrong about the nature of
> the history. Seems we used alternate signal stacks on Linux up till 1.5
> when it was explicitly dropped:
> 
> 4852809: Linux: do not use alternate signal stack
> 
> Unfortunately that bug is not public so I can't divulge the reasoning
> behind the change.
> 
> What I can say is that the stack-banging that we do with the guard pages
> was considered generally more reliable, and could be applied the same
> way across all platforms. (The Solaris version also dropped all use of
> alternate signal stacks for other reasons.)
> 
> David
> 
> Yasumasa Suenaga said the following on 08/09/11 17:26:
>> Hi, David,
>> Thank you for replying.
>>
>> (2011/08/09 15:51), David Holmes wrote:
>>> Hi,
>>>
>>> I could be mistaken here but I believe the intent/hope is that any
>>> stackoverflow will be caught when the guard pages set up by the VM are
>>> accessed. In that way we haven't run out of true native stack and so we
>>> can still process the signal that indicates the stack overflow. This is
>>> not a perfect mechanism of course and there may be situations where you
>>> can jump over the guard pages and truly exhaust the stack.
>>
>> Yes, I agree.
>>
>>> I also believe there is a bit of bad history here, where we had problems
>>> trying to use alternative signal stacks on Linux. It will take me a bit
>>> of archaeology to dig up relevant info on that.
>>
>> If you've dug up relevant info, please tell me.
>>
>>
>> BTW, my patch provides new VM option "UseAlternateSignalStack" .
>> If this option sets to false, this patch (sigaltstack) will not work.
>>
>> When it is a viewpoint of the troubleshooting, I want to this function.
>> If I can get hs_err log at native stack overflow, I can certainly suggest
>> expanding stack area (-Xss).
>>
>> Thanks,
>>
>> Yasumasa
>>
>>> David Holmes
>>>
>>> Yasumasa Suenaga said the following on 08/09/11 16:06:
>>>> Hi,
>>>>
>>>> I encountered native stack overflow at JNI code on Linux (Fedora 15 and Ubuntu 11).
>>>> I got coredump image, however, I could not get hs_err log.
>>>>
>>>> In the case of SIGSEGV, hs_err log is generated in signal handler. If native
>>>> stack overflow occurred, Linux can't use stack area. So, SIGSEGV handler
>>>> (JVM_handle_linux_signal) is never called.
>>>>
>>>> manpage of sigaltstack(2):
>>>> /****************/
>>>> NOTES
>>>>          The most common usage of an alternate signal stack is to handle the SIGSEGV sig‐
>>>>          nal that is generated if the space available for the  normal  process  stack  is
>>>>          exhausted:  in  this case, a signal handler for SIGSEGV cannot be invoked on the
>>>>          process stack; if we wish to handle it, we must use an alternate signal stack.
>>>> /****************/
>>>>
>>>>
>>>> If this patch is applied, we can get hs_err log on native stack overflow as follows:
>>>>
>>>> /****************/
>>>> #
>>>> #  SIGSEGV (0xb) at pc=0x00007fb23f1265f7, pid=25748, tid=140403650643712
>>>> #  java.lang.StackOverflowError: Native stack
>>>> #
>>>> # JRE version: 8.0
>>>> # Java VM: OpenJDK 64-Bit Server VM (22.0-b01 mixed mode linux-amd64 compressed oops)
>>>> # Problematic frame:
>>>> # C  [liboverflow.so+0x5f7]  Java_Main_doStackOverflow+0x3b
>>>> /****************/
>>>>
>>>>
>>>> I've attached this patch and testcase in this email. Please check it.
>>>>
>>>>
>>>> I would like to contribute this patch, and I hope to apply this patch to
>>>> JDK 6 / 7 / 8.
>>>>
>>>>
>>>> Please cooperate.
>>>>
>>>> Best regards,
>>>> Yasumasa
>>>>
>>