RFR: 8279124: VM does not handle SIGQUIT during initialization [v5]

[Thomas Stuefe]

On Fri, 21 Jan 2022 09:00:49 GMT, David Holmes <dholmes at openjdk.org> wrote:

>> hi, David and Thomas, 
>> 
>> I am nervous about this change. It's not complex but I don't want to break the existing java applications. Bear with me. 
>> 
>> From my reading, HotSpot can chain a user-custom signal handler because of libjsig.so.  It's not like a linked-list chain. if the user installs a handler for a signal, libjsig just saves it. JVM_HANDLE_XXX_SIGNAL is invoked first and then the user-custom handler is called [here](https://github.com/openjdk/jdk/blob/master/src/hotspot/os/posix/signals_posix.cpp#L635) if it isn't handled.
>> 
>> The reason we can hoist this logic to line 564 because we **assume** that no user application would define a handler for BREAK_SIGNAL.  It doesn't work right now because os::initialize_jdk_signal_support() will overwrite the signal handler of BREAK_SIGNAL later.
>
> Hi Xin,
> Signal chaining doesn't work for BREAK_SIGNAL - from the signal chaining docs:
> 
> 
> Note:
> 
> The SIGQUIT, SIGTERM, SIGINT, and SIGHUP signals cannot be chained. If the application must 
> handle these signals, then consider using the —Xrs option.

Exactly. If embedding application likes to handle SIGQUIT, it needs to set -Xrs. In that case, it does not matter if it installed the signal handler before or after VM was initialized. We just won't touch SIGQUIT at all in that case.

Wrt the position of handling the break signal: most of the coding between lines 564 and 588 does not hurt; `PosixSignals::unblock_error_signals();` may actually be beneficial, though it is highly unlikely that we get problems with secondary crashes when handling SIGQUIT. So, line 588 would be an okay position for SIGQUIT handling too.

-------------

PR: https://git.openjdk.java.net/jdk/pull/7003