RFR: 8292695: SIGQUIT and jcmd attaching mechanism does not work with signal chaining library

[Man Cao]

On Fri, 26 Aug 2022 01:13:11 GMT, David Holmes <dholmes at openjdk.org> wrote:

>> Hi all,
>> 
>> Could anyone review this bug fix? See https://bugs.openjdk.org/browse/JDK-8292695 for details.
>> 
>> I changed the temporary handler for SIGQUIT to use a dummy function, and use `os::signal()` to set it up, just as `os::initialize_jdk_signal_support()` does.
>> It is possible that just moving the `set_signal_handler(BREAK_SIGNAL, false);` in `install_signal_handlers()` outside of the window bounded by `JVM_{begin|end}_signal_setting()` could also fix this bug. However, `set_signal_handler()` and `JVM_HANDLE_XXX_SIGNAL()` are currently used for signals that support chaining and periodically check, which do not apply to SIGQUIT. I think it is cleaner to use different functions for SIGQUIT.
>> 
>> I also added a test to check that sending SIGQUIT should produce a thread dump on stdout, with and without using libjsig.so.
>> 
>> -Man
>
> src/hotspot/os/posix/signals_posix.cpp line 1345:
> 
>> 1343:     // JVM_begin_signal_setting and JVM_end_signal_setting above, because the
>> 1344:     // window is intended for signals that support chaining. Otherwise libjsig
>> 1345:     // would prevent us from overwriting BREAK_SIGNAL's handler to UserHandler.
> 
> Okay but doesn't that also mean that libjsig will now allow application code to install its own BREAK_SIGNAL handler, when it should be prevented from doing so???

Yes, this seems always the case regardless of this change. The application should use `-Xrs`/`-XX:+ReduceSignalUsage` if it needs to install its own handler for BREAK_SIGNAL.

This is also the reason why TestSigquit.java sets `-Xrs`, see test/hotspot/jtreg/runtime/signal/README:
> SIGQUIT, SIGTERM, SIGINT, and SIGHUP signals cannot be chained.
> If the application needs to handle these signals, the -Xrs option needs
> to be specified. So, test these signals only with -Xrs flag.

-------------

PR: https://git.openjdk.org/jdk/pull/9955