RFR: 8292695: SIGQUIT and jcmd attaching mechanism does not work with signal chaining library [v3]

[Thomas Stuefe]

On Thu, 8 Sep 2022 08:19:47 GMT, Man Cao <manc at openjdk.org> wrote:

>> Hi all,
>> 
>> Could anyone review this bug fix? See https://bugs.openjdk.org/browse/JDK-8292695 for details.
>> 
>> I changed the temporary handler for SIGQUIT to use a dummy function, and use `os::signal()` to set it up, just as `os::initialize_jdk_signal_support()` does.
>> It is possible that just moving the `set_signal_handler(BREAK_SIGNAL, false);` in `install_signal_handlers()` outside of the window bounded by `JVM_{begin|end}_signal_setting()` could also fix this bug. However, `set_signal_handler()` and `JVM_HANDLE_XXX_SIGNAL()` are currently used for signals that support chaining and periodically check, which do not apply to SIGQUIT. I think it is cleaner to use different functions for SIGQUIT.
>> 
>> I also added a test to check that sending SIGQUIT should produce a thread dump on stdout, with and without using libjsig.so.
>> 
>> -Man
>
> Man Cao has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Move jdk_misc_signal_init() earlier and clarify its comment

Can you also add a test that checks that:

if VM is started with -Xrs, and we send a SIGQUIT, the VM shall crash with a core. The crash should look like a system crash, not an hs-err file crash, like this:


Quit (core dumped)


which assures us that no hotspot signal handler is installed and we take therefore the default action on SIGQUIT.

If this string is OS dependent and gives us headaches, the test should check at least that the process exit code is not 0.

And of course the VM should be started with tiny heap etc such to make sure the core file generated will be tiny.

src/hotspot/os/posix/signals_posix.cpp line 1319:

> 1317:     // Also note that HotSpot does NOT support signal chaining for BREAK_SIGNAL.
> 1318:     // Applications that require a custom BREAK_SIGNAL handler should run with
> 1319:     // -XX:+ReduceSignalUsage. Otherwise if libjsig is used together with

Maybe also mention "AllowUserSignalHandlers"

test/hotspot/jtreg/runtime/Thread/TestBreakSignalThreadDump.java line 38:

> 36:  * @bug 8292695
> 37:  * @summary Check that Ctrl-\ causes HotSpot VM to print a full thread dump when signal chaining is used.
> 38:  * @requires os.family == "linux" | os.family == "mac"

Why not just !Windows? Aix, the BSDs, Solaris if this is downported should work too.

test/hotspot/jtreg/runtime/Thread/TestBreakSignalThreadDump.java line 62:

> 60:         public static void main(String[] argv) throws Exception {
> 61:             ProcessUtils.sendCtrlBreak();
> 62:             System.out.println("Done!");

Maybe don't finish right away but wait a second or two, for dispatcher thread on slow systems to catch up.

-------------

Changes requested by stuefe (Reviewer).

PR: https://git.openjdk.org/jdk/pull/9955