RFR: 8292695: SIGQUIT and jcmd attaching mechanism does not work with signal chaining library [v4]

[Thomas Stuefe]

On Fri, 9 Sep 2022 08:27:41 GMT, Man Cao <manc at openjdk.org> wrote:

>> Hi all,
>> 
>> Could anyone review this bug fix? See https://bugs.openjdk.org/browse/JDK-8292695 for details.
>> 
>> I changed the temporary handler for SIGQUIT to use a dummy function, and use `os::signal()` to set it up, just as `os::initialize_jdk_signal_support()` does.
>> It is possible that just moving the `set_signal_handler(BREAK_SIGNAL, false);` in `install_signal_handlers()` outside of the window bounded by `JVM_{begin|end}_signal_setting()` could also fix this bug. However, `set_signal_handler()` and `JVM_HANDLE_XXX_SIGNAL()` are currently used for signals that support chaining and periodically check, which do not apply to SIGQUIT. I think it is cleaner to use different functions for SIGQUIT.
>> 
>> I also added a test to check that sending SIGQUIT should produce a thread dump on stdout, with and without using libjsig.so.
>> 
>> -Man
>
> Man Cao has updated the pull request incrementally with one additional commit since the last revision:
> 
>   address comments in test

Looks good. Thank you for taking my suggestion!

..Thomas

> > Can you also add a test that checks that:
> > if VM is started with -Xrs, and we send a SIGQUIT, the VM shall crash with a core. The crash should look like a system crash, not an hs-err file crash, like this:
> 
> I struggled to get this to work. I found out it is due to https://bugs.openjdk.org/browse/JDK-8234262, and ProcessBuilder changes how SIGQUIT is handled in the subprocess. This is further complicated by how jtreg runs the parent process. The workaround (passing -Xrs to the parent process) does not work through jtreg using "main/othervm -Xrs".
> 
> I uploaded the not-working test in here: [caoman at 73e3a5f](https://github.com/caoman/jdk/commit/73e3a5f90985fcad9a383c704429cefb07011932)

Thank you for trying, and finding this out! I'm fine with postponing this test then to another RFE.

-------------

Marked as reviewed by stuefe (Reviewer).

PR: https://git.openjdk.org/jdk/pull/9955