SIgnal chaining, JVM_handle_(linux|bsd|aix)_signal, and backward compatibility

[David Holmes]

Hi Thomas,

On 2/11/2020 5:40 pm, Thomas Stüfe wrote:
> Hi,
> 
> While working on some signal handler fixes and cleanups ([1]), I noticed
> that we export JVM_handle_(linux|bsd)_signal() on all POSIX platforms.
> 
> I wondered why we do this. See also my first question at runtime-dev [2] -
> the initial reaction was "probably no reason anymore".
> 
> But then, I see a carefully crafted comment in [3]:
> 
> <quote>
> // This routine may be used by user applications as a "hook" to catch
> signals.
> // The user-defined signal handler must pass unrecognized signals to this
> // routine
> ...
> </quote>
> 
> and I also found some bug reports [4], [5], from 2001 and 2004:
> JDK-4864136 : "JVM_handle_linux_signal is private in 1.4.2-beta"
> JDK-4408646 : "JVM_handle_solaris_signal must be a global function"
> 
> So, to me this looks like JVM_handle_xxx_signals() was an official, or at
> least deliberate, interface for foreign code to interact with our signal
> handling. Specifically, this reads like a third party could install its
> signal handlers over ours, and as long as it queried our signal handler
> first by calling JVM_handle_xxx_signal(), we still could coexist with it.

Did you see this comment:

https://bugs.openjdk.java.net/browse/JDK-4361067?focusedCommentId=12425956&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12425956

This is tied to AllowUserSignalHandlers so we'd have to go through the 
deprecation process to make any changes in this area.

> But now we have signal chaining [6], which achieves the same by preloading
> the libjsig library. Which some consider to be ewww [7] :-), but still
> seems to me like the official solution to that problem. Using signal
> chaining there would be no need to manually call JVM_handle_xxx_signal()
> from outside since the interposed sigaction() in libjsig would take care of
> all this stuff automatically.

Aren't the two mechanisms complementary in that they work in opposite 
ways? Signal chaining keeps the VM in control and allows it to call 
application handlers.  AllowUserSignalHandlers keeps the app in control 
and requires it to call VM handlers.

> My question is:
> - if JVM_handle_xxx_signal() is (had been?) an official interface, should
> there not be some official documentation and regression tests? Was there? I
> could not find anything.
> - if it is not an official interface, would it be okay to lay it to rest?
>    - Pro: it is made obsolete by signal chaining, and removing it would
> reduce complexity of signal handling somewhat
>    - Con: applications may exist out there which use that interface, and I'd
> hate to break them. We give customers enough reasons to cling to old JDK
> versions as it is.
>       Also, arguably, this interface is useful. Not everyone likes
> preloading libjsig, and the signal chaining mechanism is also a whole lot
> more fragile.
> 
> However, if we let it live on, should we not document and test it?

It is not well documented that is for sure.

Not sure about testing ... I tend to see this more a "best effort" 
mechanism. If someone reports it stops working, or doesn't work, then 
we'll make a best effort to fix it.

> I am really interested in the history of this. Was this just a solution for
> a single customer?

Not sure of specific details but it was for hosting the VM in native 
applications that already did their own signal management.

Cheers,
David

> 
> Thanks, Thomas
> 
> 
> [1] https://bugs.openjdk.java.net/browse/JDK-8255711
> [2]
> https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-October/043145.html
> [3]
> https://github.com/openjdk/jdk/blob/64feeab70af61a52ffe4c64df87a33c16754de18/src/hotspot/os/posix/signals_posix.cpp#L411
> [4] https://bugs.openjdk.java.net/browse/JDK-4864136
> [5] https://bugs.openjdk.java.net/browse/JDK-4408646
> [6]
> https://docs.oracle.com/javase/8/docs/technotes/guides/vm/signal-chaining.html
> [7] https://mail.openjdk.java.net/pipermail/discuss/2019-April/005042.html
>