Runtime.exec and SIGPIPE + SIG_IGN

[David Holmes]

Hi Thomas,

On 29/04/2025 7:04 pm, Thomas Stüfe wrote:
> Hi,
> 
> I would like to gauge opinions on whether the following scenario is a 
> bug to fix or whether to accept it as standard behavior.
> 
> ---
> 
> A customer has the following problem:
> 
> - The JVM invokes a third-party JNI library that sets the signal 
> disposition of SIGPIPE to SIG_IGN (Boo! in this case, it is the FIPS 
> nspr library, which does this unconditionally.)
> - The JVM then spawns child processes
> - All child processes now ignore SIGPIPE, which leads to failures in 
> automation scripts
> 
> I was surprised. I always assumed the signal disposition of all signals 
> would be reset to SIG_DFL when exec'ing. However, seems I was wrong when 
> it came to SIG_IGN. Posix documents for execve() states [1]:
> 
> 1) " Signals set to the default action (SIG_DFL) in the calling process 
> image shall be set to the default action in the new process image."
> 
> and
> 
> 2) "Except for SIGCHLD, signals set to be ignored (SIG_IGN) by the 
> calling process image shall be set to be ignored by the new process image."
> 
> and
> 
> 3) "Signals set to be caught by the calling process image shall be set 
> to the default action in the new process image (see /<signal.h>/ 
> <https://pubs.opengroup.org/onlinepubs/009695399/basedefs/signal.h.html>)."
> 
> (2) and (3) are the interesting parts. (2) means that if the parent 
> process ignores SIGPIPE, child processes will also ignore SIGPIPE. (3) 
> means that if the parent process has a custom handler installed for 
> SIGPIPE, child processes will be started with SIG_DFL (default action) 
> for SIGPIPE. The default action for SIGPIPE is "terminate process".
> 
> The libjvm handles SIGPIPE. We install our `javaSignalHandler`. We do 
> this to eat up and ignore SIGPIPE. That we "manually" ignore the signal 
> beside the point here - what counts is that we set a custom signal 
> handler for SIGPIPE. Therefore, on execve, we behave according to rule 
> (3) and start the child with SIG_DFL as SIGPIPE disposition. As it 
> should be.
> 
> If third-party code sets the handler to SIG_IGN as in this example, 
> exeve will behave according to rule (2) and the child will start with 
> SIG_IGN as SIGPIPE disposition.
> 
> The libjig can be used to workaround this scenario, but I wonder if that 
> is more of an accident. The libjsig.so will preserve the JVM's SIGPIPE 
> handler even if third-party code attempts to set it to SIG_IGN. That 
> means that exeve still behaves according to rule (2): sets child's 
> SIGPIPE disposition to SIG_DFL.
> 
> ----
> 
> But I wonder whether this should not be considered a bug to fix 
> regardless of the jsig.so workaround? In jspawnhelper, we clean the 
> environment from various effects when exec'ing; among other things, we 
> reset the signal block mask for the process. The "ignore" state of 
> processes could be considered along the same line. We could reset all 
> signal handlers to SIG_DFL before execing the child.
> 
> I know that this area is super-tricky and problems are notoriously 
> difficult to analyze; we should therefore be extremely careful not to 
> break downward compatibility. Still, what do people think? Should be fix 
> this in jspawnhelper?

I think at most, the exec'ing code could restore the signals the VM has 
modified to their default disposition - which in practice would mean 
only those set to SIG_IGN would need to explicitly be changed, which in 
turn may mean only SIGPIPE is affected?

Even so, a library/application that ignores SIGPIPE and exec's other 
processes which it expects also ignore SIGPIPE, would not be happy with 
such a change.

In the customer scenario described maybe the simplest fix is to have 
them use libjsig.

Cheers,
David

> 
> Thanks, Thomas
> 
> (cc'ing Roger)
> 
> [1] https://pubs.opengroup.org/onlinepubs/009695399/functions/exec.html 
> <https://pubs.opengroup.org/onlinepubs/009695399/functions/exec.html>