RFR(s): 8223777: In posix_spawn mode, failing to exec() jspawnhelper does not result in an error

[Thomas Stüfe]

Hi all,

may I have please your opinions about the following change:

Bug: https://bugs.openjdk.java.net/browse/JDK-8223777
webrev:
http://cr.openjdk.java.net/~stuefe/webrevs/8223777-posix_spawn-no-exec-error/webrev.00/webrev/

In short, since some weeks posix_spawn() is used on Linux to spawn childs.
But the glibc implementation of posix_spawn does not report errors back to
the caller if exec()ing the target binary fails.

We only ever exec the jspawnhelper, which is part of the JDK and which then
exec()s the real target binary. The second exec() is under our control and
Martin did provide proper error reporting.

However, the first exec(), performed inside posix_spawn(), may fail too.
Most posix_spawn implementations will tell us if that happens, not glibc
(it is cumbersome to implement, one needs a pipe and somesuch, and POSIX
leaves a bit of headroom for the implementor to cheap out, so they did.)

So, if someone messes with the JDK installation and e.g. removes the
execute bits from jspawnhelper, that first exec() fails, but we get no
error. From the outside it looks like Runtime.exec() succeeded.

This is impossible to fix completely - see Martin's comment about the
impossibility to foresee whether an exec() will succeed without actually
exec()ing. But at least we can test the execute permissions on the
jspawnhelper. Which this fix does. This fixes Ubuntu 16.4 (Now, I get an
IOException if jspawnhelper is not executable).

Since I like to be super careful with runtime.exec coding, I will run this
through our nightlies and also would like feedback from the core libs
people. Also, I am not sure about the runtime costs of this hack. Seems a
bit excessive for something which is just a setup error (a correctly
installed JDK should never have this problem).

As a side note, it may help if jspawnhelper would live in the bin folder of
the JDK (currently lives in lib), because everyone should be aware that
programs in bin need execute permissions...

Cheers, Thomas