6850720: Allow POSIX_SPAWN to be used for ProcessImpl on Linux

[Thomas Stüfe]

Hi all,

First off, to be clear, I am certainly shedding no tears if I do not
get to contribute my vfork+exec*2 patch to upstream. It is a lot of
work, and I much rather do something else. However, I have spend too
much of my life with the Runtime.exec() layer to be easy about
changing it.

So far I have not read a single technical reason in this thread why
vfork needs to be abandoned now - apart from it being obsolete. If you
read my initial thread from September, you know that I think we have
understood vfork's shortcomings very well, and that our (SAPs)
proposed patch shows that they can be dealt with. In our port, our
vfork+exec*2 is solid since many years, without any issues.

I see that posix_spawn(), in its current form in the glibc and muslc,
could be a good option. Long term, maybe better than a home grown one.
But before we switch to posix_spawn(), I would like to understand
better, at least for the major distros, which distro would be affected
by regressions. Also, if we decide to switch to posix_spawn(), I would
like to prevent "accidental backporting" without checking the platform
requirements.

The current posix_spawn() implementation was added to glibc with glibc
2.24. So, what was the state of posix_spawn() before that version? Is
it safe to use, does it do the right thing, or will we encounter
regressions?

My Ubuntu 16.04 box runs glibc 2.23. Arguably, Ubuntu 16.04 is quite a
common distro. I have to check our machines at work, but I am very
sure that our zoo of SLES and RHEL servers do not all run glibc>=2.24,
especially on the more exotic architectures.

@Florian:

> and advise OS vendors to backport the
> glibc changes if they have customers that are impacted by the lack of
> process creation performance (or OOM during process creation).

Please not. We have no leverage toward the OS vendors, we cannot
"advise" anything. Even if OS vendors react and fix, we disrupt our
users and make them unhappy with us and with java. Every issue has to
be first analyzed before it even finds the way to us, let alone the OS
vendor. Lets not risk breaking userspace (don't you guys have a rule
like that :)?

Best Regards, Thomas
On Mon, Oct 22, 2018 at 3:37 PM Florian Weimer <fweimer at redhat.com> wrote:
>
> * David Lloyd:
>
> > Sure, but I don't really see this as necessary if glibc is already
> > following the vfork-like path.  Another thing to know is that at least
> > in the 2.26 sources I have checked out, the POSIX_SPAWN_USEVFORK flag
> > is completely ignored.  See also [2].
>
> Right, the manual pages are outdated.
>
> For applications, there is never a good reason to use
> POSIX_SPAWN_USEVFORK because the glibc implementation is either buggy
> when this flag is used, or the flag does nothing.  The bugs may matter
> to applications using OpenJDK, so I don't think you can set the flag
> within OpenJDK.  So the only thing you can do here is to use posix_spawn
> *without* POSIX_SPAWN_USEVFORK, and advise OS vendors to backport the
> glibc changes if they have customers that are impacted by the lack of
> process creation performance (or OOM during process creation).
>
> Another possibility would be to emulate what glibc's fixed, fork-based
> posix_spawn does, but this requires writing some machine code (for
> vfork/clone) and issuing direct system calls to bypass some abstractions
> in glibc (for setprocmask).
>
> > [2] https://github.com/bminor/glibc/commit/ccfb2964726512f6669fea99a43afa714e2e6a80
>
> Note that this neither the canonical glibc source code location, nor
> is the code actually used on Linux. 8-)
>
> Thanks,
> Florian