Replacing syscall(_NR_fork) on linux with syscall(_NR_clone)

[Thomas Stüfe]

Hi David,

it is difficult to understand Linux implementation of os::fork_and_exec()
fully. Comment claims fork() would not be async safe for use in signal
handler, but I do not see why (maybe I am being slow here):
pthread_atfork() handlers get called in the context of the thread calling
the fork(). Depending on the pthread_atfork handlers, this may make the
code non-reentrant, but async-unsafe?

Also the comment above the execve syscall is a bit of a riddle. "execve()
will instead kill every thread in the parent process.". I would love to
understand why.

But if pthread_atfork handlers are the problem, how about just deactivating
pthread_atfork handlers for the duration of os::fork_and_exec? Note that
the jdk itself does not install pthread_atfork handlers, so this only
affects user handlers.

Another alternative to using raw syscalls would be to use posix_spawn(),
which I believe does not call pthread_atfork handlers.

Regards, Thomas


On Fri, Apr 10, 2015 at 9:47 AM, David Holmes <david.holmes at oracle.com>
wrote:

> In os::fork_and_exec we provide a per platform fork/exec implementation
> for use by the OnError and OnOutOfMemory JVM options.
>
> On linux this is implemented with direct syscalls to _NR_fork (sys_fork)
> and _NR_execve.
>
> We already encountered a problem on linux-sparc:
>
> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/
> 2015-February/013811.html
>
> due to a different register usage for the return value, which is so far
> restricted to the 7u train but will need addressing in 8u and 9.
>
> Further it seems that the fork syscall has actually been deprecated for
> quite some time and we are now seeing kernels on some platforms which are
> no longer providing any of the deprecated syscalls - we simply get an error
> ENOSYS.
>
> The fork() library was updated to use clone a long time ago, but as I
> understand it we can't use the fork() library call because we might be
> executing in a signal-handling context within the error handler.
>
> So I'm considering switching to use the clone syscall for all linux
> systems.
>
> Thoughts/comments?
>
> David
>
>