Replacing syscall(_NR_fork) on linux with syscall(_NR_clone)

[David Holmes]

Hi Thomas,

On 10/04/2015 8:36 PM, Thomas Stüfe wrote:
> Hi David,
>
> it is difficult to understand Linux implementation of
> os::fork_and_exec() fully. Comment claims fork() would not be async safe
> for use in signal handler, but I do not see why (maybe I am being slow
> here): pthread_atfork() handlers get called in the context of the thread
> calling the fork(). Depending on the pthread_atfork handlers, this may
> make the code non-reentrant, but async-unsafe?

As the comment says:

  // fork() in LinuxThreads/NPTL is not async-safe.

but I suspect this is an extremely old comment and no longer applicable 
to NPTL. So I think you and Martin are right that there's no reason not 
to call fork() and execve() using the glibc calls rather than the direct 
syscalls.

> Also the comment above the execve syscall is a bit of a riddle.
> "execve() will instead kill every thread in the parent process.". I
> would love to understand why.

Again as stated in the comments this was a "feature" of the LinuxThreads 
implementation. I don't know the details - before my time :)

> But if pthread_atfork handlers are the problem, how about just
> deactivating pthread_atfork handlers for the duration of
> os::fork_and_exec? Note that the jdk itself does not install
> pthread_atfork handlers, so this only affects user handlers.

I don't think at_fork handlers are specifically an issue - this is not 
trying to solve every problem in every circumstance.

> Another alternative to using raw syscalls would be to use posix_spawn(),
> which I believe does not call pthread_atfork handlers.

"It is implementation-defined whether the fork handlers are run when 
posix_spawn() or posix_spawnp() is called." - POSIX

Of course Linux may have made a determination one way or another.

But given Martin's experience with using fork() and execve() at Google, 
I would propose just adopting that approach.

Aside: it was also suggested that we simply drop this "onError" 
mechanism and replace it with one that simply halts the VM process and 
thereby allow the user to run arbitrary external commands. That may be 
worth an RFE in itself, but I need something in the short term.

Thanks,
David

> Regards, Thomas
>
>
> On Fri, Apr 10, 2015 at 9:47 AM, David Holmes <david.holmes at oracle.com
> <mailto:david.holmes at oracle.com>> wrote:
>
>     In os::fork_and_exec we provide a per platform fork/exec
>     implementation for use by the OnError and OnOutOfMemory JVM options.
>
>     On linux this is implemented with direct syscalls to _NR_fork
>     (sys_fork) and _NR_execve.
>
>     We already encountered a problem on linux-sparc:
>
>     http://mail.openjdk.java.net/__pipermail/hotspot-runtime-dev/__2015-February/013811.html
>     <http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2015-February/013811.html>
>
>     due to a different register usage for the return value, which is so
>     far restricted to the 7u train but will need addressing in 8u and 9.
>
>     Further it seems that the fork syscall has actually been deprecated
>     for quite some time and we are now seeing kernels on some platforms
>     which are no longer providing any of the deprecated syscalls - we
>     simply get an error ENOSYS.
>
>     The fork() library was updated to use clone a long time ago, but as
>     I understand it we can't use the fork() library call because we
>     might be executing in a signal-handling context within the error
>     handler.
>
>     So I'm considering switching to use the clone syscall for all linux
>     systems.
>
>     Thoughts/comments?
>
>     David
>
>