Losing features of JVM_Open, e.g. CLOEXEC

[Martin Buchholz]

Hi guys,

In your change

8057777: Cleanup of old and unused VM interfaces

you have made changes like this:

-            zfd = JVM_Open(path, flag, 0);
+            zfd = open(path, flag, 0);

throwing away use of old shared infrastructure and replacing with "naked"
calls to the OS.  Although understandable, this abandons benefits of using
shared infrastructure.  Here I'm thinking of the close-on-exec flag.  I
just added use of O_CLOEXEC to linux hotspot, but e.g. zip file opening no
longer uses JVM_Open, which is a code hygiene regression.

What we want is to have almost all file descriptors have the close-on-exec
flag automatically set at fd creation time, using O_CLOEXEC where
available, and FD_CLOEXEC where not.  How do we get there?

I'm distressed that the JDK core libraries should be moving towards having
*more* shared native code infrastructure, but here we seem to be moving in
the opposite direction.  Having abandoned JVM_Open, the responsibility of
doing these things right belongs entirely to the core libraries team.  So
where's the core-library replacement for JVM_Open?

I'll quote from os::open:


  // All file descriptors that are opened in the Java process and not
  // specifically destined for a subprocess should have the close-on-exec
  // flag set.  If we don't set it, then careless 3rd party native code
  // might fork and exec without closing all appropriate file descriptors
  // (e.g. as we do in closeDescriptors in UNIXProcess.c), and this in
  // turn might:
  //
  // - cause end-of-file to fail to be detected on some file
  //   descriptors, resulting in mysterious hangs, or
  //
  // - might cause an fopen in the subprocess to fail on a system
  //   suffering from bug 1085341.
  //
  // (Yes, the default setting of the close-on-exec flag is a Unix
  // design flaw)
  //
  // See:
  // 1085341: 32-bit stdio routines should support file descriptors >255
  // 4843136: (process) pipe file descriptor from Runtime.exec not being
closed
  // 6339493: (process) Runtime.exec does not close all file descriptors on
Solaris 9
  //
  // Modern Linux kernels (after 2.6.23 2007) support O_CLOEXEC with open().
  // O_CLOEXEC is preferable to using FD_CLOEXEC on an open file descriptor
  // because it saves a system call and removes a small window where the
flag
  // is unset.  On ancient Linux kernels the O_CLOEXEC flag will be ignored
  // and we fall back to using FD_CLOEXEC (see below).
#ifdef O_CLOEXEC
  oflag |= O_CLOEXEC;
#endif