RFR: JDK-8271858: Handle to jimage file inherited into child processes (posix)

[Alan Bateman]

On Thu, 5 Aug 2021 14:06:17 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> On Unix systems, the JDK has always relied on the Runtime.exec implementation to close the file descriptors. On Windows the inheritance has to be disabled in the parent. So if the gtest launcher is forking directly then we may have a bit of whack-a-mole to change all the places that open file descriptors.
>
> My idea was born since we have jtreg tests that assert that certain VM internal fds, namely of log files, are not handed down to child processes. The motivation originally was Windows, since child processes would block that file from being moved. The test is done for both Unix and Windows, however. It is fragile and difficult to analyze when it fails. I wanted to throw away the Unix portion of that test and replace it with a simple gtest, either checking CLOEXEC for just that particular fd, or (my current approach) for all handles.
> 
> But if what you are saying is how we do things - we don't bother with CLOEXEC, just rely on Runtime.exec() to close all fds, and accept the fact that "foreign" forks will cause us problems - then I could just throw the Unix portion of that test away without replacing it with anything.
> 
> ATM the libs/module fd seems to be the only file descriptor tripping up my test though. Maybe it's not so bad. I am mainly afraid of situations where the gtestlauncher runs in some instrumented form, maybe with a virus scanner in its belly, with foreign code opening fds without our knowledge. I am still unsure about which direction to go.

lib/modules is opened/mapped early in the startup so I assume this is why this one shows up quickly. Adding O_CLOEXEC everywhere is open, up you if you want to do that or switch it to Runtime.exec.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4991