Proposal: Add support for Process Groups to the JDK

[Thomas Stüfe]

Hi David,

On Wed, Nov 21, 2018 at 10:57 AM David Holmes <david.holmes at oracle.com> wrote:
>
> On 21/11/2018 6:14 pm, Thomas Stüfe wrote:
> > Ping.. no-one has any thoughts on this?
>
> I can see it being useful for test harnesses/frameworks.

Oh, very much so! We use this feature in the framework that runs our
nightlies. Since there the likelihood for hanging/crashing processes
is high, this feature is an ideal fit.

We also use it in other process scheduling roles, starting/observing
server processes etc.

>
> It's not something I've ever looked at so have no first hand knowledge.
> I wonder how the semantics of Windows Jobs compare to *NIX process
> groups? Do they map nicely?

Unfortunately no. They seem to hover more in cgroups territory - e.g.
limiting resource usage etc. But if all you want to do is to group
processes together and being able to kill them and their children in
one go, that would work with Windows Jobs. I am not sure about
suspending though, or sending to background, etc.

Personally I would much rather expose more Posix-like features,
because I want to see a shell written in Java at some point :) But the
group-killing ability is to us the most important aspect of this
proposal.

Cheers, Thomas

>
> Cheers,
> David
>
> > Thanks, Thomas
> > On Mon, Nov 12, 2018 at 6:29 PM Thomas Stüfe <thomas.stuefe at gmail.com> wrote:
> >>
> >> Dear all,
> >>
> >> may I please hear your thoughts about the following proposal?
> >>
> >> We would like to add support for process groups to the JDK: the
> >> ability to put child processes into new or pre-existing process
> >> groups. We added this feature to our proprietary port some time ago
> >> and has been very useful in cases where the VM acts in a process
> >> scheduling role.
> >>
> >> With process groups we mean of course standard Unix process groups.
> >> There exists a similar concept on Windows, Job Objects, so at least a
> >> subset of what we propose could be done in a platform independent way.
> >>
> >> ----
> >>
> >> Motivation:
> >>
> >> Most importantly, the ability to safely terminate a group of processes.
> >>
> >> The established way to do this is, since Java 9, to iterate over a
> >> process tree, calling Process.children() or Process.descendants() on
> >> the root Process, and killing them using Process.destroy().
> >>
> >> In practice, that approach is not always a good fit. It leaves out any
> >> orphaned processes; any deceased non-leaf process in the tree makes
> >> its children unreachable. Worst case, if the root process dies, all
> >> children are orphaned and cannot be reached. Another limitation is
> >> that this only works for process trees - parent-child relationships -
> >> but not for unrelated processes one might want to group together. It
> >> also becomes a bit inefficient with many processes, requiring one JNI
> >> call/system call per process to kill.
> >>
> >> Process groups, OTOH, would allow us to group together any number of
> >> unrelated processes. We can then send them bulk signals, eg
> >> SIGTERM/SIGKILL with only one system call. And for that to work, the
> >> parent relationships do not matter, so we also reach processes which
> >> have been orphaned.
> >>
> >> There are more things one could do with process groups besides killing
> >> them: suspend/resume them together (SIGSTOP/CONT), or to send them to
> >> the background of the controlling terminal.
> >>
> >> In fact, one could write its own shell in Java :)
> >>
> >> ----
> >>
> >> I drew up a tiny patch to demonstrate how this could look. This is
> >> just an example, to have something to play with and talk about:
> >>
> >> http://cr.openjdk.java.net/~stuefe/webrevs/processgroup-support/webrev.01/webrev/index.html
> >>
> >> and here is a small usage example:
> >>
> >> https://github.com/tstuefe/ojdk-repros/blob/master/src/other/RuntimeExecSimpleTestWithProcessGroup.java
> >>
> >> The suggested API changes are small:
> >>
> >> - A new class ProcessGroup as the platform's notion of a process
> >> group. In this patch, it offers four functions:
> >>    - destroy()/destroyForcibly() terminate or kill the whole process group
> >>    - suspend()/resume() puts them to sleep and wakes them up.
> >>    More functionality could be added if needed. This mostly depends on
> >> how tightly we want to be bound by platform limitations on Windows,
> >> where process groups cannot be translated 1:1 to Job Objects.
> >>
> >> - ProcessBuilder has now two new attributes:
> >>    - createProcessGroup() is a boolean flag directing the builder to
> >> let sub processes create their own process group, with themselves
> >> being the leader.
> >>    - processGroup() is a reference to a ProcessGroup object; when not
> >> null, subprocesses will join that process group.
> >>
> >> - The Process class gets a new query method to retrieve a ProcessGroup
> >> object linked to its process group id.
> >>
> >> Using these building stones, a typical pattern could be:
> >>
> >> <example>
> >>          ProcessBuilder processBuilder = new ProcessBuilder(cmd);
> >>          processBuilder.createProcessGroup(true);  <-- next process is pg leader
> >>
> >>          Process leader = processBuilder.start();
> >>
> >>          ProcessGroup pgr = leader.processGroup();  <-- retrieve newly
> >> created process group
> >>          processBuilder.processGroup(pgr); <-- next processes shall be
> >> members of this process group too
> >>
> >>          processBuilder.start();
> >>          processBuilder.start();
> >>          ....
> >> </example>
> >>
> >> and then call operations on the ProcessGroup object.
> >>
> >> ----
> >>
> >> It is clear to me that this kind of change would require probably a
> >> JEP, if it is desired at all. With this mail I just wanted to gauge
> >> interest.
> >>
> >> What do you think?
> >>
> >> Thanks & Best Regards, Thomas