threads as means to an end, rather than parts of a whole

[John Rose]

Here are some comments about some of the thread-related concepts we
are adapting as we virtualize threads.  As someone on the sidelines of
Loom, I will feel successful if they spark more ideas from a Loom
developer.

Threads group somehow, and this has something to do with (varying)
concepts like groups, pools, containers, services, owners, and scopes.
Also something to do with cancellation and completion.  Also something
to do with administration.  This “something” is clearly a cluster of
“somethings” which call for further investigation and description.
Ultimately, we’d like to represent those “somethings” in a
crystal-clear set of APIs, balanced and split and lumped to
perfection.  We are not there yet.  There are uncomfortable places
where existing concepts, which have been good enough for some dozens
of “dinosaur” threads, do not scale to virtual threads in their
millions.  (I’m thinking of ThreadGroup and ThreadPool, in
particular.)

Notice that the above metaphors largely appeal to an intuition of
*placement*: A thread is *placed* in a group or pool or container, it
stands ready in a service (or is produced by one), it is in the pocket
(or other container) of its owner, and (with SC) it spends its
lifetime *within* some program block (scope).  That’s very fruitful
because our intuitions of placement and ownership are very powerful
“built in” habits of thought.

But, let me suggest another “something” in a different direction,
using another metaphor, as a thought experiment to shake us free from
our intuitions of containment and ownership.  Let’s say, just for a
moment, that *every thread serves a purpose*.  So the metaphor is no
longer of parts-and-wholes but rather means-and-ends.  There is an
isomorphism here with the other metaphors: means : end :: whole :
part.  These are both N-1 relations, and they are both relatively
static.  By that I mean that a thread is created and destroyed in one
place and/or for one purpose.  (There are use cases for changing the
relation over time, occasionally: Moving a thread from one container
to another, or changing its purpose from one end to a different end.
But these are advanced use cases.)

What does this perspective buy?  Well, compared to containment,
purpose is asymmetrically oriented towards the future.  The purpose of
something has nothing to do with *where it comes from*; it is all
about *where it’s going*.  (Students of philosophy might recognize a
certain habit of thought here, the “final cause”.)

So what is the “end” of a thread?  It is whatever that thread is going
to do in the *future*, with special emphasis on *its final acts*.  In
the future, it will continue to consume resources and contribute to
results.  Its final act will be either to be canceled (a “passive
act”) or to deliver some final completion, a result (perhaps extending
earlier results already delivered).  This focus on *ends and means*
seems worth exploring as an alternative to *parts and wholes*, because
it puts the emphasis on the most interesting part of a thread (viewed
from the outside), which is what it promises (or fails) to deliver.

Specifically, in the case of a thread pool, the metaphors yield
different answers.  Obviously, a worker thread is *always* in the same
pool, but just as obviously its purposes are deliberately changed over
time.  Over its lifetime its purpose is multiplexed.  While it is
idle, its purpose is to be ready to take on another task; while it is
running, its sole purpose is that task.  When that purpose is ended,
something funny happens: The thread doesn’t die; it is repurposed to
another task (or the idle queue).  The purpose of the task, per se, is
more clearly the purpose of a VT that runs the task, since the VT is
the means to the end of the task.

If we place threads and purposes as means and ends in a 1-1 relation,
we just get, literally, promises: A promise API which allows you to
get a completion value or notice of failure, maybe to cancel the
computation.  (One aspect of purposes is a sort of back pressure: If a
purpose is no longer useful, no longer itself has some larger purpose,
then it should be easy to abandon.  That’s cancellation, I think.  But
there might be more subtle and temporary forms of back pressure that
could apply here also.)

If we place threads and purposes as means and ends in an N-1 relation,
then we get (isomorphically to the part/whole metaphors), something
more like “projects”.  Here’s how it might go: A project has workers.
Workers may finish or fail; they may be retained or dismissed.  They
can be told to slow down, speed up, or stop in their current work.
Workers may be delegated to or inherited from other projects.  But
while a worker is on a project, it is clear that the worker has one
purpose (as a worker), to push forward some designated part of the
project.

In that metaphor, for example, a thread pool is like a temp agency,
supplying workers to whoever pays for them.  A block scope (in SC) is
like an office full of workers, working on one project.  A tree of
block scopes is like a larger organization.  An unstructured pool of
threads (reached via a global static final field) is, maybe, another
temp agency, or just that street corner where someone can hire day
laborers.  One could go on a long time in this vein.

Does this alternative metaphor help us factor thread APIs differently?
I don’t claim to know, but it seems helpfully different from the
existing metaphors.  What would an API framed in this metaphor look
like?  The name ThreadPurpose sounds silly, so I’ll switch to
“endpoint” or “destination” (verging back to spatial metaphors, but a
dynamic one).

So here’s a very rough and incomplete cut to show some possibilities:

interface /*Thread*/Destination<W /*extends Thread*/> { // joins void
   waitFor(W worker); void waitForAll(); // cancellation void cancel(W
   worker); void cancelAll(); // delivery of results (executed inside
   W, perhaps?)  void notifyCompleted(W worker, Object answer); void
   notifyCompletedExceptionally(W worker, Throwable exception); //
   advanced: transfers and delegation (probably needs a broker API)
   void transferOut(W worker, Destination<? super W> newDestination);
   void transferOutAll(Destination<? super W> newDestination); }

// debugging, reflection, internal management class
jdk.internal.DestinationUtils { // classification and discovery,
mainly for debugging: static boolean isWorker(Destination<?> dest,
Object possibleWorker); static Stream<Object>
findWorkers(Destination<?> dest, float howHardToSearch); static
findWorkers(Destination<?> dest, float howHardToSearch); // structure
(in case this is not a final destination, but a means to a larger end)
static Destination<?> nextDestination(Destination<?> dest); }

Reminder: I’m not a Loom developer or even a thread expert.  I’d be
shocked to see the above API actually built.

But I am a pretty good metaphor-monger, and so I hope that maybe the
above ideas will help other folks clarify what’s really going on with
the various form of thread grouping.

— John

P.S. I didn’t talk about confinement above, although “confinement” is
(surprise, surprise) yet another metaphor of placement in a container.
For me, the nub of the idea of confinement is to limit the number of
times and places where a proposed operation is allowed to take place.
Specifically, an operation like reading or writing a memory location,
or canceling or joining a worker thread, is safe and sane in some
times and places and unsafe and insane in others.  By defining some
time interval (which means also placement in one or more threads, in
the JMM), we might be able to (a) prove that some set of proposed
operations happen in that interval, and thus (b) that the operations
will be safe and sane.  (By which I mean, there are no dangling
pointers or race conditions or dropped results or undefined behavior;
in different applications of the idea you aim at different versions of
“safe and sane”.)

If a thread is confined to one “purpose” or destination, it might be
reasonable to say either or both of the following:

I. No other destination will control that thread’s “end”, either
receiving completions, or joining, or canceling.  As an exception to
that rule, if transfers are allowed, then transferring a worker W from
D1 to D2 means that, after the transfer is happened, D1 no longer can
control W, but instead D2 can (where it couldn’t before).

II. The operations pertaining to the “end” of each thread must be
performed within a particular and limited set of Java threads.
Compared to I above we might call this “thread-confinement”; it is a
lower-level property.  I suppose notifyCompleted is best
thread-confined to the thread W itself, which is doing the reporting.
But the control operations (join, cancel) are usefully thread-confined
to some other thread (or cooperating group of threads), such as a
control thread which is managing the whole group of workers, for some
coherent purpose.

If the control operations of some destination are thread-confined,
then they are probably also confined in Sense I above, if the
respective control logics for D1 and D2 are running in different
threads.