Thread Native ID Access

[John Rose]

On Feb 23, 2018, at 2:57 AM, Alan Bateman <Alan.Bateman at oracle.com> wrote:
> 
> things that are thread local today (ThreadLocal, thread context class loader,
> uncaught exception handler, ...) become fiber local without too much effort.

I think you mean "without much effort in the implementation".  But if this
road is taken without care, we add hidden thread-like costs to fibers,
which will make them scale disappointingly.

Specifically, I think we need to take extra care with thread-local data
that are variable in size.  By that I mean ThreadLocals in particular.

I think we are aiming to make a continuation switch cost on the order
of a few (out-of-line) method calls.  And a fiber switch should be not
much more, something like a lock handshake or two plus the
continuation switch.  And of course the scheduler overhead
(fiber selection) is another cost.  But we want to keep these small.

Likewise, the heap-size of a coroutine (and fiber) should be commensurate
with its stack frame "bulk" plus whatever local data it has.  Limiting this
data keeps switching overhead in line, since the amount of data to swap
in and out of cache (across fiber switches) is kept to the actual need.

With dinosaur threads, the switching overheads include swapping
kilobits of processor register file, plus visits to the OS, etc.  In that
regime, a few extra cache swaps to pull ThreadLocal data in and out
is not a big deal, but with fibers I think there is as serious risk that
legacy decisions (e.g. to put Thread-scale caches, like I/O buffers)
which make sense at Thread scale don't make sense at Fiber
scale.

For this reason, it's clearly not good to hack all the Thread
baggage onto fibers and then call it a day.  It may be kinder to
our developers to *not* provide automagic ThreadLocals for
fibers, and instead require that someone who created a TL
for some library must rethink the trade-offs for fibers.  They
are likely to be different, if we succeed in making fibers truly
different from threads.

Put another way, just replicating thread facilities blindly for
fibers will get us a predictable and unpleasant result:   We
will be re-implementing green threads in the JVM, and we
will have to meet every contract met by OS threads and
(poorly) by green threads.  This won't get us to a truly
higher scale levels; it's just turn us into amateur OS writers.

(Put yet another way, if there were a technique to make
fibers automagically and transparently implement all the
Thread APIs, the OS vendors would have given that to
us already in the standard thread libraries.)

Personally, I want to defend against us redoing Threads
badly by carefully questioning every burden we put on fibers.
It's not enough to say, "but Threads do it already".

An example of successfully resisting such burdens is
our carefully tuned stance toward native methods which
block or make callbacks.  Basically, we don't claim to
fiber-ize them, and we allow the fiber to entangle the
thread until it gets the native call out of its system.
If we were to be "clever" about allowing fiber dismount
across such pending calls, we'd be cleverly re-inventing
green threads—which wouldn't be so clever in the end.

Bottom line:  We must expect to make occasional
demands on users who want the scaling of fibers.
Arbitrary native calls is one compromise.  I think
TLs are likely to be another.  There are probably
more yet we haven't encountered.

I think it would be OK to have FiberContext and
ThreadContext objects which factor out the "big, heavy"
parts of Thread state, and which are shared by many
fibers and/or stuck on the dinosaur thread for multiplexed
use by mounted fibers.  This is a user model refactoring
which would reify (to the library programmer if not
the end user) the scaling trade-offs of using contextual
states and caches, without loading them unconditionally
onto the relatively light weight of fibers.

(Alan, I know you are already thinking about this stuff,
but it occurs to me that it's a good time to put the above
thoughts about Thread vs. Fiber on the record.  I've
noticed that people new to the project can slip easily
into the assumption that Fibers are just "Threads done
right", and sometimes we have to fend off various bright
ideas for reimplementing Threads "but this time better".)

Also for the record, it seems likely that we can make
java.lang.Thread a front-end for both fibers and OS
threads, and I also think we can cleverly minimize the
user model disturbances, but they won't stay completely
under the carpet, if we get (as we must) scale
improvements more like 1000x than 10x.

— John