Thread Locals (was Re: State of Loom)

[Alan Bateman]

On 27/05/2020 19:54, Brian Goetz wrote:
> There are two things being conflated regarding this use of TLs:
>
>  - What is the right mechanism (hint: TL is probably not it);
>  - Who should provide that mechanism
>
> I think Doug's main point here is that, if the JDK doesn't provide a 
> better way, people will continue using TL out of laziness, with bad 
> results.
>
> That's true, but I think we should focus on "what is the right 
> mechanism" first.
One thing to add is that we expect there will be migration paths for 
many existing usages of TL. May not be all cases initially but I think 
progress can be made.

One of the common usages of TLs is caching of objects that are expensive 
to create but are not thread safe. Many sightings of 
ThreadLocal<SimpleDateFormat> in this bucket. The migration path is 
DateTimeFormatter which can be created with a pattern to produce a 
immutable object that can be stored in a static final. We've already 
changed several cases in the loom repo to use DTF. Rémi brings up 
StringCoder which is one of several cases in the JDK where charset 
encoders/decoders are cached in TLs, same bucket. Some of this caching 
can go away, esp. with charsets that don't require an expensive lookup 
so they are cheap to create - ongoing work here and currently none of 
these cases are creating TLs when used in the context of virtual threads.

Using a TL as a flag to detect reentrancy. We have a couple of these in 
the JDK in places that lookup service providers where there is risk of 
recursive initialization. We've replaced these in the loom repo with a 
simple gate mechanism that just tracks the threads doing the 
initialization. Not a big bucket but there are ways to do this that 
don't need TLs or scope variables.

We've encountered several cases in enterprise libraries that use TLs to 
make available context to transient callees, some it due to callbacks. 
These libraries could have used a parameter to pass context around but 
that complicates some of the other code in the sandwich that doesn't 
know anything about the scope that it is being used in. Several of the 
cases that we've looked at support nested usages that translate to a 
stack of contexts (save old context, set TL to new context, do some 
action, restore TL to old context). A possible migration for these 
libraries to move to using explicit parameters of course.  Another is to 
treat them as candidates for the scope variables that is still under 
exploration and described in SoL part 2.

The JDK uses TLs to cache native resources in a few places. These 
resources are cached by carrier threads, not virtual threads, as it 
wouldn't scale. This only works because the code does not block and 
promises to not keep a reference to something on the carrier thread. If 
we do forced preemption then it require making use of the critical 
section support to temporarily pin the carrier thread. There may be a 
good case to replace these buffer/other caches with something better. 
That would at least help with the calls to expose something that is 
inheriting unsafe (too many potential reliability and security issues if 
you get it wrong).

I read the mails from Douglas as a plea for something to help creating a 
cache of mutable objects where it's critical to avoid contention. I read 
it that TLs are being used today as an approximation for CPU locals 
because there isn't anything better. This was the purposes of the 
processorid branch. It has a prototype that exposes a primitive to 
obtain the current CPU ID. It based on the Linux restartable sequences 
mechanism. There's a PoC of LongAdder in that branch but a lot more work 
is needed to see how it might be used in other advanced contexts.  So 
"approximation for CPU locals" is another set of cases that need 
attention, it's just hasn't been high priority of late.

-Alan