ScopedValue structured forking/forwarding API

[Nikita Bobko]

TLDR: The email reasons on the topic of ScopedValue structured forking
API, which is a way to forward ScopedValues to child tasks, even if
users use structured concurrency API different from
StructuredTaskScope

Lately, I've been experimenting with ScopedValue API to see how we
could integrate with it in kotlinx.coroutines library
https://github.com/Kotlin/kotlinx.coroutines

Currently, ScopedValue and StructuredTaskScope work nicely together.
Since both are structured, ScopedValues are forwarded to all
StructuredTaskScope children tasks.

    public class ScopedValue_and_StructuredTaskScope {
        public static ScopedValue<String> userName =
                ScopedValue.newInstance();

        public static void main() throws InterruptedException {
            ScopedValue.where(userName, "Duke").call(() -> {
                try (var scope = StructuredTaskScope.open()) {
                    scope.fork(() ->
                            // Prints "Duke"
                            System.out.println(userName.get()));
                    scope.join();
                }
                return null;
            });
        }
    }

And as we know, ScopedValues are not forwarded to manually created
threads/thread-pools since they are generally
unconstrained/unstructured.

    public class ScopedValue_and_Executors {
        public static ScopedValue<String> userName =
                ScopedValue.newInstance();

        public static void main() throws Exception {
            ScopedValue.where(userName, "Duke").call(() -> {
                try (var pool = Executors.newFixedThreadPool(1)) {
                    pool.submit(() -> {
                                // Throws
                                //
                                // java.util.NoSuchElementException:
                                //     ScopedValue not bound
                                System.out.println(userName.get());
                            })
                            .get();
                }
                return null;
            });
        }
    }

>From what I read in the source code, ScopedValues and
StructuredTaskScope are integrated via internal APIs – all scoped
values are captured by
jdk.internal.vm.ScopedValueContainer.captureBindings() API which is
called inside the jdk.internal.misc.ThreadFlock constructor.

Unfortunately, external libraries can't use this API, and it's clear
why, I am not asking to open these APIs up, as it would actively work
against the ScopedValues goals.

But I have come with an idea of the following "structured snapshot"
API which is symmetrical to StructuredTaskScope API:

    public class Main {
        public static ScopedValue<String> userName =
                ScopedValue.newInstance();

        public static void main(String[] args) {
            ScopedValue.where(userName, "Duke").call(() -> {
                try (ScopedValue.Snapshot bindings =
                         ScopedValue.snapshot()) {
                    try (var pool = Executors.newFixedThreadPool(1)) {
                        pool.submit(() -> {
                            bindings.forward(() -> {
                                // Should prints "Duke"
                                System.out.println(userName.get());
                            });
                            // Throws
                            //
                            // java.util.NoSuchElementException:
                            //     ScopedValue not bound
                            System.out.println(userName.get());
                        }).get();
                    }
                    bindings.join();
                }
                return null;
            });
        }
    }

ScopedValue.Snapshot is an AutoCloseable object that internally
captures all the bindings from the current Thread. Next, users can
forward all the captured values to child threads by calling
ScopedValue.Snapshot.forward inside the child thread. (All names are
WIP)

ScopedValue.Snapshot would have to maintain the integrity and make
sure that the values are indeed forwarded structurally. Basically,
most of the constraints can be taken from StructuredTaskScope:

- Once ScopedValue.Snapshot.close is called, all subsequent calls to
  ScopedValue.Snapshot.forward would throw an Exception.
- If there are still ongoing ScopedValue.Snapshot.forward calls that
  didn't finish yet, ScopedValue.Snapshot.close would block to wait
  for all of them and then throw an exception requiring an explicit
  ScopedValue.Snapshot.join call (Similar to how
  StructuredTaskScope.close works)
- ScopedValue.Snapshot.join would block and wait for all
  ScopedValue.Snapshot.forward calls to finish
- ScopedValue.Snapshot.join can be called only on the same thread
  where the respective ScopedValue.snapshot was called.

But we have to add 2 more constraints which are unique to
ScopedValue.Snapshot API:

- ScopedValue.Snapshot.forward should check that there are currently
  zero scoped value bindings in the current thread.
- ScopedValue.Snapshot.forward cannot be called on the same thread
  where the snapshot was taken.

The motivation for the first restriction is to make sure that
ScopedValue.Snapshot API couldn't be used to unpredictable override
*all* the current scoped value bindings.

The second restriction is to just forbid ScopedValue.Snapshot.forward
calls that don't make sense.

The 2 restrictions that I'm describing are emerged after reading this
previous thread on the topic of Snapshots
https://mail.openjdk.org/pipermail/loom-dev/2023-May/005664.html
https://mail.openjdk.org/pipermail/loom-dev/2023-June/005699.html

The thread is quite lengthy, but I think the conversation boils down
to the following messages:
- https://mail.openjdk.org/pipermail/loom-dev/2023-June/005722.html
- https://mail.openjdk.org/pipermail/loom-dev/2023-June/005732.html
- https://mail.openjdk.org/pipermail/loom-dev/2023-June/005728.html

which basically says that unpredicatably overriding *all* scoped
value bindings is an anti-pattern.

## Use cases

*Use case 1.* The API would be useful for any third-party libraries
that implement structured concurrency. kotlinx.coroutines is an
example of such a library.

If you are wondering why can't kotlinx.coroutines just use
StructuredTaskScope under the hood. The answer is: because
kotlinx.coroutines allows configuring which threads to use to
schedule coroutines on. It's a common scenario for users to schedule
their coroutines on the main UI thread for example.

*Use case 2.* StructuredTaskScope itself can use this public API to
integrate with ScopedValues.

## Discussion

Unfortunately, the email ended up being bigger than I wanted it to be,
but hopefully it makes sense.

I would love to hear your thoughts on the idea. I am sure that you
have already discussed it in some form internally, but hopefully this
email brings more use cases.

I realize that ScopedValues are scheduled for release in Java 25,
thankfully, the idea here is something that can be released
independently in later versions as an addendum. (shall you decide that
the idea is even a valid use case)

--
Nikita Bobko
Engineer on the Kotlin Team at JetBrains