[External] : Re: jstack, profilers and other tools

[Ron Pressler]

P.S.

Here’s something that might be of interest to those interested in the theory re inversion of control. The reason we know that synchronous and asynchronous code are “algorithmically equivalent”, i.e. able to express (using different syntax) the same algorithms is that it’s a special case of the continuation/monad equivalence proved in 1994 (https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.43.8213).

However, to make things more concrete (and hopefully, more widely understandable), consider that what delimited continuations do is turn a subroutine call into a subroutine return. In particular, in the JDK implementation, a call to Continuation.yield() becomes a return from the call to Continuation.run(), and the call to Continuation.run() turns into a return from the call to Continuation.yield(). So we can turn any decision as to which (callback) subroutine to call into a decision on which subroutine to return from.

The asynchronous programming style is a special case of what’s known as a continuation-passing style, or CPS, where we register a callback with an operation, and the operation then decides when to call the callback (“inversion of control”). With delimited continuations, and, with minor changes, threads, the registration of the callback becomes a call to yield, and the invocation of the callback becomes the *return* from yield.

Making this even more concrete, in the async style we have a set of registered callbacks which we then call at will, whereas in the synchronous style we have a set of blocked threads which we then unblock at will. Calls into a callback become returns from blocking calls.

There is one difference which I papered over, which is that threads aren’t delimited continuations, but are essentially delimited continuations with an associated scheduler, so where in the async or delimited continuation cases, the “framework” has control over scheduling, with threads, the thread scheduler has control over scheduling. In practice, that is a second-order concern for the intended use-case of threads, but we can close that gap with pluggable thread schedulers (that are being worked on but were not delivered in JEP 425).

— Ron

On 21 Jul 2022, at 12:42, Ron Pressler <ron.pressler at oracle.com<mailto:ron.pressler at oracle.com>> wrote:



On 20 Jul 2022, at 19:49, Alex Otenko <oleksandr.otenko at gmail.com<mailto:oleksandr.otenko at gmail.com>> wrote:

Waiting for requests doesn't count towards response time. Waiting for responses from downstream does, but it is doable with rather small thread counts - perhaps as witness to just how large a proportion of response time that typically is.

I am not talking about what’s *doable* but about what virtual-thread programs *do* (or can do), i.e. use a thread to wait for a response. As I made very clear, this can also be done using async APIs, which were invented for the very same reason (get around the maximum thread count limitation), but then you’re missing out on “harmony” with the Java platform, which knows about threads and not async constructs.


I agree virtual threads allow to build things differently. But it's not just the awkwardness of async API that changes. Blocking API eliminates inversion of control and changes allocation pattern.

I disagree, but no one is taking async APIs away. If you enjoy programming in that style, you’re more than welcome to continue doing so.


For example, first you allocate 16KB to read efficiently, then you are blocked - perhaps, for seconds while the user is having a think. So 10k connections pin 160MB waiting for a request. With async API nothing is pinned, because you don't allocate until you know who's read-ready. This may no longer be a problem with modern JVMs, but needs calling out.

That is incorrect. A synchronous API can allocate and return a buffer only once the number of available bytes is known. There is absolutely no difference between asynchronous and synchronous APIs in that respect. In fact, there can be no algorithmic difference between them, and they can both express the same algorithm. The difference between them is only how an algorithm is *expressed* in code, and in which constructs have built-in observability support in the runtime.


The lack of inversion of control can be seen as inability to control who reads when. So to speak, would you rather read 100 bytes of 10k requests each, or would you rather read 10KB of 100 requests each? This has impact on memory footprint and who can make progress. It makes sense to read more requests when you run out of things to do, but doesn't make sense if you saturated the bottleneck resource.

There is no difference between synchronous and asynchronous code here, either; they just express what it is that you want to do using different code. If you want to see that in the most direct way, consider that the decision of which callback to call and the decision on which thread waiting on a queue you want to unblock and return a message to is algorithmically the same.

There is nothing that you’re able to do in one of the styles and unable in the other. In terms of choice, there’s the subjective question of which style you prefer aesthetically, and the objective question of which style is more directly supported by the runtime and its tools.

— Ron

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/loom-dev/attachments/20220721/ce784788/attachment.htm>