<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<p>Extent-local memory can be shared across multple virtual threads
created by the same "scope" (e.g. a StructuredTaskScope).</p>
<p>Depending on how an application is structured, this might be
useful or not. Working with extent-local memory imposes
"structure" on your application - that is, a thread creates an
extent (typically with a try-with-resources) which is then shared
by all the threads that are forked within that extent (e.g. inside
the try-with-resources). Only the original thread can close the
extent, and it can only do so when all the "nested" computation
has completed.</p>
<p>So, if your program has some kind of structure, extents help you
manage that structure, and give you guarantees in return (which we
can use to implement more efficient memory sessions). But if your
program is non-structured, using extents will probably feel like
fitting a round peg in a square hole.</p>
<p>To bring up a C/C++ analogy, if a program does a lot of stack
allocation, then it's likely that using extent-local memory would
be a good choice: an extent-local memory session is very similar,
semantically (probably even better) to a RAII block in C++. But if
your program does a lot of "loose" malloc/free (maybe relying on
some other implicit invariants to enforce correctness),
extent-local memory is probably not going to help.</p>
<p>So, whether extent-local memory helps or not will depend a lot,
in practice, on how the framework implementation works.</p>
<p>Maurizio<br>
</p>
<p>On 01/10/2022 21:49, Gavin Ray wrote:</p>
<blockquote type="cite" cite="mid:CAFtvWZMsghQHbt2WuFYan0fA9i6rTgT_nn8K8-5iChon=pR00A@mail.gmail.com">
<div dir="auto">I was reading this guide today and noticed this
section:
<div dir="auto"><br>
</div>
<div dir="auto"><a href="https://urldefense.com/v3/__https://quarkus.io/guides/virtual-threads*the-netty-problem__;Iw!!ACWV5N9M2RV99hQ!LWNwgj4tyfm1u8uYlVxh5eVSrcZC2FC8Jrp6zKlGUKctiqOjyay9mhi2_WbT9yzgkiqhkYQLuTo1o4cnTu_B0pSsDLkq$" moz-do-not-send="true">https://quarkus.io/guides/virtual-threads#the-netty-problem</a></div>
<div dir="auto"><br>
</div>
<div dir="auto">Wouldn't extent-local Memory solve this?</div>
<div dir="auto">-----</div>
<div dir="auto"><br>
</div>
<div dir="auto">"For JSON serialization, Netty uses their custom
implementation of thread locals, FastThreadLocal to store
buffers. When using virtual threads in quarkus, then number of
virtual threads simultaneously living in the service is
directly related to the incoming traffic. It is possible to
get hundreds of thousands, if not millions, of them.</div>
<div dir="auto"><br>
</div>
<div dir="auto">If they need to serialize some data to JSON they
will end up creating as many instances of FastThreadLocal,
resulting on a massive memory consumption as well as
exacerbated pressure on the garbage collector. This will
eventually affect the performance of the application and
inhibit its scalability."</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, Sep 26, 2022, 1:55 AM
Maurizio Cimadamore <<a href="mailto:maurizio.cimadamore@oracle.com" moz-do-not-send="true" class="moz-txt-link-freetext">maurizio.cimadamore@oracle.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<p>Hi Gavin,<br>
I think such a memory session would work as follows:</p>
<p>* it will create a _fresh_ extent local on creation, and
bind it<br>
* when accessing memory, it will check whether the scope
local is bound in the thread (meaning it's one of the
thread that inherited it, e.g. via StructuredTaskScope)<br>
* when closing, it will try to unbind the extent local,
and if a structure violation arises (e.g. there are
threads still running which inherited the extent local),
an exception is thrown.</p>
<p>Last time I checked, this did not require a lot of code,
and could be done entirely in Java (although using a few
non exported functionalities of extent locals).</p>
<p>I think the trick is that, by leaning hard on extent
locals and their inheritance across threads, things will
"just work".</p>
<p>The main unknown is how much does it take to check that a
scope local has been inherited; I was afraid the cost of
this was going to be prohibitive in a use case such as
memory access, but from some quick testing I did at the
time (which I did when extent local was called
ScopeLocal), it all seemed to unroll, hoist and inline
quite nicely (probably thanks to the C2 optimizations
baked in for extent locals). Of course we will have to
retest again as the implementation for extent locals
becomes more mature.<br>
</p>
<p>Maurizio<br>
</p>
<div>On 23/09/2022 21:26, Gavin Ray wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">Maurizio,
<div><br>
</div>
<div>Going through my inbox after sending the
last message, somehow I missed this email.</div>
<div><br>
</div>
<div>Thanks a ton for such a comprehensive answer</div>
<div>(I ought to start collecting these in a scrapbook
for good measure!)</div>
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px
0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex"><i>Finally, it is
possible that we will introduce a new kind of
memory<br>
session that is confined not to a single thread,
but to an _extent_<br>
instead. This means that all threads created in a
single extent will be<br>
able to access a given memory segment, but threads
outside that extent<br>
will not be able to do so.</i></blockquote>
<div><br>
</div>
<div>This sounds like it would be fantastic,
especially if you can piggyback off of</div>
<div>work & guarantees already provided by the
Loom scheduler/executors.</div>
<div><br>
</div>
<div>It feels like there are a _lot_ of usecases where
such "extent-local" memory sharing would be
beneficial,</div>
<div>maybe it even unlocks some uses that the JVM
wasn't viable for before -- who knows?</div>
<div><br>
</div>
<div>This would be an integration with the compiler,
and not just a Java library-side feature if I
understand it correctly?</div>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, Sep 12, 2022
at 8:57 AM Maurizio Cimadamore <<a href="mailto:maurizio.cimadamore@oracle.com" target="_blank" rel="noreferrer" moz-do-not-send="true" class="moz-txt-link-freetext">maurizio.cimadamore@oracle.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px
0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">Hi Gavin,<br>
whether you can access a memory segment from multiple
threads or not <br>
generally depends on the memory session attached to
the segment. If you <br>
create a segment with a "shared" memory session, then
the resulting <br>
segment will be accessible in a "racy" way from
multiple threads. If the <br>
session associated with the segment is confined, only
one thread can <br>
access. When accessing memory in a segment that is
shared, some <br>
synchronization has to occur between the accessing
threads to make sure <br>
they don't step on each other toes. With Panama, if
you have VarHandle <br>
for memory access, you have a big set of memory access
operations at <br>
your disposal, dependning on the level of
synchronization required (e.g. <br>
plain access, acquire/relese access, volatile access,
atomic access).<br>
<br>
Then there's Loom. From Panama perspective, a virtual
thread is just a <br>
thread, so if you want to grant a memory segment
access from multiple <br>
virtual threads you need a shared session.<br>
<br>
Then there's List itself. Reading and writing on that
list concurrently <br>
from multiple thread can itself lead to issues (e.g.
missing updates). <br>
So, in general, when accessing data structures from
multiple threads, <br>
you either need a data structure that is concurrent by
design (e.g. <br>
concurrent hash map, or blocking queue, etc.). Or you
need to roll in <br>
your synchronization code. What the right answer is
often depends on the <br>
nature of your application.<br>
<br>
Finally, it is possible that we will introduce a new
kind of memory <br>
session that is confined not to a single thread, but
to an _extent_ <br>
instead. This means that all threads created in a
single extent will be <br>
able to access a given memory segment, but threads
outside that extent <br>
will not be able to do so. This would be a good
addition when working <br>
with virtual threads, because in the case of virtual
threads some <br>
additional bookkeeping is set up by the JDK runtime so
that, e.g. when <br>
using a StructuredTaskScope, it is not possible to
close the task scope <br>
before all the threads forked by that scope have
completed. This gives <br>
the memory session API a very nice semantics for its
close() operation, <br>
that we'd like to take advantage of at some point.<br>
<br>
Thanks<br>
Maurizio<br>
<br>
On 10/09/2022 16:11, Gavin Ray wrote:<br>
> Reading through the docs for the Extent-Local
preview, I was trying to <br>
> understand whether<br>
> this would be usable for sharing a buffer/memory
pool across virtual <br>
> threads?<br>
><br>
> Suppose you have some class:<br>
><br>
> class BufferPool {<br>
> private List<MemorySegment> buffers;<br>
> }<br>
><br>
> The document says that the data must be immutable<br>
> But there is "interior" immutability, and
"surface" immutability<br>
><br>
> If multiple virtual threads shared the memory and
some potentially<br>
> perform write operations on the
MemorySegment's inside of the list<br>
> would that be valid behavior?<br>
><br>
> Or does this even make sense to do?<br>
> (Concurrency/parallelism are probably the things
I know the least <br>
> about in software)<br>
><br>
><br>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</body>
</html>