<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<p>I think that's what I was referring to w.r.t. "scope of pinning".</p>
<p>We have investigated doing things like:</p>
<p>```<br>
MemorySegment heapBytes = MemorySegment.ofArray(new byte[...]);<br>
try (Arena arena = ...) {<br>
MemorySegment pinnedBytes = heapBytes.pin(arena);<br>
...<br>
} // unpinned here<br>
```</p>
<p>This works well API-wise (and it gives you all the lifetime
management options you'd come to expect when interaging with a
segment), but this could easily lead to Java programs blocking GC
indefinitively, which is a risk that has to be factored in. Yes,
we could make MS::pin a restricted method, (I see that C# went
down the unsafe path too), but the question is whether that's
enough of a deterrent, given how much accidental damage you one
cause (w/o realizing).<br>
</p>
<p>Maurizio <br>
</p>
<div class="moz-cite-prefix">On 28/08/2023 15:09, Gavin Ray wrote:<br>
</div>
<blockquote type="cite" cite="mid:CAFtvWZMFDWsWk-g+kqgj4meV3AoNMU6BEVcihcEPoRy_+3+yCA@mail.gmail.com">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">For whatever it's worth, I think that C# does
pinning fairly well.
<div>It uses a "fixed" statement around a variable which is
similar to a try-with-resources in Java.</div>
<div><br>
</div>
<div>Without introducing a new keyword this could probably
be done as a Functional Interface that takes a lamba or
something.</div>
<div><br>
</div>
<div>> withPinned(heapSegment, segment -> {});</div>
<div><br>
</div>
<div>- <a href="https://urldefense.com/v3/__https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/statements/fixed__;!!ACWV5N9M2RV99hQ!IGKbapxVG_QMwdL236-U-1XDKQLgz1imop4dEpsa2CqyUkuhuLs7lk5LYrCbzu3nH6uT0YTrJVKY329M7mInwa_nqzLD$" moz-do-not-send="true">fixed statement - pin a moveable
variable | Microsoft Learn</a></div>
<div>- <a href="https://urldefense.com/v3/__https://learn.microsoft.com/en-us/dotnet/framework/interop/copying-and-pinning__;!!ACWV5N9M2RV99hQ!IGKbapxVG_QMwdL236-U-1XDKQLgz1imop4dEpsa2CqyUkuhuLs7lk5LYrCbzu3nH6uT0YTrJVKY329M7mInweV3GGr6$" moz-do-not-send="true">Copying and Pinning - .NET
Framework | Microsoft Learn</a></div>
</div>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, Aug 28, 2023 at
9:04 AM Yasumasa Suenaga <<a href="mailto:suenaga@oss.nttdata.com" moz-do-not-send="true" class="moz-txt-link-freetext">suenaga@oss.nttdata.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi
Maurizio,<br>
<br>
Thank you for knowing ffmasm :)<br>
<br>
MS::pin/unpin in my proposal are same semantics with <br>
GetPrimitiveArrayCritical/ReleasePrimitiveArrayCritical in
JNI. They <br>
might not be leveraged a lot of developers, but they help a
part of <br>
experienced developers to improve their application
performance. In JNI, <br>
we call GetPrimitiveArrayCritical() to access large arrays
without <br>
memory copy cost. It makes sence. I just want to do same
operation in <br>
FFM.<br>
<br>
Let's think about image processing. When we want to binarize
JPEG image, <br>
we would load original image with ImageIO API. Then we can get
pixels <br>
from BufferedImage::getRGB() as int[]. So we have to copy
pixels into <br>
off-heap MemorySegment if we want to perform operations in
native (e.g. <br>
SIMD, GPU processing). I'm sure most of Java API / third party
libraries <br>
use primitive array, not MemorySegment. So I think it is
better if we <br>
pin on-heap memory and leverage it in native.<br>
<br>
Of course I understand it might be some worse behaviors
especially GC, <br>
so pinning might not be recommended for all of Java
developers. But I <br>
believe pinning is welcomed from developers who experienced in
native <br>
(C/C++/assembly/GPGPU and so on) because they want to offload
processing <br>
which needs large memory (preprocessed in Java).<br>
<br>
<br>
Thanks,<br>
<br>
Yasumasa<br>
<br>
<br>
2023-08-28 18:48 に Maurizio Cimadamore さんは書きました:<br>
> On 28/08/2023 08:29, Yasumasa Suenaga wrote:<br>
>> Hi all,<br>
>> <br>
>> I'm very interested in FFM, especially generate
assembly code in Java <br>
>> and calling them with FFM [1].<br>
> Hi,<br>
> I've seen your work a year or so ago and have been very
impressed by it <br>
> :-)<br>
>> <br>
>> I think one of the performance neck is MemorySegment
because all of <br>
>> on-heap regions should be copied into off-heap region
(native segment) <br>
>> when they are referred from foreign function. So I'm
expecting to <br>
>> implement sort of pinning operation for on-heap
MemorySegment like a <br>
>> JNI. I guess it is mentioned in FFM update in last
month [2], however <br>
>> "Pinning of heap segments" does not have any links -
I guess nobody is <br>
>> working for this yet. Do you have any updates for
pinning?<br>
>> <br>
>> I've played them with both OpenJDK 22 with pinning
support [3] and <br>
>> ffmasm (hand-assembler for Java powered by FFM) [4].
I added pin/unpin <br>
>> method into Unsafe, and they are called by
HeapMemorySegmentImpl. <br>
>> Finally I got about 16x performance gain compared to
non-pinning code <br>
>> [5] on my laptop.<br>
>> <br>
>> I guess FFM API reaches to goal step by step, but it
is still a <br>
>> preview in JDK 21. I hope that pinning feature is
supported into JDK <br>
>> 22 because I believe we can leverage FFM for more
fields! I'm happy to <br>
>> contribute/help to implement pinning feature if it
needs.<br>
> <br>
> While there's no doubt that in some applications and use
cases pinning<br>
> provides a significant performance boost, there are some
challenges:<br>
> <br>
> 1. scope of pinning: is pinning allowed on a
per-native-call basis? Or<br>
> is it something more general?<br>
> 2. does the garbage collection support region-based
pinning [1] ?<br>
> 3. why is pinning needed in the first place?<br>
> <br>
> (1) and (2) are very much linked. Not all garbage
collectors support<br>
> fine-grained pinning mechanism. Which means that, in most
of them, if<br>
> you pin, you effectively block GC for the entire duration
of the pin<br>
> operation (GC locker mechanism). This is something that,
as I'm sure<br>
> you understand, is not very desirable. For this reason,
it might be<br>
> better to consider a pinning API which only pins for the
duration of a<br>
> native call (e.g. in the shape of an additional linker
option). While<br>
> FFM could support more complex pinning policies (e.g. pin
a segment<br>
> inside an Arena, so that segment is unpinned when the
arena is<br>
> closed), given the uneven support for fine-grained
pinning across GCs,<br>
> I'm not sure such a general API (which is similar to your
"MS::pin"<br>
> method) would be a good idea. We're doing some
experiments for adding<br>
> a new linker option which allows for pin heap segments to
be pinned<br>
> when calling a downcall method handle, we're not yet sure
of its<br>
> inclusion, but it would be something worth publishing
somewhere (when<br>
> ready) so that developers (like you) can play with it and
provide<br>
> feedback.<br>
> <br>
> Then there's (3). Most of the times, pinning is used in
order to<br>
> interact with native calls from public-facing APIs that
are "stuck"<br>
> using array syntax. That is, in order to be user
friendly, such API<br>
> work with arrays - but then a problem arises when trying
to use the<br>
> contents of the array off-heap. But what if the memory
was off-heap to<br>
> begin with? Then no memory copy would be required. I
believe the<br>
> biggest impediment for off-heap memory being used
directly has to do<br>
> with the fact that, for users, interacting with an
`int[]` is<br>
> significantly easier than interacting with a
`MemorySegment`, or a<br>
> `ByteBuffer`. But what if we could provide some mechanism
to create an<br>
> "array view" over an off-heap memory region? Now clients
would be able<br>
> to use the beloved `[]` syntax, even if memory access
remained<br>
> off-heap.<br>
> <br>
> While we don't have any concrete proposal on this latter
point, we do<br>
> believe that the topic of making memory segments (or byte
buffer)<br>
> easier to access for "legacy clients" is inextricably
linked to the<br>
> topic of pinning of heap memory.<br>
> <br>
> [1] - <a href="https://openjdk.org/jeps/423" rel="noreferrer" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">https://openjdk.org/jeps/423</a><br>
> <br>
>> <br>
>> <br>
>> Thanks,<br>
>> <br>
>> Yasumasa<br>
>> <br>
>> <br>
>> [1] <a href="https://urldefense.com/v3/__https://github.com/YaSuenag/ffmasm__;!!ACWV5N9M2RV99hQ!IGKbapxVG_QMwdL236-U-1XDKQLgz1imop4dEpsa2CqyUkuhuLs7lk5LYrCbzu3nH6uT0YTrJVKY329M7mInwXvCTRxW$" rel="noreferrer" target="_blank" moz-do-not-send="true">https://github.com/YaSuenag/ffmasm</a><br>
>> [2] <br>
>> <a href="https://mail.openjdk.org/pipermail/panama-dev/2023-July/019510.html" rel="noreferrer" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">https://mail.openjdk.org/pipermail/panama-dev/2023-July/019510.html</a><br>
>> [3] <br>
>> <a href="https://urldefense.com/v3/__https://github.com/YaSuenag/jdk/commit/f0a9b3705b3ecdf3dbb6b80cac9d53456f08f967__;!!ACWV5N9M2RV99hQ!IGKbapxVG_QMwdL236-U-1XDKQLgz1imop4dEpsa2CqyUkuhuLs7lk5LYrCbzu3nH6uT0YTrJVKY329M7mInwbe47VCY$" rel="noreferrer" target="_blank" moz-do-not-send="true">https://github.com/YaSuenag/jdk/commit/f0a9b3705b3ecdf3dbb6b80cac9d53456f08f967</a><br>
>> [4] <br>
>> <a href="https://urldefense.com/v3/__https://github.com/YaSuenag/ffmasm/commit/925608538b936db1b311ae84e12fa0252058b7f4__;!!ACWV5N9M2RV99hQ!IGKbapxVG_QMwdL236-U-1XDKQLgz1imop4dEpsa2CqyUkuhuLs7lk5LYrCbzu3nH6uT0YTrJVKY329M7mInwbUJ6qi5$" rel="noreferrer" target="_blank" moz-do-not-send="true">https://github.com/YaSuenag/ffmasm/commit/925608538b936db1b311ae84e12fa0252058b7f4</a><br>
>> [5] <br>
>> <a href="https://urldefense.com/v3/__https://github.com/YaSuenag/ffmasm/blob/ffm-pinning/benchmarks/vectorapi/src/main/java/com/yasuenag/ffmasm/benchmark/vectorapi/VectorOpComparison.java__;!!ACWV5N9M2RV99hQ!IGKbapxVG_QMwdL236-U-1XDKQLgz1imop4dEpsa2CqyUkuhuLs7lk5LYrCbzu3nH6uT0YTrJVKY329M7mInwRLDJMg-$" rel="noreferrer" target="_blank" moz-do-not-send="true">https://github.com/YaSuenag/ffmasm/blob/ffm-pinning/benchmarks/vectorapi/src/main/java/com/yasuenag/ffmasm/benchmark/vectorapi/VectorOpComparison.java</a><br>
</blockquote>
</div>
</blockquote>
</body>
</html>