Pinning of on-heap MemorySegment

Mon Aug 28 14:09:23 UTC 2023

For whatever it's worth, I think that C# does pinning fairly well.
It uses a "fixed" statement around a variable which is similar to a
try-with-resources in Java.

Without introducing a new keyword this could probably be done as a
Functional Interface that takes a lamba or something.

> withPinned(heapSegment, segment -> {});

- fixed statement - pin a moveable variable | Microsoft Learn
<https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/statements/fixed>
- Copying and Pinning - .NET Framework | Microsoft Learn
<https://learn.microsoft.com/en-us/dotnet/framework/interop/copying-and-pinning>

On Mon, Aug 28, 2023 at 9:04 AM Yasumasa Suenaga <suenaga at oss.nttdata.com>
wrote:

> Hi Maurizio,
>
> Thank you for knowing ffmasm :)
>
> MS::pin/unpin in my proposal are same semantics with
> GetPrimitiveArrayCritical/ReleasePrimitiveArrayCritical in JNI. They
> might not be leveraged a lot of developers, but they help a part of
> experienced developers to improve their application performance. In JNI,
> we call GetPrimitiveArrayCritical() to access large arrays without
> memory copy cost. It makes sence. I just want to do same operation in
> FFM.
>
> Let's think about image processing. When we want to binarize JPEG image,
> we would load original image with ImageIO API. Then we can get pixels
> from BufferedImage::getRGB() as int[]. So we have to copy pixels into
> off-heap MemorySegment if we want to perform operations in native (e.g.
> SIMD, GPU processing). I'm sure most of Java API / third party libraries
> use primitive array, not MemorySegment. So I think it is better if we
> pin on-heap memory and leverage it in native.
>
> Of course I understand it might be some worse behaviors especially GC,
> so pinning might not be recommended for all of Java developers. But I
> believe pinning is welcomed from developers who experienced in native
> (C/C++/assembly/GPGPU and so on) because they want to offload processing
> which needs large memory (preprocessed in Java).
>
>
> Thanks,
>
> Yasumasa
>
>
> 2023-08-28 18:48 に Maurizio Cimadamore さんは書きました:
> > On 28/08/2023 08:29, Yasumasa Suenaga wrote:
> >> Hi all,
> >>
> >> I'm very interested in FFM, especially generate assembly code in Java
> >> and calling them with FFM [1].
> > Hi,
> > I've seen your work a year or so ago and have been very impressed by it
> > :-)
> >>
> >> I think one of the performance neck is MemorySegment because all of
> >> on-heap regions should be copied into off-heap region (native segment)
> >> when they are referred from foreign function. So I'm expecting to
> >> implement sort of pinning operation for on-heap MemorySegment like a
> >> JNI. I guess it is mentioned in FFM update in last month [2], however
> >> "Pinning of heap segments" does not have any links - I guess nobody is
> >> working for this yet. Do you have any updates for pinning?
> >>
> >> I've played them with both OpenJDK 22 with pinning support [3] and
> >> ffmasm (hand-assembler for Java powered by FFM) [4]. I added pin/unpin
> >> method into Unsafe, and they are called by HeapMemorySegmentImpl.
> >> Finally I got about 16x performance gain compared to non-pinning code
> >> [5] on my laptop.
> >>
> >> I guess FFM API reaches to goal step by step, but it is still a
> >> preview in JDK 21. I hope that pinning feature is supported into JDK
> >> 22 because I believe we can leverage FFM for more fields! I'm happy to
> >> contribute/help to implement pinning feature if it needs.
> >
> > While there's no doubt that in some applications and use cases pinning
> > provides a significant performance boost, there are some challenges:
> >
> > 1. scope of pinning: is pinning allowed on a per-native-call basis? Or
> > is it something more general?
> > 2. does the garbage collection support region-based pinning [1] ?
> > 3. why is pinning needed in the first place?
> >
> > (1) and (2) are very much linked. Not all garbage collectors support
> > fine-grained pinning mechanism. Which means that, in most of them, if
> > you pin, you effectively block GC for the entire duration of the pin
> > operation (GC locker mechanism). This is something that, as I'm sure
> > you understand, is not very desirable. For this reason, it might be
> > better to consider a pinning API which only pins for the duration of a
> > native call (e.g. in the shape of an additional linker option). While
> > FFM could support more complex pinning policies (e.g. pin a segment
> > inside an Arena, so that segment is unpinned when the arena is
> > closed), given the uneven support for fine-grained pinning across GCs,
> > I'm not sure such a general API (which is similar to your "MS::pin"
> > method) would be a good idea. We're doing some experiments for adding
> > a new linker option which allows for pin heap segments to be pinned
> > when calling a downcall method handle, we're not yet sure of its
> > inclusion, but it would be something worth publishing somewhere (when
> > ready) so that developers (like you) can play with it and provide
> > feedback.
> >
> > Then there's (3). Most of the times, pinning is used in order to
> > interact with native calls from public-facing APIs that are "stuck"
> > using array syntax. That is, in order to be user friendly, such API
> > work with arrays - but then a problem arises when trying to use the
> > contents of the array off-heap. But what if the memory was off-heap to
> > begin with? Then no memory copy would be required. I believe the
> > biggest impediment for off-heap memory being used directly has to do
> > with the fact that, for users, interacting with an `int[]` is
> > significantly easier than interacting with a `MemorySegment`, or a
> > `ByteBuffer`. But what if we could provide some mechanism to create an
> > "array view" over an off-heap memory region? Now clients would be able
> > to use the beloved `[]` syntax, even if memory access remained
> > off-heap.
> >
> > While we don't have any concrete proposal on this latter point, we do
> > believe that the topic of making memory segments (or byte buffer)
> > easier to access for "legacy clients" is inextricably linked to the
> > topic of pinning of heap memory.
> >
> > [1] - https://openjdk.org/jeps/423
> >
> >>
> >>
> >> Thanks,
> >>
> >> Yasumasa
> >>
> >>
> >> [1] https://github.com/YaSuenag/ffmasm
> >> [2]
> >> https://mail.openjdk.org/pipermail/panama-dev/2023-July/019510.html
> >> [3]
> >>
> https://github.com/YaSuenag/jdk/commit/f0a9b3705b3ecdf3dbb6b80cac9d53456f08f967
> >> [4]
> >>
> https://github.com/YaSuenag/ffmasm/commit/925608538b936db1b311ae84e12fa0252058b7f4
> >> [5]
> >>
> https://github.com/YaSuenag/ffmasm/blob/ffm-pinning/benchmarks/vectorapi/src/main/java/com/yasuenag/ffmasm/benchmark/vectorapi/VectorOpComparison.java
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/panama-dev/attachments/20230828/7b973235/attachment-0001.htm>