Garbage problem migrating from sun.misc.Unsafe to FFM API

[Remi Forax]

> From: "Remi Forax" <forax at univ-mlv.fr>
> To: "Sergio Selos" <sergio.selos.chicago at gmail.com>
> Cc: "panama-dev" <panama-dev at openjdk.org>
> Sent: Friday, December 27, 2024 2:52:17 PM
> Subject: Re: Garbage problem migrating from sun.misc.Unsafe to FFM API

> I do not think you need pooling,
> if the ByteBuffer is created from a MemorySegment, it keeps a reference to the
> MemeorySegment

> var segment = Arena . global ().allocate( 8192 );
> var buffer = segment .asByteBuffer();
> ...
> var segment2 = MemorySegment . ofBuffer ( buffer );
> assertSame( segment, segment2 );  // should be ok !

It's not :( t's a MemorySegment with the same address but not the same memory segment, 
so you have to hope that the VM is able to do escape analysis on the result of ofBuffer(). 

Rémi 

>> From: "Sergio Selos" <sergio.selos.chicago at gmail.com>
>> To: "Remi Forax" <forax at univ-mlv.fr>
>> Cc: "panama-dev" <panama-dev at openjdk.org>
>> Sent: Friday, December 27, 2024 1:12:52 PM
>> Subject: Re: Garbage problem migrating from sun.misc.Unsafe to FFM API

>> Thanks for your answer, Remi.
>>> Then I think that the idea if you want to minimize the garbage is to create the
>>> ByteBuffer from a MemorySegment with asByteBuffer() so you can then use
>> > ofBuffer() when you want the MemorySegment back.

>> That approach could work, but ideally, we want the same capability provided by
>> sun.misc.Unsafe today: the ability to work directly with external ByteBuffers.
>> Note that we’re not really using the ByteBuffer itself but its underlying
>> native memory address. Perhaps I’m overlooking something, but it seems the FFM
>> API should offer a way to copy data directly between native memory regions.

>> I also posted this question on StackOverflow and it was suggested to pool a
>> MemorySegment per ByteBuffer. Unfortunately I'm afraid that will be a
>> show-stopper due to the extra lookup in the critical path. For the record, the
>> SO question is [
>> https://stackoverflow.com/questions/79311345/is-it-possible-to-copy-from-native-memory-to-a-bytebuffer-using-the-new-ffm-api
>> |
>> https://stackoverflow.com/questions/79311345/is-it-possible-to-copy-from-native-memory-to-a-bytebuffer-using-the-new-ffm-api
>> ]

>> All the best,

>> -Sergio

>> On Fri, Dec 27, 2024 at 8:16 AM Remi Forax < [ mailto:forax at univ-mlv.fr |
>> forax at univ-mlv.fr ] > wrote:

>>> Hello,
>>> are you using an older jdk than 22, MemorySegment.ofBuffer() should only takes a
>>> ByteBuffer [1] ?

>>> Then I think that the idea if you want to minimize the garbage is to create the
>>> ByteBuffer from a MemorySegment with asByteBuffer() so you can then use
>>> ofBuffer() when you want the MemorySegment back.

>>> That said, i would love to have methods like read() and write() to take
>>> MemorySegment instead of ByteBuffer because for my students the ByteBuffer API
>>> is an useless complication (you have to understand how the indexes work) when
>>> doing serious IO. Having API closest to the C ones would help a lot.

>>> Rémi

>>> [1] [
>>> https://docs.oracle.com/en/java/javase/23/docs/api/java.base/java/lang/foreign/MemorySegment.html#ofBuffer(java.nio.Buffer)
>>> |
>>> https://docs.oracle.com/en/java/javase/23/docs/api/java.base/java/lang/foreign/MemorySegment.html#ofBuffer(java.nio.Buffer)
>>> ]

>>>> From: "Sergio Selos" < [ mailto:sergio.selos.chicago at gmail.com |
>>>> sergio.selos.chicago at gmail.com ] >
>>>> To: "panama-dev" < [ mailto:panama-dev at openjdk.org | panama-dev at openjdk.org ] >
>>>> Sent: Friday, December 27, 2024 2:05:24 AM
>>>> Subject: Garbage problem migrating from sun.misc.Unsafe to FFM API

>>>> Hello,
>>>> We have a project using sun.misc.Unsafe that we are migrating to the new FFM
>>>> API. The project is at [ https://www.github.com/coralblocks/CoralRing |
>>>> https://www.github.com/coralblocks/CoralRing ]

>>>> For our particular systems (mostly financial low latency systems) it is
>>>> important not to create any garbage (temp discarded instances) when receiving a
>>>> message.

>>>> We encountered an issue when copying from native memory straight into a direct
>>>> ByteBuffer using the FFM API.

>>>> Using sun.misc.Unsafe, copying from memory to a ByteBuffer can be done without
>>>> creating temporary objects and garbage collector overhead. However, with the
>>>> FFM API, achieving the same does not seem currently possible without generating
>>>> garbage via a call to MemorySegment.ofBuffer.

>>>> Does the FFM dev team plan to address this issue and provide a garbage-free way
>>>> to perform this copy, like we currently can with sun.misc.Unsafe?

>>>> Maybe there is already a way and we are just unaware how to do it.

>>>> Below the current code we have with sun.misc.Unsafe, which does not produce any
>>>> garbage, and the only way we are aware of to do it with the FFM API, which
>>>> produces garbage.

>>>> Thanks and let me know if you need any more details about this and/or if I can
>>>> help with anything.

>>>> All the best,

>>>> -Sergio

>>>> // With sun.misc.Unsafe:
>>>> @Override
>>>> public void getByteBuffer(long address, ByteBuffer dst, int len) {
>>>> if (!dst.isDirect()) {
>>>> throw new RuntimeException("getByteBuffer can only take a direct byte buffer!");
>>>> }
>>>> try {
>>>> long dstAddress = (long) addressField.get(dst); // get the memory address of
>>>> this ByteBuffer
>>>> dstAddress += dst.position(); // adjust the address for the ByteBuffer current
>>>> position
>>>> unsafe.copyMemory(address, dstAddress, len); // copy over
>>>> dst.position(dst.position() + len); // adjust the ByteBuffer position to reflect
>>>> the copy operation
>>>> } catch(Exception e) {
>>>> throw new RuntimeException(e);
>>>> }
>>>> }

>>>> // With FFM API:
>>>> @Override
>>>> public void getByteBuffer(long address, ByteBuffer dst, int len) {
>>>> if (!dst.isDirect()) {
>>>> throw new RuntimeException("getByteBuffer can only take a direct byte buffer!");
>>>> }

>>>> long offset = address - this.address; // offset in our 'segment'

>>>> try (Arena arena = Arena.ofConfined()) { // <==== probably creating a temp
>>>> object here too
>>>> // Wrap the destination ByteBuffer in a temporary segment
>>>> // This segment's data starts at dst.position()
>>>> MemorySegment dstSegment = MemorySegment.ofBuffer(dst, arena); // <====== temp
>>>> memory segment object

>>>> // Copy from 'segment' at 'offset' → dstSegment at offset 0 → length = 'len'
>>>> dstSegment.copyFrom(this.segment, offset, 0, len);
>>>> }
>>>> dst.position(dst.position() + len);
>>>> }
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/panama-dev/attachments/20241227/7edb99ef/attachment.htm>