Question: ByteBuffer vs MemorySegment for binary (de)serializiation and in-memory buffer pool

[Gavin Ray]

Hey Maurizio,

Thanks for the comprehensive reply!
I found your talk "ByteBuffers are dead, long live ByteBuffers!" [0] the
day after submitting my question, and that gave good insight into the
MemorySegment API + design.

> "There are var handles which you can use (as pointed out elsewhere in
this thread), and, depending on the use case, they can make your life
simpler. But you don't _have_ to use them unless you want to."

I think this is where my impression of verbosity is coming from, in [1]
I've linked a gist of ByteBuffer vs MemorySegment implementation of a page
header struct,
and it's the layout/varhandles that are the only difference, really.

If anyone's curious, I've also put up the work I have so far (may not be
correct; I'm learning both how DB's and the Panama API's work) in [2] on
the off chance it's of use.

[0] https://www.youtube.com/watch?v=RYrk4wvar6g
[1] https://gist.github.com/GavinRay97/2df37c6069ace0d7bbf90f941e29a7e8
[2]
https://gist.github.com/GavinRay97/1d5a4b22d9c07f173a052765514b4990#file-database-java-L453-L475

Gavin


On Wed, Aug 31, 2022 at 4:57 PM Maurizio Cimadamore <
maurizio.cimadamore at oracle.com> wrote:

> Hi Gavin,
> we have some documents on the foreign memory API here:
>
>
> https://github.com/openjdk/panama-foreign/blob/foreign-memaccess%2Babi/doc/panama_memaccess.md
>
> They have not been updated against the latest version of the API, but
> should be relatively in good shape (at least good enough to get started).
>
> The API is a fair bit wordier, but that's the price you pay in exchange
> for greater expressiveness and flexibility I suppose.
>
> As for wordiness, I don't think the API is significantly more verbose than
> the ByteBuffer API. In fact, at a glance the methods on MemorySegment and
> ByteBuffer are quite similar, but there are two differences:
>
> * when you allocate native memory you _have_ to pass an extra argument,
> the temporal bounds of the memory segments (MemorySession), which signals
> when the segment memory is ready to be released
> * the access methods, MemorySegment::get/set are parameterized by a layout
> argument, so instead of `buffer.getInt(offset)`, you need
> `segment.get(JAVA_INT, offset)`. This will give more extensibility (e.g.
> add support for more carriers, or value types, at a later point) as well as
> more control (e.g. alignment)
>
> In my experience, and also looking at code written by developers that
> migrated projects from ByteBuffer, the MemorySegment API seems more or less
> on par with ByteBuffer in terms of verbosity. There are var handles which
> you can use (as pointed out elsewhere in this thread), and, depending on
> the use case, they can make your life simpler. But you don't _have_ to use
> them unless you want to.
>
> I hope this helps. From the looks of it, the Foreign Memory API seems like
> a great fit for the project you are working on.
>
> Maurizio
>
>
> On 20/08/2022 18:35, Gavin Ray wrote:
>
> Hiya, I hope it's okay that I ask questions here (if not, please direct me
> elsewhere)
>
> I tried to ask on StackOverflow without much luck, and there aren't many
> resources
> available at the moment to read about the Foreign Memory API so I'm
> reaching out here.
>
> I'm implementing a basic database on the JVM and am trying to understand
> the
> technical implications of using either ByteBuffer or MemorySegment for
> heap page management and in-memory buffer allocation.
>
> Details are in this thread:
> database - Java: Performance of ByteBuffer versus jdk.incubator.foreign
> (Panama) Foreign Memory methods (MemoryLayout/Segment) - Stack Overflow
> <https://stackoverflow.com/questions/73423726/java-performance-of-bytebuffer-versus-jdk-incubator-foreign-panama-foreign-me>
>
> Would appreciate any information folks would be willing to share =)
>
> Thank you,
> Gavin Ray
>
> (By the way, I'm not subscribed to the mailing list, will I be able to
> reply to responses?)
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/panama-dev/attachments/20220901/2fcb43c3/attachment.htm>