Question: ByteBuffer vs MemorySegment for binary (de)serializiation and in-memory buffer pool

[Johannes Lichtenberger]

That's an interesting use case of the Foreign Memory API IMHO. I haven't
even thought about using it in this particular case. I'm also working on a
DBMS[1] side project in my spare time. I've used the Foreign Memory API to
memory map the storage file (for reads), but couldn't measure much impact
over a RandomAccess file / FileChannel reader. As it's an append-only store
writes are done via the FileChannel abstraction instead.
The serialization/deserialization to an array of slots is done however
using Chronicle Bytes with Bytes<ByteBuffer> mainly, as Chronicle handles
the resizing and does some Unsafe tricks, IIRC.

@Gavin, do you also have a repository? I'd love to exchange ideas as I'm
currently the only one working on the project and sharing thoughts/ideas
would be awesome.

Kind regards
Johannes

[1] https://github.com/sirixdb/sirix and the query engine
https://github.com/sirixdb/brackit

Am Mo., 22. Aug. 2022 um 20:30 Uhr schrieb Gavin Ray <ray.gavin97 at gmail.com
>:

> Thanks Paul,
>
> The API is a fair bit wordier, but that's the price you pay in exchange
> for greater expressiveness and flexibility I suppose.
> I'll likely go forward with the MemorySegment implementation then -- I
> appreciate the reply!
>
> On Mon, Aug 22, 2022 at 2:21 PM Paul Sandoz <paul.sandoz at oracle.com>
> wrote:
>
>> Hi Gavin,
>>
>> This is the right place to reach out.
>>
>> Using MemorySegment will given you far more control over the description
>> (layout) and management (freeing and pooling) than ByteBuffer. Also, if
>> it’s an issue you will also not be constrained by ByteBuffer’s size
>> limitation. Performance wise using MemorySegment should be as good as or
>> better than ByteBuffer.
>>
>> In many respects MemorySegment is a better API to interact with native
>> memory. ByteBuffer was introduced in Java 1.4 with NIO and had additional
>> design constraints in mind that are less relevant today (such as an
>> internal mutable index).
>>
>> Paul.
>>
>> > On Aug 20, 2022, at 10:35 AM, Gavin Ray <ray.gavin97 at gmail.com> wrote:
>> >
>> > Hiya, I hope it's okay that I ask questions here (if not, please direct
>> me elsewhere)
>> >
>> > I tried to ask on StackOverflow without much luck, and there aren't
>> many resources
>> > available at the moment to read about the Foreign Memory API so I'm
>> reaching out here.
>> >
>> > I'm implementing a basic database on the JVM and am trying to
>> understand the
>> > technical implications of using either ByteBuffer or MemorySegment for
>> heap page management and in-memory buffer allocation.
>> >
>> > Details are in this thread:
>> > database - Java: Performance of ByteBuffer versus jdk.incubator.foreign
>> (Panama) Foreign Memory methods (MemoryLayout/Segment) - Stack Overflow
>> >
>> > Would appreciate any information folks would be willing to share =)
>> >
>> > Thank you,
>> > Gavin Ray
>> >
>> > (By the way, I'm not subscribed to the mailing list, will I be able to
>> reply to responses?)
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/panama-dev/attachments/20220823/31190a95/attachment-0001.htm>