<div dir="auto">I think you should probably use MemorySegment::mapFile and directly obtain a MemorySegment from that call. I'd try to create a segment for the whole file and get pageSize fragments at specified offsets from this file memory segment. I think otherwise with your current approach you're calling mmap and munmap way too often as it involves costly system calls and mappings to virtual addresses (the contents should also be paged in on demand, not at once). Also I think, that even multiple calls to the `mapFile` method and the same file + start address and offset should not add up virtual memory if it's a shared mapping (<font color="#232629" face="ui-monospace, cascadia mono, segoe ui mono, liberation mono, menlo, monaco, consolas, monospace"><span style="font-size:13px;white-space:pre-wrap;background-color:rgb(227,230,232)">MAP_SHARED</span></font> and I think it is).<div dir="auto"><br></div><div dir="auto">As far as I can tell the approach works in SirixDB so far, but as I want to cache some pages in memory (your buffer manager pendant -- on-heap currently) which in my case have to be reconstructed from scattered page fragments probably it doesn't seem to be faster nor slower than a pure FileChannel based approach. Usually with memory mapping I think you'd rather not want to have an application cache as it's already available off-heap (that said if you store these page sized memory segment views in the buffer manager it should work IMHO).</div><div dir="auto"><br></div><div dir="auto">Hope any stuff which I mentioned if it's not correct will be corrected by more experienced engineers.</div><div dir="auto"><div dir="auto"><div dir="auto"><br></div><div dir="auto">Kind regards</div><div dir="auto">Johannes</div></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Gavin Ray <<a href="mailto:ray.gavin97@gmail.com">ray.gavin97@gmail.com</a>> schrieb am So., 4. Sep. 2022, 00:26:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr">Radosław, I tried to implement your advice but I think I might have implemented it incorrectly<div>With JMH, I get very poor results:</div><div><br></div><div><div>Benchmark Mode Cnt Score Error Units</div><div>DiskManagerBenchmarks._01_01_writePageDiskManager thrpt 4 552595.393 � 77869.814 ops/s</div><div>DiskManagerBenchmarks._01_02_writePageMappedDiskManager thrpt 4 174.588 � 111.846 ops/s</div><div>DiskManagerBenchmarks._02_01_readPageDiskManager thrpt 4 640469.183 � 104851.381 ops/s</div><div>DiskManagerBenchmarks._02_02_readPageMappedDiskManager thrpt 4 133564.674 � 10693.985 ops/s</div></div><div><br></div><div>The difference in writing is ~550,000 vs 174(!)</div><div>In reading it is ~640,000 vs ~130,000</div><div><br></div><div>This is the implementation code:</div><div><br></div><div><div>public void readPage(PageId pageId, MemorySegment pageBuffer) {</div><div> int pageOffset = pageId.value() * Constants.PAGE_SIZE;</div><div> MemorySegment mappedBuffer = raf.getChannel().map(FileChannel.MapMode.READ_WRITE, pageOffset, Constants.PAGE_SIZE, session);</div><div> mappedBuffer.load();</div><div> pageBuffer.copyFrom(mappedBuffer);</div><div> mappedBuffer.unload();</div><div>}</div><div><br></div><div>public void writePage(PageId pageId, MemorySegment pageBuffer) {</div><div> int pageOffset = pageId.value() * Constants.PAGE_SIZE;</div><div> MemorySegment mappedBuffer = raf.getChannel().map(FileChannel.MapMode.READ_WRITE, pageOffset, Constants.PAGE_SIZE, session);</div><div> mappedBuffer.copyFrom(pageBuffer);</div><div> mappedBuffer.force();</div><div> mappedBuffer.unload();</div><div>}</div></div><div><br></div><div>Am I doing something wrong here (I think I probably am)</div></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Sep 2, 2022 at 1:16 PM Gavin Ray <<a href="mailto:ray.gavin97@gmail.com" target="_blank" rel="noreferrer">ray.gavin97@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Thank you very much for the advice, I will implement these suggestions =)</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Sep 2, 2022 at 12:12 PM Radosław Smogura <<a href="mailto:mail@smogura.eu" target="_blank" rel="noreferrer">mail@smogura.eu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div>
<div lang="EN-US">
<div>
<p class="MsoNormal">Hi Gavin,</p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">I see you do a good progress.</p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">This is good approach. Minor improvement would be to use MemorySegment.ofBuffer(), to create memory segment from _<i>direct</i>_ byte buffer. This way you would have consistency (using only MemorySegment) and FileChannels or other methods
to manage file size.</p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Most probably you would like to use MappedByteBuffer.force() to flush changes to disk (equivalent of sync in Linux) – i.e. to be sure transaction is persisted or for write ahead log.</p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">In most cases if you want to work with zero-copy reads, you have to map a whole file as direct buffer / memory segment. You would need to enlarge file using (most probably file channel) or other methods, if you want to append new data (otherwise
sigbus or segfault can be generated – can result in exception or crash).</p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">You can compare different approaches using JMH to measure reads and writes performance.</p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Kind regards,</p>
<p class="MsoNormal">Rado Smogura</p>
<p class="MsoNormal"><u></u> <u></u></p>
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(225,225,225);padding:3pt 0in 0in">
<p class="MsoNormal" style="border:none;padding:0in"><b>From: </b><a href="mailto:ray.gavin97@gmail.com" target="_blank" rel="noreferrer">Gavin Ray</a><br>
<b>Sent: </b>Friday, September 2, 2022 5:50 PM<br>
<b>To: </b><a href="mailto:lichtenberger.johannes@gmail.com" target="_blank" rel="noreferrer">Johannes Lichtenberger</a><br>
<b>Cc: </b><a href="mailto:maurizio.cimadamore@oracle.com" target="_blank" rel="noreferrer">Maurizio Cimadamore</a>;
<a href="mailto:panama-dev@openjdk.org" target="_blank" rel="noreferrer">panama-dev@openjdk.org</a><br>
<b>Subject: </b>Re: Question: ByteBuffer vs MemorySegment for binary (de)serializiation and in-memory buffer pool</p>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<div>
<p class="MsoNormal">On a related note, is there any way to do zero-copy reads from files using MemorySegments for non-Memory-Mapped files?<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Currently I'm using "SeekableByteChannel" and wrapping the MemorySegment using ".asByteBuffer()"<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Is this the most performant way?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">========================<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<div>
<p class="MsoNormal">class DiskManager {<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> private final RandomAccessFile raf;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> private final SeekableByteChannel dbFileChannel;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"> public void readPage(PageId pageId, MemorySegment pageBuffer) {<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> int pageOffset = pageId.value() * Constants.PAGE_SIZE;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> dbFileChannel.position(pageOffset);<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> dbFileChannel.read(pageBuffer.asByteBuffer());<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> }<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"> public void writePage(PageId pageId, MemorySegment pageBuffer) {<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> int pageOffset = pageId.value() * Constants.PAGE_SIZE;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> dbFileChannel.position(pageOffset);<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> dbFileChannel.write(pageBuffer.asByteBuffer());<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> }<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">}<u></u><u></u></p>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">On Thu, Sep 1, 2022 at 6:13 PM Johannes Lichtenberger <<a href="mailto:lichtenberger.johannes@gmail.com" target="_blank" rel="noreferrer">lichtenberger.johannes@gmail.com</a>> wrote:<u></u><u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0in 0in 0in 6pt;margin-left:4.8pt;margin-right:0in">
<div>
<p class="MsoNormal">I think it's a really good idea to use off-heap memory for the Buffer Manager/the pages with the stored records. In my case, I'm working on an immutable, persistent DBMS currently storing JSON and XML with only one read-write trx per resource
concurrently and if desired in parallel to N read-only trx bound to specific revisions (in the relational world the term for a resource is a relation/table). During an import of a close to 4Gb JSON file with intermediate commits, I found out that depending
on the number of records/nodes accumulated in the trx intent log (a trx private map more or less), after which a commit and thus a sync to disk with removing the pages from the log is issued, the GC runs are >= 100ms most of the times and the objects are long-lived
and are promoted to the old gen obviously, which seems to take these >= 100ms. That is I'll have to study how Shenandoah works, but in this case, it brings no advantage regarding the latency.<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Maybe it would make sense to store the data in the record instances also off-head, as Gavin did with his simple Buffer Manager :-) that said lowering the max records number after which to commit and sync to disk also has a tremendous effect
and with Shenandoah, the GC times are less than a few ms at least.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">I'm using the Foreign Memory API however already to store the data in memory-mapped files, once the pages (or page fragments) and records therein are serialized and then written to the memory segment after compression and hopefully soon
encyrption.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Kind regards<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Johannes<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">Am Do., 1. Sept. 2022 um 22:52 Uhr schrieb Maurizio Cimadamore <<a href="mailto:maurizio.cimadamore@oracle.com" target="_blank" rel="noreferrer">maurizio.cimadamore@oracle.com</a>>:<u></u><u></u></p>
</div>
</div>
</blockquote>
</div>
</div>
<p class="MsoNormal" style="margin-right:0in;margin-bottom:12pt;margin-left:9.6pt">
<br>
On 01/09/2022 19:26, Gavin Ray wrote:<br>
> I think this is where my impression of verbosity is coming from, in <br>
> [1] I've linked a gist of ByteBuffer vs MemorySegment implementation <br>
> of a page header struct,<br>
> and it's the layout/varhandles that are the only difference, really.<br>
><br>
Ok, I see what you mean, of course; thanks for the Gist.<br>
<br>
In this case I think the instance accessor we added on MemorySegment <br>
will bring the code more or less to the same shape as what it used to be <br>
with the ByteBuffer API.<br>
<br>
Using var handles is very useful when you want to access elements (e.g. <br>
structs inside other structs inside arrays) as it takes all the offset <br>
computation out of the way.<br>
<br>
If you're happy enough with hardwired offsets (and I agree that in this <br>
case things might be good enough), then there's nothing wrong with using <br>
the ready-made accessor methods.<br>
<br>
Maurizio<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</div></blockquote></div>
</blockquote></div>
</blockquote></div>