Off heap vs on heap memory access performance for a DBS

[mail at smogura.eu]

Hi all,

If I can add my two cents.

Using native memory can be slightly faster if it’s connected with mmap. This way server can save on copying from byte array to native memory just to put it into file.

With mmap actually page cache pages (or copy-on-write pages) are read and write.

Best regards,
Radoslaw Smogura

> On 28 Jul 2023, at 15:45, Brian S O'Neill <bronee at gmail.com> wrote:
> 
> On 2023-07-28 12:15 AM, Johannes Lichtenberger wrote:
>> Hello,
>> I think I mentioned it already, but currently I'm thinking about it again.
>> Regarding the index trie in my spare time project I'm thinking if it makes sense, as currently I'm creating fine granular on heap nodes during insertions/updates/deletes (1024 per page). Once a page is read again from storage I'm storing these nodes in a byte array of byte arrays until read for the first time. One thing though is, that the nodes may store strings inline and thus are of variable size (and thus, the pages are of variable size, too, padded to word aligned IIRC).
>> I'm currently auto-committing after approx 500_000 nodes have been created (afterwards they can be garbage collected) and in total there are more than 320 million nodes in one test.
>> I think I could store the nodes in MemorySegments instead of using on heap classes / instances and dynamically reallocate memory if a node value is changed.
>> However, I'm not sure as it means a lot of work and maybe off heap memory access is always slightly worse than on heap!?
> 
> I've been maintaining a (mostly) pure Java database library for several years now, and I can give you the short answer: Yes, you should consider supporting off heap memory. Now for the long answer...
> 
> The Tupl database supports a cache backed by Java byte arrays or by off heap (native) memory. By design, the cache is very friendly to generational garbage collectors -- all the memory is allocated up front, and it's recycled without any GC involvement. Database pages are fixed in size (4096 bytes by default), which simplifies memory management and persistence. Under load, GC activity is very low.
> 
> The off heap memory is managed by using the Unsafe class, but I'm converting it to use the Panama API. The Panama version is almost as fast as using Unsafe, and Maurizio (as he mentioned earlier) is working on closing the performance gap.
> 
> For small caches (1GB), the performance of the byte[] version and the off heap version is the same. The problem is scalability. With larger caches, even generational garbage collectors slow down. With a 16GB cache, the byte[] version is about 50% slower than the off heap version. This is with the G1 collector. Generational ZGC is better, with a performance ratio of about 25%. Things get much much worse as the cache gets even larger, but I don't have numbers readily available.
> 
> Note that if you're allocating and freeing MemorySegments all the time, you might not see as much of a performance improvement compared to byte arrays. A generational collector is very good at managing short lived objects. To see a huge win, you'll need to do your own memory management.
>