compose MemorySegments

[Pedro Lamarão]

Em sex., 11 de jun. de 2021 às 11:48, Chris Vest <mr.chrisvest at gmail.com>
escreveu:

>
> On Fri, 11 Jun 2021 at 15:53, Pedro Lamarão <pedro.lamarao at prodist.com.br>
> wrote:
>
>> Em sex., 11 de jun. de 2021 às 00:17, leerho <leerho at gmail.com> escreveu:
>>
>> > This is an interesting problem and a difficult one to do efficiently
>> > without copies or additional allocations. I think the OS does similar
>> > tricks when it maps virtual memory addresses into CPU caches or main
>> > memory.  To the user's program it appears as a contiguous memory space
>> but
>> > in reality could be broken into many separate physical chunks.  I'm sure
>> > there are some ways one could do this in C++, but probably not as
>> efficient
>> > as the OS could do it.
>> >
>>
>> Virtual memory is "accelerated" by the hardware's MMU;
>> that would be a very hard performance target to replicate in userspace.
>>
>

> The tricks we'd need to play (if we can assume aligned, whole-page chunks,
> anyway) are available to us in user-space via the mmap(2) system call.
> The RUMA paper shows how you can build dynamic data structures on top of
> this:
> http://www.vldb.org/pvldb/vol9/p768-schuhknecht.pdf
>

Very interesting, thanks for sharing!
I was assuming arbitrary buffers based on my reading of the OP.
Requiring allocations in page units seems very reasonable nowadays with 4
KB pages.
I suppose a specialized page-allocator capable of "merge" and "swap" would
be an incredible API.

-- 
Pedro Lamarão
https://www.prodist.com.br
Securing Critical Systems
Tel: +55 11 4380-6585

Antes de imprimir esta mensagem e seus anexos, certifique-se que seja
realmente necessário.
Proteger o meio ambiente é nosso dever.
Before printing this e-mail or attachments, be sure it is necessary.
It is in our hands to protect the environment.