Memory Mapped Segment with offsets into the underlying file

[Johannes Lichtenberger]

Thanks a lot, that's great news. BTW: Is this[1] example still up-to-date?

I think I might have to use `madvice` hints in the future, as Trie indexes
involve random page lookups, so I guess sequential prefetching might hurt
the performance.

Right now I'm still on JDK 14, but will switch to 15 when it's officially
released (so I know it's going to be in JDK 16 or even a later JDK ;-)).

Kind regards
Johannes

[1] https://gist.github.com/mcimadamore/128ee904157bb6c729a10596e69edffd

Am Do., 10. Sept. 2020 um 22:23 Uhr schrieb Maurizio Cimadamore <
maurizio.cimadamore at oracle.com>:

>
> On 10/09/2020 21:20, Uwe Schindler wrote:
>
> Cool. Will try that out! Would it work the same way with mmapped files?
>
> Yes, works for every segment kind.
>
>
> Do I need to compile on my own or do you have a prebuilt binary?
>
> You will have to build I'm afraid - we're working on a new EA binary, but
> will take few weeks.
>
>
> Where's the VM patch, I'm interested to see how it's done.
>
> All the explanation is here:
>
> https://git.openjdk.java.net/panama-foreign/pull/304
>
> Thanks
> Maurizio
>
>
> Uwe
>
> Am September 10, 2020 8:12:12 PM UTC schrieb Maurizio Cimadamore
> <maurizio.cimadamore at oracle.com> <maurizio.cimadamore at oracle.com>:
>>
>> Sorry for resurrecting this old thread.
>>
>> We have just pushed a patch which enables safe support for shared memory
>> segments.
>>
>> In order to remove ownership from a segment you have to call the
>> `withOwnerThread(null)` method - e.g.
>>
>> MemorySegment shared =
>> MemorySegment.allocateNative(100).withOwnerThread(null);
>>
>> This segment will be accessible from any thread, and also closeable from
>> any thread. Some VM magic prevents hard crashes - but accessing threads
>> which attempt to access the segment in a racy way while it's being
>> closed might fail with an IllegalStateException.
>>
>> It would be nice if you could give this a try, and let us know how it
>> works for your use case.
>>
>> Cheers
>> Maurizio
>>
>> On 02/07/2020 00:49, Maurizio Cimadamore wrote:
>>>
>>>
>>>  On 01/07/2020 23:44, Johannes Lichtenberger wrote:
>>>>
>>>> The only thing, which is missing right now might be sharing a
>>>> MemorySegment between threads, whereas I'm making sure that only ever
>>>> one thread reads or writes (but that's something where I have to wait
>>>> for Java 15 or even later I guess). A basic operation SirixDB is
>>>> offering is a timer based auto-commit with the
>>>> ScheduledExecutorService, which commits in another thread than the
>>>> main parent thread.
>>>
>>>
>>>  This is at the top of our list of things to work on see [1] for some
>>>  background. The trick is to find a way to have shared segment which
>>>  work at speed, and retain some form of deterministic deallocation.
>>>
>>>  In the meantime though, with the Java 15 API you can workaround the
>>>  limitation by using the MemorySegment::ofNativeRestricted factory -
>>>  which allows you to set a "null" owner thread (so it's effectively a
>>>  shared segment, but an unsafe one, so if you manipulate it incorrectly
>>>  you can crash the VM).
>>>
>>>  Maurizio
>>>
>>>  [1] -
>>>  https://mail.openjdk.java.net/pipermail/panama-dev/2020-May/009004.html
>>>
>>