Memory Mapped Segment with offsets into the underlying file

[Maurizio Cimadamore]

On 10/09/2020 23:09, Johannes Lichtenberger wrote:
> Thanks a lot, that's great news. BTW: Is this[1] example still up-to-date?

Not entirely, it will need some tweaking (I'll try to get to that)

Maurizio

>
> I think I might have to use `madvice` hints in the future, as Trie 
> indexes involve random page lookups, so I guess sequential prefetching 
> might hurt the performance.
>
> Right now I'm still on JDK 14, but will switch to 15 when it's 
> officially released (so I know it's going to be in JDK 16 or even a 
> later JDK ;-)).
>
> Kind regards
> Johannes
>
> [1] 
> https://gist.github.com/mcimadamore/128ee904157bb6c729a10596e69edffd 
> <https://urldefense.com/v3/__https://gist.github.com/mcimadamore/128ee904157bb6c729a10596e69edffd__;!!GqivPVa7Brio!OW9yEpO0ZBU6qtfA3GVFR14LTJ3YR0O7KBeDAxK_qpBkM8-XgVIvxk23o6wm-laJyQChFP4$>
>
> Am Do., 10. Sept. 2020 um 22:23 Uhr schrieb Maurizio Cimadamore 
> <maurizio.cimadamore at oracle.com <mailto:maurizio.cimadamore at oracle.com>>:
>
>
>     On 10/09/2020 21:20, Uwe Schindler wrote:
>>     Cool. Will try that out! Would it work the same way with mmapped
>>     files?
>     Yes, works for every segment kind.
>>
>>     Do I need to compile on my own or do you have a prebuilt binary?
>     You will have to build I'm afraid - we're working on a new EA
>     binary, but will take few weeks.
>>
>>     Where's the VM patch, I'm interested to see how it's done.
>
>     All the explanation is here:
>
>     https://git.openjdk.java.net/panama-foreign/pull/304
>
>     Thanks
>     Maurizio
>
>>
>>     Uwe
>>
>>     Am September 10, 2020 8:12:12 PM UTC schrieb Maurizio Cimadamore
>>     <maurizio.cimadamore at oracle.com>
>>     <mailto:maurizio.cimadamore at oracle.com>:
>>
>>         Sorry for resurrecting this old thread.
>>
>>         We have just pushed a patch which enables safe support for shared memory
>>         segments.
>>
>>         In order to remove ownership from a segment you have to call the
>>         `withOwnerThread(null)` method - e.g.
>>
>>         MemorySegment shared =
>>         MemorySegment.allocateNative(100).withOwnerThread(null);
>>
>>         This segment will be accessible from any thread, and also closeable from
>>         any thread. Some VM magic prevents hard crashes - but accessing threads
>>         which attempt to access the segment in a racy way while it's being
>>         closed might fail with an IllegalStateException.
>>
>>         It would be nice if you could give this a try, and let us know how it
>>         works for your use case.
>>
>>         Cheers
>>         Maurizio
>>
>>         On 02/07/2020 00:49, Maurizio Cimadamore wrote:
>>
>>             On 01/07/2020 23:44, Johannes Lichtenberger wrote:
>>
>>                 The only thing, which is missing right now might be
>>                 sharing a MemorySegment between threads, whereas I'm
>>                 making sure that only ever one thread reads or writes
>>                 (but that's something where I have to wait for Java
>>                 15 or even later I guess). A basic operation SirixDB
>>                 is offering is a timer based auto-commit with the
>>                 ScheduledExecutorService, which commits in another
>>                 thread than the main parent thread. 
>>
>>             This is at the top of our list of things to work on see
>>             [1] for some background. The trick is to find a way to
>>             have shared segment which work at speed, and retain some
>>             form of deterministic deallocation. In the meantime
>>             though, with the Java 15 API you can workaround the
>>             limitation by using the MemorySegment::ofNativeRestricted
>>             factory - which allows you to set a "null" owner thread
>>             (so it's effectively a shared segment, but an unsafe one,
>>             so if you manipulate it incorrectly you can crash the
>>             VM). Maurizio [1] -
>>             https://mail.openjdk.java.net/pipermail/panama-dev/2020-May/009004.html
>>
>>