Memory Mapped Segment with offsets into the underlying file

[Johannes Lichtenberger]

I guess you won't have time either, but if you like to you could simply try
if this test for instance still fails if enabled:

https://github.com/sirixdb/sirix/blob/master/bundles/sirix-core/src/test/java/org/sirix/axis/concurrent/ConcurrentAxisTest.java

And the two classes (I assume I have to apply other changes as well, as the
API might have changed already?):

https://github.com/sirixdb/sirix/blob/master/bundles/sirix-core/src/main/java/org/sirix/io/memorymapped/MMFileReader.java
https://github.com/sirixdb/sirix/blob/master/bundles/sirix-core/src/main/java/org/sirix/io/memorymapped/MMFileWriter.java

But I'm pretty sure the mentioned test will succeed, because what failed
was closing the transactions on another thread (instead of the owning
thread).

So, I have to spend too much of my spare time on preparing some stuff for
the #Hacktoberfest for the project ;-)

Kind regards
Johannes

Am Fr., 11. Sept. 2020 um 00:09 Uhr schrieb Johannes Lichtenberger <
lichtenberger.johannes at gmail.com>:

> Thanks a lot, that's great news. BTW: Is this[1] example still up-to-date?
>
> I think I might have to use `madvice` hints in the future, as Trie indexes
> involve random page lookups, so I guess sequential prefetching might hurt
> the performance.
>
> Right now I'm still on JDK 14, but will switch to 15 when it's officially
> released (so I know it's going to be in JDK 16 or even a later JDK ;-)).
>
> Kind regards
> Johannes
>
> [1] https://gist.github.com/mcimadamore/128ee904157bb6c729a10596e69edffd
>
> Am Do., 10. Sept. 2020 um 22:23 Uhr schrieb Maurizio Cimadamore <
> maurizio.cimadamore at oracle.com>:
>
>>
>> On 10/09/2020 21:20, Uwe Schindler wrote:
>>
>> Cool. Will try that out! Would it work the same way with mmapped files?
>>
>> Yes, works for every segment kind.
>>
>>
>> Do I need to compile on my own or do you have a prebuilt binary?
>>
>> You will have to build I'm afraid - we're working on a new EA binary, but
>> will take few weeks.
>>
>>
>> Where's the VM patch, I'm interested to see how it's done.
>>
>> All the explanation is here:
>>
>> https://git.openjdk.java.net/panama-foreign/pull/304
>>
>> Thanks
>> Maurizio
>>
>>
>> Uwe
>>
>> Am September 10, 2020 8:12:12 PM UTC schrieb Maurizio Cimadamore
>> <maurizio.cimadamore at oracle.com> <maurizio.cimadamore at oracle.com>:
>>>
>>> Sorry for resurrecting this old thread.
>>>
>>> We have just pushed a patch which enables safe support for shared memory
>>> segments.
>>>
>>> In order to remove ownership from a segment you have to call the
>>> `withOwnerThread(null)` method - e.g.
>>>
>>> MemorySegment shared =
>>> MemorySegment.allocateNative(100).withOwnerThread(null);
>>>
>>> This segment will be accessible from any thread, and also closeable from
>>> any thread. Some VM magic prevents hard crashes - but accessing threads
>>> which attempt to access the segment in a racy way while it's being
>>> closed might fail with an IllegalStateException.
>>>
>>> It would be nice if you could give this a try, and let us know how it
>>> works for your use case.
>>>
>>> Cheers
>>> Maurizio
>>>
>>> On 02/07/2020 00:49, Maurizio Cimadamore wrote:
>>>>
>>>>
>>>>  On 01/07/2020 23:44, Johannes Lichtenberger wrote:
>>>>>
>>>>> The only thing, which is missing right now might be sharing a
>>>>> MemorySegment between threads, whereas I'm making sure that only ever
>>>>> one thread reads or writes (but that's something where I have to wait
>>>>> for Java 15 or even later I guess). A basic operation SirixDB is
>>>>> offering is a timer based auto-commit with the
>>>>> ScheduledExecutorService, which commits in another thread than the
>>>>> main parent thread.
>>>>
>>>>
>>>>  This is at the top of our list of things to work on see [1] for some
>>>>  background. The trick is to find a way to have shared segment which
>>>>  work at speed, and retain some form of deterministic deallocation.
>>>>
>>>>  In the meantime though, with the Java 15 API you can workaround the
>>>>  limitation by using the MemorySegment::ofNativeRestricted factory -
>>>>  which allows you to set a "null" owner thread (so it's effectively a
>>>>  shared segment, but an unsafe one, so if you manipulate it incorrectly
>>>>  you can crash the VM).
>>>>
>>>>  Maurizio
>>>>
>>>>  [1] -
>>>>  https://mail.openjdk.java.net/pipermail/panama-dev/2020-May/009004.html
>>>>
>>>