Memory Mapped Segment with offsets into the underlying file

[Johannes Lichtenberger]

Ah, it's testConcurrent() for instance, or testPartConcurrentDescAxis1()
and testPartConcurrentDescAxis2()

Am Fr., 11. Sept. 2020 um 00:27 Uhr schrieb Johannes Lichtenberger <
lichtenberger.johannes at gmail.com>:

> I guess you won't have time either, but if you like to you could simply
> try if this test for instance still fails if enabled:
>
>
> https://github.com/sirixdb/sirix/blob/master/bundles/sirix-core/src/test/java/org/sirix/axis/concurrent/ConcurrentAxisTest.java
>
> And the two classes (I assume I have to apply other changes as well, as
> the API might have changed already?):
>
>
> https://github.com/sirixdb/sirix/blob/master/bundles/sirix-core/src/main/java/org/sirix/io/memorymapped/MMFileReader.java
>
> https://github.com/sirixdb/sirix/blob/master/bundles/sirix-core/src/main/java/org/sirix/io/memorymapped/MMFileWriter.java
>
> But I'm pretty sure the mentioned test will succeed, because what failed
> was closing the transactions on another thread (instead of the owning
> thread).
>
> So, I have to spend too much of my spare time on preparing some stuff for
> the #Hacktoberfest for the project ;-)
>
> Kind regards
> Johannes
>
> Am Fr., 11. Sept. 2020 um 00:09 Uhr schrieb Johannes Lichtenberger <
> lichtenberger.johannes at gmail.com>:
>
>> Thanks a lot, that's great news. BTW: Is this[1] example still up-to-date?
>>
>> I think I might have to use `madvice` hints in the future, as Trie
>> indexes involve random page lookups, so I guess sequential prefetching
>> might hurt the performance.
>>
>> Right now I'm still on JDK 14, but will switch to 15 when it's officially
>> released (so I know it's going to be in JDK 16 or even a later JDK ;-)).
>>
>> Kind regards
>> Johannes
>>
>> [1] https://gist.github.com/mcimadamore/128ee904157bb6c729a10596e69edffd
>>
>> Am Do., 10. Sept. 2020 um 22:23 Uhr schrieb Maurizio Cimadamore <
>> maurizio.cimadamore at oracle.com>:
>>
>>>
>>> On 10/09/2020 21:20, Uwe Schindler wrote:
>>>
>>> Cool. Will try that out! Would it work the same way with mmapped files?
>>>
>>> Yes, works for every segment kind.
>>>
>>>
>>> Do I need to compile on my own or do you have a prebuilt binary?
>>>
>>> You will have to build I'm afraid - we're working on a new EA binary,
>>> but will take few weeks.
>>>
>>>
>>> Where's the VM patch, I'm interested to see how it's done.
>>>
>>> All the explanation is here:
>>>
>>> https://git.openjdk.java.net/panama-foreign/pull/304
>>>
>>> Thanks
>>> Maurizio
>>>
>>>
>>> Uwe
>>>
>>> Am September 10, 2020 8:12:12 PM UTC schrieb Maurizio Cimadamore
>>> <maurizio.cimadamore at oracle.com> <maurizio.cimadamore at oracle.com>:
>>>>
>>>> Sorry for resurrecting this old thread.
>>>>
>>>> We have just pushed a patch which enables safe support for shared memory
>>>> segments.
>>>>
>>>> In order to remove ownership from a segment you have to call the
>>>> `withOwnerThread(null)` method - e.g.
>>>>
>>>> MemorySegment shared =
>>>> MemorySegment.allocateNative(100).withOwnerThread(null);
>>>>
>>>> This segment will be accessible from any thread, and also closeable from
>>>> any thread. Some VM magic prevents hard crashes - but accessing threads
>>>> which attempt to access the segment in a racy way while it's being
>>>> closed might fail with an IllegalStateException.
>>>>
>>>> It would be nice if you could give this a try, and let us know how it
>>>> works for your use case.
>>>>
>>>> Cheers
>>>> Maurizio
>>>>
>>>> On 02/07/2020 00:49, Maurizio Cimadamore wrote:
>>>>>
>>>>>
>>>>>  On 01/07/2020 23:44, Johannes Lichtenberger wrote:
>>>>>>
>>>>>> The only thing, which is missing right now might be sharing a
>>>>>> MemorySegment between threads, whereas I'm making sure that only ever
>>>>>> one thread reads or writes (but that's something where I have to wait
>>>>>> for Java 15 or even later I guess). A basic operation SirixDB is
>>>>>> offering is a timer based auto-commit with the
>>>>>> ScheduledExecutorService, which commits in another thread than the
>>>>>> main parent thread.
>>>>>
>>>>>
>>>>>  This is at the top of our list of things to work on see [1] for some
>>>>>  background. The trick is to find a way to have shared segment which
>>>>>  work at speed, and retain some form of deterministic deallocation.
>>>>>
>>>>>  In the meantime though, with the Java 15 API you can workaround the
>>>>>  limitation by using the MemorySegment::ofNativeRestricted factory -
>>>>>  which allows you to set a "null" owner thread (so it's effectively a
>>>>>  shared segment, but an unsafe one, so if you manipulate it incorrectly
>>>>>  you can crash the VM).
>>>>>
>>>>>  Maurizio
>>>>>
>>>>>  [1] -
>>>>>  https://mail.openjdk.java.net/pipermail/panama-dev/2020-May/009004.html
>>>>>
>>>>