BUG: withOwnerThread closes MappedMemorySegment

Thu Sep 24 09:56:21 UTC 2020

On 24/09/2020 04:01, Ty Young wrote:
> On 9/23/20 4:31 AM, Maurizio Cimadamore wrote:
>
>> A lot of methods in MemorySegment behave this way - this has always 
>> been the behavior of "withOwnerThread" even before we had shared 
>> segments.
>
>
> withAccessModes didn't when I swapped withOwnerThread with it nor did 
> the previous restricted escape hatch method, which is why my code 
> worked before and thought it was a bug.

To be clear: I acknowledge the confusion with naming here.

Methods like asSlice, or withAccessModes are "pure withers" - they take 
some new characteristc and return a segment with that new 
characteristic, but where everything else (including temporal bounds) is 
the same.

Methods like withOwnerThread (or, the more recent withCleanupAction), 
kill the existing segment and return a new one. While this is all 
documented in the javadoc, the ambiguity can be confusing - we've been 
looking into ways to make the cut more clear, but we're yet to stumble 
on something that looks objectively _better_ than the status quo.

So, I'm not dismissing your original question.

>
>
>>
>> There are two main issues with "just mutating the existining segments":
>>
>> 1) First and foremost, C2 likes constants. So, if the thread is 
>> constant on the segment we pay nothing for the ownership (or lack of 
>> ownership) check.
>>
>> 2) Second, you can't just "flip" a switch on a shared data structure, 
>> and expect the update to be seen by the rest of the world (in a 
>> multi-thread scenario). On the other hand, publishing a "new" object 
>> has more guarantees on who can do what with that new object (meaning 
>> it's physically not possible for other threads to start writing on 
>> the new segment before it has been returned by the API)
>>
>> Note that the original segment is killed (meaning its isAlive returns 
>> false) but the new segment still points at the same mappped memory 
>> region. So if you want a shared memory segment, you just have to do:
>>
>> MappedMemorySegment nativeFile = 
>> MemorySegment.mapFromPath(file.toPath(), 0, 4096, 
>> FileChannel.MapMode.READ_WRITE)
>> .withOwnerThread(null);
>
>
> Yeah, figured that'd work. I had to add an if-else to check if a 
> segment is already unbound, which got me past that error, although my 
> projects still aren't running because of a StackOverflow and 
> "java.lang.NoClassDefFoundError: Could not initialize class 
> java.lang.StackTraceElement$HashedModules" error related to 
> MethodHandles that are being added to struct layouts. Not 100% sure 
> what's going on there...

That looks odd - if you could send the entire stack trace we could 
perhaps quickly glance if it's coming from our API, or your code.

Cheers
Maurizio

>
>
>>
>> E.g. use chaining to get to the segment you want to construct.
>>
>> As for the interaction with asSlice() - a slice is not a true 
>> standalone segment - is just another view of the existing segment 
>> with different bounds - in fact closing the slice also closes the 
>> parent segments. So, calling withOwnerThread will also kill all 
>> associated slices (or, calling withCleanupAction will attach the 
>> cleanup action to all the segments which share the same temporal bound).
>>
>> There is, currently, no way to create a segment, and then slice it so 
>> that you can N segments each owned by a different thread, if that's 
>> what you are looking for. We looked into that and it's way too messy 
>> (not only you have to worry about how to "split" the segment, but 
>> also about how you merge it back - and that's the hard part). So, if 
>> you are not ok with confinement, create a shared segment and work 
>> with it (e.g. like it was a ByteBuffer or a Java array). It is up to 
>> you then to make sure that multiple threads operate on the segment in 
>> a way that makes sense (either by assigning disjoint slices to 
>> different threads, e.g. using a spliterator, which the API supports), 
>> or by using some synchronization.
>
>
> Figured I'd have to do manual synchronization.
>
>
>>
>> Maurizio
>>
>>
>> On 23/09/2020 07:47, sundararajan.athijegannathan at oracle.com wrote:
>>> Can you access source?
>>>
>>> https://github.com/openjdk/panama-foreign/blob/foreign-jextract/src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemorySegment.java 
>>>
>>>
>>> Yes, afaik it is implementation technical reason. (Maurizio will 
>>> clarify). That said, it should be possible to map different file 
>>> segments to *different* memory segments with different owners, 
>>> right? What's the advantage of mapping the whole & slice to own 
>>> parts by different threads?
>>>
>>> -Sundar
>>>
>>> On 23/09/20 11:02 am, Ty Young wrote:
>>>> Javadoc isn't working for withOwnerThread or withCleanupAction for 
>>>> me on Netbeans:
>>>>
>>>>
>>>> https://imgur.com/a/Q21pZAC
>>>>
>>>>
>>>> Says it's downloading HTTP Javadoc for about 10 seconds then gives 
>>>> up. Javadoc of other older methods work just fine and I didn't 
>>>> think it'd close the segment as other withers(withAccessModes) do not.
>>>>
>>>>
>>>> So I wasn't able to read it, my bad.
>>>>
>>>>
>>>> Question: is this an absolute requirement because of technical 
>>>> restrains? Could this be removed so that, for example, it is 
>>>> possible to make an array segment unbound but have individual array 
>>>> index segments be bound to a specific thread?
>>>>
>>>>
>>>> On 9/23/20 12:00 AM, sundararajan.athijegannathan at oracle.com wrote:
>>>>> javadoc of MemorySegment.withOwnerThread starts as follows:
>>>>>
>>>>> "    * Obtains a new memory segment backed by the same underlying 
>>>>> memory region as this segment,
>>>>>      * but with different owner thread. As a side-effect, this 
>>>>> segment will be marked as <em>not alive</em>,
>>>>>      * and subsequent operations on this segment will result in 
>>>>> runtime errors.
>>>>> "
>>>>>
>>>>> So the behavior seen is as the specification.
>>>>>
>>>>> -Sundar
>>>>>
>>>>> On 23/09/20 7:36 am, Ty Young wrote:
>>>>>> A bug seems to have been introduced wherein using withOwnerThread 
>>>>>> causes a MappedMemorySegment to close:
>>>>>>
>>>>>>
>>>>>>     File file = new File("./test");
>>>>>>
>>>>>>         if(file.exists())
>>>>>>             file.delete();
>>>>>>
>>>>>>         file.createNewFile();
>>>>>>
>>>>>>         MappedMemorySegment nativeFile = 
>>>>>> MemorySegment.mapFromPath(file.toPath(), 0, 4096, 
>>>>>> FileChannel.MapMode.READ_WRITE);
>>>>>>
>>>>>>         System.out.println(nativeFile.isAlive());
>>>>>>
>>>>>>         MappedMemorySegment segment = 
>>>>>> (MappedMemorySegment)nativeFile.asSlice(4);
>>>>>>         segment = (MappedMemorySegment)nativeFile.asSlice(4);
>>>>>>         segment = 
>>>>>> (MappedMemorySegment)nativeFile.asSlice(4).withOwnerThread(null);
>>>>>>
>>>>>>         System.out.println(nativeFile.isAlive());
>>>>>>         System.out.println(segment.isAlive());
>>>>>>
>>>>>>
>>>>>> which prints true, false, and then true. The original 
>>>>>> MappedMemorySegment was never closed, so this is unexpected.
>>>>>>