BUG: withOwnerThread closes MappedMemorySegment

Fri Sep 25 00:46:12 UTC 2020

On 9/24/20 4:56 AM, Maurizio Cimadamore wrote:
>
> On 24/09/2020 04:01, Ty Young wrote:
>>> There are two main issues with "just mutating the existining segments":
>>>
>>> 1) First and foremost, C2 likes constants. So, if the thread is 
>>> constant on the segment we pay nothing for the ownership (or lack of 
>>> ownership) check.
>>>
>>> 2) Second, you can't just "flip" a switch on a shared data 
>>> structure, and expect the update to be seen by the rest of the world 
>>> (in a multi-thread scenario). On the other hand, publishing a "new" 
>>> object has more guarantees on who can do what with that new object 
>>> (meaning it's physically not possible for other threads to start 
>>> writing on the new segment before it has been returned by the API)
>>>
>>> Note that the original segment is killed (meaning its isAlive 
>>> returns false) but the new segment still points at the same mappped 
>>> memory region. So if you want a shared memory segment, you just have 
>>> to do:
>>>
>>> MappedMemorySegment nativeFile = 
>>> MemorySegment.mapFromPath(file.toPath(), 0, 4096, 
>>> FileChannel.MapMode.READ_WRITE)
>>> .withOwnerThread(null);
>>
>>
>> Yeah, figured that'd work. I had to add an if-else to check if a 
>> segment is already unbound, which got me past that error, although my 
>> projects still aren't running because of a StackOverflow and 
>> "java.lang.NoClassDefFoundError: Could not initialize class 
>> java.lang.StackTraceElement$HashedModules" error related to 
>> MethodHandles that are being added to struct layouts. Not 100% sure 
>> what's going on there...
>
> That looks odd - if you could send the entire stack trace we could 
> perhaps quickly glance if it's coming from our API, or your code.

Yeah, it was my code, sorry. My application at least runs now... for 
maybe about 6-8 minutes... until it crashes.

Multiple crashes look like this:

Current thread (0x00007f17e41a4f30):  JavaThread "pool-2-thread-2" 
[_thread_in_native_trans, id=2961304, 
stack(0x00007f17d09a5000,0x00007f17d0aa6000)]

Stack: [0x00007f17d09a5000,0x00007f17d0aa6000], sp=0x00007f17d0aa47b8,  
free space=1021k
Native frames: (J=compiled Java code, A=aot compiled Java code, 
j=interpreted, Vv=VM code, C=native code)
J 9169 c2 
java.lang.invoke.LambdaForm$MH+0x0000000800eb2040.invoke(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)I 
java.base at 16-internal (89 bytes) @ 0x00007f185c85867a 
[0x00007f185c858620+0x000000000000005a]

[error occurred during error reporting (printing native stack), id 0xb, 
SIGSEGV (0xb) at pc=0x00007f1871cc7075]

Although a small percentage look like:

Current thread (0x00007f300c0c1220):  JavaThread "pool-2-thread-9" 
[_thread_in_native_trans, id=3014594, 
stack(0x00007f3001f48000,0x00007f3002049000)]

Stack: [0x00007f3001f48000,0x00007f3002049000], sp=0x00007f3002047798,  
free space=1021k
Native frames: (J=compiled Java code, A=aot compiled Java code, 
j=interpreted, Vv=VM code, C=native code)
J 2028 c2 
jdk.incubator.foreign.ValueLayout.attribute(Ljava/lang/String;)Ljava/util/Optional; 
jdk.incubator.foreign at 16-internal (6 bytes) @ 0x00007f30984f1872 
[0x00007f30984f1400+0x0000000000000472]

[error occurred during error reporting (printing native stack), id 0xb, 
SIGSEGV (0xb) at pc=0x00007f30afa54075]

Errors always seem to be in relation to compilation events:

Compiled method (c1)  357892 5293  s!   3 
org.goliath.envious.nvml.local.attributes.performance.limiters.NVMLGPUPerformanceLimitSoftwareThermalSlowdownAttribute::update 
(63 bytes)

Is there any JVM or panama debug arguments I can use to find the issue?

>
> Cheers
> Maurizio
>
>>
>>
>>>
>>> E.g. use chaining to get to the segment you want to construct.
>>>
>>> As for the interaction with asSlice() - a slice is not a true 
>>> standalone segment - is just another view of the existing segment 
>>> with different bounds - in fact closing the slice also closes the 
>>> parent segments. So, calling withOwnerThread will also kill all 
>>> associated slices (or, calling withCleanupAction will attach the 
>>> cleanup action to all the segments which share the same temporal 
>>> bound).
>>>
>>> There is, currently, no way to create a segment, and then slice it 
>>> so that you can N segments each owned by a different thread, if 
>>> that's what you are looking for. We looked into that and it's way 
>>> too messy (not only you have to worry about how to "split" the 
>>> segment, but also about how you merge it back - and that's the hard 
>>> part). So, if you are not ok with confinement, create a shared 
>>> segment and work with it (e.g. like it was a ByteBuffer or a Java 
>>> array). It is up to you then to make sure that multiple threads 
>>> operate on the segment in a way that makes sense (either by 
>>> assigning disjoint slices to different threads, e.g. using a 
>>> spliterator, which the API supports), or by using some synchronization.
>>
>>
>> Figured I'd have to do manual synchronization.
>>
>>
>>>
>>> Maurizio
>>>
>>>
>>> On 23/09/2020 07:47, sundararajan.athijegannathan at oracle.com wrote:
>>>> Can you access source?
>>>>
>>>> https://github.com/openjdk/panama-foreign/blob/foreign-jextract/src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemorySegment.java 
>>>>
>>>>
>>>> Yes, afaik it is implementation technical reason. (Maurizio will 
>>>> clarify). That said, it should be possible to map different file 
>>>> segments to *different* memory segments with different owners, 
>>>> right? What's the advantage of mapping the whole & slice to own 
>>>> parts by different threads?
>>>>
>>>> -Sundar
>>>>
>>>> On 23/09/20 11:02 am, Ty Young wrote:
>>>>> Javadoc isn't working for withOwnerThread or withCleanupAction for 
>>>>> me on Netbeans:
>>>>>
>>>>>
>>>>> https://imgur.com/a/Q21pZAC
>>>>>
>>>>>
>>>>> Says it's downloading HTTP Javadoc for about 10 seconds then gives 
>>>>> up. Javadoc of other older methods work just fine and I didn't 
>>>>> think it'd close the segment as other withers(withAccessModes) do 
>>>>> not.
>>>>>
>>>>>
>>>>> So I wasn't able to read it, my bad.
>>>>>
>>>>>
>>>>> Question: is this an absolute requirement because of technical 
>>>>> restrains? Could this be removed so that, for example, it is 
>>>>> possible to make an array segment unbound but have individual 
>>>>> array index segments be bound to a specific thread?
>>>>>
>>>>>
>>>>> On 9/23/20 12:00 AM, sundararajan.athijegannathan at oracle.com wrote:
>>>>>> javadoc of MemorySegment.withOwnerThread starts as follows:
>>>>>>
>>>>>> "    * Obtains a new memory segment backed by the same underlying 
>>>>>> memory region as this segment,
>>>>>>      * but with different owner thread. As a side-effect, this 
>>>>>> segment will be marked as <em>not alive</em>,
>>>>>>      * and subsequent operations on this segment will result in 
>>>>>> runtime errors.
>>>>>> "
>>>>>>
>>>>>> So the behavior seen is as the specification.
>>>>>>
>>>>>> -Sundar
>>>>>>
>>>>>> On 23/09/20 7:36 am, Ty Young wrote:
>>>>>>> A bug seems to have been introduced wherein using 
>>>>>>> withOwnerThread causes a MappedMemorySegment to close:
>>>>>>>
>>>>>>>
>>>>>>>     File file = new File("./test");
>>>>>>>
>>>>>>>         if(file.exists())
>>>>>>>             file.delete();
>>>>>>>
>>>>>>>         file.createNewFile();
>>>>>>>
>>>>>>>         MappedMemorySegment nativeFile = 
>>>>>>> MemorySegment.mapFromPath(file.toPath(), 0, 4096, 
>>>>>>> FileChannel.MapMode.READ_WRITE);
>>>>>>>
>>>>>>>         System.out.println(nativeFile.isAlive());
>>>>>>>
>>>>>>>         MappedMemorySegment segment = 
>>>>>>> (MappedMemorySegment)nativeFile.asSlice(4);
>>>>>>>         segment = (MappedMemorySegment)nativeFile.asSlice(4);
>>>>>>>         segment = 
>>>>>>> (MappedMemorySegment)nativeFile.asSlice(4).withOwnerThread(null);
>>>>>>>
>>>>>>>         System.out.println(nativeFile.isAlive());
>>>>>>>         System.out.println(segment.isAlive());
>>>>>>>
>>>>>>>
>>>>>>> which prints true, false, and then true. The original 
>>>>>>> MappedMemorySegment was never closed, so this is unexpected.
>>>>>>>