BUG: withOwnerThread closes MappedMemorySegment

Ty Young youngty1997 at gmail.com
Fri Sep 25 13:33:57 UTC 2020 

On 9/25/20 4:58 AM, Maurizio Cimadamore wrote:
> One idea - we have recently turned on the method handle intrinsics - 
> try running with these parameters:
>
> -Djdk.internal.foreign.ProgrammableInvoker.USE_SPEC=false 
> -Djdk.internal.foreign.ProgrammableInvoker.USE_INTRINSICS=false
>
> And see if the crash still happens.


Seems like that is the issue. I can't say 100% because there are 
run-to-run outliers where it'll sometimes get past the 6-8 minute mark 
and run for awhile, only to eventually crash. Nothing yet with those 
turned off though.


>
> Maurizio
>
> On 25/09/2020 01:46, Ty Young wrote:
>>
>> On 9/24/20 4:56 AM, Maurizio Cimadamore wrote:
>>>
>>> On 24/09/2020 04:01, Ty Young wrote:
>>>>> There are two main issues with "just mutating the existining 
>>>>> segments":
>>>>>
>>>>> 1) First and foremost, C2 likes constants. So, if the thread is 
>>>>> constant on the segment we pay nothing for the ownership (or lack 
>>>>> of ownership) check.
>>>>>
>>>>> 2) Second, you can't just "flip" a switch on a shared data 
>>>>> structure, and expect the update to be seen by the rest of the 
>>>>> world (in a multi-thread scenario). On the other hand, publishing 
>>>>> a "new" object has more guarantees on who can do what with that 
>>>>> new object (meaning it's physically not possible for other threads 
>>>>> to start writing on the new segment before it has been returned by 
>>>>> the API)
>>>>>
>>>>> Note that the original segment is killed (meaning its isAlive 
>>>>> returns false) but the new segment still points at the same 
>>>>> mappped memory region. So if you want a shared memory segment, you 
>>>>> just have to do:
>>>>>
>>>>> MappedMemorySegment nativeFile = 
>>>>> MemorySegment.mapFromPath(file.toPath(), 0, 4096, 
>>>>> FileChannel.MapMode.READ_WRITE)
>>>>> .withOwnerThread(null);
>>>>
>>>>
>>>> Yeah, figured that'd work. I had to add an if-else to check if a 
>>>> segment is already unbound, which got me past that error, although 
>>>> my projects still aren't running because of a StackOverflow and 
>>>> "java.lang.NoClassDefFoundError: Could not initialize class 
>>>> java.lang.StackTraceElement$HashedModules" error related to 
>>>> MethodHandles that are being added to struct layouts. Not 100% sure 
>>>> what's going on there...
>>>
>>> That looks odd - if you could send the entire stack trace we could 
>>> perhaps quickly glance if it's coming from our API, or your code.
>>
>>
>> Yeah, it was my code, sorry. My application at least runs now... for 
>> maybe about 6-8 minutes... until it crashes.
>>
>>
>> Multiple crashes look like this:
>>
>>
>> Current thread (0x00007f17e41a4f30):  JavaThread "pool-2-thread-2" 
>> [_thread_in_native_trans, id=2961304, 
>> stack(0x00007f17d09a5000,0x00007f17d0aa6000)]
>>
>> Stack: [0x00007f17d09a5000,0x00007f17d0aa6000], 
>> sp=0x00007f17d0aa47b8,  free space=1021k
>> Native frames: (J=compiled Java code, A=aot compiled Java code, 
>> j=interpreted, Vv=VM code, C=native code)
>> J 9169 c2 
>> java.lang.invoke.LambdaForm$MH+0x0000000800eb2040.invoke(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)I 
>> java.base at 16-internal (89 bytes) @ 0x00007f185c85867a 
>> [0x00007f185c858620+0x000000000000005a]
>>
>> [error occurred during error reporting (printing native stack), id 
>> 0xb, SIGSEGV (0xb) at pc=0x00007f1871cc7075]
>>
>>
>> Although a small percentage look like:
>>
>>
>> Current thread (0x00007f300c0c1220):  JavaThread "pool-2-thread-9" 
>> [_thread_in_native_trans, id=3014594, 
>> stack(0x00007f3001f48000,0x00007f3002049000)]
>>
>> Stack: [0x00007f3001f48000,0x00007f3002049000], 
>> sp=0x00007f3002047798,  free space=1021k
>> Native frames: (J=compiled Java code, A=aot compiled Java code, 
>> j=interpreted, Vv=VM code, C=native code)
>> J 2028 c2 
>> jdk.incubator.foreign.ValueLayout.attribute(Ljava/lang/String;)Ljava/util/Optional; 
>> jdk.incubator.foreign at 16-internal (6 bytes) @ 0x00007f30984f1872 
>> [0x00007f30984f1400+0x0000000000000472]
>>
>> [error occurred during error reporting (printing native stack), id 
>> 0xb, SIGSEGV (0xb) at pc=0x00007f30afa54075]
>>
>>
>> Errors always seem to be in relation to compilation events:
>>
>>
>> Compiled method (c1)  357892 5293  s!   3 
>> org.goliath.envious.nvml.local.attributes.performance.limiters.NVMLGPUPerformanceLimitSoftwareThermalSlowdownAttribute::update 
>> (63 bytes)
>>
>>
>> Is there any JVM or panama debug arguments I can use to find the issue?
>>
>>
>>
>>
>>
>>
>>
>>>
>>> Cheers
>>> Maurizio
>>>
>>>>
>>>>
>>>>>
>>>>> E.g. use chaining to get to the segment you want to construct.
>>>>>
>>>>> As for the interaction with asSlice() - a slice is not a true 
>>>>> standalone segment - is just another view of the existing segment 
>>>>> with different bounds - in fact closing the slice also closes the 
>>>>> parent segments. So, calling withOwnerThread will also kill all 
>>>>> associated slices (or, calling withCleanupAction will attach the 
>>>>> cleanup action to all the segments which share the same temporal 
>>>>> bound).
>>>>>
>>>>> There is, currently, no way to create a segment, and then slice it 
>>>>> so that you can N segments each owned by a different thread, if 
>>>>> that's what you are looking for. We looked into that and it's way 
>>>>> too messy (not only you have to worry about how to "split" the 
>>>>> segment, but also about how you merge it back - and that's the 
>>>>> hard part). So, if you are not ok with confinement, create a 
>>>>> shared segment and work with it (e.g. like it was a ByteBuffer or 
>>>>> a Java array). It is up to you then to make sure that multiple 
>>>>> threads operate on the segment in a way that makes sense (either 
>>>>> by assigning disjoint slices to different threads, e.g. using a 
>>>>> spliterator, which the API supports), or by using some 
>>>>> synchronization.
>>>>
>>>>
>>>> Figured I'd have to do manual synchronization.
>>>>
>>>>
>>>>>
>>>>> Maurizio
>>>>>
>>>>>
>>>>> On 23/09/2020 07:47, sundararajan.athijegannathan at oracle.com wrote:
>>>>>> Can you access source?
>>>>>>
>>>>>> https://urldefense.com/v3/__https://github.com/openjdk/panama-foreign/blob/foreign-jextract/src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemorySegment.java__;!!GqivPVa7Brio!OlbtDRpO3iKUiqjySSQELP4SjIC_dCXbKMZ344YiK9XD92Rg8n6jxzYxOkrj_IvkZ8Tlf78$ 
>>>>>>
>>>>>>
>>>>>> Yes, afaik it is implementation technical reason. (Maurizio will 
>>>>>> clarify). That said, it should be possible to map different file 
>>>>>> segments to *different* memory segments with different owners, 
>>>>>> right? What's the advantage of mapping the whole & slice to own 
>>>>>> parts by different threads?
>>>>>>
>>>>>> -Sundar
>>>>>>
>>>>>> On 23/09/20 11:02 am, Ty Young wrote:
>>>>>>> Javadoc isn't working for withOwnerThread or withCleanupAction 
>>>>>>> for me on Netbeans:
>>>>>>>
>>>>>>>
>>>>>>> https://urldefense.com/v3/__https://imgur.com/a/Q21pZAC__;!!GqivPVa7Brio!OlbtDRpO3iKUiqjySSQELP4SjIC_dCXbKMZ344YiK9XD92Rg8n6jxzYxOkrj_IvkrrLx8VA$ 
>>>>>>>
>>>>>>>
>>>>>>> Says it's downloading HTTP Javadoc for about 10 seconds then 
>>>>>>> gives up. Javadoc of other older methods work just fine and I 
>>>>>>> didn't think it'd close the segment as other 
>>>>>>> withers(withAccessModes) do not.
>>>>>>>
>>>>>>>
>>>>>>> So I wasn't able to read it, my bad.
>>>>>>>
>>>>>>>
>>>>>>> Question: is this an absolute requirement because of technical 
>>>>>>> restrains? Could this be removed so that, for example, it is 
>>>>>>> possible to make an array segment unbound but have individual 
>>>>>>> array index segments be bound to a specific thread?
>>>>>>>
>>>>>>>
>>>>>>> On 9/23/20 12:00 AM, sundararajan.athijegannathan at oracle.com wrote:
>>>>>>>> javadoc of MemorySegment.withOwnerThread starts as follows:
>>>>>>>>
>>>>>>>> "    * Obtains a new memory segment backed by the same 
>>>>>>>> underlying memory region as this segment,
>>>>>>>>      * but with different owner thread. As a side-effect, this 
>>>>>>>> segment will be marked as <em>not alive</em>,
>>>>>>>>      * and subsequent operations on this segment will result in 
>>>>>>>> runtime errors.
>>>>>>>> "
>>>>>>>>
>>>>>>>> So the behavior seen is as the specification.
>>>>>>>>
>>>>>>>> -Sundar
>>>>>>>>
>>>>>>>> On 23/09/20 7:36 am, Ty Young wrote:
>>>>>>>>> A bug seems to have been introduced wherein using 
>>>>>>>>> withOwnerThread causes a MappedMemorySegment to close:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>     File file = new File("./test");
>>>>>>>>>
>>>>>>>>>         if(file.exists())
>>>>>>>>>             file.delete();
>>>>>>>>>
>>>>>>>>>         file.createNewFile();
>>>>>>>>>
>>>>>>>>>         MappedMemorySegment nativeFile = 
>>>>>>>>> MemorySegment.mapFromPath(file.toPath(), 0, 4096, 
>>>>>>>>> FileChannel.MapMode.READ_WRITE);
>>>>>>>>>
>>>>>>>>>         System.out.println(nativeFile.isAlive());
>>>>>>>>>
>>>>>>>>>         MappedMemorySegment segment = 
>>>>>>>>> (MappedMemorySegment)nativeFile.asSlice(4);
>>>>>>>>>         segment = (MappedMemorySegment)nativeFile.asSlice(4);
>>>>>>>>>         segment = 
>>>>>>>>> (MappedMemorySegment)nativeFile.asSlice(4).withOwnerThread(null);
>>>>>>>>>
>>>>>>>>>         System.out.println(nativeFile.isAlive());
>>>>>>>>>         System.out.println(segment.isAlive());
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> which prints true, false, and then true. The original 
>>>>>>>>> MappedMemorySegment was never closed, so this is unexpected.
>>>>>>>>>

More information about the panama-dev mailing list