BUG: withOwnerThread closes MappedMemorySegment
Maurizio Cimadamore
maurizio.cimadamore at oracle.com
Fri Sep 25 09:58:31 UTC 2020
One idea - we have recently turned on the method handle intrinsics - try
running with these parameters:
-Djdk.internal.foreign.ProgrammableInvoker.USE_SPEC=false
-Djdk.internal.foreign.ProgrammableInvoker.USE_INTRINSICS=false
And see if the crash still happens.
Maurizio
On 25/09/2020 01:46, Ty Young wrote:
>
> On 9/24/20 4:56 AM, Maurizio Cimadamore wrote:
>>
>> On 24/09/2020 04:01, Ty Young wrote:
>>>> There are two main issues with "just mutating the existining
>>>> segments":
>>>>
>>>> 1) First and foremost, C2 likes constants. So, if the thread is
>>>> constant on the segment we pay nothing for the ownership (or lack
>>>> of ownership) check.
>>>>
>>>> 2) Second, you can't just "flip" a switch on a shared data
>>>> structure, and expect the update to be seen by the rest of the
>>>> world (in a multi-thread scenario). On the other hand, publishing a
>>>> "new" object has more guarantees on who can do what with that new
>>>> object (meaning it's physically not possible for other threads to
>>>> start writing on the new segment before it has been returned by the
>>>> API)
>>>>
>>>> Note that the original segment is killed (meaning its isAlive
>>>> returns false) but the new segment still points at the same mappped
>>>> memory region. So if you want a shared memory segment, you just
>>>> have to do:
>>>>
>>>> MappedMemorySegment nativeFile =
>>>> MemorySegment.mapFromPath(file.toPath(), 0, 4096,
>>>> FileChannel.MapMode.READ_WRITE)
>>>> .withOwnerThread(null);
>>>
>>>
>>> Yeah, figured that'd work. I had to add an if-else to check if a
>>> segment is already unbound, which got me past that error, although
>>> my projects still aren't running because of a StackOverflow and
>>> "java.lang.NoClassDefFoundError: Could not initialize class
>>> java.lang.StackTraceElement$HashedModules" error related to
>>> MethodHandles that are being added to struct layouts. Not 100% sure
>>> what's going on there...
>>
>> That looks odd - if you could send the entire stack trace we could
>> perhaps quickly glance if it's coming from our API, or your code.
>
>
> Yeah, it was my code, sorry. My application at least runs now... for
> maybe about 6-8 minutes... until it crashes.
>
>
> Multiple crashes look like this:
>
>
> Current thread (0x00007f17e41a4f30): JavaThread "pool-2-thread-2"
> [_thread_in_native_trans, id=2961304,
> stack(0x00007f17d09a5000,0x00007f17d0aa6000)]
>
> Stack: [0x00007f17d09a5000,0x00007f17d0aa6000],
> sp=0x00007f17d0aa47b8, free space=1021k
> Native frames: (J=compiled Java code, A=aot compiled Java code,
> j=interpreted, Vv=VM code, C=native code)
> J 9169 c2
> java.lang.invoke.LambdaForm$MH+0x0000000800eb2040.invoke(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)I
> java.base at 16-internal (89 bytes) @ 0x00007f185c85867a
> [0x00007f185c858620+0x000000000000005a]
>
> [error occurred during error reporting (printing native stack), id
> 0xb, SIGSEGV (0xb) at pc=0x00007f1871cc7075]
>
>
> Although a small percentage look like:
>
>
> Current thread (0x00007f300c0c1220): JavaThread "pool-2-thread-9"
> [_thread_in_native_trans, id=3014594,
> stack(0x00007f3001f48000,0x00007f3002049000)]
>
> Stack: [0x00007f3001f48000,0x00007f3002049000],
> sp=0x00007f3002047798, free space=1021k
> Native frames: (J=compiled Java code, A=aot compiled Java code,
> j=interpreted, Vv=VM code, C=native code)
> J 2028 c2
> jdk.incubator.foreign.ValueLayout.attribute(Ljava/lang/String;)Ljava/util/Optional;
> jdk.incubator.foreign at 16-internal (6 bytes) @ 0x00007f30984f1872
> [0x00007f30984f1400+0x0000000000000472]
>
> [error occurred during error reporting (printing native stack), id
> 0xb, SIGSEGV (0xb) at pc=0x00007f30afa54075]
>
>
> Errors always seem to be in relation to compilation events:
>
>
> Compiled method (c1) 357892 5293 s! 3
> org.goliath.envious.nvml.local.attributes.performance.limiters.NVMLGPUPerformanceLimitSoftwareThermalSlowdownAttribute::update
> (63 bytes)
>
>
> Is there any JVM or panama debug arguments I can use to find the issue?
>
>
>
>
>
>
>
>>
>> Cheers
>> Maurizio
>>
>>>
>>>
>>>>
>>>> E.g. use chaining to get to the segment you want to construct.
>>>>
>>>> As for the interaction with asSlice() - a slice is not a true
>>>> standalone segment - is just another view of the existing segment
>>>> with different bounds - in fact closing the slice also closes the
>>>> parent segments. So, calling withOwnerThread will also kill all
>>>> associated slices (or, calling withCleanupAction will attach the
>>>> cleanup action to all the segments which share the same temporal
>>>> bound).
>>>>
>>>> There is, currently, no way to create a segment, and then slice it
>>>> so that you can N segments each owned by a different thread, if
>>>> that's what you are looking for. We looked into that and it's way
>>>> too messy (not only you have to worry about how to "split" the
>>>> segment, but also about how you merge it back - and that's the hard
>>>> part). So, if you are not ok with confinement, create a shared
>>>> segment and work with it (e.g. like it was a ByteBuffer or a Java
>>>> array). It is up to you then to make sure that multiple threads
>>>> operate on the segment in a way that makes sense (either by
>>>> assigning disjoint slices to different threads, e.g. using a
>>>> spliterator, which the API supports), or by using some
>>>> synchronization.
>>>
>>>
>>> Figured I'd have to do manual synchronization.
>>>
>>>
>>>>
>>>> Maurizio
>>>>
>>>>
>>>> On 23/09/2020 07:47, sundararajan.athijegannathan at oracle.com wrote:
>>>>> Can you access source?
>>>>>
>>>>> https://urldefense.com/v3/__https://github.com/openjdk/panama-foreign/blob/foreign-jextract/src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemorySegment.java__;!!GqivPVa7Brio!OlbtDRpO3iKUiqjySSQELP4SjIC_dCXbKMZ344YiK9XD92Rg8n6jxzYxOkrj_IvkZ8Tlf78$
>>>>>
>>>>>
>>>>> Yes, afaik it is implementation technical reason. (Maurizio will
>>>>> clarify). That said, it should be possible to map different file
>>>>> segments to *different* memory segments with different owners,
>>>>> right? What's the advantage of mapping the whole & slice to own
>>>>> parts by different threads?
>>>>>
>>>>> -Sundar
>>>>>
>>>>> On 23/09/20 11:02 am, Ty Young wrote:
>>>>>> Javadoc isn't working for withOwnerThread or withCleanupAction
>>>>>> for me on Netbeans:
>>>>>>
>>>>>>
>>>>>> https://urldefense.com/v3/__https://imgur.com/a/Q21pZAC__;!!GqivPVa7Brio!OlbtDRpO3iKUiqjySSQELP4SjIC_dCXbKMZ344YiK9XD92Rg8n6jxzYxOkrj_IvkrrLx8VA$
>>>>>>
>>>>>>
>>>>>> Says it's downloading HTTP Javadoc for about 10 seconds then
>>>>>> gives up. Javadoc of other older methods work just fine and I
>>>>>> didn't think it'd close the segment as other
>>>>>> withers(withAccessModes) do not.
>>>>>>
>>>>>>
>>>>>> So I wasn't able to read it, my bad.
>>>>>>
>>>>>>
>>>>>> Question: is this an absolute requirement because of technical
>>>>>> restrains? Could this be removed so that, for example, it is
>>>>>> possible to make an array segment unbound but have individual
>>>>>> array index segments be bound to a specific thread?
>>>>>>
>>>>>>
>>>>>> On 9/23/20 12:00 AM, sundararajan.athijegannathan at oracle.com wrote:
>>>>>>> javadoc of MemorySegment.withOwnerThread starts as follows:
>>>>>>>
>>>>>>> " * Obtains a new memory segment backed by the same
>>>>>>> underlying memory region as this segment,
>>>>>>> * but with different owner thread. As a side-effect, this
>>>>>>> segment will be marked as <em>not alive</em>,
>>>>>>> * and subsequent operations on this segment will result in
>>>>>>> runtime errors.
>>>>>>> "
>>>>>>>
>>>>>>> So the behavior seen is as the specification.
>>>>>>>
>>>>>>> -Sundar
>>>>>>>
>>>>>>> On 23/09/20 7:36 am, Ty Young wrote:
>>>>>>>> A bug seems to have been introduced wherein using
>>>>>>>> withOwnerThread causes a MappedMemorySegment to close:
>>>>>>>>
>>>>>>>>
>>>>>>>> File file = new File("./test");
>>>>>>>>
>>>>>>>> if(file.exists())
>>>>>>>> file.delete();
>>>>>>>>
>>>>>>>> file.createNewFile();
>>>>>>>>
>>>>>>>> MappedMemorySegment nativeFile =
>>>>>>>> MemorySegment.mapFromPath(file.toPath(), 0, 4096,
>>>>>>>> FileChannel.MapMode.READ_WRITE);
>>>>>>>>
>>>>>>>> System.out.println(nativeFile.isAlive());
>>>>>>>>
>>>>>>>> MappedMemorySegment segment =
>>>>>>>> (MappedMemorySegment)nativeFile.asSlice(4);
>>>>>>>> segment = (MappedMemorySegment)nativeFile.asSlice(4);
>>>>>>>> segment =
>>>>>>>> (MappedMemorySegment)nativeFile.asSlice(4).withOwnerThread(null);
>>>>>>>>
>>>>>>>> System.out.println(nativeFile.isAlive());
>>>>>>>> System.out.println(segment.isAlive());
>>>>>>>>
>>>>>>>>
>>>>>>>> which prints true, false, and then true. The original
>>>>>>>> MappedMemorySegment was never closed, so this is unexpected.
>>>>>>>>
More information about the panama-dev
mailing list