[External] : Re: [foreign-memaccess+abi] Integrated: 8268673: Stack walk across optimized entry frame on fresh native thread fails

Thu Jul 8 15:05:22 UTC 2021

Hi Sebastian,

I was not able to reproduce your specific problem, but I did find 
another problem with USE_INTRINSICS=true that seems to be a likely cause 
of what you were seeing. Namely: when an upcall happens on a separate 
native thread, the return value of the upcall was being corrupted when 
detaching the thread from the JVM again. So, it might be the case that 
the problem you were seeing was from an upcall returning an incorrect 
result. (I did rule out that it's a problem with the way arguments are 
passed).

I've just integrated a patch that fixes the issue: 
https://github.com/openjdk/panama-foreign/pull/565

If you have an opportunity to test out the fix, that would be greatly 
appreciated.

Thanks,
Jorn

On 15/06/2021 17:46, Jorn Vernee wrote:
>
> Ok thanks,
>
> If upcalls are involved it makes sense then that the 
> ProgrammableUpcallHandler.USE_INTRINSICS has an effect. Both 
> invocation modes should behave the same, but the implementations are 
> completely separate, so if setting the flag to false makes a 
> difference there might be a problem when passing one of the arguments 
> when USE_INTRINSICS=true.
>
> Thanks for testing, I'll try and reproduce the issue here as well.
>
> Jorn
>
> On 15/06/2021 17:10, Sebastian Stenzel wrote:
>> A reference to the first mentioned method is stored in a struct [1] 
>> (defined in fuse_operations.h) right here [2], which then gets passed 
>> to FUSE. FUSE then decides to call some of these methods when a 
>> corresponding file system access happens.
>>
>> [1]: 
>> https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/lowlevel/fuse_operations.java#L28 
>> <https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/lowlevel/fuse_operations.java*L28__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5TIPESgu$>
>> [2]: 
>> https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L25 
>> <https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java*L25__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5f8dc5As$>
>>
>>
>>> On 15. Jun 2021, at 16:52, Jorn Vernee <jorn.vernee at oracle.com 
>>> <mailto:jorn.vernee at oracle.com>> wrote:
>>>
>>> Hi Sebastian,
>>>
>>> This sounds very odd, as the code you link to doesn't seem to use 
>>> upcalls? The ProgrammableUpcallHandler.USE_INTRINSICS flag is only 
>>> used to control the invocation mode of upcalls.
>>>
>>> May I ask how exactly you're diagnosing this problem?
>>>
>>> Jorn
>>>
>>> On 15/06/2021 15:48, Sebastian Stenzel wrote:
>>>> Hi,
>>>>
>>>> I have pulled this fix (foreign-jextract commit e4f89d7b0f8) and 
>>>> can confirm that the crashes are gone. However, copying bytes from 
>>>> heap to native buffers still is a little odd. Take this code: [1]; [2]
>>>>
>>>> If I run the project with 
>>>> `-Djdk.internal.foreign.ProgrammableUpcallHandler.USE_INTRINSICS=false`, 
>>>> it copies the contents of HELLO_STR to the segment. Without the 
>>>> flag it copies 13 null-bytes, though. 13 being the correct length 
>>>> of the source buffer.
>>>>
>>>> I guess this is not related to the stack walking problem. The only 
>>>> thing I can tell is that said flag has some influence on the behaviour.
>>>>
>>>> Cheers!
>>>> Sebastian
>>>>
>>>>
>>>> [1] my upcall method: 
>>>> https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L222-L227 
>>>> <https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java*L222-L227__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5SMpro9I$>
>>>> [2] method invoked during upcall, which copies bytes to the 
>>>> specified segment: 
>>>> https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java#L109-L112 
>>>> <https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java*L109-L112__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5ZFCsni2$>
>>>>
>>>>
>>>>> On 14. Jun 2021, at 17:09, Jorn Vernee <jvernee at openjdk.java.net 
>>>>> <mailto:jvernee at openjdk.java.net>> wrote:
>>>>>
>>>>> On Mon, 14 Jun 2021 12:13:52 GMT, Jorn Vernee <jvernee at openjdk.org 
>>>>> <mailto:jvernee at openjdk.org>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> When native code creates a new thread and calls a Panama upcall, 
>>>>>> and during that upcall a stack walk is triggered, getting the 
>>>>>> sender frame for the entry frame is not possible, and should not 
>>>>>> be attempted.
>>>>>>
>>>>>> For JNI this case is handled already by indicating the end of the 
>>>>>> stack frame stream, but for Panama upcalls it is not, and the VM 
>>>>>> will either hit an assert or crash when trying to find the last 
>>>>>> Java frame before the entry frame (which does not exist in this 
>>>>>> case).
>>>>>>
>>>>>> This patch adds handling for panama upcalls frames to 
>>>>>> `frame::is_first_frame`, which is used by the stack walking code 
>>>>>> to determine when to stop walking.
>>>>>>
>>>>>> Thanks,
>>>>>> Jorn
>>>>>
>>>>> This pull request has now been integrated.
>>>>>
>>>>> Changeset: 2287ca5a
>>>>> Author:    Jorn Vernee <jvernee at openjdk.org 
>>>>> <mailto:jvernee at openjdk.org>>
>>>>> URL: 
>>>>> https://git.openjdk.java.net/panama-foreign/commit/2287ca5a530c348bc4ec7c0c6774e8ed9101dffa 
>>>>> <https://git.openjdk.java.net/panama-foreign/commit/2287ca5a530c348bc4ec7c0c6774e8ed9101dffa>
>>>>> Stats:     194 lines in 11 files changed: 193 ins; 0 del; 1 mod
>>>>>
>>>>> 8268673: Stack walk across optimized entry frame on fresh native 
>>>>> thread fails
>>>>>
>>>>> Reviewed-by: mcimadamore
>>>>>
>>>>> -------------
>>>>>
>>>>> PR: https://git.openjdk.java.net/panama-foreign/pull/558 
>>>>> <https://git.openjdk.java.net/panama-foreign/pull/558>
>>>>
>>