[External] : Re: [foreign-memaccess+abi] Integrated: 8268673: Stack walk across optimized entry frame on fresh native thread fails

Fri Jul 9 09:38:19 UTC 2021

Great!

Thanks for testing,
Jorn

On 09/07/2021 09:58, Sebastian Stenzel wrote:
> Hi Jorn,
>
> yes, I can confirm this fixes the issue.
>
> I tested on foreign-jextract, where the issue was still present 
> on 6f8f9e28c54 and fixed on 42e03fd7c6a.
>
> Cheers!
> Sebastian
>
>> On 8. Jul 2021, at 17:05, Jorn Vernee <jorn.vernee at oracle.com 
>> <mailto:jorn.vernee at oracle.com>> wrote:
>>
>> Hi Sebastian,
>>
>> I was not able to reproduce your specific problem, but I did find 
>> another problem with USE_INTRINSICS=true that seems to be a likely 
>> cause of what you were seeing. Namely: when an upcall happens on a 
>> separate native thread, the return value of the upcall was being 
>> corrupted when detaching the thread from the JVM again. So, it might 
>> be the case that the problem you were seeing was from an upcall 
>> returning an incorrect result. (I did rule out that it's a problem 
>> with the way arguments are passed).
>>
>> I've just integrated a patch that fixes the issue: 
>> https://github.com/openjdk/panama-foreign/pull/565
>>
>> If you have an opportunity to test out the fix, that would be greatly 
>> appreciated.
>>
>> Thanks,
>> Jorn
>>
>> On 15/06/2021 17:46, Jorn Vernee wrote:
>>>
>>> Ok thanks,
>>>
>>> If upcalls are involved it makes sense then that the 
>>> ProgrammableUpcallHandler.USE_INTRINSICS has an effect. Both 
>>> invocation modes should behave the same, but the implementations are 
>>> completely separate, so if setting the flag to false makes a 
>>> difference there might be a problem when passing one of the 
>>> arguments when USE_INTRINSICS=true.
>>>
>>> Thanks for testing, I'll try and reproduce the issue here as well.
>>>
>>> Jorn
>>>
>>> On 15/06/2021 17:10, Sebastian Stenzel wrote:
>>>> A reference to the first mentioned method is stored in a struct [1] 
>>>> (defined in fuse_operations.h) right here [2], which then gets 
>>>> passed to FUSE. FUSE then decides to call some of these methods 
>>>> when a corresponding file system access happens.
>>>>
>>>> [1]: 
>>>> https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/lowlevel/fuse_operations.java#L28 
>>>> <https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/lowlevel/fuse_operations.java*L28__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5TIPESgu$>
>>>> [2]: 
>>>> https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L25 
>>>> <https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java*L25__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5f8dc5As$>
>>>>
>>>>
>>>>> On 15. Jun 2021, at 16:52, Jorn Vernee <jorn.vernee at oracle.com 
>>>>> <mailto:jorn.vernee at oracle.com>> wrote:
>>>>>
>>>>> Hi Sebastian,
>>>>>
>>>>> This sounds very odd, as the code you link to doesn't seem to use 
>>>>> upcalls? The ProgrammableUpcallHandler.USE_INTRINSICS flag is only 
>>>>> used to control the invocation mode of upcalls.
>>>>>
>>>>> May I ask how exactly you're diagnosing this problem?
>>>>>
>>>>> Jorn
>>>>>
>>>>> On 15/06/2021 15:48, Sebastian Stenzel wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I have pulled this fix (foreign-jextract commit e4f89d7b0f8) and 
>>>>>> can confirm that the crashes are gone. However, copying bytes 
>>>>>> from heap to native buffers still is a little odd. Take this 
>>>>>> code: [1]; [2]
>>>>>>
>>>>>> If I run the project with 
>>>>>> `-Djdk.internal.foreign.ProgrammableUpcallHandler.USE_INTRINSICS=false`, 
>>>>>> it copies the contents of HELLO_STR to the segment. Without the 
>>>>>> flag it copies 13 null-bytes, though. 13 being the correct length 
>>>>>> of the source buffer.
>>>>>>
>>>>>> I guess this is not related to the stack walking problem. The 
>>>>>> only thing I can tell is that said flag has some influence on the 
>>>>>> behaviour.
>>>>>>
>>>>>> Cheers!
>>>>>> Sebastian
>>>>>>
>>>>>>
>>>>>> [1] my upcall method: 
>>>>>> https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L222-L227 
>>>>>> <https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java*L222-L227__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5SMpro9I$>
>>>>>> [2] method invoked during upcall, which copies bytes to the 
>>>>>> specified segment: 
>>>>>> https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java#L109-L112 
>>>>>> <https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java*L109-L112__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5ZFCsni2$>
>>>>>>
>>>>>>
>>>>>>> On 14. Jun 2021, at 17:09, Jorn Vernee <jvernee at openjdk.java.net 
>>>>>>> <mailto:jvernee at openjdk.java.net>> wrote:
>>>>>>>
>>>>>>> On Mon, 14 Jun 2021 12:13:52 GMT, Jorn Vernee 
>>>>>>> <jvernee at openjdk.org <mailto:jvernee at openjdk.org>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> When native code creates a new thread and calls a Panama 
>>>>>>>> upcall, and during that upcall a stack walk is triggered, 
>>>>>>>> getting the sender frame for the entry frame is not possible, 
>>>>>>>> and should not be attempted.
>>>>>>>>
>>>>>>>> For JNI this case is handled already by indicating the end of 
>>>>>>>> the stack frame stream, but for Panama upcalls it is not, and 
>>>>>>>> the VM will either hit an assert or crash when trying to find 
>>>>>>>> the last Java frame before the entry frame (which does not 
>>>>>>>> exist in this case).
>>>>>>>>
>>>>>>>> This patch adds handling for panama upcalls frames to 
>>>>>>>> `frame::is_first_frame`, which is used by the stack walking 
>>>>>>>> code to determine when to stop walking.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Jorn
>>>>>>>
>>>>>>> This pull request has now been integrated.
>>>>>>>
>>>>>>> Changeset: 2287ca5a
>>>>>>> Author:    Jorn Vernee <jvernee at openjdk.org 
>>>>>>> <mailto:jvernee at openjdk.org>>
>>>>>>> URL: 
>>>>>>> https://git.openjdk.java.net/panama-foreign/commit/2287ca5a530c348bc4ec7c0c6774e8ed9101dffa 
>>>>>>> <https://git.openjdk.java.net/panama-foreign/commit/2287ca5a530c348bc4ec7c0c6774e8ed9101dffa>
>>>>>>> Stats:     194 lines in 11 files changed: 193 ins; 0 del; 1 mod
>>>>>>>
>>>>>>> 8268673: Stack walk across optimized entry frame on fresh native 
>>>>>>> thread fails
>>>>>>>
>>>>>>> Reviewed-by: mcimadamore
>>>>>>>
>>>>>>> -------------
>>>>>>>
>>>>>>> PR: https://git.openjdk.java.net/panama-foreign/pull/558 
>>>>>>> <https://git.openjdk.java.net/panama-foreign/pull/558>
>>>>>>
>>>>
>