[External] : Re: [foreign-memaccess+abi] Integrated: 8268673: Stack walk across optimized entry frame on fresh native thread fails

Fri Jul 9 13:55:12 UTC 2021

Yes, we are in the process of doing that, but currently a lot of the 
reviewers are on vacation :)

Jorn

On 09/07/2021 14:27, Sebastian Stenzel wrote:
> Btw: Any chance this will get backported toJDK-17?
>
>> Am 09.07.2021 um 11:39 schrieb Jorn Vernee <jorn.vernee at oracle.com>:
>>
>> 
>>
>> Great!
>>
>> Thanks for testing,
>> Jorn
>>
>> On 09/07/2021 09:58, Sebastian Stenzel wrote:
>>> Hi Jorn,
>>>
>>> yes, I can confirm this fixes the issue.
>>>
>>> I tested on foreign-jextract, where the issue was still present 
>>> on 6f8f9e28c54 and fixed on 42e03fd7c6a.
>>>
>>> Cheers!
>>> Sebastian
>>>
>>>> On 8. Jul 2021, at 17:05, Jorn Vernee <jorn.vernee at oracle.com 
>>>> <mailto:jorn.vernee at oracle.com>> wrote:
>>>>
>>>> Hi Sebastian,
>>>>
>>>> I was not able to reproduce your specific problem, but I did find 
>>>> another problem with USE_INTRINSICS=true that seems to be a likely 
>>>> cause of what you were seeing. Namely: when an upcall happens on a 
>>>> separate native thread, the return value of the upcall was being 
>>>> corrupted when detaching the thread from the JVM again. So, it 
>>>> might be the case that the problem you were seeing was from an 
>>>> upcall returning an incorrect result. (I did rule out that it's a 
>>>> problem with the way arguments are passed).
>>>>
>>>> I've just integrated a patch that fixes the issue: 
>>>> https://github.com/openjdk/panama-foreign/pull/565
>>>>
>>>> If you have an opportunity to test out the fix, that would be 
>>>> greatly appreciated.
>>>>
>>>> Thanks,
>>>> Jorn
>>>>
>>>> On 15/06/2021 17:46, Jorn Vernee wrote:
>>>>>
>>>>> Ok thanks,
>>>>>
>>>>> If upcalls are involved it makes sense then that the 
>>>>> ProgrammableUpcallHandler.USE_INTRINSICS has an effect. Both 
>>>>> invocation modes should behave the same, but the implementations 
>>>>> are completely separate, so if setting the flag to false makes a 
>>>>> difference there might be a problem when passing one of the 
>>>>> arguments when USE_INTRINSICS=true.
>>>>>
>>>>> Thanks for testing, I'll try and reproduce the issue here as well.
>>>>>
>>>>> Jorn
>>>>>
>>>>> On 15/06/2021 17:10, Sebastian Stenzel wrote:
>>>>>> A reference to the first mentioned method is stored in a struct 
>>>>>> [1] (defined in fuse_operations.h) right here [2], which then 
>>>>>> gets passed to FUSE. FUSE then decides to call some of these 
>>>>>> methods when a corresponding file system access happens.
>>>>>>
>>>>>> [1]: 
>>>>>> https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/lowlevel/fuse_operations.java#L28 
>>>>>> <https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/lowlevel/fuse_operations.java*L28__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5TIPESgu$>
>>>>>> [2]: 
>>>>>> https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L25 
>>>>>> <https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java*L25__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5f8dc5As$>
>>>>>>
>>>>>>
>>>>>>> On 15. Jun 2021, at 16:52, Jorn Vernee <jorn.vernee at oracle.com 
>>>>>>> <mailto:jorn.vernee at oracle.com>> wrote:
>>>>>>>
>>>>>>> Hi Sebastian,
>>>>>>>
>>>>>>> This sounds very odd, as the code you link to doesn't seem to 
>>>>>>> use upcalls? The ProgrammableUpcallHandler.USE_INTRINSICS flag 
>>>>>>> is only used to control the invocation mode of upcalls.
>>>>>>>
>>>>>>> May I ask how exactly you're diagnosing this problem?
>>>>>>>
>>>>>>> Jorn
>>>>>>>
>>>>>>> On 15/06/2021 15:48, Sebastian Stenzel wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I have pulled this fix (foreign-jextract commit e4f89d7b0f8) 
>>>>>>>> and can confirm that the crashes are gone. However, copying 
>>>>>>>> bytes from heap to native buffers still is a little odd. Take 
>>>>>>>> this code: [1]; [2]
>>>>>>>>
>>>>>>>> If I run the project with 
>>>>>>>> `-Djdk.internal.foreign.ProgrammableUpcallHandler.USE_INTRINSICS=false`, 
>>>>>>>> it copies the contents of HELLO_STR to the segment. Without the 
>>>>>>>> flag it copies 13 null-bytes, though. 13 being the correct 
>>>>>>>> length of the source buffer.
>>>>>>>>
>>>>>>>> I guess this is not related to the stack walking problem. The 
>>>>>>>> only thing I can tell is that said flag has some influence on 
>>>>>>>> the behaviour.
>>>>>>>>
>>>>>>>> Cheers!
>>>>>>>> Sebastian
>>>>>>>>
>>>>>>>>
>>>>>>>> [1] my upcall method: 
>>>>>>>> https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L222-L227 
>>>>>>>> <https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java*L222-L227__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5SMpro9I$>
>>>>>>>> [2] method invoked during upcall, which copies bytes to the 
>>>>>>>> specified segment: 
>>>>>>>> https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java#L109-L112 
>>>>>>>> <https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java*L109-L112__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5ZFCsni2$>
>>>>>>>>
>>>>>>>>
>>>>>>>>> On 14. Jun 2021, at 17:09, Jorn Vernee 
>>>>>>>>> <jvernee at openjdk.java.net <mailto:jvernee at openjdk.java.net>> 
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> On Mon, 14 Jun 2021 12:13:52 GMT, Jorn Vernee 
>>>>>>>>> <jvernee at openjdk.org <mailto:jvernee at openjdk.org>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> When native code creates a new thread and calls a Panama 
>>>>>>>>>> upcall, and during that upcall a stack walk is triggered, 
>>>>>>>>>> getting the sender frame for the entry frame is not possible, 
>>>>>>>>>> and should not be attempted.
>>>>>>>>>>
>>>>>>>>>> For JNI this case is handled already by indicating the end of 
>>>>>>>>>> the stack frame stream, but for Panama upcalls it is not, and 
>>>>>>>>>> the VM will either hit an assert or crash when trying to find 
>>>>>>>>>> the last Java frame before the entry frame (which does not 
>>>>>>>>>> exist in this case).
>>>>>>>>>>
>>>>>>>>>> This patch adds handling for panama upcalls frames to 
>>>>>>>>>> `frame::is_first_frame`, which is used by the stack walking 
>>>>>>>>>> code to determine when to stop walking.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Jorn
>>>>>>>>>
>>>>>>>>> This pull request has now been integrated.
>>>>>>>>>
>>>>>>>>> Changeset: 2287ca5a
>>>>>>>>> Author:    Jorn Vernee <jvernee at openjdk.org 
>>>>>>>>> <mailto:jvernee at openjdk.org>>
>>>>>>>>> URL: 
>>>>>>>>> https://git.openjdk.java.net/panama-foreign/commit/2287ca5a530c348bc4ec7c0c6774e8ed9101dffa 
>>>>>>>>> <https://git.openjdk.java.net/panama-foreign/commit/2287ca5a530c348bc4ec7c0c6774e8ed9101dffa>
>>>>>>>>> Stats:     194 lines in 11 files changed: 193 ins; 0 del; 1 mod
>>>>>>>>>
>>>>>>>>> 8268673: Stack walk across optimized entry frame on fresh 
>>>>>>>>> native thread fails
>>>>>>>>>
>>>>>>>>> Reviewed-by: mcimadamore
>>>>>>>>>
>>>>>>>>> -------------
>>>>>>>>>
>>>>>>>>> PR: https://git.openjdk.java.net/panama-foreign/pull/558 
>>>>>>>>> <https://git.openjdk.java.net/panama-foreign/pull/558>
>>>>>>>>
>>>>>>
>>>