[foreign-memaccess+abi] RFR: 8270851: Logic for attaching/detaching native threads could be improved

[David Holmes]

Correction ...

On 27/07/2021 7:29 am, David Holmes wrote:
> On 23/07/2021 9:17 pm, Maurizio Cimadamore wrote:
>> On Fri, 23 Jul 2021 05:44:36 GMT, David Holmes <dholmes at openjdk.org> 
>> wrote:
>>
>>>   don't recall what the panama programming model is in regard to 
>>> application use of native threads and the use of upcalls, but perhaps 
>>> for the case at hand when threads are making many upcalls and the 
>>> attach/detach overhead is significant, then the application should be 
>>> responsible for attaching and later detaching these threads, with the 
>>> upcall mechanism checking for an already attached thread?
>>
>> The main issue with putting developers in charge of deciding when/how 
>> to attach/detach is that, unlike with JNI, where user code runs 
>> (albeit in C/C++) before/after the upcall (so we have a chance to 
>> attach/detach things), in Panama the code doing the upcall is some 
>> unknowing piece of native code calling a function pointer. There's 
>> nothing we can inject in the library code, of course, as that code is 
>> fixed - so the only thing we can change is what happens when the 
>> upcall starts/finishes.
> 
> Okay. I think we discussed this when the work was first reviewed to go 
> into the JDK as a preview feature (but I can't find those discussions :( 
> ). The automated attach/detach for the upcalls, is really the only 
> "correct" way to handle this - but it does involve a lot of overhead.
> 
>> I have few questions for you though.
>>
>> 1. What happens if an attached thread is never deattached?
> 
> First it will leak some VM resources.
> 
> But more significantly, because of the TLS hack we had to add to support 
> detaching from a TLS key destructor, it may hang, or trigger a pthread 
> library error, when it terminates. We generally treat a native thread 
> failing to detach as a programming error - "don't do that" - but of 
> course the JNI spec says nothing about this.

FWIW this is not an issue on modern Linux with glibc or macOS. We 
actually have a test that runs this scenario and attempts to perform all 
the available thread operations on the errant thread to ensure no 
crashes. But we don't run that test on Windows (simply because the test 
was a regression test for a Posix system and it uses native code, and we 
didn't write the Windows version ... I guess I should do that and test 
it out).

David
-----



> Also note that if a native thread terminates when it still has active 
> Java frames then that could potentially lead to a range of problems - 
> but I don't think you envisage needing to try and support this mode of 
> operation.
> 
>> 2. We can imagine we have an API for attaching threads, exposed by 
>> CLinker:
>>
>>
>> boolean attachThread(Thread t, boolean daemon, ResourceScope)
>>
>>
>> This will attach a thread (if needed) to the JVM (either in daemon 
>> mode, or normal), and will register that onto a ResourceScope, meaning 
>> that when the ResourceScope is closed, the thread will be deattached.
>>
>> This API is not impossible to do - what I'd like to understand is what 
>> happens if we jump into the upcall code _before_ we have attached the 
>> thread to the JVM. Is that a correctness issue? If not, we could go 
>> down that path - but if a native thread can _never ever_ execute a 
>> single line of Java code before being attached to the VM, well, that 
>> creates a chicken and egg problem, I think.
> 
> A non-attached thread cannot execute any Java code, so you cannot 
> provide a Java API to do the attaching. The fact the thread is not 
> attached would/should be trapped at the VM entry point and may even 
> cause a guarantee() failure.
> 
> Cheers,
> David
> 
>> -------------
>>
>> PR: https://git.openjdk.java.net/panama-foreign/pull/570
>>