JNI DetachCurrentThread() usage during JVM process exit

[David Holmes]

Hi Darryl,

On 16/10/2012 10:57 PM, Darryl L. Miles wrote:
> I am trying to understand the reasons why the JNI method
> DetachCurrentThread() blocks the thread in VM_Exit::wait_if_vm_exited()
> instead of returning control back to JNI during process exit proceedings.

The exit procedure tries to bring everything to a quiescent state. The 
thread doing the exit will wait a short while for threads in native to 
try and return into the VM. After that any thread trying to return into 
the VM is simply blocked - if the wait_if_vm_exited didn't do it then 
the state transition to _thread_in_vm would.

> This is particular problematic when the thread in question is actually
> owned by JNI (because it started out life with JNI using
> pthread_create() directly).

That thread is racing with the process being blown away. Even if the 
detach did a quick-exit (without actually detaching - which would 
violate it's specification!) there is absolutely no guarantee that the 
thread would be able to complete any clean up before process 
termination. I think it is more likely that any attempted cleanup would 
be incomplete - which might be worse than not attempting the cleanup at all.

> Sequence of events:
>
> * java.exe starts up
> * JAR loaded and JNI *.so loaded
> * addShutdownHook() thread added
> * application uses/implements JNI based thread pool, this creates a
> thread, that does work, completes and stays idle in pool. During the
> time of doing work it performs a JNI AttachCurrentThreadAsDaemon() so it
> can call into Java. Not all JNI threads created call into Java, they are
> attached to VM on their first invocation into Java and remain attached.
> If the thread pool prunes (terminates) idle workers, those threads will
> call DetachCurrentThread() if they had been attached before the worker
> thread exits.
> * The Application decides to exit, the JNI based thread pool has no
> active threads but may have management thread and one or more workers
> that are idle in the pool.
> * System.exit() is called.
> * addShutdownHook() thread started and run
> * From the shutdownHook thread this instructs the JNI based thread pool
> to close, during this close each worker thread that did an
> 'AttachCurrentThreadAsDaemon()' will perform a 'DetachCurrentThread()'.
> These threads hang because the JVM never returns control back to JNI.

At the time the shutdown hooks are run the exit procedure is still in 
Java code, we haven't called into the VM to do the exit and 
VM_Exit::block_if_vm_exited() will not block. Have you got native stack 
traces showing that the JNI threads are in fact blocked here? Something 
seems amiss.

David
-----

> These threads are owned by JNI (not by Java) so they may have further
> clean up to do and will then exit if the DetachCurrentThread() were to
> return control.
> * The addShutdownHook() thread deadlocks waiting for the JNI thread pool
> to close.
>
>
>
> I can understand if the thread started out life as Java Thread object or
> as an internal house keeping thread (that scenario is unlikely to call
> any of my JNI).
>
> But when the thread started out life from JNI using pthread_create() and
> then was introduced to the JVM, I think that JNI DetachCurrentThread()
> should not block even on exit under these circumstances. If anything the
> DetachCurrentThread() is signalling that Java has no further need to
> worry about this thread anymore, i.e. it can not invoke any Java.
> Obviously I would expect the JVM to refuse any further Attach requests
> under a VM_Exit scenario.
>
> I think JVM thread accounting should handle this case, maybe defer TLS
> cleanup if it is not possible to touch some data due to locking, or just
> never cleanup if the process is going to terminate anyway.
>
>
> TIA,
>
> Darryl
>