RFR: 8285893: Hiding dialog and showing new one causes dialog to be frozen

[Martin Fox]

On Thu, 23 May 2024 14:21:27 GMT, Marius Hanl <mhanl at openjdk.org> wrote:

> That is exactly what I also figured out. If a second new nested event loop is started directly after the first one was requested to leave, the native code will never 'return'. That's why this code exists: https://github.com/openjdk/jfx/blob/master/modules/javafx.graphics/src/main/java/com/sun/glass/ui/EventLoop.java#L118 which I slightly modified to not run `invokeLater` in my PR.

I think the invokeLater call is required. On Mac and Windows calls into Glass can only affect the innermost loop. For each loop you want to exit you have to call `Application.leaveNestedEventLoop` and then allow the execution stack to unwind to Glass so it can remove the loop. Only then can you initiate the process of leaving the next loop. So if there’s multiple loops in a LEAVING state each one needs to be handled in a separate runnable so the execution stack can unwind between runnables.

When leaving the innermost loop the steps of interest are:

1. call Application.leaveNestedEventLoop
2. disable the InvokeLaterDispatcher
3. unwind the execution stack to glass
4. enable the InvokeLaterDispatcher

My tweak to the code ensures that the InvokeLaterDispatcher is re-enabled if we start at step 1 but enter a new event loop before we can reach step 3. On Mac and Windows we’ll always restart the leaving process at step 1 for all loops. That’s not happening on Linux, the old loop is leaving without starting at step 1. I’m pretty sure this is opening a window where the InvokeLaterDispatcher can dispatch a runnable to the old loop while it’s leaving. That’s not supposed to happen but I’m not sure what the consequences might be on Linux (I know it would be very bad on Mac).

-------------

PR Comment: https://git.openjdk.org/jfx/pull/1449#issuecomment-2137836685