RFR: 8285893: Hiding dialog and showing new one causes dialog to be frozen

[Kevin Rushforth]

On Sat, 4 May 2024 22:55:13 GMT, Martin Fox <mfox at openjdk.org> wrote:

> This PR is based on a discussion that happened over in PR #1324. Some of this explanation is copied from that thread.
> 
> When `exitNestedEventLoop` is called on the innermost loop the invokeLaterDispatcher suspends operation until the loop finishes. But if you immediately start a new event loop the previous one won't finish and the dispatcher will jam up and stop dispatching indefinitely.
> 
> When the invokeLaterDispatcher is told that the innermost loop is exiting it sets `leavingNestedEventLoop` to true expecting it to be set to false when the loop actually exits. When the dispatcher is told that a new event loop has started it is not clearing `leavingNestedEventLoop` which is causing the jam. Basically it should follow the same logic used in glass; leaving the innermost loop updates a boolean indicating that the loop should exit but if a new loop is started the boolean is set back to a running state since it now applies to the new loop, not the previous one.
> 
> I suspect the invokeLaterDispatcher exists in part to deal with the specifics of how deferred runnables are handled on the Mac. I investigated this a bit and wrote up some comments in the Mac glass code.

This looks like a safe, targeted fix for the problem. It will need to be tested on all platforms with a full run of all system tests.

-------------

PR Comment: https://git.openjdk.org/jfx/pull/1449#issuecomment-2096441265