RFR: 8285893: Hiding dialog and showing new one causes dialog to be frozen

[Marius Hanl]

On Tue, 14 May 2024 18:42:47 GMT, Martin Fox <mfox at openjdk.org> wrote:

> On Linux leaveNestedEventLoop makes changes in glass that cannot be erased or undone (it calls gtk_main_quit which updates internal GTK state that we don’t have access to). In the end GTK will exit the loops in the correct order but it will exit loop A before the core code has a chance to call Application.leaveNestedEventLoop again. This is throwing off the bookkeeping including some debug checks.

That is exactly what I also figured out. If a second new nested event loop is started directly after the first one was requested to leave, the native code will never 'return'.
That's why this code exists: https://github.com/openjdk/jfx/blob/master/modules/javafx.graphics/src/main/java/com/sun/glass/ui/EventLoop.java#L118
which I slightly modified to not run `invokeLater` in my PR.

Otherwise it would never return any value and code after `enterNestedEventLoop` is never called:

        Object o = Platform.enterNestedEventLoop(this);
        System.out.println(o); // <- never called


And since you fixed the event jam so that we are not stuck anymore and Linux is implemented different with a global stateful variable, we will get that 'inconsistency', which I don't think is relevant or a problem here, hence I removed it in my PR.

I also don't found any other way to deal with that situation, as you also stated, there is nothing else we can do, and while the other OS implementation have some kind of while loop where we can try to implement something, on the Linux side we completely rely on `gtk_main` and `gtk_main_quit`.

-------------

PR Comment: https://git.openjdk.org/jfx/pull/1449#issuecomment-2127252021