<AWT Dev> [9] Review Request: JDK-7185258 [macosx] Deadlock in SunToolKit.realSync()

[Petr Pchelko]

Hello, AWT Team.

Please review the fix for the issue:
https://bugs.openjdk.java.net/browse/JDK-7185258
The fix is available at:
http://cr.openjdk.java.net/~pchelko/9/7185258/webrev/

The problem:
nativeSyncQueue synchronizes the native event queue. However, there are several situations when a dummy event posted to the native queue is not dispatched. These are our own nested loop in doAWTRunLoop and Cocoa's internal nested loop in NSView:dragImage.

Solution:

1. The interruptNativeSyncQueue was introduced. In case we are waiting for the native event queue this method interrupts waiting, otherwise it's a no-op. This is needed for the following reason: suppose the nativeSyncQueue is called on EDT. While the queue is flushed some event was processed which caused us to call doAWTRunLoop. As EDT is blocked we would have got a deadlock. Interrupting the wait lets EDT flush it's events and lets AppKit exit a nested loop. When the nativeSyncQueue is interrupted we do not immediately exit realSync, but flush EDT and try to sync native queue again. Most likely in this case we would not be in a nested loop and will successfully sync a native queue.

2. A lightweight version of nativeSyncQueue was introduced. This one does not flush the event queue, it only flushes a selector queue. This is needed to not deadlock when realSync was called during DnD. Suppose nativeSyncQueue is called while the app is in the native nested dragging loop. Until dragging operation finishes only dragging-related events will be processed. So we have no opportunity to flush the queue as our dummy event will be blocked by a dragging nested loop.

I've tested it by running almost all awt and swing tests. No new failures, some tests start to pass after the fix.

With best regards. Petr.