RFR: 8351733: [macos] Crash when creating too many nested event loops [v3]

[Martin Fox]

On Mon, 24 Mar 2025 18:30:56 GMT, Martin Fox <mfox at openjdk.org> wrote:

>> There is an undocumented limit on nesting calls to CFRunLoopRun (or the equivalent wrapper NSRunLoop methods). When the limit is hit the OS terminates the Java app. The situation arises when a JavaFX app creates too many nested event loops from within  Platform.runLater runnables.
>> 
>> This PR doesn't change the limit (which is 250+ nested loops) but it does throw an exception just before the limit is reached so a JavaFX developer will get a useful Java stack trace instead of an OS crash log.
>> 
>> On the Mac the nested event loop has two stages: first we ask the run loop to run, then we pull an event out and process it. A Platform.runLater runnable is executed in the first stage so if the runnable starts a new nested event loop the system will re-enter CFRunLoopRun. The same isn't true if an input event handler starts a new nested event loop; at that point we're in stage two and are past the call to CFRunLoopRun.
>
> Martin Fox has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Lower limit on run loop nesting

At the risk of opening a can of worms I ran some manual tests on Linux and Windows. Windows starts behaving erratically when the nesting level goes beyond 280. I've occasionally seen a Java.lang.stackOverflowError but most often it freezes up and then the next click causes it to crash with an access violation. Linux behaves well until it gets beyond 600 and then also starts to freeze up (doesn't seem to crash). I haven't dug into what's going on under the hood.

-------------

PR Comment: https://git.openjdk.org/jfx/pull/1741#issuecomment-2749090108