RFR: 8351733: [macos] Crash when creating too many nested event loops

[Kevin Rushforth]

On Fri, 21 Mar 2025 20:56:01 GMT, Martin Fox <mfox at openjdk.org> wrote:

> There is an undocumented limit on nesting calls to CFRunLoopRun (or the equivalent wrapper NSRunLoop methods). When the limit is hit the OS terminates the Java app. The situation arises when a JavaFX app creates too many nested event loops from within  Platform.runLater runnables.
> 
> This PR doesn't change the limit (which is 250+ nested loops) but it does throw an exception just before the limit is reached so a JavaFX developer will get a useful Java stack trace instead of an OS crash log.
> 
> On the Mac the nested event loop has two stages: first we ask the run loop to run, then we pull an event out and process it. A Platform.runLater runnable is executed in the first stage so if the runnable starts a new nested event loop the system will re-enter CFRunLoopRun. The same isn't true if an input event handler starts a new nested event loop; at that point we're in stage two and are past the call to CFRunLoopRun.

If the hard-coded 250 is a robust limit, then the fix looks reasonable to me. I'll test it and finish the review next week.

modules/javafx.graphics/src/main/native-glass/mac/GlassApplication.m line 759:

> 757: + (BOOL)canStartNestedEventLoop
> 758: {
> 759:     return nestedRunLoopRunCount <= 250;

How sure are you that 250 is a safe limit? Are there situations where we might not be able to do this many?

I recommend added a comment as to where the magic number "250" comes from. And maybe put it in a static const variable?

-------------

PR Review: https://git.openjdk.org/jfx/pull/1741#pullrequestreview-2707278351
PR Review Comment: https://git.openjdk.org/jfx/pull/1741#discussion_r2008341553