RFR: 8351733: [macos] Crash when creating too many nested event loops [v3]

[Kevin Rushforth]

On Mon, 24 Mar 2025 18:30:56 GMT, Martin Fox <mfox at openjdk.org> wrote:

>> There is an undocumented limit on nesting calls to CFRunLoopRun (or the equivalent wrapper NSRunLoop methods). When the limit is hit the OS terminates the Java app. The situation arises when a JavaFX app creates too many nested event loops from within  Platform.runLater runnables.
>> 
>> This PR doesn't change the limit (which is 250+ nested loops) but it does throw an exception just before the limit is reached so a JavaFX developer will get a useful Java stack trace instead of an OS crash log.
>> 
>> On the Mac the nested event loop has two stages: first we ask the run loop to run, then we pull an event out and process it. A Platform.runLater runnable is executed in the first stage so if the runnable starts a new nested event loop the system will re-enter CFRunLoopRun. The same isn't true if an input event handler starts a new nested event loop; at that point we're in stage two and are past the call to CFRunLoopRun.
>
> Martin Fox has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Lower limit on run loop nesting

I think we will eventually settle on one of two solutions. In both cases we will want to document the concept of a limited nesting depth and encourage application developers to check `Platform.canStartNestedEventLoop` if they want to be sure that the platform can start one.

The two options are:

1. Make the limit platform-specific limit. The API docs will therefore not mention any particular limit. Each glass platform would implement `Application.canStartNestedEventLoop` imposing the limit for that platform. We _might_ consider documenting a guaranteed lower limit that is well below any specific platform limit (e.g., 150).
2. Make the limit platform-independent. We could, and probably should, choose to document this limit. We could choose to implement Application.canStartNestedEventLoop in shared code rather than overriding it in the pipeline. We would need to pick a safe "lowest common denominator" (maybe 200).

My gut feel is that we should go with option 1 and keep this platform-specific, which means we would not specify the limit (other than maybe giving a guarantee of "at least 150"), unless there are good arguments in favor of option 2 that we haven't considered.

Comments?

-------------

PR Comment: https://git.openjdk.org/jfx/pull/1741#issuecomment-2752001549