RFR: 8351733: [macos] Crash when creating too many nested event loops

[Martin Fox]

On Fri, 21 Mar 2025 21:25:43 GMT, Andy Goryachev <angorya at openjdk.org> wrote:

>> There is an undocumented limit on nesting calls to CFRunLoopRun (or the equivalent wrapper NSRunLoop methods). When the limit is hit the OS terminates the Java app. The situation arises when a JavaFX app creates too many nested event loops from within  Platform.runLater runnables.
>> 
>> This PR doesn't change the limit (which is 250+ nested loops) but it does throw an exception just before the limit is reached so a JavaFX developer will get a useful Java stack trace instead of an OS crash log.
>> 
>> On the Mac the nested event loop has two stages: first we ask the run loop to run, then we pull an event out and process it. A Platform.runLater runnable is executed in the first stage so if the runnable starts a new nested event loop the system will re-enter CFRunLoopRun. The same isn't true if an input event handler starts a new nested event loop; at that point we're in stage two and are past the call to CFRunLoopRun.
>
> modules/javafx.graphics/src/main/native-glass/mac/GlassApplication.m line 759:
> 
>> 757: + (BOOL)canStartNestedEventLoop
>> 758: {
>> 759:     return nestedRunLoopRunCount <= 250;
> 
> would it be possible to determine the source of this limit - a header perhaps?
> could it be dependent on os version?

I have not been able to find any header detailing this limit. My guess is that someone at Apple decided to add an internal check for infinite recursion to ensure the app fails early with a clear message and decided 255 levels of nesting should be enough for anyone. The crash log contains these lines (which also show up in the debugger):

> Application Specific Information:
> Too many nested CFRunLoopRuns

The check is new with macOS 15. I tested some standalone code on macOS 13 and 14 and had no problem reaching a nesting level of 500. But I don't think we should make the JavaFX check conditional on OS version number; I wouldn't want a JavaFX app that worked on macOS 14 to fail mysteriously on macOS 15.

-------------

PR Review Comment: https://git.openjdk.org/jfx/pull/1741#discussion_r2008829542