JVM crashes on macOS when entering too many nested event loops

[Martin Fox]

I entered a bug https://bugs.openjdk.org/browse/JDK-8351733.

This crash doesn’t happen if you create nested event loops while processing events. I wrote a test app that started a new event loop on every mouse click and easily created over 400 nested loops. So we’re not pushing up against the native thread stack size limit, we’re running into an undocumented limit on the number of nested CFRunLoopRun calls. There’s a line to that effect in the crash log (which I overlooked for a while because I’m not used to crash logs being that helpful).

The bug is specific to the way the Mac handles runLater blocks and other internal uses of performSelectorOnMainThread. So far I can only reproduce this using Platform::runLater e.g. block A schedules runLater block B and then enters a nested event loop, block B schedules runLater block C and enters a nested event loop, and on and on. In a real app it’s also possible that some internal uses of performSelectorOnMainThread might be interacting with Platform::runLater calls in unexpected ways.

As far as I know here’s no way to ask the system what the CFRunLoopRun nesting limit is and no way to detect at runtime when we’ve hit it. The best we can do is maintain an internal counter and throw a runtime exception when it hits, say, 245 nested CFRunLoopRun calls. That at least would produce a very long but usable Java stack trace.

I could update Platform::canStartNestedEventLoop to detect this case but I’m not sure it would be useful. When you hit this condition the system will already be 240+ loops in and they will all have to be unwound to get the system back into a usable state.

Adding the code to glass that throws an exception and writing up a manual test case is easy. Writing up an automated test case is much harder (at least for me).

> On Mar 10, 2025, at 7:47 PM, Christopher Schnick <crschnick at xpipe.io> wrote:
> 
> Our code and some libraries do enter some nested event loops at a few places when it makes sense, but we didn't do anything to explicitly provoke this, this occurred naturally in our application. So it would be nice if JavaFX could somehow guard against this, especially since crashing the JVM is probably the worst thing that can happen.
> 
> I looked at the documentation, but it seems like the public API at Platform::enterNestedEventLoop does not mention this.
> From my understanding, the method Platform::canStartNestedEventLoop is potentially the right method to indicate to the caller that the limit is close by returning false.
> And even if something like an exception is thrown when a nested event loop is started while it is close to the limit, that would still be much better than a direct crash.
> 
> Best
> Christopher Schnick
> 
> On 10/03/2025 18:51, Andy Goryachev wrote:
>> This looks to me like it might be hitting the (native) thread stack size limit.
>>  
>> c.s.glass.ui.Application::enterNestedEventLoop() even warns about it:
>>  
>>      * An application may enter several nested loops recursively. There's no
>>      * limit of recursion other than that imposed by the native stack size.
>>  
>>  
>> -andy
>>  
>>  
>>  
>> From: openjfx-dev <openjfx-dev-retn at openjdk.org> <mailto:openjfx-dev-retn at openjdk.org> on behalf of Martin Fox <martinfox656 at gmail.com> <mailto:martinfox656 at gmail.com>
>> Date: Monday, March 10, 2025 at 10:10
>> To: Christopher Schnick <crschnick at xpipe.io> <mailto:crschnick at xpipe.io>
>> Cc: OpenJFX <openjfx-dev at openjdk.org> <mailto:openjfx-dev at openjdk.org>
>> Subject: Re: JVM crashes on macOS when entering too many nested event loops
>> 
>> Hi Christopher,
>> 
>> I was able to reproduce this crash. I wrote a small routine that recursively calls itself in a runLater block and then enters a nested event loop. The program crashes when creating loop 254. I’m not sure where that limit comes from so it’s possible that consuming some other system resource could lower it. I couldn’t see any good way to determine how many loops are active by looking at the crash report since it doesn’t show the entire call stack.
>> I did a quick trial on Linux and was able to create a lot more loops (over 600) but then started seeing erratic behavior and errors coming from the Java VM. The behavior was variable unlike on the Mac which always crashes when creating loop 254.
>> 
>> Martin
>> 
>> > On Mar 7, 2025, at 6:24 AM, Christopher Schnick <crschnick at xpipe.io> <mailto:crschnick at xpipe.io> wrote:
>> > 
>> > Hello,
>> > 
>> > I have attached a JVM fatal error log that seemingly was caused by our JavaFX application entering too many nested event loops, which macOS apparently doesn't like.
>> > 
>> > As far as I know, there is no upper limit defined on how often an event loop can be nested, so I think this is a bug that can occur in rare situations.
>> > 
>> > Best
>> > Christopher Schnick<hs_err_pid.txt>
>> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/openjfx-dev/attachments/20250311/f87d824e/attachment-0001.htm>