RFR: 8359108: Mac - When Swing starts First, native application menu doesn't work for JavaFX

[Kevin Rushforth]

On Mon, 15 Sep 2025 16:20:10 GMT, Pabulaner IV <duke at openjdk.org> wrote:

> This pull request fixes the system menu bar on MacOS when combining windows of Swing and JavaFX.
> 
> # Behavior before
> 
> If for some reason You needed to initialize AWT before JavaFX and You wanted to install the system menu bar from the JavaFX side, this wasn't possible. This issue is persistent even when You didn't open a Swing window. 
> 
> One scenario where this kind of issue happens is when You use install4j (see https://www.ej-technologies.com/install4j). In this case AWT is initialized by install4j and therefore You can't use the JavaFX system menu bar anymore.
> 
> 
> # Behavior after
> 
> The fix allows JavaFX to install a system menu bar even if it is initialized after AWT. This is achieved by only changing code inside JavaFX. Each JavaFX window stores the previously installed menu bar when gaining focus and will restore this menu bar if the focus was lost. This only happens if the system menu bar installed by the JavaFX window is still unchanged.
> 
> 
> # Tests
> 
> This PR introduces tests for the system menu bar in addition to verifying its own behavior / changes. The tests include single- and multi-window tests while interacting with Swing. The tests ensure that the menu bar stays the same for each window, no matter how You switch focus between them.
> 
> 
> # Additional benifits 
> 
> This fix is not specifically for AWT, but allows JavaFX to interact much more compatibly with other frameworks that make use of the system menu bar.
> 
> 
> # Review from AWT
> 
> In the previous PR related to this one, the comment was made that the folks from AWT should take a look at this fix. It would be great and much appreciated if someone could initiate it.
> 
> 
> # Add disable flag?
> 
> We could also add a flag to prevent JavaFX from installing a system menu bar for users who have found other fixes for their projects / setups. This could be used to restore the previous behavior when AWT is initialized first.
> 
> 
> Co-Author: @FlorianKirmaier

This approach seems much better than the previous in that it is activated only when a JavaFX Stage becomes the focused (key) window, and it saves / restores the menu such that whenever an different (e.g., AWT) window is focused, the AWT menu will be installed.

In addition to reviewing and checking that it only touches the menu when it should, testing will be critical.

Reviewers: @kevinrushforth @andy-goryachev-oracle @beldenfox
Cc: @prrace @honkar-jdk

As mentioned above, testing will be key here. This needs to work reliably in the case of a multi-window app when the application is hidden and restored, and when switching between the windows of the app. The automated tests should already cover some of the cases. There may be others to consider.

Here are a few cases that I can think of to check:

* Pure FX app: 2 FX Stages (AWT toolkit not initialized at all) -- the fix should do nothing
* 2 FX Stages with FX initialized first (AWT is initialized last, but no AWT/Swing windows) -- the fix should do nothing
* 2 FX Stages with AWT initialized first (but no AWT/Swing windows) -- the fix should install the FX window when either Stage is focused (there will not be any AWT menu)
* a mix of AWT Windows and FX Stages with FX initialized first -- FX will install the menu and keep it there, since AWT won't install a menu when it doesn't own the NSApplication
* a mix of AWT Windows and FX Stages with FX initialized first -- AWT menu will be installed when an AWT window is active, FX menu will be installed when an FX Stage is active
* 2 AWT Stages one of which has a JFXPanel with an FX scene, initialize AWT first -- fix should be a noop
* 2 AWT Stages one of which has a JFXPanel with an FX scene, initialize FX first -- fix should be a noop

I presume that the automated tests will cover some of these, but we might want additional tests.

-------------

PR Comment: https://git.openjdk.org/jfx/pull/1904#issuecomment-3315012432
PR Comment: https://git.openjdk.org/jfx/pull/1904#issuecomment-3315015857