RFR: 8359108: Mac - When Swing starts First, native application menu doesn't work for JavaFX [v5]

[Martin Fox]

On Sun, 31 Aug 2025 07:36:28 GMT, Pabulaner IV <duke at openjdk.org> wrote:

>> This pull request fixes the system menu bar on Mac when combining windows of Swing and JavaFX.
>> 
>> The first issue was to get the native menu bar working simultaneously on Swing and JavaFX, which was done by just returning always true inside the supportsSystemMenu method.
>> 
>> The second issue was to remove all system menu items installed by a swing window. This was fixed by checking the system menu bar every time an item is inserted or removed and removing all menu items that are not owned by JavaFX. This check is done on every insert and remove as JavaFX does not have a clear method inside the MenuBarDelegate class that could be called every time the window gets the focus.
>> 
>> I tested the fix with two Swing and two JavaFX windows that are run inside the same application and it worked without any errors, but on further testing I noticed some issues with the menu bar. I am currently writing the test and fixes for it.
>> 
>> Co-Author: @FlorianKirmaier
>
> Pabulaner IV has updated the pull request incrementally with one additional commit since the last revision:
> 
>   8359108: Mac - When Swing starts First, native application menu doesn't work for JavaFX

Using osascript to check the state of the menus is a great idea (assuming there's no policy against using that sort of thing in the system tests). Far better than what I did in PR #1881.

You need to have folks on the AWT/Swift side review this new approach. If I understand this correctly when the JavaFX window assumes focus you're stashing away NSApp.mainMenu under the assumption that it was put there by AWT. Then you're replacing it with an NSMenu built by JavaFX. When the JavaFX window loses focus you're putting the stashed AWT NSMenu back into NSApp.mainMenu. The two big questions are:

a) Will AWT malfunction since NSApp.mainMenu no longer points to the NSMenu they expect?
b) During the period of time the JavaFX window has focus is there some chance AWT will change NSApp.mainMenu out from under us?

Put another way, JavaFX currently uses window focus to figure out which window should "own" NSApp.mainMenu and this PR extends that notion to include Swing windows. I'm not sure that's matches how AWT works internally so someone on the AWT side needs to weigh in.

What you're doing on the JavaFX side is just some pointer shuffling. Swapping NSApp.mainMenu in and out makes me uneasy but it's hard for me to point to a specific problem that will arise on the JavaFX side.

There's also the larger question of whether we want to support this configuration to begin with.

-------------

PR Comment: https://git.openjdk.org/jfx/pull/1835#issuecomment-3276107668