RFR: 8359108: Mac - When Swing starts First, native application menu doesn't work for JavaFX [v5]

[Kevin Rushforth]

On Wed, 22 Oct 2025 15:57:23 GMT, Pabulaner IV <duke at openjdk.org> wrote:

>> This pull request fixes the system menu bar on MacOS when combining windows of Swing and JavaFX.
>> 
>> # Behavior before
>> 
>> If for some reason You needed to initialize AWT before JavaFX and You wanted to install the system menu bar from the JavaFX side, this wasn't possible. This issue is persistent even when You didn't open a Swing window. 
>> 
>> One scenario where this kind of issue happens is when You use install4j (see https://www.ej-technologies.com/install4j). In this case AWT is initialized by install4j and therefore You can't use the JavaFX system menu bar anymore.
>> 
>> 
>> # Behavior after
>> 
>> The fix allows JavaFX to install a system menu bar even if it is initialized after AWT. This is achieved by only changing code inside JavaFX. Each JavaFX window stores the previously installed menu bar when gaining focus and will restore this menu bar if the focus was lost. This only happens if the system menu bar installed by the JavaFX window is still unchanged.
>> 
>> 
>> # Tests
>> 
>> This PR introduces tests for the system menu bar in addition to verifying its own behavior / changes. The tests include single- and multi-window tests while interacting with Swing. The tests ensure that the menu bar stays the same for each window, no matter how You switch focus between them.
>> 
>> 
>> # Additional benifits 
>> 
>> This fix is not specifically for AWT, but allows JavaFX to interact much more compatibly with other frameworks that make use of the system menu bar.
>> 
>> 
>> # Review from AWT
>> 
>> In the previous PR related to this one, the comment was made that the folks from AWT should take a look at this fix. It would be great and much appreciated if someone could initiate it.
>> 
>> 
>> # Add disable flag?
>> 
>> We could also add a flag to prevent JavaFX from installing a system menu bar for users who have found other fixes for their projects / setups. This could be used to restore the previous behavior when AWT is initialized first.
>> 
>> 
>> Co-Author: @FlorianKirmaier
>
> Pabulaner IV has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 10 commits:
> 
>  - 8359108: Mac - When Swing starts First, native application menu doesn't work for JavaFX
>  - 8359108: Mac - When Swing starts First, native application menu doesn't work for JavaFX
>  - 8359108: Mac - When Swing starts First, native application menu doesn't work for JavaFX
>  - 8359108: Mac - When Swing starts First, native application menu doesn't work for JavaFX
>  - 8359108: Mac - When Swing starts First, native application menu doesn't work for JavaFX
>  - 8359108: Mac - When Swing starts First, native application menu doesn't work for JavaFX
>  - 8359108: Mac - When Swing starts First, native application menu doesn't work for JavaFX
>  - 8359108: Mac - When Swing starts First, native application menu doesn't work for JavaFX
>  - 8359108: Mac - When Swing starts First, native application menu doesn't work for JavaFX
>  - 8359108: Mac - When Swing starts First, native application menu doesn't work for JavaFX
>    
>    # Conflicts:
>    #	modules/javafx.graphics/src/main/native-glass/mac/GlassWindow+Overrides.m

Here are the assertion failures for the two failing tests:


MacOSSystemMenuMultiWindowFXOnlySwingLast > test() FAILED
    org.opentest4j.AssertionFailedError: Menu size is different: 5 != 2 ==> expected: <true> but was: <false>
        at app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
        at app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
        at app//org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63)
        at app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36)
        at app//org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:214)
        at app//test.com.sun.glass.ui.mac.MacOSSystemMenuTestBase.compareMenus(MacOSSystemMenuTestBase.java:350)
        at app//test.com.sun.glass.ui.mac.MacOSSystemMenuMultiWindowFXOnlySwingLast.test(MacOSSystemMenuMultiWindowFXOnlySwingLast.java:70)



MacOSSystemMenuMultiWindowTest > test() FAILED
    org.opentest4j.AssertionFailedError: Menu size is different: 5 != 2 ==> expected: <true> but was: <false>
        at app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
        at app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
        at app//org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63)
        at app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36)
        at app//org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:214)
        at app//test.com.sun.glass.ui.mac.MacOSSystemMenuTestBase.compareMenus(MacOSSystemMenuTestBase.java:350)
        at app//test.com.sun.glass.ui.mac.MacOSSystemMenuMultiWindowTest.test(MacOSSystemMenuMultiWindowTest.java:69)


I instrumented the code and it is picking up the menus of the Terminal app from which I ran `gradle test`. If I click on another app (e.g., the Finder) after starting `gradle` but before the test actually starts then the test passes.

I fired off a headful test on our Jenkins system and none of the tests pass. They all timeout or fail with an error from `osascript`.

All of this leads me to think that it might be better to convert these tests to manual test. Instead of using `osascript`, you could have each window show a `TextArea` with a list of what the top level menu items should be when you click in the window. What are your thoughts on this?

-------------

PR Comment: https://git.openjdk.org/jfx/pull/1904#issuecomment-3434284701