RFR: 8359601: Fix window button states of an extended stage [v6]

[Kevin Rushforth]

On Sat, 21 Jun 2025 23:37:13 GMT, Michael Strauß <mstrauss at openjdk.org> wrote:

>> The window button states (disabled/hidden) of extended stages with a `HeaderButtonOverlay` or custom header buttons are inconsistent with what we would expect from the OS (Windows and Linux). To figure out what we would expect, I started with gathering some data. The following table shows the button states of system-decorated windows on various platforms:
>> 
>> #### Windows 11
>> 
>> | Window attributes | Iconify | Maximize | Close |
>> |---|---|---|---|
>> | resizable, not modal | visible | visible | visible |
>> | not resizable, not modal | visible | visible, disabled | visible |
>> | resizable, modal | visible, disabled | visible | visible |
>> | not resizable, modal | hidden | hidden | visible, utility-style |
>> 
>> #### Ubuntu 24 / Fedora 41 (GNOME)
>> 
>> | Window attributes | Iconify | Maximize | Close |
>> |---|---|---|---|
>> | resizable, not modal | visible | visible | visible |
>> | not resizable, not modal | visible | hidden | visible |
>> | resizable, modal | visible, _not working_ | visible, _not working_ | visible |
>> | not resizable, modal | visible, _not working_ | hidden | visible |
>> 
>> #### Kubuntu 24 (KDE)
>> 
>> | Window attributes | Iconify | Maximize | Close |
>> |---|---|---|---|
>> | resizable, not modal | visible | visible | visible |
>> | not resizable, not modal | visible | hidden | visible |
>> | resizable, modal | visible, _not working_ | visible | visible |
>> | not resizable, modal | visible, _not working_ | hidden | visible |
>> 
>> ## Observations
>> 
>> 1. On Windows, buttons are generally disabled when their operation is not possible with the given window attributes.
>>    * Exception: modal/non-resizable windows look like utility windows (iconify and maximize are hidden)
>> 2. On GNOME and KDE, buttons are generally hidden when their operation is not possible.
>>    * Exception: iconify and maximize on modal windows is not hidden, but seems to simply not do anything (bug?)
>> 
>> ## Permitted window button operations
>> 
>> Given the gathered observations and some simple logic, this is the table of operations that are permitted for all combinations of modal and resizable window attributes:
>> 
>> | Window attributes | Iconify | Maximize | Close |
>> |---|---|---|---|
>> | resizable, not modal | yes | yes | yes |
>> | not resizable, not modal | yes | no | yes |
>> | resizable, modal | no | yes | yes |
>> | not resizable, modal | no | no | yes |
>> 
>> ## Fixes
>> 
>> This PR includes the following changes:
>> 1. Unused code relating to window modality is removed...
>
> Michael Strauß has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fix resizable state of macOS window

The code looks good. I submitted a set of headful test jobs and will report when done. I want to do a little light manual testing, especially on macOS where there are a few changes in the non-extended case (although the code looks equivalent).

modules/javafx.graphics/src/main/native-glass/mac/GlassWindow+Java.m line 215:

> 213: }
> 214: 
> 215: - (void)_setResizable:(bool)resizable

Good. It definitely seems cleaner to pass in resizable as a parameter rather than having this method be a "toggle" resizable as it was previously.

modules/javafx.graphics/src/main/native-glass/mac/GlassWindow.m line 1165:

> 1163:     {
> 1164:         GlassWindow *window = getGlassWindow(env, jPtr);
> 1165:         [window _setResizable:jResizable];

I presume you removed the `performSelectorOnMainThread` because the calling method always calls this on the main thread?

-------------

PR Review: https://git.openjdk.org/jfx/pull/1831#pullrequestreview-3053397171
PR Review Comment: https://git.openjdk.org/jfx/pull/1831#discussion_r2229616697
PR Review Comment: https://git.openjdk.org/jfx/pull/1831#discussion_r2229618920