RFR: JDK-8302618: [macOS] Problem typing uppercase letters with java.awt.Robot when moving mouse [v4]
Harshitha Onkar
honkar at openjdk.org
Wed Jul 5 23:50:19 UTC 2023
> **Problem:**
>
> On macOS, Robot erroneously produces lowercase letter when mouse is moved (manually) in unison with Robot's keyEvents. This issue was originally logged by a developer of an on-screen accessibility keyboard - TouchBoard. Originally reported at https://github.com/adoptium/adoptium-support/issues/710
>
> The issue is reproducible on JDK versions 22 to 11, but works fine on JDK-8. (details below)
>
> This issue is not restricted to the Shift modifier key and causes problems with other modifier keys as well and in some scenarios without any external mouse movement.
>
> - This works correctly on JDK-8 up to JDK-9+129 when Accessibility APIs (AXUIElementCreateSystemWide/ AXUIElementPostKeyboardEvent) were used. Later on it was changed to CGEvents.
>
> - With the present code, the issue occurs at [CRobot.m#L295](https://github.com/openjdk/jdk/blob/ac6af6a64099c182e982a0a718bc1b780cef616e/src/java.desktop/macosx/native/libawt_lwawt/awt/CRobot.m#L295.) The flags gets reset or cleared when mouse is moved physically in unison with Robot's key events.
>
> - The physical mouse movement causes the event flags to be reset.
>
> **Impact:**
>
> Modifier keys don't work as expected when using Robot with any simultaneous physical mouse movement and in case of TouchBoard, this behavior breaks the usability of the on-screen a11y keyboard. There is no known workaround for this particular use case except for reverting to JDK-8. More details on this use case [here.](https://github.com/adoptium/adoptium-support/issues/710#issuecomment-1594103280)
>
> **Proposed Fix:**
>
> - In order to avoid resetting of the CGEventFlags here [CRobot.m#L295](https://github.com/openjdk/jdk/blob/ac6af6a64099c182e982a0a718bc1b780cef616e/src/java.desktop/macosx/native/libawt_lwawt/awt/CRobot.m#L295.), the CGEvent flag state is obtained in `initRobot` (stored in initFlags) which is later used within `CRobot_keyEvent`.
>
> - The incoming keyCode is used to determine whether it is a modifier key and the corresponding modifierFlagMask is either added or cleared from the initFlags based on whether the modifier key was pressed or released.
>
> - Finally, only the required and known flag bits from initFlag are copied over to local flag which is used in `CGEventSetFlags()`.
>
> **Testing:**
>
> The newly added test - RobotModifierMaskTest.java tests for Shift, Caps, Control, Option and Command keys.
> It should be tested for two cases:
>
> CASE 1 : Run the test as an automated test WITHOUT any mouse movement and let the Robot go throu...
Harshitha Onkar has updated the pull request incrementally with one additional commit since the last revision:
automated and manual mode test changes
-------------
Changes:
- all: https://git.openjdk.org/jdk/pull/14744/files
- new: https://git.openjdk.org/jdk/pull/14744/files/0ab5b74b..962a6176
Webrevs:
- full: https://webrevs.openjdk.org/?repo=jdk&pr=14744&range=03
- incr: https://webrevs.openjdk.org/?repo=jdk&pr=14744&range=02-03
Stats: 173 lines in 2 files changed: 81 ins; 47 del; 45 mod
Patch: https://git.openjdk.org/jdk/pull/14744.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/14744/head:pull/14744
PR: https://git.openjdk.org/jdk/pull/14744
More information about the client-libs-dev
mailing list