RFR: JDK-8302618: [macOS] Problem typing uppercase letters with java.awt.Robot when moving mouse [v9]

Harshitha Onkar honkar at openjdk.org
Wed Aug 2 21:46:32 UTC 2023


> **Problem:**
> 
> On macOS, Robot erroneously produces lowercase letter when mouse is moved (manually) in unison with Robot's keyEvents. This issue was originally logged by a developer of an on-screen accessibility keyboard  - TouchBoard. Originally reported at https://github.com/adoptium/adoptium-support/issues/710
> 
> This issue is reproducible on JDK versions 22 to 11, but works fine on JDK-8. (details below) 
> 
> This issue is not restricted to the Shift modifier key and causes problems with other modifier keys as well and in some scenarios without any external mouse movement.
> 
> - This works correctly on JDK-8 up to JDK-9+129 when Accessibility APIs (AXUIElementCreateSystemWide/ AXUIElementPostKeyboardEvent) were used. Later on it was changed to CGEvents. 
> 
> - With the present code, the issue occurs at [CRobot.m#L295](https://github.com/openjdk/jdk/blob/ac6af6a64099c182e982a0a718bc1b780cef616e/src/java.desktop/macosx/native/libawt_lwawt/awt/CRobot.m#L295.)  The flags gets reset or cleared when mouse is moved physically in unison with Robot's key events.
> 
> - The physical mouse movement causes the event flags to be reset. 
> 
> **Impact:**
> 
> Modifier keys don't work as expected when using Robot with any simultaneous physical mouse movement and in case of TouchBoard, this behavior breaks the usability of the on-screen a11y keyboard. There is no known workaround for this particular use case except for reverting to JDK-8. More details on this use case [here.](https://github.com/adoptium/adoptium-support/issues/710#issuecomment-1594103280)
> 
> **Proposed Fix:**
> 
> - In order to avoid resetting of the CGEventFlags here [CRobot.m#L295](https://github.com/openjdk/jdk/blob/ac6af6a64099c182e982a0a718bc1b780cef616e/src/java.desktop/macosx/native/libawt_lwawt/awt/CRobot.m#L295.), the CGEvent flag state is obtained in `initRobot` (stored in initFlags) which is later used within `CRobot_keyEvent`.
> 
> - The incoming keyCode is used to determine whether it is a modifier key and the corresponding modifierFlagMask is either added or cleared from the initFlags based on whether the modifier key was pressed or released.
> 
> - Finally, only the required and known flag bits from initFlag are copied over to local flag which is used in `CGEventSetFlags()`.
> 
> **Testing:**
> 
> The newly added test - RobotModifierMaskTest.java tests for Shift, Caps, Control, Option and Command keys.
> The test runs in 2 modes - as automated and manual test.
> 
> CASE 1 :  As automated test. No user interaction needed.
> CASE 2 :  When r...

Harshitha Onkar has updated the pull request incrementally with one additional commit since the last revision:

  changed allModifiersMask datatype

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/14744/files
  - new: https://git.openjdk.org/jdk/pull/14744/files/54757ae4..6dbbc9eb

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=14744&range=08
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14744&range=07-08

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/14744.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/14744/head:pull/14744

PR: https://git.openjdk.org/jdk/pull/14744



More information about the client-libs-dev mailing list