RFR: JDK-8302618: [MacOS] Problem typing uppercase letters with java.awt.Robot when moving mouse [v2]
Alexey Ivanov
aivanov at openjdk.org
Mon Jul 3 20:41:57 UTC 2023
On Sat, 1 Jul 2023 00:29:11 GMT, Harshitha Onkar <honkar at openjdk.org> wrote:
>> **Problem:**
>>
>> Robot erroneously produces lowercase letter when mouse is moved (manually) in unison with Robot's keyEvents on MacOS. This issue was originally logged by a developer of an on-screen accessibility keyboard - TouchBoard. Originally reported at https://github.com/adoptium/adoptium-support/issues/710
>>
>> The issue is reproducible on JDK versions 22 to 11, but works fine on JDK-8. (details below)
>>
>> This issue is not restricted to the Shift modifier key and causes problems with other modifier keys as well and in some scenarios without any external mouse movement.
>>
>> - This works correctly on JDK-8 up to JDK-9+129 when Accessibility APIs (AXUIElementCreateSystemWide/ AXUIElementPostKeyboardEvent) were used. Later on it was changed to CGEvents.
>>
>> - With the present code, the issue occurs at [CRobot.m#L295](https://github.com/openjdk/jdk/blob/ac6af6a64099c182e982a0a718bc1b780cef616e/src/java.desktop/macosx/native/libawt_lwawt/awt/CRobot.m#L295.) The flags gets reset or cleared when mouse is moved physically in unison with Robot's key events.
>>
>> - The physical mouse movement causes the event flags to be reset.
>>
>> **Impact:**
>>
>> Modifier keys don't work as expected when using Robot with any simultaneous physical mouse movement and in case of TouchBoard, this behavior breaks the usability of the on-screen a11y keyboard. There is no known workaround for this particular use case except for reverting to JDK-8. More details on this use case [here.](https://github.com/adoptium/adoptium-support/issues/710#issuecomment-1594103280)
>>
>> **Proposed Fix:**
>>
>> - In order to avoid resetting of the CGEventFlags here [CRobot.m#L295](https://github.com/openjdk/jdk/blob/ac6af6a64099c182e982a0a718bc1b780cef616e/src/java.desktop/macosx/native/libawt_lwawt/awt/CRobot.m#L295.), the CGEvent flag state is obtained in `initRobot` (stored in initFlags) which is later used within `CRobot_keyEvent`.
>>
>> - The incoming keyCode is used to determine whether it is a modifier key and the corresponding modifierFlagMask is either added or cleared from the initFlags based on whether the modifier key was pressed or released.
>>
>> - Finally, only the required and known flag bits from initFlag are copied over to local flag which is used in `CGEventSetFlags()`.
>>
>> **Testing:**
>>
>> The newly added test - RobotModifierMaskTest.java tests for Shift, Caps, Control, Option and Command keys.
>> It should be tested for two cases:
>>
>> CASE 1 : Run the test as an autom...
>
> Harshitha Onkar has updated the pull request incrementally with one additional commit since the last revision:
>
> kCGEventFlagMaskSecondaryFn added
test/jdk/java/awt/Robot/RobotModifierMaskTest.java line 67:
> 65: CASE 1 : Run the test as an automated test and let the Robot go through all the test cases.\n
> 66: CASE 2 : Run the test in semi-automated mode. While the Robot in typing,
> 67: manually move the mouse (without clicking/dragging). Check if the test Passes or Fails.\n\n
How does a user do this?
test/jdk/java/awt/Robot/RobotModifierMaskTest.java line 75:
> 73: if (OSInfo.getOSType() != OSInfo.OSType.MACOSX) {
> 74: System.out.println("This test is for MacOS platform only");
> 75: return;
Use `jtreg.SkippedException` to indicate the test isn't applicable in this environment. The jtreg treats this exception in a special: the doesn't fail, it doesn't pass either — it's skipped.
test/jdk/java/awt/Robot/RobotModifierMaskTest.java line 83:
> 81: robot.setAutoDelay(200);
> 82:
> 83: SwingUtilities.invokeAndWait(() -> createTestUI());
I don't insist, yet IDE usually shows a warning that such a lambda expression can be replaced with a method reference.
test/jdk/java/awt/Robot/RobotModifierMaskTest.java line 176:
> 174:
> 175: private static void testAltKey() {
> 176: jTextArea.setText("");
Should be called on EDT.
test/jdk/java/awt/Robot/RobotModifierMaskTest.java line 189:
> 187: robot.delay(500);
> 188:
> 189: if (!jTextArea.getText().equals(EXPECTED_RESULT_ALT)) {
Should be called on EDT.
-------------
PR Review Comment: https://git.openjdk.org/jdk/pull/14744#discussion_r1251267910
PR Review Comment: https://git.openjdk.org/jdk/pull/14744#discussion_r1251263715
PR Review Comment: https://git.openjdk.org/jdk/pull/14744#discussion_r1251266700
PR Review Comment: https://git.openjdk.org/jdk/pull/14744#discussion_r1251265623
PR Review Comment: https://git.openjdk.org/jdk/pull/14744#discussion_r1251265712
More information about the client-libs-dev
mailing list