RFR: JDK-8302618: [macOS] Problem typing uppercase letters with java.awt.Robot when moving mouse [v7]

[Harshitha Onkar]

On Fri, 21 Jul 2023 03:39:27 GMT, Sergey Bylokhov <serb at openjdk.org> wrote:

>> @mrserb 
>> 
>>> Just to clarify an idea of this change. We would like to maintain the "flag" keys pressed by the user and maintain the list of current flags. So if the user will press SHIFT key, we will use that modifier for all next keys until shift will be released.
>> 
>> That is correct.
>> 
>>> It looks similar to how the mouseMove is implemented in CRobot, where we maintain the mouseLastX/mouseLastY, and we ignore the current location of the mouse for clicks, but use the latest saved value.
>> 
>> 
>>> Probably we can do the same on the java side in this patch? it will simplify the native code a bit.
>> 
>> Yes, the code for mouseEvent looks similar to what we are trying to achieve for keyEvent but there are a few differences too-                 
>> - `mouseEvent(int lastX, int lastY, int buttonsState, boolean isButtonsDownState, boolean isMouseMove)`,  lastX  & lastY are already part of the parameters being sent from Java to native side.
>> 
>> - `keyEvent(int javaKeyCode, boolean keydown)` - Are you suggesting that a modifier flag state be sent from Java to native side? This would involve changing the present native `keyEvent` signature and adding code to native side to evaluate the new modifier flag state from Java and then map it to the corresponding [kCGEventFlagMask](https://developer.apple.com/documentation/coregraphics/cgeventflags/kcgeventflagmaskalphashift?language=objc).
>> 
>> Having everything on native side seems to be easier because with the present fix, the equivalent modifier flag mask is obtained using javaKeyCode + key state and a quick `NSDictionary modifierKeyToMaskMap` lookup without the need of adding or changing  keyEvent signature. 
>> 
>>> BTW do we need to set all this modifiers for the mouse clicks as well?
>> 
>> I don't think it is applicable for mouse clicks, as the issue (details below) occurs only when flagState is obtained using - `CGEventSourceFlagsState(kCGEventSourceStateHIDSystemState)` which was the case for keyEvents. I can double-check on this again.
>> 
>> **Issue:** 
>> _With the original code, the issue occurs at [CRobot.m#L295](https://github.com/openjdk/jdk/blob/ac6af6a64099c182e982a0a718bc1b780cef616e/src/java.desktop/macosx/native/libawt_lwawt/awt/CRobot.m#L295.) The flags gets reset or cleared when mouse is moved physically in unison with Robot's key events. The physical mouse movement causes the event flags to be reset._
>
>>I don't think it is applicable for mouse clicks, as the issue (details below) occurs only when flagState is obtained using - CGEventSourceFlagsState(kCGEventSourceStateHIDSystemState) which was the case for keyEvents. I can double-check on this again.
> 
> yes, please.
> if the physical mouse movement causes the "event flags" to be reset, then what about  "event flags" for mouse events

@mrserb 

> BTW do we need to set all this modifiers for the mouse clicks as well?

I wanted to clarify if you mean mouse clicks generated externally or Robot here ?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/14744#issuecomment-1662979568