RFR: 8345538: Robot.mouseMove doesn't clamp bounds on macOS when trying to move mouse off screen [v3]

[Alexey Ivanov]

On Fri, 20 Dec 2024 21:09:28 GMT, Alexey Ivanov <aivanov at openjdk.org> wrote:

>> Now I ran the test on macOS Sequoia 15.1.1. With the current `Robot` implementation, it's able to move mouse across both displays. With Alisen's updated implementation, the coordinates are limited to the main screen.
>> 
>> <details>
>> <summary>Updated test</summary>
>> 
>> 
>>     private static void moveMouseAndCheck(final Robot robot,
>>                                           final int x, final int y) {
>>         System.out.println(x + ", " + y);
>>         robot.mouseMove(x, y);
>>         robot.delay(1000);
>> 
>>         PointerInfo cursor = MouseInfo.getPointerInfo();
>>         if (cursor != null) {
>>             System.out.println(cursor.getLocation() + " - "
>>                                + cursor.getDevice().getIDstring());
>>         }
>>     }
>> 
>>     public static void main(String[] args) throws Exception {
>>         Robot robot = new Robot();
>> 
>>         GraphicsDevice[] screens = getLocalGraphicsEnvironment()
>>                                    .getScreenDevices();
>>         for (GraphicsDevice screen : screens) {
>>             GraphicsConfiguration gc = screen.getDefaultConfiguration();
>>             System.out.println(screen.getIDstring());
>>             System.out.println("\t" + gc.getBounds());
>>             System.out.println("\t" + gc.getDefaultTransform());
>>         }
>>         System.out.println("\n");
>> 
>>         for (GraphicsDevice screen : screens) {
>>             System.out.println(screen.getIDstring());
>>             Rectangle bounds = screen.getDefaultConfiguration()
>>                                      .getBounds();
>> 
>>             moveMouseAndCheck(robot, bounds.x, bounds.y);
>>             moveMouseAndCheck(robot, bounds.x + bounds.width, bounds.y);
>>             moveMouseAndCheck(robot, bounds.x, bounds.y + bounds.height);
>>             moveMouseAndCheck(robot, bounds.x + bounds.width, bounds.y + bounds.height);
>>             moveMouseAndCheck(robot, bounds.x + bounds.width / 2, bounds.y + bounds.height / 2);
>>         }
>> 
>> </details>
>> 
>> Displays are arranged side by side: the built-in Display is on the left, the main display is on the right.
>> 
>> <details>
>> <summary>JDK 21 without the fix</summary>
>> 
>> **JDK 21 without the fix**
>> 
>> Display 69734208
>> 	java.awt.Rectangle[x=-1440,y=0,width=1440,height=900]
>> 	AffineTransform[[2.0, 0.0, 0.0], [0.0, 2.0, 0.0]]
>> Display 725353101
>> 	java.awt.Rectangle[x=0,y=0,width=1920,height=1080]
>> 	AffineTransform[[2.0, 0.0, 0.0], [0.0, 2.0, 0.0]]
>> 
>> 
>> Display 69734208
>> -1440...
>
> Note that even with Alisen's fix, **the test fails to get the mouse pointer location after moving it off the screen!**
> 
> The coordinates are expected to be within the screen bounds.

> (0, 200) is a location that is off the top of screen 1, so not within the virtual bounds.

@prrace  Why is it? Harshitha's screen 1 bounds are: [x=0,y=0,width=2293,height=960], so (0, 200) lies within the bounds of the screen.

Alisen's test moves mouse to (200, 200) and then to (20_000, 200). I expected to see mouse move to the right edge of the (virtual) screen but mouse pointer is instead at (0, 200) for me.

I don't know where mouse pointer is in Harshitha's case.

On Windows, I have a second monitor to the right of my main one, and main one has scale of 2.0. Harshitha has her secondary monitor on the left, I presume. Where is the mouse cursor on the screen?

By looking at the numbers, I infer Harshitha has a scale on her screens. Scaling down what's received from Windows could trim a pixel on the right, and it could be the reason why the point isn't within the bounds of any screen in her case.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/22781#discussion_r1894398669