RFR: 8345538: Robot.mouseMove doesn't clamp bounds on macOS when trying to move mouse off screen [v3]

[Harshitha Onkar]

On Fri, 20 Dec 2024 19:22:35 GMT, Alisen Chung <achung at openjdk.org> wrote:

>> @aivanov-jdk 
>> I see your point. But I'm running into NPE at `MouseInfo.getPointerInfo().getLocation()` without the fix as well (dual monitor, extended display setup)
>> 
>> Is it expected of MouseInfo.getPointerInfo() to return null for off-screen coordinates (meaning does it consider it as mouse not available if it is outside the screen devices)?
>> 
>> PointerInfo has two things associated with it - screen device and a location, so if it is not able to associate the coordinate with any of the screen device (off-screen coordinate) is it suppose to return null in this case?
>
> Interesting. Does this also happen after the fix with Robot clamping coordinates in shared code?

@alisenchung 

> I also tested windows and found different behavior from what Harshitha found - robot.mouseMove was already clamping on my primary screen, would not move to the second monitor, and did not throw any NPEs.

Tested it on windows again on different scales and found a possible rounding error that might be the cause for NPE on windows.

To save time you can run the test with multiple `@run` and different uiScales and do not have to change it manually in settings.


 * @run main/othervm -Dsun.java2d.uiScale=1    MouseMoveOffScreen
 * @run main/othervm -Dsun.java2d.uiScale=1.25 MouseMoveOffScreen
 * @run main/othervm -Dsun.java2d.uiScale=1.75 MouseMoveOffScreen
 * @run main/othervm -Dsun.java2d.uiScale=2    MouseMoveOffScreen
 * @run main/othervm -Dsun.java2d.uiScale=1.5  MouseMoveOffScreen


Following are the logs. 
Observe the width of the primary screen and the point returned from fillPointWithCoords() in getPointerInfo().

uiScale: 1 - Passes
Win32GraphicsDevice[screen=0]:java.awt.Rectangle[x=-1920,y=363,width=1920,height=1080]
Win32GraphicsDevice[screen=1]:java.awt.Rectangle[x=0,y=0,**width=3440**,height=1440]
Point: java.awt.Point[**x=3439**,y=200] 

uiScale: 1.25 - Passes
Win32GraphicsDevice[screen=0]:java.awt.Rectangle[x=-1920,y=363,width=1536,height=864]
Win32GraphicsDevice[screen=1]:java.awt.Rectangle[x=0,y=0,**width=2752**,height=1152]
Point: java.awt.Point[**x=2751**,y=200]

**uiScale: 1.5 - Fails**
Win32GraphicsDevice[screen=0]:java.awt.Rectangle[x=-1920,y=363,width=1280,height=720]
Win32GraphicsDevice[screen=1]:java.awt.Rectangle[x=0,y=0,**width=2293**,height=960]
Point: java.awt.Point[**x=2293**,y=200] =>  **x coordinate = width and bounds.contains(point) returns false for point on the edge/bounds of the window.**

uiScale: 1.75 - Passes
Win32GraphicsDevice[screen=0]:java.awt.Rectangle[x=-1920,y=363,width=1097,height=617]
Win32GraphicsDevice[screen=1]:java.awt.Rectangle[x=0,y=0,width=1966,height=823]
Point: java.awt.Point[x=0,y=200] => clamps to zero once coordinates are past the width of the screen.

We probably need to look into `fillPointWithCoords()` closely and how it is implemented on different platforms.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/22781#discussion_r1926210713