RFR: 8351907: [XWayland] [OL10] Robot.mousePress() is delivered to wrong place [v3]
Alexander Zvegintsev
azvegint at openjdk.org
Mon May 19 23:06:39 UTC 2025
> The issue is the inability to move the mouse cursor using `java.awt.Robot` on Oracle Linux 10 (which has not yet been released). FIY, the OL10 is no longer provides an Xorg session, so only Wayland is available.
> It is a JCK conformance failure.
>
> ---
>
> In our robot implementation, we emulate mouse/keyboard actions using the XTEST extension protocol.
>
> The XTEST calls are now wired to [libEI](https://libinput.pages.freedesktop.org/libei/api/index.html):
>
> 1.
>
> It now asks for a user confirmation to allow interaction, so we should now be able to click on native applications, window titles, etc.
>
> However, our robot implementation is mostly uses XTEST, but there is an exception:
> for mouse move we use XWarpPointer, which is not part of the XTEST, thus it is not propagated outside of Xwayland server.
>
> So we have a situation where mouse is moving only inside Xwayland server, and we see the mouse enter/exit events,
> on mouse press we call XTestFakeButtonEvent and it is handled by libei,
> so the click happens where is the actual mouse cursor is in the system. That is why the test fails.
>
> To avoid this situation we should use XTestFakeMotionEvent for mouseMove in Xwayland case.
>
> It would be a simple solution, but
>
>
> 2.
>
> This permission granted by a user doesn't persist across reboots, it timeouts after a period of inactivity.
>
> It still seems to conform to the specifications, but we will not be able to conveniently run automated tests in this case.
>
> ---
>
> So as a solution we can switch the [Remote Desktop XDG portal](https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.RemoteDesktop.html).
> It has session and `restore_token` functionality to keep permissions granted by a user, similar to what we use in a [Screencast](https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.ScreenCast.html) for taking screenshots.
> Most of the code can be reused, and it has everything we need.
>
>
> * It is only enabled by default on Wayland and Gnome Shell 47 and higher due to XTEST API complications that first arose in this version. For earlier versions, Screencast is used instead.
> * A new value has been introduced for the `awt.robot.screenshotMethod` system property - `dbusRemoteDesktop`. It works on Ubuntu 24.04 with Gnome 46, but is not enabled by default.
> * extra mouse buttons are not supported yet, [JDK-8357142](https://bugs.openjdk.org/browse/JDK-8357142)
> * Since we are now moving the mouse using a non-X11 API and `MouseInfo.getPointerInfo().getLo...
Alexander Zvegintsev has updated the pull request incrementally with one additional commit since the last revision:
call OpenPipeWireRemote even for remote desktop session calls
-------------
Changes:
- all: https://git.openjdk.org/jdk/pull/25265/files
- new: https://git.openjdk.org/jdk/pull/25265/files/5d08133e..b8a430ef
Webrevs:
- full: https://webrevs.openjdk.org/?repo=jdk&pr=25265&range=02
- incr: https://webrevs.openjdk.org/?repo=jdk&pr=25265&range=01-02
Stats: 7 lines in 1 file changed: 0 ins; 7 del; 0 mod
Patch: https://git.openjdk.org/jdk/pull/25265.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/25265/head:pull/25265
PR: https://git.openjdk.org/jdk/pull/25265
More information about the client-libs-dev
mailing list