RFR: 8351907: [XWayland] [OL10] Robot.mousePress() is delivered to wrong place

Alexander Zvegintsev azvegint at openjdk.org
Fri May 16 13:50:44 UTC 2025


The issue is the inability to move the mouse cursor using `java.awt.Robot` on Oracle Linux 10 (which has not yet been released). FIY, the OL10 is no longer provides an Xorg session, so only Wayland is available.
It is a JCK conformance failure. 

---

In our robot implementation, we emulate mouse/keyboard actions using the XTEST extension protocol.

The XTEST calls are now wired to [libEI](https://libinput.pages.freedesktop.org/libei/api/index.html):

1.

It now asks for a user confirmation to allow interaction, so we should now be able to click on native applications, window titles, etc.

However, our robot implementation is mostly uses XTEST, but there is an exception:
for mouse move we use XWarpPointer, which is not part of the XTEST, thus it is not propagated outside of Xwayland server.

So we have a situation where mouse is moving only inside Xwayland server, and we see the mouse enter/exit events,
on mouse press we call XTestFakeButtonEvent and it is handled by libei,
so the click happens where is the actual mouse cursor is in the system. That is why the test fails.

To avoid this situation we should use XTestFakeMotionEvent for mouseMove in Xwayland case.

It would be a simple solution, but


2.

This permission granted by a user doesn't persist across reboots, it timeouts after a period of inactivity.

It still seems to conform to the specifications, but we will not be able to conveniently run automated tests in this case.

---

So as a solution we can switch the [Remote Desktop XDG portal](https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.RemoteDesktop.html).
It has session and `restore_token` functionality to keep permissions granted by a user, similar to what we use in a [Screencast](https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.ScreenCast.html) for taking screenshots.
Most of the code can be reused, and it has everything we need.


* It is only enabled by default on Wayland and Gnome Shell 47 and higher due to XTEST API complications that first arose in this version. For earlier versions, Screencast is used instead.
* A new value has been introduced for the `awt.robot.screenshotMethod` system property - `dbusRemoteDesktop`. It works on Ubuntu 24.04 with Gnome 46, but is not enabled by default.
* extra mouse buttons are not supported yet, [JDK-8357142](https://bugs.openjdk.org/browse/JDK-8357142)
* Since we are now moving the mouse using a non-X11 API and `MouseInfo.getPointerInfo().getLocation()` uses the X11 API to get the mouse pointer location, `XRobotPeer#mouseMove` still calls the old API to update the mouse position within the XWayland server.
This workaround is necessary to pass the another conformance test.

-------------

Commit messages:
 - 8351907: [XWayland] [OL10] Robot.mousePress() is delivered to wrong place

Changes: https://git.openjdk.org/jdk/pull/25265/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=25265&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8351907
  Stats: 941 lines in 13 files changed: 804 ins; 51 del; 86 mod
  Patch: https://git.openjdk.org/jdk/pull/25265.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/25265/head:pull/25265

PR: https://git.openjdk.org/jdk/pull/25265


More information about the client-libs-dev mailing list