Robot's API createScreenCapture() question

[Alexander Zvegintsev]

Hi Olivier,


So here is an entry point of taking screenshot code for Linux's robot:

https://github.com/openjdk/jdk/blob/master/src/java.desktop/unix/native/libawt_xawt/awt/awt_Robot.c#L273


By default we are trying to use GTK 
<https://github.com/openjdk/jdk/blob/master/src/java.desktop/unix/native/libawt_xawt/awt/gtk3_interface.c#L2864>.

(operation does not fail, but producing black image on XWayland).

Anyway, it is using gdk_pixbuf_get_from_*() functions which are removed 
from GTK4 
<https://docs.gtk.org/gtk4/migrating-3to4.html#stop-using-gdk_pixbuf_get_from_window-and-gdk_cairo_set_source_window>.

> How does Robot's API createScreenCapture() deal with such composited setups
> even on plain Xorg?
>
> Corollary question, what actually fails when trying to use the
> createScreenCapture() API in Xwayland?
Regarding plain Xorg we are trying to get an image(by reusing xwd code 1 
<https://github.com/openjdk/jdk/blob/master/src/java.desktop/unix/native/libawt_xawt/awt/multiVis.c#L277>,2 
<https://github.com/openjdk/jdk/blob/master/src/java.desktop/unix/native/libawt_xawt/awt/multiVis.c#L496>) 


of Composite Overlay Window(section 3.2) 
<https://www.x.org/releases/X11R7.5/doc/compositeproto/compositeproto.txt> 
which are obtained via XCompositeGetOverlayWindow().


IIUC XWayland does not have a compositor role implemented(and will not 
have),

since compositing is controlled by Wayland's Display Servers.

--
Thanks,
Alexander.

On 22/09/2021 11:41, Olivier Fourdan wrote:
> Hi all,
>
> In a Wayland desktop, Xwayland cannot capture the content of the Wayland
> native windows (as those are handled by the Wayland compositor and not
> Xwayland), but X11's GetImage works on X11 toplevel windows in Xwayland,
> except for the root window which in Xwayland has no storage (when Xwayland
> is run rootless).
>
> IOW:
>
> $ xterm &
> $ xwd -id $WINDOWID | xwud
>
> Works fine in Xwayland, but:
>
> $ xwd -root | xwud
>
> Fails with a BadMatch for the reason mentioned above.
>
> Now I was wondering if we could make that work in Xwayland, but then I had
> some doubts as to how it currently works with Robot's API
> createScreenCapture().
>
> Typically, in a composited environment on Xorg, capturing the content of
> the root window will not fetch the content of all the other windows, just
> the bare content of the root window itself.
>
> xwd works around that by fetching the image of each toplevel window visible
> on screen and rebuilding a global image by copying those, basically, trying
> to replicate what the X11 compositor does.
>
> How does Robot's API createScreenCapture() deal with such composited setups
> even on plain Xorg?
>
> Corollary question, what actually fails when trying to use the
> createScreenCapture() API in Xwayland?
>
> I believe if we could make that work in Xwayland it would probably help the
> Robot API in Xwayland (even though it would be limited to X11 windows in
> Xwayland, it would work at least for Java own windows).
>
> Cheers
> Olivier