Wayland

[Thiago Milczarek Sayão]

Hi,

It seems there are some opportunities to explore with Wayland:

For kiosks (Ubuntu frame):
https://mir-server.io/

https://wayland.app/protocols/
There are some interesting extensions such as Tizen (samsung OS) and
"In-vehicle infotainment": https://wayland.app/protocols/ivi-application
I found it interesting because of the Wayland adoption for those scenarios.

I'm doing some experiments and it seems to fit well using wayland-client
directly.

I was thinking about keeping GTK for system dialogs such as file open - but
it can be done with DBUS and xdg-desktop-portal. So if the user is on KDE
it will display a KDE file open dialog:
https://flatpak.github.io/xdg-desktop-portal/docs/. Same for "Settings".

xdg-desktop-portal also has interfaces for "RemoteDesktop" which I think
can be used for Robot (to simulate as if it was using RDP and send cursor
movements and keys).

 -- Thiago

Em sex., 10 de nov. de 2023 às 09:43, Thiago Milczarek Sayão <
thiago.sayao at gmail.com> escreveu:

> Hi Johan,
>
> Robot will be a challenge.
>
> For mouse and keyboard simulation this could work:
> https://www.kernel.org/doc/html/v4.12/input/uinput.html
>
> of maybe:
>
> https://docs.flatpak.org/pt-br/latest/portal-api-reference.html#gdbus-org.freedesktop.portal.RemoteDesktop
>
> For screenshot:
>
> https://docs.flatpak.org/pt-br/latest/portal-api-reference.html#gdbus-org.freedesktop.portal.Screenshot
>
> Haven't tested anything yet.
>
> I did some experiments on the jfx-sandbox (it displays the window on
> wayland with software rendering), but the conclusion so far is that it's
> better to ditch gtk and use wayland-client directly, except for system
> dialogs such as file open.
>
> -- Thiago.
>
>
> Em sex., 10 de nov. de 2023 às 06:39, Johan Vos <johan.vos at gluonhq.com>
> escreveu:
>
>> Hi Thiago,
>>
>> Thanks for the work on Wayland. I spent some time on it in the past as
>> well, and I'll hope to find some time to look at your work soon.
>>
>> The main worry I had in the past was how to deal with the robot, where we
>> need to get pixels from the screen -- did you tackle that?
>>
>> - Johan
>>
>> On Fri, Nov 3, 2023 at 1:14 AM Thiago Milczarek Sayão <
>> thiago.sayao at gmail.com> wrote:
>>
>>> Hi,
>>>
>>> About Wayland:
>>>
>>> Porting es2 to use EGL instead of GLX is pretty straightforward, so
>>> converting a X11GL* to WaylandGL* is easy (GLX is X11 only, so this is why
>>> EGL is needed).
>>> The problem is Gtk4 and/or Gtk3 with Wayland won't allow you to paint
>>> directly to the window as es2 do.
>>> I've tried to use Gtk4 and composite it with a GL texture - it would
>>> work, but does't fit well with es2 as it is designed to swapBuffers.
>>>
>>> I'm looking for the shortest path to get it working.
>>>
>>> I'm thinking now to use wayland-client directly for most things, so it
>>> will be possible to create the WaylandGL* infrastructure and render
>>> directly to a surface.
>>> Compositing directly with wayland would be better anyways. Using Gtk (3
>>> or 4) would require rendering somewhere else (other than onscreen wayland
>>> surface) and then composite the result with Gtk.
>>>
>>> So, the conclusion for now is that Gtk won't work with wayland for the
>>> use of another toolkit (outside of the gtk rendering scope).
>>>
>>> wayland-client, here I go.
>>>
>>> -- Thiago.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> Em sáb., 21 de out. de 2023 às 18:12, Thiago Milczarek Sayão <
>>> thiago.sayao at gmail.com> escreveu:
>>>
>>>> EGL is the way to go for Linux on Wayland and X11.
>>>>
>>>> I don't know how to do it yet, but it seems the solution is to use
>>>> DMABUF with EGL and share it with GTK (I suspect on a GtkGLArea widget, but
>>>> not sure yet).
>>>>
>>>> Firefox and WebKit GTK uses the same technic.
>>>>
>>>> Gtk4 does not allow application painting, so direct surface context is
>>>> not possible.
>>>>
>>>> We are using gtk3 and it is still possible, but I want to use
>>>> GtkHeaderbar and clean up glass code. Since it means sharing the
>>>> window/surface with GTK, direct surface rendering also doesn't work.
>>>>
>>>>
>>>> https://mozillagfx.wordpress.com/2021/10/30/switching-the-linux-graphics-stack-from-glx-to-egl/
>>>>
>>>>
>>>> https://blogs.igalia.com/carlosgc/2023/04/03/webkitgtk-accelerated-compositing-rendering/
>>>>
>>>> https://blog.gtk.org/2021/05/10/adventures-in-graphics-apis/
>>>>
>>>> It's been nice to learn.
>>>>
>>>> Em seg., 16 de out. de 2023 20:51, Thiago Milczarek Sayão <
>>>> thiago.sayao at gmail.com> escreveu:
>>>>
>>>>> Hi,
>>>>>
>>>>> I am investigating about prism on Wayland.
>>>>>
>>>>> It looks like we could just replace the GLX calls on X11 backend to
>>>>> EGL and it will work on both (maybe rename it).
>>>>>
>>>>> It will need some work to pass the native display around - it assumes
>>>>> X11 Display everywhere and it could be a Display or a wl_display.
>>>>>
>>>>> -- Thiago.
>>>>>
>>>>> Em dom., 15 de out. de 2023 às 16:06, Thiago Milczarek Sayão <
>>>>> thiago.sayao at gmail.com> escreveu:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Update: It now works on wayland with -Dprism.order=sw
>>>>>>
>>>>>>
>>>>>>
>>>>>> Em dom., 15 de out. de 2023 às 10:49, Thiago Milczarek Sayão <
>>>>>> thiago.sayao at gmail.com> escreveu:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> https://github.com/openjdk/jfx-sandbox/tree/tsayao_wayland
>>>>>>>
>>>>>>> I did some experiments here. So far, so good.
>>>>>>>
>>>>>>> 1) Replaced GDK events for Gtk Signals - It's the way to go for
>>>>>>> newer gtk.
>>>>>>> 2) Replaced drawing directly in the window and added a
>>>>>>> GtkDrawingArea with a GtkHeaderBar which allows control over the whole
>>>>>>> window
>>>>>>> size and allows to get rid of extents* calls - this cleans up a lot.
>>>>>>> 3) Unified the WindowContext to clean it up.
>>>>>>>
>>>>>>> I also integrated the IME replacement proposed here:
>>>>>>> https://github.com/openjdk/jfx/pull/1080
>>>>>>>
>>>>>>> It almost runs with software rendering on Wayland, but something
>>>>>>> still touches X11.
>>>>>>>
>>>>>>> To finally make it work on Wayland it requires to implement it on
>>>>>>> prism es2. I see that there's a EGL part of Monocle. I still don't
>>>>>>> completely understand it, but wouldn't it work as a drop-in replacement?
>>>>>>>
>>>>>>> -- Thiago.
>>>>>>>
>>>>>>>
>>>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/openjfx-dev/attachments/20231116/8c9dd72c/attachment-0001.htm>