Monocle with VNC [was: Re: openjfx/8u-dev/rt: RT-35441 [Monocle] Provide a VNC back-end]

[John Smith]

Monocle is a very interesting and exciting development for the JavaFX team.

I think that it is an avenue of approach which may help enable the development of various innovative solutions, both in providing access to JavaFX applications through traditional web UIs or providing remote GUI access to myriad small devices which make up the Internet of Things.


Ø  Since WebSockets are part of Java EE I don't think this is something that could be part of the standard JavaFX build

Yes, I agree.  There are some excellent existing WebSocket solutions for Java, such as Tyrus https://tyrus.java.net/ and Kaazing.  Though the WebSocket protocol is simple to users, there are many tricky pitfalls in creating a robust WebSocket server, so using those complete and well-tested solutions is the best route.  As the client app would actually be installed on the server much like a normal JEE app, using a standard JEE lib is perfectly appropriate I think.  The tricky part (at least for me) would be to take the VNC (or OpenGL) commands and tunnel them through a WebSocket system.

Anyway, something to think about in my spare time.

Thanks a lot for the replies and posting to the list Daniel.

John

From: Daniel Blaukopf [mailto:daniel.blaukopf at oracle.com]
Sent: Tuesday, January 21, 2014 2:45 PM
To: John Smith; openjfx-dev at openjdk.java.net
Subject: Re: Monocle with VNC [was: Re: openjfx/8u-dev/rt: RT-35441 [Monocle] Provide a VNC back-end]

Hi John,
On 01/22/2014 12:24 AM, John Smith wrote:
Would it be possible to develop a web based VNC client rendering to HTML canvas that connects to the Monocle VNC server over WebSockets such as that discussed in Kanaka’s answer here: http://stackoverflow.com/a/3902817/1155209?

I’m just interested in the feasibility of the approach as a basis for future development of a potential 3rd party JavaFX app deployment solution outside of JavaFX core.
I understand that this is probably not the primary reason the Monocle VNC work was implemented, but I’m curious if it could be repurposed for this mechanism.
Essentially, my goal here is to enable AppStreaming JavaFX http://aws.amazon.com/appstream/ without requiring a proprietary amazon solution.

In your opinion, would such a solution be a feasible basis for developing a distribution mechanism for JavaFX applications which does not require anything on the client outside of an HTML5 compliant browser?

It would absolutely be feasible, although the lag would probably be annoying. Since WebSockets are part of Java EE I don't think this is something that could be part of the standard JavaFX build, but Monocle allows for pluggable screen implementations and it shouldn't be much work to modify VNCScreen.java to do this. Alternatively we could put just enough of an implementation of WebSockets in VNCScreen to be able to talk to the browser directly. From http://www.websocket.org/aboutwebsocket.html the protocol doesn't look complicated. Unless you need a secure authenticated connection of course, in which case the answer would have to be Java EE.

However, what would be 100x better in terms of perfomance would be to stream the OpenGL commands to a WebGL client. We'd still have the lag though. We did an experiment for a few days last month in the JavaFX team with marshalling up OpenGL calls into a stream and sending them down the wire to a client doing the rendering on another device, and the performance looked promising. A bit like http://sourceforge.net/projects/virtualgl/, but without the video stream.

Thanks,
Daniel


Thanks,
John

From: Daniel Blaukopf [mailto:daniel.blaukopf at oracle.com]
Sent: Tuesday, January 21, 2014 12:53 PM
To: John Smith; openjfx-dev at openjdk.java.net<mailto:openjfx-dev at openjdk.java.net>
Subject: Monocle with VNC [was: Re: openjfx/8u-dev/rt: RT-35441 [Monocle] Provide a VNC back-end]

Hi John,

This is a mostly complete implementation of Glass that instead of rendering to the screen, renders to an offscreen buffer and serves the buffer up to clients using the RFB protocol. So you should be able to connect to it with most VNC clients, although I haven't been able to get it to work with desktop sharing on the Mac. You can also provide mouse input to JavaFX using the VNC client.

To use (BTW, this is noted in the JIRA https://javafx-jira.kenai.com/browse/RT-35441):

Run with: -Dglass.platform=Monocle -Dmonocle.platform=VNC -Dprism.order=sw

Connect with a VNC client to port 5901. I used TigerVNC (http://sourceforge.net/projects/tigervnc/files/) since the OS X desktop sharing client didn't like the 15-year old version of the RFB protocol that VNCScreen.java implements.

As noted in the JIRA, I think it would make sense to do what Dave Hill previously did in Lens and make the VNC server an option in all embedded back-ends instead of requiring the user to make a choice between having a local display or a remote display. So there would be Linux framebuffer with a VNC server option, EGL framebuffer with a VNC server option and headless with a VNC server option.

Thanks,
Daniel
On 01/21/2014 08:46 PM, John Smith wrote:

What does the VNC backend for Monocle do and how can it be used?



-----Original Message-----

From: openjfx-dev-bounces at openjdk.java.net<mailto:openjfx-dev-bounces at openjdk.java.net> [mailto:openjfx-dev-bounces at openjdk.java.net] On Behalf Of hang.vo at oracle.com<mailto:hang.vo at oracle.com>

Sent: Tuesday, January 21, 2014 5:48 AM

To: openjfx-dev at openjdk.java.net<mailto:openjfx-dev at openjdk.java.net>

Subject: hg: openjfx/8u-dev/rt: RT-35441 [Monocle] Provide a VNC back-end



Changeset: ebf7575491e0

Author:    Daniel Blaukopf <daniel.blaukopf at oracle.com><mailto:daniel.blaukopf at oracle.com>

Date:      2014-01-21 15:43 +0200

URL:       http://hg.openjdk.java.net/openjfx/8u-dev/rt/rev/ebf7575491e0



RT-35441 [Monocle] Provide a VNC back-end



! modules/graphics/src/main/java/com/sun/glass/ui/monocle/Framebuffer.java

! modules/graphics/src/main/java/com/sun/glass/ui/monocle/MonocleApplication.java

! modules/graphics/src/main/java/com/sun/glass/ui/monocle/headless/HeadlessScreen.java

+ modules/graphics/src/main/java/com/sun/glass/ui/monocle/vnc/VNCPlatfor

+ m.java

+ modules/graphics/src/main/java/com/sun/glass/ui/monocle/vnc/VNCPlatfor

+ mFactory.java

+ modules/graphics/src/main/java/com/sun/glass/ui/monocle/vnc/VNCScreen.

+ java