API Review RT:17407 Canvas Node

[Jim Graham]

This is the same problem as:

- an FX input callback taking an inordinately long time to update the SG 
in response to its inputs
- a Swing/AWT app already deals with when it takes too long in its 
paint, or input handling methods

Basically, the rule is that you don't do any complicated processing in 
those methods.  If you are responding to input then, even if the 
rendering of everything else could be done in parallel, you have to 
return quickly or you stall the flow of events which must be taken in 
order or you have other UI problems.

The one way in which it is better than AWT/Swing is that the rendering 
is actually delayed, so at least the rendering part is just remembering 
your commands which is very quick.  So, if your calculations are simple 
then the rendering doesn't add much overhead.  And if you have long 
complicated calculations to do then those are really the problem in your 
input handler, not the drawing.

If you want to do calculations on another thread and then render the 
results, you can do that (as you can with FX node management and/or 
AWT/Swing apps), but you can't render directly from that offline 
calculation thread unfortunately - you have to pass your results back to 
the UI thread using a runLater() type of mechanism and render in the UI 
thread.

I suppose one way we could get even better would be to offer an "offline 
GC" which could be rendered from any thread, but then must be flushed 
manually as I stated in another message.  I'd have to think about the 
least confusing way to present such an API and I'd have to see if there 
are any gotchas (font metrics is a difficult cross-thread issue right 
now so that may pose problems for instance - we've solved how to do it 
between the UI and render threads, but I'm not sure the solutions scale 
to arbitrary threads).

Still, even if we provided such a mechanism, it doesn't prevent someone 
from simply stupidly doing lots of complex calculations in the input 
thread as they can now - FX nodes or immediate mode or not, and they 
still need to be familiar with the "calculate, forward results" model 
regardless.  It just makes the "calculate and forward results" technique 
one small step more convenient by allowing the actual enqueuing of the 
rendering commands to occur in the offline thread and it would still 
have to communicate back "and here are my rendering commands to flush to 
the Canvas", so the only simplification here is that the return 
communication would be just a couple of rubber stamp method calls...

			...jim

On 4/25/12 12:47 PM, Dr. Michael Paus wrote:
> Am 25.04.2012 20:23, schrieb Jim Graham:
>> Right now when you draw it has to be on the thread on which you
>> respond to inputs or do other SG modifications.
>
> So in other words you are saying that while you are drawing you can't
> respond to any other input anymore, right?
> That means while you are drawing you can't move the canvas around with
> the mouse, zoom in and out and so on?
> Isn't that the definition of a non-responsive and bad user interface?
>
>> In other words, any thread where you can appropriately call
>> cv.setWidth(), group.add(cv), etc. is a thread you can render on. The
>> rendering is "remembered" in a buffer and it isn't actually drawn
>> until the render pass. At that point we simply eat the buffer and
>> render it in a tight loop so any pauses for thought in your rendering
>> code would only affect the event and SG modification threads, not the
>> render thread. (The pauses aren't stored in the buffer. ;)
>>
>> We looked at ways of relaxing that further, but they won't come in
>> this first round of the API. It would necessitate having multiple HW
>> contexts and have a context set aside to only be used for Canvas
>> rendering into its texture with a clearly defined buffer swap that has
>> to be synchronized across the threads. Unfortunately that model is not
>> easy to support with D3D, our most common hw solution, so it is still
>> just a pet theory...
>>
>> ...jim
>>
>> On 4/25/12 1:38 AM, Dr. Michael Paus wrote:
>>> Am 25.04.2012 10:03, schrieb Jim Graham:
>>>>
>>>> Hmmm... More variants of Canvas or more attributes to set...?
>>>>
>>> I was just thinking about potential use-cases for the canvas node and
>>> stumbled over the
>>> following question. How can I still provide a super-smooth GUI if I have
>>> a canvas drawing
>>> which lasts longer than the interval between two pulses. Maybe it even
>>> lasts more than
>>> a second. My understanding of the current canvas design proposal is that
>>> when I redraw
>>> the canvas, for whatever reason, the GUI is basically blocked for this
>>> time. Is that true?
>>>
>>> If the above interpretation is right I would like to propose a
>>> modification of the design
>>> which makes it possible to have a double buffer canvas where the
>>> currently active
>>> buffer is redrawn whenever the JavaFX runtime needs to redraw the node
>>> and where
>>> I can render into the second buffer from ANY thread at any time without
>>> disturbing the
>>> smoothness of the GUI. When I am done with the rendering I just signal a
>>> buffer switch
>>> and with the next pulse the new buffer is shown.
>>>
>>> What do you think?
>>>
>>> Michael
>>>
>
>