Performant Controls (hijacking Re: Developing controls based on Canvas?)

[Jonathan Giles]

I only said that I'd consider using Canvas given my past experiences in 
doing the same kind of thing in Java 2D, so I'm familiar with it in that 
sense (but not in the real sense of having ever used Canvas in 
production). If I were to ever implement something like this I would of 
course be informed by experimentation and investigation.

The other reason for my answer is that in reality I see them both as 
much the same thing, given I spend most of my time writing custom layout 
code anyway. This is what I said right at the end of my email: if you're 
doing absolute node positioning in the scenegraph you're most of the way 
to doing canvas stuff anyway, so I see them as relatively 
interchangeable. Perhaps that is a naive point of view and the canvas 
experts will correct me though.

I don't think it is worth being confused over. I think the general 
advice is quite clear, and the repercussions of either approach is quite 
well understood. When professionally evaluating which approach to take 
you should always be driven by use cases and experimentation. If you 
need a really clear answer on how to always proceed, my (once again 
naive and subjective) suggestion is this: always use scenegraph - if 
that doesn't work, consider canvas.

Of course, I welcome the canvas experts to chime in and correct me. I'm 
not an authority here and so everything should be taken with the 
appropriate amount of salt.

-- Jonathan

On 6/08/2013 2:06 p.m., Daniel Zwolenski wrote:
> Thanks Jonathan, it's good to get your insight.
>
> You did finish by muddying the waters again though - to do something 
> complex with zooming and scrolling you'd "be tempted" to fall back 
> into Java2D paint-style programming, and use Canvas for this, not the 
> Scene graph? It's more a couldbe/maybe comment though and is in 
> contrast to your earlier suggestion that there is very little that a 
> scenegraph-based approach can't do. What's the trigger to switch from 
> one approach to the other?
>
> Previously there have been comments about the Canvas not really being 
> intended for highly dynamic stuff (that was my interpretation of 
> comments on here when Canvas was first released), and Nodes should be 
> used for most real things. Richard wanted to use Nodes in the TD game 
> for sprites. To add to the confusion, Canvas currently has some 
> drastic z-order bugs, and some clipping issues, so using it combined 
> with Nodes is currently a no-go.
>
> I'm not expecting Jonathan to have an answer here really, just 
> highlighting the fact that there is no clear answer on this. I'm still 
> confused and I imagine many others are too. I think we'll see this 
> question again.
>
>
> On Tue, Aug 6, 2013 at 11:00 AM, Jonathan Giles 
> <jonathan.giles at oracle.com <mailto:jonathan.giles at oracle.com>> wrote:
>
>     I don't think there is any particular secret sauce going on in
>     what I do compared with the general guidelines that have been
>     spelled out numerous times. It's the same old, same old: don't
>     create more nodes than you need, don't modify the scenegraph
>     needlessly, don't update the scenegraph multiple times in a single
>     pulse, change state as little as possible, use as few listeners as
>     possible, etc. I wish I had something more groundbreaking for you,
>     but that is about it :-)
>
>     With respect to TableView (and ListView, TreeView, and
>     TreeTableView), they are all based on the same virtualisation code
>     (VirtualFlow for those of you playing at home). We don't rubber
>     stamp, we create separate cell instances for the visible area of
>     the control, and reuse these exact same cells as the user scrolls.
>     Therefore, if the visible area requires 20 cells, we may create
>     ~22 cells, and as the user scrolls downwards we take the cells
>     that disappear out the top of the view and place them at the
>     bottom of the view, with their new content in place before it is
>     shown on screen.
>
>     Because all cells come from a single cell factory, and all cells
>     can be used in any location, it is up to the cell to respond to
>     the item placed into it and draw itself appropriately. Therefore,
>     we don't have 1000's of types of cells in a single control, we
>     only have one type of cell that needs to handle all the visual
>     approaches required in the app. Realistically, there aren't 1000's
>     of styles in a single control, normally there are only one, or two
>     at most. All this takes place in the Cell.updateItem(T, boolean)
>     method, and so people overriding this method need to be smart and
>     not do heavy lifting in there. The biggest mistake I see people
>     doing in updateItem(...) is throw away their entire cell
>     scenegraph and recreate the nodes and update the scenegraph. This
>     is unwise.
>
>     If you have a ListView with 100 nodes, and they are all equally
>     sized except for one (say the 50th), which is _significantly_
>     bigger, you will see the scrollbar jump in size and other
>     weirdness happen when it is scrolled into view, precisely for the
>     reason you state - we can't go off and measure every row as we'd
>     be doing a lot of busy work. We only measure what is in the visual
>     area, and we don't know where we are in the scroll range in terms
>     of pixels but rather in terms of a 0.0 - 1.0 range (which is
>     translated back to pixels when needed). Up to this point I've
>     known about this issue but I've not spent the cycles to resolve it
>     - it is a relatively rare use case (although it still happens).
>     Priority #1 for these virtualised controls is always speed.
>
>     If zooming were required on TableView, the implication (I presume)
>     is that there would be that less cells that were visible at any
>     one time, and so we would end up having less cells in the
>     scenegraph. Other than that, things would work as above.
>
>     In a past life I did a lot of work in Java 2D. This worked really
>     well for use cases like you suggest at the end of your email,
>     namely zooming and direct mouse manipulation of nodes on screen.
>     If I were to write something like you show in the screenshot, I
>     would be tempted to take a Canvas-based route nowadays, but of
>     course that decision would also be driven by the requirements and
>     use cases, and it is possible a scenegraph-based approach with
>     absolute node positioning would work just as well.
>
>     Hope that helps.
>
>     -- Jonathan
>
>     On 6/08/2013 12:38 p.m., Daniel Zwolenski wrote:
>>     Sneaking in here, as you've given an opening with "if implemented
>>     wisely, there is very little that a scenegraph-based approach
>>     can't do". The question I've been asking for a while is what does
>>     "implemented wisely" look like in JFX.
>>
>>     This has come up in the performance conversations, the game
>>     conversations, the CAD conversations, and many other places. No
>>     one seems to have an answer, but you're building extremely
>>     complex stuff on a regular basis - what's your tips?
>>
>>     When you say you only have "20 visible nodes" out of 1000's in
>>     general are the other nodes:
>>     a) in the scenegraph and set to not visible
>>     b) in memory but not in the scenegraph - added/removed when
>>     scrolled into view and out of view
>>     c) not in memory, created, added and then removed, destroyed when
>>     scrolled into view and out of view
>>     d) something else?
>>
>>     I know Table uses a rubber stamp approach, where it re-uses cell
>>     views where possible, but let's say every row in my 100,000 row
>>     Table was uniquely rendered using a different cell. What would
>>     happen under the covers?
>>
>>     How do you work out the scroll range as well? Each cell can be a
>>     unique height right? How do you know the extents of the vertical
>>     scrolling without instantiating and rendering everything? Is this
>>     what you do? What if a cell is changing size (has a collapsable
>>     pane in it, etc) - what happens to the vertical scroll range?
>>
>>     Do any of the controls have zooming on them? Have you had to deal
>>     with this and have you got a strategy for handling this with
>>     respect to scroll bounds, working out which nodes are in view,
>>     scaling fonts, etc? Could you hazard a guess at what you would do
>>     if you had to implement zooming on a Table for example?
>>
>>     Maybe the Table is lucky with its restrictive grid like layout
>>     but imagine you had to build a visualisation of the same data but
>>     in a diagram, maybe something like
>>     http://www.novell.com/communities/files/img/groupwise_8_protocol_flow_diagram_v1.3.jpg
>>     but with x100 nodes, with zooming and panning - could you outline
>>     a general strategy?
>>
>>
>>
>>
>>     On Tue, Aug 6, 2013 at 10:10 AM, Jonathan Giles
>>     <jonathan.giles at oracle.com <mailto:jonathan.giles at oracle.com>> wrote:
>>
>>         I think it would pay to take a step back and understand why
>>         you think a 'traditional' scenegraph-based (or retained mode)
>>         control is not sufficient for your needs?
>>         Unfortunately you've not detailed your use case, so it is
>>         hard to give any specific advice. Are you able to give any
>>         details about what it is you're trying to build and why you
>>         think the normal approach to building controls is not sufficient?
>>
>>         We've built some fairly complex controls using this approach,
>>         and if implemented wisely, there is very little that a
>>         scenegraph-based approach can't do. Specifically, do you
>>         think your control will render all of the 'thousands of
>>         nodes' at once, or will many of these nodes be off screen or
>>         otherwise not visible at any one time? For things like the
>>         TableView we only render the nodes that are visible. This
>>         means that regardless of whether there are 100 or 1,000,000
>>         rows of data, we only have visual nodes for the 20 visible
>>         rows, for example. Keeping your scenegraph as minimal as
>>         possible is always a very wise idea, if performance is a concern.
>>
>>         As you note, the other problem is that you will run into
>>         issues if you want to mix canvas rendering with the
>>         scenegraph-based controls like Button. The best you're likely
>>         to achieve (having not tried it personally) is to position
>>         the control on top of the canvas, rather than attempting to
>>         render the control inside the canvas (and having to then deal
>>         with event handling, etc). This will likely prove to be
>>         finicky, and more cumbersome than simply using an entirely
>>         canvas-based or entirely scenegraph-based approach.
>>
>>         -- Jonathan
>>
>>
>>         On 5/08/2013 10:11 p.m., Felix Bembrick wrote:
>>
>>             I am investigating the feasibility of developing a JavaFX
>>             8 control based
>>             on Canvas.  I have chosen Canvas as the base class as
>>             this control is of a
>>             very dynamic nature and would not be easy to implement
>>             with a retained mode
>>             style ancestor (at least as far as I can tell).
>>
>>             So is this feasible?  While I can readily see how to
>>             render the visual
>>             aspects of the control, I am not sure how to best "embed"
>>             other controls
>>             within it should that become necessary (and almost
>>             certainly will).
>>
>>             For example, how would I go about embedding a Button
>>             within my control?  It
>>             looks to me like I would need to create an actual Button
>>             node somewhere
>>             else in the scenegraph and then perhaps render it within
>>             my control using
>>             gc.drawImage() passing in a snapshot of the Button node.
>>              That's OK but
>>             then I have to somehow handle events and I am not sure
>>             how best to do that.
>>
>>             Another issue I see is that there seems to be no way to
>>             apply effects to
>>             individual graphic elements within the Canvas as the
>>             applyEffect() method
>>             applies to the entire Canvas.
>>
>>             Finally, a significant obstacle is this issue:
>>
>>             https://javafx-jira.kenai.com/browse/RT-23822
>>
>>             This issue relates to the lack of support for LCD font
>>             smoothing within
>>             Canvas.  This may not sound that serious but the
>>             difference between LCD
>>             font-smoothed text in other controls and the grey-scale
>>             text in Canvas is
>>             so distinct on my current machine that a control based on
>>             Canvas would
>>             really stick out like a sore thumb and appear
>>             significantly less appealing
>>             than a "standard" control.
>>
>>             So, am I wasting my time?
>>             Are there any other issues I am likely to face?
>>             Are there other ways to develop dynamic controls which
>>             may involve
>>             thousands of nodes (such as lines, curves etc.)?
>>
>>             Thanks,
>>
>>             Felix
>>
>>
>>
>
>