Automatic virtualisation (was Developing controls based on Canvas?)

[Felix Bembrick]

With all this talk of node virtualisation, I am wondering how feasible it
would to build some kind of "automatic virtualiser" such that you pass it
your real-world model and then it automagically works out the actual nodes
and pools required and manages them seamlessly to maximise rendering
performance and responsiveness?  That is, it determines how many nodes will
be visible at any one time and just manages all that stuff for you
completely under the hood.

Maybe Jonanthan you have come some way to building such a beast with your
work on bidirectional virtualisation in FX 8 TableView or other controls?

Felix


On 9 August 2013 07:31, Felix Bembrick <felix.bembrick at gmail.com> wrote:

> T
> hanks Jonathan.
>
> I'll have to check out the virtualisation that you refer to that's going
> on in JavaFX8 with TableView., it sounds very interesting.
>
> I am not saying that controls such as what I am proposing are *impossible*
> to implement using a scenegraph; it just seems natural to implement them in
> a more procedural/programmatic way but this could very possibly just be
> because I do not have a full grasp of the capabilities of the scenegraph
> and what can be achieved by manipulating it.  To me it seems that the costs
> of repeatedly creating nodes (or hopefully reusing them), potentially
> restructuring the hierarchy of nodes in the scenegraph and adjusting each
> node's properties to enable a truly dynamic control would logically be much
> greater than just cycling through a render loop and rendering each graphic
> element in its correct location but, again, you seem very confident that
> these costs are not significant enough to prevent such a control performing
> well.
>
> In the end if I can get away with a scenegraph representation of this
> control and that massaging said scenegraph is performant enough then I am
> in a happy place.  I have already observed that some nice stuff can happen
> just by playing around the scenegraph so it is quite possible that I am
> worrying about performance and memory issues that will never actually
> eventuate in practice.
>
> Felix
>
>
> On 9 August 2013 07:16, Jonathan Giles <jonathan.giles at oracle.com> wrote:
>
>>  Felix,
>>
>> As you are restricted in what you can say, that also restricts how I can
>> help. However, your example of a spreadsheet not being a control that can
>> be implemented in a scenegraph-based manner suggests that you might want to
>> re-evaluate your assumptions. A spreadsheet would be able to be implemented
>> in a scenegraph approach quite nicely - the TableView control comes quite
>> close to that already (especially in JavaFX 8.0 where it can virtualise in
>> both vertical and horizontal directions). In fact, I know of at least two
>> spreadsheet implementations that already exist in JavaFX based on a
>> scenegraph approach.
>>
>> As I've said elsewhere, the issues you are facing are because of merging
>> two approaches (retained and immediate modes) into one control. What you
>> are hoping to do is have your cake and eat it too, which unfortunately I
>> don't think will end up too well for you :-) My advice is to recheck your
>> assumptions - the scenegraph approach, in my opinion, should suit your
>> requirements far more than an immediate mode approach (given that your
>> requirement is to reuse existing scenegraph-based controls).
>>
>> I'm not sure why you think controls must be defined in a static,
>> structured way. You could put all nodes of a control into a single Pane (or
>> Region, etc) layout and layout all the nodes using absolute positioning.
>> You can be as dynamic as you want to be.
>>
>> Unfortunately, without more information I am really unable to give any
>> further advice, but I wish you good luck in whichever approach you decide
>> to take.
>>
>> -- Jonathan
>>
>> On 9/08/2013 9:02 a.m., Felix Bembrick wrote:
>>
>>  Hi Jonathan,
>>
>>  Thanks for your reply.
>>
>>  I am a little restricted in exactly what I can reveal about my plans
>> for this control but I can say that it is one in which its very appearance
>> could change quite significantly in a dynamic way at runtime.  The control
>> also needs to support panning and zooming in a very performant way.  There
>> will likely be a lot of graphics being draw in a manner which (to me at
>> least) fits the immediate mode of rendering better than a scenegraph
>> because their positions and attributes are most readily defined
>> programmatically rather than declaratively.
>>
>>  Though not directly analogous, consider your typical spreadsheet
>> application like Excel where the user is able to pan to the right
>> effectively without limits and that grid lines are constantly being
>> rendered as the panning takes place.  Given that screens can be very large
>> these days it is conceivable that a complex spreadsheet will be displaying
>> hundreds of lines to define the cells in a grid at any one time and that it
>> is way more concise to programmatically define how this grid is rendered
>> rather than having a scenegraph containing a node for each line etc.  Also,
>> the panning and zooming responses are much simpler to implement
>> programmatically than continually fiddling around with the scenegraph.
>>  Then there's the whole issue of virtualisation and keeping the actual
>> number of "active" nodes to a minimum if it were to be done using a
>> scenegraph.
>>
>>  I am just not convinced that the costs of memory usage and processing
>> cycles to maintain a retained mode representation of a visual structure
>> like this can be justified or made performant when all you really need is a
>> simple algorithm that draws lines on the screen according to the properties
>> of your cell model.  Then consider that the *actual* control will be
>> significantly more complex and graphically rich than this simplistic
>> spreadsheet analogy.
>>
>>  To me this is the sort of control that really lends itself to an
>> immediate mode rendering component such as Canvas but there just seem to be
>> so many impediments in the way of actually building a professional control
>> with Canvas at its base.  If we continue with the spreadsheet analogy then
>> obviously cells would contain other controls etc. and we have discussed the
>> limitations on "embedding" other controls within a Canvas that seems to
>> suggest it's not practical.
>>
>>  I think in the end it gets down to the nature of the control itself and
>> whether its appearance is best defined by data or by an algorithm just like
>> the way certain types of knowledge are best defined by nouns whereas others
>> are best defined procedurally.
>>
>>  At this stage it is looking very much as though JavaFX really only
>> supports controls most appropriately defined in a static, structured way
>> like a scenegraph.  This in turns limits its applicability to a whole range
>> of software applications.
>>
>>  Felix
>>
>>
>> On 6 August 2013 10:10, Jonathan Giles <jonathan.giles at oracle.com> wrote:
>>
>>> I think it would pay to take a step back and understand why you think a
>>> 'traditional' scenegraph-based (or retained mode) control is not sufficient
>>> for your needs?
>>> Unfortunately you've not detailed your use case, so it is hard to give
>>> any specific advice. Are you able to give any details about what it is
>>> you're trying to build and why you think the normal approach to building
>>> controls is not sufficient?
>>>
>>> We've built some fairly complex controls using this approach, and if
>>> implemented wisely, there is very little that a scenegraph-based approach
>>> can't do. Specifically, do you think your control will render all of the
>>> 'thousands of nodes' at once, or will many of these nodes be off screen or
>>> otherwise not visible at any one time? For things like the TableView we
>>> only render the nodes that are visible. This means that regardless of
>>> whether there are 100 or 1,000,000 rows of data, we only have visual nodes
>>> for the 20 visible rows, for example. Keeping your scenegraph as minimal as
>>> possible is always a very wise idea, if performance is a concern.
>>>
>>> As you note, the other problem is that you will run into issues if you
>>> want to mix canvas rendering with the scenegraph-based controls like
>>> Button. The best you're likely to achieve (having not tried it personally)
>>> is to position the control on top of the canvas, rather than attempting to
>>> render the control inside the canvas (and having to then deal with event
>>> handling, etc). This will likely prove to be finicky, and more cumbersome
>>> than simply using an entirely canvas-based or entirely scenegraph-based
>>> approach.
>>>
>>> -- Jonathan
>>>
>>>
>>> On 5/08/2013 10:11 p.m., Felix Bembrick wrote:
>>>
>>>> I am investigating the feasibility of developing a JavaFX 8 control
>>>> based
>>>> on Canvas.  I have chosen Canvas as the base class as this control is
>>>> of a
>>>> very dynamic nature and would not be easy to implement with a retained
>>>> mode
>>>> style ancestor (at least as far as I can tell).
>>>>
>>>> So is this feasible?  While I can readily see how to render the visual
>>>> aspects of the control, I am not sure how to best "embed" other controls
>>>> within it should that become necessary (and almost certainly will).
>>>>
>>>> For example, how would I go about embedding a Button within my control?
>>>>  It
>>>> looks to me like I would need to create an actual Button node somewhere
>>>> else in the scenegraph and then perhaps render it within my control
>>>> using
>>>> gc.drawImage() passing in a snapshot of the Button node.  That's OK but
>>>> then I have to somehow handle events and I am not sure how best to do
>>>> that.
>>>>
>>>> Another issue I see is that there seems to be no way to apply effects to
>>>> individual graphic elements within the Canvas as the applyEffect()
>>>> method
>>>> applies to the entire Canvas.
>>>>
>>>> Finally, a significant obstacle is this issue:
>>>>
>>>> https://javafx-jira.kenai.com/browse/RT-23822
>>>>
>>>> This issue relates to the lack of support for LCD font smoothing within
>>>> Canvas.  This may not sound that serious but the difference between LCD
>>>> font-smoothed text in other controls and the grey-scale text in Canvas
>>>> is
>>>> so distinct on my current machine that a control based on Canvas would
>>>> really stick out like a sore thumb and appear significantly less
>>>> appealing
>>>> than a "standard" control.
>>>>
>>>> So, am I wasting my time?
>>>> Are there any other issues I am likely to face?
>>>> Are there other ways to develop dynamic controls which may involve
>>>> thousands of nodes (such as lines, curves etc.)?
>>>>
>>>> Thanks,
>>>>
>>>> Felix
>>>>
>>>
>>>
>>
>>
>