API Review RT:17407 Canvas Node

[Daniel Zwolenski]

Would be good to see the prototype. I admit I'm confused on when/where this
Canvas is going to be appropriate and when it won't be. The below example
of drawing lines would have been one I'd not use the Canvas for based on
Joe's earlier general guideline on deciding between Canvas and Node graph
(i.e. it's not 'dynamic', apart from the fact it scrolls and zooms, but
then all the examples Joe indicated Canvas was not appropriate for did this
too).

My most common use case would be creating a graph/map viewer/editor with
panning and zooming. These may be schematics (e.g. CAD, P&ID, UML, Gantt,
timeline), or more image based (google maps, heat maps, digital scribble,
digital 'hand written' signatures) or usually combinations thereof (PDF or
CAD with scribble on it, maps with pipelines over them, thumbprints with
lines showing pattern matching, etc). Building this stuff in Swing was a no
brainer because there was no choice, you had your 'model' in non-Swing Java
code and then in the paint() method you would render a 'window worth' of
data into the Graphics context. The official/best way to Implement this in
in JFX is not clear to me. If Canvas is not for this, then a lot of my
comments/feedback will most likely make no sense (even less sense than
usual :) ).


On Fri, Apr 27, 2012 at 10:00 AM, Jim Graham <james.graham at oracle.com>wrote:

> Long story short - if we can get the prototype out there then I think you
> might find the following solution fairly efficient:
>
>        handleEvent(...) {
>            gc.clearRect();
>            gc.setTransform();
>            for (int i = 0; i < numlines; i++) {
>                gc.drawLine(line[i].x1, line[i].y1, line[i].x2, line[i].y2);
>            }
>        }
>
> It should perform better than N FX Line node objects (which should have
> outperformed an FX Polygon by several orders of magnitude) and have memory
> usage way, way less than N Line node objects and probably less than a
> single FX Polygon node object.
>
> It won't, however, have any better performance if you used
> gc.moveTo/gc.lineTo pairs than the FX Path would have had...
>
>                ...jim
>
>
> On 4/26/12 4:54 PM, Jim Graham wrote:
>
>> Hi Michael,
>>
>> You are pulling together a lot of issues but in the nexus of all of
>> these issues is a single problem that we don't yet have a solution for.
>> This doesn't invalidate all of the individual mechanisms because none of
>> them solve your particular problem, it just means that none of these
>> have yet addressed that one problem. I understand that problem can be
>> frustrating, but I have tried to share the issues with you in some of
>> those bug reports you have filed. Thank you for filing the issues so
>> they are on the table and we can remember they exist and track our plans
>> to eventually solve them.
>>
>> In most of your examples you are trying to draw, pan, and zoom a large
>> collection of lines. You either are able to use a large collection of
>> individual unconnected lines, or you are willing to use a large polygon
>> and not care how the segments of the polygon are connected to each other.
>>
>> Organizing them as a collection of Line nodes would probably get the
>> rendering to be very fast, but it adds a lot of overhead to create the
>> number of nodes you want to create. Using a Path or a Polygon node lets
>> you more efficiently declare the list of line segments, but we have
>> strict rendering policies about how the entire path and Polygon are
>> rendered which don't allow us to render it the same way as a bunch of
>> separate Line nodes.
>>
>> In Java2D/Swing you can render the lines either as individual calls to
>> drawLine, or as a Polygon or Path and if you set it up just right (which
>> is fairly easy because all of the defaults lead to just the right
>> conditions usually) then we can draw any of those really fast.
>> Unfortunately, FX has different defaults and we don't have (working,
>> yet) attributes you can modify to enable the conditions that allow the
>> use of simpler rendering algorithms to get your nest of lines drawn
>> equally as fast. Note that Java2D/Swing will fall down as well if you
>> turn on Antialiasing (on by default in FX with no supported way to turn
>> it off in our current implementations) or if you scale greater than 1.0
>> or use a line width greater than 1.0 where the join conditions do not
>> default well and have to be manually calculated.
>>
>> The Canvas node has the same rendering back end as the graphics nodes so
>> it isn't going to be magically faster, unfortunately, but that doesn't
>> mean that the "immediate mode" rendering API (which in this case doesn't
>> really mean rendered synchronously with the method call so perhaps
>> another term would be more appropriate) is not a boon for other
>> applications.
>>
>> Also, when you originally mentioned your doubts about being able to do
>> interactive graphics earlier in this thread, I didn't have the context
>> that you were referring to some rendering operations that we don't yet
>> efficiently handle. Yes, those long running rendering operations will
>> still delay the rendering thread whether you use the Canvas "immediate
>> mode" API or a node, unfortunately. But, for faster operations the
>> transfer of rendering calls to the rendering thread is not an overhead
>> that gets in the way.
>>
>> What Canvas would allow you to do, though, is write some code that draws
>> the collection of lines you want as individual calls to drawLine() and
>> they should go very fast. The overhead of putting the lines in a buffer
>> is a few orders of magnitude smaller than creating an equal number of
>> Line nodes, though, so I wouldn't transfer the frustration with the
>> overhead of using a Group of Line nodes onto using a Canvas with lots of
>> calls to GC.drawLine(). GC.drawLine will have overhead of 17 bytes per
>> line. It's been a while since I've seen figures on a Line node, and it
>> depends on how much binding and listening goes on, but let's just say
>> that you might find yourself using less than 1% of the memory that the
>> huge number of nodes was using - and getting the faster hw-accelerated
>> rendering of the separated line primitives...
>>
>> ...jim
>>
>> On 4/26/12 8:42 AM, Dr. Michael Paus wrote:
>>
>>> From what I have read about the new Canvas node now, I have to
>>> conclude that it will only be of very little use for me. I think
>>> it would be very important to clarify the design goals first
>>> before any decisions are made on the technical concept.
>>>
>>> My original expectation was that the Canvas is a Node which
>>> manages an image buffer (actually two of them for double
>>> buffering) and provides me with an immediate mode graphics context
>>> which would allow me to draw directly into this image buffer. My
>>> expectation was also that I could do this drawing at any time
>>> on a background thread so that I could keep long lasting renderings
>>> away from the normal rendering thread. In addition to that I
>>> expected that this graphics context would implement a general
>>> purpose 2D drawing interface (or extend an abstract class) which
>>> could later be used to implement graphics contexts for other output
>>> types like SVG, PDF, etc.
>>>
>>> I spent some of my spare time in the last couple of weeks to
>>> examine whether JavaFX could be used for GIS, CAD or similar
>>> applications which make heavy use of graphic primitives.
>>> My conclusion so far is that this would currently
>>> be very difficult if not impossible and I was hoping that the
>>> new Canvas would help me here to improve the situation but apparently
>>> it does not.
>>>
>>> Let me explain how I got to this conclusion.
>>>
>>> The first problem is drawing performance. When you have a complex
>>> drawing, a critical operation is to pan your drawing or to zoom
>>> in and out. The user expects this to be a smooth operation and
>>> so I started to examine how many graphic primitives you could
>>> have in your scene graph and still have an acceptable pan and zoom.
>>> (This was partly inspired by the first JIRA entry below.)
>>>
>>> The result is a bit frustrating because it turns out that JavaFX
>>> is actually quite slow for practical scenarios. At first I tried
>>> simple line shapes and on my laptop the limit seems to be at
>>> around 50,000 lines. This looks quite good but then I tried to
>>> put these lines into a Path object because in practice you do not
>>> deal so often with simple unconnected lines. With the lines within
>>> a Path the limit for an acceptable pan and zoom drops to around 500
>>> lines, which is not very much. This number can already be exceed
>>> by just implementing a fine background grid that way. One also
>>> has to know that even simple polylines and polygons do suffer from
>>> the same performance degradation because internally they are based
>>> on a path object too. Only simple lines, circles and rectangles
>>> are rendered in hardware. All other shapes are rendered in software
>>> and are too slow for the affore mentioned type of application.
>>> You also have to expect additional surprises. For example Rectangles
>>> are normally rendered in hardware and are thus quite fast unless
>>> you accidentally set the line join to Bevel. The performance then
>>> drops tremendously.
>>>
>>> The details and some explanations for all this can be found here:
>>> http://javafx-jira.kenai.com/**browse/RT-20405<http://javafx-jira.kenai.com/browse/RT-20405>
>>> http://javafx-jira.kenai.com/**browse/RT-20857<http://javafx-jira.kenai.com/browse/RT-20857>
>>>
>>> The conclusion here is that it is currently not possible to get
>>> an acceptable drawing performance for the non-trivial shape types
>>> even if you know that you will never use any of the features that
>>> hinder a hardware acceleration. There are no rendering hints or
>>> something like that so that the programmer could make a choice.
>>>
>>> To summarize this: Complex renderings just take too long and
>>> prevent a smooth pan and zoom operation. The new Canvas also won't
>>> be of any help here because, as I have learned, it also has to do
>>> its rendering on the rendering thread and thus delays all other
>>> rendering in the same way as the scene graph does. The only thing
>>> you get is the buffering effect but that could in principle be
>>> achieved already right now without the Canvas.
>>>
>>> In order to achieve such a buffering, and thus allowing a smooth pan,
>>> I put all my shapes in a group node and then activated the caching
>>> of this group node. All the rendering of my shapes is now captured
>>> in a bitmap and I get my smooth pan. In principle this works
>>> nicely but I have not been able to solve the zooming problem in
>>> the same way. It also does of course not solve the problem that
>>> the initial rendering is done on the rendering thread. I hoped
>>> that the Canvas would help here but apparently it doesn't.
>>>
>>> Handling a zoom in the same way did not work because there seems
>>> to be something wrong with the logic of the caching when you switch
>>> between QUALITY and SPEED. The details can be found here:
>>> http://javafx-jira.kenai.com/**browse/RT-20970<http://javafx-jira.kenai.com/browse/RT-20970>
>>>
>>> There are a lot more conceptual issues which make the use
>>> of JavaFX problematic for me at the moment. For example it is
>>> difficult to handle the selection of many overlapping shapes.
>>> Details can be found here:
>>> http://javafx-jira.kenai.com/**browse/RT-20452<http://javafx-jira.kenai.com/browse/RT-20452>
>>> http://javafx-jira.kenai.com/**browse/RT-20455<http://javafx-jira.kenai.com/browse/RT-20455>
>>>
>>> Working with Transforms is also problematic because you
>>> cannot perform any mathematical operations on them.
>>> (Not yet reported.)
>>>
>>> So coming back to the Canvas issue. I am still trying to understand
>>> which problem the current Canvas proposal is actually supposed to solve.
>>>
>>> - It does not provide a real immediate mode rendering into a bitmap.
>>> - It does not prevent the creation of a potentially huge amount of
>>> objects because it first creates a render list internally.
>>> - The render list is executed on the rendering thread and thus blocks it.
>>> - The render list cannot be created on a separate thread.
>>>
>>> Please forgive me if this posting was a bit long but I just had to
>>> write off my frustration from my soul (as we say here).
>>> You can now stone me if you like :-)
>>>
>>> Michael
>>>
>>>