Canvas memory issues and performance

[Scott Palmer]

Interesting, this made me take another look at my implementation…

I discovered I am already using the pre-multiplied BGRA PixelFormat.  The GUI in the application slows down quite a bit without an HD video preview running.  Just by updating Paths as the user drags a node around (see http://javafx-jira.kenai.com/browse/RT-25166).  Add HD-sized video at 60fps and you have a lot of garbage to process as well.  

Anything that stalls the thread that is processing the Canvas command pipe causes a backup of video frames to fill the heap - that leads to GC issues and it makes the problem worse.  I still have doubts over the usability of Canvas for realtime video in a non-trivial sample app.  But it gets worse...

There is also this issue http://javafx-jira.kenai.com/browse/RT-24903 that I filed some time ago (complete with reproducible test case).  Basically Canvas stops processing it's command queue when it is not visible.  But it could be as simple as being temporarily scrolled out of view.  It is unreasonable to detect this at stop sending it frame buffers.  So Canvas is most definitely not usable for my situation.

I was thinking of getting access to a native surface (Direct3D, or OpenGL texture?) and rendering directly into it.  But I appreciate the complexity that may be involved with such an implementation.

Scott

On 2012-12-21, at 3:13 PM, John Smith <John_Smith at symantec.com> wrote:

>> Canvas is sort of close to what is needed, but it fails completely for my use case and simply crashes the app because it runs out of memory.  The FX Thread doesn't service the changes to the canvas fast enough for me to paint live video into the canvas.
> 
> I used Canvas (and WritableImage) for doing 1024x1024px animations at 60fps and it did not crash or run out of memory.  The FX Thread serviced the changes fast enough for smooth animation.  Others have displayed smooth 1920x1080 full screen video using Canvas (see https://forums.oracle.com/forums/thread.jspa?threadID=2434712).  The PixelFormat used for the WritableImage can make a huge difference to performance.