Canvas blowing up (was Re: JavaFX Media issues)

[John Hendrikx]

On 9/08/2013 17:23, Richard Bair wrote:
>> I mean, it looks like it is working for a few seconds,
>> but then as the memory fills with the Canvas backlog it can lead to the GC
>> using a lot more CPU, thus reducing the ability for Canvas to process its
>> command queue even further, well it just collapses in on itself  and dies.
> Forking the thread.
>
> The problem with Canvas is that if you have a canvas and you scribble on it, and then scribble on it some more, and then scribble on it some more, then in order for us to get the right result in the end, we need to replay all those scribbles in order. If pulses are not happening, we still need to remember these scribbles so we can draw the right result.
That's currently how it works, but I guess for quite a few users it is 
unexpected that a Canvas, which looks like basically a texture where you 
can draw on, has any other (significant) memory requirements apart from 
the initial allocation of the buffer based on its size.  I visualize the 
Canvas as a pixel buffer with low level tools to help you manipulate it, 
and these manipulations occur... well, immediately (like settings bits 
in memory).  Any memory allocated associated with the action you just 
did should disappear as soon as the action is 'applied' to the texture.

Perhaps we'd need more of a flush() style command that forces Canvas to 
apply all outstanding actions now and consolidate them into the texture 
so memory can be freed.
> BUT, if you issue a command to the canvas which will cause it to "clear" all its contents, then we could throw away any previously buffered data. Right now the only way to do that would be a fillRect with a solid fill where the fillRect encompasses the entire canvas area, or a clearRect where the clearRect encompasses the entire canvas area.
I think that's definitely a worthwhile solution to investigate.  For my 
use-case it would likely solve any OOM issues, as I can easily "clear" 
the canvas each frame (or better yet, Canvas could notice that the 
PixelWriter I use covers its full area).  Lacking that, I'd prefer to 
have a command that just discards previous actions than requiring an 
unnecessary clear/fill when I know that my next command is going to set 
all pixels anyway.

> This seems like a very simple fix. GraphicsContext.clearRect and GraphicsContext.fillRect should both (under the right conditions) throw away the previously buffered commands. Then all you have to do is be sure to make one of these calls (likely just a clearRect) before each frame, and we'll never buffer more than a single frame's worth of data. We could also add a "clear" method which is "clearRect(0, 0, w, h)" to make this more foolproof, and then document it as a best practice to clear the canvas before each rendering if you intend to redraw the entire thing on each frame.
>
> If you're making use of manually operated "dirty rects" so that you only clear the damaged area to repaint, then we couldn't employ this technique and we'd have to buffer 'till kingdom come. So we still need a mechanism exposed in the scene graph of "liveness" and associated events so that when the scene is no longer live (for example, when minimized) you could stop your animation timer, but for your specific media use case this isn't as important.
For my media use-case (and perhaps Scott's) it would definitely solve 
the issue -- I don't intend to do anything else with Canvas, all other 
things I want to draw will occur in normal JavaFX controls layered 
transparently on top of the Canvas.

I still think though that it is rather unexpected behaviour for Canvas 
that it can use significantly more memory (under certain conditions) 
than just the pre-allocated buffer -- even more so when you use 
PixelWriter -- I really expected that to go straight to the buffer, not 
make yet another copy.

--John