JavaFX Media issues

[John Hendrikx]

On 9/08/2013 02:10, Scott Palmer wrote:
> The Media APIs are mostly useless in their current state.  Other than
> demoing that you can play a video, they don't go far enough to be of
> practical value.  I tried to get someone to pay attention to them back in
> the JavaFX 1.0 days
> https://javafx-jira.kenai.com/browse/RT-2684

Unfortunately, I have to agree here.  I'd love to use the Media APIs of 
JavaFX, but they are way to limited.  At a minimum I'd need support for 
MKV containers and some common audio formats used with these.  Decoding 
AVI containers would be rather important as well.  Without those, JavaFX 
video is basically limited to "bring your own" videos only and 
completely unsuitable for playing back videos that end users might have...

Instead, I've been using VLC + VLCJ from day one while working with 
JavaFX as a work-around.

> at least someone listened to the request to get H.264 support in there, but
> that is just a workaround.  We need to be able to get our data into the
> media pipeline.  This would allow those of us that have attempted to do a
> video window to have a fighting chance.  Canvas can't keep up and will
> likely crash the app with out of memory errors.  Support for drawing into a
> native surface (OpenGL or D3D context) has been talked about, but doesn't
> appear to be on the horizon yet.  If we just had a hook to get the dang
> pixel data into the media pipeline so we could supply the "next frame" with
> whatever we want - either from any native codec via JNI, or dynamically
> generated from Java code, whatever... that would be just so dang useful...
> (to me at least)
This would atleast allow me to decode the MKV container myself and 
supply data... not looking forward to having to write all that code, but 
I would if it meant I'd be able to go "native" JavaFX with video.

Canvas however is working for me even with HD video (copying 25 
frames/sec of HD video from VLCJ to Canvas).  Playback is smooth even 
with 1920x1080 video for hours long video, and I can't tell the 
difference with a native player or copying frame by frame using 
pixelWriter anymore.  There is some CPU penalty but on a my system it is 
well below 5%.  This is basically how that looks with VLCJ:

         new RenderCallback() {
           @Override
           public void display(DirectMediaPlayer mp, Memory[] memory, 
final BufferFormat bufferFormat) {
             final ByteBuffer byteBuffer = memory[0].getByteBuffer(0, 
memory[0].size());

             Platform.runLater(new Runnable() {
               @Override
               public void run() {
                 pixelWriter.setPixels(0, 0, bufferFormat.getWidth(), 
bufferFormat.getHeight(), byteBgraInstance, byteBuffer, 
bufferFormat.getPitches()[0]);
               }
             });
           }
         }

However, the bug where Canvas keeps buffering data when off-screen 
sometimes bites me, resulting in out of memory errors -- even slight 
display delays can fairly easily cause this when video is running at 25 
fps (takes about a second to go through 256 MB of memory).

--John

>
> Regards,
>
> Scott
>
>
>
> On Thu, Aug 8, 2013 at 5:04 PM, Fabrizio Giudici<
> Fabrizio.Giudici at tidalwave.it>  wrote:
>
>> On Thu, 08 Aug 2013 22:57:51 +0200, Joe McGlynn<joe.mcglynn at oracle.com>
>> wrote:
>>
>>   I don't know why FX Media isn't in the FX 8 API docs, but that's clearly
>>> an error.  Please file a bug on that.
>>>
>>> In the meantime, you should look at the FX 2 media docs, there isn't a
>>> lot of change from FX2 media in FX8.  Buffering and streaming (HTTP Live
>>> Streaming) are both supported, as is playback from a URL.
>>>
>> What is the strategy for codecs? I mean, now we have ImageIO (there is
>> also JAI but it seems basically dead). ImageIO provides many image codecs
>> and there's a SPI that can be used to support more formats. Will it be
>> replaced by FX2 media or co-exist with it?
>>
>>
>> --
>> Fabrizio Giudici - Java Architect @ Tidalwave s.a.s.
>> "We make Java work. Everywhere."
>> http://tidalwave.it/fabrizio/**blog<http://tidalwave.it/fabrizio/blog>  -
>> fabrizio.giudici at tidalwave.it
>>