JavaFX Media Enhancements Survey

[Scott Palmer]

> On Jul 21, 2022, at 9:48 PM, Alexander Matveev <alexander.matveev at oracle.com> wrote:
> 
> Greetings!
> 
> We are running a little survey to get input on possible JavaFX Media enhancements. Below you can find a list of possible JavaFX Media enhancements, which we might implement in the future, although there is no commitment. We will appreciate if you can rate each enhancement on how useful/important it will be for JavaFX mainline. Put “1" for very important and I need this feature; “2" for looks important, but I am not planning to use it; “3" for not important and I do not need it.
> 
> 1) ID3 metadata support for MP4 files.
> Rating:
3
> 
> 2) Support for multichannel audio rendering (more than 2 channels).
> Rating:
2
> 
> 3) Subtitles support for MP4 files and HTTP Live Streaming.
> Rating:
2
> 
> 4) HDR support.
> Rating:
2
> 
> 5) InputStream for AudioClip with PCM support.
> Rating:
1
> 
> 6) Public APIs to access raw audio and video frames.
> Rating:
1
> 
> 7) Support for multiple audio tracks in MP4 files and alternate audio streams in HTTP Live Streaming provided via #EXT-X-MEDIA tag.
> Rating:
2
> 
> 8) Media recording/capture.
> Rating:
1
> 
> 9) Opus audio codec.
> Rating:
3
> 
> 10) MKV (matroska) file format.
> Rating:
3
> 
> 11) WebM/VP9/Opus
> Rating:
2
> 
> 12) RTSP (Real Time Streaming Protocol)
> Rating:
1
> 
> 13) Pluggable codecs
> Rating:
2
> 
> Please, list any additional media formats (codecs) / protocols you would like to see in upcoming JavaFX releases:
> 
> List any additional media enhancements you consider important to JavaFX Media:

I would like to be able to tap in to various points in the media pipeline. I would like to be able to feed the player with entirely synthetic data for example. E.g. a test pattern with burnt-in timecode. 
My primary application is a tool for building media processing workflows. I currently have to make a separate window in native code to preview the media to avoid extensive copying of uncompressed video frames. Most if the media processing I’m doing will be in my own native code, but since the UI is JavaFX I’m missing convenient points at the input and output stages. It would be great to be able to have a simple cross-platform camera and mic input for testing live streams (have tried OpenCV but it wasn’t that great). Being able to feed raw uncompressed video and audio data to the UI for preview would certainly clean up that aspect of the application. 

Cheers,

Scott