Smooth video list

Another morning hack: seeing if i can use JavaFX to display a smoothly operating user interface whilst crap is going on in the background.

I was thinking of something like the PlayTV library interface and so on, where one should be able to smoothly scroll even if the video is loading or something is busy. It sort of works on the PS3 but I/O really gets in the way there.

So here's a very rough prototype of the same idea - except here every video visible is also playing at the proper speed (sans audio again) as well.

Again: without seeing it in motion it doesn't really do it justice. The shot above is after scrolling a bit, and it shows the 'busy spinner' until the video starts playing, and then the video slowly fades in to cover it.

And this is a bit later. Everything is smooth and responsive and keeps up fine (on a beastly workstation of a couple years vintage). It runs ok even with the CPU driver, but the GPU one is much better.

I used a FlowPane for the layout, so it dynamically readjusts should one resize the window. I then manually calculate which parts of the FlowPane are currently visible, and use that to initiate the transition to loading the video. The video just restarts if one moves away and comes back, although if you knew the seek was going to work that would be possible too (libavformat cannot seek on some files, and the fallback is unpleasant).

I used jjmpeg for the decoding again, although this time I hit a problem with the way i was sending frames to the display - writePixels has to be on the GUI thread too. Got around it with a frame copy ... they're only small frames. It also has some pretty major memory leak somewhere which shows up when scrolling about ... might need to poke around jjmpeg some to track that or it could be errors in the listener code - i'm not really familiar with JavaFX callback rules yet.

With these particular videos (transcoded tv stuff in mp4), it still runs pretty smooth even if I maximise the window ... that's 8x7 ... 56 concurrent decodes and display (actually 64 when it's between rows). Less than 500% cpu on a 6-core HT machine (ok, so it's basically maxed out, but it still copes). (ho hum: if only the hardware video decoding wasn't locked away from being accessed by the customers who bought it, as it is, who gives a shit about that).

MediaPlayer

I also tried MediaPlayer but it wasn't so hot. I'm not sure if you can get it to scale the image data host-side, but without it it's pretty bad. Whilst it plays (oh and with sound, which isn't so good! I muted that) it's a lot slower (only some frames shown) and the whole GUI chunks around all over the place when you scroll: i.e. exactly not what I was trying to achieve in the first place. The scaling quality is poor too.

I'm assuming 'MediaPlayer' must be created from the GUI thread here, it seems to be doing a good bit of synchronous stuff when it is.

I also seem to have some memory leaks here too, but whatever the reason they add up much much faster than with the jjmpeg stuff.

Source

I checked it into MediaZ/fxperiments/VideoLibrary, and it needs a checkout of jjmpeg to work (Netbeans).

Update: I wasn't going to post it but I also created a really shitty video of it in action: didn't realise my camera only does 720P@24, and the aliasing of the LCD pixels is very poor. I also downed the bitrate to make it smaller. But you can get an idea of how it runs anyway.

• javafx-library.mp4 (there is no sound)