How would one formulate a command for point-to-point 2K 60-90Hz streaming from one PC to another with lowest possible latency using ffmpeg?

Full story:

I built a VR headset from an eyebox off Amazon, gyro + arduino and 2K HDMI display off Alibaba. It works well with Blender Game engine but the wire drives me crazy.

I'm trying to make it wireless with a Raspberry Pi or some other small computer. (I need to add a weight at the back to balance the headset anyway)

This is how I imagine setting things up -

Server (PC): Stream 2K application window with lowest possible latency to one of the ports on the IP

Client (HMD): Run Mplayer in benchmark mode for lowest latency possible (per FFmpeg documentation)

FFmpeg is integrated fully into Blender but the input stream can be just a desktop screen capture.

The home router from my ISP supports 5Ghz Wifi. I also have a 5Ghz Wifi CSL USB stick which is plug-and-pay with Linux.

As latency is really bad for VR I guess I can shave off few milliseconds by streaming uncompressed? As streaming is only over LAN bandwidth is only limited I think by 5GHz Wifi which is I think like 100Mb/s.

There are hundreds of ways to interpret ffmpeg documentation, it looks like you can only really understand what options to use from experience. I can't see anyone else trying the same thing.

I could spend a week trying every possible setting but I'm pretty sure someone who deals with ffmpeg every day would know the best settings for this straight away.

This command gives about a second of delay and 25 FPS:

Server: avconv -f x11grab -s 2160x1200 -r 60 -i :0.0 -f mpegts udp://

Client: mplayer -benchmark udp://

If I use 'rawvideo' the stream won't be picked up by mplayer. It will say just "stream not seekable".

1 Answer 1

avconv -f x11grab -s 640x1200 -r 60 -i :0.0 -f mpegts udp://

avconv -f x11grab -s 640x1200 -r 60 -i :0.0 -f mpegts udp://

avconv -f x11grab -s 640x1200 -r 60 -i :0.0 -f mpegts udp://

avconv -f x11grab -s 640x1200 -r 60 -i :0.0 -f mpegts udp://


MPlayer in benchmark mode for no latency

More details: I found out essentially that the stream needs to be split in order to use more CPU. Using 'nice' to prioritize one single stream doesn't do anything. It's just pushing 2k at 25fps using maybe 30% of CPU. Where as if I split it into multiple streams as above I can get 60FPS on each tile with 85% CPU usage. The -threads 0 to 6 option doesn't behave as one would expect it to. It just makes rendeing slower if anything. H264 is better and the network has to be gigabit.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .