I’m attempting to get very low video latency, say below 80ms, between two TX1 boards using the provided camera. One TX1 displays the video and connected via ethernet, and another TX1 connected via WiFi which will be encoding and sending the video stream. We have it setup like this is because we’ll eventually want to use a TX1 on a hexacopter/drone.
So far I’ve only been able to get the latency down to roughly 1/4th a second using the following commands. Note that I had to build gstreamer myself to get the rtph265pay and rtph265depay elements as discussed on GStreamer RTP H.265 elements missing thread.
On the display side I run the following, via SSH cause it was easier to do hence exporting DISPLAY environment variable:
export DISPLAY=:0 cd ~/gst_1.6.0/out/bin ./gst-launch-1.0 udpsrc port=5000 ! application/x-rtp,encoding-name=H265,payload=96 ! rtph265depay ! h265parse ! queue ! omxh265dec ! nvoverlaysink sync=false async=false -e
And on the video encoding / video source side (display side’s IP is 192.168.5.2):
cd ~/gst_1.6.0/out/bin ./gst-launch-1.0 nvcamerasrc fpsRange="30 30" intent=3 ! nvvidconv flip-method=6 ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! omxh265enc low-latency=1 control-rate=2 bitrate=4000000 ! 'video/x-h265, stream-format=(string)byte-stream' ! h265parse ! rtph265pay mtu=1400 ! udpsink host=192.168.5.2 port=5000 sync=false async=false
As you may noticed that I am vertically flipping the video image (nvvidconv flip-method=6 part) cause for whatever reason the camer’s video is coming in flipped and I’m undoing that odd video result.
The overall network latency is low in our test setup since the boards are only maybe 10 feet away from the wireless access point and it’s on channel that doesn’t seem to be congested. We’re seeing an average of 3ms over 50 pings between said boards and naturally an occasional 150ms here and there as you normally do see over wireless blip up.
So there you have it. I am curious if anybody has suggestions or ideas how I go about reducing the latency to hopefully down to our maximum video latency target range.