Optimizing H.264 streaming from HDMI2CI board

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

The examples:https://github.com/InES-HPMM/linux-l4t-4.4/wiki/hdmi2csi#examples describe how to optimize display performance using a custom GStreamer build that enables Userptr and Dmabuff mode.

My goal is to stream the captured video over H.264
This code works but has high CPU load (138%) and latency.

gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw, format=RGB, width=1920, height=1080, framerate=60/1' ! videoconvert ! 'video/x-raw, format=I420, width=1920, height=1080, framerate=60/1' !   omxh264enc control-rate=2 bitrate=4000000  ! 'video/x-h264, stream-format=(string)byte-stream' ! h264parse ! mpegtsmux ! udpsink host=239.10.10.10 port=7777 sync=false async=false

I attempted to optimize using the custom GStreamer plugin.
My best result has 115% load and somewhat better latency

gst-launch-1.0 v4l2src io-mode=rw device=/dev/video0 ! 'video/x-raw, format=UYVY, width=1920, height=1080, framerate=60/1' ! nvvidconv ! 'video/x-raw(memory:NVMM), format=NV12' !   omxh264enc control-rate=2 bitrate=4000000  ! 'video/x-h264, stream-format=(string)byte-stream' ! h264parse ! mpegtsmux ! udpsink  host=239.10.10.10 port=7777 sync=false async=false

A search of the forum indicates there was a defect with nvvidconv https://devtalk.nvidia.com/default/topic/932320/jetson-tx1/nvvidconv-plugin-and-v4l2-camera-problem-/post/4912334/#4912334: “nvvidconv doesn’t totally support the v4l2 memory type”. The comment dates back to 2016 and has presumably been fixed.

A presentation by ZHAW https://digitalcollection.zhaw.ch/bitstream/11475/6187/1/211383.pdf describes how much higher performance can be achieved, but I have been unable to translate that to the TX2 and H.264 streaming

GStreamer 1.8.3
R28.2.1

2

Hi,
For pure gstreamer pipeline, your pipeline is good:

gst-launch-1.0 v4l2src io-mode=rw device=/dev/video0 ! 'video/x-raw, format=UYVY, width=1920, height=1080, framerate=60/1' ! nvvidconv ! 'video/x-raw(memory:NVMM), format=NV12' !   omxh264enc control-rate=2 bitrate=4000000  ! 'video/x-h264, stream-format=(string)byte-stream' ! h264parse ! mpegtsmux ! udpsink  host=239.10.10.10 port=7777 sync=false async=false

One more thing you can try is to execute jetson_clocks.sh.

For using MMAPI + gstreamer, you can get better performance. Please check
https://devtalk.nvidia.com/default/topic/1031967/jetson-tx2/tegra_multimedia_api-dq-buffer-from-encoder-output_plane-can-not-completed/post/5251268/#5251268
And run

appsrc ! h264parse ! mpegtsmux ! udpsink

For your reference, below post runs [appsrc ! h264parse ! qtmux ! filesink]
https://devtalk.nvidia.com/default/topic/1028387/jetson-tx1/closed-gst-encoding-pipeline-with-frame-processing-using-cuda-and-libargus/post/5256753/#5256753