H.265 encoding and decoding speed issues on Xavier

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Xavier
1

Hello,

Here is my pipeline for a sender and receiver from a usb webcam:

sender

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw, width=640, height=480 ! omxh265enc ! mpegtsmux ! udpsink host=127.0.0.1 port=5000

receiver:

gst-launch-1.0 udpsrc port=5000 ! tsparse ! tsdemux ! h265parse ! omxh265dec ! nvvidconv ! tee ! xvimagesink

When I run the receiver I get the follow error messages before it begins displaying a choppy stream:

nvidia@nvidia:~/ai/testscripts$ gst-launch-1.0 udpsrc port=5000 ! tsparse ! tsdemux ! h265parse ! omxh265dec ! nvvidconv ! tee ! xvimagesink 
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock

(gst-launch-1.0:16031): GStreamer-CRITICAL **: 14:43:46.035: gst_caps_is_empty: assertion 'GST_IS_CAPS (caps)' failed

(gst-launch-1.0:16031): GStreamer-CRITICAL **: 14:43:46.035: gst_caps_truncate: assertion 'GST_IS_CAPS (caps)' failed

(gst-launch-1.0:16031): GStreamer-CRITICAL **: 14:43:46.035: gst_caps_fixate: assertion 'GST_IS_CAPS (caps)' failed

(gst-launch-1.0:16031): GStreamer-CRITICAL **: 14:43:46.035: gst_caps_get_structure: assertion 'GST_IS_CAPS (caps)' failed

(gst-launch-1.0:16031): GStreamer-CRITICAL **: 14:43:46.036: gst_structure_get_string: assertion 'structure != NULL' failed

(gst-launch-1.0:16031): GStreamer-CRITICAL **: 14:43:46.036: gst_mini_object_unref: assertion 'mini_object != NULL' failed
NvMMLiteOpen : Block : BlockType = 279 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 279 
Allocating new output: 640x480 (x 10), ThumbnailMode = 0
OPENMAX: HandleNewStreamFormat: 3605: Send OMX_EventPortSettingsChanged: nFrameWidth = 640, nFrameHeight = 480 
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:17.149716781
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

I must be decoding something incorrectly. Please let me know if there is a way I can fix this.

Many thanks!

2

Hi,
Please execute ‘sudo nvpmodel -m 0’ and ‘sudo jetson_clocks’, and try ‘nvoverlaysink sync=false’ in receiver.

gst-launch-1.0 udpsrc port=5000 ! tsparse ! tsdemux ! h265parse ! omxh265dec ! nvoverlaysink sync=false
3

Hi Dane,

So this works in essence… But I still see artifacts occasionally.

With that said, I have a few questions:

  1. The sync=false is there to say dropping frames is okay. In my application every frame is important. Therefore, it goes against the UDP protocol. Is there a different protocol perhaps tcp? that I can use that will guarentee each frame without the studder?

  2. Once these pipelines are sound, I will be using cv2 to run these gstreamer pipes.

How can I modify my commands to not use nvoverlaysink, since I do not need it to display on the monitor. Are the following commands okay?

Sender:

# grab images from webcam
    gst_str = ('v4l2src device=/dev/video{} ! '
               'video/x-raw, width=(int){}, height=(int){}! '
               'videoconvert ! appsink').format(dev, width, height)
    return cv2.VideoCapture(gst_str, cv2.CAP_GSTREAMER)
.
.
.
# send those same images to receiver
    gst_str = 'appsrc ! videoconvert ! omxh264enc ! mpegtsmux ! udpsink host=127.0.0.1 port=5000'
    return cv2.VideoWriter(gst_str, cv2.CAP_GSTREAMER, 0, fps, (width, height))

Receiver:

gst_str = ('udpsrc port=5000 ! '
               'tsparse ! tsdemux ! '
               'h264parse ! omxh264dec ! '
               'nvvidconv ! '
               'tee !'  
               'video/x-raw, width=(int){}, height=(int){}, '
               'format=(string)BGRx ! '
               'videoconvert ! appsink').format(width, height)
    return cv2.VideoCapture(gst_str, cv2.CAP_GSTREAMER)
4

I am not sure why the same post was posted multiple times. If you can, please remove the multiple copies and keep only one. Thank you!

5

Hi,
The receiver code looks fine. Not sure about the sender code. Other users may share experience.

We have samples in
https://developer.nvidia.com/embedded/r32-2-3_Release_v1.0/Sources/T186/public_sources.tbz2

Please download the package and try.

6

Gstreamer has plugins for RTP which allows the use of the rtpjitterbuffer to smooth out the frames. Gstreamer also has support for HTTP if you need every frame delivered but this will likely have a performance impact.

gst-inspect-1.0 | grep rtp

gst-inspect-1.0 | grep http