Video tearing when modifying slice-header-spacing in omxh264enc

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

Problem:
Because of reasons I’ve modified the slice-header spacing to use more slices per I-frame in this test case, this causes tearing on the receiving end.

The problem appears as if individual slices are getting decoded without an entire I-frame getting buffered up for the omxh264dec? This is a bit strange as the Tegra decoder is supposed to only work on a frame level…

Perhaps this problem can be alleviated by correct synchronization of GstBuffer:s on the receiving end?

Repro:
The problem is easily reproduced on a standard Jetson TX2 dev kit:

# Sender:
gst-launch-1.0 nvcamerasrc ! 'video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)I420, framerate=(fraction)60/1' ! nvvidconv flip-method=0 ! 'video/x-raw(memory:NVMM), format=(string)I420' ! omxh264enc iframeinterval=1 bit-packetization=TRUE slice-header-spacing=450000 control-rate=2 preset-level=0 profile=1 qp-range=-1,-1:10,10:-1,-1 ! 'video/x-h264, stream-format=(string)byte-stream' ! h264parse ! rtph264pay mtu=60000 ! udpsink host=127.0.0.1 port=5000

# Reciever:
gst-launch-1.0 udpsrc port=5000 ! "application/x-rtp,encoding-name=H264,payload=96" ! rtph264depay ! h264parse ! omxh264dec ! videoconvert ! xvimagesink async=TRUE sync=TRUE
2

Hi Jimmy,
Is the issue observed in saved video?

gst-launch-1.0 nvcamerasrc ! 'video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)I420, framerate=(fraction)60/1' ! nvvidconv flip-method=0 ! 'video/x-raw(memory:NVMM), format=(string)I420' ! omxh264enc iframeinterval=1 bit-packetization=TRUE slice-header-spacing=450000 control-rate=2 preset-level=0 profile=1 qp-range=-1,-1:10,10:-1,-1 ! 'video/x-h264, stream-format=(string)byte-stream' ! h264parse ! qtmux ! filesink location=a.mp4
3

Not it doesn’t appear in this instance.

This could suggest its RTP packet related, MAYBE as soon as a frame is larger than the SHS
=> it will be split over multiple slices (correct)
=> multiple RTP packets per frame (correct)
=> the omxh264dec decodes one slice at a time not an entire frame (incorrect)
=> tearing

Example: direct decode and display, works:

gst-launch-1.0 nvcamerasrc ! 'video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)I420, framerate=(fraction)60/1' ! nvvidconv flip-method=0 ! 'video/x-raw(memory:NVMM), format=(string)I420' ! omxh264enc iframeinterval=1 bit-packetization=TRUE slice-header-spacing=450000 control-rate=2 preset-level=0 profile=1 qp-range=-1,-1:10,10:-1,-1 ! 'video/x-h264, stream-format=(string)byte-stream' ! queue ! h264parse ! omxh264dec ! videoconvert ! xvimagesink
4

Adding an rtpjitterbuffer element resolved the issue, it synchronizes the RTP packets.

Example:

# Example
gst-launch-1.0 udpsrc port=5000 ! "application/x-rtp,encoding-name=H264,payload=96" ! rtpjitterbuffer  ! rtph264depay ! h264parse ! omxh264dec ! videoconvert ! xvimagesink async=TRUE sync=TRUE
5

Many thanks for sharing the solution.