gstreamer h264 RTP video streaming problem.

Autonomous Machines Jetson & Embedded Systems Jetson TX1
1

Hi there,

I followed the multimedia user guide from L4t R23.2 to test the H264 video streaming, decoding the file with below pipeline work without any problem.

gst-launch-1.0 filesrc location=<filename.mp4> ! qtdemux name=demux demux.video_0 ! queue ! h264parse ! omxh264dec ! autovideosink

so next I am trying to send out the h264 data directly after qtdemux from video source, here are the two pipelines for sender and receiver.

receiver:

gst-launch-1.0 -v udpsrc address=127.0.0.1 port=1234 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! omxh264dec ! autovideosink

sender:

gst-launch-1.0 filesrc location=<filename.mp4> ! qtdemux ! rtph264pay ! udpsink host=127.0.0.1 port=1234

so those two pipeline can negotiate, but no video start playing, I can make this work by decoding file source to raw data then encoder back again with setting the “insert-sps-pps” property of omxh264enc to true, so it seems that sps and sps information is missing so that receiver can not parse the data.

gst-launch-1.0 -v filesrc location=<filename.mp4> ! qtdemux ! h264parse ! omxh264dec ! 'video/x-raw, format=(string)NV12, width=(int)4096, height=(int)1716, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, framerate=(fraction)24/1' ! omxh264enc insert-sps-pps=true ! 'video/x-h264, stream-format=(string)byte-stream' ! h264parse ! rtph264pay ! udpsink host=127.0.0.1 port=1234

the caps between omxh264dec and omxh264enc is some basic information about the video source, so do anyone know if there are pipelines can handle SPS PPS correctly, without decoding and encoding again, thank you.

2

The rtph264pay element can add this information itself.

gst-launch-1.0 filesrc location=<filename.mp4> ! qtdemux ! rtph264pay config-interval=1 ! udpsink host=127.0.0.1 port=1234

With this interval gstreamer will add SPS PPS information every second. Setting it to 2 result in information every 2 seconds etc. With newer versions of gstreamer (1.6.0+) you can also set the interval to -1 which will add the information on every I-frame.

3

Thanks CPeppenster, I tried this method, since we are streaming 4k video, on the receiver side, it seem lots of frames are dropped due to the high bitrate data, we may consider to using H265 instead.

4

While streaming over UDP dropped can be a real problem. You could consider implementing a RTSP server (gst-rtsp-server). On top of RTP for video streaming the server also sets up a backward RTCP connection for control, such as packet retransmission. This may help with the dropped frames.

5

Hello,

Are there any examples of video streaming with the RTCP elements. I haven’t found any and was wondering how this could be achieved in GStreamer.

Regards.