Streaming H264 1080p60

Hi,

This is probably an easy question, but I haven’t figured it out yet. I’m trying to stream an H264 1080p60 source from the TK1 to my desktop. The TK1 pipeline is…

gst-launch-1.0 v4l2src device=/dev/video1 ! video/x-raw,width=1280,height=720 ! omxh264enc ! rtph264pay ! udpsink host=192.168.2.100 port=1234

My receiving pipeline is…

gst-launch-1.0 udpsrc port=1234 ! application/x-rtp, encoding-name=H264,payload=96 ! rtph264depay ! decodebin ! autovideosink sync=false

Both of these pipeline seem to run, but I get no video. Any thoughts?

Thanks,
Kevin

Is there any kind of firewall issue?

No, I can stream MJPEG over RTP just fine.

I did work the following pipeline:

gst-launch-1.0 videotestsrc ! “video/x-raw, width=1500, height=1500, framerate=(fraction)1/30” ! omxh264enc insert-sps-pps=true ! rtph264pay ! udpsink host=192.168.0.56 sync=false

gst-launch-1.0 udpsrc ! application/x-rtp,payload=96 ! rtph264depay ! decodebin ! xvimagesink sync=false

Rather, the difference in line omxh264enc insert-sps-pps=true

Nice, that’s working. Quality isn’t quite there but it’s a start. I’ll play with that and post back.

My sender pipeline:

gst-launch-1.0 videotestsrc horizontal-speed=5 ! "video/x-raw, width=1920, height=1080, framerate=(fraction)1/30" ! omxh264enc insert-sps-pps=true ! rtph264pay ! udpsink host=192.168.2.100 port=1234 sync=false

My receiver pipeline:

gst-launch-1.0 -vvv udpsrc port=1234 ! application/x-rtp, encoding-name=Hpayload=96 ! rtph264depay ! avdec_h264 ! xvimagesink sync=false

Thanks,
Kevin

Ok, this is so close… Quality is looking good with my v4l2 source. My pipelines are as follows.

Sender:

gst-launch-1.0 v4l2src ! videoconvert ! "video/x-raw,format=I420, width=1920, height=1080, framerate=60/1" ! omxh264enc insert-sps-pps=true bitrate=16000000 ! rtph264pay ! udpsink host=192.168.2.100 port=1234 sync=false

Receiver:

gst-launch-1.0 -vvv udpsrc port=1234 ! application/x-rtp, encoding-name=Hpayload=96 ! rtph264depay ! avdec_h264 ! xvimagesink sync=false

The problem is now just the smoothness of the video…it’s very choppy at 1080p60 and somewhat choppy at 720p. At 1080p the cpu is only at 18% which is great, but the choppy video could be deal breaker.

I will continue to play with it…thoughts are always welcome!

Thanks,
Kevin

Hi

one potential issue leading to “choppy” video is that you set your videosink to sync=false. This means that it will spit out any frame as soon as it is received and not consider a regular timing (1/60Hz) or discard frames that arive too late. You might consider adding timestamps to the frames (e.g. in the v4l2src with do-timestamp=true) and setting your sink to sync=true.

Also you are using some software-plugins that could be replaced with hardware-accelerated plugins, like: videoconvert -> nvvidconv, avdec_h264 -> omxh264dec, xvimagesink -> nvhdmioverlaysink

For comparison, I have used the following pipelines:

Receiver:

export VCAPS='application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H264'

gst-launch-1.0 udpsrc port=5000 ! $VCAPS ! rtpjitterbuffer latency=1000 ! rtpmp2tdepay ! tsdemux ! h264parse ! omxh264dec ! nvhdmioverlaysink -vvv -e

Sender:

gst-launch-1.0 v4l2src do-timestamp=true ! 'video/x-raw,format=I420, width=1920, height=1080, framerate=60/1' ! nvvidconv ! 'video/x-raw(memory:NVMM), format=I420, width=1920, height=1080, framerate=60/1' ! omxh264enc ! h264parse ! mpegtsmux ! rtpmp2tpay ! udpsink port=5000 host=192.168.0.2 -e -vvv

Btw: for quality issues with the encoding: look at the bitrate parameter of omxh264enc

Hope that helps,
Regards
Tobias

Hi,

Thanks for your suggestions kamm. I tried messing with the timestamp and sync settings you brought up, but I didn’t see any noticeable improvement.

The hardware vs software plugins is where my concern is. First of all the receiving pipeline is running on my PC so I have no way to accelerate that, but I’m pretty confident that my CPU should be able to handle it.

On the sending side I’m using videoconvert instead of nvvidconv because my video source is YUV or RGB and nvvidconv doesn’t seem to support those formats. It’s kinda looking like my video source format is the problem.

On the other hand, the following test pipelines definitely aren’t smooth either…

Sender (Jetson)

gst-launch-1.0 videotestsrc do-timestamp=true horizontal-speed=15 ! "video/x-raw,format=I420, width=1920, height=1080, framerate=60/1" ! omxh264enc insert-sps-pps=true bitrate=16000000 ! rtph264pay ! udpsink host=192.168.2.100 port=1234 sync=true

Receiver (PC)

gst-launch-1.0 udpsrc port=1234 ! application/x-rtp, encoding-name=H264,payload=96 ! rtph264depay ! avdec_h264 ! xvimagesink sync=true

Thanks,
Kevin

Hi

(Sorry for the late answer)
First of all the topic of using nvvidconv, if you have UYVY (which is a YUV422 format http://www.fourcc.org/yuv.php#UYVY) on your input, you could still use nvvidconv. I am using it with a v4l2-source:

v4l2src device=/dev/video0 do-timestamp=true ! 'video/x-raw, width=3840, height=2160, framerate=30/1, format=UYVY' ! nvvidconv ! 'video/x-raw(memory:NVMM), width=3840, height=2160, framerate=30/1, format=I420' ! nvoverlaysink sync=false

In order to improve performance, you could also try to change the io-mode property of your v4l2src. Depending on your V4L2-driver, the element after the vl42src and the GStreamer version you are using, you could try different modes. We are seeing good results with the Userptr mode for capturing a video and showing it on the screen using xvimagesink.

gst-launch-1.0 v4l2src io-mode=3 ! 'video/x-raw, width=3840, height=2160, format=UYVY, framerate=30/1' ! xvimagesink -vvv

Lastly about the testing pipelines with videotestsrc: Be aware that the videotestsource runs on the CPU and for high resolutions and framerates it may be too slow and cause stuttering. To test this you can just view the test source directly on the output.

Regards
Tobias