Gstreamer pipeline, converting I420 to RGB

Hi there,
I am attempting to stream from my Jetson Nano to a Linux machine on the network. Everything works except at the final stage, I am unable to convert to RGB - the pipeline just hangs (no error in Verbose mode).
Here is the transmitter:

gst-launch-1.0 -e nvarguscamerasrc ! \
	'video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)NV12, framerate=(fraction)120/1' ! \
	nvv4l2h264enc maxperf-enable=1 bitrate=8000000 iframeinterval=120 preset-level=1 control-rate=1 ! \
	h264parse config-interval=1 ! \
	rtph264pay pt=96 ! \
	udpsink host=$STREAM_TO port=8001 sync=false async=false

Here is the receive code - from Python (but the same issue happens in the gstreamer command line)

        self.pipeline = Gst.parse_launch(f'''
            udpsrc port=8001 ! application/x-rtp,encoding-name=H264,payload=96 ! 
            rtph264depay ! h264parse ! queue ! avdec_h264 !
            videocrop left=280 right=280 !
            videoscale ! video/x-raw,width=64,height=64 !
            tee name=t
            t. ! xvimagesink sync=false async=false
            t. !
            videoconvert !
            video/x-raw,format=RGBA !
            fakesink name=s

I have tried a simpler pipeline with just the xvimagesink as the final sink without any resizing as well. Everything works until I try and convert to RGBA (then it hangs).

The goal is to pluck the most recent frame out of the pipeline to use in a downstream Python application (using that fakesink). I can get the frame currently, but it is in I420 format. I suppose I could convert that manually, but I feel like gstreamer can just do it right?

I am pretty new to all of this - thank you in advance for your help.

You may try:

  • adding a queue in front of each subpipeline after tee. Without these you may face synchro issues such as deadlock.
  • adding rtpjitterbuffer between udpsrc and rtph264depay. Its default latency property is 2000 ms, you may try decreasing it and see…It may depend on having sync or not.
  • Try setting video format in caps between decoder and videocrop
  • If you’re streaming to only one machine, and especially if network is wifi, also try setting udpsink property auto-multicast=false in sender.
1 Like

Wow - that was an insanely fast and super helpful reply. So it sounds like when things hang - its because there is something going on with the network / pipeline that are just preventing everything from sync’ing up. I wish there was an error message : ) That totally makes sense to me - can debug with the things you suggested and just in general like… lowering the resolution, etc. to make things nicer on the network. A huge thank you again!

If you have to decrease resolution for having it working, one possible cause is max UDP socket buffer size.
You may try setting buffer-size property of udpsink/udpsrc to a bigger size than default.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.