Nvcompositor with udpsrc not showing any results

Autonomous Machines Jetson & Embedded Systems Jetson Nano
1

hi, I’m streaming 2 cameras using udpsink with following commands:

gst-launch-1.0 v4l2src device=/dev/video0 io-mode=2 ! 'image/jpeg,width=1280,height=720,framerate=60/1' ! videorate ! 'image/jpeg,framerate=12/1' ! nvjpegdec ! 'video/x-raw' ! nvvidconv ! 'video/x-raw(memory:NVMM),width=640,height=480' ! omxh264enc iframeinterval=10 ! h264parse ! rtph264pay pt=96 ! udpsink host=10.10.87.26 port=8001 sync=false -e

gst-launch-1.0 v4l2src device=/dev/video1 io-mode=2 ! 'image/jpeg,width=1280,height=720,framerate=60/1' ! videorate ! 'image/jpeg,framerate=12/1' ! nvjpegdec ! 'video/x-raw' ! nvvidconv ! 'video/x-raw(memory:NVMM),width=640,height=480' ! omxh264enc iframeinterval=10 ! h264parse ! rtph264pay pt=96 ! udpsink host=10.10.87.26 port=8002 sync=false -e

i’ve confirmed that with filesink, the above command is working fine, i can save a video with them.

Then, I’m trying to use nvcompositor to show these two streams side-by-side with the following command:

gst-launch-1.0 nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=640 sink_0::height=480 \
sink_1::xpos=640 sink_1::ypos=0 sink_1::width=640 sink_1::height=480 ! nvoverlaysink \
udpsrc port=8001 ! 'application/x-rtp,payload=96,encoding-name=H264' ! rtph264depay ! h264parse ! omxh264dec ! comp.sink_0  \
udpsrc port=8002 ! 'application/x-rtp,payload=96,encoding-name=H264' ! rtph264depay ! h264parse ! omxh264dec ! comp.sink_1 -e

In the terminal, all commands do not return any errors, but there’s no displayed window to show the side-by-side result. Not sure what could be wrong?

3

Hi,
We are deprecating omx plugins. Please utilize nvv4l2h264enc and nvv4l2decoder in the pipeline.

4

Hi DaneLLL,

Thanks for the reply. i replaced omx with v4l2 plugins. But there’s still no display window. And it returns errors:

gst-launch-1.0 nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=640 sink_0::height=480
sink_1::xpos=640 sink_1::ypos=0 sink_1::width=640 sink_1::height=480 ! nvoverlaysink
udpsrc port=8001 ! ‘application/x-rtp,payload=96,encoding-name=H264,clock-rate=(int)90000’ ! rtph264depay ! h264parse ! nvv4l2decoder ! comp.sink_0
udpsrc port=8002 ! ‘application/x-rtp,payload=96,encoding-name=H264,clock-rate=(int)90000’ ! rtph264depay ! h264parse ! nvv4l2decoder ! comp.sink_1 -e

Setting pipeline to PAUSED ...
Opening in BLOCKING MODE 
Opening in BLOCKING MODE 
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 

(gst-launch-1.0:27246): GStreamer-CRITICAL **: 14:08:51.106: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(gst-launch-1.0:27246): GStreamer-CRITICAL **: 14:08:51.763: gst_mini_object_unref: assertion 'mini_object != NULL' failed
5

Hi,
Please check if single video playback works:

gst-launch-1.0 udpsrc port=8001 ! ‘application/x-rtp,payload=96,encoding-name=H264,clock-rate=(int)90000’ ! rtph264depay ! h264parse ! nvv4l2decoder ! nvoverlaysink
6

seems not, not showing any window, not returning any error tho

gst-launch-1.0 udpsrc port=8001 ! 'application/x-rtp,payload=96,encoding-name=H264,clock-rate=(int)90000' ! rtph264depay ! h264parse ! nvv4l2decoder ! nvoverlaysink

Setting pipeline to PAUSED ...
Opening in BLOCKING MODE 
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261
7

Hi,
Please remove h264parse and try:

gst-launch-1.0 udpsrc port=8001 ! ‘application/x-rtp,payload=96,encoding-name=H264,clock-rate=(int)90000’ ! rtph264depay ! nvv4l2decoder ! nvoverlaysink
8

It’s still not showing the window… it’s weird because it doesn’t return any errors, and it’s working with the videomixer, avdec_h264 and autovideosink

9

On encoding side, you would try either:

  • adding insert-sps-pps=1 to your H264 encoder options
  • adding config-interval=1 to h264parse options
  • adding config-interval=1 to rtph264pay options
10

I tried to debug this, and seems the nvv4l2decoder is not actually doing the work? the log is:

GST_DEBUG=3 gst-launch-1.0 udpsrc port=8001 ! 'application/x-rtp,payload=96,encoding-name=H264,clock-rate=(int)90000' ! rtph264depay ! nvv4l2decoder ! nvoverlaysink
0:00:00.110062918 32388   0x55886ded20 WARN                     omx gstomx.c:2826:plugin_init: Failed to load configuration file: Valid key file could not be found in search dirs (searched in: /home/slave/.config:/etc/xdg/xdg-unity:/etc/xdg as per GST_OMX_CONFIG_DIR environment variable, the xdg user config directory (or XDG_CONFIG_HOME) and the system config directory (or XDG_CONFIG_DIRS)
Setting pipeline to PAUSED ...
Opening in BLOCKING MODE 
0:00:00.121457094 32388   0x55886ded20 WARN                    v4l2 gstv4l2object.c:4430:gst_v4l2_object_probe_caps:<nvv4l2decoder0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:00.121535065 32388   0x55886ded20 WARN                    v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x55886e0fd0 Failed to determine interlace mode
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:02.903038090
Setting pipeline to PAUSED ...
Setting pipeline to READY ...

I’m using Jetpack4.4 L4T 32.4.3 Developer Kit Version . Is this a known issue in this version?

11

Hi,
This should be similar to

We are still working on it. Will update once we fix it.

12

Hi, I found that when I ssh -X nano to run the gstreamer pipeline using nvoverlaysink, it didn’t show any window in my laptop. But if i connect a hdmi cable to the nano and run the pipeline there, it can show the window. I’ve tried export DISPLAY=:0 (or 1) but didn’t help… Could you please help with this ?

13

nvoverlaysink expects a local display.
If you want to forward over network, you need to use X client, so use xvimagesink.
Don’t set DISPLAY for this case as ssh -X manages.

gst-launch-1.0 udpsrc port=8001 ! application/x-rtp,payload=96,encoding-name=H264 ! rtph264depay ! nvv4l2decoder ! nvvidconv ! xvimagesink
14

Hi,
So it should work if you remove h264parse from the pipeline. We don’t support X11 forwarding in NVIDIA sinks. Please try Honey Patouceul’s suggestion to use xvimagesink.