Link nvosd to nvv4l2h264enc

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
• DeepStream Version
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

I want to connect the nvdosd to nvv4l2h264enc , but shows and error

Elements could not be linked. Exiting.

I have checked the sink and src ok nvv4l2h264enc and nvosd using gst-inspect.
It shows the src of the nvosd is

SRC template: ‘src’
Availability: Always
format: { (string)NV12, (string)RGBA }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]

and the nvv4l2h264enc

SINK template: ‘sink’
Availability: Always
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
format: { (string)I420, (string)NV12, (string)P010_10LE, (string)NV24 }
framerate: [ 0/1, 2147483647/1 ]

The nvdsosd src caps can work with NV12 and RGBA and the nvv4l2h264enc sink can NV12 as well beside many format. But they cannot be linked. However if I add the nvvideoconvert between them it works.

why it worked if I add the nvvideoconvert. although it has the same caps as the nvv4l2h264enc

Hi @nomeis,

You need to understand how caps work in GStreamer. Even if there is a caps intersection between the elements, the caps event comes from the upstream elements. The source is the first one to fixate caps; then, the second element negotiates caps with the third using transform_caps, get_caps, and fixate_caps. By the time you get to nvv4l2h264enc, the pipeline has already negotiated RGBA as a format, which is not supported by the encoder. You can add a capsfilter to try and force a particular set of caps at a certain point in the pipeline, but I think you will still need the converter because nvinfer will force RGBA.

Whether the plugins are connected depends on the format of your actual stream. For example, if the video format from nvdsosd is RGBA, it cannot link the nvv4l2h264enc.

I have added a filter between nvdsosd and nvv4l2h264enc to make it NV12 which should be accepted by nvv4l2h264enc

GstElement * filter1 = gst_element_factory_make (“capsfilter”, “filter1”);
GstCaps *caps1 = gst_caps_from_string (“video/x-raw(memory:NVMM), format=NV12”);
g_object_set (G_OBJECT (filter1), “caps”, caps1, NULL);
gst_caps_unref (caps1);

Still I get the same error

It won’t work if you add capsfilter to the nvdsosd . The nvdsosd plugin cannot convert the format of the stream.