Nvivafilter and Deepstream pipeline compatibility

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) AGX Xavier
• DeepStream Version 5.0
• JetPack Version (valid for Jetson only) 4.4

I am able to run the following pipeline with nvivafilter successfully:

gst-launch-1.0
filesrc location= ~/data/ar.h264 ! h264parse ! nvv4l2decoder ! nvvidconv ! ‘video/x-raw(memory:NVMM), format=NV12, width=1280, height=720’ ! nvivafilter customer-lib-name=./lib-gst-custom-opencv_cudaprocess.so cuda-process=true ! ‘video/x-raw(memory:NVMM), format=RGBA, width=1280, height=720’ ! nvegltransform ! nveglglessink

I am aslo able to run nvinfer in the following pipeline:

gst-launch-1.0 --gst-debug-level=0 filesrc location= ~/data/ar.h264 ! h264parse ! nvv4l2decoder !'video/x-raw(memory:NVMM), width=1920, height=1080' ! nvvidconv ! 'video/x-raw(memory:NVMM), format=NV12, width=1920, height=1080' ! nvvideoconvert ! queue ! m.sink_0 nvstreammux name=m batch-size=1 width=1920 height=1080 ! nvinfer config-file-path= /opt/nvidia/deepstream/deepstream-5.0/sources/apps/sample_apps/deepstream-test1/dstest1_pgie_config.txt ! nvvideoconvert ! nvdsosd ! nvegltransform ! queue ! nveglglessink

However, when I combine the above two pipelines to nvivafilter => nvinfer into the pipeline, as below:

gst-launch-1.0 filesrc location= ~/data/ar.h264 ! h264parse ! nvv4l2decoder ! nvvidconv ! 'video/x-raw(memory:NVMM), format=NV12, width=1280, height=720' ! nvivafilter customer-lib-name=./lib-gst-custom-opencv_cudaprocess.so cuda-process=true ! 'video/x-raw(memory:NVMM), format=RGBA, width=1280, height=720' ! nvvideoconvert ! "video/x-raw(memory:NVMM), format=NV12" ! m.sink_0 nvstreammux name=m batch-size=1 width=1280 height=720 ! nvinfer config-file-path= /opt/nvidia/deepstream/deepstream-5.0/sources/apps/sample_apps/deepstream-test1/dstest1_pgie_config.txt ! nvvideoconvert ! nvdsosd ! nvegltransform ! nveglglessink

I get the following errors:

0:00:06.499927295 27125   0x558c06b190 INFO                 nvinfer gstnvinfer.cpp:602:gst_nvinfer_logger:<nvinfer0> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1681> [UID = 1]: Use deserialized engine model: /opt/nvidia/deepstream/deepstream-5.0/samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_int8.engine
0:00:06.508901994 27125   0x558c06b190 INFO                 nvinfer gstnvinfer_impl.cpp:311:notifyLoadModelStatus:<nvinfer0> [UID 1]: Load new model:/opt/nvidia/deepstream/deepstream-5.0/sources/apps/sample_apps/deepstream-test1/dstest1_pgie_config.txt sucessfully
Pipeline is PREROLLING ...
Got context from element 'eglglessink0': gst.egl.EGLDisplay=context, display=(GstEGLDisplay)NULL;
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
nvbuf_utils: nvbuffer Payload Type not supported
NvBufferGetParams failed for src_dmabuf_fd
nvbuffer_transform Failed
gst_nvvconv_transform: NvBufferTransform Failed 
ERROR: from element /GstPipeline:pipeline0/GstH264Parse:h264parse0: Internal data stream error.
Additional debug info:
gstbaseparse.c(3611): gst_base_parse_loop (): /GstPipeline:pipeline0/GstH264Parse:h264parse0:
streaming stopped, reason error (-5)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...

Any clue what am I missing? Please help. Thanks a lot.
Or does this imply the nvivafilter in multimedia SDK is not compatible with nvinfer in deepstream SDK? please advise. Thanks.

Hi,
The plugins are not compatilbe. Please check the explanation in this post:

For accessing the buffers in DeepStream SDK, you can leverage dsexample. Please look at

deepstream-5.0\sources\gst-plugins\gst-dsexample\README

Hi @DaneLLL, I recently came across the gst-nvdsvideotemplate plugin in DS5.1, which seems to be similar in spirit to nvivafilter. I.e., both allow passing along a dynamic library that looks like it can modify the source “in place”. Are dsexample and nvdsvideotemplate (with a customlib_impl) essentially two ways of accomplishing the same thing? Thanks!

Hi,
Both plugins should be able to be used fo rthis use-case. You may check source code of dsexample plugin first.