Please provide complete information as applicable to your setup.
• Hardware Platform (Jetson / GPU) AGX Xavier
• DeepStream Version 5.0
• JetPack Version (valid for Jetson only) 4.4
I have successfully created a simple full frame filter (GaussianBlur) by modifying gstdsexample.cpp as shown in this post and am able to run the following pipeline:
gst-launch-1.0 --gst-debug-level=0 filesrc location= ~/data/ar.h264 ! h264parse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m batch-size=1 width=1920 height=1080 ! nvinfer config-file-path= /opt/nvidia/deepstream/deepstream-5.0/sources/apps/sample_apps/deepstream-test1/dstest1_pgie_config.txt ! nvvideoconvert ! dsexample full-frame=1 processing-width=1920 processing-height=1080 ! nvdsosd ! nvegltransform ! nveglglessink
Then I want to run the GaussianBlur filter before the nvinfer, so I move the dsexample element in front of nvinfer as below:
gst-launch-1.0 --gst-debug-level=0 filesrc location= ~/data/ar.h264 ! h264parse ! nvv4l2decoder ! nvvideoconvert ! dsexample full-frame=1 processing-width=1920 processing-height=1080 ! m.sink_0 nvstreammux name=m batch-size=1 width=1920 height=1080 ! nvinfer config-file-path= /opt/nvidia/deepstream/deepstream-5.0/sources/apps/sample_apps/deepstream-test1/dstest1_pgie_config.txt ! nvvideoconvert ! nvdsosd ! nvegltransform ! nveglglessink
when run above pipeline, I encounter the following error message:
Setting pipeline to PAUSED …
Using winsys: x11
Opening in BLOCKING MODE
0:00:03.291576256 12973 0x55be6e1580 INFO nvinfer gstnvinfer.cpp:602:gst_nvinfer_logger: NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1577> [UID = 1]: deserialized trt engine from :/opt/nvidia/deepstream/deepstream-5.0/samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_int8.engine
INFO: [Implicit Engine Info]: layers num: 3
0 INPUT kFLOAT input_1 3x368x640
1 OUTPUT kFLOAT conv2d_bbox 16x23x40
2 OUTPUT kFLOAT conv2d_cov/Sigmoid 4x23x400:00:03.291798728 12973 0x55be6e1580 INFO nvinfer gstnvinfer.cpp:602:gst_nvinfer_logger: NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1681> [UID = 1]: Use deserialized engine model: /opt/nvidia/deepstream/deepstream-5.0/samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_int8.engine
0:00:03.294633527 12973 0x55be6e1580 INFO nvinfer gstnvinfer_impl.cpp:311:notifyLoadModelStatus: [UID 1]: Load new model:/opt/nvidia/deepstream/deepstream-5.0/sources/apps/sample_apps/deepstream-test1/dstest1_pgie_config.txt sucessfully
Pipeline is PREROLLING …
Got context from element ‘eglglessink0’: gst.egl.EGLDisplay=context, display=(GstEGLDisplay)NULL;
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
terminate called after throwing an instance of ‘cv::Exception’
what(): OpenCV(4.3.0) /home/paul/Documents/cuda_python/install/opencv_contrib-4.3.0/modules/cudafilters/src/cuda/row_filter.hpp:172: error: (-217:Gpu API call) an illegal memory access was encountered in function ‘caller’./front.agx: line 1: 12973 Aborted gst-launch-1.0 --gst-debug-level=0 filesrc location= ~/data/ar.h264 ! h264parse ! nvv4l2decoder ! nvvideoconvert ! dsexample full-frame=1 processing-width=1920 processing-height=1080 ! m.sink_0 nvstreammux name=m batch-size=1 width=1920 height=1080 ! nvinfer config-file-path= /opt/nvidia/deepstream/deepstream-5.0/sources/apps/sample_apps/deepstream-test1/dstest1_pgie_config.txt ! nvvideoconvert ! nvdsosd ! nvegltransform ! nveglglessink
Then I comment out
filter->apply (d_mat, d_mat);
in gstdsexample.cpp and “make” and “sudo make install”, then I am able to run the above pipeline without error (of course this “do-nothing” dsexample deviates from my original intention to do filtering before nvinfer)
Question: why the same filter (opencv cuda) can run after nvinfer but cannot run before nvinfer? does the order matter? or am I missing something if I place the dsexample element infront of nvinfer? Please advise. Thank you for your help. (Be aware that Here the dsexample already removes all meta data processing shown in the original example and is stripped down to a simple full frame in-place filtering as mentioned in this post )