Combining nvdsanalytics with rtsp-in-rtsp-out example

Please provide complete information as applicable to your setup.

**• Hardware Platform - Jetson **
• DeepStream Version 6.0.1
• JetPack Version: 4.6.1-b110
• TensorRT Version 8.2
• Requirement details

I am trying to combine the two deepstream_test1_rtsp_in_rtsp_out.py and deepstream_nvdsanalytics.py but I have the following error and my program is stopping. I am new to deepstream. Any idea regarding this issue? I can’t find much.

Creating Pipeline

Creating streamux

Creating source_bin 0

Creating source bin
source-bin-00
Creating Pgie

Creating nvtracker

Creating nvdsanalytics

Creating tiler

Creating nvvidconv

Creating nvosd

Creating H264 Encoder
Creating H264 rtppay
Adding elements #1 to Pipeline

Adding elements # to Pipeline

Linking elements in the Pipeline

*** DeepStream: Launched RTSP Streaming at rtsp://localhost:8554/ds-test ***

Opening in BLOCKING MODE
gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so
gstnvtracker: Batch processing is ON
gstnvtracker: Past frame output is OFF
[NvMultiObjectTracker] Initialized
0:00:01.360473460 20233 0x3581f900 WARN nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger: NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::initialize() <nvdsinfer_context_impl.cpp:1161> [UID = 1]: Warning, OpenCV has been deprecated. Using NMS for clustering instead of cv::groupRectangles with topK = 20 and NMS Threshold = 0.5
ERROR: Deserialize engine failed because file path: /opt/nvidia/deepstream/deepstream-6.0/sources/python/apps/deepstream_smarthub/…/…/…/…/samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_int8.engine open error
0:00:05.405424501 20233 0x3581f900 WARN nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger: NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1889> [UID = 1]: deserialize engine from file :/opt/nvidia/deepstream/deepstream-6.0/sources/python/apps/deepstream_smarthub/…/…/…/…/samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_int8.engine failed
0:00:05.406672551 20233 0x3581f900 WARN nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger: NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1996> [UID = 1]: deserialize backend context from engine from file :/opt/nvidia/deepstream/deepstream-6.0/sources/python/apps/deepstream_smarthub/…/…/…/…/samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_int8.engine failed, try rebuild
0:00:05.406728125 20233 0x3581f900 INFO nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger: NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1914> [UID = 1]: Trying to create engine from model files
WARNING: INT8 not supported by platform. Trying FP16 mode.
ERROR: Serialize engine failed because of file path: /opt/nvidia/deepstream/deepstream-6.0/samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_fp16.engine opened error
0:01:18.569406700 20233 0x3581f900 WARN nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger: NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1942> [UID = 1]: failed to serialize cude engine to file: /opt/nvidia/deepstream/deepstream-6.0/samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_fp16.engine
INFO: [Implicit Engine Info]: layers num: 3
0 INPUT kFLOAT input_1 3x368x640
1 OUTPUT kFLOAT conv2d_bbox 16x23x40
2 OUTPUT kFLOAT conv2d_cov/Sigmoid 4x23x40

0:01:18.623190476 20233 0x3581f900 INFO nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus: [UID 1]: Load new model:dsnvanalytics_pgie_config.txt sucessfully
Decodebin child added: source

Error: gst-resource-error-quark: Unauthorized (15): gstrtspsrc.c(6116): gst_rtspsrc_send (): /GstPipeline:pipeline0/GstBin:source-bin-00/GstURIDecodeBin:uri-decode-bin/GstRTSPSrc:source:
Unauthorized (401)
Exiting app

try to remove ugly pluging using this command :-
sudo apt remove gstreamer1.0-plugins-ugly

I tried, but it didn’t change anything. Same error.

Seems rtspsrc error. please ensure your rtsp server works fine.

the RTSP is working, it works for deepstream_test1_rtsp_in_rtsp_out.py
I think the problems come from the elements in the pipeline, I tried the following order based on deduction. Any suggestion regarding the elements linking for RTSP in RTSP out + nvdsanalytics?
pipeline.add(pgie)
pipeline.add(tracker)
pipeline.add(nvanalytics)
pipeline.add(tiler)
pipeline.add(nvvidconv)
pipeline.add(nvosd)
pipeline.add(nvvidconv_postosd)
pipeline.add(caps)
pipeline.add(encoder)
pipeline.add(rtppay)

I tried first with as in the nvdsanalytics example:
streammux.link(queue1)
queue1.link(pgie)
pgie.link(queue2)
queue2.link(tracker)
tracker.link(queue3)
queue3.link(nvanalytics)
nvanalytics.link(queue4)
queue4.link(tiler)
tiler.link(queue5)
queue5.link(nvvidconv)
nvvidconv.link(queue6)
queue6.link(nvosd)
nvosd.link(queue7)
queue7.link(nvvidconv_postosd)
nvvidconv_postosd.link(queue8)
queue8.link(caps)
caps.link(queue9)
queue9.link(encoder)
encoder.link(queue10)
queue10.link(rtppay)
if is_aarch64():
rtppay.link(queue11)
queue11.link(transform)
transform.link(sink)
else:
rtppay.link(queue11)
queue11.link(sink)

and next with:
streammux.link(pgie)
pgie.link(tracker)
tracker.link(nvanalytics)
nvanalytics.link(tiler)
tiler.link(nvvidconv)
nvvidconv.link(nvosd)
nvosd.link(nvvidconv_postosd)
nvvidconv_postosd.link(caps)
caps.link(encoder)
encoder.link(rtppay)
rtppay.link(sink)

So can you remove more plugin in your pipeline to narrow down the issue?

Found the working pipeline here Deepstream RTSP in and RTSP out with nvanalytics and tracker