Please provide complete information as applicable to your setup.
**• Hardware Platform (Jetson / GPU): Jetson Nano
**• DeepStream Version: 6.0 and 5.1 seperately on two different Nanos
**• JetPack Version (valid for Jetson only): 4.6 and 4.5.1
Hi
We are using the deepstream occupancy analytics application on two different jetson nanos and we are having trouble running them.
Jetson 1: Deepstream 5.1:
We have been using the app for a while now and then suddenly it stopped showing the camera rtsp stream, sending entry and exit kafka messages, and showing these messages:
(deepstream-test5-analytics:31631): GLib-CRITICAL **: 17:46:28.751: g_strchug: assertion 'string != NULL' failed
(deepstream-test5-analytics:31631): GLib-CRITICAL **: 17:46:28.752: g_strchomp: assertion 'string != NULL' failed
Warning: 'input-dims' parameter has been deprecated. Use 'infer-dims' instead.
gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream-5.1/lib/libnvds_nvdcf.so
gstnvtracker: Batch processing is OFF
gstnvtracker: Past frame output is OFF
ERROR: Deserialize engine failed because file path: /opt/nvidia/deepstream/deepstream-5.1/sources/apps/sample_apps/deepstream-occupancy-analytics-initial/config/peoplenet/resnet34_peoplenet_pruned.etlt_b1_gpu0_int8.engine open error
0:00:02.048396888 31631 0x5593efae40 WARN nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1691> [UID = 1]: deserialize engine from file :/opt/nvidia/deepstream/deepstream-5.1/sources/apps/sample_apps/deepstream-occupancy-analytics-initial/config/peoplenet/resnet34_peoplenet_pruned.etlt_b1_gpu0_int8.engine failed
0:00:02.048496838 31631 0x5593efae40 WARN nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1798> [UID = 1]: deserialize backend context from engine from file :/opt/nvidia/deepstream/deepstream-5.1/sources/apps/sample_apps/deepstream-occupancy-analytics-initial/config/peoplenet/resnet34_peoplenet_pruned.etlt_b1_gpu0_int8.engine failed, try rebuild
0:00:02.048525120 31631 0x5593efae40 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1716> [UID = 1]: Trying to create engine from model files
INFO: [TRT]: Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
INFO: [TRT]: Detected 1 inputs and 2 output network tensors.
0:02:00.356091717 31631 0x5593efae40 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1749> [UID = 1]: serialize cuda engine to file: /opt/nvidia/deepstream/deepstream-5.1/sources/apps/sample_apps/deepstream-occupancy-analytics-initial/config/peoplenet/resnet34_peoplenet_pruned.etlt_b2_gpu0_fp16.engine successfully
INFO: [Implicit Engine Info]: layers num: 3
0 INPUT kFLOAT input_1 3x544x960
1 OUTPUT kFLOAT output_bbox/BiasAdd 12x34x60
2 OUTPUT kFLOAT output_cov/Sigmoid 3x34x60
0:02:00.410322345 31631 0x5593efae40 INFO nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<primary_gie> [UID 1]: Load new model:/opt/nvidia/deepstream/deepstream-5.1/sources/apps/sample_apps/deepstream-occupancy-analytics-initial/config/config_infer_primary_peoplenet.txt sucessfully
Runtime commands:
h: Print this help
q: Quit
p: Pause
r: Resume
**PERF: FPS 0 (Avg) FPS 1 (Avg)
Sun Apr 24 17:48:28 2022
**PERF: 0.00 (0.00) 0.00 (0.00)
WARNING from src_bin_muxer: Rounding muxer output height to the next multiple of 4: 372
Debug info: /dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvmultistream/gstnvstreammux.c(2536): gst_nvstreammux_change_state (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstNvStreamMux:src_bin_muxer
** INFO: <bus_callback:181>: Pipeline ready
ERROR from src_elem0: Could not read from resource.
Debug info: gstrtspsrc.c(5917): gst_rtsp_src_receive_response (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstRTSPSrc:src_elem0:
Could not receive message. (System error)
ERROR from src_elem1: Could not read from resource.
Debug info: gstrtspsrc.c(5917): gst_rtsp_src_receive_response (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin1/GstRTSPSrc:src_elem1:
Could not receive message. (System error)
** INFO: <reset_source_pipeline:1433>: Resetting source 0
** INFO: <reset_source_pipeline:1433>: Resetting source 1
ERROR from src_elem0: Could not read from resource.
Debug info: gstrtspsrc.c(5917): gst_rtsp_src_receive_response (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstRTSPSrc:src_elem0:
Could not receive message. (System error)
Sun Apr 24 17:48:33 2022
**PERF: 0.00 (0.00) 0.00 (0.00)
Sun Apr 24 17:48:38 2022
**PERF: 0.00 (0.00) 0.00 (0.00)
Sun Apr 24 17:48:43 2022
**PERF: 0.00 (0.00) 0.00 (0.00)
Sun Apr 24 17:48:48 2022
**PERF: 0.00 (0.00) 0.00 (0.00)
ERROR from src_elem1: Could not open resource for reading and writing.
Debug info: gstrtspsrc.c(7469): gst_rtspsrc_retrieve_sdp (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin1/GstRTSPSrc:src_elem1:
Failed to connect. (Generic error)
Sun Apr 24 17:48:53 2022
**PERF: 0.00 (0.00) 0.00 (0.00)
Sun Apr 24 17:48:58 2022
**PERF: 0.00 (0.00) 0.00 (0.00)
Sun Apr 24 17:49:03 2022
**PERF: 0.00 (0.00) 0.00 (0.00)
Sun Apr 24 17:49:08 2022
**PERF: 0.00 (0.00) 0.00 (0.00)
Sun Apr 24 17:49:13 2022
**PERF: 0.00 (0.00) 0.00 (0.00)
Sun Apr 24 17:49:18 2022
**PERF: 0.00 (0.00) 0.00 (0.00)
Sun Apr 24 17:49:23 2022
**PERF: 0.00 (0.00) 0.00 (0.00)
Sun Apr 24 17:49:28 2022
**PERF: 0.00 (0.00) 0.00 (0.00)
** INFO: <reset_source_pipeline:1433>: Resetting source 0
** INFO: <reset_source_pipeline:1433>: Resetting source 1
Sun Apr 24 17:49:33 2022
**PERF: 0.00 (0.00) 0.00 (0.00)
Sun Apr 24 17:49:38 2022
**PERF: 0.00 (0.00) 0.00 (0.00)
Sun Apr 24 17:49:43 2022
**PERF: 0.00 (0.00) 0.00 (0.00)
Sun Apr 24 17:49:48 2022
**PERF: 0.00 (0.00) 0.00 (0.00)
ERROR from src_elem0: Could not open resource for reading and writing.
Debug info: gstrtspsrc.c(7469): gst_rtspsrc_retrieve_sdp (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstRTSPSrc:src_elem0:
Failed to connect. (Generic error)
ERROR from src_elem1: Could not open resource for reading and writing.
Debug info: gstrtspsrc.c(7469): gst_rtspsrc_retrieve_sdp (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin1/GstRTSPSrc:src_elem1:
Failed to connect. (Generic error)
Sun Apr 24 17:49:53 2022
**PERF: 0.00 (0.00) 0.00 (0.00)
Sun Apr 24 17:49:58 2022
**PERF: 0.00 (0.00) 0.00 (0.00)
Sun Apr 24 17:50:03 2022
**PERF: 0.00 (0.00) 0.00 (0.00)
**PERF: FPS 0 (Avg) FPS 1 (Avg)
Sun Apr 24 17:50:08 2022
**PERF: 0.00 (0.00) 0.00 (0.00)
Sun Apr 24 17:50:13 2022
**PERF: 0.00 (0.00) 0.00 (0.00)
Jetson 2: Deepstream 6.0:
We updated the deepstream version to see if the problem persists but what happens is the app doesn’t quite function correctly at all and we get this message:
(gst-plugin-scanner:29065): GStreamer-WARNING **: 17:50:54.813: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_inferserver.so': libtritonserver.so: cannot open shared object file: No such file or directory
(gst-plugin-scanner:29065): GStreamer-WARNING **: 17:50:54.844: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_udp.so': librivermax.so.0: cannot open shared object file: No such file or directory
(gst-plugin-scanner:29065): GStreamer-WARNING **: 17:50:55.516: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstchromaprint.so': /usr/lib/aarch64-linux-gnu/libgomp.so.1: cannot allocate memory in static TLS block
(gst-plugin-scanner:29065): GStreamer-WARNING **: 17:50:55.925: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstlibav.so': /usr/lib/aarch64-linux-gnu/libgomp.so.1: cannot allocate memory in static TLS block
(deepstream-test5-analytics:29064): GLib-CRITICAL **: 17:50:57.368: g_strchug: assertion 'string != NULL' failed
(deepstream-test5-analytics:29064): GLib-CRITICAL **: 17:50:57.368: g_strchomp: assertion 'string != NULL' failed
Warning: 'input-dims' parameter has been deprecated. Use 'infer-dims' instead.
gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream-6.0/lib/libnvds_nvdcf.so
gstnvtracker: Batch processing is OFF
gstnvtracker: Past frame output is OFF
ERROR: Deserialize engine failed because file path: /opt/nvidia/deepstream/deepstream-6.0/sources/apps/sample_apps/deepstream-occupancy-analytics-AG/config/peoplenet/resnet34_peoplenet_pruned.etlt_b1_gpu0_int8.engine open error
0:00:06.111992662 29064 0x559721d440 WARN nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1889> [UID = 1]: deserialize engine from file :/opt/nvidia/deepstream/deepstream-6.0/sources/apps/sample_apps/deepstream-occupancy-analytics-AG/config/peoplenet/resnet34_peoplenet_pruned.etlt_b1_gpu0_int8.engine failed
0:00:06.112159748 29064 0x559721d440 WARN nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1996> [UID = 1]: deserialize backend context from engine from file :/opt/nvidia/deepstream/deepstream-6.0/sources/apps/sample_apps/deepstream-occupancy-analytics-AG/config/peoplenet/resnet34_peoplenet_pruned.etlt_b1_gpu0_int8.engine failed, try rebuild
0:00:06.112189488 29064 0x559721d440 INFO nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1914> [UID = 1]: Trying to create engine from model files
WARNING: INT8 not supported by platform. Trying FP16 mode.
WARNING: INT8 not supported by platform. Trying FP16 mode.
WARNING: [TRT]: Detected invalid timing cache, setup a local cache instead
0:02:33.887904047 29064 0x559721d440 INFO nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1947> [UID = 1]: serialize cuda engine to file: /opt/nvidia/deepstream/deepstream-6.0/sources/apps/sample_apps/deepstream-occupancy-analytics-AG/config/peoplenet/resnet34_peoplenet_pruned.etlt_b1_gpu0_fp16.engine successfully
INFO: [Implicit Engine Info]: layers num: 3
0 INPUT kFLOAT input_1 3x544x960
1 OUTPUT kFLOAT output_bbox/BiasAdd 12x34x60
2 OUTPUT kFLOAT output_cov/Sigmoid 3x34x60
0:02:33.944271180 29064 0x559721d440 INFO nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<primary_gie> [UID 1]: Load new model:/opt/nvidia/deepstream/deepstream-6.0/sources/apps/sample_apps/deepstream-occupancy-analytics-AG/config/config_infer_primary_peoplenet.txt sucessfully
cb_sourcesetup set 100 latency
Runtime commands:
h: Print this help
q: Quit
p: Pause
r: Resume
**PERF: FPS 0 (Avg)
Sun Apr 24 17:53:26 2022
**PERF: 0.00 (0.00)
** INFO: <bus_callback:194>: Pipeline ready
** INFO: <bus_callback:180>: Pipeline running
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 279
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 279
** INFO: <bus_callback:180>: Pipeline running
~~ CLOG[/dvs/git/dirty/git-master_linux/deepstream/sdk/src/utils/nvdcf/src/modules/NvDCF/NvDCF.cpp, NvDCF() @line 670]: !!![WARNING] Can't open config file (/opt/nvidia/deepstream/deepstream-6.0/sources/apps/sample_apps/deepstream-occupancy-analytics-AG/config/tracker_config.yml). Will go ahead with default values
~~ CLOG[/dvs/git/dirty/git-master_linux/deepstream/sdk/src/utils/nvdcf/src/modules/NvDCF/NvDCF.cpp, NvDCF() @line 682]: !!![WARNING] Invalid low-level config file is provided. Will go ahead with default values
[NvDCF] Initialized
Segmentation fault
What we tried:
We tried emptying the cache, changing the deepstream version, also switching between source types ( rtsp and uri), and we checked on the camera stream (it was fine).
Could you help us figure out why the app is not functioning, please?
Thanks