Deepstream-test-3 failure in metropolis

deepstream-test3 failure

Executing ./deepstream-test3-app dstest3_config.yml (default values) results in failure.

Environment

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 520.61.05    Driver Version: 520.61.05    CUDA Version: 11.8     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA A10          On   | 00000000:3B:00.0 Off |                    0 |
|  0%   33C    P8    20W / 150W |      4MiB / 23028MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  NVIDIA A2           On   | 00000000:5E:00.0 Off |                    0 |
|  0%   33C    P8     7W /  60W |      4MiB / 15356MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   2  NVIDIA A30          On   | 00000000:AF:00.0 Off |                    0 |
| N/A   28C    P0    25W / 165W |      4MiB / 24576MiB |      0%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+
|   3  Tesla T4            On   | 00000000:D8:00.0 Off |                    0 |
| N/A   35C    P8    15W /  70W |      6MiB / 15360MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
+-----------------------------------------------------------------------------+

Relevant Files

Output:

root@9bcc249ef6c5:/opt/nvidia/deepstream/deepstream-6.1/sources/apps/sample_apps/deepstream-test3# ./deepstream-test3-app dstest3_config.yml
Now playing : file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4
Using file: dstest3_config.yml
0:00:02.236230880  8845 0x563bc55894d0 INFO                 nvinfer gstnvinfer.cpp:646:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1909> [UID = 1]: deserialized trt engine from :/opt/nvidia/deepstream/deepstream-6.1/samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_int8.engine
INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:610 [Implicit Engine Info]: layers num: 3
0   INPUT  kFLOAT input_1         3x368x640
1   OUTPUT kFLOAT conv2d_bbox     16x23x40
2   OUTPUT kFLOAT conv2d_cov/Sigmoid 4x23x40

0:00:02.336996975  8845 0x563bc55894d0 INFO                 nvinfer gstnvinfer.cpp:646:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2012> [UID = 1]: Use deserialized engine model: /opt/nvidia/deepstream/deepstream-6.1/samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_int8.engine
0:00:02.339678358  8845 0x563bc55894d0 INFO                 nvinfer gstnvinfer_impl.cpp:328:notifyLoadModelStatus:<primary-nvinference-engine> [UID 1]: Load new model:dstest3_pgie_config.yml sucessfully
Decodebin child added: source

(deepstream-test3-app:8845): GLib-GObject-WARNING **: 08:38:05.141: g_object_set_is_valid_property: object class 'GstFileSrc' has no property named 'drop-on-latency'
Decodebin child added: decodebin0
Running...
Decodebin child added: qtdemux0
Decodebin child added: multiqueue0
Decodebin child added: h264parse0
Decodebin child added: capsfilter0
Decodebin child added: aacparse0
Decodebin child added: avdec_aac0
Decodebin child added: nvv4l2decoder0
Frame Number = 0 Number of objects = 14 Vehicle Count = 8 Person Count = 6
Frame Number = 1 Number of objects = 12 Vehicle Count = 7 Person Count = 5
Frame Number = 2 Number of objects = 13 Vehicle Count = 7 Person Count = 6
Frame Number = 3 Number of objects = 12 Vehicle Count = 7 Person Count = 5
Frame Number = 4 Number of objects = 14 Vehicle Count = 7 Person Count = 7
Frame Number = 5 Number of objects = 14 Vehicle Count = 8 Person Count = 6
Frame Number = 6 Number of objects = 10 Vehicle Count = 6 Person Count = 4
Frame Number = 7 Number of objects = 13 Vehicle Count = 9 Person Count = 4
0:00:02.549556973  8845 0x563bc442d760 WARN                 nvinfer gstnvinfer.cpp:2300:gst_nvinfer_output_loop:<primary-nvinference-engine> error: Internal data stream error.
0:00:02.549583580  8845 0x563bc442d760 WARN                 nvinfer gstnvinfer.cpp:2300:gst_nvinfer_output_loop:<primary-nvinference-engine> error: streaming stopped, reason not-negotiated (-4)
ERROR from element primary-nvinference-engine: Internal data stream error.
Error details: gstnvinfer.cpp(2300): gst_nvinfer_output_loop (): /GstPipeline:dstest3-pipeline/GstNvInfer:primary-nvinference-engine:
streaming stopped, reason not-negotiated (-4)
Returned, stopping playback
Frame Number = 8 Number of objects = 11 Vehicle Count = 6 Person Count = 5
Frame Number = 9 Number of objects = 10 Vehicle Count = 6 Person Count = 4
Frame Number = 10 Number of objects = 11 Vehicle Count = 6 Person Count = 5
Deleting pipeline
root@9bcc249ef6c5:/opt/nvidia/deepstream/deepstream-6.1/sources/apps/sample_apps/deepstream-test3#

Steps To Reproduce

  1. Compiled
cd /opt/nvidia/deepstream/deepstream-6.1/sources/apps/sample_apps/deepstream-test3

CUDA_VER=11.7 make

  1. Executed
./deepstream-test3-app dstest3_config.yml

Please include:

  • Exact steps/commands to build your repro
  • Exact steps/commands to run your repro
  • Full traceback of errors encountered

Hi,

This looks like a Deepstream related issue. We will move this post to the Deepstream forum.

Thanks!

Can you re-run the program with "GST_DEBUG=3 " before deepstream_test3?
Please also provide the output of “xrandr --verbose” outside of docker container.

Will run this again in a few days and let you know. Thanks,

I am having the same or similar problem. Did you ever run the test again?