Please provide complete information as applicable to your setup.
• Hardware Platform (Jetson / GPU)
• DeepStream Version
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)
I am running a deepstream application using docker image nvcr.io/nvidia/deepstream:6.1.1-devel,
with rtsp sources added dynamically and an rtsp out, full pipeline below
The application is running very well without crashing on a Quadro P1000 with the same docker image and driver version 515.65.01,
However when running on Quadro RTX 5000 I keep getting this error and the application doesn’t keep running for 10 minutes and keeps crashing on random intervals
ERROR: nvdsinfer_context_impl.cpp:339 Failed to make stream wait on event, cuda err_no:700, err_str:cudaErrorIllegalAddress ERROR: nvdsinfer_context_impl.cpp:1625 Preprocessor transform input data failed., nvinfer error:NVDSINFER_CUDA_ERROR 0:23:01.478337623 1 0x7f144401b180 WARN nvinfer gstnvinfer.cpp:1338:gst_nvinfer_input_queue_loop:<primary-inference> error: Failed to queue input batch for inferencing GPUassert: an illegal memory access was encountered src/modules/NvMultiObjectTracker/context.cpp 197 Error: gst-stream-error-quark: Failed to queue input batch for inferencing (1): gstnvinfer.cpp(1338): gst_nvinfer_input_queue_loop (): /GstPipeline:pipeline0/GstNvInfer:primary-inference # Gst.MessageType.ERROR, debug is gstnvinfer.cpp(1338): gst_nvinfer_input_queue_loop (): /GstPipeline:pipeline0/GstNvInfer:primary-inference, msg.src.name primary-inference# 2023-05-28 09:13:21 INFO ds_utils 404 NoneType: None 2023-05-28 09:13:21 ERROR ds_utils 405 [WARN ] 2023-05-28 09:13:21 (cudaErrorIllegalAddress) [ERROR] 2023-05-28 09:13:21 Error destroying cuda device: 0�&
Thought it was a model error, tried different models from here.
(different models running well on the quadro p1000 machine)
But I am encountering the same error.