• Hardware Platform (Jetson / GPU) T4 GPU
• DeepStream Version Deepstream-5.0
• NVIDIA GPU Driver Version (valid for GPU only) 460.32.03
We are using the Milestone VPS plugin with Deepstream Pipeline that includes different Detection/Classification Models.
we are using Milestone 2020 R2 and VPS From MIPSDK 2020 R1. Milestone is running on a Windows OS while the VPS/Deepstream Pipeline is running on Ubuntu 18.04.
The pipeline (Deepstream) is crashing during runtime due to the error :
When we run the pipeline, all the models are starting to instantiate and start running. Yet at some random point, it crashes and starts giving this error (attached).
Also note that running the command:
watch nvidia-smi shows complete info in regards of the GPU, but when the error happens, all the processes running and its metrics (Mem allocated, GPU util, info) disappear (turn into 0) and the GPU info remains showing without any running process.
Can you please help with this?
Please find attached the nvidia-bug-report after running the command:
We also collected some data lof using nvidia-smi (sampling each 5 sec) which shows that when the error occurs the Util Mem turns to 0.
nvidia-bug-report.log.gz (1.2 MB)