cudaErrorSystemDriverMismatch error when trying to use nvinfer inside Deepstream6.4 docker container

[gabriel.monteagudo]

Please provide complete information as applicable to your setup.

• Hardware Platform: GPU (Ubuntu 22.04)
• DeepStream Version: 6.4
• JetPack Version: None
• TensorRT Version: 8.6
• NVIDIA GPU Driver Version: 535.104.12
• Issue Type: bug/malfunction
• How to reproduce the issue ?
In order to reproduce the issue, install deepstream 6.4 in the host system (Ubuntu 22.04 in my case) following the steps indicated in the QuickStart Guide, including installing the suggested driver. Then, install docker and nvidia-container-toolkit and set it up following the install guide. Finally, download the docker image deepstream. After you get it running, try running a custom gstreamer pipeline that uses the nvinfer plugin, and the error pops off.

I have tried reinstalling the drivers on the host, and the nvidia-smi ouput looks identical inside the docker and in the host; both display the driver version mentioned above (535.104.12) and the same CUDA version (12.2). On the image, left is docker and right is my host PC.

Could you point out something I could have missed/done wrong when setting up the system?

NOTE: The docker image i am using uses the one linked above as a base, and then adds a bunch of my code on top, never touching deepstream install files or libraries, same with the drivers, same with CUDA.

[gabriel.monteagudo]

Problem has been “solved”. Using nvcr.io/nvidia/deepstream:6.4-gc-triton-devel as my base image instead of nvcr.io/nvidia/deepstream:6.4-samples-multiarch solved the problem. Thread can be closed as solved, but it would be nice to know why the other image throws that error.

[yuweiw]

The deepstream:6.4-samples-multiarch is just ideal to understand and explore the DeepStream SDK using the provided samples. If you want to develop your own features, you can use other dockers.

[yingliu]

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks