After installing the TensorRT container, when I try to run the command,
docker run --gpus all -it --rm nvcr.io/nvidia/tensorrt:23.10-py3
It throws an error as
docker: Error response from daemon: could not select device driver “” with capabilities: [[gpu]]
Hence used docker run -it --rm nvcr.io/nvidia/tensorrt:23.10-py3 to run the docker container.
My system package versions:
Cuda version - 12.2
Driver version - 535.129.03
TensorRT Container version - 23.10
Tried verifying if the cuda drivers are installed inside the docker container using nvcc - V
Terminal Output:
It shows that cuda drivers are installed inside the docker container.
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Tue_Aug_15_22:02:13_PDT_2023
Cuda compilation tools, release 12.2, V12.2.140
Build cuda_12.2.r12.2/compiler.33191640_0
nvidia-smi works outside docker container but not inside docker.
Though it shows, driver and cuda versions, when I try to run the Python or C++ program from the docker container. It always shows me cuda module package not found error.
C++ error:
root@82eb7b4cf72d:/workspace/tensorrt/bin# ./sample_onnx_mnist
&&&& RUNNING TensorRT.sample_onnx_mnist [TensorRT v8601] # ./sample_onnx_mnist
[12/01/2023-12:30:41] [I] Building and running a GPU inference engine for Onnx MNIST
[12/01/2023-12:30:41] [W] [TRT] Unable to determine GPU memory usage
[12/01/2023-12:30:41] [W] [TRT] Unable to determine GPU memory usage
[12/01/2023-12:30:41] [I] [TRT] [MemUsageChange] Init CUDA: CPU +0, GPU +0, now: CPU 5, GPU 0 (MiB)
[12/01/2023-12:30:41] [W] [TRT] CUDA initialization failure with error: 35. Please check your CUDA installation: http://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html
&&&& FAILED TensorRT.sample_onnx_mnist
Python error:
root@82eb7b4cf72d:/workspace/tensorrt/samples/python/introductory_parser_samples# python onnx_resnet50.py
Traceback (most recent call last):
File "/workspace/tensorrt/samples/python/introductory_parser_samples/onnx_resnet50.py", line 30, in <module>
import common
File "/workspace/tensorrt/samples/python/introductory_parser_samples/../common.py", line 25, in <module>
from cuda import cuda, cudart
ModuleNotFoundError: No module named 'cuda'
Let me know if any of you have faced this error and the fix for this error. Thanks in advance.