Hi I want to use TensorRT in a docker container for my python3 app on my Jetson Nano device.
My setup is below;
NVIDIA Jetson Nano (Developer Kit Version)
L4T 32.3.1 [ JetPack 4.3 ]
Ubuntu 18.04.3 LTS
Kernel Version: 4.9.140-tegra
CUDA 10.0.326
CUDA Architecture: 5.3
OpenCV version: 4.1.1
OpenCV Cuda: NO
CUDNN: 7.6.3.28
TensorRT: 6.0.1.10
Vision Works: 1.6.0.500n
VPI: 0.1.0
Vulcan: 1.1.70
I use nvcr.io/nvidia/deepstream-l4t:5.0-20.07-samples image as base for my dockerfile(I did not use the latest deepstream image because of my LT4 and TensorRT versions on jetson are old).
I have already set default runtime to nvidia. But when I run āimport tensorrt as trtā I get error ; āimport tensorrt as trt ModuleNotFoundError: No module named ātensorrtāā. How can I make possible my python app to see TensorRT already installed on Jetson nano host?
Hi @hasever, you may need to upgrade/reflash your SD card with newer version of JetPack to get the TensorRT Python libraries in the containers. I believe more recent versions of JetPack automatically have the TensorRT Python libraries added to the containers.
Thanks,
Is Jetpack 4.3 not enough for that ???
So TensorRT and some libraries are added to container from host Jetson-Nano, Am I right? How this mechanism works, can I see tensorrt after that upgrade in my container(for example at āpip3 listā )? Will I have to do further things like setting default runtime to nvidia??
I donāt recall that the CSV files under /etc/nvidia-container-runtime/host-files-for-container.d/ that are responsible mounting the host files into the container included the TensorRT Python libraries on the older versions of JetPack.
You could try adding these lines to /etc/nvidia-container-runtime/host-files-for-container.d/tensorrt.csv if they arenāt already there:
Suffice it to say, if it fails to start then or isnāt work, recommend upgrading JetPack.
You should only need --runtime nvidia when you do docker run. Setting the default runtime to nvidia is when you need CUDA/ect when you are building Dockerfiles with docker build.
Thanks again for your fast reply;
I want to run containers through k3s setup. I have set runtime of k3s to docker , after that I have set docker default runtime of jetson-nano to nvidia. So I assume that , by this setup I must have tensorRT, cuda/etc in my k3s pods/containers? Is it true?
I will try adding the lines that you have specified too.
If the Jetson(s) you are deploying have JetPack and CUDA/ect in the OS, then CUDA/ect will be mounted into all containers when --runtime nvidia is used (or in your case, the default runtime is nvidia)
In the DeepStream container, check to see if you can see /usr/src/tensorrt (this is also mounted from the host)
I think the TensorRT Python libraries were only added to the CSV mounting files later on.