Need Support with installing tensorRT in a Conda env on Orin NX

[surajsingh52]

Hey,
creating this new post since i couldn’t tag the Orin NX team.
Link to the previous post: How to install tensorRT in Conda env for Orin NX

trying to install tensorrt in a conda env on Orin NX. Facing issues with adding the NVIDIA repo key and in the installation.I am looking to install just the python library.

Environment

Platform: Orin NX
Jetpack: 5.1 which includes CUDA 11.4, TensorRT 8.5.2, cuDNN 8.6.0 and VPI 2.2
Python: 3.8
Running this in a conda env.

Steps To Reproduce

Tried adding the NVIDIA Repo key before running the installation commands:
“wget https://developer.download.nvidia.com/embedded/L4T/r32_Release_v6.1/jetson-ota-public.asc
–2023-10-06 16:38:55-- https://developer.download.nvidia.com/embedded/L4T/r32_Release_v6.1/jetson-ota-public.asc
Resolving developer.download.nvidia.com (developer.download.nvidia.com)… 152.199.39.144
Connecting to developer.download.nvidia.com (developer.download.nvidia.com)|152.199.39.144|:443… connected.
HTTP request sent, awaiting response… 404 Not Found
2023-10-06 16:38:56 ERROR 404: Not Found.”

The commands i tried(won’t work till the key related issue is sorted):
“pip3 install nvidia-tensort” and “pip3 install tensort-libs”

Got similar outputs for both:
"Looking in indexes: Simple index, https://pypi.ngc.nvidia.com
ERROR: Could not find a version that satisfies the requirement nvidia-tensort (from versions: none)
ERROR: No matching distribution found for nvidia-tensort
"
"Looking in indexes: Simple index, https://pypi.ngc.nvidia.com
ERROR: Could not find a version that satisfies the requirement tensort-libs (from versions: none)
ERROR: No matching distribution found for tensort-libs
"

[AastaLLL]

Hi,

It looks like you are accessing r32 but Orin NX uses r35.
More, please install with apt instead of pip3.

$ sudo apt update
$ sudo apt install python3-libnvinfer*

Here are some instructions for adding the source list if above command fails:
https://docs.nvidia.com/jetson/archives/r35.4.1/DeveloperGuide/text/SD/SoftwarePackagesAndTheUpdateMechanism.html#

Thanks.

[surajsingh52]

Hi @AastaLLL
i tried adding the key for the r35 version of jetpack.
My command:
“wget https://developer.download.nvidia.com/embedded/L4T/r35_Release_v3.1/jetson-ota-public.asc”

Gives the same error.
→ "–2023-10-11 16:25:38-- https://developer.download.nvidia.com/embedded/L4T/r35_Release_v3.1/jetson-ota-public.asc
Resolving developer.download.nvidia.com (developer.download.nvidia.com)… 152.199.39.144
Connecting to developer.download.nvidia.com (developer.download.nvidia.com)|152.199.39.144|:443… connected.
HTTP request sent, awaiting response… 404 Not Found
2023-10-11 16:25:39 ERROR 404: Not Found.
"
Is there somewhere i am going wrong? or any other way to install the library for tensorrt?

[AastaLLL]

Hi,

Please follow the doc to set up the source list.
For r35.3.1 (JetPack 5.1), it should look like as below:

Edit ‘/etc/apt/sources.list.d/nvidia-l4t-apt-source.list’

deb https://repo.download.nvidia.com/jetson/common r35.3 main
deb https://repo.download.nvidia.com/jetson/t234 r35.3 main

Then run

$ sudo apt update
$ sudo apt install python3-libnvinfer*

Thanks.

[surajsingh52]

Hi,
the file at “/etc/apt/sources.list.d/nvidia-l4t-apt-source.list” already contains the lines you just mentioned.

Also, i am trying to install tensorrt within conda env, not system-wide. The command:

$ sudo apt install python3-libnvinfer*

installs the library system-wide.

Just to let you know, i already have tensorrt installed system-wide while flashing the board. But i need to install it in the container, for specific tensorrt versions. Will try to link/copy the globally installed TRT in the conda env as well to see if that works.

Let me know if there’s anything else i can try out for installing in the env.

[AastaLLL]

Hi,

If you want to use it inside the container, please install it again.
If you want to leverage a system-wide library, then please add the corresponding flag when activating the environment.

Which version do you want to test?

On Jetson, the TensorRT version is limited to the JetPack version.
Installing different versions might lead to unexpected errors.

Thanks.

[surajsingh52]

Hi @AastaLLL,
installing on jetson Orin NX in the container is exactly the problem. To install within the container, for Orin NX the Repo key needs to be added before running the installation commands, since tensorrt cannot be installed through the commands right away for this board. I am facing issues adding the repo key. I think this is a general problem related to installing tensorrt on Orin NX through the commands(irrespective of conda).

[AastaLLL]

Hi,

Could you try if the below command helps?

apt-key adv --fetch-key http://repo.download.nvidia.com/jetson/jetson-ota-public.asc

Thanks.

[surajsingh52]

Hi @AastaLLL,
tried the above command. Still couldn’t add the key.

The error:
"
Executing: /tmp/apt-key-gpghome.HYt72ybqry/gpg.1.sh --fetch-key http://repo.download.nvidia.com/jetson/jetson-ota-public.asc
gpg: requesting key from ‘http://repo.download.nvidia.com/jetson/jetson-ota-public.asc’
gpg: WARNING: unable to fetch URI http://repo.download.nvidia.com/jetson/jetson-ota-public.asc: Server indicated a failure
".

[AastaLLL]

Hi,

We can reproduce this scenario with virtualenv.

A WAR we found is to copy the Python library into the virtual environment workspace manually.

$ cp -r /usr/lib/python3.8/dist-packages/tensorrt* [virenv]/lib/python3.8/site-packages/
$ source [virenv]/bin/activate

Does this work for you?
If not, maybe you can try to build TensorRT python binding in the virtual environment to see if it can work as expected.

Thanks.