I have Jetpack 5.1 installed on AGX Orin: R35 (release), REVISION: 2.1. PyTorch 2.0 is installed with all GPU dependencies and runs fine. I have to benchmark TensorFlow, which I installed following the procedures from Jetson Zoo - eLinux.org. and got tensorflow 2.12.0. It seems to be missing GPU dependencies, e.g. tensorrt and nvidia-tensorrt. How do I install them?
Both pip install tensorrt and pip install nvidia-tensorrt fail:
(.venv) nick@ubuntu:~/yolo8$ pip install tensorrt
ERROR: Could not find a version that satisfies the requirement tensorrt (from versions: 0.0.1.dev5, 0.0.1)
ERROR: No matching distribution found for tensorrt
(.venv) nick@ubuntu:~/yolo8$ pip install nvidia-tensorrt
Collecting nvidia-tensorrt
Downloading nvidia-tensorrt-0.0.1.dev5.tar.gz (7.9 kB)
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [17 lines of output]
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "/tmp/pip-install-xekjcxx5/nvidia-tensorrt_af9512cb35454287be0799094414179a/setup.py", line 150, in <module>
raise RuntimeError(open("ERROR.txt", "r").read())
RuntimeError:
###########################################################################################
The package you are trying to install is only a placeholder project on PyPI.org repository.
This package is hosted on NVIDIA Python Package Index.
This package can be installed as:
```
$ pip install nvidia-pyindex
$ pip install nvidia-tensorrt
```
###########################################################################################
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
That’s not a problem. There is a compatibility problem between wheels. Here is the log:
(.venv) nick@ubuntu:~/yolo8$ pip install tensorrt -v -v
Using pip 23.1 from /home/nick/yolo8/.venv/lib/python3.8/site-packages/pip (python 3.8)
Non-user install because user site-packages disabled
Created temporary directory: /tmp/pip-build-tracker-8amp0844
Initialized build tracking at /tmp/pip-build-tracker-8amp0844
Created build tracker: /tmp/pip-build-tracker-8amp0844
Entered build tracker: /tmp/pip-build-tracker-8amp0844
Created temporary directory: /tmp/pip-install-r7p4pfj1
Created temporary directory: /tmp/pip-ephem-wheel-cache-ewtfmf8c
1 location(s) to search for versions of tensorrt:
* https://pypi.org/simple/tensorrt/
Fetching project page and analyzing links: https://pypi.org/simple/tensorrt/
Getting page https://pypi.org/simple/tensorrt/
Found index url https://pypi.org/simple/
Looking up "https://pypi.org/simple/tensorrt/" in the cache
Request header has "max_age" as 0, cache bypassed
Starting new HTTPS connection (1): pypi.org:443
https://pypi.org:443 "GET /simple/tensorrt/ HTTP/1.1" 304 0
Fetched page https://pypi.org/simple/tensorrt/ as application/vnd.pypi.simple.v1+json
Found link https://files.pythonhosted.org/packages/47/85/3565669797b70153162a7a704a6b52d5f9be50f4349a62992f5523736f97/tensorrt-0.0.1.dev5.tar.gz (from https://pypi.org/simple/tensorrt/), version: 0.0.1.dev5
Found link https://files.pythonhosted.org/packages/b0/9b/0588892fb68abccfaf26e56fc3c45a3cab3638ad8b488302fdbfce655455/tensorrt-0.0.1.tar.gz (from https://pypi.org/simple/tensorrt/), version: 0.0.1
Skipping link: none of the wheel's tags (cp310-none-manylinux_2_17_x86_64) are compatible (run pip debug --verbose to show compatible tags): https://files.pythonhosted.org/packages/67/a3/ce6be702145340eee2f3ca57a7acd65d7c4aa1652afbe54d741354d23c50/tensorrt-8.5.1.7-cp310-none-manylinux_2_17_x86_64.whl (from https://pypi.org/simple/tensorrt/)
Skipping link: none of the wheel's tags (cp36-none-manylinux_2_17_x86_64) are compatible (run pip debug --verbose to show compatible tags): https://files.pythonhosted.org/packages/c5/40/a30bfbe5b041247acf0bda5c36ada03c2fea699e397be5eb62848433a4bf/tensorrt-8.5.1.7-cp36-none-manylinux_2_17_x86_64.whl (from https://pypi.org/simple/tensorrt/)
Skipping link: none of the wheel's tags (cp37-none-manylinux_2_17_x86_64) are compatible (run pip debug --verbose to show compatible tags): https://files.pythonhosted.org/packages/95/37/71c8e660bb89fd32877cf85722a79245a98bdd1c40f76ae8c588b08d2115/tensorrt-8.5.1.7-cp37-none-manylinux_2_17_x86_64.whl (from https://pypi.org/simple/tensorrt/)
Skipping link: none of the wheel's tags (cp38-none-manylinux_2_17_x86_64) are compatible (run pip debug --verbose to show compatible tags): https://files.pythonhosted.org/packages/04/e1/2249c72a69c355140039767ad4f0664df3d27f4b509c2506e6f845f69f6f/tensorrt-8.5.1.7-cp38-none-manylinux_2_17_x86_64.whl (from https://pypi.org/simple/tensorrt/)
Skipping link: none of the wheel's tags (cp39-none-manylinux_2_17_x86_64) are compatible (run pip debug --verbose to show compatible tags): https://files.pythonhosted.org/packages/8f/9c/0ea2cf5220d3b34e215d88ca7c22c5743afddde8b0d9283ddadc0ad07893/tensorrt-8.5.1.7-cp39-none-manylinux_2_17_x86_64.whl (from https://pypi.org/simple/tensorrt/)
Skipping link: none of the wheel's tags (cp310-none-manylinux_2_17_x86_64) are compatible (run pip debug --verbose to show compatible tags): https://files.pythonhosted.org/packages/78/02/dbc1fb8b703f0eb3b20ee5e1007cf87b5f1bd165df1b059e8e578c142da4/tensorrt-8.5.2.2-cp310-none-manylinux_2_17_x86_64.whl (from https://pypi.org/simple/tensorrt/)
Skipping link: none of the wheel's tags (cp36-none-manylinux_2_17_x86_64) are compatible (run pip debug --verbose to show compatible tags): https://files.pythonhosted.org/packages/e6/f0/55d255046231874c2f1b02395a2d7197bb7b90a05e8f78d33edead1edb04/tensorrt-8.5.2.2-cp36-none-manylinux_2_17_x86_64.whl (from https://pypi.org/simple/tensorrt/)
Skipping link: none of the wheel's tags (cp37-none-manylinux_2_17_x86_64) are compatible (run pip debug --verbose to show compatible tags): https://files.pythonhosted.org/packages/0c/44/9cfe99bb2efbc8303cb51ec5b81d98580ef377e280232abf90c6e28357d3/tensorrt-8.5.2.2-cp37-none-manylinux_2_17_x86_64.whl (from https://pypi.org/simple/tensorrt/)
Skipping link: none of the wheel's tags (cp38-none-manylinux_2_17_x86_64) are compatible (run pip debug --verbose to show compatible tags): https://files.pythonhosted.org/packages/c6/9b/f6880f2518a1a0dc0592149e4d05e6c3bcfe7f4e071201e628bc49ad16a6/tensorrt-8.5.2.2-cp38-none-manylinux_2_17_x86_64.whl (from https://pypi.org/simple/tensorrt/)
Skipping link: none of the wheel's tags (cp39-none-manylinux_2_17_x86_64) are compatible (run pip debug --verbose to show compatible tags): https://files.pythonhosted.org/packages/16/7f/b64241ace9be285f24d3d508f9453d2fdb885b73caf647a9ac28b5fcbc58/tensorrt-8.5.2.2-cp39-none-manylinux_2_17_x86_64.whl (from https://pypi.org/simple/tensorrt/)
Skipping link: none of the wheel's tags (cp310-none-manylinux_2_17_x86_64) are compatible (run pip debug --verbose to show compatible tags): https://files.pythonhosted.org/packages/3e/d5/5f9dd454a89f5bf09c3740c649ba6c8dd685cae98a1255299a2e1dbac606/tensorrt-8.5.3.1-cp310-none-manylinux_2_17_x86_64.whl (from https://pypi.org/simple/tensorrt/)
Skipping link: none of the wheel's tags (cp36-none-manylinux_2_17_x86_64) are compatible (run pip debug --verbose to show compatible tags): https://files.pythonhosted.org/packages/ae/64/952a0ba49b1dbccd7d47f999bbbebb0d7c11b7f45acfca065e1035940d39/tensorrt-8.5.3.1-cp36-none-manylinux_2_17_x86_64.whl (from https://pypi.org/simple/tensorrt/)
Skipping link: none of the wheel's tags (cp37-none-manylinux_2_17_x86_64) are compatible (run pip debug --verbose to show compatible tags): https://files.pythonhosted.org/packages/83/0c/a4ce5f1dce70ca1f62572c34daccf4940a206940b6dc6f0b670b61cd685c/tensorrt-8.5.3.1-cp37-none-manylinux_2_17_x86_64.whl (from https://pypi.org/simple/tensorrt/)
Skipping link: none of the wheel's tags (cp38-none-manylinux_2_17_x86_64) are compatible (run pip debug --verbose to show compatible tags): https://files.pythonhosted.org/packages/e2/c4/4b099cdabec1528682a90093359699717b06a61391a5f5f458987654baf9/tensorrt-8.5.3.1-cp38-none-manylinux_2_17_x86_64.whl (from https://pypi.org/simple/tensorrt/)
Skipping link: none of the wheel's tags (cp39-none-manylinux_2_17_x86_64) are compatible (run pip debug --verbose to show compatible tags): https://files.pythonhosted.org/packages/e7/9d/7e45985e7281590990caf7776ac5799a22214b1a8041a83b2a83cad8cefe/tensorrt-8.5.3.1-cp39-none-manylinux_2_17_x86_64.whl (from https://pypi.org/simple/tensorrt/)
Skipping link: none of the wheel's tags (cp310-none-manylinux_2_17_x86_64) are compatible (run pip debug --verbose to show compatible tags): https://files.pythonhosted.org/packages/bb/2c/6a2a10cf2133362b1c61087b7ed35c8a52b2eb154a3d5b414920430b9467/tensorrt-8.6.0-cp310-none-manylinux_2_17_x86_64.whl (from https://pypi.org/simple/tensorrt/)
Skipping link: none of the wheel's tags (cp36-none-manylinux_2_17_x86_64) are compatible (run pip debug --verbose to show compatible tags): https://files.pythonhosted.org/packages/49/46/7fe5ef71b582cad770a3bb2af87bc9aee7095caeb0dee639a1ba3600f69c/tensorrt-8.6.0-cp36-none-manylinux_2_17_x86_64.whl (from https://pypi.org/simple/tensorrt/)
Skipping link: none of the wheel's tags (cp37-none-manylinux_2_17_x86_64) are compatible (run pip debug --verbose to show compatible tags): https://files.pythonhosted.org/packages/5e/51/91d02fb45c0055293f53ac8202229eee37c76b4b61fd370f41d60def36c1/tensorrt-8.6.0-cp37-none-manylinux_2_17_x86_64.whl (from https://pypi.org/simple/tensorrt/)
Skipping link: none of the wheel's tags (cp38-none-manylinux_2_17_x86_64) are compatible (run pip debug --verbose to show compatible tags): https://files.pythonhosted.org/packages/f5/c7/260676e8490eb768193f1c30c9e9c43fb916d316af057ff5547539634d52/tensorrt-8.6.0-cp38-none-manylinux_2_17_x86_64.whl (from https://pypi.org/simple/tensorrt/)
Skipping link: none of the wheel's tags (cp39-none-manylinux_2_17_x86_64) are compatible (run pip debug --verbose to show compatible tags): https://files.pythonhosted.org/packages/11/2c/71f54e49c4e25b15aebf11f942e0f9741cb9280ee0bfb5e7eb166f5852a6/tensorrt-8.6.0-cp39-none-manylinux_2_17_x86_64.whl (from https://pypi.org/simple/tensorrt/)
Skipping link: not a file: https://pypi.org/simple/tensorrt/
Given no hashes to check 1 links for project 'tensorrt': discarding no candidates
ERROR: Could not find a version that satisfies the requirement tensorrt (from versions: 0.0.1.dev5, 0.0.1)
ERROR: No matching distribution found for tensorrt
I’m getting the same errors when executing pip install tensorrt in a fresh virtual environment. Somehow none of existing tensorrt wheels is compatible with my current system state.
@pauljurczak on Jetson/aarch64 the TensorRT Python bindings shouldn’t be installed from pip, rather from the apt package python3-libnvinfer-dev that comes from the JetPack repo. Since you are able to import tensorrt, it seems like it was already installed from apt.
I thought that wasn’t the case, but I was wrong. I’m using VSCode and it creates virtual environments without system site packages, by default. When I created one manually with --system-site-packages option, tensorrt module became visible.