Unable to use TensorRT inside the L4T-Tensorflow container

Autonomous Machines Jetson & Embedded Systems Jetson Xavier NX
1

Hello,

I am trying to run TensorRT samples in a l4t-tensorflow container on my Jetson Xavier NX Devkit, but I keep having errors as :

AttributeError: module ‘tensorrt’ has no attribute ‘NetworkDefinitionCreationFlag’

Even just with

print(tensorrt.version)

I am using nvcr.io/nvidia/l4t-tensorflow:r32.5.0-tf2.3-py3 as base image.

I tried to install manually tensorrt for python with

python3 -m pip install nvidia-pyindex
python3 -m pip install --upgrade nvidia-tensorrt

But pip fails to install nvidia-pyindex

2

Hello @R_GSD

can you use l4t-ml image:

docker pull nvcr.io/nvidia/l4t-ml:r32.5.0-py3

I got errors about pip packages for dataset training. So that, I used this container without errors. Maybe this can help you too.

Best wishes

3

Hello @mehmetdeniz
Still having the same issues using l4t-ml :

python3 -m pip install nvidia-pyindex
Collecting nvidia-pyindex
Downloading https://files.pythonhosted.org/packages/40/e6/56a6909f95409a31865399282f5f5a1692e89e7f5f5c5e7633d201e0fbd0/nvidia-pyindex-1.0.6.tar.gz
Building wheels for collected packages: nvidia-pyindex
Running setup.py bdist_wheel for nvidia-pyindex … error
Complete output from command /usr/bin/python3 -u -c “import setuptools, tokenize;file=‘/tmp/pip-build-j8u6vfmd/nvidia-pyindex/setup.py’;f=getattr(tokenize, ‘open’, open)(file);code=f.read().replace(‘\r\n’, ‘\n’);f.close();exec(compile(code, file, ‘exec’))” bdist_wheel -d /tmp/tmp49pmgcohpip-wheel- --python-tag cp36:
running bdist_wheel
running build
running build_py
creating build
creating build/lib
creating build/lib/nvidia_pyindex
copying nvidia_pyindex/init.py → build/lib/nvidia_pyindex
copying nvidia_pyindex/utils.py → build/lib/nvidia_pyindex
copying nvidia_pyindex/cmdline.py → build/lib/nvidia_pyindex
running egg_info
writing nvidia_pyindex.egg-info/PKG-INFO
writing dependency_links to nvidia_pyindex.egg-info/dependency_links.txt
writing entry points to nvidia_pyindex.egg-info/entry_points.txt
writing top-level names to nvidia_pyindex.egg-info/top_level.txt
reading manifest file ‘nvidia_pyindex.egg-info/SOURCES.txt’
reading manifest template ‘MANIFEST.in’
writing manifest file ‘nvidia_pyindex.egg-info/SOURCES.txt’
installing to build/bdist.linux-aarch64/wheel
running install
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
COMMAND: InstallCommand
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
/bin/sh: 1: nvidia_pyindex: not found
Traceback (most recent call last):
File “/tmp/pip-build-j8u6vfmd/nvidia-pyindex/nvidia_pyindex/utils.py”, line 17, in get_configuration_files
output = subprocess.check_output(command, stderr=subprocess.STDOUT)
File “/usr/lib/python3.6/subprocess.py”, line 356, in check_output
**kwargs).stdout
File “/usr/lib/python3.6/subprocess.py”, line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command ‘[’/usr/bin/python3’, ‘-c’, ‘from pip._internal.configuration import get_configuration_files; print(get_configuration_files())’]’ returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File “”, line 1, in
File “/tmp/pip-build-j8u6vfmd/nvidia-pyindex/setup.py”, line 269, in
packages=find_packages(),
File “/usr/local/lib/python3.6/dist-packages/setuptools/init.py”, line 153, in setup
return distutils.core.setup(**attrs)
File “/usr/lib/python3.6/distutils/core.py”, line 148, in setup
dist.run_commands()
File “/usr/lib/python3.6/distutils/dist.py”, line 955, in run_commands
self.run_command(cmd)
File “/usr/lib/python3.6/distutils/dist.py”, line 974, in run_command
cmd_obj.run()
File “/usr/local/lib/python3.6/dist-packages/wheel/bdist_wheel.py”, line 335, in run
self.run_command(‘install’)
File “/usr/lib/python3.6/distutils/cmd.py”, line 313, in run_command
self.distribution.run_command(command)
File “/usr/lib/python3.6/distutils/dist.py”, line 974, in run_command
cmd_obj.run()
File “/tmp/pip-build-j8u6vfmd/nvidia-pyindex/setup.py”, line 160, in run
_install_nvidia_pypi_index()
File “/tmp/pip-build-j8u6vfmd/nvidia-pyindex/setup.py”, line 59, in _install_nvidia_pypi_index
config_filedict = get_configuration_files()
File “/tmp/pip-build-j8u6vfmd/nvidia-pyindex/nvidia_pyindex/utils.py”, line 21, in get_configuration_files
raise RuntimeError(output) from e
RuntimeError: Traceback (most recent call last):
File “”, line 1, in
ModuleNotFoundError: No module named ‘pip._internal’

Failed building wheel for nvidia-pyindex
Running setup.py clean for nvidia-pyindex
Failed to build nvidia-pyindex
Installing collected packages: nvidia-pyindex
Running setup.py install for nvidia-pyindex … error
Complete output from command /usr/bin/python3 -u -c “import setuptools, tokenize;file=‘/tmp/pip-build-j8u6vfmd/nvidia-pyindex/setup.py’;f=getattr(tokenize, ‘open’, open)(file);code=f.read().replace(‘\r\n’, ‘\n’);f.close();exec(compile(code, file, ‘exec’))” install --record /tmp/pip-1hybbt22-record/install-record.txt --single-version-externally-managed --compile:
running install
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
COMMAND: InstallCommand
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
/bin/sh: 1: nvidia_pyindex: not found
Traceback (most recent call last):
File “/tmp/pip-build-j8u6vfmd/nvidia-pyindex/nvidia_pyindex/utils.py”, line 17, in get_configuration_files
output = subprocess.check_output(command, stderr=subprocess.STDOUT)
File “/usr/lib/python3.6/subprocess.py”, line 356, in check_output
**kwargs).stdout
File “/usr/lib/python3.6/subprocess.py”, line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command ‘[’/usr/bin/python3’, ‘-c’, ‘from pip._internal.configuration import get_configuration_files; print(get_configuration_files())’]’ returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/pip-build-j8u6vfmd/nvidia-pyindex/setup.py", line 269, in <module>
    packages=find_packages(),
  File "/usr/local/lib/python3.6/dist-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/tmp/pip-build-j8u6vfmd/nvidia-pyindex/setup.py", line 160, in run
    _install_nvidia_pypi_index()
  File "/tmp/pip-build-j8u6vfmd/nvidia-pyindex/setup.py", line 59, in _install_nvidia_pypi_index
    config_filedict = get_configuration_files()
  File "/tmp/pip-build-j8u6vfmd/nvidia-pyindex/nvidia_pyindex/utils.py", line 21, in get_configuration_files
    raise RuntimeError(output) from e
RuntimeError: Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'pip._internal'


----------------------------------------

Command “/usr/bin/python3 -u -c “import setuptools, tokenize;file=‘/tmp/pip-build-j8u6vfmd/nvidia-pyindex/setup.py’;f=getattr(tokenize, ‘open’, open)(file);code=f.read().replace(‘\r\n’, ‘\n’);f.close();exec(compile(code, file, ‘exec’))” install --record /tmp/pip-1hybbt22-record/install-record.txt --single-version-externally-managed --compile” failed with error code 1 in /tmp/pip-build-j8u6vfmd/nvidia-pyindex/

Also if I run :

python3
Python 3.6.9 (default, Oct 8 2020, 12:12:24)
[GCC 8.4.0] on linux
Type “help”, “copyright”, “credits” or “license” for more information.

import tensorrt as trt
trt.version
Traceback (most recent call last):
File “”, line 1, in
AttributeError: module ‘tensorrt’ has no attribute ‘version’

4

Hello @R_GSD

Can you run this command:
pip3 install --upgrade pip

before this:
python3 -m pip install nvidia-pyindex

5

Ah yes that was it !
But pip3 install --upgrade nvidia-tensorrt still fails :

Collecting nvidia-tensorrt
Downloading nvidia-tensorrt-0.0.1.dev4.tar.gz (3.8 kB)
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 -c ‘import sys, setuptools, tokenize; sys.argv[0] = ‘"’"’/tmp/pip-install-0e91vnmf/nvidia-tensorrt_6ec640e660a34290833ff97953802338/setup.py’“'”‘; file=’“'”‘/tmp/pip-install-0e91vnmf/nvidia-tensorrt_6ec640e660a34290833ff97953802338/setup.py’“'”‘;f=getattr(tokenize, ‘"’“‘open’”’“‘, open)(file);code=f.read().replace(’”‘"’\r\n’“'”‘, ‘"’"’\n’“'”‘);f.close();exec(compile(code, file, ‘"’“‘exec’”’"‘))’ egg_info --egg-base /tmp/pip-pip-egg-info-ks5o8doi
cwd: /tmp/pip-install-0e91vnmf/nvidia-tensorrt_6ec640e660a34290833ff97953802338/
Complete output (15 lines):
Traceback (most recent call last):
File “”, line 1, in
File “/tmp/pip-install-0e91vnmf/nvidia-tensorrt_6ec640e660a34290833ff97953802338/setup.py”, line 150, in
raise RuntimeError(open(“ERROR.txt”, “r”).read())
RuntimeError:
###########################################################################################
The package you are trying to install is only a placeholder project on PyPI.org repository.
This package is hosted on NVIDIA Python Package Index.

This package can be installed as:
$ pip install nvidia-pyindex $ pip install nvidia-tensorrt
###########################################################################################
----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/bf/88/324f19e3cc94c98421072c591ccd8170e0286d818a372eb38d45cc631119/nvidia-tensorrt-0.0.1.dev4.tar.gz#sha256=70f7ce9661852730482e51470f026d12149e53ad61acdf3d79cb5874d3cf677b (from Links for nvidia-tensorrt). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement nvidia-tensorrt
ERROR: No matching distribution found for nvidia-tensorrt

Which makes me think that I can’t install tensorRT this way on arm64

6

Hi @R_GSD, the TensorRT Python library is mounted from your device (outside container), the same way that CUDA/cuDNN/TensorRT are. They aren’t actually installed inside the container, they get mounted when --runtime nvidia is used. So you don’t need to install any TensorRT package inside container (and regardless the nvidia-pyindex package isn’t for Jetson/aarch64)

I just tested this on l4t-tensorflow container and was able to run it ok:

sudo docker run -it --rm --runtime nvidia --network host nvcr.io/nvidia/l4t-tensorflow:r32.5.0-tf1.15-py3
python3
>>> import tensorrt
>>> print(tensorrt.__version__)
7.1.3.0
>>> print(tensorrt.NetworkDefinitionCreationFlag)
<class 'tensorrt.tensorrt.NetworkDefinitionCreationFlag'>

If that doesn’t work for you, can you try running those python commands outside of container? If it doesn’t work outside of container either, then I wonder if your TensorRT installation got messed up (in that case you may want to re-flash your SD card)

7

Hello @dusty_nv

Ok I understand better !

I am running the container with --gpus all instead of --runtime nvidia

When I run with --runtime nvidia I have the error :

docker: Error response from daemon: Unknown runtime specified nvidia.

So I will look into that, thanks

8

Hmm. Did you install nvidia-docker yourself? Normally it would come pre-installed and work with --runtime nvidia. You may want to flash a fresh SD-card and start with a clean slate.

9

Yes I installed it myself because I am using Yocto and not the sdk, I might have missed something