How to properly install tao-deploy ? bash: tao-deploy: command not found

glingk · March 17, 2023, 10:03am

Please provide the following information when requesting support.

• Hardware : Orin AGX
• Network Type (Detectnet_v2/Faster_rcnn/Yolo_v4/LPRnet/Mask_rcnn/Classification/etc) : All
• TLT Version (Please run “tlt info --verbose” and share “docker_tag” here) TAO Toolkit 4.0
• Training spec file(If have, please share here)
• How to reproduce the issue ? (This is for errors. Please share the command line and the detailed log here.): Follow official nvidia’s instructions

I followed the instructions below, but when I run

tao-deploy info --verbose

I get the following error:

bash: tao-deploy: command not found

So my question is how to properly install tao-deploy?

TAO Deploy is also distributed as a public wheel file at PyPI. The wheel does not include TensorRT or TensorRT OSS as part of its dependencies. Hence, you must either install these dependencies through the official TensorRT website or invoke TensorRT container available on NGC.

Run the following command to install the nvidia-tao-deploy wheel in your python environment.

pip install nvidia-tao-deploy

Then, you can run TAO Deploy tasks with the tao-deploy prefix. For example, the following command will run a detectnet_v2 TensorRT engine generation for FP16.

detectnet_v2 gen_trt_engine -e /path/to/experiment/spec.txt \
    -m /path/to/etlt/file \
    -k $KEY \
    --data_type fp16 \
    --engine_file /path/to/engine/file

Morganh · March 20, 2023, 3:42am

Could you follow TAO Deploy Installation - NVIDIA Docs and retry?

glingk · March 20, 2023, 3:53am

That is what I followed already. Not sure what to do differently? Any suggestions are appreciated.

Morganh · March 20, 2023, 4:34am

What is the log when you run above?

glingk · March 20, 2023, 9:17am

@Morganh Here’s the log

Collecting nvidia-tao-deploy
  Downloading nvidia_tao_deploy-4.0.0.1-py3-none-any.whl (2.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.5/2.5 MB 26.4 MB/s eta 0:00:00
Requirement already satisfied: PyYAML>=5.1 in /usr/local/lib/python3.8/dist-packages (from nvidia-tao-deploy) (6.0)
Collecting h5py==3.7.0
  Downloading h5py-3.7.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (8.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.6/8.6 MB 69.6 MB/s eta 0:00:00
Collecting Pillow<9.0.0,>=8.1.0
  Downloading Pillow-8.4.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.0/3.0 MB 81.8 MB/s eta 0:00:00
Collecting onnxruntime
  Downloading onnxruntime-1.14.1-cp38-cp38-manylinux_2_27_aarch64.whl (4.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.7/4.7 MB 84.4 MB/s eta 0:00:00
Collecting seaborn==0.7.1
  Downloading seaborn-0.7.1.tar.gz (158 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 158.1/158.1 kB 37.6 MB/s eta 0:00:00
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting scikit-learn==0.24.2
  Downloading scikit_learn-0.24.2-cp38-cp38-manylinux2014_aarch64.whl (26.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 26.5/26.5 MB 67.6 MB/s eta 0:00:00
Collecting pynvml==11.0.0
  Downloading pynvml-11.0.0-py3-none-any.whl (46 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 46.1/46.1 kB 12.4 MB/s eta 0:00:00
Collecting protobuf==3.20.1
  Downloading protobuf-3.20.1-cp38-cp38-manylinux2014_aarch64.whl (918 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 918.1/918.1 kB 69.9 MB/s eta 0:00:00
Collecting scipy==1.5.4
  Downloading scipy-1.5.4-cp38-cp38-manylinux2014_aarch64.whl (24.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 24.5/24.5 MB 69.3 MB/s eta 0:00:00
Collecting absl-py>=0.7.1
  Downloading absl_py-1.4.0-py3-none-any.whl (126 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 126.5/126.5 kB 32.3 MB/s eta 0:00:00
Requirement already satisfied: onnx in /usr/local/lib/python3.8/dist-packages (from nvidia-tao-deploy) (1.13.0)
Collecting opencv-python
  Downloading opencv_python-4.7.0.72-cp37-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (40.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.4/40.4 MB 56.4 MB/s eta 0:00:00
Collecting hydra-core==1.2.0
  Downloading hydra_core-1.2.0-py3-none-any.whl (151 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 151.1/151.1 kB 21.7 MB/s eta 0:00:00
Collecting matplotlib>=3.0.3
  Downloading matplotlib-3.7.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (11.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.4/11.4 MB 89.3 MB/s eta 0:00:00
Collecting tqdm==4.64.0
  Downloading tqdm-4.64.0-py2.py3-none-any.whl (78 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.4/78.4 kB 20.1 MB/s eta 0:00:00
Requirement already satisfied: six>=1.12.0 in /usr/local/lib/python3.8/dist-packages (from nvidia-tao-deploy) (1.16.0)
Collecting omegaconf==2.2.2
  Downloading omegaconf-2.2.2-py3-none-any.whl (79 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 79.1/79.1 kB 22.0 MB/s eta 0:00:00
Collecting mpi4py>=3.0.3
  Downloading mpi4py-3.1.4.tar.gz (2.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.5/2.5 MB 77.2 MB/s eta 0:00:00
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting scikit-image==0.17.2
  Downloading scikit-image-0.17.2.tar.gz (29.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 29.8/29.8 MB 66.3 MB/s eta 0:00:00
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting pycocotools-fix
  Downloading pycocotools-fix-2.0.0.9.tar.gz (124 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 124.0/124.0 kB 12.8 MB/s eta 0:00:00
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: numpy>=1.14.5 in /usr/local/lib/python3.8/dist-packages (from h5py==3.7.0->nvidia-tao-deploy) (1.24.1)
Requirement already satisfied: importlib-resources in /usr/local/lib/python3.8/dist-packages (from hydra-core==1.2.0->nvidia-tao-deploy) (5.12.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from hydra-core==1.2.0->nvidia-tao-deploy) (23.0)
Collecting antlr4-python3-runtime==4.9.*
  Downloading antlr4-python3-runtime-4.9.3.tar.gz (117 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 117.0/117.0 kB 25.8 MB/s eta 0:00:00
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting PyWavelets>=1.1.1
  Downloading PyWavelets-1.4.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (6.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.9/6.9 MB 88.3 MB/s eta 0:00:00
Collecting imageio>=2.3.0
  Downloading imageio-2.26.1-py3-none-any.whl (3.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.4/3.4 MB 84.9 MB/s eta 0:00:00
Collecting networkx>=2.0
  Downloading networkx-3.0-py3-none-any.whl (2.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 80.4 MB/s eta 0:00:00
Collecting tifffile>=2019.7.26
  Downloading tifffile-2023.3.15-py3-none-any.whl (218 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 218.6/218.6 kB 45.3 MB/s eta 0:00:00
Collecting threadpoolctl>=2.0.0
  Downloading threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting joblib>=0.11
  Downloading joblib-1.2.0-py3-none-any.whl (297 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 298.0/298.0 kB 54.5 MB/s eta 0:00:00
Collecting pandas
  Downloading pandas-1.5.3-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (11.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.5/11.5 MB 93.4 MB/s eta 0:00:00
Collecting cycler>=0.10
  Using cached cycler-0.11.0-py3-none-any.whl (6.4 kB)
Collecting kiwisolver>=1.0.1
  Using cached kiwisolver-1.4.4-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.4 MB)
Collecting contourpy>=1.0.1
  Using cached contourpy-1.0.7-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (283 kB)
Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.8/dist-packages (from matplotlib>=3.0.3->nvidia-tao-deploy) (2.8.2)
Collecting fonttools>=4.22.0
  Using cached fonttools-4.39.2-py3-none-any.whl (1.0 MB)
Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.8/dist-packages (from matplotlib>=3.0.3->nvidia-tao-deploy) (3.0.9)
Requirement already satisfied: typing-extensions>=3.6.2.1 in /usr/local/lib/python3.8/dist-packages (from onnx->nvidia-tao-deploy) (4.4.0)
Collecting onnx
  Downloading onnx-1.13.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (13.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.8/13.8 MB 87.0 MB/s eta 0:00:00
  Downloading onnx-1.12.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (13.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.0/13.0 MB 71.9 MB/s eta 0:00:00
Collecting flatbuffers
  Downloading flatbuffers-23.3.3-py2.py3-none-any.whl (26 kB)
Collecting sympy
  Downloading sympy-1.11.1-py3-none-any.whl (6.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.5/6.5 MB 77.6 MB/s eta 0:00:00
Collecting coloredlogs
  Downloading coloredlogs-15.0.1-py2.py3-none-any.whl (46 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 46.0/46.0 kB 12.4 MB/s eta 0:00:00
Collecting cython>=0.27.3
  Using cached Cython-0.29.33-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.whl (1.8 MB)
Requirement already satisfied: setuptools>=18.0 in /usr/lib/python3/dist-packages (from pycocotools-fix->nvidia-tao-deploy) (45.2.0)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from importlib-resources->hydra-core==1.2.0->nvidia-tao-deploy) (3.15.0)
Collecting humanfriendly>=9.1
  Downloading humanfriendly-10.0-py2.py3-none-any.whl (86 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 86.8/86.8 kB 24.6 MB/s eta 0:00:00
Collecting pytz>=2020.1
  Downloading pytz-2022.7.1-py2.py3-none-any.whl (499 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 499.4/499.4 kB 62.4 MB/s eta 0:00:00
Collecting mpmath>=0.19
  Downloading mpmath-1.3.0-py3-none-any.whl (536 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 536.2/536.2 kB 66.2 MB/s eta 0:00:00
Building wheels for collected packages: scikit-image, seaborn, antlr4-python3-runtime, mpi4py, pycocotools-fix
  Building wheel for scikit-image (setup.py): started
  Building wheel for scikit-image (setup.py): still running...
  Building wheel for scikit-image (setup.py): still running...
  Building wheel for scikit-image (setup.py): still running...
  Building wheel for scikit-image (setup.py): still running...
  Building wheel for scikit-image (setup.py): still running...
  Building wheel for scikit-image (setup.py): still running...
  Building wheel for scikit-image (setup.py): still running...
  Building wheel for scikit-image (setup.py): still running...
  Building wheel for scikit-image (setup.py): finished with status 'done'
  Created wheel for scikit-image: filename=scikit_image-0.17.2-cp38-cp38-linux_aarch64.whl size=34471460 sha256=c2d9e564a56c4de7acfed7e7e1cccf971941815c7a9820e62f9cdc27c1696323
  Stored in directory: /root/.cache/pip/wheels/d9/db/c4/f79b4fbcc275680296a2ea558e621fb0bee48ae9a0363f4cd0
  Building wheel for seaborn (setup.py): started
  Building wheel for seaborn (setup.py): finished with status 'done'
  Created wheel for seaborn: filename=seaborn-0.7.1-py3-none-any.whl size=165935 sha256=c020d462596f127b003d5c33b60dd94141b24ea0f83898b7a9971cc2450f2101
  Stored in directory: /root/.cache/pip/wheels/e4/5e/0b/65d26eed3baa276f2965271668fb2e2a0a3141b6fabb9c9563
  Building wheel for antlr4-python3-runtime (setup.py): started
  Building wheel for antlr4-python3-runtime (setup.py): finished with status 'done'
  Created wheel for antlr4-python3-runtime: filename=antlr4_python3_runtime-4.9.3-py3-none-any.whl size=144573 sha256=47761672b49128978d05f9ff497731dd9e447e2fe0ebb1a78c4193ab449b22a4
  Stored in directory: /root/.cache/pip/wheels/11/20/e3/33bfd201db65084a9bb7ec0fd4040c2a7b59bb155698396a7c
  Building wheel for mpi4py (pyproject.toml): started
  Building wheel for mpi4py (pyproject.toml): finished with status 'error'
  error: subprocess-exited-with-error
  
  × Building wheel for mpi4py (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [148 lines of output]
      running bdist_wheel
      running build
      running build_src
      running build_py
      creating build
      creating build/lib.linux-aarch64-cpython-38
      creating build/lib.linux-aarch64-cpython-38/mpi4py
      copying src/mpi4py/run.py -> build/lib.linux-aarch64-cpython-38/mpi4py
      copying src/mpi4py/__main__.py -> build/lib.linux-aarch64-cpython-38/mpi4py
      copying src/mpi4py/__init__.py -> build/lib.linux-aarch64-cpython-38/mpi4py
      copying src/mpi4py/bench.py -> build/lib.linux-aarch64-cpython-38/mpi4py
      creating build/lib.linux-aarch64-cpython-38/mpi4py/futures
      copying src/mpi4py/futures/server.py -> build/lib.linux-aarch64-cpython-38/mpi4py/futures
      copying src/mpi4py/futures/_lib.py -> build/lib.linux-aarch64-cpython-38/mpi4py/futures
      copying src/mpi4py/futures/__main__.py -> build/lib.linux-aarch64-cpython-38/mpi4py/futures
      copying src/mpi4py/futures/_core.py -> build/lib.linux-aarch64-cpython-38/mpi4py/futures
      copying src/mpi4py/futures/__init__.py -> build/lib.linux-aarch64-cpython-38/mpi4py/futures
      copying src/mpi4py/futures/pool.py -> build/lib.linux-aarch64-cpython-38/mpi4py/futures
      copying src/mpi4py/futures/aplus.py -> build/lib.linux-aarch64-cpython-38/mpi4py/futures
      copying src/mpi4py/futures/_base.py -> build/lib.linux-aarch64-cpython-38/mpi4py/futures
      creating build/lib.linux-aarch64-cpython-38/mpi4py/util
      copying src/mpi4py/util/__init__.py -> build/lib.linux-aarch64-cpython-38/mpi4py/util
      copying src/mpi4py/util/pkl5.py -> build/lib.linux-aarch64-cpython-38/mpi4py/util
      copying src/mpi4py/util/dtlib.py -> build/lib.linux-aarch64-cpython-38/mpi4py/util
      copying src/mpi4py/py.typed -> build/lib.linux-aarch64-cpython-38/mpi4py
      copying src/mpi4py/__main__.pyi -> build/lib.linux-aarch64-cpython-38/mpi4py
      copying src/mpi4py/bench.pyi -> build/lib.linux-aarch64-cpython-38/mpi4py
      copying src/mpi4py/MPI.pyi -> build/lib.linux-aarch64-cpython-38/mpi4py
      copying src/mpi4py/run.pyi -> build/lib.linux-aarch64-cpython-38/mpi4py
      copying src/mpi4py/__init__.pyi -> build/lib.linux-aarch64-cpython-38/mpi4py
      copying src/mpi4py/dl.pyi -> build/lib.linux-aarch64-cpython-38/mpi4py
      copying src/mpi4py/__init__.pxd -> build/lib.linux-aarch64-cpython-38/mpi4py
      copying src/mpi4py/MPI.pxd -> build/lib.linux-aarch64-cpython-38/mpi4py
      copying src/mpi4py/libmpi.pxd -> build/lib.linux-aarch64-cpython-38/mpi4py
      creating build/lib.linux-aarch64-cpython-38/mpi4py/include
      creating build/lib.linux-aarch64-cpython-38/mpi4py/include/mpi4py
      copying src/mpi4py/include/mpi4py/mpi4py.MPI_api.h -> build/lib.linux-aarch64-cpython-38/mpi4py/include/mpi4py
      copying src/mpi4py/include/mpi4py/mpi4py.h -> build/lib.linux-aarch64-cpython-38/mpi4py/include/mpi4py
      copying src/mpi4py/include/mpi4py/mpi4py.MPI.h -> build/lib.linux-aarch64-cpython-38/mpi4py/include/mpi4py
      copying src/mpi4py/include/mpi4py/mpi4py.i -> build/lib.linux-aarch64-cpython-38/mpi4py/include/mpi4py
      copying src/mpi4py/include/mpi4py/mpi.pxi -> build/lib.linux-aarch64-cpython-38/mpi4py/include/mpi4py
      copying src/mpi4py/futures/aplus.pyi -> build/lib.linux-aarch64-cpython-38/mpi4py/futures
      copying src/mpi4py/futures/_core.pyi -> build/lib.linux-aarch64-cpython-38/mpi4py/futures
      copying src/mpi4py/futures/server.pyi -> build/lib.linux-aarch64-cpython-38/mpi4py/futures
      copying src/mpi4py/futures/__main__.pyi -> build/lib.linux-aarch64-cpython-38/mpi4py/futures
      copying src/mpi4py/futures/pool.pyi -> build/lib.linux-aarch64-cpython-38/mpi4py/futures
      copying src/mpi4py/futures/__init__.pyi -> build/lib.linux-aarch64-cpython-38/mpi4py/futures
      copying src/mpi4py/futures/_lib.pyi -> build/lib.linux-aarch64-cpython-38/mpi4py/futures
      copying src/mpi4py/util/dtlib.pyi -> build/lib.linux-aarch64-cpython-38/mpi4py/util
      copying src/mpi4py/util/__init__.pyi -> build/lib.linux-aarch64-cpython-38/mpi4py/util
      copying src/mpi4py/util/pkl5.pyi -> build/lib.linux-aarch64-cpython-38/mpi4py/util
      running build_clib
      MPI configuration: [mpi] from 'mpi.cfg'
      checking for library 'lmpe' ...
      aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -c _configtest.c -o _configtest.o
      aarch64-linux-gnu-gcc -pthread _configtest.o -llmpe -o _configtest
      /usr/bin/ld: cannot find -llmpe
      collect2: error: ld returned 1 exit status
      failure.
      removing: _configtest.c _configtest.o
      building 'mpe' dylib library
      creating build/temp.linux-aarch64-cpython-38
      creating build/temp.linux-aarch64-cpython-38/src
      creating build/temp.linux-aarch64-cpython-38/src/lib-pmpi
      aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -c src/lib-pmpi/mpe.c -o build/temp.linux-aarch64-cpython-38/src/lib-pmpi/mpe.o
      creating build/lib.linux-aarch64-cpython-38/mpi4py/lib-pmpi
      aarch64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 -Wl,--no-as-needed build/temp.linux-aarch64-cpython-38/src/lib-pmpi/mpe.o -o build/lib.linux-aarch64-cpython-38/mpi4py/lib-pmpi/libmpe.so
      checking for library 'vt-mpi' ...
      aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -c _configtest.c -o _configtest.o
      aarch64-linux-gnu-gcc -pthread _configtest.o -lvt-mpi -o _configtest
      /usr/bin/ld: cannot find -lvt-mpi
      collect2: error: ld returned 1 exit status
      failure.
      removing: _configtest.c _configtest.o
      checking for library 'vt.mpi' ...
      aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -c _configtest.c -o _configtest.o
      aarch64-linux-gnu-gcc -pthread _configtest.o -lvt.mpi -o _configtest
      /usr/bin/ld: cannot find -lvt.mpi
      collect2: error: ld returned 1 exit status
      failure.
      removing: _configtest.c _configtest.o
      building 'vt' dylib library
      aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -c src/lib-pmpi/vt.c -o build/temp.linux-aarch64-cpython-38/src/lib-pmpi/vt.o
      aarch64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 -Wl,--no-as-needed build/temp.linux-aarch64-cpython-38/src/lib-pmpi/vt.o -o build/lib.linux-aarch64-cpython-38/mpi4py/lib-pmpi/libvt.so
      checking for library 'vt-mpi' ...
      aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -c _configtest.c -o _configtest.o
      aarch64-linux-gnu-gcc -pthread _configtest.o -lvt-mpi -o _configtest
      /usr/bin/ld: cannot find -lvt-mpi
      collect2: error: ld returned 1 exit status
      failure.
      removing: _configtest.c _configtest.o
      checking for library 'vt.mpi' ...
      aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -c _configtest.c -o _configtest.o
      aarch64-linux-gnu-gcc -pthread _configtest.o -lvt.mpi -o _configtest
      /usr/bin/ld: cannot find -lvt.mpi
      collect2: error: ld returned 1 exit status
      failure.
      removing: _configtest.c _configtest.o
      building 'vt-mpi' dylib library
      aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -c src/lib-pmpi/vt-mpi.c -o build/temp.linux-aarch64-cpython-38/src/lib-pmpi/vt-mpi.o
      aarch64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 -Wl,--no-as-needed build/temp.linux-aarch64-cpython-38/src/lib-pmpi/vt-mpi.o -o build/lib.linux-aarch64-cpython-38/mpi4py/lib-pmpi/libvt-mpi.so
      checking for library 'vt-hyb' ...
      aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -c _configtest.c -o _configtest.o
      aarch64-linux-gnu-gcc -pthread _configtest.o -lvt-hyb -o _configtest
      /usr/bin/ld: cannot find -lvt-hyb
      collect2: error: ld returned 1 exit status
      failure.
      removing: _configtest.c _configtest.o
      checking for library 'vt.ompi' ...
      aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -c _configtest.c -o _configtest.o
      aarch64-linux-gnu-gcc -pthread _configtest.o -lvt.ompi -o _configtest
      /usr/bin/ld: cannot find -lvt.ompi
      collect2: error: ld returned 1 exit status
      failure.
      removing: _configtest.c _configtest.o
      building 'vt-hyb' dylib library
      aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -c src/lib-pmpi/vt-hyb.c -o build/temp.linux-aarch64-cpython-38/src/lib-pmpi/vt-hyb.o
      aarch64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 -Wl,--no-as-needed build/temp.linux-aarch64-cpython-38/src/lib-pmpi/vt-hyb.o -o build/lib.linux-aarch64-cpython-38/mpi4py/lib-pmpi/libvt-hyb.so
      running build_ext
      MPI configuration: [mpi] from 'mpi.cfg'
      checking for dlopen() availability ...
      checking for header 'dlfcn.h' ...
      aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/usr/include/python3.8 -c _configtest.c -o _configtest.o
      success!
      removing: _configtest.c _configtest.o
      success!
      checking for library 'dl' ...
      aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/usr/include/python3.8 -c _configtest.c -o _configtest.o
      aarch64-linux-gnu-gcc -pthread _configtest.o -Lbuild/temp.linux-aarch64-cpython-38 -ldl -o _configtest
      success!
      removing: _configtest.c _configtest.o _configtest
      checking for function 'dlopen' ...
      aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/usr/include/python3.8 -c _configtest.c -o _configtest.o
      aarch64-linux-gnu-gcc -pthread _configtest.o -Lbuild/temp.linux-aarch64-cpython-38 -ldl -o _configtest
      success!
      removing: _configtest.c _configtest.o _configtest
      building 'mpi4py.dl' extension
      aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DHAVE_DLFCN_H=1 -DHAVE_DLOPEN=1 -I/usr/include/python3.8 -c src/dynload.c -o build/temp.linux-aarch64-cpython-38/src/dynload.o
      aarch64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 build/temp.linux-aarch64-cpython-38/src/dynload.o -Lbuild/temp.linux-aarch64-cpython-38 -ldl -o build/lib.linux-aarch64-cpython-38/mpi4py/dl.cpython-38-aarch64-linux-gnu.so
      checking for MPI compile and link ...
      aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/usr/include/python3.8 -c _configtest.c -o _configtest.o
      _configtest.c:2:10: fatal error: mpi.h: No such file or directory
          2 | #include <mpi.h>
            |          ^~~~~~~
      compilation terminated.
      failure.
      removing: _configtest.c _configtest.o
      error: Cannot compile MPI programs. Check your configuration!!!
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for mpi4py
  Building wheel for pycocotools-fix (setup.py): started
  Building wheel for pycocotools-fix (setup.py): finished with status 'done'
  Created wheel for pycocotools-fix: filename=pycocotools_fix-2.0.0.9-cp38-cp38-linux_aarch64.whl size=406522 sha256=8816bc724c724f1681e528605fc70d9a52b0c27c6cf2a4747bf8b358cf6289ef
  Stored in directory: /root/.cache/pip/wheels/49/87/16/79cbbe2d1603a1d8063b97cf9bf604d55d4bd0b61c691dd10a
Successfully built scikit-image seaborn antlr4-python3-runtime pycocotools-fix
Failed to build mpi4py
ERROR: Could not build wheels for mpi4py, which is required to install pyproject.toml-based projects

[notice] A new release of pip available: 22.3.1 -> 23.0.1
[notice] To update, run: python -m pip install --upgrade pip
The command '/bin/sh -c pip install nvidia-tao-deploy' returned a non-zero code: 1

Morganh · March 20, 2023, 3:16pm

Try another way.
Please pull the tao deploy docker and login it to run commands.
From TAO Toolkit | NVIDIA NGC , you can
$ docker pull nvcr.io/nvidia/tao/tao-toolkit:4.0.0-deploy

Then, login this docker,
$ docker run --runtime=nvidia -it --rm nvcr.io/nvidia/tao/tao-toolkit:4.0.0-deploy /bin/bash

Then, run something like

detectnet_v2 gen_trt_engine -e /path/to/experiment/spec.txt \
    -m /path/to/etlt/file \
    -k $KEY \
    --data_type fp16 \
    --engine_file /path/to/engine/file

glingk · March 21, 2023, 4:43am

@Morganh I got the following error message:

developer@orin-lab:~$ docker run --runtime=nvidia -it --rm nvcr.io/nvidia/tao/tao-toolkit:4.0.0-deploy /bin/bash
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
exec /opt/nvidia/nvidia_entrypoint.sh: exec format error

FYI, I’m trying to get an engine/plan that will run on Orin AGX

Morganh · March 21, 2023, 5:42am

Got it. Please note that the nvcr.io/nvidia/tao/tao-toolkit:4.0.0-deploy cannot run in Jetson Orin.

To run inference in Jetson Orin, you can use one of below ways.

Run with deepstream. GitHub - NVIDIA-AI-IOT/deepstream_tao_apps: Sample apps to demonstrate how to deploy models trained with TAO on DeepStream .
Run with triton application. GitHub - NVIDIA-AI-IOT/tao-toolkit-triton-apps: Sample app code for deploying TAO Toolkit trained models to Triton

glingk · March 21, 2023, 6:02am

@Morganh
I finally successfully install pip install nvidia-tao-deploy according to Installing TAO Deploy through wheel, but now when I try to run the sample command:

detectnet_v2 gen_trt_engine -e /path/to/experiment/spec.txt \
    -m /path/to/etlt/file \
    -k $KEY \
    --data_type fp16 \
    --engine_file /path/to/engine/file

I get this error instead:

/usr/local/lib/python3.8/dist-packages/pytransform_vax_001219/_pytransform_vax_001219.so: cannot open shared object file: No such file or directory

I tried adding that path to LD_LIBRARY_PATH, but I’m still getting the same error…

How should I fix this error?

Morganh · March 21, 2023, 6:06am

May I know that if you follow below?

Installing TAO Deploy on a Jetson Platform

You can download the nvidia-tao-deploy wheel to a jetson platform using the same commands as the x86 platform installation. We recommend using the NVIDIA TensorRT Docker container that already includes the TensorRT installation. Due to memory issues, you should first run the gen_trt_engine subtask on the x86 platform to generate the engine; you can then use the generated engine to run inference or evaluation on the Jetson platform and with the target dataset.

glingk · March 21, 2023, 6:18am

@Morganh Yes, I used the pip install approach according to the docs.

Morganh · March 21, 2023, 6:20am

May I know that which TRT docker you are using?

glingk · March 21, 2023, 6:33am

@Morganh
Here’s the content of the dockerfile I used for the environment:

FROM nvcr.io/nvidia/l4t-tensorrt:r8.5.2.2-devel
RUN apt update
RUN apt --fix-broken install
RUN apt install -y mpich
RUN pip install mpi4py
RUN pip install torchinfo
RUN pip install clearml
RUN pip install segmentation-models-pytorch
RUN pip install transformers
RUN pip install nvidia-tao-deploy

Morganh · March 21, 2023, 8:14am

For workaround, can you use tao-converter to generate tensorrt engine using .etlt file?

glingk · March 21, 2023, 8:16am

@Morganh will that work with a model train on TAO 4.0 ?
Or would I need to retrain the model with TAO 3 ?

Morganh · March 21, 2023, 8:17am

It can work. Not needed to retrain.

glingk · March 21, 2023, 8:17am

@Morganh Thank you