Tao-converter for NVIDIA IGX Orin Development Kit

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

I’ve been trying to run tao-converter binary in DeepStream 6.4 container, but the latest available version for ARMs (v5.1.0_jp6.0_aarch64 | November 28, 2023) present on NGC fails to run in the NVIDIA IGX Orin Development Kit.

Where can I find or build the tao-converter for this HW?

Steps to reproduce:

root@NVIDIA-IGX-Orin-Development-Kit:~# docker run --runtime=nvidia -it --rm --net=host -w /opt/nvidia/deepstream/deepstream-6.4 nvcr.io/nvidia/deepstream:6.4-triton-multiarch

root@NVIDIA-IGX-Orin-Development-Kit:/opt/nvidia/deepstream/deepstream-6.4# wget --content-disposition 'https://api.ngc.nvidia.com/v2/resources/org/nvidia/team/tao/tao-converter/v5.1.0_jp6.0_aarch64/files?redirect=true&path=tao-converter' -O tao-converter

root@NVIDIA-IGX-Orin-Development-Kit:/opt/nvidia/deepstream/deepstream-6.4# chmod 755 tao-converter

root@NVIDIA-IGX-Orin-Development-Kit:/opt/nvidia/deepstream/deepstream-6.4# ./tao-converter 
./tao-converter: error while loading shared libraries: libnvdla_compiler.so: cannot open shared object file: No such file or directory
3

Could you please try this one TAO Converter | NVIDIA NGC?

4

@Morganh the version 4.0.0_trt8.5.2.2_aarch64 also doesn’t works on the IGX (DeepStream 6.4 container):

root@NVIDIA-IGX-Orin-Development-Kit:~# docker run --runtime=nvidia -it --rm --net=host -w /opt/nvidia/deepstream/deepstream-6.4 nvcr.io/nvidia/deepstream:6.4-triton-multiarch

root@NVIDIA-IGX-Orin-Development-Kit:/opt/nvidia/deepstream/deepstream-6.4# wget --content-disposition 'https://api.ngc.nvidia.com/v2/resources/org/nvidia/team/tao/tao-converter/v4.0.0_trt8.5.2.2_aarch64/files?redirect=true&path=tao-converter' -O tao-converter

root@NVIDIA-IGX-Orin-Development-Kit:/opt/nvidia/deepstream/deepstream-6.4# chmod 755 tao-converter

root@NVIDIA-IGX-Orin-Development-Kit:/opt/nvidia/deepstream/deepstream-6.4# ./tao-converter 
./tao-converter: error while loading shared libraries: libcrypto.so.1.1: cannot open shared object file: No such file or directory
5

I tried the x86 version of tao-converter in a dgpu machine and it worked.
Need to check further.

And actually you can use trtexec instead to generate tensorrt engine.
We recommend this way. More info can be found in Optimizing and Profiling with TensorRT - NVIDIA Docs.

6

The page you shared has the following note:

TAO Converter, which has been deprecated for x86 devices, is still required for deploying to Jetson devices. TAO Converter is distributed as a separate binary for x86 and Jetson platforms. The tao-converter binaries are available as an NGC resource

Also for our use-case we need to find some other path to perform the transition between tao-converter to trtexec, as we’re still deploying several ETLT models to devices running older DeepStream versions (some proprietary, others obtained from NVIDIA NGC. From what I can see besides PeopleNet the majority of NGC models were not yet updated to a decrypted format like ONNX).

NVIDIA doesn’t plan to keep backwards support for ETLT models (and tao-converter) in the upcoming DeepStream versions?

7

Could you try the hint from ImportError: libnvdla_compiler.so: cannot open shared object file: No such file or directory - #4 by majingqiauto123? Thanks.

8

The NVIDIA container runtime tries to load those:

root@NVIDIA-IGX-Orin-Development-Kit:~# cat /etc/nvidia-container-runtime/host-files-for-container.d/drivers.csv | grep libnvdla*
lib, /usr/lib/aarch64-linux-gnu/tegra/libnvdla_compiler.so
lib, /usr/lib/aarch64-linux-gnu/tegra/libnvdla_runtime.so

But only “libnvdla_runtime.so” is present in the IGX OS image:

root@NVIDIA-IGX-Orin-Development-Kit:~# ls -lah /usr/lib/aarch64-linux-gnu/tegra/libnvdla*
-rw-r--r-- 1 root root 6.2M Dec  6 12:51 /usr/lib/aarch64-linux-gnu/tegra/libnvdla_runtime.so

root@NVIDIA-IGX-Orin-Development-Kit:~#  ls -lah /usr/lib/aarch64-linux-gnu/tegra/libnvdla_compiler.so
ls: cannot access '/usr/lib/aarch64-linux-gnu/tegra/libnvdla_compiler.so': No such file or directory
9

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

Please check if ImportError: libnvdla_compiler.so: cannot open shared object file: No such file or directory helps. If not, please create a topic under Jetson & Embedded Systems - NVIDIA Developer Forums to ask why your IGX Orin does not contain “libnvdla_runtime.so”.

10

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.