TRT on Drive OS: Failed to get number of available DLA devices

Please provide the following info (tick the boxes after creating this topic):
Software Version
DRIVE OS 6.0.6
DRIVE OS 6.0.5
DRIVE OS 6.0.4 (rev. 1)

Target Operating System
[*] Linux

Hardware Platform
DRIVE AGX Orin Developer Kit (940-63710-0010-300)
DRIVE AGX Orin Developer Kit (940-63710-0010-200)
DRIVE AGX Orin Developer Kit (940-63710-0010-100)
DRIVE AGX Orin Developer Kit (940-63710-0010-D00)
DRIVE AGX Orin Developer Kit (940-63710-0010-C00)
DRIVE AGX Orin Developer Kit (not sure its number)
[*] other (Orin IGX)

SDK Manager Version

Host Machine Version
[*] native Ubuntu Linux 20.04 Host installed with SDK Manager
native Ubuntu Linux 20.04 Host installed with DRIVE OS Docker Containers
native Ubuntu Linux 18.04 Host installed with DRIVE OS Docker Containers

I use command "sudo dpkg -i ./nv-tensorrt-repo-ubuntu2004-cuda11.4-trt8.6.12.4-d6l-target-ga-20231014_1-1_arm64.deb " to install TRT on Drive Os.
After using command “sudo make TARGET=aarch64” in /usr/src/tensorrt/samples, I get the trtexec.
Howerver, when I use trtexec to convert onnx model, the following error occurs:

2024-04-29 11-51-38 的屏幕截图

What is the problem?
Thank you for your reply!

You need to generate TRT model on hardware where you want to use(may be it is x86 host or Drive platform). You can run inference only when there are available resources. If you want to run two inference models in parallel, you can either schedule them on different resources. For example, if you take DRIVE AGX Orin platform, it has a iGPU and DLA engines. So you generate GPU/DLA TRT model using trtexec or TRT APIs to run the model on the specific engine. In case, you want to run two inference tasks on same GPU, you can use different cudaStreams for each task to achieve it.
But, note that, launching two tasks in parallel does not mean they run in parallel. if the GPU is fully used by one task, other task can not run. You can actually verify if two GPU tasks are running in nsys trace.

I did not understand your test. what is the platform your are intended to run your model? Does the testing platform already has any other GPU/DLA workload?

Please note that this platform is exclusively for developers who are part of the NVIDIA DRIVE™ AGX SDK Developer Program.
Are your using DRIVE AGX Orin platform or Jetson platform?


Thanks for your reply.
The ONNX model was created on x86 machines and wants to deploy it on the Orin IGX platform (the operating system is drive os).
The testing platform does not have any other GPU/DLA workloads.
Please note that the device I’m using is the Orin IGX with Drive OS.

Dear @liangjz,
May I know the used platform? Is it DRIVE AGX Orin devkit?
If so, you don’t need to need to install TensorRT packages on target. It gets installed automatically when flashing DRIVE OS. trtexec can be found at /usr/src/tensorrt/bin. Could you please check using that?
Also, DRIVE OS comes with TensorRT How did you get /nv-tensorrt-repo-ubuntu2004-cuda11.4-trt8.6.12.4-d6l-target-ga-20231014_1-1_arm64.deb ?

nvidia@tegra-ubuntu:/usr/src/tensorrt/bin$ sudo dpkg -l | grep TensorRT
[sudo] password for nvidia:
ii  libnvinfer-bin                                  arm64        TensorRT binaries
ii  libnvinfer-dispatch8                            arm64        TensorRT dispatch runtime library
ii  libnvinfer-lean8                                arm64        TensorRT lean runtime library
ii  libnvinfer-plugin8                              arm64        TensorRT plugin libraries
ii  libnvinfer-vc-plugin8                           arm64        TensorRT vc-plugin library
ii  libnvinfer8                                     arm64        TensorRT runtime libraries
ii  libnvonnxparsers8                               arm64        TensorRT ONNX libraries
ii  libnvparsers8                                   arm64        TensorRT parsers libraries
ii  python3-libnvinfer                              arm64        Python 3 bindings for TensorRT standard runtime

This is not we can support, you should use IGX SW on your device, not with DRIVE OS SW.
For issue on IGX platform, please open topic at Latest IGX Orin Support Forum/IGX Developer Kit topics - NVIDIA Developer Forums

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.