Help for converting .onnx to TensorRT

Hi I am a beginner in this field so I require some help. I have a NVIDIA AGX Orin 64 GP Developer Kit and want to implement Realtime Facial Recognition on it. I read the documentation (Developer Guide) but have some queries regarding model conversion

  1. Can I convert any model from .onnx to tensorRT engine (eg. arcFace, Resnet100, etc.) using trtexec ??
  2. If my TensorRT is utilizing the GPU properly is the above conversion the most efficient way to run the model ?
  3. Is the usage of –useDLACore=0 while making the tensorRT engine enough to utilize DLA or do I have to menton anything when running these models on DeepStream ?


1. In general yes, but if your model contains a special layer, TensorRT might not have a corresponding implementation for it.
2. Yes, efficient in both performance and memory.
3. You can put the model on DLA with --useDLACore=[ID] when inferring with trtexec.
For Deepstream, please find below the topic to modify the config to run on DLA.


1 Like

Thanks a lot @AastaLLL How do I verify whether my .engine file of the model I generated is using DLA.
The execution was successful and I passed --useDLACore=0 --allowGPUFallback as arguements when I converted my onnx file to engine. Just for your reference I am using arcface_r100_v1.onnx and converting that to engine. I later plan to use it in Deepstream.


You should see the corresponding info with trtexec:

For example:

$ /usr/src/tensorrt/bin/trtexec --onnx=/usr/src/tensorrt/data/resnet50/ResNet50.onnx --saveEngine=ResNet50.trt --useDLACore=0 --allowGPUFallback
[04/10/2024-05:31:18] [I] === System Options ===
[04/10/2024-05:31:18] [I] Device: 0
[04/10/2024-05:31:18] [I] DLACore: 0(With GPU fallback)
[04/10/2024-05:31:18] [I] Plugins:
[04/10/2024-05:31:25] [I] [TRT] ---------- Layers Running on DLA ----------
[04/10/2024-05:31:25] [I] [TRT] [DlaLayer] {ForeignNode[node_of_gpu_0/conv1_1 + node_of_gpu_0/res_conv1_bn_1...node_of_gpu_0/pred_1]}
[04/10/2024-05:31:25] [I] [TRT] ---------- Layers Running on GPU ----------
[04/10/2024-05:31:25] [I] [TRT] [GpuLayer] SHUFFLE: reshape_after_node_of_gpu_0/pred_1
[04/10/2024-05:31:25] [I] [TRT] [GpuLayer] SOFTMAX: (Unnamed Layer* 180) [Softmax]
&&&& PASSED TensorRT.trtexec [TensorRT v8502] # /usr/src/tensorrt/bin/trtexec --onnx=/usr/src/tensorrt/data/resnet50/ResNet50.onnx --saveEngine=ResNet50.trt --useDLACore=0 --allowGPUFallback


This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.