DLA utilization in Xavier AGX

I am currently convt. Pytorch models to Onnx and using trtexec for trt engine generation to do inference.
./trtexec --onnx=resnet18.onnx --saveEngine=resnet18.trt --fp16 --useDLACore=0 --allowGPUFallback
While generating engine I am able to check my dla0 is active by below command
cat /sys/devices/platform/host1x/158c0000.nvdla0/power/runtime_status → active

Platform : Xavier AGX
TensorRT : 8.0.2
CUDA : 10.2

  1. Is there any way to check DLA utilization similar to GPU (by tegrastats)?
  2. How can I know that “infererce.ipny” script inference is being done on (DLA + GPU) for images.

Have uploaded script, onnx model, images, label.txt and trt engine
files.tar.xz (74.5 MB)


First, you can find the hardware placement from the TensorRT log directly.

$ /usr/src/tensorrt/bin/trtexec --onnx=/usr/src/tensorrt/data/mnist/mnist.onnx --useDLACore=0 --allowGPUFallback
[02/10/2022-01:49:47] [I] [TRT] ---------- Layers Running on DLA ----------
[02/10/2022-01:49:47] [I] [TRT] [DlaLayer] {ForeignNode[Convolution28]}
[02/10/2022-01:49:47] [I] [TRT] [DlaLayer] {ForeignNode[ReLU32...Convolution110]}
[02/10/2022-01:49:47] [I] [TRT] [DlaLayer] {ForeignNode[ReLU114...Pooling160]}
[02/10/2022-01:49:47] [I] [TRT] [DlaLayer] {ForeignNode[Plus214]}
[02/10/2022-01:49:47] [I] [TRT] ---------- Layers Running on GPU ----------
[02/10/2022-01:49:47] [I] [TRT] [GpuLayer] Parameter193 + Times212_reshape1
[02/10/2022-01:49:47] [I] [TRT] [GpuLayer] Parameter194
[02/10/2022-01:49:47] [I] [TRT] [GpuLayer] Parameter6 + (Unnamed Layer* 4) [Shuffle] + Plus30
[02/10/2022-01:49:47] [I] [TRT] [GpuLayer] Parameter88 + (Unnamed Layer* 10) [Shuffle] + Plus112
[02/10/2022-01:49:47] [I] [TRT] [GpuLayer] Times212_reshape0
[02/10/2022-01:49:47] [I] [TRT] [GpuLayer] Times212
[02/10/2022-01:49:47] [I] [TRT] [GpuLayer] shuffle_Times212_Output_0
[02/10/2022-01:49:47] [I] [TRT] [GpuLayer] shuffle_(Unnamed Layer* 16) [Constant]_output

To get more detailed profiling results for DLA, please use our Nsight System profiler:


This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.