DLA utilization in Xavier AGX

[user93474]

I am currently convt. Pytorch models to Onnx and using trtexec for trt engine generation to do inference.
./trtexec --onnx=resnet18.onnx --saveEngine=resnet18.trt --fp16 --useDLACore=0 --allowGPUFallback
While generating engine I am able to check my dla0 is active by below command
cat /sys/devices/platform/host1x/158c0000.nvdla0/power/runtime_status → active

Platform : Xavier AGX
TensorRT : 8.0.2
CUDA : 10.2

1. Is there any way to check DLA utilization similar to GPU (by tegrastats)?
2. How can I know that “infererce.ipny” script inference is being done on (DLA + GPU) for images.

Have uploaded script, onnx model, images, label.txt and trt engine
files.tar.xz (74.5 MB)

[AastaLLL]

Hi,

First, you can find the hardware placement from the TensorRT log directly.

$ /usr/src/tensorrt/bin/trtexec --onnx=/usr/src/tensorrt/data/mnist/mnist.onnx --useDLACore=0 --allowGPUFallback
...
[02/10/2022-01:49:47] [I] [TRT] ---------- Layers Running on DLA ----------
[02/10/2022-01:49:47] [I] [TRT] [DlaLayer] {ForeignNode[Convolution28]}
[02/10/2022-01:49:47] [I] [TRT] [DlaLayer] {ForeignNode[ReLU32...Convolution110]}
[02/10/2022-01:49:47] [I] [TRT] [DlaLayer] {ForeignNode[ReLU114...Pooling160]}
[02/10/2022-01:49:47] [I] [TRT] [DlaLayer] {ForeignNode[Plus214]}
[02/10/2022-01:49:47] [I] [TRT] ---------- Layers Running on GPU ----------
[02/10/2022-01:49:47] [I] [TRT] [GpuLayer] Parameter193 + Times212_reshape1
[02/10/2022-01:49:47] [I] [TRT] [GpuLayer] Parameter194
[02/10/2022-01:49:47] [I] [TRT] [GpuLayer] Parameter6 + (Unnamed Layer* 4) [Shuffle] + Plus30
[02/10/2022-01:49:47] [I] [TRT] [GpuLayer] Parameter88 + (Unnamed Layer* 10) [Shuffle] + Plus112
[02/10/2022-01:49:47] [I] [TRT] [GpuLayer] Times212_reshape0
[02/10/2022-01:49:47] [I] [TRT] [GpuLayer] Times212
[02/10/2022-01:49:47] [I] [TRT] [GpuLayer] shuffle_Times212_Output_0
[02/10/2022-01:49:47] [I] [TRT] [GpuLayer] shuffle_(Unnamed Layer* 16) [Constant]_output
...

To get more detailed profiling results for DLA, please use our Nsight System profiler:

NVIDIA Nsight Systems

Thanks.