Layers Inspection on TRT model

[ashimabisla03]

While converting ONNX model to TRT with enabling DLA 0 and allow GPU fallback to true, is there any way to identify that how many layers are running on DLA and how many on GPU ?
For model conversion and inference, I am using Nvidia Jetson AGX Orin.

[AastaLLL]

Hi,

Yes, the information will print out by default.

For example:

$ /usr/src/tensorrt/bin/trtexec --onnx=/usr/src/tensorrt/data/resnet50/ResNet50.onnx --useDLACore=0 --allowGPUFallback
...
[10/31/2022-02:53:03] [W] [TRT] onnx2trt_utils.cpp:367: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[10/31/2022-02:53:03] [I] Finish parsing network model
[10/31/2022-02:53:03] [W] [TRT] Layer 'node_of_OC2_DUMMY_0': Unsupported on DLA. Switching this layer's device type to GPU.
[10/31/2022-02:53:03] [W] [TRT] Layer 'gpu_0/pred_w_0': Unsupported on DLA. Switching this layer's device type to GPU.
[10/31/2022-02:53:03] [W] [TRT] Layer 'gpu_0/pred_b_0': Unsupported on DLA. Switching this layer's device type to GPU.
[10/31/2022-02:53:03] [W] [TRT] Layer '(Unnamed Layer* 177) [Shuffle]': Unsupported on DLA. Switching this layer's device type to GPU.
[10/31/2022-02:53:03] [W] [TRT] Layer '(Unnamed Layer* 179) [Shuffle]': Unsupported on DLA. Switching this layer's device type to GPU.
[10/31/2022-02:53:03] [W] [TRT] Layer '(Unnamed Layer* 180) [Softmax]': Unsupported on DLA. Switching this layer's device type to GPU.
[10/31/2022-02:53:03] [W] [TRT] Layer '(Unnamed Layer* 181) [Shuffle]': Unsupported on DLA. Switching this layer's device type to GPU.
[10/31/2022-02:53:14] [I] [TRT] ---------- Layers Running on DLA ----------
[10/31/2022-02:53:14] [I] [TRT] [DlaLayer] {ForeignNode[node_of_gpu_0/conv1_1 + node_of_gpu_0/res_conv1_bn_1...node_of_gpu_0/pred_1]}
[10/31/2022-02:53:14] [I] [TRT] ---------- Layers Running on GPU ----------
[10/31/2022-02:53:14] [I] [TRT] [GpuLayer] SHUFFLE: reshape_after_node_of_gpu_0/pred_1
[10/31/2022-02:53:14] [I] [TRT] [GpuLayer] SOFTMAX: (Unnamed Layer* 180) [Softmax]
[10/31/2022-02:53:20] [I] [TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +534, GPU +572, now: CPU 1337, GPU 26995 (MiB)
...

Thanks.