TensorRT model inference fully on DLA is slow due to abnormally slow cudaEventSynchronize time

[AastaLLL]

Hi,

1.
GPU might need to do some data formatting for DLA.
You can find the info below:

Developer Guide :: NVIDIA Deep Learning TensorRT Documentation

2.
We expect DLA to be used in low resolution so the perf with fp16 will be slower.

DLA-v2 is slower than DLA-v1

Hi, We got further details from our internal team. Orin’s DLA has more int8 dense TOPs but fewer fp16 TOPs. So if you run the model in int8 mode, it’s expected to get better performance compared to Xavier’s DLA. Thanks.

3. INT8 is recommended to improve the DLA perf.

Thanks