Using DLA for Peoplenet tlt model in Deepstream

Accelerated Computing Intelligent Video Analytics DeepStream SDK
,
1

Please provide complete information as applicable to your setup.

• Hardware Platform: Jetson
• DeepStream Version: 5.1
• Issue Type: questions

I am trying to run the quantized peoplenet (resnet34) on Xavier NX using only DLA cores. I have set the flags enable-dla=1, use-dla-core=0 in my nvinfer config and it works. But the problem is that, while I am creating .engine file, not all of the layers are running on the DLA.

INFO: [TRT]: --------------- Layers running on DLA:
 *a long list of layers...*
INFO: [TRT]: --------------- Layers running on GPU:
INFO: [TRT]: output_cov/Sigmoid,
INFO: [TRT]: Detected 1 inputs and 2 output network tensors.

According to this log, only two layers are running on GPU, which is fine to me. But before this log, I get a long list of warnings:

::buildModel() <nvdsinfer_context_impl.cpp:1716> [UID = 1]: Trying to create engine from model files                                                                        
WARNING: [TRT]: Default DLA is enabled but layer output_bbox/bias is not supported on DLA, falling back to GPU.                                                             
WARNING: [TRT]: Default DLA is enabled but layer conv1/kernel is not supported on DLA, falling back to GPU.                                                                 
WARNING: [TRT]: Default DLA is enabled but layer conv1/bias is not supported on DLA, falling back to GPU.                                                                   
WARNING: [TRT]: Default DLA is enabled but layer bn_conv1/moving_variance is not supported on DLA, falling back to GPU.                                                     
WARNING: [TRT]: Default DLA is enabled but layer bn_conv1/Reshape_1/shape is not supported on DLA, falling back to GPU.                                                     
WARNING: [TRT]: Default DLA is enabled but layer bn_conv1/batchnorm/add/y is not supported on DLA, falling back to GPU.                                                     
WARNING: [TRT]: Default DLA is enabled but layer bn_conv1/gamma is not supported on DLA, falling back to GPU.                                                               
WARNING: [TRT]: Default DLA is enabled but layer bn_conv1/Reshape_3/shape is not supported on DLA, falling back to GPU.                                                     
WARNING: [TRT]: Default DLA is enabled but layer bn_conv1/beta is not supported on DLA, falling back to GPU.                                                                
WARNING: [TRT]: Default DLA is enabled but layer bn_conv1/Reshape_2/shape is not supported on DLA, falling back to GPU.                                                     
WARNING: [TRT]: Default DLA is enabled but layer bn_conv1/moving_mean is not supported on DLA, falling back to GPU.                                                         
WARNING: [TRT]: Default DLA is enabled but layer bn_conv1/Reshape/shape is not supported on DLA, falling back to GPU.
WARNING: [TRT]: Default DLA is enabled but layer block_1a_conv_1/kernel is not supported on DLA, falling back to GPU.
WARNING: [TRT]: Default DLA is enabled but layer block_1a_conv_1/bias is not supported on DLA, falling back to GPU.
WARNING: [TRT]: Default DLA is enabled but layer block_1a_bn_1/moving_variance is not supported on DLA, falling back to GPU.
WARNING: [TRT]: Default DLA is enabled but layer block_1a_bn_1/Reshape_1/shape is not supported on DLA, falling back to GPU.
WARNING: [TRT]: Default DLA is enabled but layer block_1a_bn_1/batchnorm/add/y is not supported on DLA, falling back to GPU.
WARNING: [TRT]: Default DLA is enabled but layer block_1a_bn_1/gamma is not supported on DLA, falling back to GPU.
WARNING: [TRT]: Default DLA is enabled but layer block_1a_bn_1/Reshape_3/shape is not supported on DLA, falling back to GPU.
WARNING: [TRT]: Default DLA is enabled but layer block_1a_bn_1/beta is not supported on DLA, falling back to GPU.
WARNING: [TRT]: Default DLA is enabled but layer block_1a_bn_1/Reshape_2/shape is not supported on DLA, falling back to GPU.
WARNING: [TRT]: Default DLA is enabled but layer block_1a_bn_1/moving_mean is not supported on DLA, falling back to GPU.

And the list goes on…
I don’t get it. If these are running on GPU as well, why the first log that I mentioned only reports two layers. The GPU utility is relatively low, compared to when specifying gpu in the config. But I need to ensure that I am avoiding taking GPU as much as possible.

I also wanted to try setting GPUFallBack to false, to try forcing DLA. But I couldn’t find the proper way of doing this in deepstream and nvinfer config. I didn’t find any relevant property.

Thank you in advance for your help.

3

Hi,

TensorRT first use layer type to check if a layer can deploy on DLA or not.
According to this, Sigmoid is not supported by DLA and has been redirected to GPU.

INFO: [TRT]: --------------- Layers running on DLA:
 *a long list of layers...*
INFO: [TRT]: --------------- Layers running on GPU:
INFO: [TRT]: output_cov/Sigmoid,

After that, DLA will check if all the configures in a layer are acceptable for DLA or not.
In your case, there are some convolution / batchnorm layers beyond DLA’s support range as below:

WARNING: [TRT]: Default DLA is enabled but layer output_bbox/bias is not supported on DLA, falling back to GPU.                                                             
...

For the details, please check DLA supported matrix below:

Thanks.