How to use DLA in deepstream-yolov5

HI,
So how do I generate the DLA version of the .ENGINE file? Are there any tutorial instructions similar to the GPU version?

Hi,

To better answer your question, could you tell us how do you convert the YOLOv5 .pt to TensorRT .engine?
Is it a public source or your implementation?

In general, you can build a DLA engine by setting DLA as the device type like below:

nvinfer1::IBuilder* builder = CREATE_INFER_BUILDER(logger);
...
builder->setDefaultDeviceType(nvinfer1::DeviceType::kDLA);

Here is our document for your reference:
https://docs.nvidia.com/deeplearning/tensorrt/api/c_api/classnvinfer1_1_1_i_builder.html#aaf9ae8a39de89a5e4e479152fa1f170d

Thanks

I generated the .engine through the tutorial on the GitHub - DanaHan/Yolov5-in-Deepstream-5.0: Describe how to use yolov5 in Deepstream 5.0 website.

Hi,

Could you insert the following line to the sample and rebuild it to see if it works?
https://github.com/DanaHan/Yolov5-in-Deepstream-5.0/blob/master/yolov5.cpp#L410

...
// Build engine
builder->setMaxBatchSize(maxBatchSize);
config->setMaxWorkspaceSize(16 * (1 << 20));  // 16MB
builder->setDefaultDeviceType(nvinfer1::DeviceType::kDLA);
config->setFlag(BuilderFlag::kFP16);
...

Thanks.

I have modified the .CPP file as you said, and regenerated the DLA version of the engine, but the GPU occupancy rate is still 100% after running.

There is no update from you for a period, assuming this is not an issue any more.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

Hi,

Could you help us validating the engine file first?

Do you see any layer been assigned to the DLA when creating the engine?
The log should look like this:

[01/20/2021-14:55:27] [I] [TRT] --------------- Layers running on DLA: 
[01/20/2021-14:55:27] [I] [TRT] {Convolution28}, {ReLU32,Pooling66,Convolution110}, {ReLU114,Pooling160}, {(Unnamed Layer* 17) [ElementWise]}, 
[01/20/2021-14:55:27] [I] [TRT] --------------- Layers running on GPU: 
[01/20/2021-14:55:27] [I] [TRT] Times212_reshape1 + (Unnamed Layer* 1) [Shuffle], Plus214 + shuffle_(Unnamed Layer* 16) [Constant]_output, Plus30 + (Unnamed Layer* 4) [Shuffle] + (Unnamed Layer* 5) [ElementWise], Plus112 + (Unnamed Layer* 10) [Shuffle] + (Unnamed Layer* 11) [ElementWise], Times212_reshape0, Times212, shuffle_Times212_Output_0, 

Thanks.

@jiyuwang Hi, may I which part you have modified except for the part above?
I added the following line to build the engine.

...
builder->setDefaultDeviceType(nvinfer1::DeviceType::kDLA);
config->setFlag(BuilderFlag::kFP16);
...

But it gives the following error.

yolov5.cpp:457: void APIToModel(unsigned int, nvinfer1::IHostMemory**): Assertion `engine != nullptr' failed.